Monday, 16 July 2018

Another Technical Debt Quandrant and Debt Days

Martin Fowler's Technical Debt Quandrant

Ages ago I blogged about The Technical Debt Quadrant as explained by Martin Fowler in his blog from some years earlier. When I re-read that article (Martin's, not mine) it looks a lot like a discussion that was of its time. The reason why I raise this again is because recently I came across a completely different kind of debt quadrant, not for helping us understand how it came to be but for helping us to understand when to pay it back. In the course of some discussions with my current client I combined it with a thing that I learned from my previous client to come up with a plan to address technical debt.

Bug Squash

At my last client I was tech lead of a team that was mainly responsible for fixing production defects in the platform that we were implementing. Leaving aside whether this is an anti-pattern (it is) we were managing a large backlog of defects whilst simultaneously adding "BAU" enhancements to the platform itself. One of the side effects of this dual focus was that annoying little defects rarely got fixed. A knock on effect of that was that our backlog was growing, we were spending money managing it but we didn't really know if the backlog was real or not.

A Bug Bash is an exercise to be done occasionally, the object of which is to find previously undiscovered, or unrecorded, issues and get them on somebody's backlog. This could be semi regular, just before a major release or upgrade or just a fun afternoon. In my experience of these occasions, anybody can take part, there is probably some kind of refreshments offered and there may be some kind of gamification and prizes on offer.

Somebody in the team proposed something that came to be known as "Bug Squash". It wasn't me, for sure, but I remember thinking it was a great idea and that we should try it. The idea is kind of the opposite of a Bug Bash. We decided that every Wednesday afternoon we would all drop everything that we were working on and we'd look at the backlog from the least important stuff upwards (instead of the usual top down way of picking stuff to fix) and we would either fix the issue straight into the code base or we would record in the ticket why it was no longer an issue and close the ticket. We left it up to everybody to decide exactly how they wanted to work. If they wanted to pair fine, if they didn't that was also fine. This had some great benefits. First, it was fun. Second, our backlog got smaller so we could all share in the success of that, third it spread context of the whole codebase amongst more people.

Another Tech Debt Quandrant

My current client thinks it has a lot of tech debt and this is one of the reasons why myself and a colleague are here for about 3 months to try and advise on some thing they can do to help themselves. As you might expect, tech debt is a symptom of some wider problems with the organisation. Unlike many symptoms though, it is always worth treating tech debt because it has a nasty habit of multiplying and making your future life more difficult.

One of the big problems is that the tech debt makes it hard to do stuff. So they know they need to be paying this debt down but they find it hard to find the time to pay it down. And one thing that makes it hard to find time to pay down tech debt is tech debt. I think everybody can see where that one ends. We've tried a few different things to help with the situation. Currently each team is asked to assign 20% (it might be 10%) of their capacity in their sprint plan to debt items. The trouble is that these are the very items that feel the axe if the teams realise that they won't make their sprint commitment. Moreover, somebody spent some time analysing the debt, creating a card, putting on the board, bringing it into the sprint cycle etc etc. So the management of the problem becomes part of the problem itself (see my earlier post on inventory management).

So in advising on a way to get to the outcome (reduced technical debt) what we needed was a method that was easy to manage and track that encouraged people to pay down the debt. The first stage was a debt quadrant (an idea that was introduced to me by a ThoughtWorks colleague last year). Instead of just having a debt wall onto which everybody pins things they'd like to fix, you split your debt wall into 4 quadrants, as per the picture below.

For those that can't read my handwriting (probably everybody) the vertical axis is labelled "Ease of Solution" and the horizontal axis is labelled "Impact". So I hope it is therefore obvious that stuff in the top right (high impact, easy to fix) should be fixed as soon as possible. The stuff in the top left should be looked at and maybe fixed, the stuff in the bottom right should be looked at, analysed and prioritised into the normal process (whatever that is). Hopefully then, this simple visualisation gives each team a good way to record, manage and pay down their tech debt.

Debt Days

But what if people still don't feel like they have the time to do any of this? Our simple suggestion was that one afternoon a week (that would be about 10% so one whole day or two half days would be 20%) the team should drop everything they are doing on the sprint. Instead, they should pick up one of the cards in the top right of the debt quadrant and do something about it. No time used on planning or estimation or arguing what is the most important thing. Just pay back some tech debt.

It is early days yet but by combining the ideas of a simple quadrant and setting aside REAL time to do something (not time in a sprint plan that might not happen) we will hopefully get some significant payback of our debt pile whilst having some fun along the way.

Saturday, 2 June 2018

Architectology Update

Some time ago I wrote a post on this site about architectology. I have had a lot of time to think further about the subject since my original post, which I have just read again. I didn't really have a conclusion or a solution in that original post. It seems that I was saying that the notion of a person who spends their time navigating legacy code feels wrong but I had no better way of dealing with the silos and extracting value from them.

From talking to a lot of software professionals over the past year I have discovered that many large businesses (and public sector bodies) have their own word for architectologists, it seems that many of them widely refer to such people as "enterprise architects".

I've recently completed a 10 month engagement with a public sector client and although they didn't call them enterprise architects they had plenty of architectologists. I now believe I have seen enough to propose that architectology is an anti pattern. According to Wikipedia, an anti patter is a common response to a recurring problem that is usually ineffective and risks being highly counterproductive. To distinguish a genuine anti-pattern from a simple bad habit, the following 2 conditions must be met:

1. A commonly used process, structure, or pattern of action that despite initially appearing to be an appropriate and effective response to a problem, has more bad consequences than good ones.

2. Another solution exists that is documented, repeatable, and proven to be effective.

Firstly, is architectology a common response to a recurring problem? I would argue that it is. The problem is how to extract value from silos of legacy systems. The response is to task some people (architectologists) with drawing pictures of the legacy with lots of arrows on it to show a software team how they might get stuff done.

Secondly, does it have more bad consequences than good ones? I think so. Among other things it prolongs the lifetime of the legacy, couples it more strongly to more processes and makes it successively harder to add new value to the enterprise.

Finally, what is the better solution? Last summer we had a good deal of success in the client we were working at in persuading them to create cross functional teams that own the entirety of a customer facing outcome. Having made that step, the logical next step was to repurpose their architectologists (IIRC they were called enterprise architects) as architects working more closely with software teams delivering those outcomes. As the teams that previously owned the legacy pieces were broken up, nobody was left illogically defending the existence of various pieces and the client was able to start sensibly decoupling the systems and strangling parts out of existence.

So it is therefore my conclusion that architecology (or enterprise architects if you prefer) fulfils the criteria to be regarded as an anti-pattern.

I recently gave a short talk postulating that architectology is an anti pattern at Devoxx UK in London.

Thursday, 3 May 2018

* Driven Development

With thanks to my colleagues - Andrew Jones Weiss, Velizar Genov, Gergo Takacs, EnterTheDragos, Robert Chow, Stuart Paton.

I had a conversation with a friend a couple of days ago about what work his team was doing. They were supposed to have done some work "in time for UAT" but they missed getting a couple of stories into the locked down UAT environment before the deadline. Thus, UAT wasn't going to cover those use cases. The next UAT wasn't due to happen for a couple of months so they changed their focus on to some different stuff.

I laughed at this and commented that this sounded like Politics Driven Development. I had never heard the term before but given the nature of the organisation my friend works in, I thought it was an apposite term for what they sometimes end up doing. That got us into discussions of what other *DDs exist.

We then set ourselves the challenge of coming up with a whole alphabet of *DDs. Some of them are serious, some of them are not, some of them we made up to get through the alphabet. Our criteria was that for something to stand it either has to be a real thing or the person suggesting it had to have experienced it at some point in their career.

Any more suggestions welcome!

A

AWS Driven Development - This is when your company migrates from the old server farm to AWS and all future development is constrained by the tools and services that AWS offers.

B

Behaviour Driven Development - this is a thing.

Bullshit Driven Development - when an influential developer blags that they understand a technology and the team goes too far down the road of relying on that technology before they realise that (a) that developer was lying and (b) the tool is not appropriate for the job. Extra points if the developer that influenced the decision leaves the company shortly after the decision was taken. Kafka is a repeat offender in this space.

Bleeding Edge Driven Development - this is when you must on NO ACCOUNT use any technology found closer to the middle of the ThoughtWorks Tech Radar than the Assess ring. If there has been a conference with this as its main subject you may still use it but only if that conference was an UnConference.

C

Career Driven Development - when a decision to use a language or technology is not driven by what is best for the problem at hand but by what the developer or developers want to learn to enhance their career within an organisation.

CV Driven Development - like Career Driven Development except the motivation is to enhance your CV with the intention of moving to a different job.

D

Domain Driven Development - this is a thing.

Developer Driven Development - This is when developers as a group decide that they want to do something interesting. So they decide that instead of using boring old Java for their Microservices that they are going to use shiny new and interesting Clojure. This could also morph into one or other of the types of CDD.

E

Ego Driven Development - It’s what Rockstar developers do.

F

Failure Driven Development - Your system has so many defects with new defects being constantly reported that you never have the bandwidth within your team to play any stories that add new value. You are sentenced to be forever jumping to the tune of the person in the business who is screaming the loudest to have their defect fixed.

Fowler Driven Development - When Martin Fowler posts a new blog post that could impact your work in progress you then make it your business to ensure that it WILL affect your work in progress. Extra points are awarded if you contact a friend you have within ThoughtWorks to ask for a clarification on the nuance of what Martin was referring to before you modify your stories.

G

Google Driven Development - Like AWS Driven Development except you use Kubernetes and the Google Cloud tools.

H

Hypothesis Driven Development - This is a thing.

HiPPO Driven Development - This is based on the HiPPO decision making process that many organisations rigidly stick to. To implement HiPPO decision making you gather a large group of people in a room to discuss the decision. The larger the better. You then discuss the thing for a long time listening to everybody's opinions, including those that will live the consequences of the decision. Then, the Highest Paid Person's Opinion (HiPPO) is aired and that is the decision taken. HDD is when the priority of features is decided purely on the seniority of the person that requested it. Actual business value is irrelevant and pointless to argue about. If the CEO requested it, it's the first thing you do.

Hype Driven Development - This is strategic decisions based on what the current "big thing" is (see also ThoughtWorks Driven Development). So if the buzz in the industry is Microservices, then the CTO will be demanding that we "should be doing Microservices" irrespective of cost of value.

I

IntelliJ Driven Development - when you join a new codebase the IDE tells you all the bad things about the code. AKA Alt + Enter Driven Development.

J

JetBrains Driven Development (see IntelliJ Driven Development)

K

KISS Driven Development - Let’s keep this one simple.

Kangaroo Driven Development - I’m struggling here, you may have noticed. This is when you have a weak or no product owner who has a bad time articulating upwards what is important and what isn’t. You therefore end up jumping from one priority to another never quite knowing from one week to the next what your purpose as a team really is.

L

Lifestyle Driven Development - As a tech lead I always think there are two kinds of decisions. Those that I care about and those that I really don’t. Some people get really exercised about the second type of decision. I’ve seen conversations over whether we should use a 2 or 3 space indent in our C# code rumble on for 3 months. Seriously. And it involved edit wars in FXCop lint rules. Any such decision, that really doesn’t matter in my view, is a lifestyle choice. Lifestyle Driven Development is when this type of Bikeshedding dominates team stand ups, team planning meetings and all team conversations. Even in the pub.

M

Marketing Driven Development - when a marketing campaign proves that a market exists for a new product that you haven't built yet, forcing you to build it.

MoSCoW Driven Development - this is a thing.

N

Negativity Driven Development - This is when your developers are so negative about the prospects of the code not falling apart that you write dozens of tests that test the edges of the edge cases that can only ever be hit when all 8 planets (and Pluto) align in such a way that frogs fall from the sky and turn into baby poodles before they hit the ground. This causes such immense scope creep that your project funding expires playing the first story and nothing ever goes live.

O

Optimism Driven Development - This is when you design for the happy paths only because obviously those are the only bits of your system that will ever get exercised. This is very true, until your project goes live at which point you have to run for the hills.

P

Politics Driven Development - Many large organisations are riven by internal politics. When you work in such an organisation you know about it. Politics can lead to some pretty illogical decision making around prioritisation of your backlog. Conversations could be something like "We think this is the most important thing but we need to keep this group happy because we need them onside to help with next year's budget approvals".

Patterns Driven Development - When one developer has recently read Design Patterns by the Gang of Four and goes through the codebase insisting that everything be refactored to closely follow some of the patterns in the book. It doesn't matter if it is an appropriate pattern for solving the problem at hand and it doesn't matter if the code being refactored is covered by solid tests that can verify the refactoring. In fact it is better if there are no tests at all so that you really have no idea if the code still behaves as expected. Patterns Driven Development is an Anti-Pattern.

Q

QA Driven Development - This is a thing but every reference I found to it calls it Tester Driven Development but we already had stuff in T.

R

Resharper Driven Development (the Visual Studio version of IntelliJ Driven Development).

Retro Driven Development - This is actually a good thing. This is when you do something for two weeks, discuss how it can be improved then realise that you were doing it wrong so now you do it right. Wash and repeat.

S

Sales Driven Development - This is when your sales team makes a bespoke deal with a new partner assuring them that we have systems in place to support these weird business rules that they just invented on the fly. They then come and talk to the tech team and say things like "we can do this, right?" It obviously doesn't matter if we can or not because we're now committed to hastily implementing the features that we have already sold. This happens a lot in start up companies.

Stack Overflow Driven Development - It’s a thing, no really! It is! This is often characterised by code comments saying “I don’t really know what this does but here’s a link to the Stack Overflow page”.

T

Test Driven Development is a thing that I hope we are all familiar with.

U

Underwear Driven Development - This is when a team works as a distributed team and most people’s default setting is working at home. During stand up everybody is on a video link and you can tell which developers aren’t dressed yet because their video feed will be a black screen with “John’s iPhone” written on it.

V

Value Driven Development is a thing.

W

Waterfall Driven Development - We’ve all been there. Hopefully not recently. Oh dear, does last year count?

Wagile Driven Development - When you are actually doing Waterfall but the project management like to claim you are doing Agile.

X

XD Driven Development - I thought this was a real thing but when I Googled it I couldn’t find it. At a place I worked recently the client had a really strong UX department and their product design was heavily influenced by what the UX people did with user research and workshopping. It seemed to work pretty well, the product was really cool that we built and had good take up from the target market.

Y

YAGNI DD

Z

Zealot Driven Development - When somebody in your team is so vehement in their beliefs that they drive the team forward to complete the work according to their standards. This was, in fact, the method used to construct this blog post. My colleagues are now desperate to order some food having been forced to finalise our definitions before I allowed us to sit down for dinner.