Outcome over Process

The Agile Manifesto

The Agile Manifesto, amongst other things, tells us that we should value "Individuals and interactions over processes and tools". The Agile Manifesto is still hugely relevant (even if the principle about regular delivery says "from a couple of weeks to a couple of months") and is not often enough referred to. I have often called out in my talk Agile is a Dirty Word that many so called Agile practitioners have never even read the manifesto, let alone understood the values and principles or attempted to get their teams to buy into them. I like to map the values and principles onto whatever my teams are doing so that they can come to their own conclusions of what is valuable based on their own context. One thing that comes up again and again is that people should value Outcome over Process.

Process over Outcome Anti-Pattern

Organisations can get into a comfort zone where people follow a process without understanding what important outcomes the process is intended to support. This is an anti-pattern. At some point, somebody, somewhere said something like "we have this problem, this process will fix it." There is a chance this person could have been right. However, many months or years later when this process is still hanging around people have lost sight of the outcome. Existing people either forgot the outcome, or never knew it or, in many cases, the outcome being driven is entirely different. So when new people join the team they only hear about the process and usually they never ask "why do we do this?" because they don't see it as their job to do that.

Risk Management Theatre

I have previously written about Risk Management Theatre in this blog. This is the end state of process over outcome. In the worst cases, the outcome of some process is not only different from what it was originally but can be obstructive to that original outcome. I worked for a company last year who was stuck in an old fashioned fear cycle of long delivery cadence. Each two weeks they had a meeting, involving around 30 people for an hour, called the "Go / No-Go Meeting". On the face of it, this meeting is there to decide whether the release should go ahead or not based on whether it is considered too risky. In fact, after observing the meeting I could see that they never once said "No-Go". So the real outcome of the meeting was that all 30 people shared the blame when the release went wrong as they knew it would. Hence, no risk was being reduced but the staff had the comfort blanket of knowing they couldn't be sacked.

In the worst type of organisations Risk Management Theatre can be so rife that there exists whole teams of people whose day to day job consists of ticking boxes on some checklist to make sure that they don't get blamed when things go wrong. It isn't hard to see how much real value that adds.

A Real World Example

If anybody has any doubt that Risk Management Theatre is a real thing, not just in technology but in all walks of life, here is a real example that I came across while walking with my girls into town a few months ago. As we walked past a new(ish) block of flats, actually the old Kodak headquarters which has been redeveloped, we saw this:

If you look closely you can see the effort that somebody went to in order to, presumably, "make the area safe". The three pieces of fencing in the foreground are carefully connected to each other and the one at the top of the stairs is connected to the metal railings on either side by those plastic ratchet connector things that are so strong that police use them as quick attaching handcuffs.

If you look REALLY closely, you'll see a minute bit of broken glass under the bush in the background. Not on the path where anybody will walk but under the bush. This type of stuff baffles me. I assume there must be a rule somewhere that says something like "when you find broken glass make the area safe so that nobody can injure themselves". This has led to somebody going to enormous trouble and expense to do this when they could have simply cleared it up in seconds. This is a classic example of people letting the tail wag the dog - the process of securing the area is obscuring the outcome of it being safe for people to use.

Ways of Working

So why am I rambling about Outcome over Process at this point? Well, I joined a new team a few months ago which was billed as being very technically able but which was unable to get things to move across their board. There was already a ThoughtWorks senior developer in the team and a ThoughtWorks principal setting technical direction but not involved too heavily in day to day work. The ask was to come in as "iteration manager", I still don't know what it means exactly but it is the second time I've been staffed by ThoughtWorks into that role, and help to get the work understood better and flowing better.

What I found was pretty much what had been advertised to me. There were 5 client developers who were all competent but no real visibility, what passed as a board didn't reflect what was going on in Jira nor what was actually happening, and absolutely no understanding of flow and why that should be valued. The TW principal asked me after a day or two what I thought and I told him exactly what I thought. "Great", he said, "can you run a Ways of Working session to get some process around this to get it moving then please." This was the point at which I pushed back and put into practice what up until this point had been a conjecture that I had been nurturing for some months.

Slowly Slowly...

I reasoned that I shouldn't try and change all the process at once, as I laid out in my previous post, and crucially we weren't under immediate delivery pressure, nor would we be for a couple of months at least. So I told Matt (the ThoughtWorks principal) that it was my belief that we should not try to change everything, rather we should be changing things little and (fairly) often. I proposed to start a "ways of working" session in week 2 but actually I said I don't like calling it that because I don't want to talk about roles or rituals in the team before we understand what outcomes the team values and whether we should be working on them. 

So in week 1 I ran a Goals and Values workshop to kick the team off into the habit of socialising and hence (hopefully) buying in to decisions. At the end of the week I casually suggested that we try a new style of standup which I called "story centric". Most people call it "walk the wall" which I proposed would better serve our needs than the traditional, "yesterday I did, today I will..." type of standup. I reasoned out loud to the team that I trusted them and that, given that I trust them to do the right thing, I don't really care about what they did yesterday. What I really care about is when our stories are going to deliver value to the company, that is to say, when they will be done.

The Big Idea - Process Driving Outcomes

My idea was that I wanted to get the team to talk through all aspects of their process in a timeline fashion. I could have sold this as a "path to production" exercise but when I've been involved in that in the past it has been to identify inefficiencies in engineering practices, coupling points (and handovers) and opportunities to improve quality. In the case of this team I was trusting that their engineering practices and quality control were good enough, at least they weren't our biggest problem at the time, and I really wanted them to start questioning why they did all the things they did. 

Part 1 - The Timeline

I used two sheets of magic whiteboard and drew a line on the left labelled "story picked up" and one on the extreme right saying "done". I explained that time is flowing from left to right. I asked them to put stickies on the timeline. Each sticky could represent something that happens, a ritual, a conversation, a meeting, writing code, reading stuff... ANYTHING to do with the process of getting that thing to done.

This took a while and there was interestingly quite a lot of debate as people added stickies at different points and discussed whether it was meant to be that way or this way or whatever. A couple of times I was asked, "should be what we're meant to do or what we actually do?" That was an interesting question. What struck me was the confusion over the process and, even without explicitly raising it which I intended to do later, confusion over why exactly things were the way they were.

Part 2 - The (disingenuous) Discussion

When we had all finally finished putting "event" stickies on the boards I told them I wanted to understand all of what was there. At this point I was being deliberately disingenuous at times. This team did not know that I was a software developer, they thought I was a project manager. So I exploited that by asking questions like "what does branch mean?", "what do you mean by merge?", "oh, you have to wait a day for a code review? Have you considered sitting with a person to do that together?" My plan was to tease out some discussion about how silly some of the processes are and how they seemingly (to the pretend ignorant) return no value. At this point we had a load of green event stickies and a load of small pink "notes" stickies at various points.

Part 3 - Outcomes Supported

The main bit I wanted to get to was a conversation about what outcomes were important from each part of the process. I wanted the team to start thinking in terms of outcomes and I wanted to start to get them understanding that flow is important. I was expecting debates around the value of various parts of the process and we certainly got that. It went probably better than I expected. 

Context Sharing

The outcome that appeared the most on the picture was "context sharing". Apparently many of the rituals were to share context. Many of the things that happened were around sharing context. Sharing context was apparently really important, they all value it but apparently they don't necessarily value it enough to do it earlier or more effectively, perhaps through pairing instead of code review.

Code Quality?

Interestingly, at no point in the process was ensuring code quality called out as an explicit outcome. I expected somebody to claim that code reviews are there to support this outcome but nobody claimed it. I suspect that they were already accepting that code review is a form of risk management theatre.

The one nod to this idea was that at one of the points in the process it was agreed that "code fit for purpose" was an outcome that was being driven. Interestingly, it wasn't clear exactly which part of the process did this - is it QA (very ill defined in the case of this team) or is it code review or is it at some earlier point? It also wasn't clear, even after a lot of discussion, what exactly "code fit for purpose" means. What was clear was that we wanted to drive this outcome somehow. So we agreed to revisit both the definition of fit for purpose and how we might drive that outcome, later.

Communication

Confidence was mentioned as an outcome in the QA and UAT process (which the team admitted wasn't actually being followed) and expectation management was also mentioned as an outcome. Advertising was called out as an outcome against "demo" or "showcase" and there were a few discussions about how we might facilitate communication with the rest of the company, in particular those teams that we are considering to be our primary customers.

Interlude - The Story so Far

This was two or three weeks into my time with the client. I must stress at this point that we had already made some significant changes to the team ways of working. I just hadn't done a "Ways of Working" session as I had been asked, for the reasons set out above.

Stand up

As mentioned above we changed the stand up style at the end of my first week. The team were buying into this change and seeing its value. It was helping us to get focused on what was blocked and what we might do to unblock them. It was also starting to get the team focused on the importance of flow (I kept using the word "flow" incessantly in those early days). The change to stand up was the ONLY change that I "imposed" on the team. I simply asked them if it was OK and they agreed. At that point I think they still had a bit of apathy for things so they probably would have agreed to anything I suggested.

Tech Huddle

At the end of my second week at the client we got an external facilitator to facilitate a "Lego retro". Before the retro we established that the retro was aiming to just get a single retro action. So I discussed with the other ThoughtWorks people what we thought was the biggest problem currently. We felt that the team were not getting a shared understanding of stories and this was causing problems with them getting past the "in progress" stage as nobody really understand what good looked like in each case. Between us we therefore felt a daily tech huddle, after stand up, would be a good way to make sure we started to share context better. So we agreed that we would try and steer the retro toward that outcome. As it turned out, the retro was really well done (I'll write about that separately) and the team came to its own conclusion that tech huddle would be a good idea.

Work in Progress Visibility and Reduction

As "Iteration Manager" it was understood that I own the board and the flow of stories across it. I spent a few days at the start writing out cards and putting them on the wall for all the stuff that was already in flight. It turned out to be over 20 things. So I tidied up the board, got rid of all the columns that really weren't useful (since our process didn't use them) and insisted that we pick nothing new up until this WIP went down significantly.

Part 4 - What Process Should we use to Support our Important Outcomes

After we had the outcomes we care about and after we'd discussed those outcomes in detail we spoke about the process to support them. I started off, with a new sheet of magic whiteboard, by putting stickies on it to represent Scrum rituals. The team still thought they were following Scrum even though I had quietly ignored Scrum planning, Backlog Grooming and all other set piece rituals other than stand up, retro and showcase. I laid out rituals in four swimlanes - once (per iteration), sporadic, daily and ?. I think I was trying to move the team from Scrum cycles thinking so I was reluctant to put anything in as "per sprint" or "per iteration". Maybe if I did this again I might use "per story", "daily", "per iteration" as headings.

We then discussed which outcomes each ritual supported and agreed which ones were needed, which ones should be dropped and which ones we needed to keep an eye on, or change, or improve, or reassess. From that point on we agreed that we will be constantly looking at our process and always asking what our biggest problem is and trying to solve it. 

The single resolution from this last meeting was that I agreed to own the story starting process better so that we only kicked them off when I had good quality on the stories. Good quality in this case was agreed to mean well understood acceptance criteria that at least a few other people in the team had agreed to and collaborated on. Essentially I had a story template to stick to that we'd all agreed upon.

The Outputs

I didn't want the team to get fatigued in any given meeting because I wanted them to be enthusiastic about wanted to understand the process and wanted to understand what outcomes they should care about. I also wanted them to make the decisions themselves and I wanted them to buy in collectively to those decisions. Decisions made at the end of along meeting driven by boredom and fatigue will not stick. Decisions made at the right point in the discussions, whether at the end of an allotted time slot or at any point in a timed meeting, if taken in the right way, have a greater chance of sticking. So the whole process took 3 separate meetings of 1 hour. The last one only lasted about half an hour.

Here are some photos of the outputs we ended up with:

Some activities (green), discussion points (pink) and desirable outcomes (blue)

More activities, outcomes and discussions, this team nearer the Done side.

The rituals (orange) with useful outcomes mapped (blue)

Continuation

These activities took place about 2 months ago at the time of writing and I am about to move on to my next challenge. The team continued to value flow, continued to be bought into the ownership of the whole process and continued to be keen to improve constantly. We since added WIP limits, changed the way Jira was set up for our project to mirror our actual wall and process (not the other way round) and added other stuff to our wall. 

The Wall

The team continued to add stuff to the wall that we thought was relevant and recently everybody has made suggestions about what might be a good idea. People are making suggestions at stand up and, if it isn't shouted down, my attitude is "sounds good, make it happen, if we like it we'll keep it, if we don't, or it doesn't solve a problem, we'll review it." This has the team working really well towards the right outcomes and we are making everything visible on the wall. Flow is now a thing that everybody values and everybody is familiar with.

Notes on the current wall:

1. This is an area to park people's avatars when they aren't working or aren't available to work for our team. For example, off sick, on holiday, at a conference.
2. This is an area to note down things that we would like to discuss in the next tech huddle.
3. The area on the left is for things that are ready for dev. The pink cards are epics, the yellow are stories, the blue are spikes. If we had any defects we'd use a different colour card for them.
4. This part is outstanding retro actions.
5. "In dev" and "Dev complete" are the only two columns we are using between ready for dev and done. This is because we don't have formal QA and we aren't sure yet whether columns such as "waiting for XX", "in review" etc are valuable to us to help us visualise our flow.
6. We drew a line around in dev and dev complete and annotated it with "WIP Limit - 4 cards". The WIP limit is across the two columns. I don't know if Jira supports WIP limits shared across columns but I don't care. The wall is giving us enough visibility to stick to it.
7. The Done column contains all the stuff we've done since the last showcase (every two weeks).
8. The little red stickies are Blocked stickies. The fact that there are so many of them tells a story of what the board looked like when we started. Loads of massive things getting snagged on loads of external dependencies in flight at the same time. The fact that nothing is currently blocked tells a story of progress. We haven't had to make a decision on whether a blocked card counts towards the WIP limit as it hasn't become a problem since we had the WIP limit.
9. This box is marked "Customer Urgent". This is stuff that is on our backlog that is blocking any of the, now 3, teams that are our platform's customers at the moment.

We also have another wall next to this one that is our roadmap. It has higher level stuff on it and some dates that we are hoping to hit for milestones on our platform. 

Conclusion

This particular team has been the first time that I have had the mandate and the courage to apply the thinking that I had been forming over the previous few years. It worked well but I understand that one success cannot be considered a scientific sample. I'll also be interested to hear in a few months time whether or not the team continued to perform well or whether there was any kind of behavioural regression. Nevertheless I will state the following things as my conclusions or perhaps as my main tips for helping a team of talented people to get flow going well.

• Don't impose stuff on the team. Find ways to get them to make their own decisions and shape their own destiny. Don't be afraid to guide them in the right direction but get the whole team to own the decision and any outcomes.
• Outcomes are more important than process. Get the team to decide and agree what the valuable outcomes are and only then talk about what process might help to support them. Never lose sight of the mantra "outcome over process".
• Transformation takes time. Be prepared for a journey of at least several weeks in length. Let the team fail, let them understand what failure and good look like. 
• Change one thing at a time. Big bang process release won't work for several reasons. Give each change time to bed in before deciding whether it was helpful or not.
• Let each team own their own process. It is clear that the process I've outlined got to one process to support one team. It would be a mistake to think you could simply take the end result and map that on to another team. Make this type of meta-process the thing that repeats, NOT the process that the team ends up using.