Tuesday 5 December 2017
Inventory Management
On my first real engagement as a ThoughtWorker, before we started doing any delivery work, my colleague Chris and I were in a meeting with some architects and business stakeholders looking at their backlog. It was large. Very large. Chris started pointing at some items near the top and asking when they might get done. "Soon", "In the next few weeks", "in this or the next sprint" were typical answers.
Next Chris pointed at some stuff just under the fold and asked the same question. "Maybe in the new year" (this was late October), "Depends if anything else comes up", "Probably in the plan for Q1" were typical of the slightly more vague answers we got at this stage.
Then Chris started asking about some stuff that was a few pages of scrolling further on, nearer the bottom of the overall backlog. Now the answers were more like "That may never get done", "Can't see that ever being a priority", "I have no idea what that even means" were the type of answers we got at this stage.
At this point Chris made what I thought at the time was a rather controversial suggestion. He asked if they had considered deleting everything. The whole backlog (he may have suggested reprieving the stuff in progress and a little bit more, I can't remember, but it sounds more dramatic if he proposed torching the whole lot). This was met with howls of protest from our clients. "We need this stuff!", "Everything on this list is business critical!", "We've promised this stuff to the business!". One of us asked how long something had been on the backlog. The tool was able to tell us it was in the order of 2 years. "How has the business managed without this business critical functionality for 2 years?" was a rather ironically posed question at this stage.
I do remember asking Chris afterwards why he was advising them to do this. The answer I got was "it's inventory". Unfortunately, I either didn't have the time or I didn't press to get a fuller explanation of why inventory was a bad thing. I hadn't read much, if anything, about lean manufacturing or the TPS at the time, and I certainly hadn't read the Goal, so I was probably associating inventory in my mind as something that was an asset (stock in hand), not a liability.
I'm not sure how that one ended up. I wasn't close to the further discussions when we started delivering and I left that client a few months later. As I was working on completely new functionality I never had to go near their backlog so I guess I just stopped immediately caring about why we should care so much.
Two years later I saw exactly what I didn't seen then. I was moved to a new project and told that there was a problem. This team that I was being asked to work with was having a problem moving stuff across their board. We are responsible for live system defects and in the months prior to me landing here the number of outstanding defects had risen slightly despite the number of new defects raised to the team being in successive months, 45 then 30 then 20 then 10. Clearly something was wrong with the flow of items across the team's board.
When I arrived I thought it might take me a few weeks to discover what problems existed. In the first week I noticed that several people in the team were spending an extraordinary amount of time looking at the card management system, updating cards with various numbers, extracting data from it and preparing reports for consumption by various people. I don't know when it started but clearly at some point somebody in the business asked this team for information about when stuff might be fixed without considering the consequences of how those answers might get produced.
I went to a meeting with 8 people in it that lasted an hour. In this meeting we reported to these people various things that they could have found out by looking on the card management system (they all had access, I checked). After that meeting I asked a few people in the team how much time they spent on what I was now recognising as inventory management. I was shocked by the answer. Some of the team were spending half their time on this type of inventory management activity!
So what did we do? In the next meeting I suggested to the business stakeholders that they were asking for all of these updates because at some point in the past they lost confidence in our ability to deal with defects in a timely fashion. This was agreed. I then postulated that they wouldn't care about these reports if we were dealing with defects in days or hours instead of weeks or months. Again, they agreed. Then I pointed out, which they probably weren't previously aware of, that we were spending so much time on servicing their reporting "requirements" that it was seriously compromising our ability to do the work that they were asking us to report on. So I asked if they minded trusting us for a little while. Could we drop all the reporting stuff as of now and then talk again in a few weeks when hopefully we'll be able to demonstrate an improvement. All of this was agreed by all of the teams we deal with.
Three weeks later and the signs are good. We've decreased the count of outstanding defects from about 70 to about 45. We have no live category 1 or 2 defects. Morale in the team is considerably improved. Nobody in the business has asked where their reports are. I am hugely simplifying of course. We also put in place a number of other things to help our team. This felt like the biggest thing though.
So for the first time I could see through direct experience why Chris was so keen to remove (or at least reduce) the backlog. The backlog is inventory. Inventory needs inventory management. Inventory management does not return value to the business. So a big backlog is bad.
I guess in conclusion I'd say if you are part of a delivery team and you find that you are spending lots of time managing inventory then ask yourself why. My advice would be not to bother jumping to these reporting "requirements". The data is probably there in your card management system or your wall. Explain to the business how inventory management stops you doing work that returns value to the business and respectfully ask if you can concentrate on valuable work. It certainly helped for us!
Wednesday 20 September 2017
Risk Management Theatre
Process Theatre?
Some time ago I put some placeholder headings on my blog for some thoughts I was having. One of them was "process theatre" which I was never happy with. Before I dive into it, some historical context...
Security Theatre
I think a lot of people are familiar with the concept of Security Theatre. This is the practice of investing in measures that give the impression of providing security when in fact they do little to improve actual security. We see this in the UK with armed police patrolling London's transport hubs. These people do a wonderful job I'm sure but they will not prevent attacks. The real work is done behind the scenes by intelligence people. I guess the problem is (for some people, not me) that this work is not visible.
"Security Theatre" has worked its way into the IT lexicon to describe things that happen that give somebody somewhere some comfort that things are all OK but may not actually add value in an Agile environment. Consider the insistence of many "infrastructure" teams on installing virus checking software on every production instance. This may have had some merit back in the day of real metal servers but it has no merit now. I've been involved in teams who have to install virus checking software every time we spin up a new instance in AWS. This adds minutes to the start up time and hence the deploy time. I have never known this virus checking software find a virus or highlight any issue. We still have to do it though because it is on somebody's checklist somewhere.
So Security Theatre is stuff that looks like security but isn't. In fact it is probably worthless. This is why I was always unhappy with "process theatre" because, whilst in my mind it was describing a worthless process, it is definitely a process.
Reactive Processes
On my last 3 major engagements I worked for mature organisations, including one bank, who were in various states of confusion, inability to innovate and general lack of corporate agility. This type of work is often dubbed as "Agile Transformation" or "Digital Transformation", certainly "transformation" needs to be in there somewhere.
The big thing that I kept seeing over and over again were processes that had to be followed no matter what. Such processes were (probably) created in reaction to some bad event that had real consequences at some point in the distant past. The business responds by installing all sorts of governance around that thing which is evermore regarded as risky. Teams then evolve to own parts of this process. Over time the original intent of the regulation is lost. The teams that manage it now exist for the purpose of that process. There is no knowledge, or desire, to understand what outcome was originally being supported. The end state is that entire teams of people exist whose day to day job involves servicing a check list to ensure that if (when) things go wrong, they don't get blamed.
So I am grateful to my colleague, Robin Weston, who introduced me to the phrase, Risk Management Theatre, apparently coined by our esteemed ex colleague, Jez Humble, to describe such processes. This is much better than what I was thinking. What we have is a process that is seen by some to manage a particular risk but which in fact does nothing to manage that risk. In fact in some cases it actually makes it worse.
So I was reassured that what I had observed and thought about was a thing. So I've been thinking about it a lot in the course of my consulting work.
Example - Code Freezes and Release Cycles
If you have worked in software delivery for any length of time you will have come across (unless you are very lucky) the "death spiral" of code freeze - testing period - release to production - pray - work late and work on weekends to fix stuff - throw out fixes - rinse and repeat etc etc.
Long before Agile was a thing somebody released some code that broke something in production. This was probably before TDD was a thing, maybe there weren't even any kind of unit tests. Old bugs were reintroduced, nobody really knew what had changed, nobody could understand why it didn't work in production because "it worked on my machine" etc etc...
In response to this undeniably bad happening the business decided that it needed to test its releases more thoroughly. This seems like a reasonable response, indeed in 2004 it may well have been a reasonable response. So they create a test team somewhere. In order to do a release now you have to build your application, put it in some environment for testing and have this test team run through their tests scripts for a prescribed period before each release. Note that after you've built into this test environment you cannot make any further changes to the code that will be released (unless they are critical bug fixes), the code freeze. This is so that everybody can be sure that the test team is testing the stuff that will end up in production.
Ten years later this governance process still exists even though software development practices have moved on. We now have TDD, pair programming, cross functional teams, continuous integration and continuous delivery. We now understand what we have changed much better and moreover we have tested it at the unit level, the integration level and the user journey level. One of the big factors in our confidence is that we changed very little. This means we know what changed, we know what we needed to test and we know how to undo it very quickly if something somehow slipped through the net.
If we still have to go through code freeze and a testing cycle what happens? We try to cram as many changes as we can into the codebase before the code freeze. We cut corners on our Agile process to beat the deadline. We start to game the process by forcing new features past the code freeze by pretending they are bug fixes. We end up doing a big bang, waterfall style release. This is precisely the thing that caused the original problem. We have good solid practices in place that mean we don't need this whole process but the insistence of somebody, somewhere that that process is necessary perpetuates the exact behaviour that made it necessary in the first place. This is why I call this a "death spiral" - once it starts it is very hard to stop.
Corporate Immune System
So why do these teams and processes not get swept away? I think there are many answers, and I don't have all of them by any stretch. A good summary of some of the reasons can be found in the Corporate Immune System.
The organisation has evolved many processes, they have to be followed. There may be a blame culture within the organisation which prevents bold, innovative, employees from challenging the status quo. They first think of avoiding blame and second think of adding value. Innovation means change, which is something to fear for many people. There are managers, or even CXOs, who are heavily invested in the people supporting some of the risk management theatre. These often powerful voices argue strongly for the continued existence of the process that they curate citing examples from ancient history to show their worth. Other people have written far more eloquently and far more knowledgeably than I about the Corporate Immune System. I'm not saying the corporate immune system, or its side effects, are the only cause of the continuance of risk management theatre, but it certainly doesn't help!
How do we break the cycle of Risk Management Theatre?
I haven't fully formulated my thoughts on solutions yet. I have had varying degrees of success in the teams I worked with on my previous engagements. I'm now working in the public sector which introduces a whole new dimension of dysfunction that I hadn't previously seen. This is what prompted me to get my thoughts down about Risk Management Theatre as they stand. I want to take that as a baseline to hopefully move forward with this new client and hopefully come up with some solutions that may help even in the public sector. That is very much a work in progress!
My best answer at the moment is to reason with people involved in Risk Management Theatre about the outcomes they are supporting. If you can find a receptive audience, with an appetite for positive change, they should be willing to reason out the answer to "what outcome are you supporting" and extrapolate that thinking on to the question of whether the process they are using is the best way to achieve that outcome.
Customer focus is a great reasoning tool here as well. If you can ask the question above in terms of customer focus, i.e. "what customer facing outcome are you supporting with this process?" this can not only help an individual to reason about a single process but it can also help resolve a conflict between opposing processes and goals. For example, we have had success using the customer as a reasoning tool to resolve the tension between the development team who wants rapid change and the infrastructure team who oppose all change.
I will return to this subject when I understand better how to challenge (and hopefully reduce) risk management theatre in a public sector environment that cares little for any customer and may not even have a clear understanding or consensus about who their customers are.
Some time ago I put some placeholder headings on my blog for some thoughts I was having. One of them was "process theatre" which I was never happy with. Before I dive into it, some historical context...
Security Theatre
I think a lot of people are familiar with the concept of Security Theatre. This is the practice of investing in measures that give the impression of providing security when in fact they do little to improve actual security. We see this in the UK with armed police patrolling London's transport hubs. These people do a wonderful job I'm sure but they will not prevent attacks. The real work is done behind the scenes by intelligence people. I guess the problem is (for some people, not me) that this work is not visible.
"Security Theatre" has worked its way into the IT lexicon to describe things that happen that give somebody somewhere some comfort that things are all OK but may not actually add value in an Agile environment. Consider the insistence of many "infrastructure" teams on installing virus checking software on every production instance. This may have had some merit back in the day of real metal servers but it has no merit now. I've been involved in teams who have to install virus checking software every time we spin up a new instance in AWS. This adds minutes to the start up time and hence the deploy time. I have never known this virus checking software find a virus or highlight any issue. We still have to do it though because it is on somebody's checklist somewhere.
So Security Theatre is stuff that looks like security but isn't. In fact it is probably worthless. This is why I was always unhappy with "process theatre" because, whilst in my mind it was describing a worthless process, it is definitely a process.
Reactive Processes
On my last 3 major engagements I worked for mature organisations, including one bank, who were in various states of confusion, inability to innovate and general lack of corporate agility. This type of work is often dubbed as "Agile Transformation" or "Digital Transformation", certainly "transformation" needs to be in there somewhere.
The big thing that I kept seeing over and over again were processes that had to be followed no matter what. Such processes were (probably) created in reaction to some bad event that had real consequences at some point in the distant past. The business responds by installing all sorts of governance around that thing which is evermore regarded as risky. Teams then evolve to own parts of this process. Over time the original intent of the regulation is lost. The teams that manage it now exist for the purpose of that process. There is no knowledge, or desire, to understand what outcome was originally being supported. The end state is that entire teams of people exist whose day to day job involves servicing a check list to ensure that if (when) things go wrong, they don't get blamed.
So I am grateful to my colleague, Robin Weston, who introduced me to the phrase, Risk Management Theatre, apparently coined by our esteemed ex colleague, Jez Humble, to describe such processes. This is much better than what I was thinking. What we have is a process that is seen by some to manage a particular risk but which in fact does nothing to manage that risk. In fact in some cases it actually makes it worse.
So I was reassured that what I had observed and thought about was a thing. So I've been thinking about it a lot in the course of my consulting work.
Example - Code Freezes and Release Cycles
If you have worked in software delivery for any length of time you will have come across (unless you are very lucky) the "death spiral" of code freeze - testing period - release to production - pray - work late and work on weekends to fix stuff - throw out fixes - rinse and repeat etc etc.
Long before Agile was a thing somebody released some code that broke something in production. This was probably before TDD was a thing, maybe there weren't even any kind of unit tests. Old bugs were reintroduced, nobody really knew what had changed, nobody could understand why it didn't work in production because "it worked on my machine" etc etc...
In response to this undeniably bad happening the business decided that it needed to test its releases more thoroughly. This seems like a reasonable response, indeed in 2004 it may well have been a reasonable response. So they create a test team somewhere. In order to do a release now you have to build your application, put it in some environment for testing and have this test team run through their tests scripts for a prescribed period before each release. Note that after you've built into this test environment you cannot make any further changes to the code that will be released (unless they are critical bug fixes), the code freeze. This is so that everybody can be sure that the test team is testing the stuff that will end up in production.
Ten years later this governance process still exists even though software development practices have moved on. We now have TDD, pair programming, cross functional teams, continuous integration and continuous delivery. We now understand what we have changed much better and moreover we have tested it at the unit level, the integration level and the user journey level. One of the big factors in our confidence is that we changed very little. This means we know what changed, we know what we needed to test and we know how to undo it very quickly if something somehow slipped through the net.
If we still have to go through code freeze and a testing cycle what happens? We try to cram as many changes as we can into the codebase before the code freeze. We cut corners on our Agile process to beat the deadline. We start to game the process by forcing new features past the code freeze by pretending they are bug fixes. We end up doing a big bang, waterfall style release. This is precisely the thing that caused the original problem. We have good solid practices in place that mean we don't need this whole process but the insistence of somebody, somewhere that that process is necessary perpetuates the exact behaviour that made it necessary in the first place. This is why I call this a "death spiral" - once it starts it is very hard to stop.
Corporate Immune System
So why do these teams and processes not get swept away? I think there are many answers, and I don't have all of them by any stretch. A good summary of some of the reasons can be found in the Corporate Immune System.
The organisation has evolved many processes, they have to be followed. There may be a blame culture within the organisation which prevents bold, innovative, employees from challenging the status quo. They first think of avoiding blame and second think of adding value. Innovation means change, which is something to fear for many people. There are managers, or even CXOs, who are heavily invested in the people supporting some of the risk management theatre. These often powerful voices argue strongly for the continued existence of the process that they curate citing examples from ancient history to show their worth. Other people have written far more eloquently and far more knowledgeably than I about the Corporate Immune System. I'm not saying the corporate immune system, or its side effects, are the only cause of the continuance of risk management theatre, but it certainly doesn't help!
How do we break the cycle of Risk Management Theatre?
I haven't fully formulated my thoughts on solutions yet. I have had varying degrees of success in the teams I worked with on my previous engagements. I'm now working in the public sector which introduces a whole new dimension of dysfunction that I hadn't previously seen. This is what prompted me to get my thoughts down about Risk Management Theatre as they stand. I want to take that as a baseline to hopefully move forward with this new client and hopefully come up with some solutions that may help even in the public sector. That is very much a work in progress!
My best answer at the moment is to reason with people involved in Risk Management Theatre about the outcomes they are supporting. If you can find a receptive audience, with an appetite for positive change, they should be willing to reason out the answer to "what outcome are you supporting" and extrapolate that thinking on to the question of whether the process they are using is the best way to achieve that outcome.
Customer focus is a great reasoning tool here as well. If you can ask the question above in terms of customer focus, i.e. "what customer facing outcome are you supporting with this process?" this can not only help an individual to reason about a single process but it can also help resolve a conflict between opposing processes and goals. For example, we have had success using the customer as a reasoning tool to resolve the tension between the development team who wants rapid change and the infrastructure team who oppose all change.
I will return to this subject when I understand better how to challenge (and hopefully reduce) risk management theatre in a public sector environment that cares little for any customer and may not even have a clear understanding or consensus about who their customers are.
Sunday 11 June 2017
Architechtologists
Recently I've had a few conversations along the lines of "what is architecture?" The natural follow on to that question is "what is an architect?" (I'll come back to that later).
Firstly, what is architecture? This is an interesting one. Depending on who I speak to I may get very different answers. It may well come down to the type of organisation people work in or perhaps the role they play within the organisation.
The best, or at least the most succinct, answer I've come across is that architecture is any decision that is hard to change later. This sounds very good to me because it plays to my feelings about drawing out diagrams and discussing a big picture and then talking over the consequences of arrows going from one place to another and what may happen if that bit is different and how that could affect the other bits it touches. But this definition also plays to my prejudice about what an architect should be. If we accept that architecture is a decision that is hard to change later then we must also believe that an architect makes such decisions or is responsible for such decisions. This idea would also closely align with how we think of as a bricks and mortar, as opposed to technology, architect.
But how often are people who either style themselves as architects or whose job title contains the word "architect" doing this kind of work, making these kind of decisions, and little else?
In my personal experience I have come across 2 other, more common, types of architects.
Firstly, it is a job title bestowed on software developers after they have been designated as senior developers. This often involves no real change in duties except more management. Sometimes there may be an "architecture group" which the architects contribute to. The architecture group is responsible for architecture decisions which in practice means "anything we do that affects more than one team".
Secondly, there are "architects" who spend their time trawling through legacy code and reading legacy (probably out of date) documentation in order to understand the existing technology landscape of their organisation. These people probably don't write code and possibly rarely even talk to people who write code. They explain, or attempt to explain, to development teams (or BAs) how things are and therefore what constraints exist on teams that attempt to create value for the organisation. These people are architectologists.
I'm interested at the moment in architectology. Why is it necessary and what does it say about the organisation that has them? The first question can be easily answered. You need them if you have a lot of legacy and a lot of knowledge silos. Somebody has to be there to help developers or BAs traverse the organisational silos in order to deliver value to the business.
What does it say about the organisation that has architectologists? Are they valuable or is there function waste? I haven't got a good answer to this question yet but instinctively it feels to me like they are waste. Instead of helping teams to traverse the silos and integrate to the legacy using the antiquated interfaces that exist it feels like the investment should be in decoupling the systems and creating a series of self service APIs that can be consumed by value adding applications. I'm currently investigating this idea more at my current client so hopefully we'll get to a better state and a better understanding of how we can get real value from the architects.
Firstly, what is architecture? This is an interesting one. Depending on who I speak to I may get very different answers. It may well come down to the type of organisation people work in or perhaps the role they play within the organisation.
The best, or at least the most succinct, answer I've come across is that architecture is any decision that is hard to change later. This sounds very good to me because it plays to my feelings about drawing out diagrams and discussing a big picture and then talking over the consequences of arrows going from one place to another and what may happen if that bit is different and how that could affect the other bits it touches. But this definition also plays to my prejudice about what an architect should be. If we accept that architecture is a decision that is hard to change later then we must also believe that an architect makes such decisions or is responsible for such decisions. This idea would also closely align with how we think of as a bricks and mortar, as opposed to technology, architect.
But how often are people who either style themselves as architects or whose job title contains the word "architect" doing this kind of work, making these kind of decisions, and little else?
In my personal experience I have come across 2 other, more common, types of architects.
Firstly, it is a job title bestowed on software developers after they have been designated as senior developers. This often involves no real change in duties except more management. Sometimes there may be an "architecture group" which the architects contribute to. The architecture group is responsible for architecture decisions which in practice means "anything we do that affects more than one team".
Secondly, there are "architects" who spend their time trawling through legacy code and reading legacy (probably out of date) documentation in order to understand the existing technology landscape of their organisation. These people probably don't write code and possibly rarely even talk to people who write code. They explain, or attempt to explain, to development teams (or BAs) how things are and therefore what constraints exist on teams that attempt to create value for the organisation. These people are architectologists.
I'm interested at the moment in architectology. Why is it necessary and what does it say about the organisation that has them? The first question can be easily answered. You need them if you have a lot of legacy and a lot of knowledge silos. Somebody has to be there to help developers or BAs traverse the organisational silos in order to deliver value to the business.
What does it say about the organisation that has architectologists? Are they valuable or is there function waste? I haven't got a good answer to this question yet but instinctively it feels to me like they are waste. Instead of helping teams to traverse the silos and integrate to the legacy using the antiquated interfaces that exist it feels like the investment should be in decoupling the systems and creating a series of self service APIs that can be consumed by value adding applications. I'm currently investigating this idea more at my current client so hopefully we'll get to a better state and a better understanding of how we can get real value from the architects.
Thursday 4 May 2017
Things I do in real life that show I'm a programmer
Sometimes the way I talk, the way I interact with people or just the way I do stuff betrays (at least in my mind) my background as a software developer.
Endless Refactoring
I am constantly annoyed by how inefficiently my wife loads the dishwasher. There are some spaces that are clearly suited to the pasta bowls and the dinner plates, some that are better suited to the cereal bowls. More than that, if you load the pasta bowls from the right and the dinner plates from the left, you can use each space on that row. If you load the pasta bowls from the left you cannot load dinner plates to the right of those spaces because of the way the bowls are shaped. This is really obvious. It is also simple. And yet, no matter how many times I point it out, she always does it wrong. The result of this is that the dishwasher appears full when it isn't. A simple re-organisation of the contents makes the space available that was otherwise wasted. I therefore constantly refactor the dishwasher after my wife has put stuff in it. I find I cannot just put more stuff in. If she puts a pasta bowl in the wrong position and it is the only thing in there, I HAVE to move it. I guess this is because I know from experience that if you leave smells lying around they morph into a horrible mess before you know it.
See also refactoring the fridge and the (very few) shelves allocated to me in the bedroom wardrobe.
Optimising Journeys
When you work in London there are many ways to get from one place to another. Many years ago a friend of mine (who is from the midlands and back then was not at all familiar with London) told me that he came to London for an interview and the instructions simply said "nearest Tube station, Euston Square". This was long before smart phones or Google Maps. So he got to Euston Station and looked on an Underground map. He saw that to get to Euston Square he needed to go to King's Cross (one stop on the Victoria Line) then change to the Circle Line and go one stop to Euston Square. The famous tube map gives no indication of how long it may take to switch lines at any station. It also gives no real indication of surface distance. So he made his journey and emerged from Euston Square about 20 minutes later. The first thing he saw was the street sign that says "Euston Station - 300 yards".
Not all journeys are so obviously sub optimal. But I have always tried to know where the exits are from the platform that I'm going to arrive at so that I can get in the most appropriate carriage. I love the latest version of London Midland's application because it tells me which platform the trains are leaving Euston by and thus which entrance to the station is optimal. Should I really be concerned with optimising my journeys to this extent?
Retrospectives
I guess this is more of an Agile thing than strictly speaking a developer thing. After a few years of indulging in blamestorming and defensiveness I started to be involved in the culture of continuous improvement as my previous employer evolved. When I moved to ThoughtWorks I finally shed the last vestiges of defensiveness and started to view regular retrospectives and feedback as opportunities to learn and improve. The trouble is when I try to engage people who aren't from this background in conversations about what went wrong or what could be improved or how we can avoid excessive risk, then I get a wall of defensiveness and accusations that I'm attacking people. Oh how I wish people could detach from the obviously over emotional reactions that have to advice or criticism and start to engage with opportunities to improve.
Am I expecting too much of people who haven't gone through an Agile transformation?
Subscribe to:
Posts (Atom)