Wednesday, 26 August 2020

Pascal's Big Bang

The Big Bang Cycle of Fear

I have written and talked in the past about the Big Bang Cycle of Fear (similar to its cousin, the Tech Debt Cycle). This is a cycle that has happened in countless places over the decades. Its end game - long cycle times, lots of failed releases, low automation and general fear around changing anything - is the state that I have been introduced to at the start of many engagements as a consultant, first for ThoughtWorks and now for Codurance.

The Big Bang Cycle of Fear (extended version) goes:

  1. There was a "bad" release that caused some production problems.
  2. The response is "we have to do more tests" but instead of automating tests, because the codebase is in such a state that test isolation is hard to achieve, we add more manual testing process and possibly more "testers".
  3. In order to be able to "test everything properly" before each release, we do a "code freeze" after a set period and then assign a set period for testing of that "release candidate".
  4. At the end of this testing period nobody is confident that there isn't a problem but the "regression packs" passed with only minor issues so it is probably OK. In any case, the product owners are screaming for the release of "my feature" to go ahead.
  5. Ostensibly to discuss the risks associated with the release and therefore whether the release should go ahead, we have a long and costly meeting with all the team leads, the product owners, some key developers, the head of engineering and possibly the CTO. In reality the purpose of this meeting is to share the blame for the failure when it comes and thus ensure that no single person is blamed and possibly fired.
  6. The release goes ahead.
  7. There is a production disaster and we go back to step one.

Understanding the Issue

The first step to solving any problem is understanding that you have a problem. I think most people involved in a BBCF would recognise that they have a problem. But would they be able to identify the probable root cause of the issue? I've recently read some books about systems thinking and one of the insights in there was that people normally look for solutions to problems at the point at which the problem manifests. I think in the case of our cycle, it isn't clear where the problem happened because it is a cycle. Thus, it might not be clear where the best place to look for a solution will be.

Of course, most organisations flounder around for months, possibly (even probably) years not understanding that they are causing their own problem. A things goes wrong in production and they blame the technology part of the organisation. Of course, they do, it is a technology problem isn't it? Sadly all too often the response of the CTO is also to not understand the issue. So instead of tackling the real cause or fixing the real problem, they decide that they have to do more testing, which manifests in some common anti-patterns.

More Manual Testing

Sometimes an organisation will hire more "testers" for their "QA team", not realising that you can't test quality in after the event. Maybe they insist on a longer code freeze to make sure that they can test everything, not understanding that by doing this, they are exponentially making it it harder to test effectively. The worst way that this gets done is when the organisation already outsources its testing (we do development in house, of course!) because it is seen as lower value, and thus they hire dozens more people at the outsourcing firm, probably several time zones distant, ensuring that any feedback loop is automatically extended by at least a working day.

More Automated Tests

A seemingly more sophisticated response is to decide that what is needed is more automated tests to cover the existing functionality. They have no people available to write these tests, or the skills needed, so they hire a "Test Automation Consultant" to write a suite of tests. This seems on the face of it to be a sensible short term expense to solve a long term problem and it may even appear at first to be effective, a classic anti pattern trait. The trouble is that these "automation test consultants" leave once they have written the tests, nobody will then maintain the tests, they will then start to fail as soon as changes are introduced. Now the company is stuck with pipelines that won't complete and which probably take hours to run because they are "end to end" tests, because that is what you asked for.

How Many Things Can Go Wrong?

A couple of years ago I was working as an adviser for a company who had undergone, and were still undergoing, this big bang cycle of fear. Instinctively it feels wrong to make a big release. The more changes there are to release, the more things can go wrong. I noticed that the testers (separate from the developers) had added tests at each new release which were by hand tests, so each successive release necessitated more testing for any subsequent release. This would have been OK, of course, if the new tests added were all automated and could be run in a reasonable amount of time. Sadly, they weren't automated and so they couldn't be run in a reasonable amount of time. So straight away they had introduced a linear scaling problem.

New Problems

The strange thing, from the perspective of the managers at this company, was that even though they had enough people to run all of these "regression packs" at each stage and thus, they thought, verify their existing functionality, they still experienced strange, unforeseen issues. "How can this be?" they asked, "when we are running all of these regression scripts?" It was clear to us that the answer was simple: over time the design of the system had degraded to the extent that there were all sorts of tight, loose and temporal couplings between the various parts of the system. Nobody had a sense for what these couplings were and nobody had any idea about what they were intended to do, let alone whether they were doing it correctly. So clearly they weren't considering enough things in their testing. They needed to consider interactions between the things.

My Mathematical Question

I went to a meeting with all of the senior managers just after another failed release in which they had released in the region of 30 different things. As usual they had gone through their ritual of blame sharing, which they called "Go / NoGo" so therefore they were safe in the knowledge that no single person in the room was going to carry any cans. In fact, the interactions between the different parts of their solution were so unpredictable that they didn't even know which "thing" had failed. They just knew the release had caused issues.

I started with a simple question:
If I release one thing, how many things can go wrong?

Clearly, the answer here is 1. So I followed up with another simple question:

If I release two things, how many things can go wrong?

At this point there was a bit of a murmur in the room but the general consensus was that two things could go wrong. "I beg to differ", I said, "Thing #1 can go wrong, Thing #2 can go wrong OR the unexpected interaction between Thing #1 and Thing #2 could cause a problem." Nobody argued with this because that was exactly the experience that they had been having.

So the next question I posed was, naturally:

If I release three things, how many things can go wrong?

Now, the maths starts getting a little complex because at this point each of three individual things can go wrong, Each or three interactions between pairs can go wrong or an interaction involving all three things can go wrong. So we now have 3 + 3 + 1 = 7 places to look for our problem. I could see the group starting to appreciate where I was going with this.

Pascal's Triangle

So naturally, my train of thought led me to think that there must be a simple formula that tells us how many things can go wrong when you release N things. At first I thought I was looking at the formula for adding the first N numbers, that is to say N(N + 1) things, or O(N2), which is bad enough, but then I realised it was even worse than that.

You may remember Pascal's Triangle from studying maths at school. My recollection of it was that it was introduced to illustrate the Binomial Theorem. I realised as I was going from 3 things to 4 things... to N things that what I was seeing was the successive rows in Pascal's triangle:


So if you start from row zero (the single 1 at the top) and number the successive rows, then row 5 contains the numbers {1, 5, 10, 10, 5, 1}. If you ignore the initial 1 (which can be regarded as representing number of cases where no things interact with no other things and thus can't be the cause of any issue) you can see that there are 5 single things that can go wrong, 10 pairs of things interacting that can go wrong, 10 triplets interacting that can go wrong, 5 sets of 4 things interacting and a single set of 5 things. So we can see that if we release 5 things, there is a potential 31 things that can cause us problems.

As well as representing the coefficients in a binomial expansion the numbers in the Nth horizontal row of Pascal's Triangle add up to N2 so given that we know the first 1 is irrelevant for this discussion, the conclusion is that if we release N things, we are causing a potential N2 - 1 issues and therefore need to test all of those things to be sure that our release will not fail. Even if we could rule out certain interactions and therefore reduce the overall universe of possible interactions, the number of things that can go wrong, and therefore could need to be tested, when we release N things is still O(N2) and thus of exponential complexity. So the scary answer for my client back in 2018 was that if you as a group agree to release 30 things simultaneously then you need to test 30- 1 = 899 things. It is easy to see that you are stretching the limits of human scalability and thus why your releases fail every time.

What is the Answer?

The answer is simple and, to a lot of people, obvious. Release one thing at a time. This means reduce your batch sizes until you are doing continuous delivery. Find a way to gain confidence around your changes so that they can be released as soon as they are "done", not weeks later. This could mean you do no new work (or reduced amounts of work) while you pay down technical debt, it could mean assigning developer effort away from new features but probably most importantly it HAS to include taking a long hard look at your Work in Progress (WIP) and aggressively reducing it. 

How can you do this and still deliver value steadily? Well, for a short period of time you can't. You have to accept that your rate of release of new features will reduce. BUT, ask yourself how often you have released new things without causing new problems and then ask yourself what is your REAL rate of return of value to the business? The calculation will be different for every system but ultimately if you don't gain confidence around your working system your throughput will eventually grind to a shuddering halt.

I think the desired end point is clear. We should all be practising continuous delivery which means tiny releases very often. Depending on the current state it could be easier or harder to get there or it may not even be possible without some kind of large scale software modernisation program. Hopefully most people will see that particular state coming and do something about it before it arrives.

Conclusion

The bigger your release the more likely it is to fail. The combinatorial mathematics shows this.

Use Pascal's Big Bang to demonstrate to stakeholders the fallacy of adding extra testing cycles.

To understand your real throughput consider releases as complete only when they have no remaining issues.

Reduce batch sizes aggressively until you are comfortable with continuous delivery.

The best form of cure is prevention! Don't suffer from boiling frog syndrome. If your levels of technical debt are increasing, slowing delivery and causing problems for releases, don't allow your company to fall into one one of the common anti-patterns that inevitably lead to the Big Bang Cycle of Fear.

Tuesday, 25 August 2020

On the Giving and Receiving of Feedback

What is Feedback?

I joined ThoughtWorks in 2015 after working at a startup for 10 years and a few other companies before that. The culture shock when I moved to ThoughtWorks from the (hero and blame) culture of my previous employer was huge. It was so huge that I struggled to adjust in my early months and at various stages in the first year was close to failing the probation period and / or quitting.

One of the weirdest things for me at the time was the culture of feedback. In all of my previous jobs personal feedback meant a once a year conversation with a manager in which the manager picked fault with your performance in the previous year and ignored anything good you had done in order to justify a miserable annual pay rise and a disappointing bonus. Feedback was thus only used for bad news, was so infrequent as to give no opportunity to improve and was strictly a one way process which, like most nasty things, only flowed downhill.

When I moved to ThoughtWorks (my current employer, Codurance, has a similar culture of feedback) my experience was very different. Firstly, in the two day induction there was a module on giving and receiving feedback. Secondly, personal feedback for the purposes of salary review was a very different thing. Instead of a manager (neither ThoughtWorks nor Codurance has the concept of reporting lines) using feedback as a way to beat down your salary expectations, the responsibility is put on the individual to gather feedback as and when you see fit in order to improve yourself in your role and possibly to support your argument for a pay raise at a later point. The key point being that it is up to you to seek feedback and also up to you how to solicit it, how to record it and how or whether to act upon it.

Instant Feedback

Instant feedback is extremely useful when used well because, at the risk of sounding obvious, it gives you a short feedback loop. Anybody familiar with Agile values should appreciate the value of short feedback loops. For example, have you ever been in some kind of feedback discussion, focussing on a large period of time, when somebody says to you something like "sometimes you can be arrogant"? This is all well and good and you might be inclined to say, or think, "I'll try to be less arrogant" but the chances are you weren't aware of being arrogant when apparently were being so. You may well, therefore, ask "can you give me an example of when you perceived me as having been arrogant?" Unfortunately, the person talking to you probably can't and therefore you are unsure what behaviour is being thus characterised and you can't act upon this feedback.

Giving Instant Feedback

I was taught to give instant feedback following three simple rules:
  • Do it as early as possible after observing something.
  • Tell the person what they did or said.
  • Tell them how it made you feel.

As Early as Possible

This is quite obvious. The earlier feedback is given referring to a specific incident the fresher in the memory of the feedback giver and receiver it will be. Moreover, you have the readymade example so need to ask for a specific example of a general behaviour.

What Happened

This should be quite clear as well. Tell the person what they did or said that is causing you to give them some feedback. Provided you have followed the first rule of instant feedback, there should be no dispute or interpretation at this stage. You are simply stating a fact of what happened.

How it Made You Feel

This may seem a little strange but is actually extremely important. The reason why you tell the person how it made you feel is because they cannot dispute this. Only you know how something made you feel. The difference between saying "you were trying to make me look small in front of the client" and "your comment made me feel like you didn't respect me" is enormous. The first comment can easily be met with an indignant (and very possibly justified) "No I wasn't!", the second comment can only prompt further useful discussion.

My Experience

Once I was used to dynamic of receiving and giving this type of fast feedback it became hugely useful for me when I received it and hugely satisfying to give it. Yes, there were sometimes hard conversations but almost always the conversation contained at some point a phrase such as "I had no idea that could make people feel like that, thanks for letting me know, please point it out if it happens again." I felt liberated and most importantly never again took part in conversations that went "You do things that to belittle people....it's just the way you talk to people....I can't think of a specific example, but you do it all the time..." The most satisfying aspect of all was that I could see, both when I gave and received feedback like this, is how much more valuable the early, instantly actionable, feedback was compared to if we had waited some considerable time before attempting the same conversation.

The Reinforcing and The Constructive

Just to be clear, this type of feedback giving should also include positive, reinforcing feedback. I realised that just as not having a specific example for behaviours that you think should be curtailed, not having a specific example for a behaviour that should be amplified is extremely frustrating. So my habit became to frequently give small nuggets of feedback, good and bad, but always immediate and actionable. It is my belief that none of that feedback was resented or ill received.

Longer Term Feedback

Longer term feedback is a bit harder to do effectively but unfortunately it seems to be much more common than instant feedback. There are many problems with it, mainly that the passage of time will have dimmed memories of certain incidents as previously mentioned. It also may be way too late to take any useful action on the back of the feedback that has been given as it could be many months after the actions that prompted it.

Why Engage in Long Term Feedback?

Annual Pay Reviews

As I've mentioned above the most likely reason that people may be having a conversation about actions and behaviours that have occurred during some long period of time will be to support the case for pay reviews. Most companies will have an annual pay review involving all their staff in which they have some kind of allegedly fair compensation formula which will reward everybody as they deserve but which in reality will force the whole of the organisation at every level of magnification to rank the peers and force their pay awards into some kind of normalised distribution that will result in a pre determined fixed rise to the total wage bill. Have I mentioned that I'm not a fan of this type of one-way-no-discussion-allowed feedback?

Project or Milestone Retrospectives

Sometimes teams break up, or reform at least, at the end of a project or at some significant milestone. It is also possible, particularly if you are a consultant, that people will regularly rotate in and out of engagements. In recent years I haven't spent more than nine months working for the same client. So it is often seen as appropriate to give and receive some kind of retrospective feedback from the people with whom you have worked for a time and from whom you may soon be separated.

Career Progression Discussions

A third use case for a retrospective style giving and receiving of feedback is to gain some kind of view on your colleagues' perceptions of your readiness or suitability for some kind of career progression. It could be that you believe that you are deserving of a promotion and you want to verify whether you are being reasonable or not. It could be that you have certain blindspots to your own shortcomings, or even your own strengths, and therefore you want to engage your colleagues in order to understand what some of these blind spots might be.

Some organisations, in fact many, will combine the annual pay review conversation with career progression conversations.

So How do I Do It?

how do you approach the giving and receiving of long-term feedback? This is a question that wasn't really addressed in a systematic fashion in the time I was at ThoughtWorks. In fact, other than saying things like "We have a feedback culture" or "We value the giving and receiving of feedback", I haven't seen many organisations that give their people much guidance on how to give and receive periodic feedback. 

Most of the time I have seen people sending emails to a group of colleagues soliciting feedback. More often than not it will say something like "Please reply to this email with some feedback." I never found it easy to respond to such requests. Too often, the response ends up being strong in terms of praise and very weak in terms of actionable points. Sometimes people request a meeting to discuss feedback. In those circumstances it can be even harder to give anything other than bland, "well done" type feedback. I think there are a number of reasons why this is but primarily it is because it is hard for many people to to give constructive feedback. Most of us are just too nice. The other reason I think is similar to what I mentioned above in that it can be very hard to come up with useful examples (because of the passage of time) of things to improve on which makes it difficult to mention certain areas.

Make it Comfortable

Having gone through a couple of frustrating cycles of less than useful feedback I alighted almost by accident on a method that seemed to work well for me, and which I shared with many of my ThoughtWorks colleagues. Firstly, one should recognise that feedback is useful and should be regarded as a gift. Therefore, make it as easy as possible for people to give you feedback. So when I would send an email soliciting feedback, I would give the recipients the option of how they felt most comfortable with giving the feedback. I would typically use something like "I'm happy for you to answer my questions by email, or if you'd prefer, we can meet face to face in the office, over Zoom, or in a pub after work if you like."

Make it Explicit

Finally, and I think this is the key, make your request for feedback more explicit. Instead of saying "please give me some feedback on my performance", try something like:

I'm looking for some feedback on my project performance. In particular could you please consider the following questions:

    • Is there anything I do as a Technical Lead [or whatever] which you think I should be doing less of?
    • Is there anything that you would expect a Technical Lead to be doing that I'm not doing?
    • Is there anything I could have done better in my direct interactions with you?
    • Can you think of any times when you haven't got what you expected from me personally?
    • What do you think I need to do more of to be considered as a Principal [or whatever the next level is]?
    • Is there anything I need to stop doing to be considered as a principal?
My experience was that five or six targeted questions yielded up actionable points that I could actively work on improving or eliminating as appropriate. I would have a pool of perhaps 8 or 10 questions that I would use for a single round of feedback and would share a different subset with different people depending on how we had worked together. For example, I would be more likely to ask another of the technical leads about interactions with our stakeholders, because they were probably in a lot of the same discussions, than I would be to ask the same question to my team's graduate developer.

Conclusion

  • There are two types of (very different) feedback. Instant feedback for immediate, actionable feedback on recent specific events and longer-term feedback on performance over time. Be clear on your reasons for giving and your expectations for receiving both.
  • The golden rules for giving instant feedback are do it soon, stick to the facts of what happened and tell that person how their actions made you feel. Most importantly, do not attempt to speculate or second guess the recipient's motivation for their actions.
  • Make it as comfortable as possible for people to have what could be an uncomfortable exchange.
  • Be explicit in your request for a long term feedback.

 

Thursday, 4 June 2020

Estimates v Priorities

Why Do You NEED That Estimate?

I have previously written, and talked, about my discomfort whenever I, or a team I am involved in, is asked to provide estimates. My reasons for discomfort have previously been chiefly:
  • How will this estimate be used? In my experience, no matter how much you emphasise that this is the best guess you can come up with today and your guess will become steadily more accurate, the date you guessed right at the start ends up becoming a pseudo contract to be used in a blame storm later about why we (a) under delivered or (b) padded our estimate. Whilst I accept that the culture of the organisation will dictate how an estimate is used, my point is that it is important to understand whether your "estimate" might (or is likely to) evolve into a "commitment". If you think it is likely that this will happen, push back about being asked to estimate.
  • Why does the business thinks it needs an estimate in the first place? I have rarely seen a powerful argument as to why an accurate estimate is more valuable than breaking the backlog into its smallest valuable units and then doing them in order of (potentially constantly adjusting) priority.
I think there is a good additional point to be made here (thanks to my colleagues Chris Bimson and Matt Belcher for pointing this out when reading my draft). Before even considering the above questions it may well pay to ask "what is the question that the estimate might help to answer?" This is entirely consistent with one of my mantras which is "tell me the problem to which this is the proposed solution". In this case an estimate is, of course, part of some solution domain, so it follows that the person requesting it must have some higher level problem which they think an estimate will help to answer. It could be that there is a better answer to that question, whatever it is, which would make the request for an estimate go away. In a healthy culture I would at least expect the person requesting the estimate to engage in this conversation.

The Iron Triangle

In project management we often talk about the Iron Triangle. In the original version, it was assumed that quality was constant and any change in one of three constraints, time, cost and scope necessitates a change in the others. In other words you have a fixed "budget" across the related constraints. 

The version I usually refer to for software delivery says that given a fixed throughput (a constant team capacity) and an unvarying level of quality, you can either fix the scope or the required time for the work (assuming some kind of accurate capacity planning technique), but you cannot fix both. This is, of course, the problem that many deliveries encounter when they fix scope in advance and attempt to force a development team to "commit" to a delivery date. The Iron Triangle tells us that this can't be possible.

The CAP Theorem

An equally well known triangular constraint based theorem is the CAP theorem. This theorem states that it is impossible for a distributed data store to provide more than two of these three guarantees:
  • Consistency (every read receives the most recent write or an error)
  • Availability (every request receives a non-error response but not necessarily the most up to date data)
  • Partition Tolerance (the system continues to operate despite an arbitrary number of dropped or delayed messages between nodes)
Given that every modern database must guarantee partition tolerance (because today's cloud infrastructure cannot guarantee that partitions won't happen), the CAP theorem in modern databases can be reduced to an acceptance that any data store can guarantee consistency or availability but not both.

Unplanned Work

Unplanned work happens all the time to all sorts of people and teams and is the enemy of accurate forecasting. Some people in some roles have some reasonably well understood and repeatable methodology for forecasting a sensible level of unplanned work, in which case they may call it a "contingency" plan. Often, building projects will add a fixed percentage for unplanned work and they call it a contingency.

Of course, no amount of contingency or forecasting of unplanned work can substitute for the real solution to unplanned work which is to make your environment more predictable so that you do much less of it.

Why Might you Plan Ahead?

I prefer this question to "why do you need that estimate?" If I have to ask a business owner, "why do you need that estimate?" it means that somebody has asked me for an estimate. Often the reply comes back that "we need certainty over planning" or something similar. Leaving aside the obvious desire to shoot this argument down using 5 Whys ("why do you need certainty over planning?" is usually far enough for anybody to stumble over their own dogma), I will assume that there is a legitimate reason to want to have certainty over planning. 

A desire to have some kind of certainty over planning implies that they backlog represents a relatively small number of large things that are considered to return large chunks of value only when they are complete. Such items are often called "features" or "epics".

Why Might you Always do the Most Important Thing Next?

In some circumstances it is legitimate to ask, when you have capacity to take on more work, what is the most important thing for me to do now? If the most important thing to do can change, and every item on your list of work can be assumed to be independent of every other piece of work and deliver some kind of independent value, then it makes little sense to plan things beyond asking "what is the most important thing for me to do now?"

A Conjecture

Drawing inspiration from the Iron Triangle and the CAP theorem and noting the similarity between a system involving three constraints (one being fixed) I have constructed the following conjecture:

Assuming that the capacity of a delivery team responsible for delivering items in a single backlog remains unchanged, you can either manage your backlog to optimise for doing the most important thing, or you can optimise it for accuracy of prediction of delivery dates. Any given backlog being serviced by a single team of unvarying capacity cannot be optimised for both the most important thing and accuracy of prediction.

Optimising for the Most Important Thing

You must accept that unplanned work at any point could change the definition of the most important thing, potentially at very short notice. 

If you optimise for doing the most important thing at any given point you can only give any sensible delivery date for something that is in progress. Anything else, even if it is at the front of the queue, is at risk of being deprioritised and relegated down the queue. So the best you can ever say is, "this is currently at the front of the queue, if nothing changes priority it will be live in X days", if it is any further down the queue you will have to make a judgement based on your experience of the level of unplanned work that your team will expect and give a confidence interval of delivery dates to the asker of the question.

Optimising for Accuracy of Prediction

How can I optimise for accuracy of prediction? Scrum, in its purest (and arguably most annoying) form, seeks to optimise for predictive accuracy, at least within a "sprint". The idea is that you have a good idea of your team's throughput for a given period (often 2 weeks) called a "sprint" and you "commit" to delivering that amount of work in the current period. But herein lies the catch. After declaring your sprint "commitment", Scrum says you cannot change it. Thus, you must not allow changes in priority, and you cannot allow your team to carry out unplanned (and by its nature unpredictable) work. 

Of course, Scrum predicates some fixed interval of planning a sprint which seems to me to be linked to old fashioned notions of fixed releases at predictable intervals. I would suggest that if you are following Scrum, you should consider shortening your sprints continually until they are a single item in duration. At that point you have moved from Scrum to Kanban with no loss of predictability, provided that no unplanned work will be taken on.

Conclusion

You must make a choice between speed of delivery of the most important item in your backlog or predictability of longer term delivery dates.

As a corollary to the above, if your team is expected to undertake unplanned work, you can never accurately estimate delivery dates.


Wednesday, 4 March 2020

Running a Futurespective

What is a Futurespective?

Most people will nowadays understand, at least broadly, what a retrospective meeting (usually abbreviated to retro) is all about. There are many different flavours of retro, and having worked for ThoughtWorks for 5 years, I met a fair few, but essentially you are looking at a period of work and working out what you did well, what you could have done better and what you might be able to do in order to improve your work in the future. But what is a futurespective?

The Goal

The goal of a futurespecitive is to imagine a future state, one would hope a desirable one, in order to then discuss what things can be done to help us to get there. So instead of saying "what did we get wrong?" which got us to where we are (which is usually where retros end up) we are saying "in order to get to where we would want to be, would do we need to do"? So in this way a futurespective is more goal oriented than a retro can ever be.

Time Frame

It is also worth noting that the time frame under discussion is usually much bigger for a futurespective than for a retro. A retro will generally focus on a period of weeks (sometimes we even talk about a sprint retro, which narrows down to the previous sprint, usually two weeks) or possibly months. At ThoughtWorks we sometimes did stuff like "milestone retros" during long running engagements, or "project retros" if the project in question was a much shorter thing. In a futurespective the focus tends to be much more strategic and therefore the question is likely to be along the lines of "imaging a year from now.... what would we need to do...?"

Different Techniques

I've been involved in a few different types of futurespective. Depending on the audience and the motivation for the workshop, you may choose a different technique. I've been involved in discussions framed around "What would good like a year from now for ThoughtWorks at this company", I've also been involved in "Imagine a year from now, the program we are kicking off is a success, what does that look like?" Right now, I'm talking about the second type of question. I try not to think about the first question in isolation, preferring a well aligned partnership with our clients.

News Headline

Our client has engaged us to talk about Software Modernisation of a particular system that is causing problems. My concern all week has been that we need to make sure that we anchor any modernisation goals within a framework of business value that is understood, articulated and shared. All too often I've seen technology-led change fail because the business value is not well understood outside of the technology group.

The technique that I facilitated today was the "news headline" technique. I asked the group to imagine a year from now that the program has already been delivering on its aims. What might some newspaper (or industry organ) have on its front page to report the remarkable success of our client in the previous year? I asked them in groups, to collaborate to produce a front page with a headline, 3 or 4 bullet points that expand on the headline, perhaps including a quote from "an industry insider" or an executive of this company and maybe a picture. We left them at it for a timeboxed period of 20 minutes. It was important to ensure that the people in each group reflected a cross section of the competences within the room. For example, the two business stakeholders, the two architects and the two product owners were split.

The Discussion

At the end of the 20 minutes we had two nice front pages (on A3 paper) reporting on this ideal future state. I'd love to share the photos here but they all had the client name on them so I can't share the photos without breaking client confidentiality. The discussion we had enabled us to pull out 6 bullet points that describe the aspirational future state from a high level viewpoint that we used to inform the subsequent discussions around the work that we should do and how we should go about doing it.

Outputs

Essentially, (and with some redaction to preserve confidentiality) we learned the following things from this exercise:
  • (our client wants to be) responsive to change and therefore improve its time to market
  • (our client wants to be able to) deal directly with its end users rather than through intermediaries
    • So they need to make it more easy to buy their stuff
    • The clients will save intermediary fees
    • (our client) will need to somehow provide directly the capability that the intermediary organisations have been offering to their customers
  • (our client wants to be able to) approach more partners and therefore needs to make its internal functions more scalable to achieve the capacity to make this possible
  • (our client wants to) compete with some massive players in its business space, this is currently not possible for several reasons which I can't go into here
  • (our client wants) to have a ubiquitous internal language to describe its product function so that it is possible to share understanding better both internally and externally

Next Steps

The outputs are useful to us on a few dimensions. Firstly this is helping us to frame what we may do in terms that mean something to the business. I need to be able to go to an executive stakeholder in a few weeks' time with a vision and a strategy that says "we should do this technology stuff in order to enable this business goal that has been identified".

Secondly, this exercise narrowed the scope of the subsequent discussion over what areas of improvement might be relevant. It helps us say "why would we want to improve that area of your estate when changing it won't contribute to these higher level goals?"

Thirdly, the workshop was fun, it aligned people in the room, and it gives us some cool photographs to use in our presentation back to the client next week when we talk about what we learnt, what we recommend and how we can help them to achieve it.

Monday, 24 February 2020

The Selfish Meme - A Conjecture

The Selfish Gene

The Selfish Gene was first published in 1976. Written by Richard Dawkins, probably now more famous, or at least more controversial, for his 2006 work, The God Delusion, its central theme is the notion that animals and plants are no more than "survival machines" manipulated by Genes to behave in a way that maximises the gene's chance to persist into subsequent generations. Clearly, individual animals and plants cannot be immortal, but genes can be. Certainly, genes persist far longer than individual survival machines.

Gene Alleles

Gene alleles are pairs of genes that compete with each other to persist into another generation. They drive behaviours that are somehow mutually exclusive to one another. The example given in The Selfish Gene is that of a gene that causes aggressive behaviour in animals that fight their own species for resources v a gene that causes passive behaviour in the confrontations in the same species.

Dawkins and Memes

In The Selfish Gene, Richard Dawkins coined a new word. This word is "Meme". His original definition is (I'm paraphrasing) "an idea, behaviour or style that spreads from person to person within a culture". Chapter 11 is entitled "Memes - the new replicators". In this chapter Dawkins describes how memes can be thought of as defining culture to an extent. In this sense, he is using culture to mean the culture of a society. Different cultures around the world and throughout human history have evolved a way to pass knowledge from one generation to the next such that the culture persists even though the individuals within the culture clearly do not.

Modern Memes

Most people probably now think of a meme as a thing, designed to be amusing in some way, that circulates, possibly "going viral", around the Internet. Some of my favourites would be Disaster Girl, XZibit - Yo Dawg (see one I made below) and the classic scene from Downfall. This last one is a sub-genre of meme where you put subtitles on some scene which are hopefully amusing in some way but are clearly not the originally intended dialog.

My XZibit - Yo Dawg Effort

Apparently XZibit is some kind of musician, a rapper I believe. He also was the presenter on a TV program, which I never watched, called "Pimp My Ride". It is from that program that I understand the Yo Dawg meme originates (I'm happy to stand corrected if I'm wrong, Know Your Meme doesn't talk about the origins of the meme, just the recursive structure). What I love about this meme is that "correct" use of it demands a recursive usage. I didn't immediately appreciate this and was told by a colleague that my use was incorrect and it had to include some kind of recursion. At the time, we were working on a Clojure implementation for our client. Imagine my joy then, when a few weeks later, I found out that the Clojure defMacro, which I had assumed was a keyword, was in fact itself a macro. I saw my opportunity to correctly use the Yo Dawg template and came up with something like this (my original is lost in the ether somewhere, so this is my best effort at a reproduction):


The Beginning of Infinity

David Deutsch's "The Beginning of Infinity" was first published in 2011. It is a study of epistemology. This isn't what I expected when I bought the book (I was expecting something about quantum physics or quantum computing and I never read the blurb), but it was still one of the best books I've ever read. Deutsch puts across an interesting conjecture. Running with the Dawkins idea that memes can define culture he argues that the rigid rules of the Spartan culture, passed from generation to generation as memes, eventually placed too many constraints on its ability to innovate. Thus, the Athenian culture, with its memes around learning and progress, was eventually able to finally conquer and all but destroy the Spartan culture. 

Transformation and Culture

So taking the two ideas together, Dawkins idea that memes can define culture, and Deutsch's idea that these memes can eventually be counter productive or damaging, led me to wonder if organisations in which I have consulted can have their culture classified or defined by their memes. If that is so, then perhaps by introducing competing memes (alleles) and somehow altering the value that the memes confer to its vectors, could I have a model for driving positive change?

Conclusion

I don't have a conclusion yet. I've been working on this idea and experimenting with clients that I've worked with. I don't yet have enough data points to draw strong conclusions but it has been fun and I've written a talk about the subject which I'll be presenting for the first time at Aginext in March this year. I was also asked to write an article for InfoQ on the subject. As soon as that goes live I will post a link to it from here.

Sunday, 29 September 2019

Quantum Supremacy and Cryptography

The story broke around September 20th that Google was claiming quantum supremacy. It merited not much more than small footnotes in the popular press. It was enthusiastically received in the technology press which makes me think that it is time to start thinking about a world after RSA is dead.

In 1994 Peter Shor published his paper “Algorithms for quantum computation: discrete logarithms and factoring”. At the time quantum computers were nothing but a theoretical figment of many fertile imaginations. Fast forward to 2019, with Google claiming quantum supremacy, and we should be taking them very seriously indeed.

“Quantum supremacy” means (if verified) that Google has a real quantum computer that can solve a real world problem more efficiently than any classical (digital) computer. If their claim turns out to be true, the chances are that the cost of using this computer is astronomical and certainly beyond the means any individual or probably any corporation. But so were IMB’s machines in their early days.

What we experienced back in the early digital age was what we should expect to happen now. As soon as practical uses exist, money will pour in and improvements in the technology will be rapid, probably exponential. Given that most research (outside of secret government research) has been funded by financial institutions, it is a fairly safe bet that those same institutions will be racing one another to translate quantum supremacy into financial market supremacy.

So why should we be concerned about this and what does it have to do with encryption? Well, Shor’s algorithm factorises numbers. If you can factorise a product of two large prime numbers in reasonable time, you break RSA and related cyphers, which account for pretty much all of the messages in the world now. And that is exactly what Shor’s algorithm promises.

We are a few years away from a time with a powerful enough quantum computer to break current RSA keys but if quantum supremacy is proved, you can be sure that time is closer than you think. And don’t forget, your messages are probably being stored by several government agencies the world over right now. Your messages are definitely secure now, but if you care about your messages being secure 5 or 10 years from now you should be asking why we aren’t using quantum safe cryptography now.

Wednesday, 14 August 2019

Why Codurance?

What Kind of Role Would you Like?

As I've said many times before, we are very lucky to work in the industry that we do. Even if we occasionally moan about what we get paid (and I certainly have done) it is very much a relative moan and not an absolute moan. My point is that most of us working in the technology sector who are any good at what we do can generally command a salary that is way above average and therefore generally "enough", whatever that means. So when I was looking for a new role this Spring and early summer I was doing so knowing that I could afford to select not only on the basis of pay. As I touched on in this post a couple of weeks ago, the fact that you can Google me and find interesting stuff made it even easier for me to get interviews for interesting things.

To be or not to be a Consultant

Very early on in my search I realised that I had a decision to make. Should I continue to be a consultant (and if so, what type of consultancy) or should I think about putting a stake in the ground as some kind of in house development leadership type person. The pros and cons, as I saw them after 4 years of consultancy, of the job were pretty clear:

In Favour of Consultancy:

  • The variety of work is interesting and leads to learning stuff about far more different technologies.
  • If you hate the gig you are on (i.e. your job) you can change to a different gig relatively easy. The ease with which you can change will be determined by the specific consultancy you are in I guess. Effectively though, you can get a new job without the pain and uncertainty of having to look for a new job.
  • There is a certain freedom of consultancy in knowing that you can't be sacked or "managed out" for being too daring or asking searching questions. I certainly played on that a few times where the conversation with a worried client might go "don't worry, if anybody asks, send them to me, I'll take responsibility, they can't sack me".
  • You get to do some travel to some places you may not have been to before.
  • You get to look at really interesting problems and when you solve them (or at least relieve some of the pains) you can move on to another equally interesting problem.
  • It never gets comfortable and therefore boring, and if it does, you can ask to move on.

Against Consultancy

  • You learn a little about a lot of things but don't always get the chance to learn a lot about any specific thing.
  • You start a lot of things and you may finish the odd thing but it is very rare to both start and finish the same thing.
  • The constant change of scenery can be unsettling and there are many periods of onboarding which can be wearing.
  • Sometimes you get sent away from home and don't get to spend time with your family.
  • Travelling on aeroplanes definitely stops being exciting.
  • Positions of influence are common but positions of true responsibility are rare.
I could go on, but the point I'm making is that it was a tough decision for me. I really enjoyed working as a consultant and I learnt loads in four years.

What was on the Table?

I interviewed for 4 or 5 roles outside consultancy. These were all either "head of", "VP of", "director of" Engineering, Development, Software type things or, in one case, a CTO role. I told the recruiters that I wasn't interested in anything that had a scope of less than several teams. So basically, in any small to medium sized organisation this means "head of" or CTO, in a larger organisation it could mean "architect" or "head of" or whatever they use internally. Based on what I enjoyed doing back in my Viagogo days and what I enjoyed doing during my 4 years at ThoughtWorks I think I had a reasonable idea of the type of company that I would like to work for (if I was to go to an in-house role) and, more importantly, the type of organisation I didn't want to work for.

Things I'd Like to work for

  • Somewhere with significant organisational problems that recognises it has such problems and has a good appetite to fix them.
  • An place that has been around for a while as a bricks and mortar, has loads of legacy systems but realises the need to move on and has the courage to appoint the right people to do that.

Things I wouldn't like to work for

  • Brilliantly run startups or scale-ups that have great technology, great engineering capability and practice and no significant organisational problems.
  • Companies that need to improve but either don't realise that they need to change or don't have the courage to do what needs to be done.
So what I really wanted was something with issues but a mandate to fix them. This is the kind of profile of the organisations that I consulted in to for ThoughtWorks, for the most part. I did have experience of one place where the CEO was blissfully unaware that they were heading (still are as far as I can tell) for a massive car crash at some point in the next few years and thus was unwilling to change what he thought was a successful course. That was an example of a company that was doing well on its bottom line despite its practices and capabilities, not because of them and was certainly a lesson for me.

Well funded Startup

One company I interviewed with up to the final interview state was an extremely well funded start up. Its parent is a well known multi national group. This new company was created to implement a brand loyalty scheme across the whole of the group. So it is a greenfield thing but there would be lots of potential pain in integrating the new systems with all the legacy of the companies within the group. The role here was "VP Engineering", reporting to the CTO. The initial responsibility would be building a team and working on the MVP of the new solution.

Well Run Tech Company

The other company I went quite deep with was a fairly well known company with a well known web presence. This company is renowned for good engineering practice and is often cited as a "centre of excellence" (I've never quite understood exactly what this means and it seems to be a bit of a self nominated, self selecting thing). In any case, I started to lose a bit of interest in this company somewhere between the third and fourth stage as I wondered exactly what type of deep problems they may have that would keep me interested.

Consultancies and Organisational Takeover

I spoke to a few consultancies. Some bigger (and uglier) than others. The more I spoke to, the more I realised that I had been lucky in working for ThoughtWorks. It seemed that once a consultancy gets beyond a certain size, whatever principles they started out with seem to get thrown away and replaced with a simple goal to rinse as much money from the clients as possible. I had heard this of really big players in the past ("organisational takeover" is apparently a real business model for some) but I, somewhat naively, assumed this was the preserve of only the biggest few.

Fake Agile

The most disappointing thing about consultancies in general is the promise of being an "Agile Consultant". There are many of these around, a lot of the smaller variety. I can't speak for all of them but I spoke to people in one or two and specifically asked them how they go about selling Agile to their customers. Sadly, I didn't get any kind of good answer. I was left, in every case, with the overwhelming impression that these consultancies were not only selling snake oil to unsuspecting clients but that they really didn't even get it themselves.

Worrying Conclusion

So after speaking to a few consultancies and going quite deeply into some interview processes for non consultancy work, I was left with the worrying conclusion that I would find it hard to find a company to work for that was sufficiently broken to need me (or keep me interested) but that also had the sufficient level of executive support for the work that I would know would need to be done. On the other hand, I was struggling to find a consultancy company that I could feel comfortable working with.

Meeting Codurance

I spoke to a recruitment consultant some time in May about a company called Codurance. I was starting to get disillusioned with my search and was resigned to a long summer of frustration. The consultant made all the right noises about this fairly small company and their ethos, largely based around software craftsmanship. He also spoke about how Codurance has no internal hierarchy, an interesting approach to innovation and a progressive set of policies on salaries. So I agreed to find out more.

Software Craftsmanship

The <Title> tag of the Codurance website includes the phrase "Software craftsmanship and Agile Delivery". It is clear that Sandro feels quite strongly about the importance of software craftsmanship. Indeed, in the early chapters of his book, the Software Craftsman, he argues that software craftsmanship should be a core part of Agile delivery but that this is often overlooked in the belief that if you follow the right process everything will work out. I can't argue with this assertion at all. For years I worked under the assumption that code quality was essential to the long term health of a system. I love the emphasis on craftsmanship (although I wonder if we can invent a term that is a bit more gender agnostic) and I do agree that it can be a forgotten element of effective delivery.

London Software Craftsmanship Community

I was even more encouraged when I discovered that Sandro founded the London Software Craftsmanship Community, a meetup group. After my experiences of the previous few years I realise how important it is for a company not just to make the right noises about culture and community but to actually do the right things, and believe in the right things, as well. In addition to the meetup group, Codurance also runs the Software Craftsmanship London conference every year at Skillsmatter in the autumn. So it was immediately apparent to me that Codurance does not just talk the talk but quite clearly it walks the walk too.

Agile Delivery

The other part of the title tag is Agile Delivery. In 2019 I would expect anybody doing software delivery to at least claim that they are Agile (or agile). I know from my travels that fake agile is everywhere and I was keen to understand what Agile means to Codurance. So when I spoke to Steve Lydford, head of PS, before being invited in for a face to face interview, it was very pleasing when he told me that he had watched a video of my Agile is a Dirty Word talk in which I vent (and despair a bit) about the prevalence of fake Agile practitioners, fake Agile methodologies and fake Agile consultants. That formed the basis of a good discussion between us which convinced me that Codurance as an organisation properly understands what Agile (and agile) means and that my experience of Agile delivery would be appreciated

The Opportunity

Codurance is a much smaller company than ThoughtWorks, which is itself pretty small compared to the global players that most people will have heard of. We are starting to grow organically and the current challenge, or opportunity as I like to call it, is to win more work that we would consider to be partnership type work rather than "pair of hands" staff augmentation work. We haven't got much experience of creating fully cross functional teams and this is our current challenge. How do we win and retain this type of work? This is the challenge we are facing now and my experience before Codurance seems to have been the most interesting thing for them looking at me.

Self Managing "Teal" Organisations

Steve recommended me a book to read, Reinventing Organisations, which describes a relatively new type of organisation that has evolved in recent decades. The author calls these organisations "Teal Organisations", which are essentially post hierarchical organisations based on management through self organisation. As I remember it, he used colours to categorise different types of organisation merely to avoid the name implying any kind of meaning. Having worked at ThoughtWorks, I know all about what it means to work in an organisation without hierarchy and I would absolutely not be able to work in an organisation that had anything other than a flat structure so this was all encouraging.

But there is flat structure and there is self organisation and true buying in to self organisation. Certainly I have encountered organisations that claim to have a flat structure but really they have a hidden hierarchy and command and control is everywhere. My early conversations with Steve suggested that Codurance was more genuinely bought in to self organising entities.

Innovation Circles

Through reading Reinventing Organisations and talking to people who have been involved in organisations that are trying to be self organising to a greater or lesser extent, the problem of how to change policy and ways of working comes up again and again. There are many different solutions and I would recommend anybody to read the book to find out how some organisations deal with this. In Codurance if you have an idea to change something you start an innovation circle. This has to be open to anybody that is interested and the discussions must be public. There is a rule on what constitutes a quorate group but the innovation circle is empowered to make changes and to implement them provided that the proposal passes the culture and financial tests. There is no need for approval from any "higher" power.

Open Salary Policy

Codurance has an open salary policy. I was told this by the recruiter when I first spoke to him and this was mentioned in every conversation before I started. Essentially this means that we have a spreadsheet that we can all look at that lists everybody in the company and what their salary is. At first this seemed a little alien and maybe scary but the more I considered it, the more I thought it was a great idea. I ended up reasoning to myself about what the difference could be between a company with open salaries and a company (i.e. every one I've ever worked for previously) that does not have an open salary policy.

Why Not have an Open Salary Policy?

In almost every place I have ever worked there is suspicion and rumours around salaries. Why would you not just publish everybody's salary so that everybody knows? In almost every company it is perceived that the barriers to promotions and pay rises are lower for people coming in from outside than they are for internal people. I don't know why this is but it leads to people changing jobs frequently in many cases as this is perceived to be the only way to get a decent pay rise or a promotion. If this is true, then I can see why you wouldn't publish. If somebody comes from outside with the same job as you but with no domain knowledge and they were getting paid more than you, you will likely be very upset, and rightly so.

I think perhaps the bigger point is that to move to an open salaries policy would mean that fairness in salaries across the organisation would inevitably have to follow and that fairness would cost a lot of money to implement properly. Either lots of people would have to be given pay rises to bring them in line or many people would leave as they perceived that their pay was still unfair even after some adjustments. Certainly nobody would be volunteering to take a pay cut. The cost of these adjustments would just be too great for most organisations to contemplate.

How This Worked Immediately

At my final interview I was told that I would be offered a role, the only question was how much the offer would be. I asked for a number and was told that this was quite a bit more than the only other person at the proposed grade. I pointed out that they had already told me that I would bring additional expectations because of my previous experience. This was accepted and they then told me that any offer had to be agreed by all the people that had interviewed me. As one of those was the person in question, who might be upset by my proposed salary, this seemed perfect to me. Not only would this other person know the outcome of any offer but would actually be involved in that decision.

Conclusion

I took the offer at Codurance because I was excited by the challenge of helping us to grow and mature our offerings, I was enthused by what I see as a genuinely flat, self organising structure and I love the culture of genuine openness that is obviously real and not just a veneer. I'm so far very happy with my decision and very happy to be a Principal at Codurance.