Thursday 4 June 2020

Estimates v Priorities

Why Do You NEED That Estimate?

I have previously written, and talked, about my discomfort whenever I, or a team I am involved in, is asked to provide estimates. My reasons for discomfort have previously been chiefly:
  • How will this estimate be used? In my experience, no matter how much you emphasise that this is the best guess you can come up with today and your guess will become steadily more accurate, the date you guessed right at the start ends up becoming a pseudo contract to be used in a blame storm later about why we (a) under delivered or (b) padded our estimate. Whilst I accept that the culture of the organisation will dictate how an estimate is used, my point is that it is important to understand whether your "estimate" might (or is likely to) evolve into a "commitment". If you think it is likely that this will happen, push back about being asked to estimate.
  • Why does the business thinks it needs an estimate in the first place? I have rarely seen a powerful argument as to why an accurate estimate is more valuable than breaking the backlog into its smallest valuable units and then doing them in order of (potentially constantly adjusting) priority.
I think there is a good additional point to be made here (thanks to my colleagues Chris Bimson and Matt Belcher for pointing this out when reading my draft). Before even considering the above questions it may well pay to ask "what is the question that the estimate might help to answer?" This is entirely consistent with one of my mantras which is "tell me the problem to which this is the proposed solution". In this case an estimate is, of course, part of some solution domain, so it follows that the person requesting it must have some higher level problem which they think an estimate will help to answer. It could be that there is a better answer to that question, whatever it is, which would make the request for an estimate go away. In a healthy culture I would at least expect the person requesting the estimate to engage in this conversation.

The Iron Triangle

In project management we often talk about the Iron Triangle. In the original version, it was assumed that quality was constant and any change in one of three constraints, time, cost and scope necessitates a change in the others. In other words you have a fixed "budget" across the related constraints. 

The version I usually refer to for software delivery says that given a fixed throughput (a constant team capacity) and an unvarying level of quality, you can either fix the scope or the required time for the work (assuming some kind of accurate capacity planning technique), but you cannot fix both. This is, of course, the problem that many deliveries encounter when they fix scope in advance and attempt to force a development team to "commit" to a delivery date. The Iron Triangle tells us that this can't be possible.

The CAP Theorem

An equally well known triangular constraint based theorem is the CAP theorem. This theorem states that it is impossible for a distributed data store to provide more than two of these three guarantees:
  • Consistency (every read receives the most recent write or an error)
  • Availability (every request receives a non-error response but not necessarily the most up to date data)
  • Partition Tolerance (the system continues to operate despite an arbitrary number of dropped or delayed messages between nodes)
Given that every modern database must guarantee partition tolerance (because today's cloud infrastructure cannot guarantee that partitions won't happen), the CAP theorem in modern databases can be reduced to an acceptance that any data store can guarantee consistency or availability but not both.

Unplanned Work

Unplanned work happens all the time to all sorts of people and teams and is the enemy of accurate forecasting. Some people in some roles have some reasonably well understood and repeatable methodology for forecasting a sensible level of unplanned work, in which case they may call it a "contingency" plan. Often, building projects will add a fixed percentage for unplanned work and they call it a contingency.

Of course, no amount of contingency or forecasting of unplanned work can substitute for the real solution to unplanned work which is to make your environment more predictable so that you do much less of it.

Why Might you Plan Ahead?

I prefer this question to "why do you need that estimate?" If I have to ask a business owner, "why do you need that estimate?" it means that somebody has asked me for an estimate. Often the reply comes back that "we need certainty over planning" or something similar. Leaving aside the obvious desire to shoot this argument down using 5 Whys ("why do you need certainty over planning?" is usually far enough for anybody to stumble over their own dogma), I will assume that there is a legitimate reason to want to have certainty over planning. 

A desire to have some kind of certainty over planning implies that they backlog represents a relatively small number of large things that are considered to return large chunks of value only when they are complete. Such items are often called "features" or "epics".

Why Might you Always do the Most Important Thing Next?

In some circumstances it is legitimate to ask, when you have capacity to take on more work, what is the most important thing for me to do now? If the most important thing to do can change, and every item on your list of work can be assumed to be independent of every other piece of work and deliver some kind of independent value, then it makes little sense to plan things beyond asking "what is the most important thing for me to do now?"

A Conjecture

Drawing inspiration from the Iron Triangle and the CAP theorem and noting the similarity between a system involving three constraints (one being fixed) I have constructed the following conjecture:

Assuming that the capacity of a delivery team responsible for delivering items in a single backlog remains unchanged, you can either manage your backlog to optimise for doing the most important thing, or you can optimise it for accuracy of prediction of delivery dates. Any given backlog being serviced by a single team of unvarying capacity cannot be optimised for both the most important thing and accuracy of prediction.

Optimising for the Most Important Thing

You must accept that unplanned work at any point could change the definition of the most important thing, potentially at very short notice. 

If you optimise for doing the most important thing at any given point you can only give any sensible delivery date for something that is in progress. Anything else, even if it is at the front of the queue, is at risk of being deprioritised and relegated down the queue. So the best you can ever say is, "this is currently at the front of the queue, if nothing changes priority it will be live in X days", if it is any further down the queue you will have to make a judgement based on your experience of the level of unplanned work that your team will expect and give a confidence interval of delivery dates to the asker of the question.

Optimising for Accuracy of Prediction

How can I optimise for accuracy of prediction? Scrum, in its purest (and arguably most annoying) form, seeks to optimise for predictive accuracy, at least within a "sprint". The idea is that you have a good idea of your team's throughput for a given period (often 2 weeks) called a "sprint" and you "commit" to delivering that amount of work in the current period. But herein lies the catch. After declaring your sprint "commitment", Scrum says you cannot change it. Thus, you must not allow changes in priority, and you cannot allow your team to carry out unplanned (and by its nature unpredictable) work. 

Of course, Scrum predicates some fixed interval of planning a sprint which seems to me to be linked to old fashioned notions of fixed releases at predictable intervals. I would suggest that if you are following Scrum, you should consider shortening your sprints continually until they are a single item in duration. At that point you have moved from Scrum to Kanban with no loss of predictability, provided that no unplanned work will be taken on.

Conclusion

You must make a choice between speed of delivery of the most important item in your backlog or predictability of longer term delivery dates.

As a corollary to the above, if your team is expected to undertake unplanned work, you can never accurately estimate delivery dates.