This episode is part of a wider mini-series looking at Estimation in Software Development. In this episode I want to take a deeper dive into the cost of achieving that - what is the correct cost for the perceived value of the estimate to you and your organisation. This episode continue on from 190 where I asked "Don't invest in estimates unless there are clear demonstrable value in having them"
Or listen at:
Published: Wed, 27 Nov 2024 01:00:00 GMT
Links
- #189: Estimation - a review
- #190: Estimation - do you get value
- #191: Estimation - what is a valuable estimate
- Wikipedia: Cone of Uncertainty
- Software Estimation: Demystifying the Black Art by Steve McConnell
- Thinking, Fast and Slow by Daniel Kahneman
Transcript
Hello and welcome back to the Better RIO from Software Development podcast
This episode is part of a wider mini-series looking at estimation in software development.
I started the mini-series back in episode 189 by providing the following guidelines: 1. Don't invest in estimates unless there are clear demonstrable value in having them. 2. Agree what a valuable estimate looks like. This will likely be a desirable level of accuracy and precision for an estimate. 3. Provide the team with the training and time to develop their estimation skills. 4. Collect data on points 1 to 3 and regularly review if you have the correct balance.
Subsequent episodes take a deeper dive into specific aspects for estimation in software development. And while long-term listeners may find an amount of repetition across the series, I want each episode to be understandable in its own right, as much as practical to be self-contained advice.
In this episode, I want to take a deeper dive into the cost of achieving that. What is the correct cost for the perceived value of the estimate to you and your organisation?
This episode continues on from 190 where I asked, don't invest in estimates unless there are clear demonstrable value in having them.
Back in episode 191, I introduced the shorthand of "Valuable Estimate", an estimate that is desirable for the organisation asking for it. And for it to be valuable, it is likely to need to have an acceptable level of accuracy and precision.
Let's start with a bold statement: Producing estimates cost - Producing valuable estimates costs a lot
In short, if you want valuable estimates, be prepared to invest time and resources in obtaining them.
Anecdotally, I would say that we attach more value to an estimate than we are prepared to invest in achieving it. It's super important that we have an estimate, but we shouldn't be spending time creating it.
For me, it's very much why the "off-the-cuff" estimate is so prevalent.
Off-the-cuff is a phrase I will use many times in this series. It refers to doing or saying something spontaneously without prior preparation or thought. In the context of estimates, an off-the-cuff estimate would be one made quickly and without detailed analysis or consideration. Such estimates are typically based on a person's immediate intuition or a rough guess, rather than a thorough examination of the relevant data or a structured estimation process.
It should be assumed that an off-the-cuff estimate will have poor accuracy, poor precision or both. Ultimately, we should consider an off-the-cuff estimate to be a low-value estimate.
In a future episode I will discuss two approaches to estimation, qualitative and quantitative, but in summary, qualitative estimation is predominantly based on expert judgment, something based on subjective thought process, whereas quantitative is based on something we can count or calculate, a use of statistical analysis based on historic data.
The off-the-cuff is a form of qualitative estimate and will be based on our intuition. System 1 brain as described in the book Thinking Fast and Slow by Daniel Kahneman. And as many books and studies show, the human ability to intuit a valuable estimate is poor at best.
Once again for this series, I reference back to Steve McConnell's book, Software Estimation - Demystifying the Black Art. In it, he says, "Don't give the off-the-cuff estimates. Even a 15-minute estimate will be much more accurate".
Thus, why are we so often expected to just provide the off-the-cuff estimate?
Ok, so fair warning, before I answer this, I will caveat this with, "in my opinion". And while it's an opinion built up over 30 years of industry experience, I'm not going to pretend it isn't somewhat biased from historical scar tissue.
Ok, so now the disclaimer is out of the way.
I honestly believe that in many cases estimates are just being asked for to just tick a box. It's purely a data point to be added to a spreadsheet, or worse, a Gantt chart. Thus, more often than not, we are purely being asked to keep the cogs of some project management process turning. Without my estimate, the spreadsheet can't be completed. And without the spreadsheet being completed, we can't produce a PowerPoint presentation. And without the PowerPoint presentation, we can't convince the higher ups that we're in control. And without convincing the higher ups we're in control, then we won't be left alone to get on with the work that we should have been doing in the first place.
In short, providing the off-the-cuff estimate is an easy and seductive way of getting management off your back.
Thus, we don't want to be investing valuable time into generating valuable estimates. Simply having something is enough.
This, for me, all comes back to the legacy approach of Taylorism and Command and Control. It's the lazy approach of gathering data, formatting data, presenting data. It's not being involved with the actual work being undertaken. It's the lazy approach of the higher-ups looking at a single slide red-amber-green status and then believing their responsibilities are done until the next red-amber-green status update in six months' time.
The modern world simply does not agree with this approach.
There are simply too many unknowns, too many variables, too many fluctuations in our organisations and markets for traditional command and control structures to flourish.
If we put my rant to one side, there are probably two main reasons for the prevalence of off-the-cuff estimates. We know no better, or we simply don't value estimates enough to invest properly.
So, let's hope the we-know-no-better is at least partially addressed through this series of podcast episodes. I freely admit that the research for these episodes has highlighted a number of approaches that I hadn't encountered before. And if I haven't come across them in my 30 plus years of professional experience and my questionable of obsession on all things software development, it probably isn't a surprise many would fall into that camp also.
So, let's focus more on the we simply don't value estimates enough to invest properly.
For this, we neatly segue back to how valuable that valuable estimate actually is to us. If we know how valuable the estimate is to us, then we can use that to inform the correct amount of investment to put into it.
Estimation comes at a cost. Getting good at estimation then comes at a further cost. And more often than not, it's time that the team could have been using to do the actual work.
But it's like any skill, it has to be practised and refined to obtain a good level of proficiency. We can't get a brand new set of oil paints for Christmas and expect to be exhibiting in the Louvre by New Year.
So if we are going to make that investment, the first step is making sure there is value in having the estimate in the first place.
Once you are confident there is real value in that estimate, we can then move on to what is the right amount of investment.
Unfortunately, as I said back in episode 191, there is no one single correct answer, and I feel myself slipping back into the consultant stock answer of "it depends".
But it really does depend on how valuable that estimate is to you. And of course, that will be very much dependent on your own requirements and ways of working.
There is obviously a substantial difference in value between a request for a new report and a new digital estate to support a new venture.
The former could be the proverbial 15-minute job, while the latter may require substantial outside investment to fund, let alone staff.
And then we have predictability, as I talked about in last week's episode. Having confidence our estimates are both valuable and have a level of predictability. So much so that business can make decisions against them.
I suggested a couple of approaches last week to help with creating a trend of better predictability over time, the blameless post-mortem to assess how we did against our estimate, and periodic re-estimation as we learn more about the work. Both great opportunities to improve and refine our estimation skill, but also to build trust with the stakeholders.
But if you're thinking this is all going to take time, then you're right, it will. There is the investment in time to create the initial estimate, and again in those reviews to improve and refine our estimation skills. There is considerably more investment needed than the simple off-the-cuff estimate.
And thus, with the potential for so much cost, we come back to my age-old question. Why are we asking for an estimate?
What is the purpose for it? What are we hoping to achieve with the estimate?
What changes are based on the response?
In short, there are a myriad of possible reasons why you want the estimate. And those reasons will impact the investment that is sensible. Because ultimately, this is a balance. There is no single correct answer.
Thus, in lieu of a better answer, I suggest you ask, why?
Or even better, the 5 Whys.
If you've not come across the 5 Whys before, it's a problem-solving technique developed within Toyota production systems. It's designed to identify the root cause of a problem by repeatedly asking the question, why?
The basic idea is that by asking why 5 times, you can move past symptoms and understand the underlying issues.
We start with the first why, then we continue to dig further asking the why against each response until we have gone 5 levels deep. This forces us to really understand the rationale.
This often reminds me of the story of a husband that asks his wife why she always cuts off the legs before cooking the Christmas turkey.
His wife responds with "it's how my mum did it, I've always done it this way".
Not happy with the response, the husband rings his mother-in-law, and she dutifully replies "Yes, I've always done it that way. It was something my mother did".
Again, not happy with the response, the husband rings the grandmother-in-law and she replies with "Yes, I always used to cut the legs off. My oven was quite small and it wasn't big enough to take the full turkey".
In our case, I suspect the response to the why are we asking for estimates will likely be "we've always done it this way", or "I'm being asked to", or some other incomplete reason.
We need to dig further. If it's something we've always done this way, why? Is there still value to it? Or are we still trying to reduce the size of the turkey that would have no problem fitting into our modern ovens?
As I've talked about in previous episodes, it's often that the true request for an estimate is not actually for the estimate. Someone's asking for the estimate to answer a different question. A different question that could be answered much better via other methods.
By getting to the real question, you may find that there simply is no value in the estimate, or that the estimate is not the most viable manner of actually answering the real question.
In this episode, I wanted to get you to think about how much you want to invest to get a valuable estimate.
As I've said in prior episodes, a valuable estimate would depend on individual circumstances, teams and organisations. Sometimes an off-the-cuff may be the correct balance between cost to produce and the value it holds.
But, as I've said, I would say that most attach more value to an estimate than they are prepared to invest. It's very much why the off-the-cuff estimate is so prevalent.
Other times there may be much greater value to having an accurate estimate.
Or it could be a case of digging into the "why" will produce an underlying question that is answered in a much more effective manner.
In next week's episode, I want to discuss what the term "estimate" may mean to you and the colleagues that you work with.
It often gets used interchangeably for effort to achieve, date by which it will be done, and even a commitment by which the work will have been done.
Often the meaning of an estimate changes between people within even a single team, let alone an organisation. Thus it is often a cause of friction and dysfunction due to it being very much in the eye of the beholder.
Thank you for taking the time to listen to this podcast. I look forward to speaking to you again next week.