ROI of Estimates

The real world of estimates

Let me start this article with a story;

Last Friday I was to have the house to myself (wife was visiting son at University), so like a true nerd I planned to spend the evening working on a development project.

For those that read my weekly updates, its part of my blog rewrite. The details of the project are not important; rather the acknowledgement that it is a reasonable neat and self-contained project that is only worked on by me.

It has been a slow project that I’ve been working on for a number of weeks on-and-off as infrequent spare time allowed. So the opportunity of a good few hours uninterrupted to get some traction was one which I did not want to waste.

As such I left work with a clear plans of what I wanted to achieve that evening.

During the 90 minute drive home however, a revelation stuck me – I’d not taken something into account in my earlier work. Again the details are unimportant, more that it would require me to make changes to existing work.

So that night, I spent 5 hours making those changes – largely to be at the same place I was when I left work.

The moral of this story

It was right for me to make the 5 hours of changes. Any further work on the project would have been difficult without it.

Should I have spotted the revelation earlier in my work?

While I would never claim to be world’s best developer; I know that I’m exceptionally capable and able to hold my own with some very talented people. I had developed the project with best principals. It was clean and certainly some good solid code.

So if I missed this, then so would most developers 9 out of 10 times.

So the moral of the story; estimates can (and statistically will) be wrong. If I missed this additional work in a project to which I am so intimately engaged – it will be happening on a daily basis within most development teams.

Does it matter if the estimate is wrong?

In my story above, other than a little frustrating that I didn’t get as much done? No.

It’s my own code, my own time – it doesn’t really matter.

If this was a tightly controlled project, then yes it very much could. On a smallish project with a committed deadline, then a days’ worth of “unexpected” work would likely cause many a project manager to breakout into a nervous sweat.

And in organisations where there isn’t an acceptance that an estimate is just an estimate, then this can drive some terrible behaviour. Rather than accept that this is a natural part of development (when and how often – not if), it is immediately seen as a negative – a failure.

Project Managers and even developers will look at how to “absorb” that additional time to avoid that “failure” being seen. And this can push otherwise great developers into cutting corners in an attempt to hiding that perceived failure.

Take my story above; I could have ignored it or tried to work round it. At some point I would have paid the price for it though. Be it difficult & expensive bugs or increased complexity (and thus cost) for next time I had to make changes. For the longer term ROI of the software, fixing it there and then was 100% the correct thing to do (and just accept my disappointment at not being further forward).

At this point, I will acknowledge that there are instances where it may be appropriate to do a quick fix – but in doing so, everyone should be aware (and accept) that the project is taking on a Technical Debt (see ROI of Technical Debt). The key though is to accept that this is the case and not sweep it under the carpet.

Not hitting an estimate is not an excuse to incur Technical Debt on its own.

Planning and ROI

Around this point, you may be wondering if I’m going to go into a rant about not using estimates in Software Development.

And indeed there is a #NoEstimates movement that advocates exactly that. Ron Jeffries provides an excellent article here on the subject.

Here are a couple of quotes taken from the article to give a flavour:

“Estimation is difficult, perhaps impossible, and we would be well off if we could avoid it. That’s why the #NoEstimates idea is so compelling.”

“…estimates are often misused. Estimates are often used as a bludgeon to try to get programmers to work faster. This leads to unhappy programmers and to poor software. No one wins.”

Ron is without doubt a fan of the #NoEstimates movement – but is also keen to highlight its problems – primary for me is the ability to assess expected ROI.

Any investment should be considered in terms of anticipated ROI – roughly speaking the benefit vs the cost.

So if you have an idea that is expected to make £1000 but will cost you £10,000 – then you’ll likely bin the idea and look at something else.

But life is never that black and white – benefits will be a guess and so will the cost (our Software Estimate in this case). And sometimes that gets lost in management layers – people want absolutes – How much exactly will this cost? How much exactly will this make?

This simply isn’t realistic to expect from any project – and if bluntly shows naivety and lack of experience in the manager that expects it. A manager makes his career through an ability make judgement on fuzzy details … generally the higher they go, the fuzzier they are.

If the question was absolute then we wouldn’t need the manager to make the decision – it would be automatic.

We do however need to provide some level of educated guess – both in terms of the benefit and the cost – otherwise we are in dart throwing territory.

So what we are looking for is a balance between a “ball park” and not wasting everyone’s time generating an estimate which we know to be wrong.

An estimate is an estimate – not a commitment

Let me reiterate that last point – the estimate will be wrong.

Every estimate every given has been wrong. Every single one.

The question is not if an estimate is wrong – rather by how much. It will simply never by an absolute – that is the nature of an estimate.

If really want to know how long it will take, let me do the work, then I’ll tell you exactly how long it too. Simply put that is the only reliable value you will ever get. Anything else is … well a guess.

And as long as everyone understand that an estimate is a guess, then we can provide sensible estimates for the ROI planning, but not drive dysfunctional results.

How do we control spend?

It would be wrong of me to conclude this article without briefly talking about controls.

If you are funding some Software Development, you want be confident that it will not spiral out of control – incur huge costs and fail to deliver – and this is one of primary reasons why the estimate is asked for AND then treated as a commitment.

So we know that estimates are fairly useless beyond giving a general idea – so how do we avoid spending millions on something that won’t work?

By regular small iterations.

We break the work down in to small chunks (my preference is 2 weeks), work to deliver and then review at the end of that time. We then repeat over and over again.

And we do that until one of two things happens – the work is completed or the investment is no longer viable.

At each of those review points, we know exactly what has been done – even if we are vague on how long it will take us to complete the rest. Having working code at every review point, allows the investor to confirm to themselves if the project is on track – much better than having a Project Manager confirm that the project is in line with their Gantt chart.

And of course it allows the investment to be killed if the project just simply isn’t working – maybe the idea was wrong, maybe the team are struggling to turn idea into reality – whatever the reason, it produces a much safer environment from an RIO perspective. Take a look at my Agile Software Development articles (Part 1, Part 2) for more on this.

Summary

For me the key takeaway from this article is that estimates are just that … they are estimates.

To treat them as anything else is encourage dysfunction and poor ROI.

Used correctly they provide benefit. Just make sure you are using them for the correct thing.