In this episode I want to introduce the term Technical Debt. The term Technical debt was coined in 1992 by Ward Cunningham to describe what happens when we fail to adequately maintain what he calls “immature” code. I find technical debt a great way to express choices we make in prioritising our software development investment.
Or listen at:
Published: Wed, 06 Nov 2019 16:50:40 GMT
Transcript
Technical Debt is a means of measuring and explain problems in our Software Solutions using the metaphor of traditional financial debt. As with the house building analogy (discussed in episode 4), financial debt has considerable shared understanding and is incredibly useful to increase shared meaning.
Technical Debt allows us to assign a metric to something that is otherwise overlooked, but is nevertheless extolling a cost on the organisation.
When Ward Cunningham introduced the term he commented that:
"Although immature code may work fine and be completely acceptable to the customer, excess quantities will make a program unmasterable, leading to extreme specialization of programmers and finally an inflexible product. Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a standstill under the debt load of an unconsolidated implementation.”
Wikipedia describes technical debt as:
"Technical debt (also known as design debt or code debt) is a concept in software development that reflects the implied cost of additional rework caused by choosing an easy (limited) solution now instead of using a better approach that would take longer.
Technical debt can be compared to monetary debt. If technical debt is not repaid, it can accumulate 'interest', making it harder to implement changes later on. Unaddressed technical debt increases software entropy. Technical debt is not necessarily a bad thing, and sometimes (e.g., as a proof-of-concept) technical debt is required to move projects forward."
Personally, I take this a little further and expand it to be anything that would be sub-optimal expense when operating and maintain a Software Solution.
As I discussed previously, every Software Solution will have an ongoing cost.
This is often overlooked in the understanding of its cost to an organisation.
Even the best written and designed Software will have some cost associated with its upkeep.
You will never get it to zero.
But there is a point of optimal cost – beyond that is what can be considered the “interest” you are paying on your technical debt. Each software system will have a different optimal cost – and there isn’t necessarily an easy way to establish that – which is possibly why technical debt can be quite a subjective measure.
While it is possible to measure – and thus report on – technical debt, it is not necessarily an easy thing to establish.
For me; just having an awareness of how “indebt” a system feels should be enough – rather than expending considerable effort in reporting and managing.
As long as the entire organisation is bought into that; it can be much more cost effective than performing all of that work – just to provide a business case.
That being said, the Software Development world isn’t perfect and sometimes we do need to put a number against it.
I’ve seen a few options, but they basically boil down to the amount of effort to return the Software to that optimal point.
To establish this with your Development Team, I’d ask them to draw up a list of anything that feels sub-optimal, then provide a development estimate for it. From that, you can establish a cost to fix.
Even this however doesn’t give a true view of the debt. This is how much it would be to remove the debt. You may be paying interest on the debt in other ways – low productivity, additional resources to operate, delays in changes, etc.
Some of these can quickly outweigh the debt cost.
If a cost can be attributed to the any sub-optimal item – great, throw it into any technical debt calculation you are preparing – but trying to actively seek them out can be expensive in its own right.
So what could be consider sub-optimal?
Again this can be difficult to quantify – it can differ from Software product to Software product.
There are things that would be considered universal good Software Development practice – many of the topics I cover in this podcast - such as automated testing.
You will then have those less universal affects that are specific to your environment and the Software. Those are the ones that would require review and analysis.
One good way to capture technical debt, is after a problem. If a system failure or change delay can be identified as being due to something being sub-optimal – great, make a note of it. You should also be able to value it as you’ve suffered for it.
Once you have some view of the Technical Debt, it is up to you to decide how much you are prepared to live with.
As with financial debt, it can be seen as an enabler, so not all debt is bad.
There will obviously come a point where it will cost too much for your business to afford on a day-to-day basis.
This will be similar to any risk analysis – impact + frequency. If it has minimal impact, but happens all the time then the net affect can be high. By the same token, it may have a high impact but happens exceptionally rarely, you could choose to accept the risk and carry on.
So how much is too much debt?
If you’re Development Team are spending any level of time keeping a sub-optimal system running, then I’d start to question if the debt shouldn’t be paid off.
If they are spending 5% of their time a week, that soon becomes a considerable impact over the longer term. If the “cost” to pay off the debt is two weeks development, then that will of course pay off over the lifetime of the Software.
Again, there isn’t a simple rule for this – however I err on the side of caution and advise paying off any technical debt as soon as possible.
Like financial debt, it is easy to ignore until the debt becomes un-serviceable. And as I say, technical debt can be difficult to measure (even with consider effort).
Martyn Cagan, author of "Inspired: How to Create Tech Products Customers Love":
"The deal [between product owners and] engineering goes like this: Product management takes 20% of the team’s capacity right off the top and gives this to engineering to spend as they see fit. They might use it to rewrite, re-architect, or re-factor problematic parts of the code base...whatever they believe is necessary to avoid ever having to come to the team and say, ‘we need to stop and rewrite [all our code].’ If you’re in really bad shape today, you might need to make this 30% or even more of the resources. However, I get nervous when I find teams that think they can get away with much less than 20%."
In the book The DevOps Handbook, a case study talks of LinkedIn in 2011.
Following a successful IPO, LinkedIn found themselves struggling to get new features released due to their technical debt.
LinkedIn would try to add a bunch of new things at once, then the site would crumble into a broken mess, requiring engineers to work long into the night and fix problems.
They reached the point that their technical debt was so great that they needed to stop adding new features - and fix the technical debt first.
Not a great position following an IPO and the market has its eyes on you.
The 2019 State of DevOps report commented that:
"We found that technical debt negatively impacts productivity, that respondents with high technical debt were 1.6 times less productive, and that the highest performers were 1.4 times more likely to have low technical debt."
I wanted to end with the acknowledgement that debt can be an enabler. It can be a good thing.
If you choose to actively cut corners to get some value quickly – great.
Just make sure to have a plan of how & when to service your Technical Debt. If it isn’t going to be a priority to service the debt, then I’d really advise against incurring it.
If used wisely, using Technical Debt (in the same way you use Financial Debt) can be hugely advantageous – but only if used wisely.