How to measure technical debt

Technical debt is a reality of software development – the result of needing to weigh up long-term quality against short-term deadlines. The trick of course is striking the right balance to keep technical debt at an acceptable level, but that begs the question: how much ‘debt’ are you dealing with?

Measuring the levels of technical debt present in your systems – and particularly large, complex legacy software systems – is the first step to managing it. This can be a tricky task, as technical debt is a wide-ranging and often abstract concept, however having clear metrics in place is vital to identifying the right path forward, to ensure that your software systems can continue to support your business, and your users for years to come.

In this post I’ll cover what these metrics may look like – from understanding the specifics of your code to quantifying the wider organisational impact – and explore how you can use this information to improve your systems in a constructive and considered way.

What is technical debt?

First, a quick recap on what we mean when we talk about ‘technical debt’. The term refers to the consequences of not tidying up software as it’s built, and includes underlying issues in your systems that may be negatively affecting performance, security, scalability and more.

There are a number of reasons why technical debt may accrue, and often it’s the result of deliberate compromises or shortcuts made in order to deliver value more quickly, which over time can grow into major issues or blockers. Technical debt can also occur when software isn’t regularly maintained, and may be accelerated if the technologies underpinning your systems become unsupported.

Additionally, although accidental mistakes or poor implementation choices aren’t exactly examples of technical debt, their impact on the performance of your systems is similar – so it can also be a useful way to think about these kinds of issues.

The business impact

As you can see, technical debt covers multiple different and difficult to pin down concepts, which can make communicating these issues to non-technical stakeholders a challenge. However, while these stakeholders may not particularly care about the specifics of technical debt, they will certainly care about the impact on the business if it’s not addressed.

High levels of technical debt put your systems at risk of crashes and downtime, along with security and data breaches. Added complexity also leads to longer development cycles – increasing costs, delaying time to market, and reducing your ability to react quickly to new opportunities. Technical debt can also cause usability issues that make it more difficult for administrators to work with your systems, and contribute to a poor customer experience that negatively impacts satisfaction and trust.

Why measure technical debt?

Of course, the key is to act on technical debt before it leads to any of the above symptoms, which is why it’s so important to maintain a clear and current understanding of the health of your systems. Providing objective evidence of any technical debt is also crucial when making a business case to address it, to ensure all relevant stakeholders recognise technical debt as a priority.

Knowing how much debt you’re dealing with is additionally vital to support your subsequent planning efforts and deliver you a roadmap that effectively balances technical fixes and improvements alongside feature development (something I’ll look at in more detail a little later in this post).

Ways to measure technical debt

Static code analysis tools

Attempting to manually assess every line of your system’s code for errors, risks and other issues is practically impossible – however there are numerous sophisticated tools available that can help you with this. Known as ‘static code analysis’ tools, they examine your source code and report potential flaws without actually executing the programs, and may span multiple languages and code quality issues or be targeted at specific languages and aspects of code quality.

One of the benefits of using a static code analysis tool is that they typically output findings in a highly visual and accessible way, making it easy to communicate these findings to other stakeholders. To take an example, one of the tools we use regularly when conducting code reviews for our clients is PHP Metrics, which (among other things) uses a ‘cyclomatic complexity model’ where each file is symbolised by a circle. The size of the circle represents cyclomatic complexity, and its colour refers to the maintainability index – so the smaller and more green your diagram is, the better.

Other tools we use at Box UK include Sensiolabs Insight to discover bugs, Nessus to identify vulnerabilities, and PHPCS to assess coding standards. Between these tools we’re able to capture a number of metrics that help us to understand technical debt, including:

• Dependencies and inheritance
• Levels of test coverage
• Runtime errors such as missing commands, incorrect syntax, logical inconsistencies, control flow issues
• Security violations like nested passwords or unauthorised access to sensitive data logs

Technical Debt Ratio

Capturing these and other metrics will give you a good picture of the overarching quality of your code, and enable you to pinpoint specific issues. The information you gather here will also inform your Technical Debt Ratio (TDR) – an important measure when deciding how, and whether, to address your technical debt.

Technical Debt Ratio is calculated as follows:

(Cost of remediation / Cost of development) x 100

That is, the difference between how much it will cost to remove technical debt from your systems, compared to the cost of the original development.

By calculating the TDR across different elements of your system – as well as for your codebase as a whole – you can identify where action is needed, and prioritise areas for improvement. Understanding your Technical Debt Ratio can also help you make informed decisions about the future of your systems, as if your TDR is very high then it may be more time- and cost-effective to replace legacy elements of your system, or to replatform entirely.

Operational and business costs

Of course, in order to make a strong business case to stakeholders, it’s also important to measure the cost of technical debt to your organisation at an operational and business level. Referring back to the symptoms mentioned earlier can help here, and other metrics to quantify costs include:

• Development velocity (e.g. how many tasks are completed per sprint)
• Development flow (e.g. the time between starting a task and completing it)
• Time spent fixing security issues caused by technical debt, and on other general maintenance tasks that could have been avoided
• Lost revenue due to downtime, poor performance or abandoned purchases
• Unnecessary manual effort spent on complex/disconnected processes, or processes that should be automated
• Developer turnover (as a result of poor satisfaction working on low-quality platforms, or frustration at a lack of development progress)

How to use this information

Even with a clear understanding of what your technical debt looks like, and how it’s affecting your organisation, it can still be difficult to make a plan to address it – particularly when you’re also managing requests for new features and functionality, where the value is evident to all.

To ensure that you’re able to tackle problem areas of technical debt, consider setting aside a portion of your time and budget to specifically address these items. The work covered so far to measure your debt will be incredibly valuable in helping define exactly what this portion should be, and when prioritising the order in which you tackle technical debt issues. Other factors to consider in your prioritisation activities include:

• How frequently a component/service is used
• Its importance to your business
• Whether there are any dependencies/integrations
• Any risks associated with your potential fix

Technical debt in Agile

As they are focused on continuous improvement and iteration, adopting Agile methodologies can further help you effectively manage your technical debt.

What is technical debt in Agile?

In contrast to traditional waterfall methods where development typically ends with the delivery of a finished product, Agile facilitates the ongoing evolution and optimisation of your systems. As such, technical debt items can be raised as part of your development backlog, and built into your future roadmap.

It’s a good idea to review your current backlog and tag any issues related to technical debt, and create an overarching technical debt ‘epic’ to enable clear categorisation that will support you as you continue to develop your systems. It’s also important to gather information from your wider team, as areas of technical debt can be easily overlooked, particularly if team members have put their own workarounds in place – the collaboration and ceremonies of an Agile framework such as Scrum can help you manage this, and ensure all relevant items are added to your backlog.

As above, when dealing with technical debt in Agile/Scrum, you may want to set aside a certain portion of the sprint (e.g. a set number of tickets or story points) for technical debt items. You may also want to dedicate a whole sprint (or multiple sprints) to technical debt, to get your systems under control and provide a stable base for future development.

Long-term management

It’s important to recognise that addressing technical debt isn’t a one-time thing, and you’ll need to have regular conversations about potential debt – as you continue to develop your systems and in response to changes in the technology and business landscape – to ensure you can keep your levels of debt at an acceptable and manageable level.

Documenting your conversations is also recommended, to provide a record of the decisions made and (crucially) the reasons behind these decisions, adding valuable context for future discussions. If you’re using Agile methodologies you should also use your project backlog to keep a record of technical debt – for example, if a compromise is agreed to bring a feature to market quickly, the additional work to fix this should be ticketed and factored into your development plans accordingly.

External factors can also add to technical debt, as technologies become outdated or, worse, unsupported, so it’s important to understand the lifespan of each of the technologies you rely upon, and make a plan that accommodates these, before they become an issue.

Your roadmap to success

In this post I’ve taken you through the what, why and how of measuring technical debt, and while it may seem intimidating, getting to grips with the levels of debt in your systems will make managing this much easier in the long-term.

A code review is a great first step to help you understand how your systems are performing and where improvements can be made, and at Box UK we’ve seen how organisations including Sodexo, RS Components and Jaguar Land Rover have used the outputs of their code reviews to expand and evolve their digital offering.

To learn more about the role a code review can play in helping address technical debt, download our white paper on the subject, and get in touch with a member of our team to find out how we can help you move your software systems forward.