Notes on metrics
Metrics can be a powerful and legible signal about what has happened in the business. The best way to use metrics is to dig deeper to understand and learn. Some important things can never be measured.
Last week, I shared a few articles on the limitations of metrics. Despite these limitations, I know that metrics are an important tool for building a great business.
Notes:
So I have summarised below my own thoughts on the do’s and don’ts of metrics.
Measure the right things. This is hard. The temptation is to measure what can be measured easily and discard that which cannot be measured. This is the McNamara Fallacy.
The specific metrics vary between businesses. Try to focus on three things: growth, customer delight and cash runway.
Measure outcomes not inputs.
Keep it simple. A small number of metrics that are easy to understand.
Don’t forget to track the things that can’t be measured: learning, employee development, changes in the market environment.
Metrics not targets. Metrics are backward looking. They tell you something about what has happened. That causes you to ask why. That is how you learn. Setting targets gets in the way of that process. This is called Goodhart’s Law:
"When a measure becomes a target, it ceases to be a good measure"
Don’t set limits. One dimension of the target problem. If you aim for a certain level of growth, what happens if you reach it? Do you want to stop there?
Incentives and disincentives. The other downside of targets. Once you report a metric, you create an incentive for others to drive that number. Every incentive is also a disincentive to do something else. Make sure you are incentivising the right behaviour.
No individual metrics or targets. Making one person “accountable” for a number is high risk. Splitting an overall metric into numbers for each individual is really dangerous. This always creates a disincentive for teamwork and collaboration.
The most important thing is in the middle of these notes. Metrics can be a powerful and legible signal about what has happened in the business. The best way to use metrics is to dig deeper to understand and learn. Don’t limit yourself to metrics. Some important things cannot be learned from data alone.
Reading:
The McNamara Fallacy by Matt Rickard. I only discovered the name for this a couple of years ago. Its one of the biggest traps in business: using the wrong metric just because you can measure it.
If you can’t measure it, you better manage it by Henry Mintzberg. Short but great article. Metrics are not essential for management. Far too often they are a substitute for good management and a way of avoiding dealing with things that really matter.
Organizational Behavior: Incentive systems get gamed by Arnold Kling. On a metrics theme this week. Every metric creates an incentive. In large organisations, every incentive gets gamed. The metrics become the score in the game instead of the story of the business.
“What gets measured gets managed” - It’s wrong and Drucker never said it by Danny Buerkli. Good article about some of the long known problems with metrics. Metrics matter but everyone should understand the risks of getting the wrong measures.
Ask them by Roy Lilley. One of the most engaging and realistic commentators on the NHS. Here he makes a simple point: all those NHS targets are inputs not outcomes. Patients' real concerns are about the outcome.