I have read a lot about metrics from many different contexts: Six Sigma, project management, and Scrum among others. Below are the top five things that I think need to be considered when selecting and interpreting metric data.

5—Metrics are meaningless without context

All metrics get their meaning from their context. What does it mean for a Scrum team to have a cost per sprint of $60,000? Without context, it means nothing. $60,000 for a four week sprint with a team of ten might be considered a bargain. But if the sprint is one week and the team size is four, then some might consider that an expensive proposition.

However, it might not actually be expensive at all if the team is highly specialized and is expected to deliver $25 million of value in ten sprints. A $600,000 investment with a return of $25 million is arguably a good investment.

4—Metrics are only useful if you know what to change

Before you start tracking a metric, ask yourself, “what should happen if a ‘bad’ value occurs?” If nobody will care or if there is nothing that can be done to make the next measurement better, then why measure it? The project team should focus on measuring things that can help them earn more value for the company faster.

3—Metrics are only useable by the person or persons that can fake them

Velocity is useful to the team because they are responsible for measuring it. If they fudge the numbers to get some artificial value, they will know it. Nobody, other than the team, should make any business decisions based upon team velocity. It is an absolute meaningless number to anyone other than the team. As soon as someone outside of the team begins to care about its value, the team will make an adjustment to give them what they want.

2—There is no such thing as a “good” or “evil” metric

I don’t believe in the idea of assigning strengths and weaknesses to people, but that is another story. Likewise, there is no such thing as a good or evil metric. Every metric that tells someone anything can be used for good or it can be used for evil. In this case, good means to help the team become more productive (earn value faster) and evil means hindering the team from earning value.

1—The only metric the business should care about is Net Value Earned

The, one, most important metric to the business is the net value of the team’s output. The gross value of the team’s output less the cost of creating the output is the net value to the business. This number can’t be fudged by the team. The business is responsible for assigning value to the work the team commits to perform each sprint. The sum of the value of each completed sprint is the gross value of each sprint. The sum of all completed sprints is the total gross value earned. Subtracting the total cost gives us the net value earned.

Sure the business can assign any value to the sprints but, as I mentioned in item number three, they only have a reason to be dishonest if they are going to share the metric with someone else, like the company’s shareholders.

I can’t think of any other metric that doesn’t devolve into net business value earned.

Customer satisfaction? Customer sat numbers from after a new feature is released may make us adjust the value of the product to the company, but we still end up with the same calculation.

Team productivity? How would you measure that? How about thousands of lines of code? That might be a meaningful number to the team, but if the business starts measuring KLOCs, they are guaranteed to magically see it increase to whatever they are told it needs to be. The only way to really measure true team productivity is by the value of the thing they build; which, again, gets us back to the same calculation.

Summary

Measurements are the only way we can tell where we are. They can warn us of impending doom and tell us when we are doing a great job. But they can only do this given the right context. Understanding the context without having the number is not helpful. But, having the number without understanding the context could be disastrous.

See you next time…