Evaluating Metrics
Good Continous Delivery podcast where they talked about how to evaluate metrics and their suitability to the situation.
If you don’t know how to respond to a metric, don’t measure it. This is a quote from How to Measure Anything) and is a clever way of asking why we need to measure something. If we’re not willing to act when we see a metric, what’s the point?
If there is additional work needed to create the metric and it isn’t a natural byproduct of the work, then we have to ask the question whether the effort of creating an observable metric is worth the opportunity cost?
Is the metic at the right level of granularity? If a team metric is shown to management or vice-versa, what effects will it have? Will it send the wrong signals which can snowball into waste?
Is the metric measuring output or outcome? Pure ouput metrics like Line of Code are not helpful. Metrics such as subscribers gained or software quality which are outcome-oriented are better.
Is the metric being observed producing the right behavours? If people are incentivized to do the wrong thing by the metric then it’s harmful.