In a previous post, I described using a measure-question-learn-improve cycle to drive improvements to development processes. In this post, I assert that this cycle can also help people understand their own opportunities for improvement and growth. It can be a powerful coaching tool — when used correctly.
At Indeed, we’ve developed an internal web app called Hindsight that rolls up measurements of work done by individuals. This tool makes contributions more transparent for that person and their manager.
Each individual has a Hindsight card that shows their activity over time (quarter by quarter). Many of the numbers come from Jira, such as issues resolved, reported, commented on, etc. Others come from other SDLC tools. All numbers are clickable so that you can dive down into the details.
When we introduced Hindsight, we worried about the Number Six Principle and Goodhart’s Law (explained in the earlier post). To protect against these negative effects, we constantly emphasize two guidelines:
- Hindsight is a starting point for discussion. It can’t tell the whole story, but it can surface trends and phenomena that are worth digging into.
- There are no targets. There’s no notion of a “reasonable number” for a given role and level, because that would quickly become a target. We even avoid analyzing medians/averages for the metrics included.
Hindsight in action: How’s your quality?
To see how Hindsight fits into the measure-question-learn-improve cycle, consider this example: Suppose my card shows that for last quarter I resolved 100 issues and had 30 issues reopened during testing. As my manager, you might be tempted to say, “Jack is really productive, but he tries to ship a lot of buggy code and should pay more attention to quality.”
But remember — the metrics are only a starting point for discussion. You need to ask questions and dig into the data. When you read through the 30 reopened issues, you discover that only 10 of them were actual bugs, and all of those bugs were relatively minor. Now the story is changing. In fact, your investigation might drive insight into how the team can improve their communication around testing.
Measure, question, learn, improve
In this five-part series, I’ve explored how metrics help us improve how we work at Indeed. Every engineering organization can and should use data to drive important conversations. Whether you use Imhotep, spreadsheets, or other tools, it’s worth doing. Start by measuring everything you can. Then question your measurements, learn, and repeat. You’ll soon find yourself in a rewarding cycle of continuous improvement.
Read the full series of blog posts:
- Imhotep: Scalable, Efficient, and Fast
- Using Metrics to Improve the Development Process (and Coach People)
- Metrics-Driven Process Improvement: A Case Study
- What’s Up, ASF? Using Imhotep to Understand Project Activity
- The Benefits of Hindsight: A Metrics-Driven Approach to Coaching
The Benefits of Hindsight: A Metrics-Driven Approach to Coaching cross-posted on Medium.