Alchemy and Science: Choosing Metrics That Work
This talk was held on Wednesday, May 30, 2018
Update: Ketan delivered a newer version of this talk at SXSW in Austin on March 11, 2019.
Quantitative measurement is essential to scaling businesses, processes, and products and making them better. It sounds easy: just pick a number, like clicks or conversions, and improve it. It’s easy to do, but it’s not so easy to get it right.
When it goes wrong, the problem is often misdiagnosed as using metrics at all rather than which metrics were used. Every story of metrics gone wrong is really a story of badly chosen metrics. Choosing a metric has been an unsolvable mystery so enigmatic most people don’t even know they’re trapped inside. These choices are a haphazard result of folklore, recipes, and blind guessing, a.k.a. “intuition,” rather than a rigorous, scientific set of reusable principles. Until now.
In this talk, I will explore the artful science of choosing metrics. You’ll walk away with deep understanding of principles and concepts that determine which metrics are useful for your product and which ones are a waste of time. You’ll gain practical knowledge you can apply to your product to make it even more awesome. This talk will also be presented at O’Reilly’s Strata big data conference.
Summary in PDF format: go.indeed.com/metricsthatwork
Speaker
Ketan Gangatirkar leads Engineering at Indeed for Job Seeker products. For the last 9 years, he's been helping millions of people get jobs. Along the way, he's made some good product decisions and some terrible ones. He used to think he had great ideas, but being wrong all the time cured him of that. Now all he wants is scientific evidence. He was analytical before it was cool. It's not cool, and it never has been, but he's still hoping. Someday. Someday…