« all talks

Alchemy and Science: Choosing Metrics That Work

This talk was held on Wednesday, May 30, 2018

Update: Ketan delivered a newer version of this talk at SXSW in Austin on March 11, 2019.

Quantitative measurement is essential to scaling businesses, processes, and products and making them better. It sounds easy: just pick a number, like clicks or conversions, and improve it. It’s easy to do, but it’s not so easy to get it right.

When it goes wrong, the problem is often misdiagnosed as using metrics at all rather than which metrics were used. Every story of metrics gone wrong is really a story of badly chosen metrics. Choosing a metric has been an unsolvable mystery so enigmatic most people don’t even know they’re trapped inside. These choices are a haphazard result of folklore, recipes, and blind guessing, a.k.a. “intuition,” rather than a rigorous, scientific set of reusable principles. Until now.

In this talk, I will explore the artful science of choosing metrics. You’ll walk away with deep understanding of principles and concepts that determine which metrics are useful for your product and which ones are a waste of time. You’ll gain practical knowledge you can apply to your product to make it even more awesome. This talk will also be presented at O’Reilly’s Strata big data conference.

Summary in PDF format: go.indeed.com/metricsthatwork