This talk was held on Wednesday, May 30, 2018 at 6:30pm
We’re updating the accompanying videos. Please check back later.
Alchemy and Science: Choosing Metrics That Work
Update: Ketan delivered a newer version of this talk at SXSW in Austin on March 11, 2019.
Quantitative measurement is essential to scaling businesses, processes, and products and making them better. It sounds easy: just pick a number, like clicks or conversions, and improve it. It’s easy to do, but it’s not so easy to get it right.
When it goes wrong, the problem is often misdiagnosed as using metrics at all rather than which metrics were used. Every story of metrics gone wrong is really a story of badly chosen metrics. Choosing a metric has been an unsolvable mystery so enigmatic most people don’t even know they’re trapped inside. These choices are a haphazard result of folklore, recipes, and blind guessing, a.k.a. “intuition,” rather than a rigorous, scientific set of reusable principles. Until now.
In this talk, I will explore the artful science of choosing metrics. You’ll walk away with deep understanding of principles and concepts that determine which metrics are useful for your product and which ones are a waste of time. You’ll gain practical knowledge you can apply to your product to make it even more awesome. This talk will also be presented at O’Reilly’s Strata big data conference.
Summary in PDF format: go.indeed.com/metricsthatwork
Weapons of Math Instruction: Shifting from Data-Driven to Science-Driven Product Development
Data is a hammer, science is carpentry.
Building great technology products is difficult — scaling them presents even more challenges. The sheer amount of data now available can help with decision making for your products, and many companies are striving to be more data-driven. Data is only the start of the solution. Without a scientific process for consuming this data and deriving its meaning, you can end up with lower quality decisions and potentially slower decision velocity due to analysis paralysis. Teams can become distracted by the noise of large quantities of data and miss vital information. Thankfully, you can rely on scientific principles to refine and scope your use of data so that you’re asking the right questions at the right time in search of meaningful answers that will help you improve your products.
In this talk, I will introduce you to key data science concepts and illustrate how to use them in making robust decisions at scale. This talk leverages established principles from statistics, information theory, economics, and machine learning, to supply your team with a shared vernacular and evaluation framework. There will be math, but also practical examples and even some March madness in May!