Introduction To Proxy Metrics - Issue 204
A guide to identifying and implementing effective proxy metrics with examples and use cases.
Welcome to my Data Analytics Journal, where I write about data science and analytics.
“It’s usually hard to measure the things we care about. So we compromise.” Sophie Alpert.
Today, let’s talk about proxy metrics.
A month ago, I published How To Find Optimal Proxy Metrics, in which I referenced a recent research study from Google and Stanford on the importance of using sensitive metrics instead of KPIs for A/B testing. Researchers introduced the concept of Pareto Optimal Proxy Metrics, which significantly optimized the accuracy and sensitivity of lift predictions.
That publication provoked a lot of discussion, and I received many questions about proxy metrics. So today, I want to take a step back and offer a “proper” introduction to the concept of proxies.
What are proxies? Why do we need them, or when do we need them? What are the methods for finding proxy metrics? How do you know when your proxy metric is not an appropriate representation of the KPIs? Examples of proxies and their use cases.
Keep reading with a 7-day free trial
Subscribe to Data Analysis Journal to keep reading this post and get 7 days of free access to the full post archives.