A Guide To A/B Test Evaluation - Issue 46

How to measure A/B test success, and why product experiments can be tricky and not always effective.

Let’s say you have successfully shipped an experiment with a winning +15% increase in your conversion. But somehow, this isn’t reflecting in your revenue data yet. Even more concerning, it didn’t improve overall retention nor increased LTV as you expected (and the test proved). Before you question the experiment data or blame your analyst (never do that…

This post is for paying subscribers