How Notion Scaled Experimentation - Issue 286

How the team grew and scaled A/B testing without losing rigor - lessons from StatSig Summit.

Today I want to share a recap of my favorite talk from the Notion data science team at the recent StatSig Summit in San Francisco.

It’s a great one for anyone leading data or analytics teams, or trying to scale experimentation from a few tests to dozens running at once. The Notion team shared what it really takes to build a strong experimentation setup: the right infrastructure, project process, analytics framework, and, most importantly, the right mindset.

I liked this talk the most because they didn’t chase flashy milestones like “50 active tests at a time”. Instead, they focused on quality over speed - how to build trust in results and make every test count. After all, if you can’t interpret one test correctly, running 50 more won’t help much.