Before You Build an Analytics Agent - Issue 307
Getting started with agentic analytics - how to set up business context, benchmarks, and structured workflows
I’m starting a new series in my newsletter on AI for analytics: how to set up agents, connect multiple data sources into a single agentic system, evaluate models for analytics tasks, and build AI systems for reporting.
This will be a long series, which I’ll alternate between my usual product analytics, monetization analytics, and data science projects. There is too much here to cover well in one or two editions, so I’ll break it into sections:
Data preparation and AI readiness
Agents 101, including the difference between agent-powered and agentic analytics
Prompting for analytics tasks across Gemini, Claude, and OpenAI (won’t be covering Grok)
Setting up guardrails, baselines, and evaluation for agentic workflows
Hallucinations, failure modes, and ways to make AI-generated reporting more trusted and reliable through governance, audit trails, and cross-model validation.
There are countless ways AI can be used across product development, marketing, and monetization. But as you have probably guessed already, I’ll stay focused on analytics only, and most of my examples and case studies will stay within this domain.
Today’s publication is the first step in this series and focuses on AI readiness for analytics and data preparation. It is aimed at team leads and senior managers responsible for shaping the roadmap for AI-driven reporting, self-serve analytics, and broader data workflows. I’ll walk through the steps I use to bring business and product context into AI assistants and workflows. That includes building metadata layers, documenting evolving business logic, setting trusted benchmarks, and organizing question types before the model starts reasoning.
Before I begin, a quick announcement: I’ll be speaking at Apps in Motion on April 8 at NASDAQ in New York about driving mobile growth with AI. I’ll be joining FunnelFox, Adapty, and Paddle for a one-day conference focused on scaling subscription apps - from paid UA and web-to-app funnels to paywalls, payment infrastructure, and retention. If you’ll be there, come say hi!
The AI data readiness is a myth
There is a myth that teams can become “100% AI-ready” by June 25 at 11:00 am, right after the final alignment meeting.
Organizations hire agencies and security-compliant consultants to make their data “AI-ready,” then wait for the green light to finally plug in AI and expect it to do everything.
Some organizations build full project plans and deadlines around this. In reality, it never fully happens. Data keeps growing, and tools keep adding complexity. Every improvement in data quality reveals new gaps and new opportunities. The better your data gets, the more doors it opens, the more data gets loaded and processed, and the cycle continues.
