In early November, OpenAI said ChatGPT's search volume is now about 20% of Google's. In February, Ahrefs had estimated 12%. The gap looks like someone's mistake. It isn't, necessarily: the two companies are measuring different things.
I've spent more than 20 years building data pipelines for an airline, fintech and biometrics. If I've learned one thing, it's that a metric's definition decides the answer before any calculation.
What OpenAI said
Speaking with Alex Kantrowitz of the Big Technology newsletter, ChatGPT's consumer product lead, Adam Fry, shared strong numbers:
- 1.2 billion weekly active users, up from 1 billion two months earlier.
- 250 million weekly voice users, counting dictation and conversation.
- 1.5 billion images created each week.
- Search volume of about 20% of Google's.
All of these are the company's own figures. Fry also explained what counts as a search: every question that makes ChatGPT look something up on the web. He didn't say how he estimated Google's volume.
What Ahrefs measured
Ahrefs looked only at topics people have traditionally searched for, and arrived at about 12% of Google's volume. The same study found something else: Google still sends about 190 times more visits to websites than ChatGPT does.
One count includes every question that triggers the web. The other includes only what used to be a search. Both can be right and still tell very different stories.
Why it matters to decision-makers
At WEX, a US payments company, I automated data-quality checks with Great Expectations, and we cut critical incidents by 30%. At BairesDev, we used DataDog and Monte Carlo to hold data-quality SLAs. In both cases, the gain didn't come from more data. It came from agreeing on what each number means.
Every AI metric that lands on your desk deserves four questions:
- What's the unit? A conversation, a question or a web lookup?
- Who measured it? The company selling the product, or someone independent?
- Compared with what? What's the baseline, and how was it estimated?
- Over what period? A week, a month or a launch peak?
Without those answers, 20% and 12% are just nice numbers for a slide.
What I'd take to the next meeting
ChatGPT's growth is real and it matters. But if your company is going to change its marketing, SEO or customer service because of it, treat the number as you would any production data: with a clear definition, a known source and a plan to measure the effect on your own business.
A metric without a definition isn't data. It's an opinion with decimal places.