Turn up the prompt volume

At 10 prompts, a brand’s AI visibility score moves about 4x as much day to day as at 1,000, even when nothing changed. More prompts give you a number you can trust.

··
Ask AI about this postChatGPTClaudePerplexity

Key takeaways

  • At 10 prompts, a brand’s AI visibility score moved about 4x as much day to day as at 1,000 prompts, for the exact same brand.
  • At 10 prompts the score jumped 10+ points on almost 1 in 4 days. At 100 prompts that dropped to 3% of days.
  • AI breaks a long prompt into a handful of short background searches. Track the topics those searches cover, not the exact sentence someone types.
  • Visibility tells you the direction. Whether AI gets your brand right matters more, and about 9% of brand facts AI gave were wrong.

There’s a lot of debate right now about whether you can actually trust AI visibility tracking. Rand Fishkin raised some good questions on Ross Hudgens’ podcast this week. Prompt volume is basically a guess, most tools don’t run prompts enough times, and personalization means nobody sees the same answer anyway.

That same morning, the same topics came up in a conversation I was having with someone who runs AI search at a very large brand. So I wanted to see what the data actually says.

I’ve been in AI search since it basically started (founding team at Scrunch, now on the founding team at Searchable). Below is what I found in our data and in the research out there, and where I think it leaves anyone tracking their brand in AI.

What we know for sure

Nobody outside of OpenAI, Google, Anthropic and Microsoft knows how often people actually ask a certain prompt. Every prompt volume number you see in a tool (ours included) is an estimate built from things like search volume, clickstream data and the searches AI runs in the background. That’s fine, as long as everyone is honest that it’s an estimate.

One answer also tells you almost nothing. In a SparkToro study, 600 volunteers ran the same prompts close to 3,000 times, and there was less than a 1 in 100 chance of getting the same list of brands twice. Getting the same list in the same order happened about 1 in 1,000 times. So if a tool tells you that you rank #3 in ChatGPT, I’d take that with a big grain of salt.

That same study also found that how often a brand shows up across lots of prompts, each run lots of times, is a reasonable thing to measure. So the thing to figure out is how you get to a number you can trust.

More runs or more prompts?

One common fix is to run each prompt a lot more times, and a few tools do that, running every prompt up to 100 times per model. From what I’ve seen, the bigger problem is tracking too few prompts. If you only track 15, each one is a big piece of your score, so one answer flips and your number jumps even though nothing really changed for your brand.

So we tested it. We took Searchable brands that track 1,000+ prompts every day and looked at the last six weeks. Then we pulled random smaller sets of prompts from the same brand (10, 25, 50, 100, 250 and 500) and watched how much each score moved day to day. Same brand, same days, so the only thing that changed was how many prompts we counted.

Bar chart of the average daily change in a brand's AI visibility score by number of prompts tracked: 5.0 points at 10 prompts, 3.8 at 25, 3.1 at 50, 2.1 at 100, 1.7 at 250, 1.4 at 500 and 1.2 at 1,000. Going from 10 to 1,000 prompts cut the daily movement by 77%.

At 10 prompts, the score jumped 10+ points on almost 1 in 4 days. At 100 prompts, that happened on 3% of days. Going from 10 to 1,000 prompts cut the daily movement by about 77%, for the exact same brand.

Bar chart of the share of days a brand's AI visibility score moved 10 or more points: 23% at 10 prompts, 12% at 25, 6% at 50, 3% at 100 and 2% at 250, 500 and 1,000. A 10-prompt set lands about 12 points from the brand's real score on average.

The small sets were also just off. A 10-prompt set landed about 12 points away from the brand’s real score on average. That’s a lot of the gaps people find when they first start tracking. Some of them are real, and some are just what happens when you only look at a handful of prompts.

Running a prompt more times does help if you care about that one prompt (one academic study from the University of St. Gallen found it takes around 7 runs a day for a single prompt to settle). But if you want to know how your brand is doing overall, adding prompts gets you there faster than rerunning the same few over and over.

I’ll be honest about the trade-off though. The person I was talking with that morning pushed back that more prompts means more gaps, and their team is already stretched filling the ones they have. That’s fair. I’d still rather see every gap and pick the ones that matter than miss them because the sample was too small.

Track what the AI actually searches

Real prompts are also long and super personal, so trying to track them word for word doesn’t make sense. I don’t think you should try. When I use Claude for work, my prompt looks something like this. I’m a GTM engineer at Searchable, I’m trying to find the biggest gaps in our inbound and outbound, is there a tool that can help with all of it? Nobody is going to track that exact sentence (iPullRank found AI prompts run about twice as long as Google searches, with about 6x more personal detail).

The AI doesn’t search that sentence either. It breaks it into a handful of smaller searches in the background, things like lead routing software or best sales pipeline tools, and builds the answer off those. Seer Interactive looked at Gemini 3 and found about 10.7 of these searches per prompt on average. That’s the layer I think you should be tracking.

Example of one long prompt turning into five short background searches, such as lead routing software and best sales pipeline tools. Gemini 3 ran about 10.7 background searches per prompt on average, and 95% of those searches had zero search volume in keyword tools.

One catch. 95% of those background searches had zero search volume in keyword tools, and the exact wording changes from run to run. So I’d track the topics they cover, and use search volume to decide which topics matter most (not to count how many people ask).

Personalization is real too. One recent study found a single neutral answer caught about 63% of the brands a user with chat history saw, and 16 neutral answers caught about 95%. So a clean baseline, measured enough, covers most of what real people end up seeing.

Same idea with personas. A lot of teams want to know what results a CMO sees vs. a marketing manager. I’d start with what each of them is asking, use search and clickstream data to confirm those questions are real, and track more prompts that are specific to each one.

Is AI right about you?

I honestly don’t love visibility as the main number. It’s good for telling you if things are moving the right way, and that’s about where it stops being useful. The number I care about more is whether AI gets your brand right. Is it describing your product, your pricing and what you’re good at correctly, and is the tone positive?

We looked at this in Searchable’s brand data earlier this year. About 9% of the brand facts AI gave were false, and 75% of brands that checked at least 20 facts found at least one that was wrong. Pricing was wrong the most, about 13.5% of the time (brands graded their own facts, so grain of salt).

Brand facts AI got wrong: 9.2% overall, 75% of brands that checked 20 or more facts found at least one wrong, and pricing facts were wrong 13.5% of the time. By engine, Perplexity got 11.5% of brand facts wrong, Google AI Overviews 11.4% and ChatGPT 8.1%.

A lot of it comes from the brand itself. In almost every demo I do, we find a negative or wrong claim in the sentiment data that traces back to the company’s own website. One study of B2B tech companies traced 18% of AI’s mistakes to the company’s own pages. You’d only catch most of that with enough prompts to cover the questions people actually ask about you.

Nobody’s an expert yet

I didn’t come into this as a marketer. I’d been in martech for years but never in a marketing role, and it wasn’t until Scrunch that I started thinking like one (running tests, digging through data, looking at websites and learning SEO as I went). Honestly, I used to wonder if that would hurt me.

I don’t think it does anymore. Everyone in AI search is about 18 months in, including people with 20 years of SEO behind them. People who’ve been doing SEO for decades are still working this part out with the rest of us. The person I talked with that morning has been in the industry for 20 years too, and they said pretty much the same thing.

So I’d be a little careful with anyone calling themselves an AEO expert right now, me included. What I can do is share what our data shows as we keep testing. And if you’re only tracking 15 prompts today, I’d add more before you change anything else.

Sources

Preston Boling

Written by Preston Boling

AI search (AEO) expert. I was on the founding team at Scrunch and I’m on the founding US team at Searchable. Here I test what gets a site cited in AI search and write up what I find.

Get new posts by email

One email when I publish something new. What I tested, what moved and what didn’t.

Subscribe