Why classic SEO audits miss AI search, and what to check instead

Classic SEO audits miss AI search because they measure whether Google can crawl and rank a page, while AI engines decide something else: whether they can read the raw HTML without JavaScript, whether a specific passage is worth quoting, whether your brand resolves to a known entity, and whether you actually appear in their answers. A site can pass every on-page check and fail all four. The fix is not to throw the old audit away. It is to add the checks it was never designed to run.

This is not a knock on the incumbents. Tools like Seobility, Screaming Frog, and the site audits inside Ahrefs and Semrush are good at what they were built for, and several have added AI features over the last year. The issue is structural: a crawler-and-rules audit grades the page, and AI visibility is decided partly by the page, partly by the rest of the web, and partly by a non-deterministic model.

Ranking on Google and being cited by AI have come apart

The strongest evidence that you need a second scorecard is how little the two outcomes now overlap. In a March 2026 study of 863,000 keyword results and 4 million AI Overview citations, Ahrefs found that only 37.9 percent of URLs cited in Google's AI Overviews also appeared in the top 10 results for that query. Its earlier analysis had put the overlap near 76 percent. The likely cause is query fan-out: Google's own documentation says AI Overviews and AI Mode issue "multiple related searches across subtopics and data sources" and cite pages that rank for those related searches, not just for yours.

Outside Google the divergence is larger. When Seer Interactive compared ChatGPT search citations against Bing and Google for the same 100 queries, more than 87 percent of the citations matched Bing's top organic results, while the Google match rate was 56 percent, with a median Google rank of 17 for the cited pages. An audit that only asks "how does this page rank on Google" is blind to the index ChatGPT leans on.

The five things a classic audit was not built to measure

1. Whether AI crawlers can read the page at all

Many desktop crawlers offer a JavaScript rendering mode, and in that mode they report the page a browser sees. That is the right view for Googlebot. It is the wrong view for GPTBot, ClaudeBot, and PerplexityBot, which the Vercel and MERJ crawler study found do not execute JavaScript at all. A client-rendered page can therefore pass a rendered audit with a perfect heading structure and still be an empty shell to every AI crawler. The check you need compares the raw HTML with the rendered DOM; we walk through it in which rendering modes AI crawlers can read.

2. Whether each AI crawler is actually allowed in

Classic audits read robots.txt for Googlebot and usually stop there. AI visibility depends on a dozen separate user agents, with separate rules for training and for search: blocking GPTBot does not block OAI-SearchBot, and blocking ClaudeBot does not block Claude-SearchBot. On top of robots.txt, CDNs and firewalls increasingly challenge or block AI bots by default, which a robots.txt parser cannot see. Our AI crawler directory lists every agent and what blocking it actually does.

3. Whether a passage is quotable, not just whether the page is optimized

Answer engines lift passages, not pages. A page can have a perfect title tag and a 1,500-word body and still contain no paragraph that answers a question cleanly on its own. Classic on-page checks look at the page as a unit: title length, heading presence, keyword placement, word count. None of them ask whether the paragraph under your main heading would make sense if it were pasted, alone, into an AI answer. That is the property that gets you quoted, and it is measurable paragraph by paragraph. See passage extractability.

4. Whether your brand resolves as an entity off the page

Some of the strongest correlates of AI visibility are not on your site at all. In Ahrefs' study of 75,000 brands, branded web mentions correlated with visibility in AI Overviews at 0.664, while backlinks correlated at 0.218. A page audit cannot see how often the rest of the web talks about you, or whether your Organization schema links your brand to the profiles a knowledge graph trusts. Correlation is not causation, but the gap between those two numbers is large enough to change where effort goes. We cover the off-site side in brand mentions are the new backlinks.

5. Whether you actually appear in AI answers, measured honestly

The final gap is the outcome itself. A classic audit infers visibility from inputs. AI visibility has to be observed: ask the engine the question a buyer would ask and record whether you are in the answer. That sounds simple, and it is where most tools get misleading. In January 2026 SparkToro had 600 volunteers run the same 12 prompts through ChatGPT, Claude, and Google's AI 2,961 times. The engines returned the same list of brands less than 1 percent of the time, and the same list in the same order less than 0.1 percent of the time. A single screenshot, or a confident "AI rank" from one run, is noise. What holds up is how often you appear across repeated runs, reported with its uncertainty.

What a classic audit still gets right

None of this makes the old checks obsolete. Google states plainly that "there are no additional requirements to appear in AI Overviews or AI Mode": a page must be indexed and eligible for a snippet. Crawlability, indexability, canonicals, sensible titles, internal linking, and valid structured data are still the foundation every AI surface builds on. A site that fails them will not be fixed by AI-specific work. Treat the classic audit as necessary and the AI checks as the part that was missing.

A checklist that closes the gap

1. Fetch your top templates without JavaScript and confirm the H1, the first paragraph, and the JSON-LD are in the raw HTML.
2. List every AI user agent your robots.txt allows or blocks, then send a real request with each one to catch CDN or firewall blocks robots.txt does not mention.
3. Under each important heading, write a direct, self-contained answer paragraph that would still make sense quoted on its own.
4. Ship Organization schema with sameAs links to the profiles that identify you, and put a named, linked author on long-form content.
5. Track your presence in AI answers as a rate over repeated runs, per engine, never as a single position.
6. Keep running your classic audit for crawl, index, and on-page hygiene. It is the floor, not the ceiling.

How SEO for AI Agents measures this

We built the audit around those five gaps and kept the classic checks alongside them. The render-parity check compares a no-JavaScript fetch with a real browser render. The AI crawler access check reads robots.txt for each named AI agent and, for a small high-value set, sends a real request with that agent's User-Agent and records the HTTP status. The passage extractability check maps every paragraph on the page. The entity graph check reads your Organization schema and the identity links it anchors to.

For the outcome itself, the citation measurement asks each engine the same prompt several times, reports how many runs cited you out of how many succeeded, and publishes a confidence interval anyone can recompute. When results are too volatile to act on, it says so instead of printing a confident number. Every result links to the verbatim answer, the model, and the timestamp, so you can check it yourself. If you are comparing tools, our Seobility comparison sets out where each one is the better fit.

The goal is not a higher score. It is a scorecard that matches the way buyers now research: across several engines, in answers rather than lists of links, with evidence you can verify.