AI crawler readability

AI crawler readability is the share of a page's meaningful content that is visible to a crawler that fetches raw HTML and does not run JavaScript. Because GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Google-Extended, and the rest read the raw response, any content that only appears after client-side hydration is invisible to them.

We quantify readability as the ratio of raw-HTML body text to rendered-DOM body text, with an additional check that every H1 and H2 in the rendered page also appears in the raw HTML. The heading subset is a sharper signal than aggregate token count, because the headings carry the page's answer structure. A page that returns most of its content in raw HTML scores high; a single-page app whose root div is empty until JavaScript runs scores near zero.

The fix is architectural, not cosmetic. Move primary content into server-rendered HTML: Server Components in the Next.js App Router, static export, or a server-render layer for an SPA. The cheapest verification is a curl that does not run JavaScript, piped through grep for the text you expect a model to cite. If the text is not there, neither is your AI visibility.