AI Visibility pillar / check vertical-benchmarks

Vertical benchmarks: how you compare, without exposing anyone

We publish anonymized cohort benchmarks for each vertical, for example "the median brand in this category is cited by 2 of 6 engines," computed only from organizations that opt in, and only when a cohort is large enough that no single organization can ever be identified.

By Shimon Carroll, Founder, SEO for AI Agents · Last updated

What this check measures

For each vertical, we compute cohort-level benchmarks that describe how AI citation is distributed across the organizations in that category: for example, the median number of engines that cite a brand, and the spread between the strongest and weakest. These benchmarks are summary statistics over a group, never a list of organizations and never a per-organization value. They are computed only from organizations that have explicitly opted in to contribute to benchmarks, and only at the cohort level. When you view a benchmark, you see the anonymized cohort statistics alongside your own result, computed for you at request time, so you can place yourself against the category without ever seeing, or being seen by, any other specific organization.

Why it matters

A score in isolation does not tell you whether it is good. Being cited by 2 of 6 engines could be category-leading or merely average depending entirely on the vertical, and the only way to know is to compare against your peers. Cohort benchmarks give you that context honestly: where you sit relative to the median and the spread of organizations actually competing for the same AI attention. The catch is that this comparison is only worth having if it can be provided without compromising anyone, including you. That is why the benchmark is built to be useful and private at the same time: you get the context of the cohort, and no organization, yours or anyone else's, is ever individually exposed to get it.

How we score it

Benchmarks are computed from the opted-in cohort as group-level summary statistics: typically a median and a sense of the spread across the cohort, plus the size of the cohort behind the figure. We never store or serve a per-organization value in a benchmark, and we never attach an identity to a contributed measurement; the rollup reads the contributing measurements through a privileged process and keeps only the anonymized summary. Your position relative to the cohort is computed for you, at the moment you request it, by comparing your own result against the cohort statistics, so you see your own value and the anonymous cohort summary, and never another organization's value. The specifics of how the cohort is composed and how contributed measurements are reconciled before summarizing are our internal design and are not published, because that design is what keeps the benchmark both representative and private.

Confidence-flag rules

Privacy is enforced by a hard floor, not by good intentions. We never compute or serve a benchmark cell for a cohort smaller than a minimum number of distinct organizations, currently five; below that floor the cell is suppressed entirely rather than shown, because a small cohort could let a single contributor be inferred. That floor is rechecked every time benchmarks are recomputed: if a cohort drops below it for a given window, the cell is re-suppressed rather than shown stale. Only opted-in organizations are ever included, and opting out removes future contribution. Every benchmark carries the size of the cohort behind it, so you can see how much confidence a figure deserves; a benchmark sitting just above the floor is shown honestly as a thin cohort rather than dressed up as a settled category norm. No benchmark ever contains, or can be reversed into, a per-organization value.

Common mistakes

  • Reading a raw score without a peer comparison. The same citation rate can be category-leading in one vertical and below average in another; context is what makes the number actionable.
  • Assuming a benchmark exposes other organizations. Benchmarks are group-level summaries only; no per-organization value is ever stored or shown.
  • Trusting a thin cohort as a settled norm. A benchmark just above the minimum-cohort floor is a weak signal, and we publish the cohort size precisely so you can judge that.
  • Expecting your data to be included without opting in. Only organizations that explicitly opt in contribute, and opting out stops future contribution.

How to fix it

A benchmark is context, not a defect, so the response is strategic rather than a fix. If you sit below the cohort median, the gap quantifies how much room there is and the rest of your report names the levers; if you sit above it, the benchmark tells you the lead is real and worth defending. Re-check your position over time as the cohort grows, since the median itself moves as more of your category gets measured. If you want your own results to help build a more representative benchmark for your vertical, you can opt in, knowing that your contribution is only ever used as part of an anonymized cohort above the minimum-cohort floor, never as an identifiable value. The benchmark you see is always paired with your own result and the cohort size behind it, so the comparison is transparent about exactly how much it is built on.

Primary sources

Changelog

  • · Initial publication. Documents anonymized vertical benchmarks: opt-in only, computed from a cohort, served only above a minimum-cohort floor so no single org is identifiable, and aggregate-only with no per-org value ever stored or shown.
← Back to the methodology indexRun an audit