A growing share of developers meet your product as a paraphrase. A coding assistant summarizes your docs inside the editor; an answer engine compares you against two competitors in a chat window; an agent reads your API reference to decide whether it can do the job. The discipline that has formed around this goes by several names — AEO (answer engine optimization), GEO (generative engine optimization) — and it arrives with a venture-funded tooling category and a large pile of ritual attached. Sorting the real work from the ritual is most of the job.

The one-line position: machine-mediated discovery is a real channel with real conversion numbers, but almost everything sold under the AEO label is either premature or free — the durable work is making your docs and API surfaces machine-legible, which is work you should want anyway.

Three layers, three different bets

Everything in this space sorts into three layers with very different risk profiles. Conflating them is what makes the category feel like snake oil.

  • The reading layer — real, cheap, do it now. Models already mediate first impressions. The work is unglamorous: serve docs as static HTML rather than client-rendered JavaScript; make one section answer one question, so a chunk retrieved alone still makes sense; put real numbers — pricing, rate limits, latency — on ungated pages where a model can quote them instead of guessing; publish OpenAPI for the reference; and check that robots.txt isn’t blocking retrieval bots while you deliberate about training bots, which are a separate population and a separate decision. The accidental version is common: a September 2026 audit of the Tranco top 5,000 (one AI-operated author, dataset public, not reproduced) found 238 of the 535 sites blocking GPTBot also block OAI-SearchBot — the crawler that produces ChatGPT’s citations — while welcoming every other search engine, which reads as a catch-all rule nobody decided on. The file and the actual policy also drift apart on their own — enough that Cloudflare shipped Bot Preference Sync (August 2026) to auto-write dashboard-set search/agent/training rules into the published robots.txt; if you manage the file by hand, diff it against what you actually enforce as part of the recurring check below. Cloudflare then made the split itself a setting (2026-09-15): “Disallow AI Training” keeps a site indexable by AI search and answer engines while blocking training crawlers, backed by an “Accountable” designation for operators that run the two crawlers apart and promise that opting out of training does not touch search — Apple, Google, Microsoft, Amazon, Anthropic, Meta and OpenAI qualify at launch. It is available on every plan and off by default, so the decision the accidental block was making for you is now one checkbox you have to go and set.
  • The selection layer — early, design for it if you sell an API. Nobody can show you conversion data on agent-initiated purchases yet, but the pattern is legible: machine-readable everything, quote-then-execute pricing so an agent can’t be surprise-billed, idempotency keys so a retry can’t double-buy, scoped tokens so an agent can search without holding purchase permission, and consent records of what was agreed and when. GoDaddy’s developer platform (July 2026) is the most copyable artifact the thread has produced. Every element doubles as good API design for humans, which is why it’s worth adopting before the data arrives. The flow itself, and the directories and marketplaces an agent finds you through, are agent rails’ subject; this section stays on the reader.
  • The measurement layer — forming, mostly don’t buy yet. Enterprises will pay to know how AI describes them; Profound raised $96M at a $1B valuation in February 2026 with 700+ enterprise customers, then $180M at $1.8B seven months later (2026-09-15), citing more than 1,000 enterprise clients and revenue tripled in six months — the company’s own figures. At Fortune-500 scale, across hundreds of SKUs and ten AI surfaces, a dashboard may earn its keep. For a typical devtool company the math is different, and the free check gets you most of the signal.

What the numbers actually support

The honest read requires holding two sets of evidence at once.

The channel is real. Vercel reported ChatGPT referring 10% of new signups in April 2025, up from under 1% six months earlier — first-party attribution on its own funnel. The mechanism was plumbing, not magic: docs rendered as static HTML and structured so a model could retrieve a clean answer to a task-shaped question. The conversion premium shows up in every dataset that measures it, though never at the same number — Adobe’s retail read (reported via eMarketer) had AI-referred traffic converting 42% better than non-AI traffic by March 2026, a full reversal from a year earlier when it converted worse, and Visibility Labs’ analysis of 94 ecommerce sites had ChatGPT traffic converting 31% above non-branded organic in 2025 (1.81% vs 1.39%) — so treat any single rate as that vendor’s panel, not the channel’s truth. The direction, though, is consistent: a visitor arriving from an AI answer arrives pre-qualified — the model already matched your tool to their task and answered their first three objections.

The traffic exchange is lopsided, and that’s fine for you. Cloudflare’s crawl-to-refer ratios (July 2025) ran to roughly 38,000:1 for Anthropic’s crawlers, ~1,100:1 for OpenAI and ~195:1 for Perplexity, with around 80% of AI crawler activity being model training rather than answering a live question. Publishers who sell pageviews have a real grievance here. You don’t sell pageviews: if a model ingests your docs ten thousand times to produce one well-timed “use X for this, here’s the code” in a developer’s editor, you got the good end of that trade.

The ranking metaphor doesn’t hold. Citations churn — by the measure of the vendor selling the fix, up to 90% of sources cited in AI answers shift over time, and different models draw on largely distinct source sets. The split runs deeper than model families: Profound’s August 2026 observational study (24,135 responses; vendor-collected) found Claude and Claude Code behave as two distinct answer engines — Claude Code invoked web search in only 13% of responses against over 93% for Claude, and when it visited a site it went to documentation and pricing pages nearly three-quarters of the time, where Claude’s visits skewed to robots.txt, sitemaps and homepages. The reader inside the editor mostly isn’t searching; it is fetching your pages directly. There is no stable position to hold, which is precisely why per-model rank tracking is the weakest thing to buy first. Nor is a citation share one number: Profound’s September 2026 persona study (71,147 responses across ChatGPT, Claude and Gemini; vendor-collected) found that identical questions asked by different personas shared about one in four brand mentions and one in five cited domains, and higher-income personas drew more of both — so a tool reporting one score per query is averaging over people, and the free check below should be run as the buyer you actually sell to. The evidence layer itself is no steadier than the rankings: when Google rolled Gemini 3.8 Flash into AI Mode in September 2026, citations and links simply stopped appearing for top-of-funnel queries until Google confirmed the bug and promised a fix — a platform-wide outage of the very signal presence dashboards measure, caused by a routine model swap.

llms.txt: ship it, expect nothing

This one is settled enough to state plainly, because the two authorities finally agree from opposite directions.

Ahrefs checked all 137,210 domains in its Web Analytics that received traffic in May 2026. About 28% had shipped a valid llms.txt — remarkable adoption for an unofficial spec — but 97% of those files received zero requests. Of requests that did arrive, 96% were bots, and the largest single category was SEO audit tools (21.7%), the industry checking its own homework; AI retrieval bots were 1.1%. AI bots made essentially no requests for files that don’t exist, meaning they aren’t probing for it.

Google then made the split official. Search Central’s AI-features guidance states that sites don’t need machine-readable files or Markdown mirrors to appear in Google Search including its AI features, “as Google Search itself doesn’t use them” — so the file is not a ranking or visibility input. Meanwhile Chrome’s Lighthouse added an Agentic Browsing audit category that checks for llms.txt and WebMCP as agent-readiness signals — explicitly experimental, not on by default: Chrome’s own docs say testing it requires Chrome 150 or later (the WebMCP audits additionally need an origin trial), and it is deliberately unscored because the standards are still emerging.

That is retrieval versus ranking, made explicit by the one company that owns both a search index and a browser: nothing for the index, something still unmeasured for browser-borne agents. So ship it — an afternoon’s cheap insurance, and its absence reads badly to the audit tools your prospects run — but expect nothing from it, and never let a checkbox file substitute for the docs work underneath. The file is the index; the content is the product.

And audit it like code, because an agent executes what it reads. In August 2026 researcher Alon Hertz scanned 6,214 corporate domains and found more than 237 install commands in their llms.txt files pointing at package names nobody had registered — names an attacker can claim. He registered a handful with a harmless beacon and got a callback from inside a Fortune 500 in under four minutes; in Clerk’s docs, an unknown party had already registered a referenced package name and loaded it with live malware. Every install command in an llms.txt, an AGENTS.md or an agent-facing quickstart must resolve to a package you own, and the check belongs in CI, not in a quarterly review.

And hold the evidence for it — yours or a vendor’s — to a referral standard. In August 2026 SEO Mark Williams-Cook published cats.txt, a satirical standard for declaring office cats, and collected every proof point the industry cites for llms.txt: AI bots crawled it, Google indexed it, ChatGPT endorsed it and an AI Overview cited its fictional data — then ChatGPT reversed itself once public discussion called it a joke, showing the endorsement was echoed consensus, not evaluation. Crawled, indexed and cited are observations that occur whether or not a tactic works. The only proof that survives is an attributed number from your own funnel, which is what the measurement section below is for.

Measuring it without buying anything

The free check gets most of the signal and costs twenty minutes a month:

  • Open three assistants and ask the task-shaped questions your buyers actually ask — “how do I add auth to a Next.js app”, not “best auth provider”. Record whether you appear, what’s said about your pricing and limits, and whether it is true. Accuracy matters more than presence: a confident wrong answer about your free tier costs you more than absence does. And when an answer is wrong, trace it to the page it cites before assuming the model hallucinated: Profound’s August 2026 analysis of 158,000+ brand claims found the most common source of an inaccurate claim was the brand’s own site — 54% of brands had a wrong claim citing their own content, ahead of earned media — with pricing the concentration point (12% of evaluated claims, 24% of the inaccuracies). The fix is usually deleting or correcting a stale page you forgot you owned, not filing a bug against the model.
  • Add an AI-referral segment in analytics and a free-text “how did you hear about us” field. Vercel’s 10% figure came from exactly that kind of first-party attribution, not from a platform — and randomize any survey’s answer order: Val Town found whichever option listed first got picked more, before it could trust its own July 2026 reading that most known-source new Pro signups arrived via AI.
  • Turn on Search Console’s generative AI performance reports, rolled out to every site worldwide as of 2026-08-31 after a phased start in June: impressions from AI Overviews, AI Mode and AI-powered Discover by page, country and device, with a matching opt-out that removes a site from those surfaces without touching standard ranking. It reports impressions, not clicks, so treat it as a free presence meter — weather, not attribution.
  • Watch your own logs for retrieval bots. Which crawlers fetch which pages is a fact you own, and it tells you more than a rank tracker. The population is unstable enough that this check has to recur: Meta’s own indexer went from roughly 2% to 37.8% of observed AI-crawler requests in under a month (Promptwatch, August 2026) — the single heaviest AI crawler in those logs, arriving almost from nowhere — so a robots policy or allowlist tuned to last year’s bot names quietly goes stale.
  • Buy tooling when the free check stops scaling — hundreds of pages, many surfaces, someone accountable for the number — not before.

What earns citations, and what repels

The work that gets you quoted is mostly the work that serves human readers, which is the tell that it’s durable:

  • Specific, quotable facts. Concrete numbers, named limits and dated benchmarks are what a model can lift into an answer. Marketing adjectives are unquotable by construction.
  • One question per section. Retrieval returns chunks, not pages. A section that only makes sense after reading the three above it is invisible to the reader that matters.
  • Verifiable claims over positioning language. When the evaluator is a machine comparing three tools, published, checkable numbers beat superlatives.
  • Loud deprecation. Machine readers make this a survival trait: an agent that receives a clean, versioned deprecation notice reroutes; an agent that hits a silent 404 tells its developer your product is broken.
  • Gating and client-side rendering repel. Content behind a form or assembled by JavaScript is content the model cannot cite. This is the oldest lesson in the guide arriving through a new door.

The honest caveat

This section describes a channel that is moving faster than the evidence about it. The lab result everyone quotes — visibility gains “up to 40%” from adding quotable statistics and citations — comes from an academic benchmark (Aggarwal et al., KDD 2024), not from the field, and the paper itself warns the effect varies widely by domain. Treat any vendor number about your own visibility as a hypothesis you test against your funnel. The reading-layer work is the part that pays regardless of how the rest of it resolves, because it is indistinguishable from doing your docs properly.