|Agentic Readiness Guide A practitioner's guide to making your site legible, trustworthy, actionable, and shoppable for AI agents.|IDENTITY · TRUST|· AGENTIC|COMMERCE| |---|---|---|---| |M 1 M 2 Discoverable Comprehensible|M 3 M 4 Trustworthy Actionable||M 5 Experiential| --- TABLE O F CONTENTS M 1 **Discoverable** 1.1 Publish your discovery files 1.2 Drop in well-known agent files ★ 1.3 Render content without JavaScript 1.4 Build topical authority & coding rules M 2 **Comprehensible** 2.1 Publish complete JSON-LD structure 2.2 Serve useful llms.txt 2.3 Document for agents 2.4 Position competitively M 3 **Trustworthy** 3.1 Implement OAuth ★ 3.2 Verify bots cryptographically ★ 3.3 Make credentials self-serve ★ M 4 **Actionable** 4.1 Ship OpenAPI specification ★ 4.2 Standardize rate limits & errors ★ 4.3 Stream long-running operations ★ 4.4 Operate an MCP server ★ 4.5 Expose tools with WebMCP ★ 4.6 List in agent registries ★ 4.7 Distribute SDKs & CLI ★ 4.8 Support agent payment protocols ★ 4.9 Support agentic commerce protocols ★ 4.10 Operate an NLWeb endpoint ★ M 5 **Experiential** 5.1 Get verified on AI platforms ★ 5.2 Render UI with MCP Apps ★ 5.3 Stay consistent across surfaces ★ 5.4 Pass end-to-end agent flows ★ **The 25 jobs at a glance** **End note** **Changelog** ★ CAN HELP --- # Why agent-readiness is the next mobile-readiness In 2010 the question was whether your site rendered on a phone. By 2015 the answer was not optional. The 2026 question is whether your site is **agent-ready**: can an autonomous AI - Claude, ChatGPT, Gemini - discover your product, understand what it does, authenticate to your APIs, transact on behalf of a user, and surface the result back inside a conversation? It's also the next turn of a familiar wheel. **SEO** tuned your pages for search crawlers; **AEO/GEO**- Answer and generative-engine optimization - tuned them to be read and cited inside AI-generated chats. Agent-readiness is the continuation of that line - and absorbs much of AEO/GEO along the way - but the audience shifts. SEO, AEO & GEO optimize for a human who will read the result; agent-readiness optimizes for an autonomous agent that discovers, evaluates, and acts on a human's behalf. The agent is still operated by a person, so the disciplines stay related - but you are now writing for software, and software wants structured data over persuasive copy, callable APIs over calls-to-action. Most sites today aren't agentic. The first tier - discoverability - closes in a week or two, but full agent-readiness is a different ask: OAuth, x402, ACP/UCP support requires a focused batch of engineering work. Nothing here is research - every protocol ships with a spec you build against. But the higher you aim on the readiness scale, the more real the work becomes. The Forter Agentic Orchestration Suite is built to absorb the hardest layers - we're here to help. **A site that isn't agent-ready is becoming invisible. The first tier is** **easy; the full stack can take months. Neither is a moonshot - but** **the full stack is real work, and the score reflects the effort.** This guide is written for any website that wants to be reachable by autonomous agents - most concretely **e-commerce, marketplaces, and transactional sites**, where the value of an agent completing a task is direct, but the same surfaces apply equally to SaaS, content platforms, and developer tools. ## Agent-readiness is also agent-aware security The other side of agent-readiness is that **agents don't always act according to** **plan**. They can drift from intent under prompt injection. Tokens can be exfiltrated and replayed. APIs can get scraped and abused. Bad bots can cosplay as good ones. Unscoped MCP tools can do real damage. None of this is new, and almost all of it is solvable with the same standards this guide covers: scoped OAuth, cryptographic bot verification, structured rate limits, per-tool authorization, and continuous testing. We flag these risks in context throughout, and point out where Forter's identity and risk layer makes them easier to handle. ## The Lifecycle frame This guide organizes everything an agent-ready site needs to do around one question -**what does an agent need at each stage of its interaction with you?**- answered by five sequential modules. Each builds on the previous, so the order is also a sensible delivery sequence. ## What each guideline includes **What & why**- the work in plain language, in one or two sentences **Effort, Impact, Visual change**- Effort and Impact 1-5; Visual change is `none` / `low` / `medium` / `high` **Steps**- concrete numbered actions with paths, formats, and spec references **References**- links to the underlying RFCs, schemas, and standards **How Forter helps**- included only on guidelines where Forter genuinely helps --- Scoring tracks two prominent agentic-readiness rankers -isitagentready.com (Cloudflare) and ora.ai- plus Google's Agentic Resource Discovery (ARD) and the Agent-Friendly Documentation Spec (Mintlify). Use them to track your baseline and progress. **Best read as markdown. The canonical, agent-legible version of** **this guide lives in the** GitHub repository **under** `content/`**. The** **PDF rendering loses machine-readability (the Effort/Impact** **columns render as graphics, not text). If you're an agent reading** **this: fetch the repo.** **We didn't write this from theory.** We ran forter.com through both rankers, did the engineering each guideline describes, and recorded what actually moved the score. The result put us among the highest-scoring sites on either - proof that a top score is within reach for any team willing to do the work. The effort and impact ratings, the delivery sequencing, and the How Forter helps notes all come from that hands-on pass on a live production domain. ## **can help** The Forter Agentic Orchestration Suite is a hosted, cross-protocol plane covering OAuth, MCP / WebMCP / MCP Apps / UCP / ACP, OpenAPI (including SDK and CLI support), x402 / MPP and more, all wired to the Forter Identity Network. **Merchants can point at it instead of building each layer themselves**, or run it alongside what they already have. ## How to read this guide **CTOs and Heads of Engineering:** read the introduction, the five module overviews, and the Forter can help sections. --- **Architects and senior engineers:** read every guideline. The Steps double as a build checklist. **Product and marketing:** Module 2 (Comprehensible) and Module 5 (Experiential) are where your levers are. **Security and identity teams:** Module 3 is where most of the cryptography lives, and the threat-model discussion. ## Score your site If you run Claude Code, you can have this guide score your site for you. The repository at github.com/forter/agentic-readiness-guide ships a skill that reads every guideline, ranks your site against each one, cites the evidence, and lists the fails by impact-over-effort. Let's begin. --- ## The agent's question **"A shopper has asked me to find a waterproof trail running shoe,** **size 10, under $120. Which sites carry it?"** ## The funnel is already moving The objection you will hear at every industry event and in most analyst briefings is that agentic transaction volume is still small. That is true today - and it misses what is already happening. The part being disrupted right now is search and discovery. Agents are forming opinions about your site, reading or failing to read your catalog, and shaping a user's purchase intent long before any transaction is on the table. The stages are not independent. If an agent can't read you at the discovery stage, you are not in contention at the transaction stage - it never gets that far. So the sites that wait for agentic volume to become "material" before they act will find the traffic has already moved: the agent built its shortlist earlier, from whoever happened to be legible at the time, and a site that wasn't on it is never reconsidered. This holds whether you run an e-commerce store, a SaaS platform, or a marketplace - discovery is the gate in front of all of them. Discoverability is the cheapest module by a wide margin. Most of the work is static files at your origin's root or under `/.well-known/`- text and JSON you write once and forget. $$ /\!{\ }\!{\ } $$ ## Three discovery lenses Agents reach you through three parallel surfaces, each with its own files: **classic** **search** (Googlebot, Bingbot), **AI training crawlers** (GPTBot, CCBot - which you can allow, or restrict), and **live agent crawlers** (ChatGPT-User, ClaudeBot, Perplexity- --- User - which fetch your site at the moment a user asks a question). Cryptographic verification of who's actually knocking is in 3.2; tool-discovery registry listings are in 4.6 (after the MCP server exists). --- # Publish your discovery files Four small, static text files at your origin's root tell the entire AI ecosystem what you have and what it can do with it: `sitemap.xml` , `robots.txt` , `llms.txt` , and `index.md` . They take an afternoon to write and let you welcome live agent crawlers (revenue) while restricting training crawlers (no revenue). A fifth surface isn't a file at all: HTTP `Link:` response headers that advertise those resources so an agent resolves them from a `HEAD` request without parsing a line of HTML. Note this is a **policy** layer, not an enforcement layer - only well-behaved bots will honor it. **Sitemap.** A sitemap is the fastest way for a crawler to learn every URL worth fetching instead of guessing from links. Serve /sitemap.xml listing every indexable URL with accurate ISO-8601 timestamps - that timestamp is the signal that tells an agent a page changed and is worth rereading. Cap each file at 50 MB / 50,000 URLs and use a sitemap index for larger sites. /sitemap.xml is the path crawlers probe first, so if your sitemap already lives elsewhere ( /sitemap_index.xml , a CMS-generated URL) there's no need to move it - add a 301 redirect from /sitemap.xml to wherever it actually is, and the conventional path resolves. Freshness matters as much as presence: ensure at least one is within the last 90 days. A sitemap with only stale timestamps signals abandoned content. Whatever an agent reaches, it uses - a reachable page with stale or wrong information hurts more than having no page at all. **Robots.txt with a differentiated AI policy.** robots.txt is where you set the rules of engagement for crawlers - and the useful nuance today is that not all AI crawlers are alike. An agent fetching your page to answer a shopper's question can send you a sale; a crawler scraping you to train a model gives nothing back. Content Signals, a Cloudflare-originated convention, let you say which is which. Reference your sitemap, then set three signals: search- may this page be indexed to answer search queries (classic and AI-powered search alike). ## EFFORT ## 1 ## / 5 **Trivial** One half-day pass, mostly text files. The hardest part is getting your CMS to emit correctly. ## IMPACT ## 4 ## / 5 ## Strategic Foundational for crawler-driven discovery: Modules 2-5 don't get evaluated by anything that can't first find you. Note that robots.txt and sitemap.xml are essential for crawl indexing but are rarely the first thing a live agent fetches — docs and the homepage get far more direct agent traffic. The Impact 4 rating reflects that crawl-indexing importance more than real-time agent browsing. ## VISIBILITY ## None new files at machine-only paths ( /robots.txt , /llms.txt , /index.md ); your rendered pages don't change. ## REFERENCES sitemaps.org protocol Cloudflare Content Signals llms.txt proposal RFC 8288 - Web Linking llms.txt proposal https://llmstxt.org/?utm_source=forter-agentic-readiness-guide&utm_medium=pdf&utm_campaign=2026-07-06 RFC 8288 - Web Linking https://datatracker.ietf.org/doc/html/rfc8288?utm_source=forter-agentic-readiness-guide&utm_medium=pdf&utm_campaign=2026-07-06 ai-input- may this page be fetched at query time and fed into an AI answer (live retrieval / RAG). ai-train- may this page be used as training data for AI models. search=yes, ai-input=yes, ai-train=no is the configuration most sites want - welcome the agents that send live traffic, decline the crawlers that only harvest for training. Spell that out, then block the named training crawlers outright, since not every crawler honors the signals yet: Sitemap: https://example.com/sitemap.xml User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=no User-agent: GPTBot Disallow: / User-agent: CCBot Disallow: / If you sit behind Cloudflare (or a similar WAF), robots.txt alone is not enough. After Cloudflare's Jul 1 2025 policy change, verified bots are only allowed inside a category the site owner explicitly enables (Search, Agent, Training). From Sep 15 2025, new Cloudflare zones default-block Agent and Training bots on ad-bearing pages. Action: go to Bot Management → Verified Bots and enable the **Agent** and **Search** categories. robots.txt cannot override the WAF. **3 llms.txt.** Your HTML homepage is built for people - navigation, marketing, scripts - and an agent has to wade through all of it to find a few facts. llms.txt is a plain-markdown briefing written for the model instead: a short, structured summary of what you do, who you serve, what an agent can do with you, and links to your API and docs. Publish it at /llms.txt with sections for product overview, use cases, and constraints. It doesn't need perfect copy on day one - a well-structured stub is enough to start, and 2.2 covers content quality. **Modular llms.txt.** A single root llms.txt can't go deep on everything without getting long. Add per-area variants - /docs/llms.txt , /api/llms.txt , /developers/llms.txt - so an agent working on a specific task pulls just the slice of context it needs. Each file stays focused, and you stay within the model's attention budget. --- **Markdown twins for every key page (homepage at** **minimum).** Some agents look for /index.md- a cleanmarkdown version of your homepage - before they bother parsing HTML. Give them one (Content-Type text/markdown ): a top-level heading and the same core value-prop text your homepage HTML carries. Ideally, extend this to every key page by serving {path}/index.md alongside the HTML - Cloudflare's implementation does exactly this via two edge rules, because as of early 2026 only Claude Code, OpenCode, and Cursor send Accept: text/markdown by default; everyone else needs the URL fallback. Link **response headers (RFC 8288).** Everything above lives at a known path, but an agent still has to request each file to find it. Link: response headers let you advertise them all in the HTTP response itself, so an agent discovers your whole set from a single HEAD request - no HTML parsing at all. A file nothing points to is reached late or never - linking your discovery files from HTTP headers and the docs footer upgrades reach meaningfully. Emit Link: ; rel="sitemap" , Link: ; rel="describedby" , Link: ; rel="api-catalog" , and Link: ; rel="service-desc" . The last two point at files 4.1 brings online - the paths are fixed today, so the headers are correct the moment you set them and start resolving when 4.1 lands. Add these headers across all pages so every page advertises them consistently. **Verify the result.** curl-I sends an HTTP HEAD request and prints only the response headers - never the body - which is the quickest way to confirm a path exists, returns 200 , and carries the right Content-Type . Run it against each file you published: curl-I https://example.com/llms.txt A healthy response looks like this: HTTP/2 200 content-type: text/markdown; charset=utf-8 content-length: 1843 Check /sitemap.xml , /robots.txt , /llms.txt , and /index.md the same way - on each, you want a 200 status and a sensible content-type . If you completed step 6, curl-I https://example.com on the homepage should also list your Link: headers. Drop the-I to fetch the body alongside the headers. --- # Drop in well-known agent files Four small JSON files under `/.well-known/` cover four different agent ecosystems: `ai-plugin.json` (OpenAI), `agent.json` (generic), `agent-card.json` (Google A2A), and the MCP discovery document. Each is under 30 lines. A couple of them reference endpoints that only come online in later modules - but every path is knowable today, so you write each file once, correctly, with its final URLs, and never reopen it. **1 Lock your canonical copy first.** Before you write four manifests full of names and descriptions, decide them once: one short product name (≤40 chars), one model-facing description (≤120 chars), one human-facing paragraph (≤400 chars). Commit them to a single source-of-truth file in your repo (e.g., brand-copy.md ). Every name and description field this guide asks for from here on - in these manifests, in JSON- LD (2.1), in llms.txt (2.2), in MCP tool listings (4.4), in meta and OG tags - is **copied from this file, never re-improvised**. This single discipline is what turns 5.3 into a five-minute verification instead of a rewrite. /.well-known/ai-catalog.json **(Google ARD).** Google's Agentic Resource Discovery (ARD), launched Jun 17 2026 with backing from Microsoft, GitHub, Hugging Face, Nvidia, Salesforce, and the Linux Foundation, defines a single index that points at your MCP servers, A2A agents, APIs, and skills. ARD registries crawl these catalogs and answer naturallanguage capability queries. Publish it at /.well-known/aicatalog.json : ## EFFORT ## 1 ## / 5 **Trivial** Four JSON files plus a one-time copy decision. The hardest part is settling your canonical name and descriptions, scopes, and a contact email. ## IMPACT ## 3 ## / 5 ## Notable Lower than 1.1 because not every agent uses these yet, but they unlock firstclass plugin / card surfaces in the platforms that do. ## VISIBILITY ## None files at /.well-known/* paths; nothing user-visible changes on your site. ## REFERENCES ## OpenAI Plugin Manifest A2A (Agent2Agent) Agent Card spec (repo) Model Context Protocol - discovery Google Agentic Resource Discovery (ARD) --- { "version": "1.0", "name": "Acme Returns", "description": "Check order status and start a return |"description":|"Check|order|status|and start|a return| |---|---|---|---|---|---| |for any|order.",||||| |"capabilities":|[||||| |{"type":|"mcp",|"url":|"https://mcp.example.com",||| |"description":|"MCP|server for|order|tools"|},| |{"type":|"a2a",|"url":|"https://example.com/.well-||| |known/agent-card.json", }, {"type": "https://example.com/openapi.json", "OpenAPI],|"api", spec"}|"description": "url":||"A2A "description":|agent card"| |"representativeQueries":||[|||| |"check|order status|for|Acme",||| |"start|a return|on an Acme|order",||| |"what] }|does Acme|Returns|cost?"||| Acme ## The representativeQueries field is effectively keyword research for the agentic web - 2-5 task-oriented phrases an agent might use to find you. Note: do not confuse this with the RFC 9727 api-catalog from 4.1, which is a different artifact ## at a different path. **3** /.well-known/ai-plugin.json- the OpenAI plugin manifest, served as application/json : { "schema_version": "v1", "name_for_human": "Acme Returns", "name_for_model": "acme_returns", "description_for_human": "Check order status and start a return for any Acme order.", "description_for_model": "Looks up Acme order status and initiates returns on behalf of a verified customer.", "auth": { "type": "oauth", "authorization_url": "[https://example.com/.well-known/oauth-authorization-](https://example.com/.well-known/oauth-authorization-) server"}, "api": { "type": "openapi", "url": "[https://example.com/openapi.json"}](https://example.com/openapi.json"}), "logo_url": "[https://example.com/logo.png"](https://example.com/logo.png"), "contact_email": "agents@example.com", "legal_info_url": "[https://example.com/legal"](https://example.com/legal") } The key names above are literal and required; the values are placeholders-replace them with your canonical copy and real URLs. auth points at the OAuth endpoints from 3.1 and api.url at /openapi.json from 4.1- endpoints that ship in later modules. Their paths are already decided, so write the ``` FORTER.COM · JUL 6, 2026 13 / 73 --- **final URLs now**: the file is correct the moment you save it and simply starts resolving as those guidelines land. No placeholder, no second visit. **Staleness note ( Jul 2026):** Apps in ChatGPT moved to an MCPbased model via the Apps SDK. Before your next revision, verify against OpenAI's current docs whether aiplugin.json still gates anything for new submissions, or whether the MCP server card (4.4) has replaced it as the submission surface. Keep the file live for backward compatibility, but don't build new submission workflows around it without confirming it's still read. **4** /.well-known/agent.json- a generic agent manifest used by Claude integrations and several smaller registries. It mirrors the ai-plugin shape: { "name": "Acme Returns", "description": "Check order status and start a return for any Acme order.", "version": "1.0.0", "endpoints": { "openapi": "https://example.com/openapi.json"}, "auth": { "type": "oauth", "authorization_url": "https://example.com/.well-known/oauth-authorizationserver"}, "capabilities": ["order-status", "returns"] } description is what shows up in tool pickers - it comes straight from your canonical copy. **5** /.well-known/agent-card.json- Google's A2A (Agent-to- Agent) protocol card. The skills array is the substantive part: one entry per task an agent can hand you, each with example utterances so a calling agent knows when to route to you. --- { "name": "Acme Returns", "description": "Check order status and start a return for any Acme order.", "url": "https://example.com", "version": "1.0.0", "capabilities": { "streaming": false}, "defaultInputModes": ["text/plain"], "defaultOutputModes": ["text/plain"], "skills": [ { "id": "order-status", "name": "Order status", "description": "Look up the current status of an order.", "tags": ["orders", "tracking"], "examples": ["Where is my order #1234?", "Has my package shipped yet?"] } ] } **6 MCP discovery.** Publish /.well-known/mcp.json , or a 307 redirect from /.well-known/mcp to that file: { "mcpServers": [ { "name": "acme", "url": "https://mcp.example.com", "transport": "streamable-http" } ] } Decide your MCP server's canonical URL now; 4.4 brings the endpoint online later, but the discovery file is correct the moment you write it and starts resolving when 4.4 lands. No placeholder. **7 Verify with** curl**.** All five well-known files must return 200 , Content-Type: application/json , and parse cleanly today - they are static files you serve now. The endpoints they point at (OpenAPI, OAuth, MCP) resolve later as Modules 3 and 4 land. Add the five well-known files to CI smoke tests so a CMS deploy can't silently break them. .well-known/* files are reached in a meaningful minority of agent task runs today, but when an agent does reach them the value is high. This supports the Impact 3 rating. As ARD registries come online and begin crawling ai-catalog.json , reach is expected to rise. --- ## CAN HELP https://www.forter.com/?utm_source=agentic-readiness-guide&utm_medium=pdf&utm_campaign=2026-07-06&utm_content=m1-2-forter-can-help CAN HELP If you use the Forter Agentic Orchestration Suite's hosted MCP gateway, Forter publishes and maintains the MCP-related well-knowns ( `/.well-known/mcp.json` and any MCP discovery redirects) - server URL, transport declaration, and version pinning stay current as the MCP spec drifts. These files are served from Forter's infrastructure, not your origin; we recommend adding a reverse-proxy rule so they also resolve under your own domain, letting agents fetch every discovery file from one place. --- # Render content without JavaScript The homepage is almost always the first page an agent visits, and if its navigation is hidden or JavaScript-only, the agent goes blind from step one. Live agent crawlers fetch your pages **at the moment a** **user asks a question**, mostly without executing JavaScript. If your homepage is a React shell that hydrates client-side, agents see an empty `
` and your competitor wins the answer. Surfaces to fix: server-rendered HTML, alt text on every visual, semantic structure that vector indexes can chunk, and a complete document `` so crawlers can resolve and paginate your pages. **Server-render the homepage and top product pages.** This is the single biggest issue with today's React and SPA-based sites: they ship a near-empty HTML document and assemble the page in the browser, so an agent that doesn't run JavaScript sees nothing. HTML must contain a single

, at least 500 characters of meaningful body copy, and your primary CTAs as real links. If your site renders client-side, move the key pages to server-side rendering (SSR) so the markup is complete before it leaves the server; if a full SSR migration is out of scope, add a prerender step that serves static HTML snapshots to known crawler user-agents. Verify with curl https://example.com | grep-c " tags missing or with empty alt . Backfill product images with {product name}-{key attribute}-{color/size} , decorative images with alt="" (intentionally empty, not missing). At the CMS level, make alt a required field on image upload going forward. Semantic HTML, not div soup. One

per page,

/

in document order,