Is your site agent-ready? What isitagentready.com checks, and how we went from 27 to 40

A site is “agent-ready” when AI agents can find it, read it cheaply, understand what it allows, and use what it offers without scraping. Cloudflare's free scanner at isitagentready.com measures exactly that. We ran linkinseconds.com through it and scored 27, Level 1 “Basic Web Presence”. Two small changes (Content Signals in robots.txt and Markdown on request) took us to 40, Level 4 “Agent-Integrated”. Then we built a real API and MCP server so the discovery checks had something true to point at.

This is the honest version: what the scanner looks at, what we fixed, what we skipped on purpose and why, and what we would tell anyone about to run the same scan.

What isitagentready.com checks

The scanner groups its checks into five areas:

• Discoverability. Can an agent find your content? robots.txt, a sitemap, HTTP Link headers, and DNS-based agent discovery (DNS-AID).
• Content. Can an agent read your pages cheaply? Mainly: do you return Markdown when asked for it?
• Bot Access Control. Do you state what AI bots may do? Rules for AI crawlers, Content Signals, and Web Bot Auth (an informational check).
• API, Auth, MCP and Skill discovery. If you offer an API or tools, can an agent find them from well-known URLs? API catalogs, OpenAPI, MCP server cards, agent skills and auth metadata.
• Commerce. Agent checkout protocols, for sites that sell things to agents. It was not checked for us.

Our first scan: 27

The basics were already there, because they are plain SEO hygiene: a robots.txt, a sitemap and an llms.txt file. What we lacked was anything aimed at agents specifically.

Fix 1: Content Signals in robots.txt

Our robots.txt let every crawler in, but it said nothing about what they may do with our pages. Content Signals fill that gap with one line:

Content-Signal: search=yes, ai-input=yes, ai-train=no

In words: index us and use our pages to answer questions and cite us, but do not train models on them. That matters more for us than for most sites, because people upload their own files here, and those are not ours to license for training. The full reasoning and the Next.js code are in Content Signals in robots.txt.

Fix 2: Markdown for agents

An agent that fetches a web page gets navigation, footers, scripts and styling, then has to dig out the text. That costs tokens and adds noise. So now, if a request sends Accept: text/markdown, any indexable page on our site answers with a Markdown version at the same URL. Browsers never send that header, so people see exactly what they saw before. The English home page answers with our llms.txt, which is already a hand-written Markdown summary of the site.

Only pages in the sitemap qualify. The dashboard, sign-in and anyone's uploaded files get a 406, never their content. How we built it is in Serve Markdown to AI agents with content negotiation.

Fix 3: no locale routing on /.well-known

This one was a bug we only spotted because of the scan. Our site has translated pages, and the language router was attaching hreflang alternates to responses for /.well-known/ probes, pointing at language versions that do not exist. Discovery files live at fixed, language-free paths, so we took /.well-known/* out of locale routing entirely.

The rescan: 40

With those three changes, the score went from 27 to 40, and the level jumped from 1 to 4, “Agent-Integrated”. Content went to 1/1 and Bot Access Control to 2/2. None of it needed a new product feature. It was one robots.txt line, a rewrite rule, a converter and a routing fix.

Then: a real API and MCP server

The biggest block of points, 0 out of 8 in API, Auth, MCP and Skill discovery, was not something we could fix with a header. Those checks look for machine-readable descriptions of an API. Publishing those files without a real API behind them would be the agent version of a fake button.

So we built the real thing first: an upload API and an MCP server that lets Claude, Cursor and VS Code publish files as links. Then we published the discovery files that describe them:

• /.well-known/api-catalog, an RFC 9727 catalog that points to the API, its docs and its status endpoint.
• /openapi.json, the full API description.
• /.well-known/mcp/server-card.json, which tells an MCP client the server URL, transport, required header and tools.
• /.well-known/agent-skills/index.json, which lists a SKILL.md file with step by step instructions for agents, plus its sha256 digest so a client can check it has not changed.
• /.well-known/ai-catalog.json, one catalog that lists the MCP server, the API and the skill, with example requests each one is good for.

All of them are generated from one module, so the URLs and tool lists cannot drift apart. We have not rescanned since, so we are not quoting a new score here. What we can say is that these checks now have real files behind them, describing a real API. The full list is in Agent discovery files.

What we skipped, and why

Not every check is worth passing. These we left alone on purpose:

• OAuth and OpenID metadata, and auth.md. Our API uses API keys, not OAuth. Publishing OAuth discovery documents would tell agents to try a sign-in flow that does not exist.
• DNS-AID. It is still a draft and needs DNSSEC. Not worth reworking our DNS for it yet.
• WebMCP. Still experimental. Our MCP server already covers the real use case.
• Web Bot Auth. This is for sites that send bot requests and want to sign them. We do not send any, so there is nothing to sign.
• Commerce protocols. Agents do not buy our plans on anyone's behalf, and we would rather not pretend otherwise.

Lessons if you are about to scan your own site

1. Do the cheap content wins first. Content Signals is one line. Markdown negotiation is a middleware rule and a converter. With the /.well-known fix, they took us from Level 1 to Level 4.
2. Read every failed check as a question, not an order. “Should we have this?” often has the answer no.
3. Build the capability, then describe it. Discovery files are only useful when they describe something real.
4. Generate discovery files from one source. Hand-written JSON in five places will drift.
5. Keep private things private. Serve Markdown only for pages you already let search engines index, and say no to training on anything users upload.

Common questions

Does an agent-ready score help SEO?

Not directly. Most of the checks are about AI agents and assistants, not search rankings. But the basics it rewards (a clean robots.txt, a sitemap, readable content) overlap with good SEO.

Do I need an MCP server to be agent-ready?

No. If your site has nothing for an agent to do, the content and bot access checks matter most. An MCP server only makes sense when you have an action worth offering. If you are unsure, what is an MCP server explains the idea.

How do AI crawlers read your site today?

See AI crawlers, Markdown and content signals for exactly what we allow and how to request Markdown, or browse the rest of the AI agents and developers guides.