What a bot reads when it visits your website
ChatGPT, Claude and Perplexity don't look at your site the way a person does: they ask for specific files and read the reply. Type your domain and we'll make the same requests they do.
The requests we'll make
GEThttps://example.com/
the homepage, as served
GEThttps://example.com/
Accept: text/markdown
the homepage, asked for in Markdown
GEThttps://example.com/robots.txt
the rules for robots
GEThttps://example.com/sitemap.xml
the page map
GEThttps://example.com/.well-known/api-catalog
Accept: application/linkset+json, application/json
the API catalogue
GEThttps://example.com/.well-known/mcp/server-card.json
If missing: /.well-known/mcp/server-cards.json · /.well-known/mcp.json
the MCP server card
GEThttps://example.com/.well-known/agent-skills/index.json
If missing: /.well-known/skills/index.json
the index of agent skills
GEThttps://example.com/.well-known/openid-configuration
If missing: /.well-known/oauth-authorization-server
how an agent signs in
8 read-only requests to public addresses, plus up to 4 alternatives when one is missing. We change nothing.
Prefer to ask directly? Chat with us on WhatsApp
What it looks at, in four parts
Each request answers a simple question. You don't need to code to understand the result; fixing it sometimes takes a developer.
- Discoverability
- Can a bot find every page and know where to start?
- GET /GET /sitemap.xml
- Content
- If an agent asks for clean text in Markdown, does it get it, or does it have to decode the HTML?
- GET /Accept: text/markdown
- Access control
- Have you told AI bots what they may read, and what they may use it for?
- GET /robots.txt
- API, MCP and skills
- Can an agent do things on your site (look up, book) and not just read it? Only needed if you offer those services.
- GET /.well-known/api-catalogAccept: application/linkset+json, application/jsonGET /.well-known/mcp/server-card.jsonGET /.well-known/agent-skills/index.jsonGET /.well-known/openid-configuration
Why it matters to a small business
People ask before they search
More and more customers ask ChatGPT, Claude or Perplexity which company suits them. These assistants read websites to answer, and what they can't read well they can't cite.
The standards are new
Six of the ten checks look at standards that appeared from 2023 onwards, and some are still drafts. Knowing where you stand now avoids surprises when they settle.
Often it's little work
Many fixes are one text file or a few lines in robots.txt. The advanced ones (MCP, OAuth) only make sense if your site offers services to agents.
Based on public standards: robots.txt (RFC 9309), sitemaps.org, Web Linking (RFC 8288), Content Signals (IETF draft), API Catalog (RFC 9727), MCP, Agent Skills and OAuth/OIDC. Inspired by isitagentready.com.