Search and AI answers

Atlas prepares everything search engines and AI assistants read about your website. Your website only has to output it.

What Atlas prepares

For	What you get
Every page	A ready head: title, description, robots, canonical and language links, Open Graph and X/Twitter tags, and JSON-LD
Structured data	Organization, WebSite, WebPage, Article, Product, BreadcrumbList and FAQPage, already escaped for a script tag
sitemap.xml	Every indexable published page of the live release, with language alternates
robots.txt	Allows search engines, points at the sitemap and, if you choose, keeps AI crawlers out
llms.txt	A plain summary of your site and its pages for AI assistants

Development pages and files are always marked noindex, so a test site never shows up in search.

Set up your search profile

1. In Studio, open SEO & redirects → Search & AI.
2. Under Organization and summary, fill in Organization name, Logo, Profiles elsewhere (your social and company pages, one per line) and What the site is about (up to 500 characters).
3. Choose AI crawlers: Allow AI assistants to read the site or Keep AI crawlers out.
4. Click Save settings. The live website gets them with the next release.

The llms.txt card previews the file and lists pages without a summary. Structured data per page shows exactly what a page will output, for the current draft or the published version.

Per page, the page editor’s Page tab adds a Canonical page (a path on your site) and a Sharing image (the first image on the page by default).

Output it on your website

The pages response of the Delivery API includes head: { title, meta[], links[], jsonLd[] }. Map it to your framework’s metadata and print the JSON-LD blocks in <script type="application/ld+json"> tags. The blocks are already escaped, but parse and re-serialize them in your code rather than trusting any CMS output blindly.

head is null with headProblem: "SITE_ADDRESS_MISSING" when the environment has no website address in Atlas. Add the website address for that environment in Studio.

Serve the three files from your own domain by proxying them on your server, so the delivery token stays private:

// app/robots.txt/route.ts (do the same for sitemap.xml and llms.txt)
export async function GET() {
  const response = await fetch(
    `${process.env.ATLAS_CMS_ORIGIN}/v1/delivery/environments/${process.env.ATLAS_ENVIRONMENT_ID}/robots.txt`,
    { headers: { Authorization: `PublishedConsumer ${process.env.ATLAS_DELIVERY_TOKEN}` } },
  );
  const headers = new Headers({
    'content-type': response.headers.get('content-type') ?? 'text/plain',
  });
  const robotsTag = response.headers.get('x-robots-tag');
  if (robotsTag) headers.set('x-robots-tag', robotsTag);
  return new Response(await response.text(), { status: response.status, headers });
}

When AI crawlers are blocked, robots.txt disallows GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, anthropic-ai, PerplexityBot, Google-Extended, Applebot-Extended, CCBot and meta-externalagent.

Answer-engine checks

The SEO audit now also checks what helps AI assistants quote your pages: one H1 per page, heading order, FAQ questions with answers, answers whose first sentence is short (40 words or fewer), image alt text, canonical links, a sharing image and an llms.txt summary. These are suggestions, never blockers.

Limits

• Canonical links must point to a path on your own site.
• llms.txt lists up to 2,000 pages.
• There is no Google Search Console connection yet.

PerformanceDelivery API