llms.txt — Complete Guide
llms.txt is a plain-text file placed at the root of your domain that gives AI systems a direct, structured brief about your product. This is the complete technical reference.
Contents
What is llms.txt?
llms.txt is a plain text file located at /llms.txt on your domain (e.g., https://yourdomain.com/llms.txt). It provides AI language models with a structured, human-readable summary of your website and product.
The standard was proposed by Jeremy Howard (co-founder of fast.ai) in September 2024. It uses a simplified Markdown-like format optimized for LLM consumption — headings, short paragraphs, and bullet lists. The goal is to give AI systems exactly the information they need to accurately describe, cite, and recommend your product.
Why it exists
When AI systems like ChatGPT, Claude, and Perplexity browse your website to learn about your product, they face a problem: most websites are built for humans, not language models. Pages have navigation menus, JavaScript-rendered content, modal popups, cookie banners, and varying structures that make it hard for LLMs to quickly extract the core product information they need.
As a result, AI systems often get product details wrong — misattributing features, confusing your product with competitors, citing outdated pricing, or describing use cases you no longer support.
llms.txt solves this by giving AI systems a direct, unambiguous brief. Instead of inferring your product details from a complex marketing page, AI crawlers read your llms.txt and get the information you want them to have, formatted the way they work best.
How AI crawlers use llms.txt
When an AI crawler visits your domain — whether as part of a scheduled index crawl or in response to a real-time user query — it typically checks for llms.txt early in the crawl sequence, before or alongside fetching your homepage.
The content of your llms.txt is treated as a high-authority, first-party declaration about your product. AI systems use it to:
- Generate accurate product descriptions in answer responses
- Identify the correct category and use case for your product
- Find the right URL to link to when recommending your product
- Understand pricing, target audience, and key differentiators
- Discover additional documentation via the Optional Links section
llms.txt vs. robots.txt vs. sitemap.xml
What to include
A well-written llms.txt has these sections, in order:
H1 — Product nameRequiredThe exact name of your product or company. Used by AI systems as the primary identifier.
# YourProduct
Blockquote — One-line descriptionRequiredA single sentence describing what your product does and who it's for. This is the most important line — it's what AI systems quote when summarizing your product.
> The AI visibility audit for founders and developers. Helps your brand appear in ChatGPT, Perplexity, Claude & Gemini answers.
## OverviewRequired2-4 sentences expanding on the description. What it does, how it works at a high level, and what makes it different.
## Target AudienceRecommendedWho the product is for — job roles, company sizes, use contexts. Helps AI match your product to relevant queries.
## Key FeaturesRecommendedBulleted list of major capabilities. Keep each item to one line. AI systems use this to answer 'what does X do?' queries.
## PricingRecommendedPricing tiers with key inclusions. Helps AI answer 'how much does X cost?' queries accurately.
## Optional LinksOptionalMarkdown links to key pages: homepage, docs, pricing, blog. AI crawlers follow these to discover more of your content.
- [Homepage](https://yourdomain.com) - [Pricing](https://yourdomain.com/pricing) - [Documentation](https://yourdomain.com/docs)
Example llms.txt
This is the llms.txt for AIExposureTool — a real-world example you can use as a template.
# AIExposureTool > AI visibility audit for founders and developers. AIExposureTool helps your brand appear in ChatGPT, Google AI, Perplexity, Claude & Gemini answers. ## Product Overview AIExposureTool is a web-based platform that measures how visible and recommendable your product is to AI assistants like ChatGPT, Claude, Perplexity, and Gemini. It produces an AI Exposure Score (0-100) across 25+ signals and generates the files needed to improve that score: llms.txt, llms-full.txt, JSON-LD structured data, and copy-paste fix prompts. It also includes a dedicated security scanner that runs 19 passive checks — finding exposed API keys in JavaScript bundles, publicly accessible .env files, missing security headers, and more. ## Target Audience - Founders and SaaS teams who want ChatGPT, Claude, and Perplexity to recommend their product - Vibe-coders using Claude, Cursor, ChatGPT, or Gemini to build products - Marketing and SEO teams optimizing for AI-powered search and AEO - Agencies adding AI visibility audits to their service offering ## Key Features - AI Exposure Score (0-100) measuring 25+ signals across 6 categories - Security Grade (A-F) from 19 passive security checks - Auto-generated llms.txt, llms-full.txt, and JSON-LD structured data - Fix prompts for Claude, Cursor, ChatGPT, and Gemini - Re-scan any URL to verify fixes - JavaScript bundle scanning for API key leaks (17 patterns) ## Pricing Free: 1 scan per URL, top 3 issues, basic fix prompts — no credit card required Starter ($19/month): 25 AI audits + 25 security scans/month, full reports, all generated files Pro ($49/month): Unlimited scans, priority processing, everything in Starter ## Optional - [Homepage](https://aiexposuretool.com) - [AI Exposure Audit](https://aiexposuretool.com/audit) - [Security Scanner](https://aiexposuretool.com/security) - [Pricing](https://aiexposuretool.com/pricing) - [Methodology](https://aiexposuretool.com/methodology/ai-exposure-score) - [llms-full.txt](https://aiexposuretool.com/llms-full.txt)
llms-full.txt — the extended version
llms-full.txt is the extended companion to llms.txt. While llms.txt is a concise brief (50-200 words), llms-full.txt is a comprehensive document (typically 500-2000 words) with complete product documentation.
llms.txt — for
- Quick product summaries
- "What does X do?" queries
- Product comparison responses
llms-full.txt — for
- Deep research queries
- Technical documentation requests
- Detailed pricing/feature questions
Add a link to /llms-full.txt in your llms.txt Optional Links section so AI crawlers can discover it.
llm.json — structured variant
llm.json is a machine-readable JSON file (typically at /llm.json) that provides the same core product information in a structured format that AI systems can parse programmatically.
Key fields: name, tagline, description, category, target_audience, use_cases, features, pricing, links, contact. AIExposureTool generates llm.json alongside llms.txt.
How to generate yours
Use AIExposureTool (automated)
Paste your URL into AIExposureTool. After the audit, Starter and Pro plan users can download auto-generated llms.txt, llms-full.txt, and llm.json files created from your scanned site content. No writing required.
Write it manually
Follow the template in the "What to include" section above. The file is plain text — no special tooling needed. Save as llms.txt and serve it from your domain root.
Deploy and verify
Place the file at the root of your domain so it's accessible at https://yourdomain.com/llms.txt. Verify it returns a 200 status with Content-Type: text/plain. Re-run your AIExposureTool audit to confirm the Agent Readiness score improves.
FAQ
Is llms.txt an official standard?
llms.txt was proposed by Jeremy Howard (fast.ai) in September 2024 as an emerging standard. It is not yet an official W3C or IETF standard, but it has been widely adopted and is now recognized by major AI crawlers. The spec is documented at llmstxt.org.
Do AI crawlers actually look for llms.txt?
Yes. Claude (Anthropic), ChatGPT (OpenAI), and Perplexity crawlers actively look for llms.txt when indexing sites. Having one increases the accuracy and completeness of how these systems describe your product.
What's the difference between llms.txt and llms-full.txt?
llms.txt is a concise brief — 50-200 words — with the essential product information. llms-full.txt is a comprehensive document — typically 500-2000 words — with full feature descriptions, all pricing tiers, technical details, and complete documentation. AI systems use llms.txt for quick summaries and llms-full.txt for detailed research queries.
Does having llms.txt improve my AI Exposure Score?
Yes — llms.txt is worth up to 5 points and llms-full.txt is worth up to 5 points in the AI Exposure Score, contributing to both the AI Crawl Access and Agent Readiness categories. Together they are worth up to 10 of the 100 possible points.
What if I don't have a technical team to create llms.txt?
AIExposureTool generates llms.txt and llms-full.txt for you automatically based on your scanned site content. You can copy the generated file and paste it into your codebase, or give it to a developer with instructions to place it at /llms.txt on your domain.
Should I update my llms.txt when my product changes?
Yes — keep your llms.txt current when you add major features, change pricing, rebrand, or shift target audience. Stale llms.txt files can cause AI systems to describe your product using outdated information.
Generate your llms.txt automatically
Scan your site and get auto-generated llms.txt, llms-full.txt, and JSON-LD ready to deploy. Free to try — no signup required.