# mdream > Convert HTML to Markdown online. 33x faster than Turndown, 38% fewer tokens. Free, zero dependencies, LLM optimized. Available as npm package, Rust crate, WASM, and CLI. Canonical Origin: https://mdream.dev/ ## LLM Resources - [Full Content](https://mdream.dev/llms-full.txt): Complete page content in markdown format. --- - **Page:** Guides · GEO, AEO, llms.txt, and markdown for agents · mdream - **Source:** https://mdream.dev/guides - **Description:** Deep-dive guides on generative engine optimization, answer engine optimization, markdown for agents, the Accept-header standard, and AI crawler control. [Home](https://mdream.dev/) # **Guides ** Long-form references on AI search visibility, markdown content negotiation, and the tooling that makes your site readable to agents. [

**Generative engine optimization (GEO)**

Generative engine optimization (GEO) is the discipline of getting your content cited by ChatGPT, Claude, Perplexity, and Google AI Overviews. It blends classic SEO, structured data, and new formats like llms.txt + Accept-header markdown.generative engine optimization tools](https://mdream.dev/guides/generative-engine-optimization) [

**Answer engine optimization (AEO)**

Answer engine optimization (AEO) is the practice of structuring content so that ChatGPT, Claude, Perplexity, Google AI Overviews, and voice assistants pick it as the canonical answer.answer engine optimization tools](https://mdream.dev/guides/answer-engine-optimization) [

**Markdown for agents**

AI agents read markdown 2–5× more efficiently than HTML. Serve it via Accept-header negotiation, sitemap.md, and \ and every agent-originated request instantly becomes cheaper and smarter.markdown for agents](https://mdream.dev/guides/markdown-for-agents) [

**The accept-markdown header**

Accept: text/markdown is an HTTP content-negotiation header that signals an AI agent prefers clean markdown over HTML. Implementing it correctly means a Content-Type match, a Vary: Accept header, honoring q-values, and 406 on unsatisfiable requests.accept markdown header](https://mdream.dev/guides/accept-markdown-header) [

**AI crawlers in 2026**

Nine AI crawlers account for virtually all agent-originated traffic in 2026. Blanket-blocking them means no citations from ChatGPT, Claude, Perplexity, or Google AI Overviews. Block selectively with robots.txt and guide the rest with llms.txt.ai crawlers](https://mdream.dev/guides/ai-crawlers) --- - **Page:** AI search visibility audit · Check your site for ChatGPT, Claude & Perplexity · mdream - **Source:** https://mdream.dev/tools/ai-audit - **Description:** Free AI search visibility audit. Check your site for Accept-header markdown negotiation, llms.txt, sitemap.md, robots.txt AI bot policy, and HTML → Markdown token savings. Answer engine optimization tools and generative engine optimization tools roll up into this one report. [Home](https://mdream.dev/) # **AI search visibility audit ** Is your site ready for ChatGPT, Claude, Perplexity, and Google AI Overviews? One click, seven checks: Accept-header negotiation, llms.txt, sitemap.md, robots.txt AI rules, and HTML → Markdown token savings. The tool behind the answer to *which generative engine optimization tools actually move the needle*. zzz Ready Enter a URL to start your first audit. What gets checked - Accept: text/markdown negotiation - Vary: Accept header - \ - /llms.txt spec-compliance - /sitemap.md presence - robots.txt AI bot coverage - HTML → Markdown token savings ## How the audit works 1. **1**### **Fetch with agent fingerprints** We replay Claude Code and Cursor requests to see what your server returns. 2. **2**### **Inspect headers and surfaces** Accept negotiation, Vary, \, llms.txt, sitemap.md, robots.txt AI rules. 3. **3**### **Convert + tokenize** Run mdream HTML → markdown on your page and count the token delta. 4. **4**### **Score + grade** Weighted 0-100 score with per-check remediation links to our other tools. ## Why audit? Every generative engine optimization tool and answer engine optimization tool you've seen measures some version of these checks. We surface them directly against your live URL, flag the broken ones, and link remediation to our [~~llms.txt generator ~~](https://mdream.dev/tools/llms-txt/generator), [~~accept-header tester ~~](https://mdream.dev/tools/accept-header), and [~~token counter ~~](https://mdream.dev/tools/token-counter). No fluff, no dashboard sales motion. ## Frequently Asked Questions Seven checks across content negotiation, discovery, and efficiency: Accept: text/markdown negotiation, Vary: Accept, \, /llms.txt presence and spec validity, /sitemap.md, robots.txt AI policy coverage, and HTML → Markdown token savings. Generative engine optimization tools and answer engine optimization tools all converge on the same few signals: can agents fetch you, can they parse you efficiently, do you have structured discovery surfaces. This audit measures those signals directly against your live site rather than inferring them from keywords. Token counts use OpenAI's official cl100k\_base BPE vocabulary (exact for GPT-4 and a very close approximation for Claude). The savings percentage is real, billing-grade for those models. Audits run in-flight on Cloudflare Workers. The report is returned to your browser; we log aggregate stats for rate-limiting but not the report body. ## Related [

**GEO guide **

What actually moves the needle.](https://mdream.dev/guides/generative-engine-optimization) [

**Accept-header tester **

Replay agent requests manually.](https://mdream.dev/tools/accept-header) [

**Token counter **

Count what agents actually pay for.](https://mdream.dev/tools/token-counter) --- - **Page:** HTML to Markdown Benchmarks · mdream - **Source:** https://mdream.dev/tools/benchmarks - **Description:** Live browser benchmarks comparing HTML to Markdown converters. mdream WASM vs pure JS vs Turndown vs node-html-markdown. Run tests in your browser. [Home](https://mdream.dev/) # **Benchmarks ** Live browser benchmarks. Converters are lazy-loaded per run. Results show min real time across 1000 iterations. HTML Sample Converters mdream (wasm)mdream (js)Turndown (js)node-html-md (js) Iterations zzzSelect converters and click Run Benchmark ## Related [**Converter Comparison**·Feature matrix and static speed comparison](https://mdream.dev/tools/compare) [**Online Converter**·Convert HTML to Markdown in your browser](https://mdream.dev/tools/llms-txt/generator) --- - **Page:** Generate llms.txt Free · Online AI Sitemap Generator · mdream - **Source:** https://mdream.dev/tools/llms-txt/generator - **Description:** Generate an llms.txt file free. Enter any URL and our generator crawls your site, converts pages to Markdown, and produces a spec-compliant /llms.txt for ChatGPT, Claude, and Perplexity. No signup, works with WordPress, Next.js, and Nuxt. [llms.txt Tools](https://mdream.dev/llms-txt) # **Generate llms.txt **free Generate a [~~spec-compliant~~](https://llmstxt.org)`/llms.txt` file for any website. Create an AI sitemap so ChatGPT, Claude, and Perplexity can understand your site. Free, no signup, powered by [~~mdream ~~](https://mdream.dev/). ## **Need more? ** The generator is free for up to 100 pages. We're exploring a pro tier for teams and larger sites. - llms-full.txt generation - LLM analytics - Scheduled regeneration [Join the waitlist ](https://mdream.dev/auth/github?redirect=/tools/llms-txt/generator) ## How It Works This generator uses [~~mdream ~~](https://mdream.dev/) to crawl your site and produce a structured llms.txt that AI systems can understand. ### **Crawl** Discovers pages from your sitemap, internal links, and common paths. ### **Convert** Extracts content from each page and converts HTML to clean Markdown using mdream. ### **Organize** Groups pages into logical sections based on URL patterns and site structure. ### **Generate** Produces a spec-compliant llms.txt with proper headings, links, and descriptions. ## Why Add llms.txt to Your Site? AI tools are how a growing number of people discover and interact with your content. A well-structured llms.txt helps them get accurate answers from your site. 1. **1**### **AI search visibility** ChatGPT, Perplexity, Claude Browse, and Google AI Overviews use structured data to cite your content accurately. An llms.txt acts like an AI sitemap for your site. 2. **2**### **Developer experience** Developers using AI coding assistants get better answers when your docs have a clear llms.txt index. Essential for open-source projects and API documentation. 3. **3**### **RAG and chat-with-docs** Teams building RAG pipelines use llms.txt and llms-full.txt to populate vector databases. A well-structured file means better retrieval quality. 4. **4**### **Complement robots.txt** robots.txt controls crawl access (exclusion). llms.txt provides context and structure for the content you do want shared (inclusion). Use both together. ## Tips for a Great llms.txt `Curate` Edit the generated file. Remove low-value pages. The median file across 523+ projects is 16 KB with 83 links. A focused file under 50 KB performs best. `Describe` Add descriptions after each link. Only 54% of links in the wild include descriptions, but they dramatically improve LLM comprehension. `Section` Group links under meaningful H2 headings. The most common across 523+ files: "Docs", "Guides", "Reference", "Examples". `Optional` Add an "Optional" section for lower-priority links. Only 23% of files use this pattern, but it lets LLMs skip non-essential content when context is limited. `Validate` Run your file through the llms.txt validator to catch formatting issues. 22% of files in the wild violate the spec with H3+ headings. ## Frequently Asked Questions It crawls your website, extracts page titles and descriptions, converts HTML to clean Markdown, organizes pages into sections based on your URL structure, and produces a spec-compliant llms.txt file that AI systems like ChatGPT, Claude, and Perplexity can consume. Most sites complete in under 30 seconds. Larger sites with hundreds of pages may take a minute or two. The progress indicator shows pages found and processed in real time. Place it at the root of your domain so it's accessible at https://yourdomain.com/llms.txt. For WordPress, add it to your root directory or use a plugin. For Next.js, put it in public/ or create a route handler. For Nuxt, use @mdream/nuxt for automatic generation or place it in public/llms.txt. Absolutely. The generated file is a starting point. Review it, reorder sections, add descriptions to links, and remove pages that aren't useful for AI context. A curated llms.txt with 20 to 50 focused links performs better than a massive auto-generated list. This tool generates llms.txt (the concise index with links). For llms-full.txt (which embeds full page content inline for RAG pipelines and "chat with docs" applications), use the @mdream/crawl package or the @mdream/nuxt module. No. The crawl runs server-side, generates the llms.txt output, and discards all data after delivery. Nothing is stored or logged. robots.txt controls which pages crawlers can access (exclusion). Sitemaps list URLs for search engines to index. llms.txt provides structured context for AI systems (inclusion): which pages matter, how they relate, and what each one covers. Think of llms.txt as an "AI sitemap" or "robots.txt for AI." Use all three together for complete discoverability. Yes. AI search engines like Perplexity, ChatGPT Browse, and Google AI Overviews use structured data to cite your content. A well-structured llms.txt acts as an AI sitemap, making it easier for these tools to find, understand, and accurately reference your pages. This is part of what some call Generative Engine Optimization (GEO). For WordPress, place a static llms.txt in your site root or use a plugin that generates it from your posts and pages. For Next.js, add it to public/llms.txt for a static file, or create a route handler at app/llms.txt/route.ts that generates it dynamically from your content. This generator works with any site regardless of platform. llms.txt is an excellent starting point for RAG (Retrieval Augmented Generation) pipelines. Use llms.txt as a structured index, or use llms-full.txt (which embeds all content inline) as a retrieval source for vector databases. The section structure maps naturally to document chunks for embedding. ## References [llmstxt.org Specification The official llms.txt proposal and spec.](https://llmstxt.org) [llms.txt Directory Browse sites that have adopted llms.txt.](https://directory.llmstxt.cloud) [mdream on GitHub The HTML to Markdown engine powering this generator.](https://github.com/harlan-zw/mdream) ## Framework Guides Prefer a walkthrough for your stack? Each guide covers install options, dynamic routes, and build-time generation. [

**llms.txt for Next.js**

App Router route handler, static file, or build-time generation.](https://mdream.dev/tools/llms-txt/nextjs) [

**llms.txt for Nuxt**

@mdream/nuxt module, Nitro route, or public/llms.txt.](https://mdream.dev/tools/llms-txt/nuxt) [

**llms.txt for WordPress**

Plugin, mu-plugin rewrite, or upload to site root.](https://mdream.dev/tools/llms-txt/wordpress) [

**llms.txt for SvelteKit**

Static asset, +server.ts endpoint, or prerender from MDsveX.](https://mdream.dev/tools/llms-txt/sveltekit) [

**llms.txt for Astro**

Static file, API route, or generate from content collections.](https://mdream.dev/tools/llms-txt/astro) [

**llms.txt for Remix**

Resource route, static file, or loader-driven MDX output.](https://mdream.dev/tools/llms-txt/remix) ## Related Tools [

**Validator **

Check your file against the spec.](https://mdream.dev/tools/llms-txt/validator) [

**Insights **

See how 95+ projects structure their files.](https://mdream.dev/insights) [

**mdream **

HTML to Markdown. Fastest in JS and Rust.](https://mdream.dev/) --- - **Page:** llms.txt Insights · How 523+ Projects Structure Their AI Sitemaps · mdream - **Source:** https://mdream.dev/insights - **Description:** Data-driven analysis of 523+ real-world llms.txt files. See size distributions, section patterns, link coverage, compliance scores, and best practice adoption across the ecosystem. [llms.txt Tools](https://mdream.dev/llms-txt) # **llms.txt Insights ** Data from 523 real-world llms.txt files, analyzed for structure, size, compliance, and patterns. Last updated March 30, 2026 Projects **523** Total Links **151,268** Median Size **15.6 KB** Unique Sections **5,256** ## File Size Distribution How large are llms.txt files across the ecosystem? The smallest is 213 B (/llms.txt directory), the largest 1.8 MB (What is Datadog?). < 1 KB **18** 1–5 KB **98** 5–20 KB **187** 20–50 KB **92** 50–100 KB **52** 100–500 KB **64** 500 KB+ **12** Min **213 B** Median **15.6 KB** Max **1.8 MB** ## Best Practices Adoption How well do real-world llms.txt files follow the spec and best practices? Under 100K tokens **97%** Has H2 sections **96%** No code blocks **92%** Has H1 title **90%** No H3+ headings **78%** Uses absolute URLs **77%** Under 50 KB **76%** Has blockquote description **50%** All links described **35%** Has Optional section **23%** ## Link Description Coverage Across 151,268 links, only 54% include descriptions. Descriptions give LLMs context about each page without fetching it. **54%**described With descriptions: **81,194** Without descriptions: **70,074** Median: 83 links per file ## Most Common Section Names 5,256 unique H2 section names across all files. These are the most popular. "Docs" 177x "Installation" 136x "API Reference" 130x "Optional" 121x "Usage" 94x "OpenAPI Specs" 87x "Getting started" 83x "Next steps" 76x "Example" 47x "Troubleshooting" 46x "Install" 45x "How it works" 44x ## URL Patterns URL Style 85% absolute15% relative File Placement /llms.txt (root)**87%** /docs/llms.txt**13%** Other paths**1%** Link Target Extensions .md101,691.none57,235.txt9,719.mdx6,614.html1,688.png222.com188.json187 ## By Category How do different categories of projects approach llms.txt? llms.txt usage statistics by project category | Category | Count | Avg Size | Avg Links | Size Distribution | | --- | --- | --- | --- | --- | | Libraries | 232 | 71.6 KB | 307 | | | AI | 54 | 67.9 KB | 170 | | | Documentation Tools | 51 | 36.3 KB | 208 | | | UI Libraries | 37 | 19.5 KB | 100 | | | Frameworks | 34 | 64.7 KB | 245 | | | Databases | 30 | 50.5 KB | 294 | | | Platforms | 28 | 143.5 KB | 354 | | | Developer Tools | 18 | 37.8 KB | 160 | | | Vue Ecosystem | 10 | 16.4 KB | 117 | | | CMS | 9 | 24.4 KB | 180 | | | Build Tools & Runtimes | 8 | 34.6 KB | 318 | | | Monitoring | 5 | 431.7 KB | 3161 | | | Authentication | 3 | 214.6 KB | 1470 | | | E-commerce | 3 | 54.0 KB | 300 | | | React Ecosystem | 1 | 14.0 KB | 177 | | ## Structural Patterns Has blockquote description**50%** References llms-full.txt**10%** Has "Optional" section**23%** Uses H3+ headings (non-spec)**22%** 19 projects use a flat structure (no H2 sections): Next.js, Ark UI, Supabase, Ri Xu Online, Appwrite, Kotlin Documentation, Strapi Documentation, http://docs.copilotkit.ai llms.txt, https://docs.litellm.ai/ llms.txt, Documentation, Memvid, https://www.sardine.ai llms.txt, Alpaca Main Page, sitespeak.ai llms.txt, https://tolgee.io llms.txt, https://latte.nette.org/ llms.txt, WordLift, https://nette.org/en/ llms.txt, ScreenshotOne.com Documentation 45 projects have multiple H1 headings: Svelte, SvelteKit, Ark UI, SWR, Remotion, Turso... 44 projects include code blocks: Svelte, SvelteKit, Waku, daisyUI, Zustand, SWR... ## Token Estimates How much LLM context do these files consume? Min **54** Median **3,992** Mean **17,090** Max **471,765** 18 files exceed 100K tokens What is Datadog?: 471,765 tokens Documentation: 447,604 tokens Add a New Provider: 371,002 tokens LLMs.txt - Sitemap for AI content discovery: 307,831 tokens Tamagui Complete Documentation: 267,532 tokens Kilo Code Documentation: 254,109 tokens Snowflake Documentation: 231,479 tokens Gemini CLI Documentation: 219,925 tokens PraisonAI: 209,780 tokens SvelteKit: 137,599 tokens Vultr Documentation: 135,514 tokens Mailchimp - Complete Web Content Analysis: 131,208 tokens Modular: 125,191 tokens Svelte: 117,430 tokens LaunchDarkly | Documentation: 115,318 tokens Agno: 114,069 tokens Documentation: 111,033 tokens Auth0 blog: 100,823 tokens ## Compliance Leaderboard Top Compliance [1.Cloudflare Workers**10/10**](https://mdream.dev/llms-txt/cloudflare-workers) [2.Trigger.dev**10/10**](https://mdream.dev/llms-txt/trigger-dev) [3.reveal.js Documentation**10/10**](https://mdream.dev/llms-txt/reveal-js-documentation) [4.Javalin**10/10**](https://mdream.dev/llms-txt/javalin) [5.Milvus**10/10**](https://mdream.dev/llms-txt/milvus) [6.Magic UI**10/10**](https://mdream.dev/llms-txt/magicui) [7.SurrealDB**10/10**](https://mdream.dev/llms-txt/surrealdb) [8.Super Productivity**10/10**](https://mdream.dev/llms-txt/super-productivity) Needs Improvement [Kilo Code Documentation**1/10**](https://mdream.dev/llms-txt/kilocode) [Gemini CLI Documentation**1/10**](https://mdream.dev/llms-txt/gemini-cli) [SvelteKit**1/10**](https://mdream.dev/llms-txt/sveltekit) [Svelte**1/10**](https://mdream.dev/llms-txt/svelte) [Preact Documentation**2/10**](https://mdream.dev/llms-txt/preactjs) [--banner:js hack from https://github.com/evanw/esbuild/pull/2067**2/10**](https://mdream.dev/llms-txt/middy) [Hurl Documentation**2/10**](https://mdream.dev/llms-txt/hurl) [Tamagui Complete Documentation**2/10**](https://mdream.dev/llms-txt/tamagui) [Add a New Provider**2/10**](https://mdream.dev/llms-txt/novu) ## Size Extremes Largest Files [What is Datadog?1.8 MB](https://mdream.dev/llms-txt/datadoghq) [Add a New Provider1.4 MB](https://mdream.dev/llms-txt/novu) [Tamagui Complete Documentation1.0 MB](https://mdream.dev/llms-txt/tamagui) [Kilo Code Documentation992.6 KB](https://mdream.dev/llms-txt/kilocode) [Snowflake Documentation904.2 KB](https://mdream.dev/llms-txt/snowflake) [Gemini CLI Documentation859.1 KB](https://mdream.dev/llms-txt/gemini-cli) [PraisonAI819.5 KB](https://mdream.dev/llms-txt/praisonai) [SvelteKit537.5 KB](https://mdream.dev/llms-txt/sveltekit) Smallest Files [/llms.txt directory213 B](https://mdream.dev/llms-txt/llms-txt-directory) [Video.js v10229 B](https://mdream.dev/llms-txt/video-js) [Three.js274 B](https://mdream.dev/llms-txt/three-js) [Sherlock Project523 B](https://mdream.dev/llms-txt/sherlock) [Supabase546 B](https://mdream.dev/llms-txt/supabase) [WireMock560 B](https://mdream.dev/llms-txt/wiremock) [LITIENGINE 🎮: The pure 2D Java Game Engine596 B](https://mdream.dev/llms-txt/litiengine-the-pure-2d-java-game-engine) [Terragrunt615 B](https://mdream.dev/llms-txt/terragrunt) [llms.txt648 B](https://mdream.dev/llms-txt/llmstxt-org) ## Recommendations from the Data **Describe every link ** 53% of links across the ecosystem lack descriptions. Adding a short description after each link (e.g. `- [Page](url): What this page covers`) dramatically improves LLM comprehension and saves token budget. **Target 5 to 50 KB ** The median file is 16 KB. Files under 50 KB fit easily in any LLM context. Larger files should consider an "Optional" section or splitting into llms.txt (index) and llms-full.txt (full content). Only 12% of files use the Optional pattern. **Use standard section names ** "Docs", "Guides", "Reference", "Examples", and "Getting Started" are the most recognized patterns. Using familiar names helps LLMs navigate your file structure predictably. Avoid deeply nested H3+ headings (27% of files violate this). **Always use absolute URLs ** 21% of projects use relative URLs. Relative URLs break when the file is consumed outside its original domain (AI agents, documentation aggregators, IDE integrations). Always use full https:// URLs. **Consider llms-full.txt ** 28% of projects reference an llms-full.txt companion file. This pattern is growing because it enables RAG pipelines and deep context loading without requiring the LLM to fetch each page individually. ## Frequently Asked Questions We track and analyze llms.txt files from 523+ projects across frameworks, libraries, platforms, databases, and developer tools. The standard has seen rapid adoption since its introduction, particularly among developer documentation sites. The median llms.txt file is around 16 KB with about 3,992 tokens. This fits comfortably in any LLM context window. Files over 100 KB start to consume significant context, and files over 500 KB may exceed smaller model limits. Aim for 5 to 50 KB with well-described links. The most common section names are "Docs", "Guides", "Reference", "Examples", and "Getting Started". Most effective files use 3 to 8 sections. An "Optional" section is a useful pattern that lets LLMs skip less critical content when context is limited. Yes. Only 54% of links across all analyzed files include descriptions, but descriptions dramatically improve AI comprehension. A link description tells the LLM what the page covers without needing to fetch it, saving tokens and improving accuracy. About 10% of analyzed projects reference an llms-full.txt companion file. This file embeds the full content of linked pages inline, so AI systems can consume everything in one request. It is especially valuable for RAG pipelines and deep context loading. We fetch and parse every llms.txt file from our registry of 523+ projects, validate structure against the spec, extract statistics on size, sections, links, and URL patterns, then compute compliance scores. The data is regenerated periodically to track changes. ## Related Tools [**llms.txt Generator**Generate an llms.txt for your own site.](https://mdream.dev/tools/llms-txt/generator) [**llms.txt Validator**Validate your file against the spec.](https://mdream.dev/tools/llms-txt/validator) --- - **Page:** Answer Engine Optimization · Tools and tactics for AEO · mdream - **Source:** https://mdream.dev/guides/answer-engine-optimization - **Description:** Answer engine optimization guide: tools, tactics, structured data, and measurement. Rank in ChatGPT, Perplexity, Google AI Overviews, and voice assistants. [All guides](https://mdream.dev/guides) Guide # **Answer engine optimization (AEO)** Answer engine optimization (AEO) is the practice of structuring content so that ChatGPT, Claude, Perplexity, Google AI Overviews, and voice assistants pick it as the canonical answer. Updated 2026-04-22 Rendering guide… ## Check this on your site [

**AI search visibility audit **

One-click audit: llms.txt, Accept-header, robots.txt, sitemap.md, token savings.](https://mdream.dev/tools/ai-audit) ## Related guides [

**Generative engine optimization (GEO)**

Generative engine optimization (GEO) is the discipline of getting your content cited by ChatGPT, Claude, Perplexity, and Google AI Overviews. It blends classic SEO, structured data, and new formats like llms.txt + Accept-header markdown.](https://mdream.dev/guides/generative-engine-optimization) [

**Markdown for agents**

AI agents read markdown 2–5× more efficiently than HTML. Serve it via Accept-header negotiation, sitemap.md, and \ and every agent-originated request instantly becomes cheaper and smarter.](https://mdream.dev/guides/markdown-for-agents) [

**The accept-markdown header**

Accept: text/markdown is an HTTP content-negotiation header that signals an AI agent prefers clean markdown over HTML. Implementing it correctly means a Content-Type match, a Vary: Accept header, honoring q-values, and 406 on unsatisfiable requests.](https://mdream.dev/guides/accept-markdown-header) --- - **Page:** llms.txt for SvelteKit · Add an AI Sitemap to SvelteKit · mdream - **Source:** https://mdream.dev/tools/llms-txt/sveltekit - **Description:** How to add llms.txt for SvelteKit. Drop a static file in static/, expose a +server.ts endpoint at /llms.txt, or prerender the route from your MDsveX collection. Works with every SvelteKit adapter. [llms.txt Directory](https://mdream.dev/llms-txt) # **llms.txt for **SvelteKit Ship a spec-compliant /llms.txt from any SvelteKit app. Static asset, server route, or a +server.ts endpoint that builds from MDsveX. ## Why llms.txt on SvelteKit? SvelteKit serves everything under static/ directly, so a hand-written llms.txt lands at the site root with no routing needed. For dynamic content, a +server.ts endpoint can return text/plain and is portable across every adapter: adapter-vercel, adapter-cloudflare, adapter-node, and adapter-static. If you build docs with MDsveX or a content collection, mark the endpoint with export const prerender = true so llms.txt ships as a static asset on every deploy. Zero runtime cost, stays in sync with your pages. ## Install options Pick whichever fits your stack. For most teams the module or a static file is enough; reach for the dynamic route when you need request-time content. Static file Drop a file at `static/llms.txt`. No build step, no server logic. ```` ```md # My SvelteKit Project > A short summary of what this project does and who it is for. ## Docs - [Getting Started](https://example.com/docs/getting-started): Quick setup guide - [API Reference](https://example.com/docs/api): Full API docs ``` ```` Dynamic route Generate at request time from a CMS, MDX collection, or database. Handler at `src/routes/llms.txt/+server.ts`. ```` ```ts export const prerender = true export async function GET() { const body = await buildLlmsTxt() return new Response(body, { headers: { 'Content-Type': 'text/plain; charset=utf-8', 'Cache-Control': 'public, max-age=0, s-maxage=3600, stale-while-revalidate=86400', }, }) } async function buildLlmsTxt() { // Replace with your MDsveX collection or CMS query return `# My SvelteKit Project\n\n> Short summary.\n\n## Docs\n\n- [Getting Started](https://example.com/docs/getting-started)\n` } ``` ```` SvelteKit endpoint. Mark prerender = true for static output, or leave false for request-time generation. ## Serve markdown to agents Claude Code, Cursor, and Codex increasingly request markdown via the `Accept: text/markdown` header. Pairing your llms.txt with per-page markdown responses makes your whole site legible to agents, not just the index. [~~Generate llms.txt from your URL ~~](https://mdream.dev/tools/llms-txt/generator) or browse the [~~llms.txt directory ~~](https://mdream.dev/llms-txt) for real examples. ## Frequently Asked Questions Use static/llms.txt for a hand-curated file you rarely change. Use src/routes/llms.txt/+server.ts when you want to generate the file from your content collection. Don't ship both — the static asset wins and the endpoint never runs. Yes, as long as you add export const prerender = true. The adapter pre-renders the endpoint into a static llms.txt file at build time. Works with adapter-cloudflare, adapter-vercel, adapter-netlify, and adapter-node too. In your +server.ts endpoint, use globalThis.\_importMeta\_.glob() to import all your markdown posts, iterate over them, and return a formatted llms.txt body. Prerender the route so the file builds once per deploy. text/plain; charset=utf-8 is the safest default. text/markdown; charset=utf-8 is also valid per the spec and plays well with agents that negotiate on Accept: text/markdown. Either works; pick text/plain if you want zero compatibility risk. ## Related tools [

**Generator **

Crawl any site, generate a spec-compliant llms.txt.](https://mdream.dev/tools/llms-txt/generator) [

**Validator **

Check your file against the spec.](https://mdream.dev/tools/llms-txt/validator) [

**Directory **

Browse 95+ real llms.txt files.](https://mdream.dev/llms-txt) --- - **Page:** llms.txt for Astro · Add an AI Sitemap to Astro · mdream - **Source:** https://mdream.dev/tools/llms-txt/astro - **Description:** How to add llms.txt for Astro. Drop a file in public/, expose an API route at /llms.txt, or iterate your content collections to build one at ship time. Works with every Astro adapter and with Starlight. [llms.txt Directory](https://mdream.dev/llms-txt) # **llms.txt for **Astro Add a spec-compliant /llms.txt to your Astro site. Static file, API route, or a tiny integration that builds from your content collections. ## Why llms.txt on Astro? Astro is docs-first: Starlight sites, blogs, and marketing pages usually have a well-structured content collection already. An API route at src/pages/llms.txt.ts can iterate getCollection() and emit a spec-compliant file that stays in sync with your posts automatically. For fully static sites, add export const prerender = true and the route is compiled into a static asset at build time. There is no runtime cost, and the output works with adapter-cloudflare, adapter-vercel, adapter-netlify, and adapter-node. ## Install options Pick whichever fits your stack. For most teams the module or a static file is enough; reach for the dynamic route when you need request-time content. Static file Drop a file at `public/llms.txt`. No build step, no server logic. ```` ```md # My Astro Project > A short summary of what this project does and who it is for. ## Docs - [Getting Started](https://example.com/docs/getting-started): Quick setup guide - [Guides](https://example.com/guides): How-to guides ## Blog - [Latest Posts](https://example.com/blog): Recent posts ``` ```` Dynamic route Generate at request time from a CMS, MDX collection, or database. Handler at `src/pages/llms.txt.ts`. ```` ```ts import type { APIRoute } from 'astro' import { getCollection } from 'astro:content' export const prerender = true export const GET: APIRoute = async ({ site }) => { const docs = await getCollection('docs') const origin = site?.origin ?? 'https://example.com' const lines = ['# My Astro Project', '', '> Short summary.', '', '## Docs'] for (const entry of docs) { const url = new URL(`/docs/${entry.slug}`, origin).href lines.push(`- [${entry.data.title}](${url}): ${entry.data.description ?? ''}`) } return new Response(lines.join('\n'), { headers: { 'Content-Type': 'text/plain; charset=utf-8' }, }) } ``` ```` Astro API route. Prerender is true by default in static output — leave it for static, or switch to output: "server" in astro.config.mjs for request-time generation. ## Serve markdown to agents Claude Code, Cursor, and Codex increasingly request markdown via the `Accept: text/markdown` header. Pairing your llms.txt with per-page markdown responses makes your whole site legible to agents, not just the index. [~~Generate llms.txt from your URL ~~](https://mdream.dev/tools/llms-txt/generator) or browse the [~~llms.txt directory ~~](https://mdream.dev/llms-txt) for real examples. ## Frequently Asked Questions Yes. Starlight uses Astro content collections under the hood, so the src/pages/llms.txt.ts example above works unchanged. Iterate the "docs" collection, emit one link per entry, and you get an llms.txt that mirrors your Starlight sidebar. public/llms.txt is the fastest path for a hand-curated file. src/pages/llms.txt.ts is the better choice when your content lives in collections — the route regenerates the file every deploy with zero manual upkeep. Yes. Remove export const prerender = true and the route runs on every request. Cache it with Cache-Control or at the CDN layer if you don't want to hit your content layer per request. Duplicate the endpoint as src/pages/llms-full.txt.ts and inline the full rendered body of each entry instead of emitting a link. Astro's content render() returns the full HTML or MDX, which you can pipe through an HTML-to-Markdown converter like mdream. ## Related tools [

**Generator **

Crawl any site, generate a spec-compliant llms.txt.](https://mdream.dev/tools/llms-txt/generator) [

**Validator **

Check your file against the spec.](https://mdream.dev/tools/llms-txt/validator) [

**Directory **

Browse 95+ real llms.txt files.](https://mdream.dev/llms-txt) --- - **Page:** llms.txt for Remix · Add an AI Sitemap to Remix · mdream - **Source:** https://mdream.dev/tools/llms-txt/remix - **Description:** How to add llms.txt for Remix or React Router v7. Drop a static file in public/, add a resource route at app/routes/llms[.]txt.ts, or generate it from MDX. Works with Vercel, Cloudflare, Fly, and every Remix adapter. -- title: "llms.txt for Remix · Add an AI Sitemap to Remix · mdream" canonical_url: "https://mdream.dev/tools/llms-txt/remix" last_updated: "2026-10-07T05:19:59.715Z" meta: description: "How to add llms.txt for Remix or React Router v7. Drop a static file in public/, add a resource route at app/routes/llms[.]txt.ts, or generate it from MDX. Works with Vercel, Cloudflare, Fly, and every Remix adapter." "og:description": "How to add llms.txt for Remix: static file, resource route, or MDX-driven loader." "og:title": "llms.txt for Remix — Add an AI Sitemap" --- [llms.txt Directory](https://mdream.dev/llms-txt) # **llms.txt for **Remix Serve /llms.txt from any Remix or React Router v7 app. Static asset, resource route, or a loader that builds from your MDX content. ## Why llms.txt on Remix? Remix (and React Router v7 in framework mode) treats any route that only exports a loader as a resource route — perfect for non-HTML responses like llms.txt. The handler runs on every Remix adapter without changes: Vercel, Cloudflare Workers, Fly, Node. Because Remix loaders run server-side on every request, you can read from a CMS, Contentlayer, fumadocs, or your MDX collection at build or runtime. Add a long Cache-Control header and Remix's platform handlers will cache the response at the edge. ## Install options Pick whichever fits your stack. For most teams the module or a static file is enough; reach for the dynamic route when you need request-time content. Static file Drop a file at `public/llms.txt`. No build step, no server logic. ```` ```md # My Remix App > A short summary of what this app does and who it is for. ## Docs - [Getting Started](https://example.com/docs/getting-started): Quick setup guide - [API Reference](https://example.com/docs/api): Full API docs ``` ```` Dynamic route Generate at request time from a CMS, MDX collection, or database. Handler at `app/routes/llms[.]txt.ts`. ```` ```ts import type { LoaderFunctionArgs } from '@remix-run/node' export async function loader(_args: LoaderFunctionArgs) { const body = await buildLlmsTxt() return new Response(body, { headers: { 'Content-Type': 'text/plain; charset=utf-8', 'Cache-Control': 'public, max-age=0, s-maxage=3600, stale-while-revalidate=86400', }, }) } async function buildLlmsTxt() { // Replace with your MDX collection, Contentlayer, or CMS query return `# My Remix App\n\n> Short summary.\n\n## Docs\n\n- [Getting Started](https://example.com/docs/getting-started)\n` } ``` ```` Escape the dot in the filename so Remix treats /llms.txt as a literal route. Export only a loader — Remix classifies it as a resource route and skips the document shell. ## Serve markdown to agents Claude Code, Cursor, and Codex increasingly request markdown via the `Accept: text/markdown` header. Pairing your llms.txt with per-page markdown responses makes your whole site legible to agents, not just the index. [~~Generate llms.txt from your URL ~~](https://mdream.dev/tools/llms-txt/generator) or browse the [~~llms.txt directory ~~](https://mdream.dev/llms-txt) for real examples. ## Frequently Asked Questions Remix uses dots in route filenames to indicate nested segments. To treat llms.txt as a literal URL, escape the dot with square brackets: app/routes/llms\[.]txt.ts. The resulting route serves at /llms.txt. Yes. React Router v7 in framework mode uses the same resource-route pattern: export a loader, return a Response, and the route serves raw text. The filename convention is identical. Remix does not have built-in static generation, but you can pre-build llms.txt into public/ via a prebuild script that writes to disk. For adapters that support prerendering (adapter-static forks, Vercel's ISR), cache the loader response for 24h. A resource route is any Remix route that exports a loader or action without a default component export. Remix skips the HTML document shell and returns your Response directly. Perfect for llms.txt, sitemap.xml, RSS feeds, and OG images. ## Related tools [

**Generator **

Crawl any site, generate a spec-compliant llms.txt.](https://mdream.dev/tools/llms-txt/generator) [

**Validator **

Check your file against the spec.](https://mdream.dev/tools/llms-txt/validator) [

**Directory **

Browse 95+ real llms.txt files.](https://mdream.dev/llms-txt) --- - **Page:** Markdown for agents · Why serving markdown wins · mdream - **Source:** https://mdream.dev/guides/markdown-for-agents - **Description:** Why AI agents prefer markdown over HTML, how to serve it via Accept negotiation, sitemap.md, and , and which agents actually respect the standard. [All guides](https://mdream.dev/guides) Guide # **Markdown for agents** AI agents read markdown 2–5× more efficiently than HTML. Serve it via Accept-header negotiation, sitemap.md, and \ and every agent-originated request instantly becomes cheaper and smarter. Updated 2026-04-22 Rendering guide… ## Check this on your site [

**AI search visibility audit **

One-click audit: llms.txt, Accept-header, robots.txt, sitemap.md, token savings.](https://mdream.dev/tools/ai-audit) ## Related guides [

**Generative engine optimization (GEO)**

Generative engine optimization (GEO) is the discipline of getting your content cited by ChatGPT, Claude, Perplexity, and Google AI Overviews. It blends classic SEO, structured data, and new formats like llms.txt + Accept-header markdown.](https://mdream.dev/guides/generative-engine-optimization) [

**Answer engine optimization (AEO)**

Answer engine optimization (AEO) is the practice of structuring content so that ChatGPT, Claude, Perplexity, Google AI Overviews, and voice assistants pick it as the canonical answer.](https://mdream.dev/guides/answer-engine-optimization) [

**The accept-markdown header**

Accept: text/markdown is an HTTP content-negotiation header that signals an AI agent prefers clean markdown over HTML. Implementing it correctly means a Content-Type match, a Vary: Accept header, honoring q-values, and 406 on unsatisfiable requests.](https://mdream.dev/guides/accept-markdown-header) --- - **Page:** llms.txt for Next.js · Add an AI Sitemap to Next.js · mdream - **Source:** https://mdream.dev/tools/llms-txt/nextjs - **Description:** How to add llms.txt for Next.js. Drop a static file into public/, ship a dynamic App Router route handler, or generate at build time. Works with App Router, Pages Router, and edge runtimes. [llms.txt Directory](https://mdream.dev/llms-txt) # **llms.txt for **Next.js Add a spec-compliant /llms.txt to any Next.js app. Static file, App Router route handler, or build-time generation. ## Why llms.txt on Next.js? Next.js apps ship both static assets and server-rendered pages, so there are two equally valid homes for llms.txt. A static file in public/ is the simplest path; a dynamic route handler lets you pull from your CMS, file system, or docs collection at request time and set cache headers per deploy target. App Router route handlers run on Node and the edge, so the same /llms.txt handler works on Vercel, Cloudflare, and AWS. For docs sites built with Contentlayer, Velite, or fumadocs, generating llms.txt at build time keeps the file in sync with your MDX collection with zero runtime cost. ## Install options Pick whichever fits your stack. For most teams the module or a static file is enough; reach for the dynamic route when you need request-time content. Static file Drop a file at `public/llms.txt`. No build step, no server logic. ```` ```md # My Next.js Project > A short summary of what this project does and who it is for. ## Docs - [Getting Started](https://example.com/docs/getting-started): Quick setup guide - [API Reference](https://example.com/docs/api): Full API docs ## Examples - [Starter Template](https://example.com/examples/starter): Minimal Next.js starter ``` ```` Dynamic route Generate at request time from a CMS, MDX collection, or database. Handler at `app/llms.txt/route.ts`. ```` ```ts import { NextResponse } from 'next/server' export const revalidate = 3600 export async function GET() { const body = await buildLlmsTxt() return new NextResponse(body, { headers: { 'Content-Type': 'text/plain; charset=utf-8', 'Cache-Control': 'public, max-age=0, s-maxage=3600, stale-while-revalidate=86400', }, }) } async function buildLlmsTxt() { // Replace with your own content source (MDX, CMS, filesystem…) return `# My Next.js Project\n\n> Short summary.\n\n## Docs\n\n- [Getting Started](https://example.com/docs/getting-started)\n` } ``` ```` App Router route handler. Revalidates once per hour — swap \`revalidate\` for \`dynamic = "force-dynamic"\` if you want per-request generation. ## Serve markdown to agents Claude Code, Cursor, and Codex increasingly request markdown via the `Accept: text/markdown` header. Pairing your llms.txt with per-page markdown responses makes your whole site legible to agents, not just the index. [~~Generate llms.txt from your URL ~~](https://mdream.dev/tools/llms-txt/generator) or browse the [~~llms.txt directory ~~](https://mdream.dev/llms-txt) for real examples. ## Frequently Asked Questions For most sites, drop it in public/llms.txt — Next.js serves everything under public/ at the site root. Use an App Router route handler at app/llms.txt/route.ts when you need to generate the file dynamically from a CMS, MDX collection, or database. Yes. Next.js serves static files from public/ before hitting the app router. If you keep both, delete public/llms.txt so your route handler receives requests. Yes. Run a script in your prebuild step (for example, via a package.json "prebuild" hook) that writes public/llms.txt. This works with Contentlayer, Velite, fumadocs, or any MDX collection, and avoids runtime cost. Yes. Route handlers marked with export const runtime = "edge" can return text/plain responses. For large llms-full.txt files, prefer the Node runtime so you have more memory headroom. The spec accepts both. text/plain is the safer default for caching and CDN compatibility; text/markdown is the right content-type when you also want clients that negotiate on Accept: text/markdown to treat the response as markdown. You can send text/markdown; charset=utf-8 without breaking plain-text clients. ## Related tools [

**Generator **

Crawl any site, generate a spec-compliant llms.txt.](https://mdream.dev/tools/llms-txt/generator) [

**Validator **

Check your file against the spec.](https://mdream.dev/tools/llms-txt/validator) [

**Directory **

Browse 95+ real llms.txt files.](https://mdream.dev/llms-txt) --- - **Page:** Accept: text/markdown · The authoritative guide · mdream - **Source:** https://mdream.dev/guides/accept-markdown-header - **Description:** Everything you need to implement Accept-header content negotiation for markdown: spec references, Vary, q-values, 406 responses, Cloudflare and Vercel examples. [All guides](https://mdream.dev/guides) Guide # **The accept-markdown header** Accept: text/markdown is an HTTP content-negotiation header that signals an AI agent prefers clean markdown over HTML. Implementing it correctly means a Content-Type match, a Vary: Accept header, honoring q-values, and 406 on unsatisfiable requests. Updated 2026-04-22 Rendering guide… ## Check this on your site [

**AI search visibility audit **

One-click audit: llms.txt, Accept-header, robots.txt, sitemap.md, token savings.](https://mdream.dev/tools/ai-audit) ## Related guides [

**Generative engine optimization (GEO)**

Generative engine optimization (GEO) is the discipline of getting your content cited by ChatGPT, Claude, Perplexity, and Google AI Overviews. It blends classic SEO, structured data, and new formats like llms.txt + Accept-header markdown.](https://mdream.dev/guides/generative-engine-optimization) [

**Answer engine optimization (AEO)**

Answer engine optimization (AEO) is the practice of structuring content so that ChatGPT, Claude, Perplexity, Google AI Overviews, and voice assistants pick it as the canonical answer.](https://mdream.dev/guides/answer-engine-optimization) [

**Markdown for agents**

AI agents read markdown 2–5× more efficiently than HTML. Serve it via Accept-header negotiation, sitemap.md, and \ and every agent-originated request instantly becomes cheaper and smarter.](https://mdream.dev/guides/markdown-for-agents) --- - **Page:** Accept-header tester · Replay AI agent requests for markdown · mdream - **Source:** https://mdream.dev/tools/accept-header - **Description:** Replay real AI agent requests against your URL. See what Claude Code, Cursor, Codex, ChatGPT, Perplexity, and Gemini get back when they ask for markdown. Pass/fail lights for Vary, 406, and q-values. [Home](https://mdream.dev/) # **Accept-header tester ** Replay real AI-agent requests against any URL. See what Claude Code, Cursor, and Codex actually get back, with pass/fail lights for `Vary`, `406`, and q-values. zzz Ready to probe Pick an agent and enter a URL to test. ## What do these lights mean? - MD — server responded with `text/markdown` or `text/plain`. - Vary — response includes `Vary: Accept`, so caches won't mix HTML and markdown responses. - 406 — server returns spec-compliant 406 on unsatisfiable Accept values. Optional but nice. - q= — server honors Accept q-values, serving markdown when the client marks it as preferred. [Read the full guide ](https://mdream.dev/guides/accept-markdown-header) ## Agent support matrix Markdown response support by AI agent | Agent | Accept | User-Agent | MD | | --- | --- | --- | --- | | Claude Code | text/markdown, text/html, \*/\* | Claude-User/1.0 | full support | | Cursor | text/markdown, text/plain;q=0.9, \*/\*;q=0.8 | Cursor/1.0 | full support | | Codex CLI | text/html, \*/\* | codex/1.0 | partial support | | ChatGPT | \*/\* | ChatGPT-User/1.0 | none support | | Perplexity | \*/\* | PerplexityBot/1.0 | none support | | Gemini | \*/\* | Google-Extended | none support | ## Frequently Asked Questions Content negotiation is an HTTP mechanism where the client (browser, AI agent) advertises which response formats it understands via the Accept header. The server picks the best match and serves it with a matching Content-Type. For AI agents, Accept: text/markdown signals that the client prefers clean markdown over rendered HTML. Markdown is 40–80% smaller than the equivalent HTML, strips noise (nav, scripts, styles), and maps directly to the token-efficient format LLMs internally convert to anyway. Serving markdown on Accept negotiation is faster, cheaper, and gives agents better answers. As of 2026, only Claude Code and Cursor advertise text/markdown in their Accept header. OpenAI Codex CLI follows \ hints in HTML instead. ChatGPT, Perplexity, and Gemini fetchers still send \*/\* and parse whatever comes back. 406 Not Acceptable is the spec-compliant response when the client explicitly only wants a format the server cannot produce. Correctly returning 406 for Accept: application/vnd.pandoc is a sign of a well-implemented content negotiation layer. ## Scan the whole site [

**AI search visibility audit **

Accept-header + llms.txt + sitemap.md + robots + token savings. One shareable report.](https://mdream.dev/tools/ai-audit) --- - **Page:** llms.txt for Nuxt · Add an AI Sitemap to Nuxt · mdream - **Source:** https://mdream.dev/tools/llms-txt/nuxt - **Description:** How to add llms.txt for Nuxt. Use the @mdream/nuxt module for automatic generation, a Nitro server route for dynamic content, or a static file in public/. Works with Nuxt Content, Nitro presets, and any deploy target. [llms.txt Directory](https://mdream.dev/llms-txt) # **llms.txt for **Nuxt Ship a spec-compliant /llms.txt from your Nuxt site. Drop-in module, runtime Nitro route, or static asset. ## Why llms.txt on Nuxt? Nuxt apps build through Nitro, which serves static assets from public/ and dynamic handlers from server/routes/. That gives you three paths for llms.txt without changing any Nuxt configuration. If your docs live in Nuxt Content or @nuxt/content, the @mdream/nuxt module reads your rendered pages and produces llms.txt and llms-full.txt automatically at build time. No runtime work, and the output re-generates on every deploy. ## Install options Pick whichever fits your stack. For most teams the module or a static file is enough; reach for the dynamic route when you need request-time content. Recommended — module Install the official module. Reads your pages at build and writes llms.txt into the output. ```` ```bash npx nuxi module add @mdream/nuxt ``` ```` ```` ```ts export default defineNuxtConfig({ modules: ['@mdream/nuxt'], }) ``` ```` [~~@mdream/nuxt docs → ~~](https://github.com/harlan-zw/mdream) Static file Drop a file at `public/llms.txt`. No build step, no server logic. ```` ```md # My Nuxt Project > A short summary of what this project does and who it is for. ## Docs - [Getting Started](https://example.com/docs/getting-started): Quick setup guide - [API Reference](https://example.com/docs/api): Full API docs ## Examples - [Starter Template](https://example.com/examples/starter): Minimal Nuxt starter ``` ```` Dynamic route Generate at request time from a CMS, MDX collection, or database. Handler at `server/routes/llms.txt.ts`. ```` ```ts export default defineEventHandler(async (event) => { setResponseHeader(event, 'Content-Type', 'text/plain; charset=utf-8') setResponseHeader(event, 'Cache-Control', 'public, max-age=0, s-maxage=3600, stale-while-revalidate=86400') return await buildLlmsTxt() }) async function buildLlmsTxt() { // Replace with your own content source (Nuxt Content, filesystem…) return `# My Nuxt Project\n\n> Short summary.\n\n## Docs\n\n- [Getting Started](https://example.com/docs/getting-started)\n` } ``` ```` Nitro server route. Runs on every deploy target Nuxt supports — Node, Cloudflare, Vercel, Netlify, Deno. ## Serve markdown to agents Claude Code, Cursor, and Codex increasingly request markdown via the `Accept: text/markdown` header. Pairing your llms.txt with per-page markdown responses makes your whole site legible to agents, not just the index. [~~Generate llms.txt from your URL ~~](https://mdream.dev/tools/llms-txt/generator) or browse the [~~llms.txt directory ~~](https://mdream.dev/llms-txt) for real examples. ## Frequently Asked Questions Install @mdream/nuxt with nuxi module add @mdream/nuxt. The module reads your rendered pages and writes llms.txt and llms-full.txt into the build output. Both files are generated at build time, so there is no runtime overhead. Yes. @mdream/nuxt works alongside @nuxt/content and will include your rendered markdown pages. If you want full control, a Nitro server route at server/routes/llms.txt.ts can call queryCollection() (Nuxt Content v3) or queryContent() (v2) directly and format the output yourself. Yes. The module writes a static file into .output/public/, so every Nitro preset (cloudflare-pages, vercel, netlify, node-server, bun, deno) serves it with no extra configuration. Dynamic server routes also work on every preset. @mdream/nuxt generates both llms.txt and llms-full.txt by default. llms-full.txt embeds the full markdown of every page inline, which is useful for RAG pipelines and chat-with-docs apps. No extra configuration is needed. Prefer build-time generation via @mdream/nuxt — your llms.txt stays in sync with your pages automatically. Commit a static public/llms.txt only when you want a fully hand-curated file that never changes. ## Related tools [

**Generator **

Crawl any site, generate a spec-compliant llms.txt.](https://mdream.dev/tools/llms-txt/generator) [

**Validator **

Check your file against the spec.](https://mdream.dev/tools/llms-txt/validator) [

**Directory **

Browse 95+ real llms.txt files.](https://mdream.dev/llms-txt) --- - **Page:** Token counter · Free LLM token counter for OpenAI, Claude & Gemini · mdream - **Source:** https://mdream.dev/tools/token-counter - **Description:** Free online token counter for LLMs. Paste text, HTML, or a URL and get instant token counts for GPT-4, GPT-4o, Claude 3+, Llama 3 and Gemini. See how many tokens you save by converting HTML to Markdown with mdream. [Home](https://mdream.dev/) # **Token counter ** Count tokens for ChatGPT, Claude, Gemini, and Llama. Paste text, HTML, or a URL; we show counts across every major tokenizer and how many tokens you save by converting HTML to Markdown with [~~mdream ~~](https://mdream.dev/). zzz Ready Paste text, HTML, or a URL to count tokens. ## How mdream reduces tokens Every HTML page carries tag overhead that tokenizers charge you for. Converting to focused markdown before the LLM sees it keeps the context and drops the tax. 1. **1**### **Convert HTML to markdown** mdream strips scripts, nav, footers, and inline styles. What remains is the content an LLM actually needs. 2. **2**### **Tokenize both sides** We estimate cl100k\_base, o200k\_base, Llama 3, and Gemini counts so you can see the delta per model family. 3. **3**### **Compare and save** Typical pages drop 50–80% of tokens after conversion. That's real money back on every LLM call. ## Which tokenizer? - cl100k\_base — official OpenAI BPE, exact. GPT-3.5, GPT-4, and a very close approximation for Claude 3 & 3.5. Use this if you're unsure which model you're targeting. - o200k\_base — official OpenAI BPE, exact. GPT-4o and GPT-4o-mini. Roughly 8% fewer tokens than cl100k\_base on English text. - llama3 — estimate. Meta Llama 3's tokenizer isn't open; we scale cl100k by +8% based on published vocab size. - gemini — estimate. Google hasn't released Gemini's SentencePiece; we use a char-frequency heuristic. Treat this column as a ballpark. ## Frequently Asked Questions GPT-4, GPT-3.5, and Claude 3+ counts come from OpenAI's official cl100k\_base BPE vocabulary via gpt-tokenizer; GPT-4o counts use o200k\_base. These are exact. Llama 3 and Gemini tokenizers are not publicly released, so those columns are heuristic estimates calibrated against published model documentation. HTML carries tag overhead (class names, inline styles, script tags, nav menus, cookie banners). LLM tokenizers count every angle bracket, attribute, and piece of whitespace. Markdown strips all of that, keeping only the content an agent needs to answer a question. No. Text, HTML, and URL inputs are tokenized in-flight on a Cloudflare Worker and discarded on response. Nothing is logged or persisted. OpenAI's tokenizer only counts one tokenizer at a time and only accepts plain text. This tool also fetches URLs, runs mdream's HTML-to-markdown pipeline, and shows the token delta across four tokenizer families side by side. Yes. Anthropic has stated Claude 3 and 3.5 use a tokenizer very close to OpenAI's cl100k\_base. For most content, the cl100k\_base column is within a few percent of Anthropic's official count. ## Related [

**mdream **

Convert HTML to Markdown.](https://mdream.dev/) [

**Converters compared **

mdream vs Turndown vs node-html-markdown.](https://mdream.dev/tools/compare) [

**AI visibility audit **

Is your site ready for AI agents?](https://mdream.dev/tools/ai-audit) --- - **Page:** llms.txt for WordPress · Add an AI Sitemap to WordPress · mdream - **Source:** https://mdream.dev/tools/llms-txt/wordpress - **Description:** How to add llms.txt for WordPress. Upload a static file to your site root, install a plugin that builds one from your posts, or add a rewrite that generates it on request. Works with any host and theme. [llms.txt Directory](https://mdream.dev/llms-txt) # **llms.txt for **WordPress Add a spec-compliant /llms.txt to any WordPress site. Plugin, wp-content drop-in, or a rewrite rule that generates from your posts. ## Why llms.txt on WordPress? WordPress powers \~43% of the web, so AI systems hit a WordPress site more often than any other stack. A well-structured llms.txt is the fastest way to make your posts, docs, and product pages legible to ChatGPT, Claude, and Perplexity, especially when your theme buries important pages behind archives and taxonomies. WordPress gives you three clean install paths: upload a static file via SFTP or the file manager, install a plugin that generates it from your post types, or add a tiny snippet that registers /llms.txt as a rewrite and renders on demand. Pick the one that matches how much you want to automate. ## Install options Pick whichever fits your stack. For most teams the module or a static file is enough; reach for the dynamic route when you need request-time content. Static file Drop a file at `/llms.txt`. No build step, no server logic. ```` ```md # My WordPress Site > A short summary of what this site covers and who it is for. ## Guides - [Getting Started](https://example.com/getting-started): Quick setup guide - [Frequently Asked Questions](https://example.com/faq): Common questions answered ## Products - [Pricing](https://example.com/pricing): Plans and pricing - [Features](https://example.com/features): Feature list ``` ```` Dynamic route Generate at request time from a CMS, MDX collection, or database. Handler at `wp-content/mu-plugins/llms-txt.php`. ```` ```php {$desc}", '', '## Pages']; $pages = get_posts([ 'post_type' => ['page', 'post'], 'posts_per_page' => 50, 'post_status' => 'publish', ]); foreach ($pages as $p) { $url = get_permalink($p); $title = get_the_title($p); $excerpt = wp_trim_words(strip_tags($p->post_content), 20); $lines[] = "- [{$title}]({$url}): {$excerpt}"; } header('Content-Type: text/plain; charset=utf-8'); header('Cache-Control: public, max-age=3600'); echo implode("\n", $lines); exit; }); ``` ```` Drop this in wp-content/mu-plugins/ (create the folder if needed). The file is auto-loaded; no activation required, and /llms.txt is generated from your published posts and pages. ## Serve markdown to agents Claude Code, Cursor, and Codex increasingly request markdown via the `Accept: text/markdown` header. Pairing your llms.txt with per-page markdown responses makes your whole site legible to agents, not just the index. [~~Generate llms.txt from your URL ~~](https://mdream.dev/tools/llms-txt/generator) or browse the [~~llms.txt directory ~~](https://mdream.dev/llms-txt) for real examples. ## Frequently Asked Questions Upload a hand-written llms.txt to your site root via SFTP or your host's file manager. It lives alongside wp-config.php and is served directly without touching WordPress. This is the right choice for sites that want a fully curated, hand-edited file. Several plugins exist in the directory. They generate llms.txt from your published posts and pages and re-run on every publish. If you want zero-setup automation, pick one; if you want more control over what is included, use the mu-plugin snippet or upload a static file. Yes. Drop a tiny PHP snippet into wp-content/mu-plugins/llms-txt.php that registers /llms.txt as a rewrite rule and iterates over your published posts and pages. Must-use plugins auto-load, so there is no activation step. No. llms.txt is its own file at /llms.txt, separate from your robots.txt and sitemap.xml. SEO plugins do not touch it. Keep all three in sync: robots.txt for crawl access, sitemap.xml for search engines, llms.txt for AI systems. If you use the rewrite-rule approach, yes — visit Settings → Permalinks once and hit Save to flush the rewrite cache. Static files uploaded to the site root never need a flush. ## Related tools [

**Generator **

Crawl any site, generate a spec-compliant llms.txt.](https://mdream.dev/tools/llms-txt/generator) [

**Validator **

Check your file against the spec.](https://mdream.dev/tools/llms-txt/validator) [

**Directory **

Browse 95+ real llms.txt files.](https://mdream.dev/llms-txt) --- - **Page:** AI crawlers · Who crawls your site and how to control them · mdream - **Source:** https://mdream.dev/guides/ai-crawlers - **Description:** List of AI crawlers (GPTBot, ClaudeBot, CCBot, PerplexityBot, Google-Extended, Bytespider, Applebot-Extended) and how to control them with robots.txt and llms.txt. [All guides](https://mdream.dev/guides) Guide # **AI crawlers in 2026** Nine AI crawlers account for virtually all agent-originated traffic in 2026. Blanket-blocking them means no citations from ChatGPT, Claude, Perplexity, or Google AI Overviews. Block selectively with robots.txt and guide the rest with llms.txt. Updated 2026-04-22 Rendering guide… ## Check this on your site [

**AI search visibility audit **

One-click audit: llms.txt, Accept-header, robots.txt, sitemap.md, token savings.](https://mdream.dev/tools/ai-audit) ## Related guides [

**Generative engine optimization (GEO)**

Generative engine optimization (GEO) is the discipline of getting your content cited by ChatGPT, Claude, Perplexity, and Google AI Overviews. It blends classic SEO, structured data, and new formats like llms.txt + Accept-header markdown.](https://mdream.dev/guides/generative-engine-optimization) [

**Answer engine optimization (AEO)**

Answer engine optimization (AEO) is the practice of structuring content so that ChatGPT, Claude, Perplexity, Google AI Overviews, and voice assistants pick it as the canonical answer.](https://mdream.dev/guides/answer-engine-optimization) [

**Markdown for agents**

AI agents read markdown 2–5× more efficiently than HTML. Serve it via Accept-header negotiation, sitemap.md, and \ and every agent-originated request instantly becomes cheaper and smarter.](https://mdream.dev/guides/markdown-for-agents) --- - **Page:** Generative Engine Optimization · Tools, techniques, and measurement · mdream - **Source:** https://mdream.dev/guides/generative-engine-optimization - **Description:** Practical guide to generative engine optimization. Compare GEO tools, understand how ChatGPT, Perplexity, and Claude pick sources, and ship the changes that move citation share. [All guides](https://mdream.dev/guides) Guide # **Generative engine optimization (GEO)** Generative engine optimization (GEO) is the discipline of getting your content cited by ChatGPT, Claude, Perplexity, and Google AI Overviews. It blends classic SEO, structured data, and new formats like llms.txt + Accept-header markdown. Updated 2026-04-22 Rendering guide… ## Check this on your site [

**AI search visibility audit **

One-click audit: llms.txt, Accept-header, robots.txt, sitemap.md, token savings.](https://mdream.dev/tools/ai-audit) ## Related guides [

**Answer engine optimization (AEO)**

Answer engine optimization (AEO) is the practice of structuring content so that ChatGPT, Claude, Perplexity, Google AI Overviews, and voice assistants pick it as the canonical answer.](https://mdream.dev/guides/answer-engine-optimization) [

**Markdown for agents**

AI agents read markdown 2–5× more efficiently than HTML. Serve it via Accept-header negotiation, sitemap.md, and \ and every agent-originated request instantly becomes cheaper and smarter.](https://mdream.dev/guides/markdown-for-agents) [

**The accept-markdown header**

Accept: text/markdown is an HTTP content-negotiation header that signals an AI agent prefers clean markdown over HTML. Implementing it correctly means a Content-Type match, a Vary: Accept header, honoring q-values, and 406 on unsatisfiable requests.](https://mdream.dev/guides/accept-markdown-header) --- - **Page:** Free llms.txt Validator & Checker · Validate Your AI Sitemap · mdream - **Source:** https://mdream.dev/tools/llms-txt/validator - **Description:** Free llms.txt validator and checker. Validate your AI sitemap against the official llmstxt.org spec. Server-side fetch avoids CORS errors. Check heading structure, markdown links, blockquote summary, and formatting for ChatGPT, Perplexity, and RAG pipelines. No signup. [llms.txt Tools](https://mdream.dev/llms-txt) # **llms.txt Validator ** Check your `/llms.txt` file against the [~~official spec~~](https://llmstxt.org). Verify heading structure, markdown links, blockquote summary, and formatting. Enter a site URL or paste content directly. ## What is llms.txt? llms.txt is a proposed standard for making your website understandable to AI systems. Sometimes called an "AI sitemap" or "robots.txt for AI," it's a structured Markdown file placed at the root of your domain (`/llms.txt`) that provides an overview of your site's content, documentation, and APIs for tools like ChatGPT, Claude, and Perplexity. ### **AI Discovery ** Helps LLMs like ChatGPT and Claude understand your site without crawling every page. ### **Structured Context ** Organized Markdown with headings and links gives AI systems the right context fast. ### **Simple Format ** Just Markdown. No XML, no JSON. Easy to write, easy to maintain, easy to validate. ## What Makes a Valid llms.txt? The [~~llmstxt.org specification~~](https://llmstxt.org) defines the structure AI systems expect. This validator checks each rule automatically. 1. **1**### **H1 title** Exactly one H1 heading as the first line, naming the project or site 2. **2**### **Blockquote summary** A blockquote immediately after the H1 with a short description 3. **3**### **H2 sections** Content organized under H2 headings like ## Docs, ## API, ## Examples 4. **4**### **Markdown links** Resources as markdown links: - \[Name](url): Optional description 5. **5**### **Optional sections** Sections may be marked "Optional" to indicate non-essential content 6. **6**### **Plain text** File must be valid Markdown, UTF-8 encoded, served as text/plain or text/markdown ## Common llms.txt Errors `Missing H1` File must start with a single # heading. Multiple H1s or starting with H2 is invalid. `No blockquote` A > blockquote summary should follow the H1 heading immediately. `Bad link format` Links must use markdown syntax: - \[Label](https://url): Description `Relative URLs` All links should be absolute URLs (https://...), not relative paths. `Too large` Keep llms.txt concise. AI systems have context limits; a focused file performs better. `Wrong MIME` Serve as text/plain or text/markdown, not text/html. ## llms.txt vs robots.txt A common question: how does llms.txt relate to robots.txt? They serve opposite purposes and work together. ### `robots.txt`** Exclusion ** Tells crawlers which pages to **avoid**. Controls access for search engines and AI bots. Blocks paths, enforces crawl-delay. ### `llms.txt`** Inclusion ** Tells AI systems which pages to **prioritize**. Provides structure, context, and descriptions. Guides LLMs to your best content. Use both together: robots.txt to control who can access what, llms.txt to help AI systems understand the content you do want to share. Neither replaces the other. ## Example llms.txt A minimal, spec-compliant llms.txt file. Copy this as a starting point and customize for your site. ``` # My Project > A brief description of what this project does and who it's for. ## Docs - [Getting Started](https://example.com/docs/getting-started): Quick setup guide - [API Reference](https://example.com/docs/api): Full API documentation - [Configuration](https://example.com/docs/config): Configuration options ## Examples - [Basic Usage](https://example.com/examples/basic): Minimal working example - [Advanced](https://example.com/examples/advanced): Complex use cases ## Optional - [Blog](https://example.com/blog): Latest updates and tutorials - [Changelog](https://example.com/changelog): Version history ``` ## Frequently Asked Questions llms.txt is a proposed standard that provides a structured Markdown file at /llms.txt on your website. Sometimes called an "AI sitemap" or "robots.txt for AI," it helps AI systems like ChatGPT, Claude, and Perplexity understand your site's content, documentation, and APIs without crawling every page. Enter your domain above (e.g., example.com) and click Validate. This checker fetches your /llms.txt server-side (avoiding CORS issues that browser-based validators hit), parses the Markdown structure, and validates it against the official specification including heading hierarchy, link formatting, and required sections. llms.txt is a concise overview with links to detailed resources. llms-full.txt is an optional companion file that contains the full expanded content inline, so AI systems can consume everything in a single request without following links. This is especially useful for RAG pipelines and "chat with your docs" applications. robots.txt controls crawler access by telling bots which pages to exclude. llms.txt is the opposite: it tells AI systems which pages to include and how they relate to each other. They complement each other. Use robots.txt for exclusion, llms.txt for structured inclusion. llms.txt is designed for AI systems and LLM-powered tools, not traditional search engine crawlers. Search engines use robots.txt and XML sitemaps. However, AI search products like Perplexity, ChatGPT Browse, and Google AI Overviews can benefit from a well-structured llms.txt. llms.txt is a community proposal by Jeremy Howard, not a W3C or IETF standard. It has gained adoption across many sites and AI tools. The specification is published at llmstxt.org and continues to evolve. For WordPress, place the file in your site root or use a plugin. For Next.js, add it to your public/ directory or create a route handler at /llms.txt. For Nuxt, use the @mdream/nuxt module for automatic generation, or place it in public/llms.txt for a static file. The ideal range is 20 to 50 links for a standard llms.txt. Focus on your most important documentation, guides, and API references. AI systems have context limits, so a curated file with good descriptions outperforms a massive list of every page on your site. No. They serve different audiences. sitemap.xml tells search engine crawlers which URLs to index. llms.txt tells AI systems which content matters and how it is structured. Think of llms.txt as an "AI sitemap" that complements your existing sitemap.xml and robots.txt. Browser-based validators often fail because servers block cross-origin requests. This validator fetches your llms.txt server-side, bypassing CORS entirely. If you still see errors, the server may be blocking non-browser user agents or the file may not exist at the expected path. Yes. llms.txt provides a structured index ideal for populating vector databases. Use the concise llms.txt as a retrieval map, or use llms-full.txt (which embeds all content inline) for direct ingestion into RAG systems. The section structure maps naturally to document chunks for embedding. ## References [llmstxt.org Specification The official llms.txt proposal and spec.](https://llmstxt.org) [llms.txt Directory Browse sites that have adopted llms.txt.](https://directory.llmstxt.cloud) [Anthropic llms.txt Example from Anthropic's documentation.](https://docs.anthropic.com/llms.txt) ## Related Tools [

**llms.txt Generator **

Crawl any site and generate a spec-compliant llms.txt.](https://mdream.dev/tools/llms-txt/generator) [

**llms.txt Insights **

See how your file compares to 95+ real-world llms.txt files.](https://mdream.dev/insights) [

**mdream HTML to Markdown **

Convert HTML to LLM-optimized Markdown. Fastest in JS and Rust.](https://mdream.dev/) --- - **Page:** HTML to Markdown Converters Compared · Mdream - **Source:** https://mdream.dev/tools/compare - **Description:** Compare HTML to Markdown converters: mdream vs Turndown vs node-html-markdown. Benchmarks, token output, features, and npm package size. Find the best converter for your project. [Home](https://mdream.dev/) # **HTML to Markdown Converters Compared ** How does mdream stack up against [~~Turndown~~](https://github.com/mixmark-io/turndown), [~~node-html-markdown~~](https://github.com/crosstype/node-html-markdown), and [~~html-to-markdown~~](https://github.com/johanneskaufmann/html-to-markdown)? Speed benchmarks, token output, and features at a glance. ## Speed Benchmarks 166 KB HTML · lower is better mdream (Rust) mdream (NAPI) mdream (JS) Turndown (JS) node-html-md (JS) [Run live benchmarks in your browser ](https://mdream.dev/tools/benchmarks) ## Feature Comparison HTML to Markdown converter feature comparison | Feature | mdream | Turndown | node-html-markdown | html-to-markdown (Go) | | --- | --- | --- | --- | --- | | Language | Rust + JS + WASM | JavaScript | JavaScript | Go | | Speed (166KB) | **0.34ms (Rust), 3.26ms (JS)** | 11.26ms | 14.31ms | \~2ms | | Token Output | **Lowest** | High | High | Medium | | Bundle Size | 23.4kB (JS), 101.6kB (WASM) | \~30kB | \~25kB | Go binary | | Streaming | | | | | | Dependencies | 0 | 1 | 3 | 0 | | Maintained | | | | | ## mdream The fastest HTML to Markdown converter in any runtime. Built in Rust with bindings for Node.js (NAPI), browsers (WASM), and a pure JavaScript fallback. Designed for LLM pipelines where token efficiency and speed matter. [GitHub](https://github.com/harlan-zw/mdream) [ npm](https://npmx.dev/package/mdream) ## Turndown The most popular HTML to Markdown converter for JavaScript. Turndown has a clean plugin API and good browser support, but is [~~no longer actively maintained~~](https://github.com/mixmark-io/turndown). At 11.26ms for 166kB HTML, it is 3.4x slower than mdream's JS engine. On the same fixture, mdream minimal output uses 38% fewer tokens. ## node-html-markdown A Node.js focused converter with custom translation rules. Actively maintained but the slowest JavaScript option at 14.31ms for 166KB HTML. Produces similar token counts to Turndown. Best suited for Node.js projects that need custom element handling. ## html-to-markdown (Go) A well-regarded Go implementation by Johannes Kaufmann. Fast at \~2ms, good structure preservation, but only available as a Go module. If your stack is Go, this is a solid choice alongside mdream's Rust crate. ## Frequently Asked Questions mdream is the fastest HTML to Markdown converter across all runtimes. The Rust implementation processes 166kB of HTML in 0.34ms. The pure JavaScript version runs in 3.26ms, compared with Turndown at 11.26ms. mdream is the recommended Turndown alternative. On the 166kB benchmark fixture, it produces 38% fewer tokens and its Rust engine is 33x faster. It has zero dependencies and supports streaming for large documents. On the real 166kB Wikipedia fixture, mdream minimal output uses 38% fewer o200k\_base tokens than Turndown. This matters for LLM context windows where every token counts. mdream has a different API than Turndown, but covers the same use cases. Migration is straightforward: replace the Turndown import with mdream, pass your HTML string, and get Markdown back. mdream also supports options like minimal mode, table handling, and custom element filtering. mdream is designed specifically for LLM pipelines. It strips unnecessary whitespace, removes redundant formatting, and produces token-efficient Markdown. It also supports streaming for processing large documents without loading everything into memory, and includes a crawler for generating llms.txt files. Yes. mdream runs everywhere: Node.js (via Rust NAPI), browsers (via WASM or pure JS), edge runtimes (Cloudflare Workers, Deno, Bun), and as a Rust crate. The pure JS core is 23.4kB gzipped with zero dependencies. ## Try It [

**Online Converter **

Convert HTML to Markdown in your browser.](https://mdream.dev/) [

**Live Benchmarks **

Run speed tests in your own browser.](https://mdream.dev/tools/benchmarks) --- - **Page:** llms.txt Examples · Real-World AI Sitemaps · mdream - **Source:** https://mdream.dev/examples - **Description:** Browse real llms.txt examples from Stripe, Vercel, Nuxt, Vue.js, Cloudflare, and more. Learn the llms.txt format, best practices, and how to write your own AI sitemap file. [Home](https://mdream.dev/) # **llms.txt Examples ** Real-world llms.txt files from popular sites. See how leading companies and open source projects structure their AI sitemaps following the [~~llmstxt.org specification~~](https://llmstxt.org). ## Live Examples [**Nuxt**nuxt.com Full framework documentation covering modules, composables, configuration, deployment, and the Nitro server engine.FrameworkOpen SourceBrowse interactively](https://mdream.dev/llms-txt/nuxt) [**Vue.js**vuejs.org Core Vue documentation including composition API, reactivity, components, routing, and state management guides.FrameworkOpen SourceBrowse interactively](https://mdream.dev/llms-txt/vue) [**Vercel**vercel.com Platform documentation covering deployment, edge functions, serverless, domains, and integrations.PlatformInfrastructureView raw](https://vercel.com/llms.txt) [**Stripe**docs.stripe.com Payment API documentation with guides for charges, subscriptions, webhooks, and Connect.API docsFintechView raw](https://docs.stripe.com/llms.txt) [**Cloudflare**developers.cloudflare.com Developer platform docs covering Workers, Pages, D1, R2, KV, and AI services.PlatformInfrastructureView raw](https://developers.cloudflare.com/llms.txt) [**Supabase**supabase.com Backend-as-a-service docs for database, auth, storage, edge functions, and realtime features.PlatformOpen SourceView raw](https://supabase.com/llms.txt) [**Anthropic**docs.anthropic.com API documentation for Claude models, including messages, tool use, prompt caching, and safety guidelines.API docsAIView raw](https://docs.anthropic.com/llms.txt) [**Cursor**docs.cursor.com Editor documentation covering AI features, tab completion, context management, and MCP integration.Developer ToolAIView raw](https://docs.cursor.com/llms.txt) ## What Makes a Good llms.txt? Based on our [~~analysis of 95+ real-world llms.txt files ~~](https://mdream.dev/insights), the best files share these patterns. 1. **1**### **Start with a clear H1** The first line should be a single # heading with your project or company name. 90% of analyzed files get this right. 2. **2**### **Add a blockquote summary** Follow the H1 with a > blockquote that explains what the site or project does. Only 50% of files include one, so this is an easy differentiator. 3. **3**### **Use H2 sections for structure** Group related links under ## headings. The most popular names across 523+ files: "Docs", "Guides", "Reference", "Examples", "Getting Started". 4. **4**### **Describe every link** Use the format - \[Title](url): Description. Only 54% of links include descriptions, but they dramatically improve LLM comprehension. 5. **5**### **Curate, do not dump** The median file has 83 links across 3 sections. A focused llms.txt under 50 KB outperforms a massive list. 6. **6**### **Keep links absolute** Always use full URLs (https://...). 15% of projects still use relative URLs, which break when consumed outside the original domain. ## llms.txt Template Copy this template as a starting point for your own llms.txt file. ### **# Your Project Name ** > *Short description of what your project does and who it's for. * #### **## Docs ** - \- \[Getting Started](https://example.com/docs/getting-started): Quick setup guide - \- \[Configuration](https://example.com/docs/config): All configuration options - \- \[Deployment](https://example.com/docs/deploy): Deploy to production #### **## API Reference ** - \- \[REST API](https://example.com/api): Full REST API documentation - \- \[Authentication](https://example.com/api/auth): Auth endpoints and tokens #### **## Guides ** - \- \[Tutorial](https://example.com/guides/tutorial): Step-by-step first project - \- \[Migration](https://example.com/guides/migration): Upgrading from v1 to v2 ## Frequently Asked Questions A good llms.txt starts with a clear H1 heading and blockquote summary, organizes links under H2 sections, describes each link, and focuses on the 20 to 50 most important pages. The examples above from Stripe, Vercel, and Nuxt demonstrate this well. Start with # Your Project Name, add a > blockquote summary, then create sections with ## headings. Under each section, list your most important pages as - \[Title](url): Description. Or use our free llms.txt generator to crawl your site and create a starting point automatically. llms.txt is a concise index with links to your pages. llms-full.txt embeds the full content of each linked page inline, so AI systems can consume everything in one request. llms.txt is better for navigation and context; llms-full.txt is better for RAG pipelines and deep analysis. Any site that wants to be discoverable by AI systems benefits from llms.txt. It is especially valuable for documentation sites, API references, open source projects, and developer tools. Marketing sites and personal blogs can also benefit if they want AI search engines to cite their content accurately. Most effective llms.txt files have 3 to 8 sections. Common sections include Docs, API Reference, Guides, Tutorials, Examples, and Getting Started. Too many sections dilute the structure; too few make it hard for AI systems to find specific content. Yes. Check the llms.txt directory at llmstxt.org for a growing list of sites that publish llms.txt files. You can also validate any site's llms.txt using our free validator tool. ## Related Tools [**Generator**Generate an llms.txt for your own site.](https://mdream.dev/tools/llms-txt/generator) [**Validator**Check your file against the spec.](https://mdream.dev/tools/llms-txt/validator) [**Insights**Data from 95+ real-world llms.txt files.](https://mdream.dev/insights) --- - **Page:** llms.txt Directory · Browse 95+ AI Sitemaps by Category · mdream - **Source:** https://mdream.dev/llms-txt - **Description:** Searchable llms.txt directory. Browse 95+ AI sitemaps organized by category: frameworks, platforms, APIs, docs tools, and more. Inspect how Vue, Nuxt, Stripe, Cloudflare, and Anthropic structure their files. [Home](https://mdream.dev/) # **llms.txt Directory ** Browse 95+ real llms.txt files grouped by category: frameworks, platforms, APIs, and docs. Search, filter, and inspect how production sites structure their AI sitemaps. ## Browse by Category ### Frameworks [Vue.js](https://mdream.dev/llms-txt/vue) [Nuxt](https://mdream.dev/llms-txt/nuxt) [Next.js](https://mdream.dev/llms-txt/nextjs) [Svelte](https://mdream.dev/llms-txt/svelte) [SvelteKit](https://mdream.dev/llms-txt/sveltekit) [SolidJS](https://mdream.dev/llms-txt/solidjs) [Analog](https://mdream.dev/llms-txt/analog) [Waku](https://mdream.dev/llms-txt/waku) [Elysia](https://mdream.dev/llms-txt/elysia) [Hono](https://mdream.dev/llms-txt/hono) [Nitro](https://mdream.dev/llms-txt/nitro) [H3](https://mdream.dev/llms-txt/h3) [Expo](https://mdream.dev/llms-txt/expo) [Tauri](https://mdream.dev/llms-txt/tauri) [TanStack](https://mdream.dev/llms-txt/tanstack) [Vendure Documentation](https://mdream.dev/llms-txt/vendure-documentation) [React Native · Learn once, write anywhere](https://mdream.dev/llms-txt/react-native-learn-once-write-anywhere) [fast-check Documentation](https://mdream.dev/llms-txt/fast-check-documentation) [reveal.js Documentation](https://mdream.dev/llms-txt/reveal-js-documentation) [Javalin](https://mdream.dev/llms-txt/javalin) [Midway Framework Documentation](https://mdream.dev/llms-txt/midway) [Fastify](https://mdream.dev/llms-txt/fastify) [Engineering Productivity Software for Delivery Leaders | Middleware](https://mdream.dev/llms-txt/engineering-productivity-software-for-delivery-leaders-middleware) [Fiber](https://mdream.dev/llms-txt/gofiber) [Laravel Herd](https://mdream.dev/llms-txt/laravel-herd) [Remix - Ethereum IDE](https://mdream.dev/llms-txt/remix-ide) [Laravel Spark](https://mdream.dev/llms-txt/laravel-spark) [FastHTML](https://mdream.dev/llms-txt/fasthtml) [FastAPI MCP](https://mdream.dev/llms-txt/fastapi-mcp) [Flutter documentation](https://mdream.dev/llms-txt/flutter-documentation) [Semaphore: The best CI/CD tool for high-performance engineering teams.](https://mdream.dev/llms-txt/semaphore-the-best-ci-cd-tool-for-high-performance-engineering-teams) [LITIENGINE 🎮: The pure 2D Java Game Engine](https://mdream.dev/llms-txt/litiengine-the-pure-2d-java-game-engine) [LoginRadius](https://mdream.dev/llms-txt/loginradius) ### Platforms [SST](https://mdream.dev/llms-txt/sst) [Vercel](https://mdream.dev/llms-txt/vercel) [Stripe](https://mdream.dev/llms-txt/stripe) [Cloudflare](https://mdream.dev/llms-txt/cloudflare) [Cloudflare Workers](https://mdream.dev/llms-txt/cloudflare-workers) [Netlify](https://mdream.dev/llms-txt/netlify) [Railway](https://mdream.dev/llms-txt/railway) [Render](https://mdream.dev/llms-txt/render) [AWS](https://mdream.dev/llms-txt/aws) [Resend](https://mdream.dev/llms-txt/resend) [LaunchDarkly | Documentation](https://mdream.dev/llms-txt/launchdarkly) [Twilio Documentation](https://mdream.dev/llms-txt/twilio) [Add a New Provider](https://mdream.dev/llms-txt/novu) [Appwrite](https://mdream.dev/llms-txt/appwrite) [Knock Documentation](https://mdream.dev/llms-txt/knock) [DigitalOcean Documentation](https://mdream.dev/llms-txt/digitalocean) [Coolify](https://mdream.dev/llms-txt/coolify) [Azure](https://mdream.dev/llms-txt/azure) [Pinata Docs](https://mdream.dev/llms-txt/pinata-docs) [TutorialEdge.net](https://mdream.dev/llms-txt/tutorialedge-net) [FeatBit](https://mdream.dev/llms-txt/featbit) [Heroku: Cloud Application Platform](https://mdream.dev/llms-txt/heroku-cloud-application-platform) [CircleCI](https://mdream.dev/llms-txt/circleci) [Aptabase](https://mdream.dev/llms-txt/aptabase) [OctoBot](https://mdream.dev/llms-txt/octobot) [RepoFlow](https://mdream.dev/llms-txt/repoflow) [Azul | The Java Platform for the Modern Cloud Enterprise](https://mdream.dev/llms-txt/azul-the-java-platform-for-the-modern-cloud-enterprise) [Product Home – DigitalOcean Documentation](https://mdream.dev/llms-txt/product-home-digitalocean-documentation) ### Vue Ecosystem [Vue Router](https://mdream.dev/llms-txt/vue-router) [Nuxt UI](https://mdream.dev/llms-txt/nuxt-ui) [Nuxt Content](https://mdream.dev/llms-txt/nuxt-content) [Nuxt Image](https://mdream.dev/llms-txt/nuxt-image) [Nuxt Scripts](https://mdream.dev/llms-txt/nuxt-scripts) [Nuxt i18n](https://mdream.dev/llms-txt/nuxt-i18n) [Nuxt SEO](https://mdream.dev/llms-txt/nuxt-seo) [NuxtHub](https://mdream.dev/llms-txt/nuxthub) [Vuetify](https://mdream.dev/llms-txt/vuetify) ### UI Libraries [shadcn-vue](https://mdream.dev/llms-txt/shadcn-vue) [shadcn/ui](https://mdream.dev/llms-txt/shadcn-ui) [Chakra UI](https://mdream.dev/llms-txt/chakra-ui) [Mantine](https://mdream.dev/llms-txt/mantine) [Ark UI](https://mdream.dev/llms-txt/ark-ui) [daisyUI](https://mdream.dev/llms-txt/daisyui) [React Spectrum](https://mdream.dev/llms-txt/react-spectrum) [Panda CSS](https://mdream.dev/llms-txt/panda-css) [Remotion](https://mdream.dev/llms-txt/remotion) [UnoCSS](https://mdream.dev/llms-txt/unocss) [Ant Design - Enterprise-class React UI library](https://mdream.dev/llms-txt/ant-design) [Magic UI](https://mdream.dev/llms-txt/magicui) [MDBootstrap](https://mdream.dev/llms-txt/material-design-for-bootstrap) [ButterCMS – The Fast, Flexible Headless CMS for Teams That Need to Move](https://mdream.dev/llms-txt/buttercms-the-fast-flexible-headless-cms-for-teams-that-need-to-move) [Syncfusion: Enterprise-Grade AI-Ready UI Components, Document SDKs, and Feature-rich Document Viewers/Editors](https://mdream.dev/llms-txt/syncfusion-enterprise-grade-ai-ready-ui-components-document-sdks-and-feature-rich-document-viewers-editors) [LiveCodes](https://mdream.dev/llms-txt/livecodes) [LabEx](https://mdream.dev/llms-txt/labex) [SQLPage](https://mdream.dev/llms-txt/sqlpage) [ThemeSelection: Our vision is to empower designers \& developers with meticulously crafted dashboard templates and UI kits, accelerating their development journey while maximizing efficiency and innovation.](https://mdream.dev/llms-txt/themeselection-our-vision-is-to-empower-designers-amp-developers-with-meticulously-crafted-dashboard-templates-and-ui-kits-accelerating-their-development-journey-while-maximizing-efficiency-and-innovation) [ng-zorro-antd - Ant Design of Angular](https://mdream.dev/llms-txt/ng-zorro-antd-ant-design-of-angular) [WooCommerce: The most customizable ecommerce platform for building your online business. Get started today for free.](https://mdream.dev/llms-txt/woocommerce-the-most-customizable-ecommerce-platform-for-building-your-online-business-get-started-today-for-free) [Skip](https://mdream.dev/llms-txt/skip) [CKEditor - The Only Rich Text Editor Your Application Will Ever Need](https://mdream.dev/llms-txt/ckeditor-the-only-rich-text-editor-your-application-will-ever-need) [Lark Developer](https://mdream.dev/llms-txt/lark-developer) [Shelf](https://mdream.dev/llms-txt/shelf) [Marmite Documentation](https://mdream.dev/llms-txt/marmite-documentation) [Apache Camel](https://mdream.dev/llms-txt/apache-camel) [Apache Superset](https://mdream.dev/llms-txt/apache-superset) [Quill](https://mdream.dev/llms-txt/quill) [Tally – The simplest way to build forms, for free](https://mdream.dev/llms-txt/tally-the-simplest-way-to-build-forms-for-free) [Domoticz](https://mdream.dev/llms-txt/domoticz) [micro1](https://mdream.dev/llms-txt/micro1) [Meetup](https://mdream.dev/llms-txt/meetup) [Svix (www.svix.com)](https://mdream.dev/llms-txt/svix-www-svix-com) [Manifest](https://mdream.dev/llms-txt/manifest) ### Build Tools & Runtimes [Vite](https://mdream.dev/llms-txt/vite) [Turborepo](https://mdream.dev/llms-txt/turborepo) [Rspack](https://mdream.dev/llms-txt/rspack) [Deno](https://mdream.dev/llms-txt/deno) [Prettier](https://mdream.dev/llms-txt/prettier) [Bun](https://mdream.dev/llms-txt/bun) [The JavaScript Oxidation Compiler](https://mdream.dev/llms-txt/oxc) [Nx](https://mdream.dev/llms-txt/nx) [Rolldown](https://mdream.dev/llms-txt/rolldown) ### Developer Tools [Vitest](https://mdream.dev/llms-txt/vitest) [Docker](https://mdream.dev/llms-txt/docker) [Cursor](https://mdream.dev/llms-txt/cursor) [Linear](https://mdream.dev/llms-txt/linear) [Trigger.dev](https://mdream.dev/llms-txt/trigger-dev) [Inngest](https://mdream.dev/llms-txt/inngest) [Cal.com](https://mdream.dev/llms-txt/cal-com) [Grafbase](https://mdream.dev/llms-txt/grafbase) [GitHub Docs](https://mdream.dev/llms-txt/github-docs) [OpenClaw 从入门到精通 | OpenClaw Guide](https://mdream.dev/llms-txt/docker-practice) [Zed](https://mdream.dev/llms-txt/zed) [Pulumi](https://mdream.dev/llms-txt/pulumi) [GitHub](https://mdream.dev/llms-txt/github) [Spatie](https://mdream.dev/llms-txt/spatie) [ToolsHref](https://mdream.dev/llms-txt/toolshref) [BulkPicTools - Professional Image Toolset](https://mdream.dev/llms-txt/bulkpictools-professional-image-toolset) [Tuleap](https://mdream.dev/llms-txt/tuleap) [WireMock](https://mdream.dev/llms-txt/wiremock) ### Libraries [tRPC](https://mdream.dev/llms-txt/trpc) [Zod](https://mdream.dev/llms-txt/zod) [Valibot](https://mdream.dev/llms-txt/valibot) [Zustand](https://mdream.dev/llms-txt/zustand) [SWR](https://mdream.dev/llms-txt/swr) [Apollo GraphQL](https://mdream.dev/llms-txt/apollo-graphql) [unstorage](https://mdream.dev/llms-txt/unstorage) [Wagmi](https://mdream.dev/llms-txt/wagmi) [Tiptap](https://mdream.dev/llms-txt/tiptap) [NativeWind](https://mdream.dev/llms-txt/nativewind) [Mastra](https://mdream.dev/llms-txt/mastra) [Three.js](https://mdream.dev/llms-txt/three-js) [Snowflake Documentation](https://mdream.dev/llms-txt/snowflake) [PraisonAI](https://mdream.dev/llms-txt/praisonai) [Modular](https://mdream.dev/llms-txt/modular) [Agno](https://mdream.dev/llms-txt/agno) [LearnOpenCV](https://mdream.dev/llms-txt/learnopencv) [Infisical](https://mdream.dev/llms-txt/infisical) [NocoDB](https://mdream.dev/llms-txt/nocodb) [learning](https://mdream.dev/llms-txt/learning) [Documentation](https://mdream.dev/llms-txt/chat2db) [Daytona Documentation v0.0.0-dev](https://mdream.dev/llms-txt/daytona) [Gemini CLI Documentation](https://mdream.dev/llms-txt/gemini-cli) [FiftyOne documentation](https://mdream.dev/llms-txt/fiftyone) [Ri Xu Online](https://mdream.dev/llms-txt/excelize) [Kilo Code Documentation](https://mdream.dev/llms-txt/kilocode) [Apify Documentation](https://mdream.dev/llms-txt/apify) [Prefect](https://mdream.dev/llms-txt/prefect) [EventCatalog](https://mdream.dev/llms-txt/eventcatalog) [Tamagui Complete Documentation](https://mdream.dev/llms-txt/tamagui) [Baeldung on Ops](https://mdream.dev/llms-txt/baeldung-on-ops) [openmediavault](https://mdream.dev/llms-txt/openmediavault) [Nylas | Powerful Communication APIs](https://mdream.dev/llms-txt/nylas-mail) [Hurl Documentation](https://mdream.dev/llms-txt/hurl) [Dagster Docs](https://mdream.dev/llms-txt/dagster) [Weights & Biases Documentation](https://mdream.dev/llms-txt/wandb) [Awesome Python](https://mdream.dev/llms-txt/awesome-python) [Unsloth Documentation](https://mdream.dev/llms-txt/unsloth) [Checkly Docs](https://mdream.dev/llms-txt/puppeteer-examples) [MCP Toolbox for Databases - Documentation Index](https://mdream.dev/llms-txt/genai-toolbox) [Opik Documentation](https://mdream.dev/llms-txt/opik) [LiveKit docs](https://mdream.dev/llms-txt/livekit) [Kotlin Documentation](https://mdream.dev/llms-txt/dokka) [--banner:js hack from https://github.com/evanw/esbuild/pull/2067](https://mdream.dev/llms-txt/middy) [Temporal Platform Documentation](https://mdream.dev/llms-txt/temporal) [HTTP/3 explained](https://mdream.dev/llms-txt/http3-explained) [Meilisearch Documentation](https://mdream.dev/llms-txt/meilisearch) [GSAP Documentation](https://mdream.dev/llms-txt/gsap) [Inspira UI](https://mdream.dev/llms-txt/inspira-ui) [cohere](https://mdream.dev/llms-txt/cohere) [Tinybird](https://mdream.dev/llms-txt/tinybird) [Lago](https://mdream.dev/llms-txt/lago) [Visual Studio Code](https://mdream.dev/llms-txt/vscode) [Humanloop Docs](https://mdream.dev/llms-txt/humanloop) [Pinecone Docs](https://mdream.dev/llms-txt/pinecone) [Courier](https://mdream.dev/llms-txt/courier) [Promptfoo](https://mdream.dev/llms-txt/promptfoo) [Klavis AI](https://mdream.dev/llms-txt/klavis) [Baseten](https://mdream.dev/llms-txt/baseten) [Aptible](https://mdream.dev/llms-txt/aptible) [Fireworks AI Docs](https://mdream.dev/llms-txt/fireworks) [Ably Documentation](https://mdream.dev/llms-txt/ably) [Developer Hub](https://mdream.dev/llms-txt/developer-hub) [ProjectDiscovery Documentation](https://mdream.dev/llms-txt/nuclei) [Aceternity UI Documentation](https://mdream.dev/llms-txt/ui) [New API Docs (en)](https://mdream.dev/llms-txt/new-api) [LimeSurvey](https://mdream.dev/llms-txt/limesurvey) [tldraw SDK](https://mdream.dev/llms-txt/tldraw) [Modern.js](https://mdream.dev/llms-txt/modern-js) [Dify Docs](https://mdream.dev/llms-txt/dify) [Milvus](https://mdream.dev/llms-txt/milvus) [Dub](https://mdream.dev/llms-txt/dub) [Taiga UI - Angular Components Library](https://mdream.dev/llms-txt/taiga-ui) [Stedi](https://mdream.dev/llms-txt/stedi) [Together AI Docs](https://mdream.dev/llms-txt/together) [Authelia](https://mdream.dev/llms-txt/authelia) [smartcar](https://mdream.dev/llms-txt/smartcar) [HeroUI v3 Documentation](https://mdream.dev/llms-txt/heroui) [Overview](https://mdream.dev/llms-txt/zag) [Helicone OSS LLM Observability](https://mdream.dev/llms-txt/helicone) [Labelbox](https://mdream.dev/llms-txt/labelbox) [Activepieces](https://mdream.dev/llms-txt/activepieces) [Scanopy Documentation](https://mdream.dev/llms-txt/scanopy) [Kaneo](https://mdream.dev/llms-txt/kaneo) [E2B Docs](https://mdream.dev/llms-txt/e2b) [Databricks Documentation](https://mdream.dev/llms-txt/databricks) [AutoGPT](https://mdream.dev/llms-txt/autogpt) [Modal llms.txt](https://mdream.dev/llms-txt/modal) [Better Auth](https://mdream.dev/llms-txt/better-auth) [Juspay Hyperswitch](https://mdream.dev/llms-txt/hyperswitch) [Pydantic AI](https://mdream.dev/llms-txt/pydantic-ai) [Filament](https://mdream.dev/llms-txt/filament) [Ragas](https://mdream.dev/llms-txt/ragas) [Firecrawl Docs](https://mdream.dev/llms-txt/firecrawl) [Puck](https://mdream.dev/llms-txt/puck) [Chroma Docs](https://mdream.dev/llms-txt/trychroma) [React Bits](https://mdream.dev/llms-txt/react-bits) [Fabric](https://mdream.dev/llms-txt/fabric) [Loops](https://mdream.dev/llms-txt/loops) [DSPy](https://mdream.dev/llms-txt/dspy) [Preact Documentation](https://mdream.dev/llms-txt/preactjs) [FalkorDB](https://mdream.dev/llms-txt/falkordb) [Easy!Appointments](https://mdream.dev/llms-txt/easy-appointments) [OpenAI API docs](https://mdream.dev/llms-txt/platform) [Browserless.io Documentation](https://mdream.dev/llms-txt/browserless) [Egg](https://mdream.dev/llms-txt/egg) [Split docs Documentation](https://mdream.dev/llms-txt/split) [Elastic Documentation](https://mdream.dev/llms-txt/elastic) [Gogs: A painless self-hosted Git service](https://mdream.dev/llms-txt/gogs) [Font Awesome Docs](https://mdream.dev/llms-txt/font-awesome) [Convoy](https://mdream.dev/llms-txt/convoy) [AST-GREP](https://mdream.dev/llms-txt/ast-grep) [Plane](https://mdream.dev/llms-txt/plane) [XState](https://mdream.dev/llms-txt/xstate) [Chainlit](https://mdream.dev/llms-txt/chainlit) [Finch](https://mdream.dev/llms-txt/finch) [Perplexity](https://mdream.dev/llms-txt/perplexity) [Model Context Protocol](https://mdream.dev/llms-txt/servers) [http://docs.copilotkit.ai llms.txt](https://mdream.dev/llms-txt/copilotkit) [Element Plus - A Vue 3 based component library](https://mdream.dev/llms-txt/element-plus) [run every day at 8am](https://mdream.dev/llms-txt/run-every-day-at-8am) [About TagSpaces](https://mdream.dev/llms-txt/about-tagspaces) [Angular](https://mdream.dev/llms-txt/angular) [VoltAgent: The Comprehensive Developer & LLM Guide (v1.0.x)](https://mdream.dev/llms-txt/voltagent) [Shuttle Docs](https://mdream.dev/llms-txt/shuttle) [Slidev](https://mdream.dev/llms-txt/slidev) [singlestore](https://mdream.dev/llms-txt/singlestore) [semantic-release](https://mdream.dev/llms-txt/semantic-release) [Trellis Doc](https://mdream.dev/llms-txt/trellis) [飞书开放平台](https://mdream.dev/llms-txt/feishu-open-platform) [Letta](https://mdream.dev/llms-txt/letta) [Goody](https://mdream.dev/llms-txt/goody) [MistralAI](https://mdream.dev/llms-txt/mistral) [33 JavaScript Concepts](https://mdream.dev/llms-txt/33-js-concepts) [Memori](https://mdream.dev/llms-txt/memori) [Easy Dataset](https://mdream.dev/llms-txt/easy-dataset) [LlamaIndex Documentation](https://mdream.dev/llms-txt/llama-index) [https://qdrant.tech/ llms.txt](https://mdream.dev/llms-txt/qdrant) [Vibe Kanban](https://mdream.dev/llms-txt/vibe-kanban) [Context7 MCP](https://mdream.dev/llms-txt/context7) [iii](https://mdream.dev/llms-txt/iii) [Medama Analytics](https://mdream.dev/llms-txt/medama-analytics) [Onyx AI](https://mdream.dev/llms-txt/onyx) [nut.js](https://mdream.dev/llms-txt/nut-js) [每日一荐](https://mdream.dev/llms-txt/leetcode) [https://docs.litellm.ai/ llms.txt](https://mdream.dev/llms-txt/litellm) [AgentOps](https://mdream.dev/llms-txt/agentops) [TOON](https://mdream.dev/llms-txt/toon) [PixiJS Documentation for LLMs](https://mdream.dev/llms-txt/pixijs) [SpacetimeDB](https://mdream.dev/llms-txt/spacetimedb) [PixiJS Documentation for LLMs](https://mdream.dev/llms-txt/pixijs-documentation-for-llms) [amCharts 5](https://mdream.dev/llms-txt/amcharts-5) [Liam ERD](https://mdream.dev/llms-txt/liam) [Durable Execution Solutions | Temporal](https://mdream.dev/llms-txt/durable-execution-solutions-temporal) [21st.dev (https://21st.dev)](https://mdream.dev/llms-txt/21st) [Architecture](https://mdream.dev/llms-txt/chatbot) [Load testing designed for DevOps and CI/CD | Gatling](https://mdream.dev/llms-txt/load-testing-designed-for-devops-and-ci-cd-gatling) [Framer](https://mdream.dev/llms-txt/framer) [Aipolabs agent-computer interface (ACI)](https://mdream.dev/llms-txt/aci) [Super Productivity](https://mdream.dev/llms-txt/super-productivity) [DeepWiki-Open](https://mdream.dev/llms-txt/deepwiki-open) [Midscene - Vision-Driven UI Automation](https://mdream.dev/llms-txt/midscene) [Vue Macros](https://mdream.dev/llms-txt/vue-macros) [LobeHub](https://mdream.dev/llms-txt/lobehub) [Traefik Labs is the Leader in Cloud-Native, GitOps-Driven AI and API Management](https://mdream.dev/llms-txt/traefik) [Apache Superset](https://mdream.dev/llms-txt/superset) [World Monitor](https://mdream.dev/llms-txt/worldmonitor) [Seth Hobson](https://mdream.dev/llms-txt/agents) [llms.txt](https://mdream.dev/llms-txt/scrapegraph-ai) [botpress](https://mdream.dev/llms-txt/botpress) [Claude](https://mdream.dev/llms-txt/claude) [Teleport: The easiest, most secure way to access and protect all your infrastructure](https://mdream.dev/llms-txt/teleport) [Netdata](https://mdream.dev/llms-txt/netdata) [nbdev](https://mdream.dev/llms-txt/nbdev) [algolia](https://mdream.dev/llms-txt/algolia) [prompt-kit](https://mdream.dev/llms-txt/prompt-kit) [Kitchen Stories](https://mdream.dev/llms-txt/kitchen-stories) [Mistral AI](https://mdream.dev/llms-txt/mistral-inference) [BeeAI Framework](https://mdream.dev/llms-txt/beeai-framework) [Dittofeed (dittofeed.com)](https://mdream.dev/llms-txt/dittofeed-dittofeed-com) [Repomix](https://mdream.dev/llms-txt/repomix) [Plotly](https://mdream.dev/llms-txt/plotly) [devenv](https://mdream.dev/llms-txt/devenv) [Campsite](https://mdream.dev/llms-txt/campsite) [Langflow Documentation](https://mdream.dev/llms-txt/langflow) [Skyvern](https://mdream.dev/llms-txt/skyvern) [Reflex](https://mdream.dev/llms-txt/reflex) [App Store Connect CLI (\`asc\`)](https://mdream.dev/llms-txt/app-store-connect-cli) [Portkey](https://mdream.dev/llms-txt/gateway) [BrowserOS](https://mdream.dev/llms-txt/browseros) [Vector search](https://mdream.dev/llms-txt/weaviate) [Doks](https://mdream.dev/llms-txt/doks) [Alex Tselegidis](https://mdream.dev/llms-txt/alex-tselegidis) [Documentation](https://mdream.dev/llms-txt/gopeed) [Poimandres](https://mdream.dev/llms-txt/pmnd) [DVC: Data Version Control](https://mdream.dev/llms-txt/dvc) [Haystack](https://mdream.dev/llms-txt/haystack) [Wasmer (https://wasmer.io)](https://mdream.dev/llms-txt/wasmer) [llms.txt for \[https://www.runanywhere.ai/\](https://www.runanywhere.ai/)](https://mdream.dev/llms-txt/runanywhere-sdks) [Chroma](https://mdream.dev/llms-txt/chroma) [Stagehand: AI Browser Automation Framework](https://mdream.dev/llms-txt/stagehand) [Sim](https://mdream.dev/llms-txt/sim) [ArrowJS](https://mdream.dev/llms-txt/arrow-js) [Talk Python to Me](https://mdream.dev/llms-txt/talk-python-to-me) [OpenBB](https://mdream.dev/llms-txt/openbb) [Ollama](https://mdream.dev/llms-txt/ollama) [9GAG](https://mdream.dev/llms-txt/9gag) [kubefwd](https://mdream.dev/llms-txt/kubefwd) [Atoms](https://mdream.dev/llms-txt/metagpt) [screenpipe](https://mdream.dev/llms-txt/screenpipe) [Memvid](https://mdream.dev/llms-txt/memvid) [GenAIScript](https://mdream.dev/llms-txt/genaiscript) [ngrok](https://mdream.dev/llms-txt/ngrok) [APILayer: APILayer APIs are feature-rich and easy to integrate, offering low latency for an enhanced developer experience.](https://mdream.dev/llms-txt/public-apis) [Sherlock Project](https://mdream.dev/llms-txt/sherlock) [YAFFA: Yet Another Free Financial App](https://mdream.dev/llms-txt/yaffa-yet-another-free-financial-app) [Video.js v10](https://mdream.dev/llms-txt/video-js) [Langfuse](https://mdream.dev/llms-txt/langfuse) [Lo - A Go library for functional programming](https://mdream.dev/llms-txt/lo) [Terragrunt](https://mdream.dev/llms-txt/terragrunt) ### Databases [Kysely](https://mdream.dev/llms-txt/kysely) [Drizzle](https://mdream.dev/llms-txt/drizzle) [Prisma](https://mdream.dev/llms-txt/prisma) [Turso](https://mdream.dev/llms-txt/turso) [Neon](https://mdream.dev/llms-txt/neon) [PlanetScale](https://mdream.dev/llms-txt/planetscale) [Convex](https://mdream.dev/llms-txt/convex) [MongoDB](https://mdream.dev/llms-txt/mongodb) [Redis](https://mdream.dev/llms-txt/redis) [Supabase](https://mdream.dev/llms-txt/supabase) [Upstash](https://mdream.dev/llms-txt/upstash) [Couchbase](https://mdream.dev/llms-txt/couchbase) [ClickHouse](https://mdream.dev/llms-txt/clickhouse) [Convex Documentation](https://mdream.dev/llms-txt/convex-backend) [Tiger Data Documentation](https://mdream.dev/llms-txt/timescale) [TypeORM](https://mdream.dev/llms-txt/typeorm) [YugabyteDB Documentation](https://mdream.dev/llms-txt/yugabyte) [SurrealDB](https://mdream.dev/llms-txt/surrealdb) [DuckDB](https://mdream.dev/llms-txt/duckdb) [Xata](https://mdream.dev/llms-txt/xata) [InfluxData Documentation](https://mdream.dev/llms-txt/influxdata) [NearForm](https://mdream.dev/llms-txt/nearform) [MikroORM](https://mdream.dev/llms-txt/mikroorm) [Formisch](https://mdream.dev/llms-txt/formisch) [Ably Documentation](https://mdream.dev/llms-txt/ably-documentation) [Form.io](https://mdream.dev/llms-txt/form-io) [Documentation - Formbricks](https://mdream.dev/llms-txt/documentation-formbricks) [O’Reilly Media - Technology and Business Training](https://mdream.dev/llms-txt/o-reilly-media-technology-and-business-training) [HitKeep](https://mdream.dev/llms-txt/hitkeep) [Onetime Secret](https://mdream.dev/llms-txt/onetime-secret) ### CMS [Payload](https://mdream.dev/llms-txt/payload) [Sanity](https://mdream.dev/llms-txt/sanity) [Storyblok](https://mdream.dev/llms-txt/storyblok) [Directus Documentation](https://mdream.dev/llms-txt/directus) [Ghost Developer Docs](https://mdream.dev/llms-txt/ghost) [Strapi Documentation](https://mdream.dev/llms-txt/strapi) [TinaCMS](https://mdream.dev/llms-txt/tina) [The Road To Enterprise](https://mdream.dev/llms-txt/the-road-to-enterprise) [Ghost Developer Docs](https://mdream.dev/llms-txt/ghost-developer-docs) ### Monitoring [PostHog](https://mdream.dev/llms-txt/posthog) [What is Datadog?](https://mdream.dev/llms-txt/datadoghq) [LogTape](https://mdream.dev/llms-txt/logtape) [Honeybadger](https://mdream.dev/llms-txt/honeybadger) [Axiom Docs](https://mdream.dev/llms-txt/axiom-docs) ### Documentation Tools [Fumadocs](https://mdream.dev/llms-txt/fumadocs) [Mintlify](https://mdream.dev/llms-txt/mintlify) [GitBook Documentation](https://mdream.dev/llms-txt/gitbook) [ReadMe Documentation](https://mdream.dev/llms-txt/readme) [VitePress](https://mdream.dev/llms-txt/vitepress) [Redocly](https://mdream.dev/llms-txt/redocly) [Dynamic Documentation](https://mdream.dev/llms-txt/dynamic-documentation) [Kirby CMS Docs](https://mdream.dev/llms-txt/kirby-cms-docs) [Tyk Documentation](https://mdream.dev/llms-txt/tyk-documentation) [ElevenLabs Documentation](https://mdream.dev/llms-txt/elevenlabs-documentation) [Junction](https://mdream.dev/llms-txt/junction) [Mangopay docs](https://mdream.dev/llms-txt/mangopay-docs) [Meilisearch Documentation](https://mdream.dev/llms-txt/meilisearch-documentation) [Velt](https://mdream.dev/llms-txt/velt) [Unstructured](https://mdream.dev/llms-txt/unstructured) [Conductor Docs](https://mdream.dev/llms-txt/conductor-docs) [Zapier](https://mdream.dev/llms-txt/zapier) [Hyperline](https://mdream.dev/llms-txt/hyperline) [Method API Docs](https://mdream.dev/llms-txt/method-api-docs) [Bytebase Docs](https://mdream.dev/llms-txt/bytebase-docs) [Cosmos Docs](https://mdream.dev/llms-txt/cosmos-docs) [Ecotone - DDD, CQRS, Event Sourcing in PHP](https://mdream.dev/llms-txt/ecotone-ddd-cqrs-event-sourcing-in-php) [Inkeep](https://mdream.dev/llms-txt/inkeep) [Intuned](https://mdream.dev/llms-txt/intuned) [Datafold](https://mdream.dev/llms-txt/datafold) [Windsurf Docs](https://mdream.dev/llms-txt/windsurf-docs) [OpenCage Documentation](https://mdream.dev/llms-txt/opencage-documentation) [Flatfile Developer Docs](https://mdream.dev/llms-txt/flatfile-developer-docs) [Unify](https://mdream.dev/llms-txt/unify) [IonQ Quantum Cloud Documentation](https://mdream.dev/llms-txt/ionq-quantum-cloud-documentation) [Frigade Engage](https://mdream.dev/llms-txt/frigade-engage) [LuxAlgo](https://mdream.dev/llms-txt/luxalgo) [BaseHub](https://mdream.dev/llms-txt/basehub) [ScreenshotOne.com Documentation](https://mdream.dev/llms-txt/screenshotone-com-documentation) [Salesbricks](https://mdream.dev/llms-txt/salesbricks) [Aporia](https://mdream.dev/llms-txt/aporia) [Axle](https://mdream.dev/llms-txt/axle) [Mesh](https://mdream.dev/llms-txt/mesh) [Pomerium Documentation](https://mdream.dev/llms-txt/pomerium-documentation) [Metabase Documentation](https://mdream.dev/llms-txt/metabase-documentation) [Cryptomator Documentation](https://mdream.dev/llms-txt/cryptomator-documentation) [Coinbase Developer Documentation](https://mdream.dev/llms-txt/coinbase-developer-documentation) [Quo Docs](https://mdream.dev/llms-txt/quo-docs) [Speedtest Tracker](https://mdream.dev/llms-txt/speedtest-tracker) [Server Side Up](https://mdream.dev/llms-txt/server-side-up) [Primev Documentation](https://mdream.dev/llms-txt/primev-documentation) [API Documentation](https://mdream.dev/llms-txt/api-documentation) [Envoyer](https://mdream.dev/llms-txt/envoyer) [Taleshape Documentation](https://mdream.dev/llms-txt/taleshape-documentation) [Qt Documentation](https://mdream.dev/llms-txt/qt-documentation) ### AI [LangChain](https://mdream.dev/llms-txt/langchain) [https://www.sardine.ai llms.txt](https://mdream.dev/llms-txt/https-www-sardine-ai-llms-txt) [Mailchimp - Complete Web Content Analysis](https://mdream.dev/llms-txt/mailchimp-complete-web-content-analysis) [Monterail](https://mdream.dev/llms-txt/monterail) [Livestorm | All-in-one Webinar Software for Marketing Teams](https://mdream.dev/llms-txt/livestorm-all-in-one-webinar-software-for-marketing-teams) [Alpaca Main Page](https://mdream.dev/llms-txt/alpaca-main-page) [Tinybird LLM Agent Sitemap](https://mdream.dev/llms-txt/tinybird-llm-agent-sitemap) [FlowFuse](https://mdream.dev/llms-txt/flowfuse) [Chatwoot: AI-powered, open-source customer support platform. Self-host or cloud. Alternative to Intercom & Zendesk. - llms.txt](https://mdream.dev/llms-txt/chatwoot-ai-powered-open-source-customer-support-platform-self-host-or-cloud-alternative-to-intercom-zendesk-llms-txt) [Docs](https://mdream.dev/llms-txt/docs) [Writer AI Studio](https://mdream.dev/llms-txt/writer-ai-studio) [CrewAI](https://mdream.dev/llms-txt/crewai) [Galileo](https://mdream.dev/llms-txt/galileo) [AI Squared](https://mdream.dev/llms-txt/ai-squared) [Intlayer Documentation for Agents](https://mdream.dev/llms-txt/intlayer-documentation-for-agents) [codewithmukesh](https://mdream.dev/llms-txt/codewithmukesh) [Sophos: Defeat Cyberattacks with AI-Native Security](https://mdream.dev/llms-txt/sophos-defeat-cyberattacks-with-ai-native-security) [Embedchain](https://mdream.dev/llms-txt/embedchain) [ProjectDiscovery llms.txt](https://mdream.dev/llms-txt/projectdiscovery-llms-txt) [sitespeak.ai llms.txt](https://mdream.dev/llms-txt/sitespeak-ai-llms-txt) [Plain Docs](https://mdream.dev/llms-txt/plain-docs) [Argil](https://mdream.dev/llms-txt/argil) [Hello world](https://mdream.dev/llms-txt/hello-world) [https://tolgee.io llms.txt](https://mdream.dev/llms-txt/https-tolgee-io-llms-txt) [Postmark](https://mdream.dev/llms-txt/postmark) [OpenPipe](https://mdream.dev/llms-txt/openpipe) [Best Password Manager for Business, Enterprise & Personal | Bitwarden](https://mdream.dev/llms-txt/best-password-manager-for-business-enterprise-personal-bitwarden) [Leantime: Project Management for Non-project Managers](https://mdream.dev/llms-txt/leantime-project-management-for-non-project-managers) [Papra](https://mdream.dev/llms-txt/papra) [Operational.co docs for LLMs](https://mdream.dev/llms-txt/operational-co-docs-for-llms) [Apache Spark](https://mdream.dev/llms-txt/apache-spark) [https://latte.nette.org/ llms.txt](https://mdream.dev/llms-txt/https-latte-nette-org-llms-txt) [RapidForge - LLM Content Corpus](https://mdream.dev/llms-txt/rapidforge-llm-content-corpus) [WRITER: The enterprise AI platform for agentic work](https://mdream.dev/llms-txt/writer-the-enterprise-ai-platform-for-agentic-work) [WordLift](https://mdream.dev/llms-txt/wordlift) [micro1 API](https://mdream.dev/llms-txt/micro1-api) [Tiledesk: Conversational Automation for Customer Engagement](https://mdream.dev/llms-txt/tiledesk-conversational-automation-for-customer-engagement) [Judge0 - Code execution made simple for every business](https://mdream.dev/llms-txt/judge0-code-execution-made-simple-for-every-business) [Localization That Works For You](https://mdream.dev/llms-txt/localization-that-works-for-you) [Primev](https://mdream.dev/llms-txt/primev) [fastcore](https://mdream.dev/llms-txt/fastcore) [https://nette.org/en/ llms.txt](https://mdream.dev/llms-txt/https-nette-org-en-llms-txt) [Lowdefy](https://mdream.dev/llms-txt/lowdefy) [llms.txt](https://mdream.dev/llms-txt/llmstxt-org) [Answer.AI company website](https://mdream.dev/llms-txt/answer-ai-company-website) [/llms.txt directory](https://mdream.dev/llms-txt/llms-txt-directory) [Slack Platform LLM development guide](https://mdream.dev/llms-txt/slack-platform-llm-development-guide) [gomponents: HTML Components in Pure Go - LLM Documentation](https://mdream.dev/llms-txt/gomponents-html-components-in-pure-go-llm-documentation) ### React Ecosystem [React](https://mdream.dev/llms-txt/react) ### Authentication [Clerk](https://mdream.dev/llms-txt/clerk) [Auth0 blog](https://mdream.dev/llms-txt/auth0) [WorkOS Documentation](https://mdream.dev/llms-txt/workos) ### E-commerce [Medusa](https://mdream.dev/llms-txt/medusa) [Spree Commerce documentation](https://mdream.dev/llms-txt/spree-commerce-documentation) [Saleor Commerce](https://mdream.dev/llms-txt/saleor-commerce) ## Framework Guides [**llms.txt for Next.js**App Router handler, static file, build-time.](https://mdream.dev/tools/llms-txt/nextjs) [**llms.txt for Nuxt**@mdream/nuxt module, Nitro route, or public file.](https://mdream.dev/tools/llms-txt/nuxt) [**llms.txt for WordPress**Plugin, mu-plugin rewrite, or upload to root.](https://mdream.dev/tools/llms-txt/wordpress) [**llms.txt for SvelteKit**+server.ts endpoint, static/, or MDsveX.](https://mdream.dev/tools/llms-txt/sveltekit) [**llms.txt for Astro**API route from content collections, or public/.](https://mdream.dev/tools/llms-txt/astro) [**llms.txt for Remix**Resource route, static, or loader.](https://mdream.dev/tools/llms-txt/remix) ## Related Tools [**Generator**Crawl any site, generate a spec-compliant llms.txt.](https://mdream.dev/tools/llms-txt/generator) [**Validator**Check your llms.txt against the official spec.](https://mdream.dev/tools/llms-txt/validator) [**Guide**What is llms.txt, format rules, and implementation.](https://mdream.dev/llms-txt/guide) [**Examples**Real-world llms.txt files from top projects.](https://mdream.dev/examples) [**Insights**Data analysis of 95+ llms.txt files across the ecosystem.](https://mdream.dev/insights) --- - **Page:** Mdream: HTML to Markdown Converter for LLMs - **Source:** https://mdream.dev/ - **Description:** Free, zero-dependency HTML to Markdown for LLMs. 33x faster than Turndown with 38% fewer tokens. Available for npm, Rust, WASM, and CLI. # **mdream ** Parse \ with fewer tokens. Zero-dependency HTML to Markdown for LLMs; fastest in JS and Rust, streaming, runs everywhere. **23.4kB**gzip **33x**faster than Turndown **38%**fewer tokens [GitHub](https://github.com/harlan-zw/mdream) [ npmx](https://npmx.dev/package/mdream) ## Tools [**llms.txt Explorer**·Browse, validate, and generate llms.txt files.](https://mdream.dev/llms-txt) [**llms.txt Generator**·Crawl any site, generate a spec-compliant llms.txt. Free.](https://mdream.dev/tools/llms-txt/generator) [**llms.txt Validator**·Check your llms.txt against the official spec.](https://mdream.dev/tools/llms-txt/validator) [**llms.txt Guide**·What is llms.txt and how to implement it.](https://mdream.dev/llms-txt/guide) [**Converter Comparison**·Feature and speed comparison of HTML to Markdown converters.](https://mdream.dev/tools/compare) [**Live Benchmarks**·Run HTML-to-Markdown benchmarks in your browser.](https://mdream.dev/tools/benchmarks) [**Token Counter**·Count tokens for ChatGPT, Claude, Gemini, and Llama.](https://mdream.dev/tools/token-counter) [**Accept-header Tester**·Replay agent requests; see what Claude Code gets back.](https://mdream.dev/tools/accept-header) [**AI Visibility Audit**·Check your site for ChatGPT, Claude, Perplexity readiness.](https://mdream.dev/tools/ai-audit) [**Guides**·GEO, AEO, markdown for agents, and AI crawler policy.](https://mdream.dev/guides) ## What is Mdream? A zero-dependency, LLM-optimized HTML to Markdown converter. Faster and leaner than Turndown, node-html-markdown, and html-to-markdown, with output tuned for token efficiency and readability. [

**#1 Fastest**

33x faster than Turndown on 166kB HTML (0.34ms Rust vs 11.26ms Turndown). Pure JS: 3.26ms.](https://mdream.dev/tools/benchmarks) ### **#1 Token Optimizer** 38% fewer o200k\_base tokens than Turndown on a real 166kB Wikipedia fixture. ### **Streaming** Memory efficient streaming for large documents and real-time pipelines. ### **Tiny & Universal** 23.4kB gzip JS core, 101.6kB Rust WASM. Zero dependencies. Runs everywhere. ## Packages [**mdream** Rust NAPI engine + WASM for edge. Includes CLI.](https://github.com/harlan-zw/mdream/tree/main/packages/mdream) [**@mdream/js** Pure JS engine. Full hook access, zero native deps.](https://github.com/harlan-zw/mdream/tree/main/packages/js) [**@mdream/crawl** Site-wide crawler to generate llms.txt artifacts](https://github.com/harlan-zw/mdream/tree/main/packages/crawl) [**@mdream/vite** Auto .md generation for Vite sites](https://github.com/harlan-zw/mdream/tree/main/packages/vite) [**@mdream/nuxt** Automatic .md and llms.txt for Nuxt sites](https://github.com/harlan-zw/mdream/tree/main/packages/nuxt) [**@mdream/action** GitHub Action for .md and llms.txt from static HTML](https://github.com/harlan-zw/mdream/tree/main/packages/action) [**Docker** Pre-built image with Playwright Chrome for crawling](https://github.com/harlan-zw/mdream/blob/main/DOCKER.md) [**mdream (crate)** Native Rust crate with CLI. Zero deps, streaming.](https://github.com/harlan-zw/mdream/tree/main/crates) [**Browser CDN** Use via unpkg/jsDelivr with no build step](https://github.com/harlan-zw/mdream#browser-cdn) Community funded ## **Kept going by the community. ** Mdream stays free and open source because developers and teams using it help fund the work. [Become a sponsor](https://github.com/sponsors/harlan-zw) ### Top sponsors - [**Daniel Roe**](https://roe.dev) - [**Jan-Henrik Damaschke**](https://visorian.com) ### Gold sponsors - [Massive Monster](https://massivemonster.co) - [Localazy](https://localazy.com) ### Backers - [Marko](https://github.com/aussieboi) - [Christian Ducrot](https://www.ducrot.de) - [Maik](https://github.com/just-maik) - [Leopold Kristjansson](https://www.leopold.is) - [Email Testing for Developers - SMTP Labs, Inc.](https://smtp.dev) - [Laioutr GmbH](https://www.laioutr.com) - [Elmo](https://www.elmohq.com) - [AdKit](https://adkit.so) - [AMZ Cart Share](https://amzcartshare.com)