Enables agents to discover the llms.txt index covering any site, fetch its linked documentation pages as markdown, and search across parsed indexes by section heading, link title, and notes.
Enables crawling and extracting clean content from documentation websites with optional LLM-powered analysis for intelligent summaries, code example extraction, and content classification.
Auto-discovers llms.txt documentation from project dependencies and exposes it to AI coding agents via MCP, enabling agents to read first-party docs without scraping or guessing.
Enables AI agents to search, fetch, and clean live documentation from LangChain, LlamaIndex, and OpenAI via Model Context Protocol, with token-optimized extraction.
Enables AI agents to check whether a public website is crawlable, understandable, and ready for AI search workflows through local-only audits of robots.txt, sitemaps, metadata, and llms.txt.