Enables AI agents to crawl and scrape websites, converting HTML to clean Markdown and structured metadata with support for JavaScript rendering, bot evasion, and SSRF protection.
Enables fetching web pages as clean Markdown with automatic escalation from direct requests to residential proxies and headless browsers to overcome blocks, plus built-in politeness handling like robots.txt and rate limiting.
Crawl entire websites, sitemaps, and documentation portals into clean, LLM-ready Markdown without ads, cookie popups, or noise for RAG pipelines and AI agents.