crawl
Start an asynchronous BFS crawl of a site from a seed URL, converting each page to Markdown. Returns a job id — poll with crawl_status.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Seed URL | |
| depth | No | Max link depth (default 3) | |
| limit | No | Max pages (default 50) | |
| country | No | ISO country code for the proxy exit | |
| exclude | No | URL substrings/globs to exclude | |
| include | No | URL substrings/globs to include | |
| content_mode | No | Per-page content scope: smart (default) | article | full |