Crawl
crawlDiscover and scrape a whole site as one job. Returns a crawl id straight away; read it with crawl_status. Use this instead of calling scrape in a loop.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The site to start from. | |
| limit | No | Maximum pages to scrape. | |
| maxDepth | No | How far from the seed to follow. |