spider-cloud-mcp
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@spider-cloud-mcpCrawl https://docs.spider.cloud and limit to 10 pages"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
spider-cloud-mcp
MCP server for the Spider web crawling and scraping API. Crawl, scrape, search, and extract web data for AI agents, RAG pipelines, and LLMs.
Setup
1. Get your API key
Sign up at spider.cloud and get your API key from the API Keys page.
2. Configure your MCP client
Claude Desktop
Add to your claude_desktop_config.json:
{
"mcpServers": {
"spider": {
"command": "npx",
"args": ["-y", "spider-cloud-mcp"],
"env": {
"SPIDER_API_KEY": "your-api-key"
}
}
}
}Claude Code
claude mcp add spider -- npx -y spider-cloud-mcpThen set your API key in the environment or .env file:
SPIDER_API_KEY=your-api-keyCursor
Add to your Cursor MCP settings:
{
"mcpServers": {
"spider": {
"command": "npx",
"args": ["-y", "spider-cloud-mcp"],
"env": {
"SPIDER_API_KEY": "your-api-key"
}
}
}
}Related MCP server: crawl4ai-mcp
Tools
Core Tools
Tool | Description |
| Crawl a website and extract content from multiple pages |
| Scrape a single page (no crawling) |
| Search the web with optional page content fetching |
| Extract all links from a page |
| Capture page screenshots |
| Access bot-protected content with anti-bot bypass |
| Transform HTML to markdown/text/other formats |
| Check your credit balance |
AI Tools (Subscription Required)
These tools require an active AI subscription plan.
Tool | Description |
| AI-guided crawling with natural language prompts |
| Extract structured data using plain English |
| AI-enhanced semantic web search |
| Automate browser interactions with natural language |
| Intelligent link extraction and filtering |
Examples
Crawl a website to markdown
Use spider_crawl to crawl https://example.com with limit 10 and return_format "markdown"Search the web
Use spider_search to search for "latest AI research papers" with fetch_page_content true and num 5AI-powered extraction
Use spider_ai_scrape on https://news.ycombinator.com with prompt "Extract all article titles, URLs, points, and comment counts as structured JSON"Check credits
Use spider_get_credits to check my balanceAPI Reference
Full API documentation: spider.cloud/docs/api
License
MIT
Available Tools
13 toolsspider_ai_browserA
AI-powered browser automation using natural language. Describe actions like 'click login, fill email, submit form' and Spider automates the browser. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to automate | |
| prompt | Yes | Natural language automation instructions (e.g. 'Click the login button, fill in email field, submit') | |
| cookies | No | HTTP cookies | |
| proxy_enabled | No | Enable premium proxies | |
| return_format | No | Output format |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It discloses a critical prerequisite ('REQUIRES an active AI subscription plan') but does not discuss operational behaviors such as navigation side effects, timeouts, or failure outcomes. This is a moderate amount of transparency for a nontrivial automation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, with each sentence serving a distinct purpose: stating the core function, giving a usage example, and flagging the subscription requirement. It is front-loaded and contains no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is complex (browser automation with natural language), yet the description omits details about expected output, rate limits, and how it compares to sibling tools. The subscription requirement and purpose are present, but the overall guidance is minimal for a tool with this scope.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides descriptions for all five parameters, achieving 100% schema_description_coverage. The description adds little beyond an example prompt, so the schema already carries the parameter meaning. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'AI-powered browser automation using natural language' and provides concrete examples ('click login, fill email, submit form'). This specific verb+resource distinguishes it from sibling tools like spider_scrape or spider_crawl, which focus on data extraction rather than interaction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear usage context by showing how to phrase actions ('Describe actions like...'), implying this tool is for interactive tasks rather than retrieval. However, it does not explicitly mention alternatives or when not to use it, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_ai_crawlA
AI-guided crawling using natural language prompts. Describe what content to find and Spider's AI will guide the crawl. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to crawl | |
| limit | No | Maximum pages to crawl | |
| prompt | Yes | Natural language prompt to guide the crawl (e.g. 'Find all product pages and extract pricing') | |
| cookies | No | HTTP cookies | |
| request | No | Request type | |
| proxy_enabled | No | Enable premium proxies | |
| return_format | No | Output format |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behaviors. It discloses a key operational constraint—the mandatory AI subscription plan—which is not visible in the schema or annotations. It also states that the AI 'will guide the crawl,' signaling autonomous behavior. Yet it could elaborate on what happens without the plan or how AI decisions affect outcomes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise: two sentences and a URL. The core action and differentiator appear in the first sentence, with the subscription requirement in the second. Every word earns its place without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters, no output schema, and AI-driven behavior, the description is adequate but sparse. It covers the primary capability and the subscription prerequisite, but does not explain what a successful response looks like, how the prompt guides the crawl in practice, or any error/edge-case behavior. More context would improve operational confidence.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters clearly. The description adds a modest reinforcement of the 'prompt' role ('Describe what content to find and Spider's AI will guide the crawl'), but does not provide significant new parameter-level meaning beyond what the schema offers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'crawling' with a clear resource ('AI-guided') and method ('using natural language prompts'). It immediately distinguishes itself from the sibling tool 'spider_crawl' by emphasizing AI guidance and prompt-driven content discovery.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It clearly establishes the usage context: when you need AI-guided crawling based on natural language descriptions. The explicit 'REQUIRES an active AI subscription plan' provides a critical prerequisite, implying that without such a plan this tool should not be used. However, it does not explicitly compare against alternatives like spider_crawl or state when to choose one over the other.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_ai_linksA
AI-powered intelligent link extraction and filtering. Describe what links you want and Spider uses AI to find and categorize them. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to extract links from | |
| limit | No | Maximum links | |
| prompt | Yes | Natural language link filter (e.g. 'Find all product pages and documentation links') | |
| request | No | Request type | |
| return_format | No | Output format |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the AI requirement and the AI-driven behavior ('uses AI to find and categorize them'), but does not mention potential side effects, rate limits, or whether the operation is read-only. Given it's a link extraction tool, the lack of explicit safety disclosure is a minor gap, resulting in a 3.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded: three short sentences cover purpose, usage, and a critical requirement. There is no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description provides enough context for an agent to understand the tool's purpose, usage, and a key constraint (AI subscription). It also hints at the output by saying 'find and categorize them'. With all parameters documented in the schema and no output schema, this is sufficient, though it does not explain return formats in detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters have schema descriptions, so the baseline is 3. The description's phrase 'Describe what links you want' adds a small amount of context for the 'prompt' parameter, but it does not elaborate on other parameters like 'limit' or 'return_format'. Schema coverage is 100%, so no further compensation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with specific verbs ('extraction and filtering') and describes the AI-powered approach. It distinguishes from sibling tools like spider_links by emphasizing AI and natural language prompts ('Describe what links you want').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear usage context: users describe what links they want. It also states a prerequisite (active AI subscription plan). However, it does not explicitly name alternative tools (e.g., spider_links) or exclusions, so it earns a 4 rather than a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_ai_scrapeA
AI-powered structured data extraction using plain English. Describe what data you want and get structured JSON back — no CSS selectors needed. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to scrape | |
| prompt | Yes | Natural language extraction prompt (e.g. 'Extract article title, author, and publish date') | |
| cookies | No | HTTP cookies | |
| request | No | Request type | |
| proxy_enabled | No | Enable premium proxies | |
| return_format | No | Output format |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It does disclose the AI subscription requirement and the return format (structured JSON). However, it does not mention potential side effects like credit consumption, rate limits, or behavior on failure, which would be useful for a scraping tool. The description gives some key context but is not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two sentences that are front-loaded with the core purpose. It includes the essential subscription warning without unnecessary fluff, making it efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters and a sibling (spider_scrape) that serves a similar function, the description is somewhat brief. It explains the AI-based approach and the subscription requirement but does not elaborate on output formats, limitations, or explicit comparison to non-AI scraping. The schema fills in parameter details, so overall completeness is moderate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description indirectly references the 'prompt' parameter ('Describe what data you want'), but it does not add significant meaning beyond the schema. It is adequate but not enhanced.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'AI-powered structured data extraction using plain English' and mentions it returns structured JSON without CSS selectors. This distinguishes it from sibling tools like spider_scrape, which likely requires selectors.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: when you want to describe data in plain English rather than writing CSS selectors. It also mentions the prerequisite of an AI subscription. However, it does not explicitly name alternatives or state when not to use it, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_ai_searchB
AI-enhanced semantic web search. Uses intent understanding and relevance ranking to find the most relevant results. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).
| Name | Required | Description | Default |
|---|---|---|---|
| num | No | Maximum results | |
| tbs | No | Time range filter | |
| prompt | No | Additional AI guidance for search results | |
| search | Yes | The search query | |
| country | No | Two-letter country code | |
| language | No | Two-letter language code | |
| return_format | No | Output format | |
| fetch_page_content | No | Fetch full page content |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It discloses a key behavioral constraint: requirement for an active AI subscription plan. However, it does not explain what happens if the subscription is absent, nor any other side effects, rate limits, or return behavior. This is useful but limited transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with purpose and followed by the subscription requirement. Every word contributes value, and the structure is efficient. The all-caps 'REQUIRES' is a minor stylistic issue but does not detract from clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 8 parameters, no output schema, and no annotations. The description explains what the tool does and the subscription requirement, but does not describe return format, result structure, error behavior, or how it differs from spider_search. For a search tool with this complexity, critical information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 8 parameters. The description adds no parameter-specific semantics, only general statements about intent understanding and relevance ranking, which apply to the tool's behavior overall rather than individual parameters. The baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'AI-enhanced semantic web search' and 'finds the most relevant results' using intent understanding and relevance ranking. It identifies the tool's specific verb (search) and resource (web), and the 'AI-enhanced' qualifier distinguishes it from the sibling spider_search, though it does not explicitly name the sibling. This is clear but slightly less explicit than the get_calls example.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit usage guidance, such as when to use this tool versus spider_search or other siblings. It implies using it for AI-enhanced search but gives no comparison or 'when not to use' information. The subscription requirement is a prerequisite, not a usage guideline. This is a clear gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_crawlB
Crawl a website and extract content from multiple pages. Returns page content in the specified format (markdown, HTML, text, etc.). Powered by Spider - the fastest web crawler at 100K+ pages/sec.
| Name | Required | Description | Default |
|---|---|---|---|
| tld | No | Allow TLDs | |
| url | Yes | The URL to crawl. Can be comma-separated for multiple URLs. | |
| cron | No | Schedule crawl | |
| cache | No | HTTP caching. Object: {maxAge, allowStale, period} | |
| delay | No | Crawl delay in ms (max 60000). Disables concurrency | |
| depth | No | Maximum crawl depth. Default: 25. 0 for no limit. | |
| limit | No | Maximum pages to crawl per website. 0 for all pages. Default: 0 | |
| proxy | No | Proxy pool type | |
| budget | No | Crawl budget by path (e.g. {'*':100}) | |
| locale | No | Locale for content (e.g. 'en-US') | |
| cookies | No | HTTP cookies for SSR authentication | |
| request | No | Request type. Default: smart | |
| session | No | Persist HTTP headers and cookies. Default: true | |
| sitemap | No | Include links from sitemaps | |
| timeout | No | Overall request timeout | |
| metadata | No | Collect page metadata (title, description, keywords) | |
| sitemaps | No | Specific sitemap URLs to use | |
| timezone | No | Timezone for content | |
| viewport | No | Device viewport settings | |
| wait_for | No | Chrome-only readiness gate (request: 'chrome' or 'smart'). Object with any of: selector ({selector, timeout}), idle_network ({timeout}), idle_network0, almost_idle_network0, dom ({selector, timeout}), delay ({timeout}), page_navigations (bool). Each timeout is a Rust Duration: { secs, nanos }. | |
| webhooks | No | Webhook config for events (on_find, on_credits_depleted, etc.) | |
| blacklist | No | Paths to exclude (supports regex) | |
| block_ads | No | Block advertisements. Default: true | |
| whitelist | No | Paths to include (supports regex) | |
| automation | No | Web automation actions (Click, Fill, Wait, Scroll, etc.) | |
| clean_html | No | Clean HTML of unwanted attributes | |
| filter_svg | No | Filter SVG elements from markup | |
| subdomains | No | Allow subdomains | |
| user_agent | No | Custom HTTP user agent | |
| fingerprint | No | Advanced fingerprint detection for Chrome. Default: true | |
| readability | No | Use readability algorithm for content preprocessing | |
| storageless | No | Prevent data storage. Default: true | |
| chunking_alg | No | Segment content: bysentence, bylines, bycharacterlength, bywords | |
| country_code | No | ISO country code for proxy (e.g. 'gb') | |
| remote_proxy | No | External proxy connection URL | |
| event_tracker | No | Track requests, responses, automation | |
| filter_images | No | Filter image elements from markup | |
| preserve_host | No | Preserve HOST header | |
| proxy_enabled | No | Enable premium proxies. Multiplies cost by 1.5x | |
| return_format | No | Output format. Default: raw | |
| root_selector | No | Root CSS query selector for content extraction | |
| full_resources | No | Download all website resources including assets | |
| respect_robots | No | Respect robots.txt. Default: true | |
| return_cookies | No | Return HTTP response cookies | |
| return_headers | No | Return HTTP response headers | |
| block_analytics | No | Block analytics. Default: true | |
| redirect_policy | No | Redirect policy. Default: Loose | |
| request_timeout | No | HTTP request timeout in ms | |
| exclude_selector | No | CSS selector for content to ignore | |
| external_domains | No | External domains to include. Use ['*'] for all | |
| filter_main_only | No | Filter to main content only. Default: enabled | |
| return_json_data | No | Return JSON data from SSR scripts | |
| block_stylesheets | No | Block stylesheets. Default: true | |
| concurrency_limit | No | Concurrency limit for slower websites | |
| disable_intercept | No | Disable request interception | |
| filter_output_svg | No | Filter SVG tags from output | |
| return_page_links | No | Return links found on each page | |
| run_in_background | No | Run in background. Requires storageless=false or webhooks | |
| css_extraction_map | No | CSS/XPath selectors for structured extraction. Shape: { '<url-path>': [{ name, selectors: [...] }] }. '/' matches all paths. Results returned under css_extracted. | |
| request_max_retries | No | Maximum request retries | |
| filter_output_images | No | Filter images from output | |
| request_redirect_limit | No | Maximum redirects to follow | |
| filter_output_main_only | No | Filter nav, aside, footer from output |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only states return format and a promotional speed claim ('fastest web crawler'); it does not mention potential side effects like network cost, premium proxy multipliers, background execution, or external service dependencies. The schema descriptions cover some parameters, but the description itself lacks critical behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences, front-loading the core purpose and output format. It is concise, but the phrase 'Powered by Spider - the fastest web crawler at 100K+ pages/sec' is promotional and adds little actionable information, though it does identify the external provider.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is highly complex (63 parameters, nested objects) and has no output schema. The description provides only a minimal high-level purpose and return format. While the schema descriptions are detailed, the overall description does not explain typical usage patterns, output structure, or how parameters interact, leaving the agent under-oriented for correct invocation and result interpretation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for all 63 parameters, so the schema thoroughly documents each parameter. The tool description adds no additional parameter semantics beyond what the schema already provides, hence the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Crawl a website' and the scope 'extract content from multiple pages', giving a specific verb and resource. It implies differentiation from siblings like spider_scrape (which likely handles single pages), but does not explicitly name any alternative, so it stops short of full sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'from multiple pages' provides clear context for when to use this tool (multi-page crawling) versus alternatives. However, it does not explicitly mention when not to use it or name specific sibling tools, so it has context but no exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_get_creditsA
Check your available Spider API credit balance.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. The verb 'check' implies a read-only operation, but the description does not disclose whether this call consumes credits, requires authentication, or returns any specific format. Such details are minimal but valuable for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that directly states the tool's purpose. No wasted words or redundant details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple zero-parameter, read-only operation, the description is nearly complete. It explains what the tool does but could add a note about whether the check consumes credits or what the response shape is, especially since there is no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is nothing to describe. The schema coverage is 100% (empty object), and the description adds no parameter semantics because none exist. Baseline 4 for zero-parameter tools is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Check your available Spider API credit balance' uses a specific verb ('check') and resource ('credit balance'), clearly distinguishing it from sibling tools that handle crawling, scraping, and other data operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The usage is implied by the purpose: this is a read-only balance check. However, the description does not explicitly state when to use it (e.g., before a large batch of operations) or mention that no alternatives exist among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_linksA
Extract all links from a page without fetching content. Fast way to discover URLs on a site.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to extract links from | |
| limit | No | Maximum links to return | |
| request | No | Request type | |
| return_format | No | Output format |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It discloses a key behavioral trait ('without fetching content') and performance ('Fast'), but lacks details on return format, authentication, or limits. The disclosure is partial but adds some value beyond the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no fluff. It front-loads the core purpose and adds a practical use case, every word contributes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given four parameters and no output schema, the description is minimal. It covers the basic purpose but omits information about return format, limit behavior, or how it handles dynamic pages. The description is sufficient for a simple tool but lacks depth.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all four parameters (url, limit, request, return_format) are already documented. The description adds no extra parameter semantics, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (extract links), the resource (a page), and the key differentiator ('without fetching content'). It distinguishes spider_links from sibling tools like spider_scrape or spider_crawl.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for quick URL discovery ('Fast way to discover URLs on a site') but does not explicitly mention when to avoid it or suggest alternatives like spider_crawl or spider_search. Usage guidance is indirect rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_scrapeB
Scrape a single page and extract its content. No crawling — just fetches and processes one URL. Supports all output formats and screenshot capture.
| Name | Required | Description | Default |
|---|---|---|---|
| tld | No | Allow TLDs | |
| url | Yes | The URL to crawl. Can be comma-separated for multiple URLs. | |
| cron | No | Schedule crawl | |
| cache | No | HTTP caching. Object: {maxAge, allowStale, period} | |
| proxy | No | Proxy pool type | |
| binary | No | Return image as binary instead of base64 | |
| budget | No | Crawl budget by path (e.g. {'*':100}) | |
| locale | No | Locale for content (e.g. 'en-US') | |
| cookies | No | HTTP cookies for SSR authentication | |
| request | No | Request type. Default: smart | |
| session | No | Persist HTTP headers and cookies. Default: true | |
| sitemap | No | Include links from sitemaps | |
| timeout | No | Overall request timeout | |
| metadata | No | Collect page metadata (title, description, keywords) | |
| sitemaps | No | Specific sitemap URLs to use | |
| timezone | No | Timezone for content | |
| viewport | No | Device viewport settings | |
| wait_for | No | Chrome-only readiness gate (request: 'chrome' or 'smart'). Object with any of: selector ({selector, timeout}), idle_network ({timeout}), idle_network0, almost_idle_network0, dom ({selector, timeout}), delay ({timeout}), page_navigations (bool). Each timeout is a Rust Duration: { secs, nanos }. | |
| webhooks | No | Webhook config for events (on_find, on_credits_depleted, etc.) | |
| blacklist | No | Paths to exclude (supports regex) | |
| block_ads | No | Block advertisements. Default: true | |
| full_page | No | Screenshot full page. Default: true | |
| whitelist | No | Paths to include (supports regex) | |
| automation | No | Web automation actions (Click, Fill, Wait, Scroll, etc.) | |
| cdp_params | No | Chrome DevTools Protocol settings | |
| clean_html | No | Clean HTML of unwanted attributes | |
| filter_svg | No | Filter SVG elements from markup | |
| screenshot | No | Enable screenshot capture | |
| subdomains | No | Allow subdomains | |
| user_agent | No | Custom HTTP user agent | |
| fingerprint | No | Advanced fingerprint detection for Chrome. Default: true | |
| readability | No | Use readability algorithm for content preprocessing | |
| storageless | No | Prevent data storage. Default: true | |
| block_images | No | Block image loading | |
| chunking_alg | No | Segment content: bysentence, bylines, bycharacterlength, bywords | |
| country_code | No | ISO country code for proxy (e.g. 'gb') | |
| remote_proxy | No | External proxy connection URL | |
| event_tracker | No | Track requests, responses, automation | |
| filter_images | No | Filter image elements from markup | |
| preserve_host | No | Preserve HOST header | |
| proxy_enabled | No | Enable premium proxies. Multiplies cost by 1.5x | |
| return_format | No | Output format. Default: raw | |
| root_selector | No | Root CSS query selector for content extraction | |
| full_resources | No | Download all website resources including assets | |
| respect_robots | No | Respect robots.txt. Default: true | |
| return_cookies | No | Return HTTP response cookies | |
| return_headers | No | Return HTTP response headers | |
| block_analytics | No | Block analytics. Default: true | |
| omit_background | No | Omit background | |
| redirect_policy | No | Redirect policy. Default: Loose | |
| request_timeout | No | HTTP request timeout in ms | |
| exclude_selector | No | CSS selector for content to ignore | |
| external_domains | No | External domains to include. Use ['*'] for all | |
| filter_main_only | No | Filter to main content only. Default: enabled | |
| return_json_data | No | Return JSON data from SSR scripts | |
| block_stylesheets | No | Block stylesheets. Default: true | |
| concurrency_limit | No | Concurrency limit for slower websites | |
| disable_intercept | No | Disable request interception | |
| filter_output_svg | No | Filter SVG tags from output | |
| return_page_links | No | Return links found on each page | |
| run_in_background | No | Run in background. Requires storageless=false or webhooks | |
| css_extraction_map | No | CSS/XPath selectors for structured extraction. Shape: { '<url-path>': [{ name, selectors: [...] }] }. '/' matches all paths. Results returned under css_extracted. | |
| request_max_retries | No | Maximum request retries | |
| filter_output_images | No | Filter images from output | |
| request_redirect_limit | No | Maximum redirects to follow | |
| filter_output_main_only | No | Filter nav, aside, footer from output |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It claims 'No crawling' and 'one URL,' but the schema includes crawl-related parameters (sitemap, subdomains, budget, cron, whitelist, blacklist) and the URL parameter explicitly allows comma-separated multiple URLs, creating a significant misrepresentation. It also fails to mention important behaviors like proxy usage, cost multipliers, data storage, or output details beyond 'all output formats and screenshot.'
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately short and front-loaded with the main purpose, which is good for conciseness. However, the sentences do not earn their place: 'No crawling' is misleading, and 'Supports all output formats and screenshot capture' merely repeats information already in the schema. It is concise but not effectively informative, striking a balance between brevity and clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This tool has 66 parameters, no output schema, and no annotations, yet the description is only two sentences. It fails to convey essential context like return formats, error handling, cost implications, proxy usage, or even the correct scope (single vs. multiple URLs). The description is grossly inadequate for a tool of this complexity and actively misleads about crawling capabilities.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, but the description actively contradicts the schema: it says 'one URL' while the schema states 'Can be comma-separated for multiple URLs,' and it says 'No crawling' while the schema includes crawl-related parameters. The description adds no useful parameter context and instead creates confusion, reducing its helpfulness below the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with a specific verb and resource: 'Scrape a single page and extract its content.' It explicitly distinguishes itself from crawling tools by saying 'No crawling — just fetches and processes one URL,' which differentiates it from the sibling 'spider_crawl' tool. This is a clear, unambiguous purpose statement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear usage context: use this for single-page scraping, not for crawling. The phrase 'No crawling' serves as an explicit exclusion, telling the agent when not to use this tool. However, it does not name the alternative tool (e.g., 'spider_crawl') or provide guidance on multi-page scenarios, so it lacks an explicit alternative recommendation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_screenshotC
Capture screenshots of web pages. Returns base64-encoded images or binary data.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The URL to screenshot | |
| binary | No | Return image as binary instead of base64 | |
| locale | No | Locale | |
| cookies | No | HTTP cookies | |
| timeout | No | Request timeout | |
| timezone | No | Timezone | |
| viewport | No | Device viewport settings | |
| block_ads | No | Block advertisements | |
| full_page | No | Screenshot full page. Default: true | |
| automation | No | Web automation actions before screenshot | |
| cdp_params | No | Chrome DevTools Protocol settings | |
| screenshot | No | Enable screenshot capture | |
| fingerprint | No | Advanced fingerprint detection | |
| block_images | No | Block image loading | |
| country_code | No | ISO country code for proxy | |
| proxy_enabled | No | Enable premium proxies | |
| block_analytics | No | Block analytics | |
| omit_background | No | Omit background | |
| block_stylesheets | No | Block stylesheets |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only mentions the return format (base64 or binary) and ignores aspects like page loading, navigation, or automation actions, which are implied by the many parameters but not explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise with two clear sentences: one stating the purpose and one stating the return format. Every word earns its place, and the front-loaded purpose makes it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (19 parameters, nested objects, no output schema), this description is under-specified. It lacks any context on when to use the tool, how the many parameters interrelate, or what the return data looks like beyond base64/binary, making it insufficient for effective tool selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% parameter description coverage, so the baseline is 3. The tool description adds no extra semantic meaning beyond what the schema already provides, but it does not need to due to full schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool captures screenshots of web pages, which is a specific verb and resource. 'Screenshot' is distinct from sibling tools like spider_scrape or spider_crawl, but it does not explicitly contrast them, so it falls short of a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention any exclusions, prerequisites, or scenarios where this tool is preferred, leaving the agent without contextual direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_searchC
Search the web and optionally crawl results. Returns search results with optional full page content.
| Name | Required | Description | Default |
|---|---|---|---|
| num | No | Maximum number of results to return | |
| tbs | No | Time range: qdr:h (hour), qdr:d (24h), qdr:w (week), qdr:m (month), qdr:y (year) | |
| url | No | Optional URL context | |
| page | No | Page number for results | |
| limit | No | Page crawl limit for fetched results | |
| search | Yes | The search query to perform | |
| cookies | No | HTTP cookies | |
| country | No | Two-letter country code (e.g. 'us') | |
| request | No | Request type | |
| language | No | Two-letter language code (e.g. 'en') | |
| location | No | Location origin (e.g. 'United Kingdom') | |
| quick_search | No | Prioritize speed over quantity | |
| search_limit | No | Max URLs to fetch from results. 0 for all | |
| proxy_enabled | No | Enable premium proxies | |
| return_format | No | Output format for fetched content | |
| auto_pagination | No | Auto-paginate to exact desired result count | |
| fetch_page_content | No | Fetch full website content. Default: false |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It only mentions that it returns search results and optionally full page content, but omits any details about side effects, read-only behavior, rate limits, or operational traits. This is insufficient for a tool that likely performs network calls.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences and front-loaded with the core purpose. However, the second sentence ('Returns search results with optional full page content') is partially redundant with the first ('optionally crawl results'), adding slight bloat but still keeping it concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 17 parameters, no output schema, and no annotations, this description is far too minimal. It doesn't explain the return format structure, how crawling works, what 'full page content' means, or how parameters like fetch_page_content, search_limit, or auto_pagination interplay. This leaves major gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% parameter coverage, so the baseline is 3. The description itself adds no parameter-specific meaning; it doesn't mention any of the 17 parameters or how they affect behavior. But since the schema already documents each param, this is acceptable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as a web search with optional result crawling and full page content. The verb 'search' and resource 'web' are specific, and the optional full content distinguishes it from pure search. However, it doesn't explicitly differentiate from siblings like spider_crawl or spider_ai_search, so it's not a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool over other spider_* siblings. It doesn't mention alternatives or exclusions. The phrase 'optionally crawl results' could actually confuse usage with spider_crawl, making the guidance weak.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_transformA
Transform HTML content to markdown, text, or other formats. No network requests — processes HTML you provide directly.
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | List of HTML data to transform | |
| clean | No | Clean for AI (remove footers, navigation) | |
| clean_full | No | Clean HTML fully | |
| readability | No | Use readability preprocessing | |
| return_format | No | Output format |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses a key behavioral trait—'No network requests'—which is important and distinguishes it from siblings. However, it does not describe output behavior, error handling, processing limits, or how multiple inputs are handled, leaving notable gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the main action in the first sentence and a clarifying constraint in the second. Every sentence adds value with no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The combination of a complete schema and the description provides enough for an agent to understand the tool's purpose and key limitation (no network). The schema covers parameters, and the description supplies the critical distinction from siblings. However, without an output schema or annotations, it could have elaborated on response format or additional preprocessing behaviors, leaving minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all 5 parameters with 100% coverage. The description only adds a high-level summary of formats, which aligns with the return_format enum but adds no new semantic information beyond the schema. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Transform' and identifies the resource as HTML content, listing target formats (markdown, text, other). It explicitly distinguishes itself from sibling fetch/crawl tools by stating 'No network requests — processes HTML you provide directly.' This makes it clear what the tool does and how it differs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for processing already-obtained HTML, contrasting with network-fetching siblings like spider_scrape or spider_crawl. It provides clear context for when to use it but does not explicitly name alternatives or state when not to use it, so it lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
spider_unblockerB
Access blocked or protected content with advanced anti-bot bypass. Uses enhanced fingerprinting and proxy rotation. Adds 10-40 extra credits per successful unblock.
| Name | Required | Description | Default |
|---|---|---|---|
| tld | No | Allow TLDs | |
| url | Yes | The URL to crawl. Can be comma-separated for multiple URLs. | |
| cron | No | Schedule crawl | |
| cache | No | HTTP caching. Object: {maxAge, allowStale, period} | |
| proxy | No | Proxy pool type | |
| binary | No | Return image as binary instead of base64 | |
| budget | No | Crawl budget by path (e.g. {'*':100}) | |
| locale | No | Locale for content (e.g. 'en-US') | |
| cookies | No | HTTP cookies for SSR authentication | |
| request | No | Request type. Default: smart | |
| session | No | Persist HTTP headers and cookies. Default: true | |
| sitemap | No | Include links from sitemaps | |
| timeout | No | Overall request timeout | |
| metadata | No | Collect page metadata (title, description, keywords) | |
| sitemaps | No | Specific sitemap URLs to use | |
| timezone | No | Timezone for content | |
| viewport | No | Device viewport settings | |
| wait_for | No | Chrome-only readiness gate (request: 'chrome' or 'smart'). Object with any of: selector ({selector, timeout}), idle_network ({timeout}), idle_network0, almost_idle_network0, dom ({selector, timeout}), delay ({timeout}), page_navigations (bool). Each timeout is a Rust Duration: { secs, nanos }. | |
| webhooks | No | Webhook config for events (on_find, on_credits_depleted, etc.) | |
| blacklist | No | Paths to exclude (supports regex) | |
| block_ads | No | Block advertisements. Default: true | |
| full_page | No | Screenshot full page. Default: true | |
| whitelist | No | Paths to include (supports regex) | |
| automation | No | Web automation actions (Click, Fill, Wait, Scroll, etc.) | |
| cdp_params | No | Chrome DevTools Protocol settings | |
| clean_html | No | Clean HTML of unwanted attributes | |
| filter_svg | No | Filter SVG elements from markup | |
| screenshot | No | Enable screenshot capture | |
| subdomains | No | Allow subdomains | |
| user_agent | No | Custom HTTP user agent | |
| fingerprint | No | Advanced fingerprint detection for Chrome. Default: true | |
| readability | No | Use readability algorithm for content preprocessing | |
| storageless | No | Prevent data storage. Default: true | |
| block_images | No | Block image loading | |
| chunking_alg | No | Segment content: bysentence, bylines, bycharacterlength, bywords | |
| country_code | No | ISO country code for proxy (e.g. 'gb') | |
| remote_proxy | No | External proxy connection URL | |
| event_tracker | No | Track requests, responses, automation | |
| filter_images | No | Filter image elements from markup | |
| preserve_host | No | Preserve HOST header | |
| proxy_enabled | No | Enable premium proxies. Multiplies cost by 1.5x | |
| return_format | No | Output format. Default: raw | |
| root_selector | No | Root CSS query selector for content extraction | |
| full_resources | No | Download all website resources including assets | |
| respect_robots | No | Respect robots.txt. Default: true | |
| return_cookies | No | Return HTTP response cookies | |
| return_headers | No | Return HTTP response headers | |
| block_analytics | No | Block analytics. Default: true | |
| omit_background | No | Omit background | |
| redirect_policy | No | Redirect policy. Default: Loose | |
| request_timeout | No | HTTP request timeout in ms | |
| exclude_selector | No | CSS selector for content to ignore | |
| external_domains | No | External domains to include. Use ['*'] for all | |
| filter_main_only | No | Filter to main content only. Default: enabled | |
| return_json_data | No | Return JSON data from SSR scripts | |
| block_stylesheets | No | Block stylesheets. Default: true | |
| concurrency_limit | No | Concurrency limit for slower websites | |
| disable_intercept | No | Disable request interception | |
| filter_output_svg | No | Filter SVG tags from output | |
| return_page_links | No | Return links found on each page | |
| run_in_background | No | Run in background. Requires storageless=false or webhooks | |
| css_extraction_map | No | CSS/XPath selectors for structured extraction. Shape: { '<url-path>': [{ name, selectors: [...] }] }. '/' matches all paths. Results returned under css_extracted. | |
| request_max_retries | No | Maximum request retries | |
| filter_output_images | No | Filter images from output | |
| request_redirect_limit | No | Maximum redirects to follow | |
| filter_output_main_only | No | Filter nav, aside, footer from output |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It does add useful context by mentioning 'enhanced fingerprinting and proxy rotation' and the extra cost of '10-40 extra credits per successful unblock.' However, it doesn't disclose failure behaviors, whether it bypasses robots.txt, or other side effects, so transparency is partial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, front-loads the primary purpose, and every sentence adds value: purpose, method, and cost. No fluff or redundancy. It is appropriately concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the tool's high complexity (66 parameters, no output schema, no annotations), the description is minimal. It lacks guidance on return format, failure handling, or when to prefer this over spider_crawl. For an AI agent to use this correctly in diverse scenarios, the description is insufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and all 66 parameters have individual descriptions in the schema, so the description doesn't need to compensate. The description adds no parameter-specific guidance beyond naming 'proxy' and 'fingerprint' mechanisms, which already appear as schema fields. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's core purpose: 'Access blocked or protected content with advanced anti-bot bypass.' This uses a specific verb and resource, and differentiates from siblings like spider_crawl by focusing on anti-bot bypass. However, it doesn't explicitly name alternatives, so it's not a perfect 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for blocked or protected content, which provides context, but it doesn't explicitly state when to use this tool versus alternatives like spider_crawl, nor does it mention exclusions. This is implied guidance, not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
13 tool updates
v1.2.2- First observed
spider_ai_browser - First observed
spider_ai_crawl - First observed
spider_ai_links - First observed
spider_ai_scrape - First observed
spider_ai_search - First observed
spider_crawl - First observed
spider_get_credits - First observed
spider_links - First observed
spider_scrape - First observed
spider_screenshot - First observed
spider_search - First observed
spider_transform - First observed
spider_unblocker
TDQS
Most tools have distinct purposes (crawl vs. scrape vs. screenshot), but the AI variants (spider_ai_crawl, spider_ai_links) overlap conceptually with their non-AI counterparts, requiring careful description reading to select the right one.
All tools share the spider_ prefix, but the second word mixes verbs (crawl, scrape, search, transform), nouns (links, unblocker), and verb phrases (get_credits), and the AI group includes both verbs and nouns (ai_browser). This inconsistency makes the API less predictable.
13 tools is slightly above the typical sweet spot but still reasonable for the server's scope, which includes crawling, scraping, linking, screenshots, transformation, and multiple AI-powered variants.
The tool surface covers crawling, scraping, link extraction, screenshots, transformation, and credit checking thoroughly. The AI tools extend the surface with additional capabilities. Minor gaps might include batch processing or bulk URL handling, but core workflows are well served.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Scrape, crawl and search the web for AI agents via MCP.
- IndicesOAuthio.indices
Official Indices MCP server. Turn any website into a reliable API.
- UnifAPIOAuthcom.unifapi
Hosted MCP server for live public-data APIs and Skills for AI agents.
MCP server for web extraction and rendering via AceDataCloud WebExtrator
Related MCP Servers
- FlicenseBqualityDmaintenanceAn MCP Server for Web scraping and Crawling, built using Crawl4AI224-
- AlicenseNot gradedqualityCmaintenanceMCP server integrating Crawl4AI for universal web crawling and data extraction. Enables AI agents to crawl, extract markdown/HTML, take screenshots, generate PDFs, and execute JavaScript on web pages.336MIT
- AlicenseNot gradedqualityDmaintenanceA lightweight MCP server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents, enabling single-page scraping and multi-page crawling with adaptive stopping.107MIT
- -licenseNot gradedqualityCmaintenanceSelf-hosted MCP server that provides web scraping and crawling tools, integrating seamlessly with AI frameworks like OpenAI Agents SDK, Cursor, and Claude Code.4-
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/spider-rs/spider-cloud-mcp-server'
If you have feedback or need assistance with the MCP directory API, please join our Discord server