Skip to main content
Glama
spider-rs

spider-cloud-mcp

by spider-rs

spider-cloud-mcp

MCP server for the Spider web crawling and scraping API. Crawl, scrape, search, and extract web data for AI agents, RAG pipelines, and LLMs.

Setup

1. Get your API key

Sign up at spider.cloud and get your API key from the API Keys page.

2. Configure your MCP client

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Claude Code

claude mcp add spider -- npx -y spider-cloud-mcp

Then set your API key in the environment or .env file:

SPIDER_API_KEY=your-api-key

Cursor

Add to your Cursor MCP settings:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Related MCP server: crawl4ai-mcp

Tools

Core Tools

Tool

Description

spider_crawl

Crawl a website and extract content from multiple pages

spider_scrape

Scrape a single page (no crawling)

spider_search

Search the web with optional page content fetching

spider_links

Extract all links from a page

spider_screenshot

Capture page screenshots

spider_unblocker

Access bot-protected content with anti-bot bypass

spider_transform

Transform HTML to markdown/text/other formats

spider_get_credits

Check your credit balance

AI Tools (Subscription Required)

These tools require an active AI subscription plan.

Tool

Description

spider_ai_crawl

AI-guided crawling with natural language prompts

spider_ai_scrape

Extract structured data using plain English

spider_ai_search

AI-enhanced semantic web search

spider_ai_browser

Automate browser interactions with natural language

spider_ai_links

Intelligent link extraction and filtering

Examples

Crawl a website to markdown

Use spider_crawl to crawl https://example.com with limit 10 and return_format "markdown"

Search the web

Use spider_search to search for "latest AI research papers" with fetch_page_content true and num 5

AI-powered extraction

Use spider_ai_scrape on https://news.ycombinator.com with prompt "Extract all article titles, URLs, points, and comment counts as structured JSON"

Check credits

Use spider_get_credits to check my balance

API Reference

Full API documentation: spider.cloud/docs/api

License

MIT

Available Tools

13 tools
spider_ai_browserA

AI-powered browser automation using natural language. Describe actions like 'click login, fill email, submit form' and Spider automates the browser. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to automate
promptYesNatural language automation instructions (e.g. 'Click the login button, fill in email field, submit')
cookiesNoHTTP cookies
proxy_enabledNoEnable premium proxies
return_formatNoOutput format

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden. It discloses a critical prerequisite ('REQUIRES an active AI subscription plan') but does not discuss operational behaviors such as navigation side effects, timeouts, or failure outcomes. This is a moderate amount of transparency for a nontrivial automation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, with each sentence serving a distinct purpose: stating the core function, giving a usage example, and flagging the subscription requirement. It is front-loaded and contains no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex (browser automation with natural language), yet the description omits details about expected output, rate limits, and how it compares to sibling tools. The subscription requirement and purpose are present, but the overall guidance is minimal for a tool with this scope.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides descriptions for all five parameters, achieving 100% schema_description_coverage. The description adds little beyond an example prompt, so the schema already carries the parameter meaning. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'AI-powered browser automation using natural language' and provides concrete examples ('click login, fill email, submit form'). This specific verb+resource distinguishes it from sibling tools like spider_scrape or spider_crawl, which focus on data extraction rather than interaction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear usage context by showing how to phrase actions ('Describe actions like...'), implying this tool is for interactive tasks rather than retrieval. However, it does not explicitly mention alternatives or when not to use it, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_ai_crawlA

AI-guided crawling using natural language prompts. Describe what content to find and Spider's AI will guide the crawl. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to crawl
limitNoMaximum pages to crawl
promptYesNatural language prompt to guide the crawl (e.g. 'Find all product pages and extract pricing')
cookiesNoHTTP cookies
requestNoRequest type
proxy_enabledNoEnable premium proxies
return_formatNoOutput format

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behaviors. It discloses a key operational constraint—the mandatory AI subscription plan—which is not visible in the schema or annotations. It also states that the AI 'will guide the crawl,' signaling autonomous behavior. Yet it could elaborate on what happens without the plan or how AI decisions affect outcomes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exceptionally concise: two sentences and a URL. The core action and differentiator appear in the first sentence, with the subscription requirement in the second. Every word earns its place without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 7 parameters, no output schema, and AI-driven behavior, the description is adequate but sparse. It covers the primary capability and the subscription prerequisite, but does not explain what a successful response looks like, how the prompt guides the crawl in practice, or any error/edge-case behavior. More context would improve operational confidence.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters clearly. The description adds a modest reinforcement of the 'prompt' role ('Describe what content to find and Spider's AI will guide the crawl'), but does not provide significant new parameter-level meaning beyond what the schema offers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'crawling' with a clear resource ('AI-guided') and method ('using natural language prompts'). It immediately distinguishes itself from the sibling tool 'spider_crawl' by emphasizing AI guidance and prompt-driven content discovery.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly establishes the usage context: when you need AI-guided crawling based on natural language descriptions. The explicit 'REQUIRES an active AI subscription plan' provides a critical prerequisite, implying that without such a plan this tool should not be used. However, it does not explicitly compare against alternatives like spider_crawl or state when to choose one over the other.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_ai_scrapeA

AI-powered structured data extraction using plain English. Describe what data you want and get structured JSON back — no CSS selectors needed. REQUIRES an active AI subscription plan (https://spider.cloud/ai/pricing).

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to scrape
promptYesNatural language extraction prompt (e.g. 'Extract article title, author, and publish date')
cookiesNoHTTP cookies
requestNoRequest type
proxy_enabledNoEnable premium proxies
return_formatNoOutput format

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It does disclose the AI subscription requirement and the return format (structured JSON). However, it does not mention potential side effects like credit consumption, rate limits, or behavior on failure, which would be useful for a scraping tool. The description gives some key context but is not comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, consisting of two sentences that are front-loaded with the core purpose. It includes the essential subscription warning without unnecessary fluff, making it efficient and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 6 parameters and a sibling (spider_scrape) that serves a similar function, the description is somewhat brief. It explains the AI-based approach and the subscription requirement but does not elaborate on output formats, limitations, or explicit comparison to non-AI scraping. The schema fills in parameter details, so overall completeness is moderate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description indirectly references the 'prompt' parameter ('Describe what data you want'), but it does not add significant meaning beyond the schema. It is adequate but not enhanced.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'AI-powered structured data extraction using plain English' and mentions it returns structured JSON without CSS selectors. This distinguishes it from sibling tools like spider_scrape, which likely requires selectors.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool: when you want to describe data in plain English rather than writing CSS selectors. It also mentions the prerequisite of an AI subscription. However, it does not explicitly name alternatives or state when not to use it, but the context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_crawlB

Crawl a website and extract content from multiple pages. Returns page content in the specified format (markdown, HTML, text, etc.). Powered by Spider - the fastest web crawler at 100K+ pages/sec.

ParametersJSON Schema
NameRequiredDescriptionDefault
tldNoAllow TLDs
urlYesThe URL to crawl. Can be comma-separated for multiple URLs.
cronNoSchedule crawl
cacheNoHTTP caching. Object: {maxAge, allowStale, period}
delayNoCrawl delay in ms (max 60000). Disables concurrency
depthNoMaximum crawl depth. Default: 25. 0 for no limit.
limitNoMaximum pages to crawl per website. 0 for all pages. Default: 0
proxyNoProxy pool type
budgetNoCrawl budget by path (e.g. {'*':100})
localeNoLocale for content (e.g. 'en-US')
cookiesNoHTTP cookies for SSR authentication
requestNoRequest type. Default: smart
sessionNoPersist HTTP headers and cookies. Default: true
sitemapNoInclude links from sitemaps
timeoutNoOverall request timeout
metadataNoCollect page metadata (title, description, keywords)
sitemapsNoSpecific sitemap URLs to use
timezoneNoTimezone for content
viewportNoDevice viewport settings
wait_forNoChrome-only readiness gate (request: 'chrome' or 'smart'). Object with any of: selector ({selector, timeout}), idle_network ({timeout}), idle_network0, almost_idle_network0, dom ({selector, timeout}), delay ({timeout}), page_navigations (bool). Each timeout is a Rust Duration: { secs, nanos }.
webhooksNoWebhook config for events (on_find, on_credits_depleted, etc.)
blacklistNoPaths to exclude (supports regex)
block_adsNoBlock advertisements. Default: true
whitelistNoPaths to include (supports regex)
automationNoWeb automation actions (Click, Fill, Wait, Scroll, etc.)
clean_htmlNoClean HTML of unwanted attributes
filter_svgNoFilter SVG elements from markup
subdomainsNoAllow subdomains
user_agentNoCustom HTTP user agent
fingerprintNoAdvanced fingerprint detection for Chrome. Default: true
readabilityNoUse readability algorithm for content preprocessing
storagelessNoPrevent data storage. Default: true
chunking_algNoSegment content: bysentence, bylines, bycharacterlength, bywords
country_codeNoISO country code for proxy (e.g. 'gb')
remote_proxyNoExternal proxy connection URL
event_trackerNoTrack requests, responses, automation
filter_imagesNoFilter image elements from markup
preserve_hostNoPreserve HOST header
proxy_enabledNoEnable premium proxies. Multiplies cost by 1.5x
return_formatNoOutput format. Default: raw
root_selectorNoRoot CSS query selector for content extraction
full_resourcesNoDownload all website resources including assets
respect_robotsNoRespect robots.txt. Default: true
return_cookiesNoReturn HTTP response cookies
return_headersNoReturn HTTP response headers
block_analyticsNoBlock analytics. Default: true
redirect_policyNoRedirect policy. Default: Loose
request_timeoutNoHTTP request timeout in ms
exclude_selectorNoCSS selector for content to ignore
external_domainsNoExternal domains to include. Use ['*'] for all
filter_main_onlyNoFilter to main content only. Default: enabled
return_json_dataNoReturn JSON data from SSR scripts
block_stylesheetsNoBlock stylesheets. Default: true
concurrency_limitNoConcurrency limit for slower websites
disable_interceptNoDisable request interception
filter_output_svgNoFilter SVG tags from output
return_page_linksNoReturn links found on each page
run_in_backgroundNoRun in background. Requires storageless=false or webhooks
css_extraction_mapNoCSS/XPath selectors for structured extraction. Shape: { '<url-path>': [{ name, selectors: [...] }] }. '/' matches all paths. Results returned under css_extracted.
request_max_retriesNoMaximum request retries
filter_output_imagesNoFilter images from output
request_redirect_limitNoMaximum redirects to follow
filter_output_main_onlyNoFilter nav, aside, footer from output

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only states return format and a promotional speed claim ('fastest web crawler'); it does not mention potential side effects like network cost, premium proxy multipliers, background execution, or external service dependencies. The schema descriptions cover some parameters, but the description itself lacks critical behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is only two sentences, front-loading the core purpose and output format. It is concise, but the phrase 'Powered by Spider - the fastest web crawler at 100K+ pages/sec' is promotional and adds little actionable information, though it does identify the external provider.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is highly complex (63 parameters, nested objects) and has no output schema. The description provides only a minimal high-level purpose and return format. While the schema descriptions are detailed, the overall description does not explain typical usage patterns, output structure, or how parameters interact, leaving the agent under-oriented for correct invocation and result interpretation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for all 63 parameters, so the schema thoroughly documents each parameter. The tool description adds no additional parameter semantics beyond what the schema already provides, hence the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Crawl a website' and the scope 'extract content from multiple pages', giving a specific verb and resource. It implies differentiation from siblings like spider_scrape (which likely handles single pages), but does not explicitly name any alternative, so it stops short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'from multiple pages' provides clear context for when to use this tool (multi-page crawling) versus alternatives. However, it does not explicitly mention when not to use it or name specific sibling tools, so it has context but no exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_get_creditsA

Check your available Spider API credit balance.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. The verb 'check' implies a read-only operation, but the description does not disclose whether this call consumes credits, requires authentication, or returns any specific format. Such details are minimal but valuable for an agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that directly states the tool's purpose. No wasted words or redundant details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple zero-parameter, read-only operation, the description is nearly complete. It explains what the tool does but could add a note about whether the check consumes credits or what the response shape is, especially since there is no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is nothing to describe. The schema coverage is 100% (empty object), and the description adds no parameter semantics because none exist. Baseline 4 for zero-parameter tools is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Check your available Spider API credit balance' uses a specific verb ('check') and resource ('credit balance'), clearly distinguishing it from sibling tools that handle crawling, scraping, and other data operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The usage is implied by the purpose: this is a read-only balance check. However, the description does not explicitly state when to use it (e.g., before a large batch of operations) or mention that no alternatives exist among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_scrapeB

Scrape a single page and extract its content. No crawling — just fetches and processes one URL. Supports all output formats and screenshot capture.

ParametersJSON Schema
NameRequiredDescriptionDefault
tldNoAllow TLDs
urlYesThe URL to crawl. Can be comma-separated for multiple URLs.
cronNoSchedule crawl
cacheNoHTTP caching. Object: {maxAge, allowStale, period}
proxyNoProxy pool type
binaryNoReturn image as binary instead of base64
budgetNoCrawl budget by path (e.g. {'*':100})
localeNoLocale for content (e.g. 'en-US')
cookiesNoHTTP cookies for SSR authentication
requestNoRequest type. Default: smart
sessionNoPersist HTTP headers and cookies. Default: true
sitemapNoInclude links from sitemaps
timeoutNoOverall request timeout
metadataNoCollect page metadata (title, description, keywords)
sitemapsNoSpecific sitemap URLs to use
timezoneNoTimezone for content
viewportNoDevice viewport settings
wait_forNoChrome-only readiness gate (request: 'chrome' or 'smart'). Object with any of: selector ({selector, timeout}), idle_network ({timeout}), idle_network0, almost_idle_network0, dom ({selector, timeout}), delay ({timeout}), page_navigations (bool). Each timeout is a Rust Duration: { secs, nanos }.
webhooksNoWebhook config for events (on_find, on_credits_depleted, etc.)
blacklistNoPaths to exclude (supports regex)
block_adsNoBlock advertisements. Default: true
full_pageNoScreenshot full page. Default: true
whitelistNoPaths to include (supports regex)
automationNoWeb automation actions (Click, Fill, Wait, Scroll, etc.)
cdp_paramsNoChrome DevTools Protocol settings
clean_htmlNoClean HTML of unwanted attributes
filter_svgNoFilter SVG elements from markup
screenshotNoEnable screenshot capture
subdomainsNoAllow subdomains
user_agentNoCustom HTTP user agent
fingerprintNoAdvanced fingerprint detection for Chrome. Default: true
readabilityNoUse readability algorithm for content preprocessing
storagelessNoPrevent data storage. Default: true
block_imagesNoBlock image loading
chunking_algNoSegment content: bysentence, bylines, bycharacterlength, bywords
country_codeNoISO country code for proxy (e.g. 'gb')
remote_proxyNoExternal proxy connection URL
event_trackerNoTrack requests, responses, automation
filter_imagesNoFilter image elements from markup
preserve_hostNoPreserve HOST header
proxy_enabledNoEnable premium proxies. Multiplies cost by 1.5x
return_formatNoOutput format. Default: raw
root_selectorNoRoot CSS query selector for content extraction
full_resourcesNoDownload all website resources including assets
respect_robotsNoRespect robots.txt. Default: true
return_cookiesNoReturn HTTP response cookies
return_headersNoReturn HTTP response headers
block_analyticsNoBlock analytics. Default: true
omit_backgroundNoOmit background
redirect_policyNoRedirect policy. Default: Loose
request_timeoutNoHTTP request timeout in ms
exclude_selectorNoCSS selector for content to ignore
external_domainsNoExternal domains to include. Use ['*'] for all
filter_main_onlyNoFilter to main content only. Default: enabled
return_json_dataNoReturn JSON data from SSR scripts
block_stylesheetsNoBlock stylesheets. Default: true
concurrency_limitNoConcurrency limit for slower websites
disable_interceptNoDisable request interception
filter_output_svgNoFilter SVG tags from output
return_page_linksNoReturn links found on each page
run_in_backgroundNoRun in background. Requires storageless=false or webhooks
css_extraction_mapNoCSS/XPath selectors for structured extraction. Shape: { '<url-path>': [{ name, selectors: [...] }] }. '/' matches all paths. Results returned under css_extracted.
request_max_retriesNoMaximum request retries
filter_output_imagesNoFilter images from output
request_redirect_limitNoMaximum redirects to follow
filter_output_main_onlyNoFilter nav, aside, footer from output

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It claims 'No crawling' and 'one URL,' but the schema includes crawl-related parameters (sitemap, subdomains, budget, cron, whitelist, blacklist) and the URL parameter explicitly allows comma-separated multiple URLs, creating a significant misrepresentation. It also fails to mention important behaviors like proxy usage, cost multipliers, data storage, or output details beyond 'all output formats and screenshot.'

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately short and front-loaded with the main purpose, which is good for conciseness. However, the sentences do not earn their place: 'No crawling' is misleading, and 'Supports all output formats and screenshot capture' merely repeats information already in the schema. It is concise but not effectively informative, striking a balance between brevity and clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This tool has 66 parameters, no output schema, and no annotations, yet the description is only two sentences. It fails to convey essential context like return formats, error handling, cost implications, proxy usage, or even the correct scope (single vs. multiple URLs). The description is grossly inadequate for a tool of this complexity and actively misleads about crawling capabilities.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description actively contradicts the schema: it says 'one URL' while the schema states 'Can be comma-separated for multiple URLs,' and it says 'No crawling' while the schema includes crawl-related parameters. The description adds no useful parameter context and instead creates confusion, reducing its helpfulness below the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb and resource: 'Scrape a single page and extract its content.' It explicitly distinguishes itself from crawling tools by saying 'No crawling — just fetches and processes one URL,' which differentiates it from the sibling 'spider_crawl' tool. This is a clear, unambiguous purpose statement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear usage context: use this for single-page scraping, not for crawling. The phrase 'No crawling' serves as an explicit exclusion, telling the agent when not to use this tool. However, it does not name the alternative tool (e.g., 'spider_crawl') or provide guidance on multi-page scenarios, so it lacks an explicit alternative recommendation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_screenshotC

Capture screenshots of web pages. Returns base64-encoded images or binary data.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to screenshot
binaryNoReturn image as binary instead of base64
localeNoLocale
cookiesNoHTTP cookies
timeoutNoRequest timeout
timezoneNoTimezone
viewportNoDevice viewport settings
block_adsNoBlock advertisements
full_pageNoScreenshot full page. Default: true
automationNoWeb automation actions before screenshot
cdp_paramsNoChrome DevTools Protocol settings
screenshotNoEnable screenshot capture
fingerprintNoAdvanced fingerprint detection
block_imagesNoBlock image loading
country_codeNoISO country code for proxy
proxy_enabledNoEnable premium proxies
block_analyticsNoBlock analytics
omit_backgroundNoOmit background
block_stylesheetsNoBlock stylesheets

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only mentions the return format (base64 or binary) and ignores aspects like page loading, navigation, or automation actions, which are implied by the many parameters but not explained.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exceptionally concise with two clear sentences: one stating the purpose and one stating the return format. Every word earns its place, and the front-loaded purpose makes it easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (19 parameters, nested objects, no output schema), this description is under-specified. It lacks any context on when to use the tool, how the many parameters interrelate, or what the return data looks like beyond base64/binary, making it insufficient for effective tool selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% parameter description coverage, so the baseline is 3. The tool description adds no extra semantic meaning beyond what the schema already provides, but it does not need to due to full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool captures screenshots of web pages, which is a specific verb and resource. 'Screenshot' is distinct from sibling tools like spider_scrape or spider_crawl, but it does not explicitly contrast them, so it falls short of a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention any exclusions, prerequisites, or scenarios where this tool is preferred, leaving the agent without contextual direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_transformA

Transform HTML content to markdown, text, or other formats. No network requests — processes HTML you provide directly.

ParametersJSON Schema
NameRequiredDescriptionDefault
dataYesList of HTML data to transform
cleanNoClean for AI (remove footers, navigation)
clean_fullNoClean HTML fully
readabilityNoUse readability preprocessing
return_formatNoOutput format

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses a key behavioral trait—'No network requests'—which is important and distinguishes it from siblings. However, it does not describe output behavior, error handling, processing limits, or how multiple inputs are handled, leaving notable gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main action in the first sentence and a clarifying constraint in the second. Every sentence adds value with no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The combination of a complete schema and the description provides enough for an agent to understand the tool's purpose and key limitation (no network). The schema covers parameters, and the description supplies the critical distinction from siblings. However, without an output schema or annotations, it could have elaborated on response format or additional preprocessing behaviors, leaving minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all 5 parameters with 100% coverage. The description only adds a high-level summary of formats, which aligns with the return_format enum but adds no new semantic information beyond the schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Transform' and identifies the resource as HTML content, listing target formats (markdown, text, other). It explicitly distinguishes itself from sibling fetch/crawl tools by stating 'No network requests — processes HTML you provide directly.' This makes it clear what the tool does and how it differs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for processing already-obtained HTML, contrasting with network-fetching siblings like spider_scrape or spider_crawl. It provides clear context for when to use it but does not explicitly name alternatives or state when not to use it, so it lacks explicit exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_unblockerB

Access blocked or protected content with advanced anti-bot bypass. Uses enhanced fingerprinting and proxy rotation. Adds 10-40 extra credits per successful unblock.

ParametersJSON Schema
NameRequiredDescriptionDefault
tldNoAllow TLDs
urlYesThe URL to crawl. Can be comma-separated for multiple URLs.
cronNoSchedule crawl
cacheNoHTTP caching. Object: {maxAge, allowStale, period}
proxyNoProxy pool type
binaryNoReturn image as binary instead of base64
budgetNoCrawl budget by path (e.g. {'*':100})
localeNoLocale for content (e.g. 'en-US')
cookiesNoHTTP cookies for SSR authentication
requestNoRequest type. Default: smart
sessionNoPersist HTTP headers and cookies. Default: true
sitemapNoInclude links from sitemaps
timeoutNoOverall request timeout
metadataNoCollect page metadata (title, description, keywords)
sitemapsNoSpecific sitemap URLs to use
timezoneNoTimezone for content
viewportNoDevice viewport settings
wait_forNoChrome-only readiness gate (request: 'chrome' or 'smart'). Object with any of: selector ({selector, timeout}), idle_network ({timeout}), idle_network0, almost_idle_network0, dom ({selector, timeout}), delay ({timeout}), page_navigations (bool). Each timeout is a Rust Duration: { secs, nanos }.
webhooksNoWebhook config for events (on_find, on_credits_depleted, etc.)
blacklistNoPaths to exclude (supports regex)
block_adsNoBlock advertisements. Default: true
full_pageNoScreenshot full page. Default: true
whitelistNoPaths to include (supports regex)
automationNoWeb automation actions (Click, Fill, Wait, Scroll, etc.)
cdp_paramsNoChrome DevTools Protocol settings
clean_htmlNoClean HTML of unwanted attributes
filter_svgNoFilter SVG elements from markup
screenshotNoEnable screenshot capture
subdomainsNoAllow subdomains
user_agentNoCustom HTTP user agent
fingerprintNoAdvanced fingerprint detection for Chrome. Default: true
readabilityNoUse readability algorithm for content preprocessing
storagelessNoPrevent data storage. Default: true
block_imagesNoBlock image loading
chunking_algNoSegment content: bysentence, bylines, bycharacterlength, bywords
country_codeNoISO country code for proxy (e.g. 'gb')
remote_proxyNoExternal proxy connection URL
event_trackerNoTrack requests, responses, automation
filter_imagesNoFilter image elements from markup
preserve_hostNoPreserve HOST header
proxy_enabledNoEnable premium proxies. Multiplies cost by 1.5x
return_formatNoOutput format. Default: raw
root_selectorNoRoot CSS query selector for content extraction
full_resourcesNoDownload all website resources including assets
respect_robotsNoRespect robots.txt. Default: true
return_cookiesNoReturn HTTP response cookies
return_headersNoReturn HTTP response headers
block_analyticsNoBlock analytics. Default: true
omit_backgroundNoOmit background
redirect_policyNoRedirect policy. Default: Loose
request_timeoutNoHTTP request timeout in ms
exclude_selectorNoCSS selector for content to ignore
external_domainsNoExternal domains to include. Use ['*'] for all
filter_main_onlyNoFilter to main content only. Default: enabled
return_json_dataNoReturn JSON data from SSR scripts
block_stylesheetsNoBlock stylesheets. Default: true
concurrency_limitNoConcurrency limit for slower websites
disable_interceptNoDisable request interception
filter_output_svgNoFilter SVG tags from output
return_page_linksNoReturn links found on each page
run_in_backgroundNoRun in background. Requires storageless=false or webhooks
css_extraction_mapNoCSS/XPath selectors for structured extraction. Shape: { '<url-path>': [{ name, selectors: [...] }] }. '/' matches all paths. Results returned under css_extracted.
request_max_retriesNoMaximum request retries
filter_output_imagesNoFilter images from output
request_redirect_limitNoMaximum redirects to follow
filter_output_main_onlyNoFilter nav, aside, footer from output

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It does add useful context by mentioning 'enhanced fingerprinting and proxy rotation' and the extra cost of '10-40 extra credits per successful unblock.' However, it doesn't disclose failure behaviors, whether it bypasses robots.txt, or other side effects, so transparency is partial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, front-loads the primary purpose, and every sentence adds value: purpose, method, and cost. No fluff or redundancy. It is appropriately concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the tool's high complexity (66 parameters, no output schema, no annotations), the description is minimal. It lacks guidance on return format, failure handling, or when to prefer this over spider_crawl. For an AI agent to use this correctly in diverse scenarios, the description is insufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and all 66 parameters have individual descriptions in the schema, so the description doesn't need to compensate. The description adds no parameter-specific guidance beyond naming 'proxy' and 'fingerprint' mechanisms, which already appear as schema fields. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's core purpose: 'Access blocked or protected content with advanced anti-bot bypass.' This uses a specific verb and resource, and differentiates from siblings like spider_crawl by focusing on anti-bot bypass. However, it doesn't explicitly name alternatives, so it's not a perfect 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for blocked or protected content, which provides context, but it doesn't explicitly state when to use this tool versus alternatives like spider_crawl, nor does it mention exclusions. This is implied guidance, not explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 13 tool updatesv1.2.2
    • First observedspider_ai_browser
    • First observedspider_ai_crawl
    • First observedspider_ai_links
    • First observedspider_ai_scrape
    • First observedspider_ai_search
    • First observedspider_crawl
    • First observedspider_get_credits
    • First observedspider_links
    • First observedspider_scrape
    • First observedspider_screenshot
    • First observedspider_search
    • First observedspider_transform
    • First observedspider_unblocker

TDQS

B3.4/5.0
Disambiguation4/5

Most tools have distinct purposes (crawl vs. scrape vs. screenshot), but the AI variants (spider_ai_crawl, spider_ai_links) overlap conceptually with their non-AI counterparts, requiring careful description reading to select the right one.

Naming Consistency3/5

All tools share the spider_ prefix, but the second word mixes verbs (crawl, scrape, search, transform), nouns (links, unblocker), and verb phrases (get_credits), and the AI group includes both verbs and nouns (ai_browser). This inconsistency makes the API less predictable.

Tool Count4/5

13 tools is slightly above the typical sweet spot but still reasonable for the server's scope, which includes crawling, scraping, linking, screenshots, transformation, and multiple AI-powered variants.

Completeness4/5

The tool surface covers crawling, scraping, link extraction, screenshots, transformation, and credit checking thoroughly. The AI tools extend the surface with additional capabilities. Minor gaps might include batch processing or bulk URL handling, but core workflows are well served.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    C
    maintenance
    MCP server integrating Crawl4AI for universal web crawling and data extraction. Enables AI agents to crawl, extract markdown/HTML, take screenshots, generate PDFs, and execute JavaScript on web pages.
    33
    6
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    A lightweight MCP server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents, enabling single-page scraping and multi-page crawling with adaptive stopping.
    107
    MIT
  • -
    license
    Not graded
    quality
    C
    maintenance
    Self-hosted MCP server that provides web scraping and crawling tools, integrating seamlessly with AI frameworks like OpenAI Agents SDK, Cursor, and Claude Code.
    4
    -

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/spider-rs/spider-cloud-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server