Skip to main content
Glama
crawlbase

Crawlbase MCP

Official
by crawlbase

What is Crawlbase MCP?

Crawlbase MCP is a Model Context Protocol (MCP) server that bridges AI agents and the live web. Instead of relying on outdated training data, your LLMs can now fetch fresh, structured, real-time content — powered by Crawlbase’s proven crawling infrastructure trusted by 70,000+ developers worldwide.

It handles the complexity of scraping for you:

  • JavaScript rendering for modern web apps

  • Proxy rotation & anti-bot evasion

  • Structured outputs (HTML, Markdown, screenshots)

How It Works

  • Get Free Crawlbase Tokens → Sign up at Crawlbase ↗️, get free Normal, and JavaScript tokens.

  • Set Up MCP Configuration → Configure the MCP server in your preferred client (Claude, Cursor, or Windsurf) by updating the MCP Servers settings.

  • Start Crawling → Use commands like crawl, crawl_markdown, or crawl_screenshot to bring live web data into your AI agent.

Related MCP server: Raccoon AI MCP Server

Setup & Integration

Claude Desktop

  1. Open Claude Desktop → Settings → Developer → Edit Config

  2. Add to claude_desktop_config.json:

  3. Replace your_token_here and your_js_token_here with the tokens from your dashboard.

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": ["@crawlbase/mcp@latest"],
      "env": {
        "CRAWLBASE_TOKEN": "your_token_here",
        "CRAWLBASE_JS_TOKEN": "your_js_token_here"
      }
    }
  }
}

Claude Code

Add to your claude.json configuration:

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": ["@crawlbase/mcp@latest"],
      "env": {
        "CRAWLBASE_TOKEN": "your_token_here",
        "CRAWLBASE_JS_TOKEN": "your_js_token_here"
      }
    }
  }
}

Cursor IDE

  1. Open Cursor IDE → File → Preferences → Cursor Settings → Tools and Integrations → Add Custom MCP

  2. Add to mcp.json:

  3. Replace your_token_here and your_js_token_here with the tokens from your dashboard.

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": ["@crawlbase/mcp@latest"],
      "env": {
        "CRAWLBASE_TOKEN": "your_token_here",
        "CRAWLBASE_JS_TOKEN": "your_js_token_here"
      }
    }
  }
}

Windsurf IDE

  1. Open WindSurf IDE → File → Preferences → WindSurf Settings → General → MCP Servers → Manage MCPs → View raw config

  2. Add to mcp_config.json:

  3. Replace your_token_here and your_js_token_here with the tokens from your dashboard.

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": ["@crawlbase/mcp@latest"],
      "env": {
        "CRAWLBASE_TOKEN": "your_token_here",
        "CRAWLBASE_JS_TOKEN": "your_js_token_here"
      }
    }
  }
}

HTTP Transport Mode

For scenarios where you need a shared MCP server accessible over HTTP (e.g., multi-user environments, custom integrations), you can run the server in HTTP mode:

# Clone and install
git clone https://github.com/crawlbase/crawlbase-mcp.git
cd crawlbase-mcp
npm install

# Start HTTP server with tokens (default port: 3000)
CRAWLBASE_TOKEN=your_token CRAWLBASE_JS_TOKEN=your_js_token npm run start:http

# Or with custom port
CRAWLBASE_TOKEN=your_token CRAWLBASE_JS_TOKEN=your_js_token MCP_PORT=8080 npm run start:http

The server exposes:

  • POST /mcp - MCP Streamable HTTP endpoint

  • GET /health - Health check endpoint

Per-Request Token Authentication

HTTP mode supports per-request tokens via headers, allowing multiple users to share a single server:

curl -X POST http://localhost:3000/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "X-Crawlbase-Token: your_token" \
  -H "X-Crawlbase-JS-Token: your_js_token" \
  -d '{"jsonrpc": "2.0", "method": "tools/list", "id": 1}'

Headers:

  • X-Crawlbase-Token - Normal token for HTML requests

  • X-Crawlbase-JS-Token - JavaScript token for JS-rendered pages/screenshots

Headers override environment variables when provided, enabling multi-tenant deployments.

🔑 Get your free tokens at Crawlbase ↗️.

Usage

Once configured, use these commands inside Claude, Cursor, or Windsurf:

  • crawl → Fetch raw HTML

  • crawl_markdown → Extract clean Markdown

  • crawl_screenshot → Capture screenshots

Example prompts:

  • “Crawl Hacker News and return top stories in markdown.”

  • “Take a screenshot of TechCrunch homepage.”

  • “Fetch Tesla investor relations page as HTML.”

Async Crawling with Cloud Storage

For larger jobs, Crawlbase MCP can push crawl results to Crawlbase Cloud Storage instead of returning them immediately. Your AI agent can then come back later to read, list, or clean up those pages — useful when crawling many URLs at once, revisiting a dataset across sessions, or keeping heavy HTML out of the chat until you actually need it.

Example prompts:

  • “Crawl these 50 product pages and save them to my Crawlbase storage. Once they're saved, summarize each one.”

  • “Save the Hacker News front page to storage so I can analyze it later.”

  • “How many pages do I have stored in Crawlbase right now? Show me the most recent 20.”

  • “Pull back everything I saved yesterday from my Crawlbase storage and give me a report.”

  • “Delete all the pages I have in Crawlbase storage — I'm done with that project.”

Use Cases

  • Market research → Pull live data from competitors, news, and reports

  • E-commerce monitoring → Track products, reviews, and prices in real time

  • News & finance feeds → Keep AI agents up-to-date with live events

  • Autonomous AI agents → Give them vision to act on fresh web data

Resources & Next Steps

Looking to supercharge your AI agents with live web data? Get started here:


MSeeP.ai Security Assessment Badge

Copyright 2026 Crawlbase

Available Tools

9 tools
crawlB

Crawl a URL and return HTML content. Pass store=true to push the result to Crawlbase Cloud Storage instead of returning the body — the response will then contain only the RID and metadata, which can later be retrieved via storage_get.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to crawl
storeNoPush the result to Crawlbase Cloud Storage. When true, the response returns only RID + metadata (no body) so the body can be fetched later via storage_get.
deviceNoDevice type for crawling
countryNoCountry code for geo-targeting
ajax_waitNoWait time for AJAX requests in milliseconds
page_waitNoWait time for page load in milliseconds
screenshotNoTake a screenshot of the page
user_agentNoCustom user agent string

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It explains the store=true behavior (no body returned, only RID and metadata) which is good. However, it omits other behavioral traits such as default return format, error handling, rate limits, or permissions. It does not contradict annotations (none exist).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, with the first sentence stating the core purpose and the second explaining the critical optional behavior. No redundant information. It is appropriately front-loaded and every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 parameters and no output schema or annotations, the description covers only the core function and store behavior. It fails to describe other parameters, return values, error conditions, or how it relates to sibling tools. This leaves significant gaps for an agent to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds meaning to the store parameter by explaining the behavioral change. For other parameters (device, country, etc.), it adds no additional context beyond the schema. Overall, it provides some extra value but does not fully compensate for the low coverage that would otherwise be needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it crawls a URL and returns HTML content, which is a specific verb+resource. It also details the store parameter behavior. However, it does not explicitly distinguish from sibling tools like crawl_markdown or crawl_screenshot, leaving some ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides guidance on using store=true for storage, but lacks any when-to-use or when-not-to-use guidance compared to alternative tools. There is no mention of prerequisites or context for choosing crawl over its siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

crawl_markdownA

Crawl a URL and extract clean markdown content. Pass store=true to persist the original page in Cloud Storage and return only the RID + metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to crawl
storeNoPush the result to Crawlbase Cloud Storage. When true, returns only RID + metadata; retrieve later with storage_get (use as=markdown to convert).
deviceNoDevice type for crawling
user_agentNoCustom user agent string

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses main behavior (crawl → markdown) and the store option's alternative return. However, it omits error handling, rate limits, or response format details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, direct and front-loaded. Every word is necessary; no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 4 parameters and no output schema. Description explains the two operational modes but does not specify the return structure for the default mode (e.g., the markdown content format) or error cases. It's adequate but not thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions cover all 4 parameters (100% coverage), so baseline is 3. The description rephrases the store parameter's behavior but adds no new meaning beyond the schema for url, device, or user_agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it crawls a URL and extracts clean markdown content, specifying both verb and resource. It implies differentiation from siblings like crawl (likely raw) and crawl_screenshot by focusing on markdown output.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the store parameter's effect on behavior, but lacks explicit guidance on when to use this tool versus siblings (e.g., crawl for raw HTML). No disclaimers or alternatives are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

crawl_screenshotA

Take a screenshot of a webpage. Pass store=true to persist the underlying HTML page to Cloud Storage (the screenshot itself is not stored and is still returned as an ephemeral screenshot_url).

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to screenshot
modeNoScreenshot mode (default: fullpage)
storeNoPersist the underlying HTML page to Cloud Storage. The screenshot itself is not stored; only RID + metadata + screenshot_url are returned (no image download). Storage is scoped to the JS token — use_js_token=true is required when retrieving it later.
widthNoMaximum width in pixels (only with mode=viewport)
deviceNoDevice type for screenshot
heightNoMaximum height in pixels (only with mode=viewport)
page_waitNoWait time before taking screenshot

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses that the screenshot is ephemeral (returns an ephemeral screenshot_url), explains that store=true persists the HTML page (not the screenshot), and notes storage scoping to JS token. It does not mention rate limits or auth beyond token hint, but provides key behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no waste. First sentence states main action, second sentence adds the critical nuance about store parameter. Information is front-loaded and every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 7 parameters and no output schema, the description covers the main behavior but does not explain return format (beyond mentioning screenshot_url) or common use cases for parameters like mode, device, page_wait. It is adequate but could be more complete for an agent to invoke correctly without schema cross-referencing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by clarifying the store parameter behavior (persists HTML, not screenshot) and mentioning the use_js_token requirement, which is not in the schema description. This exceeds minimal schema explanation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool takes a screenshot of a webpage, distinguishing it from sibling tools like crawl and crawl_markdown which have different purposes (crawling and markdown extraction). The verb 'Take' with resource 'screenshot of a webpage' is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides guidance on using the store parameter (when to persist HTML) but does not explicitly state when to use this tool over siblings like crawl or crawl_markdown. It implies usage for visual capture but lacks explicit when-not or alternative comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

storage_bulk_deleteB

Bulk-delete up to 100 items from Crawlbase Cloud Storage by RID. Irreversible.

ParametersJSON Schema
NameRequiredDescriptionDefault
ridsYesUp to 100 storage RIDs to delete.
use_js_tokenNoOperate on the JS token's storage instead of the normal token's. Set to true if the target RIDs came from a crawl whose response showed token_type=js.

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. The description only notes irreversibility, which is critical, but lacks other behavioral details like partial failures, idempotency, or required authentication. Minimal disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste. Front-loaded with purpose and key constraint, then immediate behavioral warning. Appropriate for a straightforward tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and moderate complexity, the description covers purpose and irreversibility. However, lacks usage context, error handling notes, and batch semantics, making it adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters have clear descriptions in the schema. The description adds no additional parameter-specific information beyond the schema, so baseline is 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Bulk-delete', the resource 'Crawlbase Cloud Storage', the constraint 'by RID', and the limit 'up to 100 items'. It distinguishes from siblings like storage_delete (single) and storage_bulk_get.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool vs alternatives such as storage_delete for single deletes. No prerequisites or contexts mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

storage_bulk_getA

Bulk-fetch up to 100 stored items by RID. Bodies are decoded automatically. Default as=metadata_only returns just RID/URL/timestamps to keep context lean; use as=html or as=markdown to include bodies.

ParametersJSON Schema
NameRequiredDescriptionDefault
asNometadata_only (default) returns RID/URL/timestamps only; json returns full records; html/markdown include bodies.
ridsYesUp to 100 storage RIDs to retrieve.
auto_deleteNoDelete each item from storage after it is retrieved.
use_js_tokenNoOperate on the JS token's storage instead of the normal token's. Set to true if the target RIDs came from a crawl whose response showed token_type=js.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, description carries full burden. It mentions automatic decoding and default return format, but does not disclose side effects of auto_delete, authentication needs, or what happens on errors. Could be improved.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with key information (limit, decoding, default behavior). No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a tool with 4 parameters and no output schema. Covers main use and important parameter. Could mention return format and clarify auto_delete as destructive action.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers all parameters with 100% description coverage. Description adds value by explaining default 'as' behavior and trade-offs, but does not elaborate on auto_delete or use_js_token beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool bulk-fetches up to 100 stored items by RID, with automatic decoding. It distinguishes from siblings like storage_get (single item) and storage_bulk_delete (delete operation).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies usage for fetching multiple items, but does not explicitly mention alternatives like storage_get for single items or storage_bulk_delete for deleting. Provides guidance on the 'as' parameter but lacks when-not-to-use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

storage_countA

Get the total number of documents in Crawlbase Cloud Storage.

ParametersJSON Schema
NameRequiredDescriptionDefault
use_js_tokenNoQuery the JS token's storage instead of the normal token's. Set to true if the prior crawl response showed token_type=js.

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It clearly indicates a read operation (get count), which is transparent for this simple tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One concise sentence that front-loads the purpose, with no unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description should hint at the return format. It doesn't, though for a simple count it's minimally adequate given sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and describes the parameter well. The description adds no additional meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it retrieves the total number of documents, distinguishing it from sibling tools that list, get, or delete specific documents.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like storage_list or storage_get. The context is implied but not stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

storage_deleteA

Delete a single item from Crawlbase Cloud Storage by RID.

ParametersJSON Schema
NameRequiredDescriptionDefault
ridYesStorage request ID to delete.
use_js_tokenNoOperate on the JS token's storage instead of the normal token's. Set to true if the target RIDs came from a crawl whose response showed token_type=js.

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not disclose behavioral traits such as whether deletion is permanent, what happens if the RID does not exist, or any side effects. The description is too brief to add meaningful context beyond the tool's basic function.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no wasted words. It is front-loaded and efficiently communicates the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (2 params, no output schema), the description is mostly adequate. It could be improved by noting that deletion is irreversible and by hinting at sibling tools for bulk operations, but it covers the essential functionality.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with descriptions for both parameters (rid and use_js_token). The description adds no extra meaning beyond the schema; it only mentions deletion by RID, ignoring the use_js_token parameter. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (Delete), resource (a single item from Crawlbase Cloud Storage), and method (by RID). It effectively distinguishes from the sibling tool storage_bulk_delete, which handles bulk deletions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for deleting a single item by RID but does not explicitly state when to use this tool versus alternatives like storage_bulk_delete or when not to use it. No prerequisites or context are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

storage_getA

Retrieve a single stored page from Crawlbase Cloud Storage by RID or URL. Only works for pages previously crawled with store=true. Returns the raw JSON record by default; pass as=html to return just the body, or as=markdown to convert it.

ParametersJSON Schema
NameRequiredDescriptionDefault
asNoHow to render the body. Default: json (full record).
ridNoStorage request ID. Either rid or url must be provided.
urlNoOriginal crawled URL. Either rid or url must be provided.
use_js_tokenNoQuery the JS token's storage instead of the normal token's. Storage is per-token. Set this to true if the prior crawl response showed token_type=js (i.e. screenshot or JS-rendered pages); otherwise the lookup will return 'Not found'.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes core behavior: retrieval method, precondition, and return format options (as=html returns body, as=markdown converts). No annotations exist, so the description carries the full burden, which it meets for a non-destructive retrieval tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences that efficiently convey purpose, precondition, and key parameter behavior. No redundant information; every word contributes to clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Fully covers what the tool does, when it applies, what it returns, and how to modify output format. No output schema exists, but description sufficiently explains return value. No gaps for a simple retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds value beyond the schema by explaining the effect of the 'as' parameter ('return just the body' for html, 'convert it' for markdown). For rid and url, description reinforces schema hints. No extra for use_js_token, but schema already covers it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Retrieve a single stored page' with specific verb and resource, specifies identification by RID or URL, and includes precondition (previously crawled with store=true). Distinguishes from sibling tools like storage_list or storage_bulk_get.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the precondition for use ('Only works for pages previously crawled with store=true'), providing clear context. Does not explicitly mention alternatives or when not to use, but the precondition effectively limits appropriate contexts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

storage_listA

List RIDs of pages previously crawled with store=true in Crawlbase Cloud Storage. Supports scroll-based pagination (scroll session expires after 15s of inactivity). Returns at most 1000 per call.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax RIDs to return (≤1000).
scrollNoEnable scroll pagination.
scroll_idNoContinuation token from a previous response.
scroll_orderNoSort direction (default desc).
use_js_tokenNoQuery the JS token's storage instead of the normal token's. Set to true if the prior crawl response showed token_type=js.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. Discloses scroll session expiry (15s inactivity) and max 1000 per call. Implies read-only nature (listing RIDs). Could mention rate limits or error scenarios, but covers key behaviors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose, second adds key behavioral constraints (scroll expiry, limit). No redundancy, every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, so description must clarify response format. It mentions listing RIDs and scroll pagination but does not specify whether response includes a scroll_id or other fields. Incomplete for a tool with 5 parameters and no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by mentioning scroll session expiry and max return limit, which are not in parameter descriptions. Provides behavioral context over schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it lists RIDs of stored crawled pages with store=true, using specific verbs and resource naming. Distinct from sibling storage_get (single) and storage_count (count). Includes scroll pagination and limit details.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Describes what the tool does but does not explicitly guide when to use it over alternatives like storage_get or storage_count. Agent must infer usage context from sibling names. Lacks 'when-not-to-use' or explicit differentiation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 9 tool updatesv1.3.0
    • First observedcrawl
    • First observedcrawl_markdown
    • First observedcrawl_screenshot
    • First observedstorage_bulk_delete
    • First observedstorage_bulk_get
    • First observedstorage_count
    • First observedstorage_delete
    • First observedstorage_get
    • First observedstorage_list

TDQS

A4/5.0
Disambiguation5/5

Each tool has a clearly distinct purpose: three crawl tools for different output formats (HTML, markdown, screenshot) and six storage tools for managing stored pages (get, list, delete, count, bulk operations). No overlap or ambiguity.

Naming Consistency5/5

All tool names follow a consistent pattern: 'crawl_*' for crawling actions and 'storage_*' for storage operations. The verbs are clear and the prefix clearly indicates the domain.

Tool Count5/5

9 tools is well-scoped for a web crawling and storage server. It covers crawling with three output formats and full CRUD for storage, without unnecessary bloat or missing essential operations.

Completeness5/5

The tool surface provides comprehensive coverage: crawling with HTML, markdown, and screenshot output; storage with create (via store=true), read (single, bulk, list), delete (single, bulk), and count. No obvious gaps for the intended purpose.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/crawlbase/crawlbase-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server