Skip to main content
Glama
rayss868

Web-curl MCP Server

by rayss868
OverviewSchemaRelated ServersScoreDiscussions
JavaScript
Hybrid

Google Custom Search API

Google Custom Search API is free with usage limits (e.g., 100 queries per day for free, with additional queries requiring payment). For full details on quotas, pricing, and restrictions, see the official documentation.

Web-curl

Web-curl Logo

Developed by Rayss

🚀 Open Source Project
🛠️ Built with Node.js & TypeScript (Node.js v18+ required)

Node.js License Status

🎬 Demo Video

Watch the demo

Click here to watch the demo video directly in your browser.

If your platform supports it, you can also download and play demo/demo_1.mp4 directly.

Related MCP server: Fetch MCP Server

📚 Table of Contents

📝 Changelog / Update History

See CHANGELOG.md for a complete history of updates and new features.

📝 Overview

Web-curl is a powerful tool for fetching and extracting text content from web pages and APIs. Use it as a standalone CLI or as an MCP (Model Context Protocol) server. Web-curl leverages Puppeteer for robust web scraping and supports advanced features such as resource blocking, custom headers, authentication, and Google Custom Search.

✨ Features

🚀 Deep Research & Automation (v1.4.2)

  • Advanced Browser Automation: Full control over Chromium via Puppeteer (click, type, scroll, hover, key presses).

  • Always-On Session Persistence: Browser profiles are now always persistent. Login sessions, cookies, and cache are automatically saved in a local user_data/ directory.

  • Token-Efficient Snapshots:

    • Accessibility Tree: Clean, structured snapshots instead of messy HTML.

    • HTML Slice Mode: Raw HTML with startIndex/endIndex for safe chunking when needed.

    • Viewport Filtering: Automatically filters out elements not visible on screen, saving up to 90% of context tokens on long pages.

  • Chrome DevTools Integration (implemented, but hidden from list_tools):

    • Network Monitoring (browser_network_requests)

    • Console Logs (browser_console_messages)

  • Parallel Search:

    • multi_search: Run multiple Google searches at once (only exposed search tool).

  • Intelligent Resource Management:

    • Idle Auto-Close: Browser automatically shuts down after 15 minutes of inactivity to save RAM/CPU.

    • Tab Rotation: Automatically replaces the oldest tab when the 10-tab limit is reached.

  • Media & Documents:

    • Full-Page Screenshots: Capture high-quality screenshots with a 5-day auto-cleanup lifecycle and custom destination support.

    • Document Parsing: Extract text from PDF and DOCX files directly from URLs.

Storage & Download Details

  • 🗂️ Error log rotation: logs/error-log.txt is rotated when it exceeds ~1MB (renamed to error-log.txt.bak) to prevent unbounded growth.

  • 🧹 Logs & temp cleanup: old temporary files in the logs/ directory are cleaned up at startup.

  • 🛑 Browser lifecycle: Puppeteer browser instances are closed in finally blocks to avoid Chromium temp file leaks.

  • 🔎 Content extraction:

    • Returns raw text, HTML, and Readability "main article" when available. Readability attempts to extract the primary content of a webpage, removing headers, footers, sidebars, and other non-essential elements, providing a cleaner, more focused text.

    • Readability output is subject to startIndex/maxLength/chunkSize slicing when requested.

  • 🚫 Resource blocking: blockResources is now always forced to false, meaning resources are never blocked for faster page loads.

  • ⏱️ Timeout control: navigation and API request timeouts are configurable via tool arguments.

  • 💾 Output: results can be printed to stdout or written to a file via CLI options.

  • ⬇️ Download behavior (download_file):

    • destinationFolder accepts relative paths (resolved against the project root) or absolute paths.

    • The server creates destinationFolder if it does not exist.

    • Downloads are streamed using Node streams + pipeline to minimize memory use and ensure robust writes.

    • Filenames are derived from the URL path (e.g., https://.../path/file.jpg -> file.jpg). If no filename is present, the fallback name is downloaded_file.

    • Overwrite semantics: by default the implementation will overwrite an existing file with the same name.

  • 🖥️ Usage modes: CLI and MCP server (stdin/stdout transport).

  • 🌐 REST client: fetch_api returns JSON/text when appropriate and base64 for binary responses.

  • 🔍 Google Custom Search: requires APIKEY_GOOGLE_SEARCH and CX_GOOGLE_SEARCH.

  • 🤖 Smart command:

    • Auto language detection (franc-min) and optional translation (dynamic translate import).

    • Query enrichment is heuristic-based; results depend on the detected intent.

🏗️ Architecture

This section outlines the high-level architecture of Web-curl.

graph TD
    A[User/MCP Host] --> B(CLI / MCP Server)
    B --> C{Tool Handlers}
    C -- browser_flow --> D["Puppeteer (Web Scraping)"]
    C -- fetch_api --> E["REST Client"]
    C -- multi_search --> F["Google Custom Search API"]
    C -- parse_document --> G["Document Parser (PDF/DOCX)"]
    C -- download_file --> H["File System (Downloads)"]
    D --> I["Web Content"]
    E --> J["External APIs"]
    F --> K["Google Search Results"]
    H --> L["Local Storage"]

  • CLI & MCP Server: src/index.ts Implements both the CLI entry point and the MCP server.

  • Web Scraping: Uses Puppeteer for headless browsing and content extraction.

  • REST Client: src/rest-client.ts Provides a flexible HTTP client for API requests.

⚙️ MCP Server Configuration Example

To integrate web-curl as an MCP server, add the following configuration to your mcp_settings.json:

{
  "mcpServers": {
    "web-curl": {
      "command": "node",
      "args": [
        "build/index.js"
      ],
      "disabled": false,
      "alwaysAllow": [
        "browser_flow",
        "browser_configure",
        "browser_close",
        "multi_search",
        "fetch_api",
        "download_file",
        "parse_document"
      ],
      "env": {
        "APIKEY_GOOGLE_SEARCH": "YOUR_GOOGLE_API_KEY",
        "CX_GOOGLE_SEARCH": "YOUR_CX_ID"
      }
    }
  }
}

🔑 How to Obtain Google API Key and CX

  1. Get a Google API Key:

    • Go to Google Cloud Console.

    • Create/select a project, then go to APIs & Services > Credentials.

    • Click Create Credentials > API key and copy it.

  2. Get a Custom Search Engine (CX) ID:

  3. Enable Custom Search API:

    • In Google Cloud Console, go to APIs & Services > Library.

    • Search for Custom Search API and enable it.

Replace YOUR_GOOGLE_API_KEY and YOUR_CX_ID in the config above.

🛠️ Installation

# Clone the repository
git clone https://github.com/rayss868/MCP-Web-Curl
cd web-curl

# Install dependencies
npm install

# Build the project
npm run build

  • Prerequisites: Ensure you have Node.js (v18+) and Git installed on your system.

Puppeteer installation notes

  • Windows: Just run npm install.

  • Linux / Ubuntu Server: You must install extra dependencies for Chromium to handle rendering and screenshots in a headless environment. Run:

    sudo apt-get update && sudo apt-get install -y \
  fonts-liberation \
  libasound2 \
  libatk-bridge2.0-0 \
  libatk1.0-0 \
  libc6 \
  libcairo2 \
  libcups2 \
  libdbus-1-3 \
  libexpat1 \
  libfontconfig1 \
  libgbm1 \
  libgcc1 \
  libglib2.0-0 \
  libgtk-3-0 \
  libnspr4 \
  libnss3 \
  libpango-1-0-0 \
  libpangocairo-1.0-0 \
  libstdc++6 \
  libx11-6 \
  libx11-xcb1 \
  libxcb1 \
  libxcomposite1 \
  libxcursor1 \
  libxdamage1 \
  libxext6 \
  libxfixes3 \
  libxi6 \
  libxrandr2 \
  libxrender1 \
  libxss1 \
  libxtst6 \
  lsb-release \
  wget \
  xdg-utils

For more details, see the Puppeteer troubleshooting guide.

🚀 Usage

CLI Usage

The CLI supports fetching and extracting text content from web pages.

# Basic usage
node build/index.js https://example.com

# With options
node build/index.js --timeout 30000 https://example.com

# Save output to a file
node build/index.js -o result.json https://example.com

Command Line Options

  • --timeout <ms>: Set navigation timeout (default: 60000)

  • -o <file>: Output result to specified file

MCP Server Usage

Web-curl can be run as an MCP server for integration with Roo Context or other MCP-compatible environments.

Exposed Tools (v1.4.2)

Only the tools below are exposed via list_tools to reduce tool-chaining in agent clients.

  • browser_flow: One-call browser workflow (optional navigate → optional actions → return ONE result).

  • browser_configure: Set proxy/user-agent/viewport (session persistence is always on via user_data/).

  • browser_close: Close browser and tabs (also auto-closes after 15 minutes of inactivity).

  • multi_search: Run multiple Google searches in parallel (the only exposed search entrypoint).

  • fetch_api: REST API request with response truncation (limit).

  • download_file: Download a file from a URL.

  • parse_document: Extract text from PDF/DOCX URLs.

Running as MCP Server

npm run start

The server will communicate via stdin/stdout and expose the tools as defined in src/index.ts.

🚦 HTML Slicing Example (Recommended for Large Pages)

Use browser_flow with result: { type: "snapshot", mode: "html" } when you need raw HTML but want to keep the response small.

Client request for first slice:

{
  "name": "browser_flow",
  "arguments": {
    "result": {
      "type": "snapshot",
      "mode": "html",
      "startIndex": 0,
      "endIndex": 20000
    }
  }
}

Response (example):

{
  "mode": "html",
  "totalLength": 123456,
  "startIndex": 0,
  "endIndex": 20000,
  "remainingCharacters": 103456,
  "content": "<html>...first slice...</html>"
}

🧩 Configuration

  • Session Persistence: Always enabled. Logins and cookies are automatically reused across restarts.

  • Timeout: Set navigation and API request timeouts.

  • Environment Variables: Used for Google Search API integration (used by multi_search).

💡 Examples {#examples}

{
  "name": "fetch_api",
  "arguments": {
    "url": "https://api.github.com/repos/nodejs/node",
    "method": "GET",
    "headers": {
      "Accept": "application/vnd.github.v3+json"
    },
    "limit": 10000
  }
}
{
  "name": "download_file",
  "arguments": {
    "url": "https://example.com/image.jpg",
    "destinationFolder": "downloads"
  }
}

Note: destinationFolder can be either a relative path (resolved against the project root) or an absolute path. The server will create the destination folder if it does not exist.

{
  "name": "browser_configure",
  "arguments": {
    "proxy": "http://proxy.example.com:8080",
    "viewport": { "width": 1920, "height": 1080 }
  }
}

Note: Session persistence is always enabled. Cookies and login sessions are automatically stored in the user_data/ directory.

🛠️ Troubleshooting {#troubleshooting}

  • Timeout Errors: Increase the timeout parameter if requests are timing out.

  • Google Search Fails: Ensure APIKEY_GOOGLE_SEARCH and CX_GOOGLE_SEARCH are set in your environment.

  • Error Logs: Check the logs/error-log.txt file for detailed error messages.

🧠 Tips & Best Practices {#tips--best-practices}

  • For large pages, use maxLength and startIndex to fetch content in slices.

  • Always validate your tool arguments to avoid errors.

  • Secure your API keys and sensitive data using environment variables.

  • Review the MCP tool schemas in src/index.ts for all available options.

🤝 Contributing & Issues {#contributing--issues}

Contributions are welcome! If you want to contribute, fork this repository and submit a pull request.
If you find any issues or have suggestions, please open an issue on the repository page.

📄 License & Attribution {#license--attribution}

This project was developed by Rayss.
For questions, improvements, or contributions, please contact the author or open an issue in the repository.

Note: Google Search API is free with usage limits. For details, see: Google Custom Search API Overview

Available Tools

5 tools
download_fileC

Download a file from a given URL to a specified folder.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL of the file to download.
destinationFolderYesThe destination folder (relative to the workspace directory) to save the file.

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions downloading a file but fails to address critical aspects like authentication needs, rate limits, error handling, file size constraints, or network behavior. This leaves significant gaps in understanding the tool's operational traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's function without any unnecessary words. It is front-loaded and appropriately sized, making it easy to understand at a glance while avoiding redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a download operation with no annotations and no output schema, the description is insufficient. It lacks details on return values, error conditions, security considerations, and performance implications, which are crucial for effective tool invocation in an AI agent context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, clearly documenting both parameters ('url' and 'destinationFolder'). The description adds minimal value beyond the schema, as it only reiterates the purpose without providing additional syntax, format details, or constraints. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Download a file') and the target resource ('from a given URL to a specified folder'), which is specific and unambiguous. However, it does not explicitly differentiate from sibling tools like 'fetch_api' or 'fetch_webpage', which might have overlapping functionality, so it doesn't reach the highest score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as 'fetch_api' or 'fetch_webpage'. It lacks context about prerequisites, exclusions, or specific use cases, offering only a basic functional statement without comparative or situational advice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_apiC

Make a REST API request with various methods, headers, and body.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL for the API endpoint.
methodYesHTTP method for the request.
headersNoRequest headers (e.g., for authorization).
bodyNoRequest body (JSON object, string, etc.).
timeoutNoRequest timeout in milliseconds (default: 60000).
limitYesMaximum number of characters to return in the response body (required).
redirectNoRedirect mode for the request (default: follow).

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions making requests with 'various methods, headers, and body' but lacks critical details: it doesn't specify authentication requirements, rate limits, error handling, response format, or side effects. For a general-purpose API tool with 7 parameters, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core functionality. Every word earns its place without redundancy or fluff, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (7 parameters, no annotations, no output schema), the description is insufficient. It doesn't explain what the tool returns, how errors are handled, or practical constraints like authentication. For a versatile API tool, more context is needed to use it effectively beyond the parameter definitions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema—it mentions 'various methods, headers, and body' which are covered in the schema's enum and descriptions. No additional parameter semantics are provided, meeting the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose as making REST API requests with various methods, headers, and body. It specifies the verb ('Make a REST API request') and resource ('API endpoint'), but doesn't explicitly differentiate from sibling tools like fetch_webpage or download_file, which might have overlapping functionality for HTTP requests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like fetch_webpage or download_file, nor does it specify use cases, prerequisites, or exclusions. The agent must infer usage from the tool name and parameters alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_webpageC

Retrieve text content from a web page

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL of the webpage to fetch
blockResourcesNoWhether to block images, stylesheets, and fonts to improve performance (default: true)
resourceTypesToBlockNoList of resource types to block (e.g., "image", "stylesheet", "font")
timeoutNoNavigation timeout in milliseconds (default: 120000)
maxLengthNoMaximum number of characters to return (default: 10000).
startIndexYesStart character index for content extraction (required; default: 0).
headersNoCustom headers to include in the request
usernameNoUsername for basic authentication
passwordNoPassword for basic authentication
nextPageSelectorNoCSS selector for next page button/link (for auto-pagination, optional)
maxPagesNoMaximum number of pages to crawl (for auto-pagination, optional, default: 1)
evaluateScriptNoJavaScript code to execute on the page after loading. The result of the script will be returned.

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states the basic operation. It doesn't disclose important behavioral traits like whether this makes network requests, has rate limits, requires authentication for certain sites, handles errors, returns structured data, or supports pagination beyond what's implied in parameters. The description is minimal compared to the tool's complexity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 12 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what 'text content' means (HTML, plain text, structured data?), doesn't mention error handling, doesn't describe return format, and provides no context about the tool's capabilities beyond the minimal purpose statement.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 12 parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema descriptions. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose as 'Retrieve text content from a web page' with a specific verb ('Retrieve') and resource ('text content from a web page'). It distinguishes from siblings like 'download_file' (which likely downloads files) and 'fetch_api' (which likely calls APIs), but doesn't explicitly differentiate from 'google_search' or 'smart_command'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'fetch_api' or 'google_search'. There's no mention of prerequisites, limitations, or typical use cases beyond the basic purpose statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smart_commandC

Free-form command: automatically fetch if a link is detected, automatically search if a search query is detected.

ParametersJSON Schema
NameRequiredDescriptionDefault
commandYesFree-form user instruction

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions automatic detection and actions (fetching links, searching queries), but lacks details on error handling, rate limits, authentication needs, or what constitutes a 'link' or 'search query'. This leaves significant gaps in understanding the tool's behavior and constraints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded, consisting of a single sentence that directly states the tool's function. It avoids unnecessary words, but could be more structured by explicitly separating the link and query cases or adding brief examples to enhance clarity without sacrificing brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a tool that handles multiple actions (fetching and searching) and lacks annotations and an output schema, the description is incomplete. It does not explain return values, error conditions, or how the detection logic works, leaving the AI agent with insufficient context to use the tool effectively compared to its more specific siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with the parameter 'command' documented as a 'free-form user instruction'. The description adds marginal value by reiterating the auto-fetch and auto-search behavior, but does not provide additional semantics beyond what the schema already states, such as examples or format details. Baseline 3 is appropriate given the high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool's purpose as a 'free-form command' that automatically fetches links or searches queries, which is clear but vague. It specifies the verb ('fetch', 'search') and resource types ('link', 'query'), but does not distinguish it from sibling tools like 'fetch_webpage' or 'google_search', leaving ambiguity about when to use this versus those specific tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides minimal guidance by stating it triggers based on detecting links or search queries, but it does not specify when to use this tool versus the sibling tools (e.g., 'fetch_webpage' for links, 'google_search' for queries). There are no explicit alternatives, exclusions, or context for usage, offering little help for an AI agent in tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 5 tool updatesv1.0.0
    • Addeddownload_file
    • Changedfetch_api5 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / description
        Added value: +"HTTP API request (GET/POST/etc), custom header/body, timeout, debug mode for verbose output/logging."
      • addedInput schema / properties / limit
        Added value: +{
+  "description": "Maximum number of characters to return in the response body (required).",
+  "type": "number"
+}
      • addedInput schema / properties / redirect
        Added value: +{
+  "description": "Redirect mode for the request (default: follow).",
+  "enum": [
+    "follow",
+    "error",
+    "manual"
+  ],
+  "type": "string"
+}
      • changedInput schema / required
        Previous value: -[
-  "url",
-  "method"
-]New value: +[
+  "url",
+  "method",
+  "limit"
+]
    • Changedfetch_webpage9 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / description
        Added value: +"Fetch web content (text, html, mainContent, metadata, supports multi-page crawling, and AI-friendly regex extraction). Debug option for verbose output/logging."
      • addedInput schema / properties / evaluateScript
        Added value: +{
+  "description": "JavaScript code to execute on the page after loading. The result of the script will be returned.",
+  "type": "string"
+}
      • changedInput schema / properties / maxLength / description
        Previous value: -"Maximum number of characters to return for content extraction (default: 2000 if not provided)"New value: +"Maximum number of characters to return (default: 10000)."
      • addedInput schema / properties / maxPages
        Added value: +{
+  "description": "Maximum number of pages to crawl (for auto-pagination, optional, default: 1)",
+  "type": "number"
+}
      • addedInput schema / properties / nextPageSelector
        Added value: +{
+  "description": "CSS selector for next page button/link (for auto-pagination, optional)",
+  "type": "string"
+}
      • changedInput schema / properties / startIndex / description
        Previous value: -"Start character index for content extraction (default: 0)"New value: +"Start character index for content extraction (required; default: 0)."
      • changedInput schema / properties / timeout / description
        Previous value: -"Navigation timeout in milliseconds (default: 60000)"New value: +"Navigation timeout in milliseconds (default: 120000)"
      • changedInput schema / required
        Previous value: -[
-  "url"
-]New value: +[
+  "url",
+  "startIndex"
+]
    • Changedgoogle_search3 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / description
        Added value: +"Search the web using Google Custom Search API."
      • changedInput schema / properties / query / description
        Previous value: -"Search query"New value: +"Search query, including any operators like site:, filetype:, etc."
    • Changedsmart_command3 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / description
        Added value: +"Free-form command: auto fetch if link detected, auto search if query. Debug option for verbose output/logging."
      • changedInput schema / properties / command / description
        Previous value: -"Instruksi bebas dari user"New value: +"Free-form user instruction"
  2. 4 tool updates
    • First observedfetch_api
    • First observedfetch_webpage
    • First observedgoogle_search
    • First observedsmart_command
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

C2.9/5.0
Disambiguation3/5

There is significant overlap between fetch_api, fetch_webpage, and download_file, as all three involve retrieving content from URLs, with fetch_api being a general-purpose tool that could subsume the others. Smart_command further blurs boundaries by automatically choosing between fetching and searching, potentially causing confusion about when to use the dedicated tools. However, google_search is clearly distinct for search operations.

Naming Consistency2/5

The naming is inconsistent with mixed conventions: fetch_api and fetch_webpage use a verb_noun pattern, download_file uses verb_noun but with a different verb style, google_search is a compound noun, and smart_command uses an adjective_noun pattern. This lack of a predictable naming scheme makes the tool set harder to navigate and remember.

Tool Count5/5

With 5 tools, the count is well-scoped for a web/curl server, covering key operations like downloading files, making API requests, fetching web content, searching, and a smart utility. Each tool appears to serve a purpose without obvious bloat or thinness, fitting the server's domain appropriately.

Completeness4/5

The tool set covers core web interaction tasks: downloading files, API requests, webpage fetching, and web searching, with smart_command adding flexibility. Minor gaps might include more advanced operations like handling cookies or sessions, but the surface is largely complete for basic web utilities, allowing agents to perform common workflows without major dead ends.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • MCP Webscan Server

    bsmi021
    A
    license
    A
    quality
    D
    maintenance
    Enables web content scanning and analysis by fetching, analyzing, and extracting information from web pages using tools like page fetching, link extraction, site crawling, and more.
    6
    13
    MIT

  • Fetch MCP Server

    phpmac
    A
    license
    Not graded
    quality
    C
    maintenance
    Provides functionality to fetch and transform web content in various formats (HTML, JSON, plain text, and Markdown) through simple API calls.
    102,856
    1
    MIT

  • FreeCrawl MCP Server

    dylan-gluck
    A
    license
    Not graded
    quality
    D
    maintenance
    Enables web scraping and document processing with JavaScript execution, anti-detection measures, batch processing, and structured data extraction. Supports multiple formats including markdown, HTML, screenshots, and handles PDFs with OCR capabilities.
    4
    MIT

  • Web Search MCP Server

    Mantraa-Zzz
    F
    license
    B
    quality
    D
    maintenance
    Enables web searching and content scraping through Google Custom Search API. Provides tools to search the internet, extract webpage content, and automatically scrape search results for comprehensive information gathering.
    3
    -

Appeared in Searches

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/rayss868/MCP-Web-Curl'

If you have feedback or need assistance with the MCP directory API, please join our Discord server