Skip to main content
Glama
naku111

Web Scraper MCP Server

by naku111
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Hybrid

Web Scraper MCP Server

基于 Model Context Protocol (MCP) 的 TypeScript 网页爬取服务器，内置多种常用爬取与命令工具。

使用npx配置MCP（无需clone代码）

"web-scraper-puppeteer": {
  "command": "npx",
  "args": [
    "-y",
    "web-scraper-mcp-puppeteer",
    "mcpserver-puppeteer-nakiwang"
  ]
}

Related MCP server: ScrapeGraph MCP Server

功能特性

  • 🚀 多种导出格式: 支持将爬取内容导出为 Markdown、Text、HTML 和 JSON 格式

  • 🎭 Puppeteer 无头浏览器: 解决未实现静态化的 SPA 单页应用渲染问题

  • 📋 规则集支持: 支持设置规则集，实现抓取网页的特定部分内容

  • 🔧 自定义域名 Headers: 支持设置自定义域名 Headers，绕过网站登录限制

  • 📦 批量爬取: 支持同时爬取多个 URL

  • 🎯 智能内容提取: 自动识别和提取网页主要内容

安装

# 克隆项目
git clone <repository-url>
cd web-scraper-mcp

# 安装依赖
npm install

# 构建项目
npm run build

使用方法

启动服务器

npm start

可用工具

1. scrape_url - 爬取单个网页

{
  "name": "scrape_url",
  "arguments": {
    "url": "https://example.com",
    "format": "markdown",
    "usePuppeteer": false,
    "ruleSet": "blog",
    "customHeaders": {
      "Authorization": "Bearer token"
    }
  }
}

参数说明:

  • url (必需): 要爬取的网页 URL

  • format (可选): 导出格式，支持 markdown、text、html、json，默认为 markdown

  • usePuppeteer (可选): 是否使用 Puppeteer 无头浏览器，默认为 false

  • ruleSet (可选): 规则集名称，用于提取特定内容

  • customHeaders (可选): 自定义请求头

2. create_rule_set - 创建内容提取规则集

{
  "name": "create_rule_set",
  "arguments": {
    "name": "my_blog_rule",
    "rules": {
      "title": "h1, .post-title",
      "content": ".post-content, article",
      "links": ".post-content a",
      "images": ".post-content img",
      "exclude": ["nav", "footer", ".sidebar"]
    }
  }
}

参数说明:

  • name (必需): 规则集名称

  • rules (必需): 提取规则配置

    • title: 标题选择器

    • content: 内容选择器

    • links: 链接选择器

    • images: 图片选择器

    • exclude: 要排除的选择器列表

3. set_domain_headers - 设置域名请求头

{
  "name": "set_domain_headers",
  "arguments": {
    "domain": "example.com",
    "headers": {
      "Authorization": "Bearer your-token",
      "X-Custom-Header": "custom-value"
    }
  }
}

4. batch_scrape - 批量爬取

{
  "name": "batch_scrape",
  "arguments": {
    "urls": [
      "https://example1.com",
      "https://example2.com",
      "https://example3.com"
    ],
    "format": "markdown",
    "usePuppeteer": false,
    "ruleSet": "blog"
  }
}

预定义规则集

系统内置了以下规则集：

  • blog: 适用于大多数博客文章页面

  • news: 适用于新闻网站文章

  • product: 适用于电商产品页面

  • documentation: 适用于技术文档页面

  • forum: 适用于论坛帖子页面

导出格式

Markdown

将网页内容转换为 Markdown 格式，包含标题、内容、链接和图片。

Text

纯文本格式，去除所有 HTML 标签。

HTML

清理后的 HTML 格式，移除脚本和样式。

JSON

完整的结构化数据，包含所有提取的信息。

开发

项目结构

src/
├── index.ts              # 主服务器文件
├── scraper/
│   └── webScraper.ts     # 网页爬取器
├── export/
│   └── exportManager.ts  # 导出管理器
├── rules/
│   └── ruleEngine.ts     # 规则引擎
├── headers/
│   └── headerManager.ts  # 请求头管理器
└── utils/
    └── logger.ts         # 日志工具

开发模式

# 监听文件变化并自动重新构建
npm run dev

测试

npm test

配置

环境变量

  • LOG_LEVEL: 日志级别 (debug, info, warn, error)

  • PUPPETEER_TIMEOUT: Puppeteer 超时时间（毫秒）

  • REQUEST_TIMEOUT: HTTP 请求超时时间（毫秒）

自定义配置

可以通过修改源代码来自定义：

  1. 默认请求头: 编辑 src/headers/headerManager.ts

  2. 预定义规则集: 编辑 src/rules/ruleEngine.ts

  3. 导出格式: 编辑 src/export/exportManager.ts

常见问题

Q: 为什么某些网站爬取失败？

A: 可能是因为网站有反爬虫机制。尝试：

  1. 使用 usePuppeteer: true 启用无头浏览器

  2. 设置合适的自定义请求头

  3. 使用预定义的反检测请求头

Q: 如何处理需要登录的网站？

A: 使用 set_domain_headers 工具设置包含认证信息的请求头，如 Cookie 或 Authorization。

Q: SPA 应用内容为空怎么办？

A: 设置 usePuppeteer: true 使用 Puppeteer 渲染动态内容。

许可证

MIT License

贡献

欢迎提交 Issue 和 Pull Request！

Available Tools

7 tools
cleanup_imagesC

清理所有下载的图片文件

ParametersJSON Schema
NameRequiredDescriptionDefault
outputDirNo要清理的图片目录路径（可选，默认为 ./scraped-images）

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden but only states the action ('清理' meaning clean/delete). It doesn't disclose critical behavioral traits: whether this is destructive (likely yes, but not confirmed), if it requires specific permissions, what happens on failure, or if it's reversible. The description adds minimal value beyond the basic purpose.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence in Chinese ('清理所有下载的图片文件') that directly states the tool's purpose with zero wasted words. It's appropriately sized and front-loaded, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's likely destructive nature (cleaning/deleting files), no annotations, and no output schema, the description is incomplete. It lacks details on safety (e.g., confirmation prompts, error handling), output (what's returned), and integration with siblings (e.g., works with 'scrape_images'). For a mutation tool, this is inadequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the parameter 'outputDir' well-documented in the schema. The description doesn't add any parameter-specific information beyond implying a default location (though not explicitly stated). This meets the baseline of 3 since the schema handles the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description '清理所有下载的图片文件' clearly states the action (clean/delete) and target resource (downloaded image files). It distinguishes from siblings like 'cleanup_texts' (different resource) and 'list_images' (different action). However, it doesn't specify the scope beyond 'all' or mention the optional directory parameter, keeping it from a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., after scraping), exclusions (e.g., avoid if images are needed), or relationships to siblings like 'scrape_images' (which might create the files to clean). The agent must infer usage from context alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cleanup_textsC

清理所有提取的文本文件

ParametersJSON Schema
NameRequiredDescriptionDefault
outputDirNo要清理的文本目录路径（可选，默认为 ./scraped-text）

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states '清理' (clean up), which implies a destructive operation, but doesn't clarify what 'clean up' entails (e.g., deletion, archiving, or moving files), whether it's reversible, or any permissions/rate limits. This leaves significant gaps for a tool that likely modifies or removes files.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple tool and front-loaded with the core purpose, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's likely destructive nature (implied by 'clean up'), no annotations, and no output schema, the description is incomplete. It should address behavioral aspects like what 'clean up' does to files, any confirmation prompts, or error handling. The simplicity of the tool (1 optional parameter) doesn't compensate for these gaps in safety and outcome clarity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with the single parameter 'outputDir' fully documented in the schema. The description adds no parameter-specific information beyond what the schema provides (e.g., it doesn't explain the default path './scraped-text' or typical use cases for overriding it). Baseline 3 is appropriate as the schema handles the parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('清理' meaning 'clean up') and the resource ('所有提取的文本文件' meaning 'all extracted text files'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'cleanup_images' or 'list_texts', which would require mentioning it's specifically for text files (not images) and that it performs cleanup (not listing).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., after scraping), when-not-to-use scenarios (e.g., if you need to preserve files), or explicit alternatives among siblings like 'list_texts' for viewing files or 'cleanup_images' for image cleanup.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_scraping_statusC

获取爬虫状态信息

ParametersJSON Schema
NameRequiredDescriptionDefault
outputDirNo输出目录路径（可选）

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states what the tool does ('get crawler status information') without describing traits like whether it's read-only (implied but not explicit), what the output format might be (e.g., JSON, text), if it requires authentication, or if it has side effects (e.g., logging). For a tool with no annotation coverage, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence ('获取爬虫状态信息') that directly states the tool's purpose without any wasted words. It is appropriately front-loaded and concise, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (involves status retrieval), no annotations, no output schema, and sibling tools that suggest a scraping context, the description is incomplete. It doesn't explain what 'status' entails (e.g., active jobs, errors, metrics), how results are returned, or how it fits with tools like 'scrape_images'. For a tool in this environment, more context is needed to ensure proper use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 1 parameter with 100% coverage (outputDir described as '输出目录路径（可选）' - output directory path, optional), so the schema does the heavy lifting. The description adds no parameter-specific information beyond the tool's overall purpose. With high schema coverage and only one optional parameter, a baseline of 3 is appropriate, but the simplicity (0 required params) and clear schema justify a 4, as minimal additional semantics are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description '获取爬虫状态信息' (Get crawler status information) states a clear verb ('获取' - get) and resource ('爬虫状态信息' - crawler status information), providing basic purpose. However, it doesn't distinguish this tool from potential siblings like 'list_images' or 'list_texts' that might also retrieve status-like information, nor does it specify what aspects of 'status' are included (e.g., running state, progress, errors).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., only after starting a scrape), exclusions (e.g., not for historical data), or comparisons to sibling tools like 'scrape_images' or 'cleanup_images'. Without such context, an agent must infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_imagesC

列出所有已下载的图片信息

ParametersJSON Schema
NameRequiredDescriptionDefault
outputDirNo图片目录路径（可选，默认为 ./scraped-images）

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but offers minimal behavioral insight. It implies a read-only operation by using '列出' (list), but doesn't disclose format of returned information, pagination, error conditions, or performance characteristics. The description adds little beyond the basic action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence with zero wasted words. It's appropriately sized for a simple listing tool and front-loads the core purpose immediately. Every word earns its place in conveying the essential function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description is insufficient. It doesn't explain what '图片信息' (image information) includes (e.g., filenames, sizes, metadata), how results are structured, or any limitations. Given the context of sibling tools for scraping and cleanup, more guidance on this tool's role would be valuable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with one optional parameter clearly documented in the schema. The description adds no parameter-specific information beyond what the schema provides, so it meets the baseline of 3 where the schema does the heavy lifting for parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('列出' - list) and resource ('所有已下载的图片信息' - all downloaded image information), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_texts' beyond the resource type, nor does it specify if this includes metadata vs. just filenames.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'scrape_images' (for downloading) or 'cleanup_images' (for deletion). It doesn't mention prerequisites, such as whether images need to be downloaded first, or context for when listing is appropriate versus other operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_textsC

列出所有已提取的文本文件

ParametersJSON Schema
NameRequiredDescriptionDefault
outputDirNo文本目录路径（可选，默认为 ./scraped-text）

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states it lists files but doesn't describe the return format (e.g., list of filenames, metadata), pagination, error handling, or whether it's a read-only operation. This leaves significant gaps for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence in Chinese that directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple tool, though it could be slightly more structured (e.g., by including usage context).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the output looks like (e.g., format, structure), error conditions, or behavioral details like whether it's safe to call repeatedly. This is inadequate given the lack of structured data to compensate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with the single parameter 'outputDir' documented as optional with a default path. The description doesn't add any parameter semantics beyond what the schema provides, so the baseline score of 3 is appropriate given the schema handles the parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('列出所有' - list all) and the resource ('已提取的文本文件' - extracted text files). It's specific about what it does, though it doesn't explicitly differentiate from sibling tools like 'list_images' or 'scrape_text'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_images' for images or 'scrape_text' for scraping new text, nor does it specify prerequisites or appropriate contexts for use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scrape_imagesC

从指定URL爬取网站上的所有图片并保存到本地

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes要爬取图片的网站URL
outputDirNo图片保存目录（可选，默认为 ./scraped-images）
maxConcurrentNo并发下载数量（可选，默认为5）

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions saving images locally but fails to describe critical traits like error handling, rate limits, authentication needs, file formats supported, or whether it overwrites existing files. This leaves significant gaps for safe and effective use.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action and outcome without unnecessary details. However, it could be slightly improved by structuring key behavioral information, but it earns high marks for zero waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a web scraping tool with no annotations and no output schema, the description is incomplete. It omits essential context such as return values, error conditions, performance constraints, and how it interacts with sibling tools, making it inadequate for reliable agent use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters (url, outputDir, maxConcurrent) with their types and defaults. The description adds no additional meaning beyond implying the tool processes images from a URL, which the schema's url description already covers, resulting in a baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('scrape images from a specified URL') and the outcome ('save to local'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'scrape_text' or 'list_images' beyond the resource type (images vs. text), which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'scrape_text' for text content or 'list_images' for listing without saving. It also lacks context about prerequisites, such as network access or file system permissions, leaving usage scenarios unclear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scrape_textC

从指定URL爬取网站的文本内容并保存为Markdown文件

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes要爬取文本的网站URL
outputDirNo文本保存目录（可选，默认为 ./scraped-text）

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions saving as a Markdown file but doesn't cover important aspects like error handling, rate limits, authentication needs, what happens if the URL is inaccessible, or whether this is a read-only or destructive operation. For a web scraping tool with zero annotation coverage, this leaves significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that communicates the core functionality without unnecessary words. It's appropriately sized and front-loaded with the main action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a web scraping tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., success/failure status, file path), error conditions, or behavioral constraints. Given the complexity of web scraping operations, more context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents both parameters thoroughly. The description doesn't add any meaningful semantic information beyond what's in the schema (e.g., it doesn't explain URL format requirements or output directory behavior). Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('scrape text from a specified URL') and the outcome ('save as a Markdown file'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'scrape_images' or 'list_texts', which would be needed for a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'scrape_images' or 'list_texts', nor does it mention any prerequisites or exclusions. It simply states what the tool does without contextual usage information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 7 tool updates
    • First observedcleanup_images
    • First observedcleanup_texts
    • First observedget_scraping_status
    • First observedlist_images
    • First observedlist_texts
    • First observedscrape_images
    • First observedscrape_text
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A3.5/5.0
Disambiguation5/5

Every tool has a clearly distinct purpose with no ambiguity. The tools are cleanly divided between scraping operations (scrape_images, scrape_text), listing operations (list_images, list_texts), cleanup operations (cleanup_images, cleanup_texts), and status monitoring (get_scraping_status). There is no overlap in functionality.

Naming Consistency5/5

The naming follows a perfectly consistent verb_noun pattern throughout. All tools use snake_case with clear action prefixes (scrape_, list_, cleanup_, get_) followed by specific resource nouns (images, texts, scraping_status). There are no deviations in naming conventions.

Tool Count5/5

The 7 tools are well-scoped for a web scraping server. Each tool earns its place by covering distinct aspects of the scraping workflow: initiating scrapes, listing results, cleaning up files, and checking status. This is an appropriate number for the domain without being too thin or bloated.

Completeness4/5

The tool surface provides excellent coverage for core web scraping operations with clear CRUD-like patterns (create via scrape, read via list, delete via cleanup). The only minor gap is the lack of update operations for existing scraped content, but agents can work around this by re-scraping. Status monitoring is included, which is valuable for workflow management.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • ScrapeGraph MCP Server by ScrapeGraphAI

    ScrapeGraph MCP Serverofficial

    ScrapeGraphAI
    A
    license
    A
    quality
    B
    maintenance
    A production-ready Model Context Protocol server that enables language models to leverage AI-powered web scraping capabilities, offering tools for transforming webpages to markdown, extracting structured data, and executing AI-powered web searches.
    8
    99
    MIT

  • Firecrawl MCP Server

    ampcome-mcps
    A
    license
    A
    quality
    D
    maintenance
    A Model Context Protocol server that enables web scraping, crawling, and content extraction capabilities through integration with Firecrawl.
    8
    40,139
    2
    MIT

Appeared in Searches

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/naku111/mcpServer'

If you have feedback or need assistance with the MCP directory API, please join our Discord server