Skip to main content
Glama
KhaledDev

yet-another-web-scraper-mcp

by KhaledDev

Yet another web scraper MCP

A minimal MCP server that lets an LLM drive a real Chromium browser to scrape websites, collect structured rows, and export them to CSV.

Setup

uv sync
uv run playwright install chromium

Related MCP server: Ruishu MCP

Running

uv run main.py

This starts the MCP server over stdio. Point your MCP client (e.g. Claude Desktop / Claude Code) at uv run --directory <this folder> main.py.

No need to clone this repo. The npm package still runs the Python implementation under the hood via uv (which it will tell you how to install if it's missing), but gives a familiar one-line install/run command that works the same on Windows, macOS, and Linux:

npx -y yet-another-web-scraper-mcp

Add it to your MCP client config, e.g. Claude Desktop's claude_desktop_config.json:

{
  "mcpServers": {
    "web-scraper": {
      "command": "npx",
      "args": ["-y", "yet-another-web-scraper-mcp"]
    }
  }
}

On first run, npm install triggers a postinstall step that runs uv sync and downloads the Chromium browser for Playwright. This can take a minute the very first time.

Requires uv and Python 3.13+ to be resolvable on the machine; npm alone cannot provide the Python runtime.

Adding it to CLI coding agents

Claude Code

claude mcp add web-scraper -- npx -y yet-another-web-scraper-mcp

Codex CLI: add to ~/.codex/config.toml:

[mcp_servers.web-scraper]
command = "npx"
args = ["-y", "yet-another-web-scraper-mcp"]

Gemini CLI: add to ~/.gemini/settings.json (or a project's .gemini/settings.json):

{
  "mcpServers": {
    "web-scraper": {
      "command": "npx",
      "args": ["-y", "yet-another-web-scraper-mcp"]
    }
  }
}

Cursor: add to .cursor/mcp.json (project-level) or ~/.cursor/mcp.json (global):

{
  "mcpServers": {
    "web-scraper": {
      "command": "npx",
      "args": ["-y", "yet-another-web-scraper-mcp"]
    }
  }
}

Windsurf: same mcpServers JSON shape as Cursor above, in Windsurf's MCP config panel or ~/.codeium/windsurf/mcp_config.json.

Any other MCP-compatible client that takes a raw command/args pair (stdio transport) can point at the same npx -y yet-another-web-scraper-mcp command; the JSON snippets above are all equivalent to that one line.

Tools

  • open_browser(headless=True) / close_browser(): start/stop the browser.

  • navigate(url): go to a URL, returns cleaned page content.

  • click(selector): click a CSS-selected element (e.g. a "next page" button), returns the resulting cleaned content.

  • get_page_content(max_length): re-read the current page's HTML, with <script> and <svg> tags stripped out since they're noise for an LLM.

  • get_page_links(): list {text, href} for links on the page.

  • add_record(record): append one scraped row (dict of column -> value).

  • get_records(): review everything collected so far.

  • export_csv(filename): write the collected rows to a CSV via pandas.

Typical flow: open_browser -> navigate -> get_page_content / get_page_links -> click to page through results -> add_record per item -> export_csv when done -> close_browser.

Available Tools

15 tools
add_recordA

Add one row of scraped data (column name -> value) to the in-memory dataset.

ParametersJSON Schema
NameRequiredDescriptionDefault
recordYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden for behavioral disclosure. It does reveal that the tool mutates the in-memory dataset and highlights the in-memory nature, but it does not mention return values, validation behavior, or error handling, which leaves some behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-front-loaded sentence with no redundant information. Every word contributes to clarity, making it concise and to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool, the description covers the essential aspects: what it does (adds a row), the data format (column name -> value), and the destination (in-memory dataset). It falls slightly short on edge cases, but the low complexity and presence of an output schema reduce the need for more detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides only the parameter name 'record' with an object type and string values, with 0% description coverage. The description's 'column name -> value' explanation adds meaningful semantic guidance, effectively compensating for the sparse schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool adds one row of scraped data to the in-memory dataset, using a specific verb ('Add') and resource ('row of scraped data'). It distinguishes itself from siblings like get_records, export_csv, and clear_records, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for accumulating scraped rows during a scraping workflow, and the mention of 'in-memory dataset' clarifies that data is not persisted. No alternatives are needed since no other sibling tool performs this add function, and the guidance is clear enough for the simple operation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

clear_recordsA

Discard all rows collected so far. Use this when starting over, e.g. you've been exploring one source and want to switch to another without mixing rows.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description carries the full burden. 'Discard all rows' is direct and implies destructiveness, but it doesn't explicitly state whether the action is permanent/irreversible or whether it affects persistent storage. More transparency could include 'this cannot be undone' or 'clears only the current session's rows.'

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no wasted words. The first sentence states the action, the second explains when to use it. Perfectly front-loaded and concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple no-parameter tool with an output schema, the description explains both the function and the appropriate context. It is complete, covering the essential information an agent needs to decide when to invoke it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, and the schema is empty with 100% coverage. Per the rubric, a 0-param tool gets a baseline of 4. The description doesn't need to add parameter details, and none are added.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: discarding all rows collected so far. The verb 'discard' combined with the resource 'rows' is specific and distinguishes it from sibling tools like add_record or get_records.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides a clear use case: 'when starting over, e.g. you've been exploring one source and want to switch to another without mixing rows.' This gives explicit contextual guidance, though it doesn't explicitly mention when not to use it or name alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

clickA

Click an element by CSS selector (e.g. a pagination button) and return the resulting page's cleaned content.

On stateful pages (large SPA/ASP.NET markup) re-returning the whole page every click is wasteful. Pass result_selector (a CSS selector, e.g. "table#resultsGrid tbody") to get back only that part of the post-click page.

If the click triggers a client-side fetch (e.g. a filter or "load more" button) rather than a full reload, pass wait_for_selector/wait_for_idle the same way as navigate so the returned content isn't read too early.

ParametersJSON Schema
NameRequiredDescriptionDefault
selectorYes
timeout_msNo
wait_for_idleNo
result_selectorNo
wait_for_selectorNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden and discloses that clicking returns cleaned content, that re-returning the full page is wasteful on stateful pages, and that waiting is needed for client-side fetches to avoid early reads. This provides useful operational context beyond the basic action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is densely informative, with each sentence adding meaningful guidance and examples. It is well-structured, starting with the main action and then providing scenario-based refinements for stateful pages and client-side fetches.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema and the tool's complexity, the description provides sufficient context for correct invocation, covering return content scoping and async waiting. The only minor gap is `timeout_ms`, but the overall guidance is comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description explains the meaning and use cases for `selector`, `result_selector`, `wait_for_selector`, and `wait_for_idle`, critical since schema coverage is 0%. However, `timeout_ms` is not addressed, leaving its semantics to inference.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool clicks an element by CSS selector and returns the resulting page's cleaned content, which distinguishes it from navigation and extraction siblings. The specific verb 'Click' and resource 'element by CSS selector' make the purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use `result_selector` (stateful pages) and `wait_for_selector`/`wait_for_idle` (client-side fetch), with concrete examples. However, it does not explicitly contrast with alternatives like `navigate`, though the mention of 'navigate' implies a sibling relationship.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

close_browserA

Close the Chromium browser instance.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. The description only states the action 'Close the Chromium browser instance' but does not disclose side effects, whether it closes all tabs, or if it can be called multiple times.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, directly to the point, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool, the description covers the core function adequately. However, it lacks details on when to use and behavioral effects, though the simplicity of the tool mitigates this gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the schema coverage is 100% trivially. The description does not need to elaborate on parameters; the empty schema is self-explanatory.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly specifies the verb 'Close' and the resource 'Chromium browser instance', making it unambiguous and distinct from sibling tools like open_browser.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to close the browser, prerequisites, or alternatives. It only states the action without context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

export_csvA

Write all collected rows to a CSV file (via pandas) and return its path.

ParametersJSON Schema
NameRequiredDescriptionDefault
filenameNoscraped_data.csv

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the main action (writing to CSV) and the return value (path), but does not mention potential side effects like overwriting existing files or behavior when no rows are collected. This is moderate disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that front-loads the primary action ('Write all collected rows to a CSV file') and then states the return value. No wasted words; it efficiently conveys the core behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, and an output schema exists, so return values are covered. However, the description lacks explicit usage guidelines and parameter semantics, making it not fully complete for an agent to invoke correctly in all contexts.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not mention the 'filename' parameter at all. The schema provides only a default value, so the description fails to compensate for the missing parameter explanation, leaving the agent to infer that filename controls the output file name.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: writing all collected rows to a CSV file and returning the path. This distinguishes it from sibling tools like get_records or add_record, which handle data retrieval and manipulation, making its purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'all collected rows' implies the tool is used after data collection, but there is no explicit when-to-use guidance or mention of alternatives. No exclusions or prerequisites are stated, leaving usage context inferred rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_repeating_elementsA

Scan the current page for groups of repeated sibling elements (product cards, list items, search results, ...) and return candidate CSS selectors for each group, with a count and a text sample.

Use this on an unfamiliar site instead of eyeballing raw HTML for class names: pick a promising selector from the results and feed it to get_list (as container_selector) to actually extract the data.

ParametersJSON Schema
NameRequiredDescriptionDefault
min_countNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It indicates a read-only scan operation ('scan the current page') and describes the return format, which implies no page mutation. However, it does not explicitly state non-destructive behavior or limitations like handling dynamic content, so it falls short of a perfect score.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately concise, with the core purpose in the first sentence and usage guidance in the second. Every sentence earns its place, no redundant information or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's purpose, usage workflow, and return value shape, and it references the output schema. However, the complete omission of the min_count parameter prevents it from being fully complete, even though the tool is simple and the output schema exists.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has one parameter, min_count, with no description (0% schema coverage). The description never mentions min_count, leaving the agent without any clue about its meaning or effect. Since the schema is not self-explanatory, the description was required to compensate but failed entirely.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool scans the current page for repeated sibling elements and returns candidate CSS selectors with a count and text sample. It specifies the exact resource (current page) and action (scan and return selectors), clearly differentiating itself from sibling tools by mentioning its role in discovering selectors for get_list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use it on unfamiliar sites instead of eyeballing raw HTML, and instructs the agent to pick a promising selector and feed it to get_list. This provides both when-to-use context and a concrete integration path with a sibling tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_json_ldA

Return schema.org structured data (JSON-LD) embedded in the current page, e.g. Recipe/Product/Article objects. Many sites (WordPress recipe plugins, e-commerce) embed clean structured data this way. Check here before falling back to get_page_content.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It clearly states the return behavior (JSON-LD data) and gives a fallback hint, implying that absence of data should lead to get_page_content. It doesn't mention edge cases like multiple JSON-LD blocks or malformed data, but for a zero-parameter read-only tool, it is sufficiently transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured: first sentence states the core function, second adds context and usage guidance. No wasted words, and it front-loads the key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity, empty input schema, and existing output schema, the description provides complete context. It states what the tool does, gives examples, explains the fallback relationship, and refers to the current page context. No significant gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description adds no parameter-specific details, but none are needed since schema coverage is 100% (empty properties).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns schema.org structured data (JSON-LD) embedded in the current page, citing specific examples (Recipe/Product/Article). It differentiates from siblings by explicitly directing users to 'Check here before falling back to get_page_content'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use guidance by instructing to check this tool before falling back to get_page_content. It also motivates usage by noting that many sites embed clean structured data, making it the preferred first choice for extracting structured content.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_listA

Extract one record per element matching container_selector (e.g. a product card in a grid), pulling fields out of each one. This is the div/card equivalent of get_table for sites that don't use markup.

fields maps an output field name to a CSS selector relative to each container, e.g. {"title": ".product-title a", "price": "[data-a-color='base']"}. Use "" to target the container itself, or append "@attr" to read an attribute instead of text, e.g. "link": "a@href" or "image": "img@src".

Because each field is looked up independently within its own container, results never get misaligned the way separate get_page_content calls + manual zipping do when some cards are missing a field (e.g. a strikethrough "was" price only some products have). Not sure of the right selectors? Call find_repeating_elements first.

ParametersJSON Schema
NameRequiredDescriptionDefault
fieldsYes
container_selectorYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite lacking annotations, the description fully discloses the lookup mechanism: each field is resolved independently within its own container, thus preventing misalignment. It also specifies selector syntax including container self-targeting and '@attr' for attribute extraction, going far beyond the bare function name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Despite its length, every sentence contributes: the opening defines scope, the middle explains the mapping syntax, and the final note points to a companion tool. The structure is logical and front-loads the core purpose, making it an efficient, well-organized description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with only two required parameters, the description covers purpose, parameter semantics, usage context, and even a companion tool tip. With an existing output schema, the description need not explain return values, so it is fully complete within its context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema only defines container_selector as a string and fields as a string-to-string object, with 0% description coverage. The description explains that fields maps output names to relative CSS selectors, provides concrete examples, and documents the '' and '@attr' conventions, filling the entire coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Extract one record per element matching container_selector' and defines it as 'the div/card equivalent of get_table,' giving a specific verb, resource, and scope while distinguishing it from sibling tools like get_table and get_page_content.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly frames this as the non-table alternative to get_table and advises calling find_repeating_elements first when unsure of selectors. This provides clear when-to-use guidance and names a companion tool, leaving no ambiguity about alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_page_contentA

Return the current page's HTML with // tags stripped out.

If selector is given (a CSS selector, e.g. ".recipe-card" or "#content"), only the matching element(s) are returned instead of the whole page, which avoids truncation on large pages. If the content is longer than max_length, it's truncated and the response tells you the offset to pass on the next call to read the following chunk.

If the page looks like a Cloudflare/Akamai/CAPTCHA challenge page rather than real content, a "bot-challenge page" warning is prepended (only on the first chunk, offset=0).

ParametersJSON Schema
NameRequiredDescriptionDefault
offsetNo
selectorNo
max_lengthNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden and does so excellently. It discloses tag stripping, truncation behavior, offset continuation, and the bot-challenge warning, giving the agent a thorough understanding of side effects and edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence adds value: the main purpose is stated first, followed by selector behavior, truncation mechanics, and an edge case warning. The structure is logical and efficient without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no annotations and an output schema not shown, the description covers the tool's behavior comprehensively: input parameters, output format (offset for pagination, bot-challenge warning), and real-world usage considerations. This is fully adequate for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage, but the description fully explains all three parameters: selector (CSS selector for element filtering), max_length (truncation limit), and offset (chunk read position). This compensates completely for the schema's silence.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns the current page's HTML with script/style/svg tags stripped, and optionally scoped by a CSS selector. This distinguishes it from sibling tools like get_page_links (links), get_table (tables), and get_json_ld (structured data).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on when to use the selector (to avoid truncation on large pages) and how to handle pagination with offset. However, it does not explicitly mention when not to use this tool or name alternatives like get_page_links or get_table.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_recordsA

Return every row collected so far, for review before exporting.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. 'Return' and 'for review' clearly indicate a non-destructive read operation, though it does not detail output format or any limitations (mitigated by the presence of an output schema).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that is front-loaded with the action and includes context ('for review before exporting') without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (no parameters) and the existence of an output schema, the description adequately covers what the tool does and when to use it. The phrase 'for review before exporting' ties it into a workflow.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description adds context about the tool's purpose but no parameter-specific semantics are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Return every row collected so far') and identifies the resource (rows/records). It distinguishes from siblings like export_csv by framing this as a review step before export.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'for review before exporting' provides clear context for when to use this tool: before an export, to inspect the collected data. It doesn't explicitly name alternatives or exclusions, but the workflow context makes it clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_tableA

Parse an HTML table on the current page into rows of {column_header: cell_text}, using the first matching element's first as headers.

selector picks which table (default: the first on the page, e.g. "table#programsGrid"). A cell containing a link also gets a <header>_href key. Use this instead of eyeballing get_page_content for tabular data: it does the row/cell transcription for you, so pass the results straight to add_record instead of hand-copying values.

ParametersJSON Schema
NameRequiredDescriptionDefault
selectorNotable

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It explains key behaviors: uses the first matching element's first <tr> as headers, defaults to the first table, and adds a `<header>_href` key for cells containing links. It does not mention error handling for absent tables, but the disclosed parsing rules are sufficient for a read-only operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is tightly written, front-loading the core purpose in the first sentence. Additional details about selector behavior and usage guidance are relevant and concise, with no obvious filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one optional parameter and no annotations, the description covers the essential invocation details: input selection, default behavior, output format, and extra href key. It stops short of explaining edge cases like missing tables or multi-table ambiguity, but given the low complexity, it is reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description fully explains the only parameter, `selector`: it picks which table, defaults to the first <table>, and gives a concrete example ('table#programsGrid'). This adds substantial meaning beyond the raw schema definition.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Parse an HTML table on the current page into rows of {column_header: cell_text}'. It clearly distinguishes itself from siblings by explicitly naming get_page_content as the alternative for non-tabular data, and by noting that results can be passed directly to add_record.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool: 'Use this instead of eyeballing get_page_content for tabular data'. It provides an alternative tool and gives downstream workflow guidance ('pass the results straight to add_record'), making the intended usage context unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

open_browserB

Launch the Chromium browser. Call this once before navigating anywhere.

ParametersJSON Schema
NameRequiredDescriptionDefault
headlessNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It only hints that the tool should be called once but does not explain what happens if called multiple times, whether it is idempotent, or any side effects. The absence of such details leaves significant behavioral ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exceptionally concise: two short sentences that immediately state the primary action and add a usage note. Every word earns its place, with no wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Even though the tool is simple, the description is incomplete because it fails to explain the 'headless' parameter and does not clarify the consequences of repeated calls. The lack of annotations makes this incompleteness more impactful, leaving essential context missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema includes a 'headless' parameter with a default of true, but the description does not mention it at all. With 0% schema description coverage, the agent receives no explanation of what 'headless' means or when to set it, making this a major gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action: 'Launch the Chromium browser.' This is a specific verb+resource pair that distinguishes it from siblings like navigate or close_browser. The additional clause 'Call this once before navigating anywhere' further clarifies its role as the initial setup step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit timing guidance: 'Call this once before navigating anywhere,' which tells the agent when to use it. It does not mention when-not-to-use or alternatives, but no direct alternative exists among the siblings, so the guidance is clear enough for a 4.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wait_forA

Wait for selector to appear and/or network activity to go idle, without navigating or clicking. Use this if a previous get_page_content/get_list read caught the page before client-side content finished loading.

ParametersJSON Schema
NameRequiredDescriptionDefault
idleNo
selectorNo
timeout_msNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavior itself. It reveals that it does not navigate or click, and that it handles client-side loading delays. However, it omits what happens on timeout or when both selector and idle are specified, which is a notable gap for a blocking operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the primary action and conditions, then a targeted usage scenario. No fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, but the lack of annotation coverage and 0% schema descriptions mean the description must compensate. It explains the main purpose and a usage scenario, but omits parameter semantics for idle/timeout and error behavior on timeout, leaving the description incomplete for full invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain all parameters. It only explains 'selector' implicitly via the main sentence. 'idle' and 'timeout_ms' are not described, leaving their meaning (especially the boolean 'idle' and its interaction with 'selector') unclear.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: waiting for a selector to appear and/or network activity to go idle, with explicit exclusion of navigating or clicking. This distinguishes it from sibling tools like navigate, click, and get_page_content.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides a concrete use case: 'if a previous get_page_content/get_list read caught the page before client-side content finished loading.' This is clear context, though it does not explicitly list when not to use or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 15 tool updatesv0.1.1
    • First observedadd_record
    • First observedclear_records
    • First observedclick
    • First observedclose_browser
    • First observedexport_csv
    • First observedfind_repeating_elements
    • First observedget_json_ld
    • First observedget_list
    • First observedget_page_content
    • First observedget_page_links
    • First observedget_records
    • First observedget_table
    • First observednavigate
    • First observedopen_browser
    • First observedwait_for

TDQS

A4.2/5.0
Disambiguation5/5

Each tool maps to a distinct action: browser lifecycle (open/close), navigation/clicking/waiting, content retrieval (navigate/get_page_content/click), extraction (get_table/get_list/get_json_ld/get_page_links/find_repeating_elements), and data management (add/get/clear/export). While navigate, click, and get_page_content all return cleaned content, their triggering actions are distinct and clearly described, so an agent can pick the right one.

Naming Consistency5/5

All tool names are snake_case and follow a verb-first pattern: open_browser, close_browser, navigate, click, wait_for, get_*, add_record, clear_records, export_csv, find_repeating_elements. No mixing of camelCase or inconsistent verb styles. The single-word verbs (navigate, click) are minor deviations but remain clear and consistent.

Tool Count5/5

15 tools is within the typical 3-15 well-scoped range. Each tool addresses a distinct need in the scraping workflow, from browser control to extraction to data export, so none feels redundant or missing.

Completeness4/5

The surface covers the full scraping lifecycle: launch, navigate, interact (click/wait), extract (content/table/list/JSON-LD/links), and manage/export data. A few advanced scenarios like infinite-scroll (no scroll tool) or form filling are not covered, but these are minor gaps for a generic scraper and workarounds exist (e.g., click for load-more buttons).

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    D
    maintenance
    An MCP server that enables LLM applications to control web browsers via Browserbase, supporting features like web navigation, screenshots, cookie management, and persistent contexts.
    17
    5,333
    Apache 2.0
  • F
    license
    A
    quality
    A
    maintenance
    An MCP server that bridges LLMs with dynamic real-world data by leveraging Chrome DevTools Protocol to intercept and reconstruct network traffic, enabling AI agents to extract high-quality structured data from complex web environments.
    3
    88
    -
  • A
    license
    B
    quality
    D
    maintenance
    An MCP server that provides browser automation capabilities, enabling LLMs to control a web browser for navigation, interaction, and data extraction.
    32
    232
    13
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    MCP server for web scraping and browser automation, enabling AI agents to extract clean, token-efficient content from web pages.
    1
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/KhaledDev/yet-another-web-scraper-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server