Skip to main content
Glama
DmitriyOT

MCP Web Search Server

by DmitriyOT
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Remote

MCP Web Search Server

MCP server for web search with results optimized for LLMs and advanced bot-detection evasion.

Features

  • Search: DuckDuckGo (no API key), Serper.dev, Bing Web Search

  • Search aggregation: fallback mode or merge mode that queries multiple providers in parallel, deduplicates and ranks results by query relevance

  • Content extraction: Headless browser with stealth injection, browser page pool, and plain HTTP text fallback when pages are blocked

  • Anti-detection: Dynamic fingerprint generation (viewport, UA, locale, timezone), human-like behavior, proxy support

  • LLM formatting: Clean markdown via Turndown, metadata, structured data, links, images, PDF text extraction

  • OCR: Extract text from image URLs using tesseract.js

  • Broken link checker: Scan a page and report the status of its outgoing links

  • Robustness: Retry with exponential backoff, circuit breaker per provider, token-bucket rate limiting, concurrency limiting, in-flight request deduplication, persistent cache, graceful shutdown

  • Observability: Structured JSON logging via Pino, latency/error metrics per provider and fetch operation

  • Config hot reload: .env changes are picked up automatically while the server is running

  • Multiple transports: stdio (default) or HTTP Streamable MCP transport

Related MCP server: webmcp

Installation

npm install
npm run build

Configuration

Copy .env.example to .env and configure:

# Search providers (optional — DuckDuckGo works without API keys)
SERPER_API_KEY=your_key
BING_API_KEY=your_key

# Search aggregation: "fallback" or "merge"
SEARCH_MERGE_MODE=fallback

# Rate limits (requests per second per provider)
SERPER_RATE_LIMIT=10
BING_RATE_LIMIT=10
DUCKDUCKGO_RATE_LIMIT=1

# Anti-detect
STEALTH_ENABLED=true
HEADLESS=true
PROXY_LIST=http://proxy1:8080,http://proxy2:8080
USER_DATA_DIR=

# Browser page pool size
BROWSER_POOL_SIZE=2

# Limits
MAX_RESULTS=10
MAX_CONTENT_LENGTH=8000
MAX_RESPONSE_SIZE_BYTES=10000000
REQUEST_TIMEOUT=30000
CACHE_TTL=300

# Behavior
MIN_DELAY=500
MAX_DELAY=3000
MAX_CONCURRENT=2
SCROLL_TO_BOTTOM=true

# Fetch fallback when the browser is blocked
TEXT_FETCH_FALLBACK=true

# Optional persistent cache directory
CACHE_DIR=./cache

# Logging
LOG_LEVEL=info

# Transport: stdio or http
MCP_TRANSPORT=stdio
HTTP_HOST=127.0.0.1
HTTP_PORT=8080

# Ethics / safety
ROBOTS_TXT_ENABLED=true
ALLOWED_DOMAINS=
BLOCKED_DOMAINS=

# Debug only — weakens browser security
ALLOW_INSECURE_BROWSER_FLAGS=false

MCP Tools

web_search

Search by query.

{
  "query": "latest AI developments 2025",
  "num_results": 10,
  "provider": "auto",
  "recency_days": 7
}

fetch_url

Fetch and clean a page.

{
  "url": "https://example.com/article",
  "max_length": 8000,
  "include_images": false,
  "include_links": false
}

search_and_fetch

Search and automatically fetch top-N results.

{
  "query": "quantum computing breakthrough",
  "num_results": 5,
  "fetch_content": true,
  "max_content_length": 5000,
  "include_images": false,
  "include_links": false
}

health_check

Check provider availability, circuit breaker state, and rate limits.

{}

ocr_image

Extract text from an image URL.

{
  "url": "https://example.com/screenshot.png",
  "language": "eng"
}

check_links

Fetch a page and check its outgoing links.

{
  "url": "https://example.com/article",
  "max_links": 20
}

Claude Desktop Integration

{
  "mcpServers": {
    "web-search": {
      "command": "node",
      "args": ["C:\\Files\\git\\mcp-web-search\\dist\\index.js"],
      "env": {
        "SERPER_API_KEY": "..."
      }
    }
  }
}

HTTP Transport

Set MCP_TRANSPORT=http and HTTP_PORT=8080, then point an MCP client that supports Streamable HTTP at http://127.0.0.1:8080.

Anti-Detection & Reliability

The server uses a layered approach:

  1. puppeteer-extra-plugin-stealth — hides automation fingerprints

  2. Dynamic fingerprint generation — random Chrome on Windows/macOS/Linux with matching timezone

  3. Browser page pool — reusable Puppeteer pages to reduce launch overhead

  4. Human-like behavior — random delays, scroll, mouse events

  5. Proxy support — random proxy selection from PROXY_LIST

  6. Text fetch fallback — plain HTTP fetch when the headless browser is blocked

  7. Search result merging & ranking — combine multiple providers and rank by relevance

  8. Circuit breaker — temporarily disables failing providers

  9. Token-bucket rate limiting — per-provider rate limiting

  10. Retry & concurrency limiting — exponential backoff, bounded parallelism, in-flight deduplication

  11. Graceful shutdown — waits for active requests on SIGINT/SIGTERM

  12. Robots.txt respect — honors site crawl rules (can be disabled)

Development

npm run dev          # watch mode
npm test             # run unit tests
npm run build        # compile TypeScript
npm run lint         # run ESLint
npm run format       # format with Prettier

Docker

docker build -t mcp-web-search .
docker run --rm -e SERPER_API_KEY=... mcp-web-search

License

MIT

Available Tools

3 tools
fetch_urlC

Fetch and extract clean text content from a URL. Optimized for LLM consumption with metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesURL to fetch
max_lengthNoMaximum content length in characters
include_imagesNoInclude image references

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does not disclose behavioral aspects such as potential slowness, error handling, caching, or whether the fetch is destructive. Only mentions 'optimized for LLM consumption' without specifics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no unnecessary words, achieving brevity and focus. It effectively communicates the core function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema and the presence of three parameters, the description fails to explain what metadata is returned, how to interpret the response, or any error conditions. This leaves the agent without enough context to fully rely on the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description does not add any semantic value beyond the schema; it merely restates the tool's purpose rather than elaborating on parameter usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool fetches a URL and extracts clean text, with a specific verb and resource. It distinguishes from siblings by implying this is for direct URL fetching, but does not explicitly differentiate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus sibling tools like 'search_and_fetch' or 'web_search'. There is no indication of prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_and_fetchA

Search the web and automatically fetch content from top results. Returns combined LLM-formatted output.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesSearch query
num_resultsNoNumber of results to fetch (1-20)
fetch_contentNoWhether to fetch full page content
max_content_lengthNoMax length per page

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must bear the burden. It mentions automatic fetching from 'top results' but lacks details on selection criteria, rate limits, or processing behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences efficiently convey the core functionality, though additional structure could enhance clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Without annotations or output schema, the description adequately explains the combined search-and-fetch process and output format, but could specify the number of top results and provide more behavioral constraints.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all 4 parameters. The description adds minimal value beyond the schema, meeting the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool combines web search and automatic content fetching, distinguishing it from siblings 'web_search' and 'fetch_url'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this vs. alternatives; only implied by the combined nature and mention of 'LLM-formatted output'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 3 tool updatesv1.0.0
    • First observedfetch_url
    • First observedsearch_and_fetch
    • First observedweb_search
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A3.5/5.0
Disambiguation5/5

Each tool serves a clearly distinct purpose: web_search returns a list of results, fetch_url extracts content from a specific URL, and search_and_fetch combines search with automatic fetching. No overlap or ambiguity.

Naming Consistency4/5

Tool names consistently use snake_case and mostly follow a verb_noun pattern (fetch_url, web_search). The exception is search_and_fetch which uses two verbs, but it still clearly conveys its combined functionality.

Tool Count4/5

With 3 tools, the set is on the smaller side but still covers the core needs of web searching and URL fetching. It feels slightly minimal but not insufficient for the stated purpose.

Completeness4/5

The tool surface covers searching, fetching individual pages, and a combined operation. Minor gaps like lacking custom search parameters or batch fetching exist, but the core workflow is functional.

Maintenance

ActivitySlowing
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • webmcp by AuthBits

    webmcpofficial

    AuthBits
    A
    license
    Not graded
    quality
    C
    maintenance
    MCP server for web search and content extraction using DuckDuckGo or SearXNG, with Playwright-based fetching and LLM-powered data extraction.
    139
    MIT

  • evo-scry

    krich11
    A
    license
    Not graded
    quality
    D
    maintenance
    MCP server for internet search via direct Google and DuckDuckGo HTML scraping with AI-powered result normalization and optional summarization, requiring no API keys for search.
    MIT

  • web-mcp

    Chang-Tong
    A
    license
    Not graded
    quality
    B
    maintenance
    MCP server for multi-engine web search and web page fetching, supporting parallel search, content extraction, and optional LLM-powered search summarization and deep search.
    2
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/DmitriyOT/mcp-web-search'

If you have feedback or need assistance with the MCP directory API, please join our Discord server