Skip to main content
Glama
RichardDillman

Googlebot Simulator MCP

by RichardDillman
OverviewSchemaRelated ServersScoreDiscussions
TypeScript

Googlebot Simulator MCP

An MCP server that simulates Googlebot crawling pages and captures analytics events. Use this to verify your site's bot detection and analytics tracking work correctly.

Features

  • Authentic Googlebot User-Agents: Uses real Googlebot desktop and mobile user-agent strings

  • Idle Detection Simulation: Mimics Googlebot's requestIdleCallback-based idle detection

  • Event Interception: Captures analytics/tracking requests matching your pattern

  • Detailed Reporting: Returns timing info, captured payloads, and screenshots

Related MCP server: gsc-mcp

Installation

Add to Claude Code

claude mcp add googlebot-sim -- npx googlebot-simulator-mcp

Or add to .claude.json

{
  "mcpServers": {
    "googlebot-sim": {
      "type": "stdio",
      "command": "npx",
      "args": ["googlebot-simulator-mcp"]
    }
  }
}

From source

git clone https://github.com/RichardDillman/googlebot-simulator-mcp.git
cd googlebot-simulator-mcp
npm install
npm run build

Usage

Once added to Claude, you can use natural language:

"Test https://example.com/jobs as Googlebot, watching for requests containing 'page_viewed'"

Or be more specific:

"Simulate mobile Googlebot on https://talent.com/jobs, capture events to **/serp-event-producer/send"

Tool: simulate_googlebot

Input

{
  // Required: URL(s) to test
  urls: string | string[];

  // Required: Pattern to match analytics events
  eventPattern: {
    urlPattern?: string;    // e.g., "**/analytics/**"
    bodyContains?: string;  // e.g., "page_viewed"
  };

  // Optional configuration
  options?: {
    userAgent?: "desktop" | "mobile";  // Default: "desktop"
    idleTimeout?: number;              // Default: 5000ms
    maxWaitTime?: number;              // Default: 10000ms
    captureScreenshot?: boolean;       // Default: true
    screenshotDir?: string;            // Default: cwd
  };
}

Output

{
  results: Array<{
    url: string;
    success: boolean;              // true if matching event was captured
    eventsFired: Array<{
      timestamp: number;
      url: string;
      method: string;
      payload: object;
      matchedPattern: boolean;
      responseStatus?: number;
    }>;
    timing: {
      navigationStart: number;
      domContentLoaded: number;
      idleDetected: number;
      totalTime: number;
      firstIdleCallback?: number;
    };
    screenshot?: string;           // Path to screenshot file
    errors: string[];
    idleCallbacksObserved: number;
  }>;
  summary: {
    totalUrls: number;
    passed: number;
    failed: number;
    totalEventsCaptured: number;
  };
}

Example Output

{
  "results": [
    {
      "url": "https://talent.com/jobs",
      "success": true,
      "eventsFired": [
        {
          "timestamp": 1704825600000,
          "url": "https://events.talent.com/serp-event-producer/send",
          "method": "POST",
          "payload": {
            "event_type": "page_viewed_bot",
            "bot_type": "google",
            "page_name": "serp"
          },
          "matchedPattern": true,
          "responseStatus": 200
        }
      ],
      "timing": {
        "navigationStart": 1704825595000,
        "domContentLoaded": 1704825596500,
        "idleDetected": 1704825600000,
        "totalTime": 5000,
        "firstIdleCallback": 3200
      },
      "screenshot": "/tmp/googlebot_desktop_jobs_1704825600000.png",
      "errors": [],
      "idleCallbacksObserved": 3
    }
  ],
  "summary": {
    "totalUrls": 1,
    "passed": 1,
    "failed": 0,
    "totalEventsCaptured": 1
  }
}

How It Works

  1. Launches headless Chromium with Googlebot user-agent and viewport settings

  2. Injects tracking script to monitor requestIdleCallback usage

  3. Sets up network interception to capture matching requests

  4. Navigates to URL and waits for page to reach idle state

  5. Captures screenshot at the "idle" moment (what Googlebot sees)

  6. Returns detailed report with all captured events and timing

Use Cases

  • Verify bot detection: Ensure your analytics correctly identifies Googlebot

  • Test event timing: Confirm events fire before Googlebot's idle timeout

  • Debug rendering: See what content is visible when Googlebot considers the page "done"

  • Compare desktop vs mobile: Test both Googlebot variants

License

MIT

Available Tools

1 tool
simulate_googlebotA

Simulate Googlebot crawling a page and capture analytics events.

This tool:

  • Uses authentic Googlebot user-agent strings

  • Simulates Googlebot's idle detection behavior (requestIdleCallback)

  • Intercepts and captures network requests matching your pattern

  • Returns detailed timing, events captured, and optional screenshots

Use this to verify:

  • Analytics pageview events fire correctly for bots

  • Bot detection is working (e.g., page_viewed vs page_viewed_bot)

  • Content is rendered correctly at the "idle" state

  • Timing of event firing relative to page load

ParametersJSON Schema
NameRequiredDescriptionDefault
urlsYesURL(s) to simulate Googlebot crawling
optionsNoOptional configuration for the simulation
eventPatternYesPattern to identify analytics/tracking events. At least one of urlPattern or bodyContains required.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description fully carries the transparency burden. It discloses key behaviors: authentic user-agent, requestIdleCallback simulation, network interception, and capture of timing/events/screenshots. It doesn't mention limits or error handling, but the transparency is high.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured: a one-line summary, a bulleted list of features, and a bulleted list of use cases. Every sentence is informative with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given three parameters (one nested), no output schema, and no annotations, the description covers the tool's purpose, behavior, and use cases adequately. It mentions return values (timing, events, screenshots) but lacks details on output format. Still, it is complete for the complexity level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents parameters well. The description adds context about the tool's purpose (e.g., capturing analytics events) that complements the eventPattern parameter, but it does not provide new parameter-level details beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Simulate Googlebot crawling a page and capture analytics events.' It provides a specific verb and resource, and lists detailed behaviors and use cases. Without sibling tools, it effectively distinguishes itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly lists four verification scenarios ('Use this to verify:...'), providing clear usage context. It lacks explicit alternatives or when-not-to-use statements, but the absence of sibling tools reduces the need for exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 1 tool updatev1.0.0
    • First observedsimulate_googlebot
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.2/5.0
Disambiguation5/5

With only one tool, there is no possibility of confusion between tools. The tool's purpose is clearly distinct by default.

Naming Consistency5/5

The single tool name 'simulate_googlebot' follows a clear verb_noun pattern and is descriptive. With only one tool, there are no naming inconsistencies.

Tool Count3/5

One tool is on the low end, but it is well-scoped to a specific use case (simulating Googlebot). It feels slightly thin but not inappropriate for its narrow domain.

Completeness4/5

The tool covers the main aspects of simulating Googlebot crawling (user-agent, idle detection, event capture, timing, screenshots). No obvious gaps for its stated purpose, though it is a single tool.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • gsc-mcp

    mikusnuz
    A
    license
    A
    quality
    A
    maintenance
    MCP server for Google Search Console, URL Inspection & Indexing API — search analytics, sitemap management, and batch indexing
    13
    203
    7
    MIT

  • robotstxt-ai-mcp

    sharozdawa
    A
    license
    A
    quality
    C
    maintenance
    Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
    5
    1
    MIT

  • Argus

    chriswu727
    A
    license
    A
    quality
    A
    maintenance
    AI-powered exploratory QA agent. Explores web apps like a real user — 18 MCP tools for clicking, filling forms, and navigating. Automatically verifies that actions persist (fake deletes, failed edits). Runs 16 detection types including dead links, SEO, accessibility, and performance checks.
    29
    2
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/RichardDillman/googlebot-simulator-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server