Fetch Crawl MCP
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| NODE_ENV | No | Node.js environment | production |
| PUPPETEER_EXECUTABLE_PATH | No | Path to Chromium executable | /usr/bin/chromium |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| fetch_pageA | Fetch a web page and return its content in the specified format (HTML, plain text, or markdown). Includes basic metadata. |
| crawl_siteA | Crawl a website recursively starting from a URL. Follows internal links up to a specified depth and max pages. Returns a list of discovered pages with their titles and status codes. |
| extract_contentB | Extract structured content from a web page: headings hierarchy, paragraphs, images, links, and plain text content with statistics. |
| extract_linksA | Extract all links from a web page. Can filter by internal, external, or all links. Returns anchor text, URL, rel attributes, and nofollow status. |
| audit_onpageA | Technical on-page HTML audit. Checks title tag, meta description, canonical, robots, lang attribute, heading hierarchy (H1-H6), images alt attributes, Open Graph, Twitter Card, and structured data (JSON-LD). Returns a list of technical issues found. Does NOT analyze keywords, rankings, backlinks, or traffic — use Semrush for that. |
| parse_sitemapA | Parse a sitemap.xml file (or auto-detect it from a website URL). Supports sitemap index files and returns all discovered URLs with lastmod, changefreq, and priority. |
| check_linksA | Check all links on a web page for broken links (404, timeout, connection errors). Returns broken, redirected, and OK links with their status codes. |
| screenshotA | Capture a screenshot of a web page using a headless browser. Returns an image as base64 (PNG or JPEG). Supports custom viewport size, full-page capture, waiting for a CSS selector, and dismissing cookie consent banners. |
| check_performanceA | Measure web page performance metrics (TTFB, FCP, LCP, DOM Content Loaded, Fully Loaded) using a headless browser. Supports mobile (with network throttling) and desktop profiles. Returns a 0-100 score, detailed timings, network stats, and actionable issues. |
| check_redirect_chainA | Follow the redirect chain of a URL hop by hop (using manual redirect). Returns each hop with status, Location header, and Server header. Detects redirect loops, long chains, and HTTP-to-HTTPS upgrades. |
| check_mobileA | Audit a web page for mobile-friendliness using a headless browser with iPhone viewport (375x812) and mobile User-Agent. Checks viewport meta, horizontal scroll, font sizes, tap target sizes, and returns a mobile screenshot. |
| check_structured_dataA | Extract and validate structured data from a web page: JSON-LD (with type-specific validation for Product, Organization, BreadcrumbList, Article), Microdata, Open Graph, and Twitter Card meta tags. |
| check_robots_txtA | Analyze a site's robots.txt: parse rules per User-Agent (Allow/Disallow), Crawl-delay, declared sitemaps, and cross-check sitemap accessibility. Detects undeclared sitemaps and inconsistencies. |
| check_indexabilityA | Check if a page is indexable by search engines. Analyzes HTTP status, meta robots, X-Robots-Tag, canonical tag, hreflang, and sitemap presence. Returns a verdict with detailed reasoning. |
| compare_pagesA | Compare two web pages side by side on SEO criteria: title, meta description, headings, word count, links, images alt, Open Graph, Twitter Card, JSON-LD, and canonical. Optionally captures screenshots of both pages. |
| audit_site_batchA | Batch audit multiple pages of a site. Collects URLs from sitemap, crawl, or a provided list, then runs a lightweight SEO audit on each page. Returns aggregate scores, top problems, quick wins, and critical pages. |
| detect_orphan_pagesA | Detect orphan pages by cross-referencing sitemap URLs with crawled pages and internal link graph. Identifies pages with no inbound links (orphans), sitemap-only pages, crawl-only pages, and deep pages. Returns classification, link graph hubs, and sitemap/crawl coherence stats. |
| detect_duplicate_contentA | Detect duplicate and near-duplicate content across a site's pages. Analyzes titles, meta descriptions, H1 headings, and text content. Groups exact duplicates and identifies near-duplicates based on word similarity. Supports sitemap, crawl, or custom URL list as page source. |
| extract_with_schemaA | Extract structured data from a web page using configurable CSS selectors. Supports custom schemas and built-in presets (ecommerce-product, article, local-business, recipe). Each field defines a CSS selector, optional attribute, multiple flag, and transform (text, html, number, trim, href). Fallback selectors can be provided for resilience. |
| check_gtm_snippetA | Check a page for Google Tag Manager (GTM) and gtag.js snippets. Detects GTM container IDs, GA4 measurement IDs, verifies snippet placement (head vs body), noscript fallback, and duplicate IDs. |
| check_datalayerA | Inspect window.dataLayer at runtime using a headless browser. Checks if dataLayer exists, its contents (events), whether GTM and gtag are loaded, and detects suspicious patterns like dataLayer redefinition after GTM init. |
| intercept_tracking_requestsA | Intercept and analyze all tracking network requests (GA4 hits, GTM, gtag) fired during page load using Puppeteer request interception. Parses GA4 /g/collect hits for event names and measurement IDs. Detects obsolete Universal Analytics hits and duplicate events. |
| audit_trackingA | Full tracking audit: runs check_gtm_snippet, check_datalayer, and intercept_tracking_requests in parallel, then produces a unified report with a global score, severity summary, and cross-tool diagnosis. |
| check_security_headersA | Audit HTTP security headers (HSTS, CSP, X-Frame-Options, Referrer-Policy, Permissions-Policy, etc.). Returns a 0-100 score with grade (A-F), per-header analysis, and actionable recommendations. Lightweight — uses a single HTTP request, no browser needed. |
| check_hreflangA | Validate hreflang alternate language tags on a page. Checks language code validity (ISO 639-1), URL accessibility, reciprocal linking between language versions, x-default presence, canonical consistency, and language set coherence. Returns a 0-100 score with detailed per-alternate analysis. |
| audit_content_qualityA | Analyze content quality of a web page: word count, readability score (multilingual), text-to-HTML ratio, heading structure, link density, media richness, and engagement signals (TOC, FAQ, CTA). Returns a 0-100 score with per-category breakdown and actionable recommendations. Does NOT check SEO meta tags — use audit_onpage for that. |
| check_accessibilityA | Run a lightweight WCAG accessibility audit on a web page. Checks images alt text, form labels, semantic structure (landmarks, lang), heading hierarchy, link text quality, viewport zoom restrictions, table structure, media controls, and ARIA usage. Returns a 0-100 score with per-category breakdown, WCAG criteria references, and actionable issues with CSS selectors. Does NOT check color contrast or keyboard navigation (requires browser rendering). |
| extract_images_auditA | Comprehensive image audit for a web page. Uses Puppeteer to detect all images (img, picture, CSS backgrounds) including lazy-loaded ones. Analyzes format (WebP/AVIF adoption), alt text quality, responsive images (srcset), sizing optimization, lazy loading correctness, LCP candidate optimization, and file sizes. Returns a 0-100 score with per-image details and actionable recommendations. |
| check_consent_modeA | Audit Google Consent Mode v2 implementation and cookie compliance. Detects CMP vendor (Cookiebot, OneTrust, Didomi, Axeptio, etc.), verifies consent default/update configuration, checks IAB TCF API presence, analyzes GA4 hit consent signals (gcs/gcd parameters), and audits pre-consent cookie behavior. Returns a 0-100 GDPR compliance score with detailed findings. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/GeorgesAdSim/fetch-crawl-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server