MCP Observatory
MCP Observatory is an MCP server testing tool that enables AI agents to autonomously scan, test, monitor, and verify other MCP servers for regressions, schema drift, and security issues.
Core Tools:
scan– Auto-discover MCP servers from config files (Claude configs,.claude.json,.mcp.json) and run health checks, returning a summary of tools, prompts, and resources for every discovered servercheck_server– Test a specific MCP server by launch command, verifying its capabilities respond correctlydiff_runs– Compare two run artifact JSON files to identify regressions, recoveries, and schema drift between server versionsget_last_run– Retrieve the most recent run artifact for a given target ID to review previous results without re-running a scan
Additional Capabilities:
Security scanning – Analyze tool schemas for dangerous patterns like shell injection, broad filesystem access, and credential leakage
Lock file management – Snapshot server schemas and verify no drift has occurred since last lock
Health scoring – Generate 0–100 health scores and SVG badges for server READMEs
CI integration – Generate reports for GitHub Actions, block merges on regressions, and provide health badges
Record and replay – Capture server interactions to cassette files for offline/CI testing
Server recommendations – Suggest MCP servers from the registry based on your project's tech stack
Multi-transport support – Works with stdio, HTTP/SSE, and Docker-based MCP servers
It operates both as a CLI tool and as an MCP server itself, allowing AI agents to use its tools to autonomously test other servers.
Generates human-readable Markdown reports from MCP server test runs and comparison artifacts for sharing and documentation.
Secure the MCP servers you're building. MCP Observatory is the CI-native security tool for teams shipping custom MCP servers. Test during development, catch schema drift, simulate attacks, and generate compliance evidence — before agents depend on your servers.
Also available in Simplified Chinese.
Runtime enforcement: Use mcp-seatbelt to block dangerous MCP tool calls at runtime based on observatory scan results.
Get Started
Step-by-step setup guide · macOS, Windows, Linux · Node.js 20+ and npm (install Node.js LTS).
1. Run your first scan
Open Terminal or PowerShell in your project folder and paste:
npx -y @kryptosai/mcp-observatory@latestThis downloads Observatory and checks your first configured MCP server. With no configured server, it uses an included example. To try only that example, run npx -y @kryptosai/mcp-observatory@latest demo --example.
2. Read the result
The terminal shows the target, grade, findings, and saved receipt path. Failed checks are findings to review, not a request to pay. If connection failed, fix the startup command or credentials and rerun. A demo grade describes the example, not your own server.
3. Save one hosted snapshot free (optional)
In the same project folder, run:
npx -y @kryptosai/mcp-observatory@latest cloud uploadOpen the verification link printed in the terminal, confirm the code, connect with GitHub, then return to the terminal. Wait for Upload complete and open the dashboard link. No separate account setup or payment method is needed. Free includes one target, the latest snapshot, seven-day retention, and ten uploads per month.
Local scans and local CI stay free. Individual Pro is $29/month when you need 10 targets, 90-day history, hosted CI ingestion, or hosted HTTP scans.
MCP Observatory uses jurisdiction-aware product telemetry after showing the collection notice. Review the exact fields with telemetry preview, check the current policy with telemetry status, or stop collection at any time with telemetry disable, MCP_OBSERVATORY_TELEMETRY=0, or DO_NOT_TRACK=1. See the complete privacy and telemetry notice.
npx -y @kryptosai/mcp-observatory@latest test npx -y @modelcontextprotocol/server-everything- uses: KryptosAI/mcp-observatory/action@v1If this repo is an MCP package, that tests npx -y <package.json name>. Otherwise set command:.
Or generate a full workflow: npx -y @kryptosai/mcp-observatory@latest setup-ci --all --command "npx -y my-mcp-server" --sarif
Install: npx -y @kryptosai/mcp-observatory@latest · brew tap kryptosai/mcp-observatory https://github.com/KryptosAI/mcp-observatory && brew install mcp-observatory · docker pull ghcr.io/kryptosai/mcp-observatory:latest · curl -fsSL https://raw.githubusercontent.com/KryptosAI/mcp-observatory/main/scripts/install.sh | sh
docker run --rm ghcr.io/kryptosai/mcp-observatory:latest demo
docker run --rm ghcr.io/kryptosai/mcp-observatory:latest test npx -y @modelcontextprotocol/server-everythingScanning a server on the host needs network and, for local files, a volume mount (-v "$PWD:$PWD" -w "$PWD").
Agents: npx -y @kryptosai/mcp-observatory@latest serve
Default-deny connect: npx -y @kryptosai/mcp-observatory@latest protect rewrites every discovered client config (Cursor, Claude, VS Code, .mcp.json, …). Undo: protect --undo. HTTP/SSE servers are left alone.
Runtime deny-default: npx -y @kryptosai/mcp-observatory@latest enforce --start-proxy writes a Seatbelt policy from the scan and starts the proxy. Local scan stays free.
Proof: Safety Index. Upload one hosted snapshot free with cloud upload; Individual Pro is $29/month when you need retained history and hosted CI ingestion.
Related MCP server: SilentFail
Why MCP Observatory
MCP servers are becoming production dependencies. If agents rely on them, teams need a way to catch broken tools, unsafe schemas, schema drift, slow responses, and security footguns before those failures reach users.
Observatory gives maintainers and teams:
One-command CI setup with
setup-ci --allProfile-mapped audits with
audit --profile nsa-mcpMCP receipts that package target, evidence, verdict, action, and reproduction commands
MCP risk graphs that group servers by capability boundary, receipt state, CI posture, and recommended action
Action receipts that say
allow,gate,rerun,quarantine, orescalateGitHub PR comments for compatibility, drift, and security findings
GitHub Code Scanning SARIF for normalized MCP findings
Health score badges for public trust signals
Record/replay/verify workflows for regression testing
MCP server mode so agents can inspect other MCP servers directly
Production support path for one-user hosted history, CI ingestion, and owner-ready Release Gate remediation
See GitHub Code Scanning for MCP servers, MCP Receipts, Troubleshooting, Safety Methodology, MCP Server Safety Index, MCP Observatory Contributors, hosted client contract, repository boundary, open core boundary, and commercial support.
Self-Assessment
We scan ourselves with mcp-observatory on every release. See results →
For Security And Platform Teams
MCP servers are becoming part of the AI software supply chain. Agents need reliable, testable, auditable tools before those tools become dependencies in mission-critical workflows.
Whether you're shipping one MCP server or running a fleet, MCP Observatory gives you CI-native security scoring, attack simulation, schema drift detection, SARIF/HTML/Markdown reports, and GitHub Code Scanning — from your first npx command to production deployment. Local development stays free; teams with a near-term production approval decision can use the fixed-scope MCP Release Gate Pilot.
Production Support
Local OSS use stays free under MIT. Teams running MCP in production can use the MCP Release Gate Pilot for safe-mode evidence, SARIF/Code Scanning setup, CI rollout, private reporting, and owner-ready remediation notes. The fixed public entry offer is $15,000 for 1-3 critical MCP servers over ten business days; broader work is scoped after the release decision.
The open source repo is the portable evidence engine. Hosted authentication, retention, organization workflows, fleet coordination, and private intelligence stay outside the OSS package; see the repository boundary.
Run npx @kryptosai/mcp-observatory cloud, open a pilot request from the issue chooser, or see COMMERCIAL.md. Also see privacy, campaign attribution, and terms for production use.
How It Compares
Feature | mcp-observatory | Snyk agent-scan | Cisco mcp-scanner | agent-shield |
MCP-native | ✓ | ✓ | ✓ | ✓ |
Attack simulation | ✓ | ✗ | ✗ | ✗ |
Schema drift detection | ✓ | ✗ | ✗ | ✗ |
Record/replay/verify | ✓ | ✗ | ✗ | ✗ |
Health scoring (0-100) | ✓ | ✗ | ✗ | ✗ |
SARIF output | ✓ | ✓ | ✓ | ✓ |
CI/CD native (setup-ci) | ✓ | ✓ | ✓ | ✓ |
Public Safety Index | ✓ | ✗ | ✗ | ✗ |
Runtime enforcement via mcp-seatbelt | ✓ | ✗ | ✗ | ✗ |
Quick Start
Run a first check. No arguments starts demo (your servers, or the packaged local demo):
npx @kryptosai/mcp-observatoryGo deeper — also invoke safe tools to verify they actually run:
npx @kryptosai/mcp-observatory scan deepTest a specific server:
npx @kryptosai/mcp-observatory test npx -y @modelcontextprotocol/server-everythingAdd it to Claude Code as an MCP server:
claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serveOr add it manually to your config:
{
"mcpServers": {
"mcp-observatory": {
"command": "npx",
"args": ["-y", "@kryptosai/mcp-observatory", "serve"]
}
}
}Commands
Command | What it does |
| Auto-discover servers, check them, and run safe attack-readiness simulation by default |
| Scan, run safe attack simulation, and also invoke safe tools to verify they execute |
| Test one server and emit an action receipt by command or target config |
| Record a server session to a cassette file for offline replay |
| Replay a cassette offline — no live server needed |
| Verify a live server still matches a recorded cassette |
| Compare two run artifacts for regressions and schema drift |
| Watch a server for changes, alert on regressions |
| Detect your stack and recommend MCP servers from the registry |
| Start as an MCP server for AI agents |
| Snapshot MCP server schemas into a lock file |
| Verify live servers match the lock file |
| Show health score trends for your MCP servers |
| Create a GitHub Action and badge snippet for MCP compatibility/security checks |
| Generate a workflow that uploads normalized findings to GitHub Code Scanning |
| Inspect whether the repository has a complete CI adoption kit |
| Inspect or control product telemetry and deliberately provide a contact email |
| Merge receipts and run artifacts into JSON, Markdown, and HTML MCP risk graphs |
| Opt out of the default safe attack simulation on |
| Generate CI report for GitHub issue creation |
| Generate a static production/security report from run artifacts |
| Score an MCP server's health (0-100) |
| Generate an SVG health score badge for README |
| Show the free hosted snapshot, Individual Pro, and Release Gate Pilot options |
Run with no arguments to start demo. Use --menu for the interactive catalog.
What It Does
Check capabilities — connects to a server and verifies tools, prompts, and resources respond correctly.
Invoke tools — goes beyond listing. Actually calls safe tools (no required params / readOnlyHint) and reports which ones work and which ones crash.
npx @kryptosai/mcp-observatory scan deepDetect schema drift — diffs two runs and surfaces added/removed fields, type changes, and breaking parameter changes.
npx @kryptosai/mcp-observatory diff run-a.json run-b.jsonRecommend servers — scans your project for languages, frameworks, databases, and cloud providers, then cross-references the MCP registry to suggest servers you're missing.
npx @kryptosai/mcp-observatory suggestOr ask your agent "what MCP servers should I add?" when running in MCP server mode.
Security scanning — analyzes tool schemas for dangerous patterns: shell injection surfaces, broad filesystem access, missing auth, and credential leakage in responses.
npx @kryptosai/mcp-observatory test --security npx -y my-mcp-serverRecord / replay / verify — capture a live session, replay it offline in CI, and verify nothing changed. Like VCR for MCP.
# Record a session
npx @kryptosai/mcp-observatory record npx -y @modelcontextprotocol/server-everything
# Replay offline (no server needed)
npx @kryptosai/mcp-observatory replay .mcp-observatory/cassettes/latest.cassette.json
# Verify the live server still matches
npx @kryptosai/mcp-observatory verify cassette.json npx -y @modelcontextprotocol/server-everythingWatch for regressions — re-runs checks on an interval and alerts when something changes.
npx @kryptosai/mcp-observatory watch target.jsonScan locations
When you run scan, it looks for MCP configs in:
~/.claude.json(Claude Code)~/Library/Application Support/Claude/claude_desktop_config.json(Claude Desktop, macOS)%APPDATA%/Claude/claude_desktop_config.json(Claude Desktop, Windows).claude.jsonand.mcp.json(current directory)
Architecture
┌─────────────────────────┐
│ MCP Observatory CLI │
│ npx @kryptosai/mcp- │
│ observatory scan │
└───────────┬─────────────┘
│
┌───────────▼─────────────┐
│ Config Discovery │
│ (Claude, Cursor, etc.) │
└───────────┬─────────────┘
│
┌─────────────────┼─────────────────┐
▼ ▼ ▼
┌─────────────────┐ ┌──────────────┐ ┌──────────────────┐
│ Security Scan │ │ Attack Sim │ │ Schema Drift │
│ (shell, creds) │ │ (tool poison)│ │ (version diff) │
└────────┬────────┘ └──────┬───────┘ └────────┬─────────┘
│ │ │
└─────────────────┼───────────────────┘
▼
┌─────────────────────┐
│ Health Score │
│ (0-100 + verdict) │
└──────────┬──────────┘
│
┌────────────────┼────────────────┐
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ SARIF │ │ Markdown │ │ CI Gateway │
│ (Code Scan) │ │ Report │ │ (setup-ci) │
└──────────────┘ └──────────────┘ └──────────────┘CI / GitHub Action
Add Observatory to your MCP server's CI pipeline:
npx @kryptosai/mcp-observatory setup-ci --all --command "npx -y my-mcp-server" --sarif --schedule weeklyCheck the adoption kit:
npx @kryptosai/mcp-observatory setup-ci --doctorSuccessful test, run, and single-target scan checks also offer to convert the passing result into a CI adoption kit. That automatic conversion enables SARIF/Code Scanning and weekly scheduled checks by default; pass --no-ci-sarif when you only want a conservative workflow without Code Scanning upload.
Or create the workflow manually:
# .github/workflows/observatory.yml
name: MCP Server Check
on: [pull_request]
permissions:
contents: read
jobs:
observatory:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: KryptosAI/mcp-observatory/action@v1
with:
command: npx -y my-mcp-server
deep: true
security: true
comment-on-pr: false
set-status: falseGitLab CI
Runs the same test scan on merge requests and on main, publishing the SARIF as a GitLab SAST report so findings appear in the merge request security widget. This matches what setup-ci --ci-provider gitlab-ci generates.
# .gitlab-ci.yml
mcp-observatory:
image: node:22
rules:
- if: $CI_PIPELINE_SOURCE == 'merge_request_event'
- if: $CI_COMMIT_BRANCH == 'main'
script:
- npx @kryptosai/mcp-observatory test npx -y my-mcp-server --deep --security --sarif mcp-observatory.sarif
artifacts:
reports:
sast: mcp-observatory.sarifOr let the CLI write it for you:
npx @kryptosai/mcp-observatory setup-ci --ci-provider gitlab-ci --command "npx -y my-mcp-server"Action inputs:
Input | Description | Default |
| Server command to test | (optional; omit to scan discovered servers) |
| Path to target config JSON | |
| Path to MCP config file for multi-server matrix scan | |
| Also invoke safe tools |
|
| Run security analysis |
|
| Fail the action on issues |
|
| Fail the action when baseline verification detects drift |
|
| Post report as PR comment. Requires |
|
| Set a commit status check (green/red) on the HEAD SHA. Requires |
|
| Token for PR comments and commit statuses |
|
The action can comment on PRs and set commit statuses when the workflow grants write permissions. setup-ci generates read-only third-party-friendly workflows by default and lets maintainers opt into comments/statuses later. init-ci remains available as a backward-compatible alias. See action/README.md for all options.
Production teams with a near-term MCP approval decision can use the fixed-scope MCP Release Gate Pilot: an approve, gate, or defer decision for 1–3 servers in ten business days. See COMMERCIAL.md or request a decision at mcp-observatory.com/release-gate-pilot.
Evidence badges for MCP Observatory
MCP server maintainers can add a public compatibility/security signal to their README:
[](https://github.com/KryptosAI/mcp-observatory)Or generate a score badge from a live check:
npx @kryptosai/mcp-observatory badge npx -y my-mcp-server --output docs/mcp-health.svgSee the evidence distribution loop for the GitHub Action template, maintainer PR body, and badge rollout playbook. A badge is a public evidence signal, not a certification or endorsement.
Generate a pilot-ready production/security report from local run artifacts:
npx @kryptosai/mcp-observatory enterprise-report \
--account "Your Company" \
--format html \
--output observatory-enterprise-report.htmlFor clearer internal account attribution in CI, set:
MCP_OBSERVATORY_ORG=your-company.comTo deliberately associate a contact email and optional contact-channel slug with private telemetry analytics, run mcp-observatory telemetry identify --email <address> [--channel <slug>]. Setting an organization or identifying a contact does not authorize outreach.
Testing Feishu/Lark integrations? See the Feishu/Lark MCP guide.
Lock Files
$ npx @kryptosai/mcp-observatory lock # Snapshot all server schemas
$ npx @kryptosai/mcp-observatory lock verify # Verify no drift since last lockLock files are the package-lock for AI tools: commit the MCP contract, then make every tool, schema, prompt, or resource drift visible in CI. See MCP lock files.
Trend Tracking
$ npx @kryptosai/mcp-observatory history # Show health trends over timeNightly Scans
$ npx @kryptosai/mcp-observatory ci-report # Generate regression report for CIMCP Server Mode
No other testing tool is itself an MCP server. Add Observatory as a server and your AI agent can autonomously test, diagnose, and monitor your other MCP servers.
claude mcp add mcp-observatory -- npx -y @kryptosai/mcp-observatory serveYour agent gets 10 tools:
Tool | When to use it |
| Check if all your configured MCP servers are healthy |
| Test a specific server before installing or after updating |
| Get a quick health score and grade for a server |
| Capture a baseline of a working server for future comparison |
| Test against a recorded session — no live server needed |
| Confirm a server update didn't break anything |
| Check a server and see what changed since the last check |
| Find regressions between two check results |
| Retrieve previous check results for a server |
| Discover MCP servers that match your project stack |
An AI tool that checks other AI tools. It is a tool testing tools that serve tools.
Security
The MCP server runs inside AI hosts where an LLM chooses which tools to call. To prevent prompt-injection attacks:
Command allowlist: Only
npx,node,python,python3,uvx,docker,deno,bunare permitted as base executables. The CLI has no restrictions.Path validation: File-reading tools are constrained to the runs/cassettes directories.
No arbitrary execution: Use the CLI for unrestricted commands.
CLI vs MCP: Intentional Differences
Feature | CLI | MCP Server | Why |
| Polling loop | Single check + diff | Request/response doesn't support long-polling |
Interactive menu | Arrow-key navigation | Not available | MCP has no interactive UI |
Color output |
| Always plain text | MCP returns structured content |
| Renders saved artifacts | Not available | Agents read artifacts directly |
| Starts MCP server | N/A | Is the MCP server |
| Reads target config files | Inline params | MCP tools accept params directly |
| Not available (use | Available | Convenience for agents |
Compatibility
Works with any MCP server that uses standard transports:
Transport | Examples | Adapter |
stdio (most servers) | filesystem, memory, context7, brave-search, sentry, notion, stripe |
|
HTTP/SSE (remote) |
| |
Docker | All |
|
Servers needing API keys work via env in the target config. Python servers work via uvx. See the full compatibility matrix for tested servers and known issues.
Target config files
For more control (env vars, metadata, custom timeout):
{
"targetId": "filesystem-server",
"adapter": "local-process",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "."],
"timeoutMs": 15000,
"skipInvoke": false
}npx @kryptosai/mcp-observatory run --target ./target.jsonHTTP / SSE targets
{
"targetId": "my-remote-server",
"adapter": "http",
"url": "https://mcp.example.com/mcp",
"authToken": "${MCP_SERVER_TOKEN}",
"headers": {
"X-Api-Key": "$MCP_SERVER_API_KEY"
},
"timeoutMs": 15000
}Target configs support ${VAR}, $VAR, and env:VAR references in authToken, headers, and local-process env values.
How It Compares
Feature | Observatory | |||
Auto-discover servers | ✅ | — | — | — |
Check capabilities | ✅ | — | ✅ | ✅ |
Invoke tools | ✅ | — | — | ✅ |
Schema drift detection | ✅ | — | — | — |
Record / replay | ✅ | ✅ | — | — |
Verify against cassette | ✅ | — | — | — |
Response snapshot diffs | ✅ | — | — | — |
Benchmarking / latency | — | — | ✅ | — |
Jest integration | — | — | — | ✅ |
Works as MCP server | ✅ | — | — | — |
Each tool has strengths. Observatory focuses on regression detection and CI-friendly workflows. mcp-recorder is great as a transparent proxy. MCPBench is the go-to for performance benchmarking. mcp-jest is ideal if you're already in a Jest workflow.
Prior Art
The record/replay/verify pattern is inspired by:
VCR (Ruby) — pioneered cassette-based HTTP record/replay
Polly.js (Netflix) — HTTP interaction recording for JavaScript
mcp-recorder — MCP-specific traffic recording proxy
MCPBench — MCP server benchmarking
mcp-jest — Jest-style testing for MCP servers
Limitations
Servers requiring interactive OAuth (e.g., Google Drive) need pre-authentication before Observatory can connect
Custom WebSocket transports (e.g., BrowserTools MCP) are not supported
A few servers time out or close before init — see known issues and compatibility
Works with mcp-seatbelt
Scan before you trust. Then run npx -y @kryptosai/mcp-observatory@latest enforce --start-proxy — Observatory writes a deny-default mcp-seatbelt policy and starts the proxy. Observatory validates; seatbelt enforces.
Works with agent-obs
Secure your servers with Observatory. Trace your agents with agent-obs — an open-source agent execution tracer that records every tool call, computes A-F session grades, and shows you exactly where your agents spend time, burn tokens, and hit errors. Observatory tells you if a server is safe. agent-obs tells you what your agent did with it. Free, local-first, npm install -g agent-obs.
Contributors ✨
Thanks to these amazing people who have contributed:
leemeo3 — 3 Safety Index targets (Git, Chrome DevTools, Filesystem MCP)
albatrossflyon-coder — GitHub MCP Safety Index (#201)
tanishxdev — Legacy CLI deprecation warnings (#187)
sansynx — CLI format validation (#182)
Contributing
We welcome contributors! This project follows a Contributor Covenant Code of Conduct. The fastest way to get involved:
git clone https://github.com/KryptosAI/mcp-observatory.git && cd mcp-observatory && npm install && npm testThe most common first contribution is adding an MCP server to the Safety Index (10-15 minutes). See CONTRIBUTING.md for full guidelines, code standards, and the contributor recognition ladder.
If Observatory saved you a broken deploy, consider giving it a star. It helps others find the project.
Available Tools
13 toolscheck_serverA
Use this to test a specific MCP server before installing or after updating it. Launches the server by command, checks all capabilities, and saves a run artifact for future comparison. Example: check_server({ command: 'npx -y @modelcontextprotocol/server-everything' }). Use deep=true to invoke tools, security=true to analyze schemas for vulnerabilities.
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Additional arguments for the command. | |
| deep | No | Also invoke safe tools to verify they execute. | |
| command | Yes | The command to launch the MCP server (e.g. 'npx -y @modelcontextprotocol/server-everything'). | |
| security | No | Run security analysis on tool schemas. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes key behaviors: launches server, checks capabilities, saves artifact. Explains deep and security parameters' effects. No annotations, so description carries burden; it is thorough but could mention if there are any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences plus an example; no fluff. Purpose front-loaded, each sentence adds information. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a testing tool with 4 parameters and no output schema, description explains all necessary usage, example, and parameter purposes. Complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, providing baseline. Description adds value by explaining deep and security flags beyond schema descriptions, and gives a concrete example.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool tests a specific MCP server before installation or after update, with actions: launch, check capabilities, save artifact. It distinguishes from siblings by focusing on testing rather than scanning or scoring.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use (pre-install, post-update), providing clear context. Lacks explicit when-not-to-use or alternatives, but the purpose is well-defined.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
ci_reportB
Generate a CI regression report from run artifacts.
| Name | Required | Description | Default |
|---|---|---|---|
| artifactsDir | No | Directory containing run artifacts. Defaults to .mcp-observatory/runs/ |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden but only states it generates a report, implying a read operation. It does not disclose side effects, required permissions, or output format, leaving significant ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence of 9 words, perfectly front-loaded and efficient with no extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple schema (1 optional parameter) and no output schema, the description lacks specifics about the report's nature, contents, or return value, leaving the agent insufficiently informed for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds no new meaning beyond the schema's parameter details, merely echoing 'from run artifacts', which is already apparent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Generate a CI regression report from run artifacts' uses a specific verb (Generate) and resource (CI regression report from run artifacts), clearly distinguishing it from sibling tools like get_last_run or scan.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description offers no guidance on when to use this tool versus alternatives, such as diff_runs or check_server, nor does it specify prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
diff_runsA
Use this to find what changed between two server checks. Compares two run artifacts and surfaces regressions (things that broke), recoveries (things that got fixed), schema drift (added/removed/changed tool parameters), and gate status changes. Essential after updating a server.
| Name | Required | Description | Default |
|---|---|---|---|
| base | Yes | Path to the base run artifact JSON file. | |
| head | Yes | Path to the head run artifact JSON file. | |
| format | No | Output format: 'markdown' (default) or 'json'. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes what the tool surfaces (changes, regressions, etc.), but with no annotations, the description lacks information on side effects, permissions, or whether it reads/writes. The behavior is implied but not fully transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no fluff. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the main use case and output types, but lacks details on output format (though format parameter is in schema) and prerequisites like file existence. Given no output schema, it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions; the description adds context about the comparison but does not provide additional semantics beyond the schema for each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it finds changes between two run artifacts, listing specific types (regressions, recoveries, schema drift, gate status changes). However, it does not differentiate from sibling tools that may also compare runs or history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a specific use case 'Essential after updating a server', but does not include when to avoid using it or alternative tools for similar tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_historyC
Get health score trends for MCP servers from run history.
| Name | Required | Description | Default |
|---|---|---|---|
| target | No | Filter to a specific target ID. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It states it 'gets' data (read operation) but does not mention potential impacts, auth requirements, or rate limits. The behavior is minimally transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 10 words, highly concise and front-loaded with the core purpose. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should explain the return format (e.g., list of scores, timeline). It does not. Also missing details about required run history context. Incomplete for a tool with no annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter described. The description adds no additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves health score trends from run history, using a specific verb and resource. However, it does not explicitly distinguish from sibling tools like get_last_run or diff_runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as get_last_run (single result) or diff_runs (comparison). The description lacks context about appropriate use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_last_runA
Return the last Observatory run for a server, including handshake allow/deny from the local receipt. No hosted account required.
| Name | Required | Description | Default |
|---|---|---|---|
| targetId | Yes | The target ID to find the last run for (e.g. server name or command). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the operation returns data and adds context about handshake details and that no account is needed. However, it does not explicitly mention that this is a read-only operation, what happens if no run exists (e.g., returns null or errors), or any other behavioral nuances. It adds some value but lacks full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary purpose, and includes only essential additional context (handshake detail and no-account requirement). There is no wasted wording; every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is simple (one parameter, no output schema), the description covers the main operation and a key prerequisite. It would be more complete with a note on error handling (e.g., what happens if no run is found) or a mention that it's read-only. However, for its complexity, it is nearly complete and clear enough for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter `targetId`, which already includes an explanation. The tool description adds minimal semantic value beyond the schema—it repeats that it's for a server, but does not elaborate on the format, required patterns, or how it is used in the operation. The description is adequate but not enriching.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Return the last Observatory run for a server'. It specifies the resource (Observatory run), the scope (last run for a server), and adds a distinctive detail (including handshake allow/deny). This differentiates it from siblings like get_history (which likely returns all runs) and check_server (which likely checks status).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives. It mentions 'No hosted account required', which is a prerequisite/condition, but does not say 'use this to get the most recent run' or contrast it with siblings like get_history or scan. The usage context is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
lock_verifyA
Verify that live MCP servers still match a previously saved lock file. Detects schema drift, added/removed tools, and breaking changes.
| Name | Required | Description | Default |
|---|---|---|---|
| config | No | Path to MCP config file. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states what is detected but omits key behavioral traits: whether the tool is read-only, whether it modifies state, what triggers an error, or how the result is presented. For a verification tool, these gaps reduce transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One concise sentence that conveys the core purpose without wasted words. Information density is high.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (verification against a lock file) and no output schema, the description should hint at the return format (e.g., boolean, diff report). It doesn't explain what a lock file is or how it is created, assuming context from sibling tools. Adequate but with gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'config' has a description in the schema ('Path to MCP config file.'). The tool description does not add any additional information about the parameter, such as format, defaults, or how it relates to the lock file. With 100% schema coverage, baseline is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'verify' and the resource 'live MCP servers match a previously saved lock file'. It specifies the exact purpose: detecting schema drift, added/removed tools, and breaking changes. This distinguishes it from sibling tools like 'verify' (generic) and 'check_server' (single server check).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage context (when a lock file exists and drift detection is needed), but does not explicitly say when to use this tool vs alternatives. No exclusions or prerequisites are provided. With nine siblings, more guidance would be helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
recordA
Use this to capture a baseline of a working MCP server. Records all JSON-RPC traffic to a cassette file that can be replayed offline (no server needed) or used to verify future versions haven't broken anything. Like VCR for MCP.
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Additional arguments for the command. | |
| command | Yes | The command to launch the MCP server. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explains the core behavior (records all JSON-RPC traffic to a cassette file) and mentions offline replay capability. However, it lacks details about side effects, required permissions, or the format of the cassette file.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (three sentences), front-loaded with the purpose, and uses an analogy ('Like VCR for MCP') for clarity. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 parameters, no nested objects), the description is largely complete. However, it does not explicitly state the return value (e.g., the path to the cassette file), which would be helpful for an AI agent using the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters are described in the input schema (100% coverage). The description does not add any additional semantic meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: capture a baseline of a working MCP server by recording JSON-RPC traffic to a cassette file. It uses a specific verb ('capture', 'records') and resource, and distinguishes from siblings like 'replay' and 'verify'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool (to capture a baseline for offline replay or verification), but it does not explicitly state when not to use it or mention alternatives among siblings. The context is sufficient for an AI agent to decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
replayA
Use this to test a server without running it. Replays a previously recorded cassette offline and runs all checks against the recorded responses. Useful in CI or when the live server is unavailable.
| Name | Required | Description | Default |
|---|---|---|---|
| cassette | Yes | Path to a cassette JSON file. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool runs offline and replays recorded responses, but does not specify side effects (e.g., no modifications) or output format. Since no annotations are provided, the description carries the full burden and could be more detailed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with purpose, and every sentence adds value. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains purpose, behavior, and use cases but does not mention the output format or return value, which is important since no output schema is provided. For a simple tool, it's adequate but incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a description for the 'cassette' parameter. The description adds no additional semantic value beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool replays a recorded cassette to test a server offline, which is a specific verb-resource combination. It distinguishes from sibling tools like 'check_server' which likely test live servers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions usefulness in CI or when the live server is unavailable, giving clear context for when to use. It lacks explicit when-not-to-use or alternative mentions, but the context is strong enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scanA
Use this to check if all your MCP servers are healthy. Auto-discovers servers from Claude config files, connects to each one, and verifies tools/prompts/resources respond correctly. Use with deep=true to also invoke tools and confirm they actually execute. Returns pass/fail status for every server.
| Name | Required | Description | Default |
|---|---|---|---|
| deep | No | Also invoke safe tools to verify they execute. | |
| config | No | Path to a specific MCP config file. If omitted, scans default locations. | |
| security | No | Run security analysis on tool schemas. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations present, so description fully covers behavior: auto-discovers servers from config files, connects, verifies responses, and in deep mode invokes tools. Discloses return is pass/fail status.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise paragraph with front-loaded purpose, followed by parameter guidance. Every sentence adds value; no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Complete description for a health-check tool with 3 optional parameters and no output schema. Explains return format, auto-discovery, and verification steps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. Description adds usage context (e.g., deep=true invokes tools, config path specific, security analysis) beyond schema definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it checks MCP server health with auto-discovery and verification of tools/prompts/resources. Distinguishes from siblings like check_server (single server) and verify (different scope) by describing the comprehensive health check.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use this to check if all your MCP servers are healthy' and suggests when to use deep=true. Does not mention when not to use or alternatives, but provides sufficient context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
score_serverB
Use this to get a quick health grade for an MCP server. Runs all checks (capabilities, tool invocation, security) and returns a 0-100 score with A-F grade and detailed breakdown across protocol compliance, schema quality, security, reliability, and performance.
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Additional arguments for the command. | |
| command | Yes | The command to launch the MCP server. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries full burden. It states 'runs all checks' but does not disclose potential side effects (e.g., launching a server or modifying state), whether it requires permissions, or what happens on failure. This is insufficient transparency for a tool that executes commands.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the primary use case, and efficiently enumerates the checks and output. No unnecessary words; each sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema, the description adequately explains the return value (0-100 score, A-F grade, breakdown across multiple dimensions). It lists the areas checked but could mention prerequisites (e.g., server must be running) or result interpretation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond the schema's parameter descriptions; it merely repeats the concept of 'command' and 'args' without deepening understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action ('get a quick health grade') and the target ('an MCP server'), listing the checks performed and output format. It is distinct from sibling tools like 'scan' or 'check_server' but does not explicitly differentiate itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'Use this to get a quick health grade', implying usage context. However, it does not provide when-not-to-use guidance or compare with alternatives such as 'verify' or 'check_server' for more detailed checks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
suggest_serversA
Use this when setting up a project or wondering what MCP servers to add. Scans the working directory for languages, frameworks, databases, and cloud providers, lists currently configured servers, and cross-references the MCP registry to recommend servers you're missing.
| Name | Required | Description | Default |
|---|---|---|---|
| cwd | No | Working directory to scan for environment signals. Defaults to process.cwd(). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It describes scanning, listing, and cross-referencing behavior but does not disclose potential side effects, network usage, error handling, or whether it modifies any files. Adequate but not detailed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. First sentence gives usage context, second explains the process. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (scanning, listing, cross-referencing) and no output schema, the description explains input and general behavior but does not specify the output format or what exactly 'recommend servers you're missing' means. Slightly incomplete but still good for a suggestion tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only parameter 'cwd' has schema description 'Working directory to scan for environment signals. Defaults to process.cwd().' Description adds context about the scanning purpose, but the schema already covers the meaning. With 100% schema coverage, baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool scans working directory for languages, frameworks, databases, and cloud providers, lists configured servers, and cross-references the MCP registry to recommend missing servers. It sets it apart from siblings like 'scan' or 'check_server' by focusing on recommendations for project setup.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use this when setting up a project or wondering what MCP servers to add.' Provides clear context for when to use, but does not mention when to use alternative sibling tools like 'scan' or 'check_server' instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
verifyA
Use this after updating a server to confirm nothing broke. Connects to the live server, sends the same requests from a recorded cassette, and compares responses. Reports exactly what changed — added tools, removed parameters, different response shapes.
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Additional arguments for the command. | |
| command | Yes | The command to launch the MCP server. | |
| cassette | Yes | Path to a cassette JSON file. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explains key behaviors: connects to live server, sends requests from a recorded cassette, compares responses, and reports changes. It lacks details on potential side effects (e.g., read-only nature) or prerequisites, but covers core actions well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at two sentences, with no redundant information. Every sentence adds value, clearly stating purpose, method, and output.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description partially explains the output ('reports exactly what changed'). However, it doesn't specify the format or structure of the report, which could be critical for programmatic use. Otherwise, context is sufficient for a verification tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds no new information beyond the schema parameter descriptions (e.g., 'path to cassette', 'command', 'args'). It does not enhance semantic understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: verifying a server after an update by comparing responses against a recorded cassette. It specifies the verb ('confirm nothing broke') and resource ('server'), and effectively distinguishes from sibling tools like 'record' and 'replay'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit context for when to use ('after updating a server'), which is helpful. However, it does not mention when not to use it or name alternative tools (e.g., 'replay' vs 'verify'), leaving some ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
watchA
Use this to check a server and see what changed since the last check. Runs all checks, saves the result, and diffs against the previous run for the same target. Shows regressions, recoveries, and schema drift in one call.
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Additional arguments for the command. | |
| command | Yes | The command to launch the MCP server. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool runs all checks, saves results, diffs against previous runs, and shows regressions, recoveries, and schema drift. It does not mention side effects or auth needs, but the behavioral description is comprehensive for the tool's purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no wasted words. The purpose is front-loaded, and every sentence adds value. Excellent conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description explains what the tool shows (regressions, recoveries, schema drift). It clarifies that a previous run is needed by mentioning 'since the last check' and 'previous run for the same target.' Complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes both parameters (command and args). The description adds no additional parameter meaning, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it checks a server and shows changes since last check, specifying verb, resource, and scope. It distinguishes from siblings like 'scan' and 'diff_runs' by noting it runs all checks, saves results, and diffs against previous runs in one call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says 'Use this to check a server and see what changed since the last check,' providing clear context. It implies a usage scenario but does not explicitly mention when not to use or provide alternatives. Still, the guidance is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
3 tool updates
v1.35.10- Added
ci_report - Added
get_history - Added
record
8 tool updates
v1.35.4- Added
check_server - Added
get_last_run - Added
lock_verify - Added
replay - Added
scan - Added
score_server - Added
suggest_servers - Added
watch
11 tool updates
v1.33.0- Removed
check_server - Removed
ci_report - Removed
get_history - Removed
get_last_run - Removed
lock_verify - Removed
record - Removed
replay - Removed
scan - Removed
score_server - Removed
suggest_servers - Removed
watch
TDQS
Most tools have distinct purposes (e.g., scan auto-discovers, check_server tests a specific server, score_server returns a grade), but some comparison tools like diff_runs, verify, watch, and lock_verify could be confused due to overlapping functions.
Most tools follow a verb_noun pattern (get_last_run, check_server, diff_runs), but some are single verbs (scan, verify, watch) and ci_report is noun_noun, causing minor inconsistency.
13 tools cover the domain of MCP server monitoring, checking, recording, and verification without being excessive. Each tool earns its place.
The tool surface is complete: health checks, history, comparison, recording/replaying, suggestions, and CI reporting. No obvious missing functionality for the stated purpose.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Scan any MCP server for tool-poisoning, security, auth & license. Trust score before install.
Check if your MCP server is ready to publish on the MCP Registry, Smithery, or npm.
Conformance checker for MCP servers. Free, no key, verdicts recomputable and re-measured daily.
MCP Spec Compliance MCP — audits any MCP server.json against the official Model Context Protocol
Related MCP Servers
- AlicenseAqualityDmaintenanceDiagnose, secure, and benchmark your MCP servers. Zero-config CLI for Claude Code, Cursor, VS Code, and Windsurf.4183MIT
- AlicenseNot gradedqualityCmaintenanceDiagnose MCP servers — health checks, tool testing, token cost audits, conflict detection, and security scanning with 50+ prompt injection patterns. Works as CLI or MCP server inside Claude Desktop.1MIT
- AlicenseBqualityDmaintenanceAn MCP server that connects Claude Code to your codebase for automated code cleanup with scanning, planning, atomic fixes, and rollback safety.102MIT
- AlicenseNot gradedqualityAmaintenanceScans MCP servers for deprecated and removed protocol features, providing a dated migration checklist.18MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/KryptosAI/mcp-observatory'
If you have feedback or need assistance with the MCP directory API, please join our Discord server