mcpgrade is an MCP server that evaluates other MCP servers on agent usability. It provides three tools:
grade_mcp_server: Score an MCP server (A–F) on description quality, schema design, naming, token cost, and consistency. Accepts a remote URL, local launch command, or a JSON snapshot. Returns overall grade, per-category scores, finding counts, and a prioritized list of defects with concrete fixes. Optionally limit the number of findings returned.
explain_rule: Get the full rationale for a specific grading rule by its identifier (e.g., D004, N002, S008), including what it detects, why it degrades agent behavior, and how to fix it.
list_grading_rules: List all grading rules with identifiers, severities, and one-line summaries. Can be filtered by category (descriptions, naming, schema, token-cost, consistency, or all).
Lighthouse for MCP servers. Your server can be 100% spec-compliant and still fail agents — vague descriptions, token-bloated schemas, confusable tool names. mcpgrade scores what compliance checkers can't: whether an LLM can actually use your tools.
npx mcpgrade https://your-server.example.com/mcp # streamable HTTP
npx mcpgrade --stdio "node ./my-server.js" # local stdio server
npx mcpgrade --snapshot tools.json # saved tools/list output
npx mcpgrade https://mcp-us.example.com/mcp/streamable \
--header "Authorization: Bearer $TOKEN" # authenticated remote serverZero config. No API key. Report in seconds.
Category | Weight | Examples |
Descriptions | 30% | missing/too-short descriptions, undocumented params, placeholder text, duplicate descriptions |
Schema design | 30% | missing types, no |
Naming | 15% | confusable names ( |
Token cost | 15% | catalog total budget, per-tool budget — agents pay your schema on every request |
Consistency | 10% | catalog-wide uniformity; with |
Every finding comes with a concrete fix. Scores are density-normalized: 3 broken tools out of 3 is an F; 3 out of 30 is a dent.
Related MCP server: AgentRank
mcpgrade — agent usability report
target: examples/bad-server.json · 4 tools
F 37/100
Descriptions ░░░░░░░░░░░░░░░░░░░░ 0
Naming ███████████░░░░░░░░░ 55
Schema design ███░░░░░░░░░░░░░░░░░ 13
Token cost ████████████████████ 100
Consistency ████████████████████ 100
findings: 6 errors · 10 warnings · 2 info
Descriptions
✖ D002 [get_user] Description of "get_user" is only 12 chars ("Gets a user.").
↳ Expand to at least one full sentence: what it does, when to use it, what it returns.
...mcpgrade also runs as an MCP server, so an agent can grade other servers on your behalf:
claude mcp add mcpgrade -- npx -y mcpgrade serveOr add it manually:
{
"mcpServers": {
"mcpgrade": { "command": "npx", "args": ["-y", "mcpgrade", "serve"] }
}
}Three tools, deliberately: grade_mcp_server, explain_rule, list_grading_rules.
Security. In serve mode the target string is chosen by a model, so local launch
commands are restricted to an allowlist (npx, node, python, python3, uv,
uvx, deno, bun, docker). URLs and .json snapshots are always allowed. The
CLI has no such restriction.
Dogfooding. The serve catalog is graded by mcpgrade in CI and must score an A with zero errors (test/serve.test.ts) — if a change drops the grade, the fix is the catalog, not the threshold. Current self-score:
A 96/100 3 tools · 0 errors · 1 warningThe one warning is a rule I disagree with on this catalog: S005 flags target
for describing a fixed value set in prose without an enum. The prose lists
permitted command prefixes for an otherwise free-form string, so an enum is not
expressible. Left in place rather than suppressed — the ruleset is opinionated by
design, and disagreements belong in the open (#10).
Most hosted MCP servers require a bearer token. Pass headers with --header
(repeatable), or set MCPGRADE_HEADERS="Authorization: Bearer …; X-Tenant: acme":
npx mcpgrade https://your-host/mcp --header "Authorization: Bearer $TOKEN"Streamable HTTP is tried first, with an automatic SSE fallback for servers on the
older transport. mcpgrade only calls tools/list — it never invokes a tool unless
you pass --probe.
Header values are treated as secrets: they go to the transport and nowhere
else — not the report, not --json output, not the eval envFingerprint. MCP
serve mode accepts no headers at all, since there the target is chosen by a model
and a model has no business handing out credentials.
mcpgrade <target> --json # machine-readable
mcpgrade <target> --fail-on error # exit 1 on errors — gate your PRs
mcpgrade <target> --disable S008,N001 # tune rules
mcpgrade rules # list all rulesI integrate first-party and third-party MCP connectors into a production AI agent for a living. Most MCP servers fail agents in the same ten ways — none of which show up in a spec compliance check. So I wrote the linter I wished server authors had run before shipping.
Different tools, different questions. mcp-lint checks whether your tool schemas parse correctly across clients (Claude, Cursor, OpenAI strict mode, ...) — syntax-level compatibility. mcpgrade measures whether a model can actually use your tools — description quality, naming confusion, token economics, and live LLM tool-selection accuracy. A server can pass mcp-lint cleanly and still score an F here, and vice versa. They compose well: lint for compatibility, grade for usability. Full side-by-side with concrete outputs: docs/comparison.md.
v0.1 — static lint engine, 24 rules, A–F scoring
v0.2 — --eval: LLM-powered live testing — synthetic task generation, blind tool selection, argument validation, refusal accuracy, confusion pairs. Calibrated on real servers (methodology); costs ~$0.05–0.2 per server on Haiku. Bring your own ANTHROPIC_API_KEY, or any OpenAI-compatible endpoint via --eval-base-url (DeepSeek, OpenRouter, ...); --eval-mock runs offline. Respects HTTPS_PROXY.
v0.3 — mcpgrade serve: runs as an MCP server so an agent can grade other servers (allowlisted launchers; the catalog is graded by mcpgrade in CI and must hold an A). Plus envFingerprint on every eval result — catalog hash, model, temperature, prompt version, task policy — so two scores are comparably or visibly incomparable.
Also shipped: GitHub Action for CI gating, and a public leaderboard of 36 popular servers.
v0.4 — the failure taxonomy work, driven by reader feedback: four-outcome scoring, held-out task authoring, silent-vs-observable failures, cross-server collisions, multi-hop evaluation, rule-entailment dedup. Dynamic badges when the scoring model settles.
MIT
Returns the full rationale for one grading rule: what it detects, why it degrades agent behaviour, and how to fix it. Use this after grade_mcp_server reports a finding you want to understand or dispute, rather than guessing what a rule ID means.
| Name | Required | Description | Default |
|---|---|---|---|
| rule_id | Yes | The rule identifier exactly as it appears in a finding, e.g. "D004", "N002", "S008". Case-insensitive. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden for behavioral disclosure. It clearly states the tool 'returns' information, implying a read-only, non-mutating operation. It also details the content of the rationale, which provides realistic expectations. However, it does not explicitly mention error handling, side effects, or authentication requirements, which would be beneficial for full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the purpose, and every clause adds value. The first sentence explains what it returns, and the second gives usage context. There is no redundancy or irrelevant detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one parameter, no output schema, no annotations). The description compensates for the missing output schema by listing the content of the rationale, which suggests the return structure. It also explains the tool's role in the broader workflow. While it could be more explicit about the exact return format, the description is complete enough for an experienced user to understand and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the existing parameter documentation is complete. The tool description does not add additional meaning about the parameter beyond what the schema already provides (e.g., format, case-insensitivity, examples). It refers to 'rule ID' generically but does not elaborate further, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Returns') and identifies a concrete resource ('the full rationale for one grading rule') with explicit content categories ('what it detects, why it degrades agent behaviour, and how to fix it'). It clearly distinguishes itself from siblings by focusing on explaining a single rule rather than listing or grading.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool ('after grade_mcp_server reports a finding you want to understand or dispute') and what not to do ('rather than guessing what a rule ID means'). This provides clear situational guidance and references the sibling tool as the trigger context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scores an MCP server on agent usability (A–F) and returns the specific defects that cost it points, each with a concrete fix. Grades description quality, schema design, tool naming, token cost and catalog consistency — the properties that determine whether a model picks the right tool and fills valid arguments, which spec-compliance checks do not measure. Returns a grade, per-category scores, finding counts by severity, and a prioritized finding list. Use this before depending on a third-party server, or after changing your own catalog.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes | What to grade. Three accepted forms: a remote server URL, e.g. "https://mcp.example.com/mcp"; a local launch command, e.g. "npx -y @modelcontextprotocol/server-memory"; or a path to a saved tools/list JSON snapshot, e.g. "./tools.json". Local commands must start with one of: npx, node, python, python3, uv, uvx, deno, bun, docker. | |
| max_findings | No | Maximum number of findings to return, most severe first. Defaults to 20. Raise it when you intend to fix everything, e.g. 100. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It transparently describes the outputs (grade, per-category scores, finding counts, prioritized list) but does not mention potential side effects such as executing local commands or making network calls when grading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured into four sentences, front-loaded with the core action, and each sentence adds distinct information: grading scope, graded properties, return format, and usage scenarios. It is dense but avoids redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, what is measured, return format, and when to use it, compensating for the lack of an output schema. Minor gaps like execution side effects exist, but overall it is a well-rounded description for a two-parameter evaluation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with detailed explanations of both target and max_findings. The tool description adds no additional parameter-specific meaning beyond the schema, so the baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Scores an MCP server on agent usability (A–F)' which clearly identifies the action and resource. It distinguishes from sibling tools by focusing on grading a server rather than explaining or listing rules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage context: 'Use this before depending on a third-party server, or after changing your own catalog.' It does not mention exclusions or alternatives, but the primary use case is clearly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Lists every rule mcpgrade applies, with its identifier, severity and one-line summary. Use this to see what is checked before grading, or to find the identifier of a rule to pass to explain_rule.
| Name | Required | Description | Default |
|---|---|---|---|
| category | Yes | Which rules to list. Pass "all" for the complete ruleset, or one category to narrow it, e.g. "descriptions". |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the burden. It discloses the return content (identifier, severity, one-line summary) and implies read-only behavior. It does not cover edge cases or rate limits, but for a simple list query, this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core function, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter and no output schema, the description fully explains the purpose, return contents, and usage context. Minor omission: no mention of output list shape or size, but this is not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a detailed enum description for category, so the description adds no additional semantics. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly identifies the action (lists) and resource (every rule mcpgrade applies), and specifies the output fields (identifier, severity, one-line summary). Distinguishes itself from explain_rule by noting it can provide identifiers for that tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states two use cases: to see what is checked before grading, and to find an identifier to pass to explain_rule. This differentiates when to use this vs. explain_rule, though it doesn't mention grade_mcp_server directly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v0.1.0explain_rulegrade_mcp_serverlist_grading_rulesEach tool has a clearly distinct purpose: listing rules, explaining a rule, and grading a server. There is no overlap or potential for confusion between them.
All tool names follow a consistent verb_noun pattern: explain_rule, list_grading_rules, grade_mcp_server. The verbs are action-oriented and the objects are specific, making the names predictable and readable.
Three tools is well-scoped for this niche server. Each tool earns its place by covering the complete workflow of discovering, understanding, and applying grading rules.
The tool surface is complete for its stated purpose: list rules to discover them, explain rules to understand them, and grade a server to put them into action. There are no obvious gaps or dead ends.
Reproducible benchmarks and reliability evidence for agent tools.
65+ AI tools as MCP: research, write, code, scrape, translate, RAG, agent memory, workflows
Graph-native persistent memory for AI agents — 33 MCP tools, zero-LLM writes.
Free public MCP for AI agents — 193 tools, 44 workflows. No API key.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/TengByte/mcpgrade'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
