Skip to main content
Glama
studiomeyer-io

mcp-protocol-conformance

Part of the StudioMeyer MCP Stack โ€” Built in Mallorca ๐ŸŒด ยท โญ if you use it

mcp-protocol-conformance

npm version npm downloads License Last commit GitHub stars

This is a Foundation build of the StudioMeyer MCP Factory: every other Factory build runs through this harness before promotion (npm publish, marketplace submit, upstream PR).

A note from us

We have been building tools and systems for ourselves for the past two years. The fact that this repo is small and has few stars is not because it is new. It is because we only just decided to share what we have built. It is not a fresh experiment, it is a long story with a recent commit.

We love building things and sharing them. We do not love social media tactics, growth hacks, or chasing stars and followers. So this repo is small. The code is real, it gets used, issues get answered. Judge for yourself.

If it helps you, sharing, testing, and feedback help us. If it could be better, an issue is more useful. If you build something with it, tell us at hello@studiomeyer.io. That genuinely makes our day.

From a small studio in Palma de Mallorca.

Related MCP server: MCP Test Server

Install

npm install --save-dev mcp-protocol-conformance

The package ships both a CLI (mcp-conformance) and a TypeScript library entry (import ... from "mcp-protocol-conformance").

CLI usage

Run against a stdio server

mcp-conformance run \
  --target stdio \
  --cmd node \
  --cmd-arg dist/server.js \
  --spec 2025-06-18 \
  --suite all \
  --format terminal

Run against an HTTP server (Streamable HTTP, 2025-03-26+)

mcp-conformance run \
  --target http \
  --url https://memory.studiomeyer.io/mcp \
  --header "Authorization:Bearer ${TOKEN}" \
  --spec 2025-06-18 \
  --suite full \
  --format json --out report.json

Run only a subset of suites

mcp-conformance run --target stdio --cmd ./server --spec 2025-06-18 \
  --suite jsonrpc,version,schema

Compare two manifests

mcp-conformance compare \
  --expected manifests/v1.json \
  --actual   manifests/v2.json

Assert no breaking changes between two reports

mcp-conformance assert-no-breaking \
  --baseline reports/main.json \
  --current  reports/pr-42.json

Exit codes: 0 clean, 1 failures, 2 invocation error.

Library usage

import {
  runFullSuite,
  generateReport,
} from "mcp-protocol-conformance";

const report = await runFullSuite(
  { kind: "stdio", cmd: "node", args: ["dist/server.js"] },
  "2025-06-18",
  { suite: "all" },
);

console.log(generateReport(report, "terminal"));
if (report.status === "fail") process.exit(1);

MCP server usage

The harness is itself an MCP server. Start it over stdio and any MCP client can call its 12 tools:

node dist/server.js

Tools (all read-only, all destructiveHint: false):

#

Tool

Purpose

1

runJsonRpcCompliance

JSON-RPC 2.0 error-code matrix + response-envelope (result/error mutual exclusivity)

2

runSpecVersionAssertion

Verify advertised protocolVersion

3

runTransportSuite

Transport-layer ping + session-id

4

runOauthPkceFlow

OAuth 2.1 PKCE S256 (mock-AS or real-tenant)

5

runToolSchemaValidation

inputSchema (+ 2025-11-25 outputSchema / title) is valid JSON-Schema

6

runCapabilityIntrospection

initialize.capabilities matches behaviour (+ 2025-11-25 tasks)

7

runRoundtripSmoke

One tools/call per advertised tool

8

runAnnotationsAudit

readOnlyHint / destructiveHint hygiene

9

runFullSuite

All suites + summary

10

generateReport

Render JUnit / JSON / terminal

11

compareManifests

Diff two tool manifests

12

assertNoBreakingChanges

Diff two FullReports

Compatibility matrix

2024-11-05

2025-03-26

2025-06-18

2025-11-25

jsonrpc

yes

yes

yes

yes

version (handshake)

yes

yes

yes

yes

transport (stdio)

yes

yes

yes

yes

transport (http)

n/a

yes

yes

yes

oauth (mock)

n/a

yes

yes

yes

oauth (real)

n/a

yes

yes

yes

schema

yes

yes

yes

yes

capability

yes

yes

yes

yes

smoke

yes

yes

yes

yes

annotations

warn-only

warn-only

yes

yes

tasks (capability)

n/a

n/a

n/a

yes

tool title/output

n/a

n/a

n/a

warn-only

yes = suite runs and produces actionable results. n/a = capability not in spec; suite skips automatically. warn-only = suite runs but the spec does not formally require the feature.

The 2026-07-28 RC (stateless core, SEP-2575/2567) is intentionally not here โ€” it ships final on 2026-07-28; conformance for it lands in a later release against the final SDK.

Integration in Factory builds

In every Factory build's package.json:

{
  "scripts": {
    "factory:conformance": "mcp-conformance run --target stdio --cmd 'node dist/server.js' --spec 2025-06-18 --suite full"
  }
}

In CI:

- run: npm run factory:conformance

Spec references

About StudioMeyer

StudioMeyer is an AI and design studio based in Palma de Mallorca, working with clients worldwide. We build custom websites and AI infrastructure for small and medium businesses. Production stack on Claude Agent SDK, MCP and n8n, with Sentry, Langfuse and LangGraph for observability and an in-house guard layer.

License

MIT โ€” Copyright (c) 2026 Matthias Meyer (StudioMeyer)

Available Tools

12 tools
assertNoBreakingChangesA
Read-onlyIdempotent

Compare a baseline FullReport against a current FullReport. Returns a list of breaking changes and warnings.

ParametersJSON Schema
NameRequiredDescriptionDefault
currentNo
baselineNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive. Description adds that it returns a list of breaking changes and warnings, but no additional behavioral context like side effects or request patterns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single clear sentence conveying action, inputs, and output without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, describing return type helps. However, lacking references to FullReport origin (e.g., from generateReport) and not explaining what constitutes a breaking change leaves some gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema coverage, the description names the two parameters as 'baseline FullReport' and 'current FullReport', providing basic semantic meaning that the schema lacks. However, no details on FullReport structure are given.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool compares two FullReport objects and returns breaking changes and warnings. It uses specific verbs and nouns, distinguishing it from siblings like compareManifests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives (e.g., compareManifests). No preconditions or exclusions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compareManifestsB
Read-onlyIdempotent

Diff two tool manifests โ€” added/removed/changed tools.

ParametersJSON Schema
NameRequiredDescriptionDefault
actualYes
expectedYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint and idempotentHint are true, so the description doesn't need to repeat that. However, it adds no additional behavioral context (e.g., permissions, side effects, or what 'changed' means). Neutral value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Extremely concise single sentence that frontloads the core purpose. Every word earns its place. No fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description fails to specify the output format (diff result), how 'changed' is determined, or any details about the nested object structure. For a diff tool with no output schema, this is insufficient for an agent to understand the return value.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not explain the two parameters ('expected' and 'actual'). It only hints at two tool manifests but gives no detail about expected format, which is critical given both are complex nested objects.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it diffs two tool manifests and identifies added/removed/changed tools. It uses a specific verb ('diff') and resource ('tool manifests'), and it distinguishes from sibling tools that focus on running compliance tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool vs alternatives. No mention of prerequisites, context, or when not to use it. The description provides no usage direction beyond stating the basic function.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generateReportB
Read-onlyIdempotent

Render a FullReport as JUnit XML, JSON, or human-readable terminal output.

ParametersJSON Schema
NameRequiredDescriptionDefault
formatYes
reportNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, establishing safety. The description adds format output options (beyond annotations) but does not elaborate on behavior like handling of missing report fields or performance characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one sentence long, direct, and contains no extraneous information. Every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (2 params, no output schema) and presence of annotations, the description minimally covers the tool's purpose and output formats, but lacks detail on input parameters and edge cases, making it adequate but not thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so description must compensate. It hints at the 'format' parameter values by listing output types, but it does not describe the 'report' parameter at all (type, structure, or constraints). This is insufficient for a 2-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Render'), identifies the resource ('FullReport'), and lists all output formats (JUnit XML, JSON, terminal), clearly distinguishing it from sibling tools that run tests or perform assertions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives (e.g., other report tools or the test-running siblings). It only states what it does, leaving usage context implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runAnnotationsAuditB
Read-onlyIdempotent

Apply the heuristic annotation rules (destructive name regex, readOnlyHint conflicts) to every advertised tool.

ParametersJSON Schema
NameRequiredDescriptionDefault
targetYes
specVersionNo2025-06-18

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive. The description adds specifics about applying heuristic rules (destructive name regex, readOnlyHint conflicts), providing behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no wasted words, front-loads the key action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description adequately conveys the core functionality but omits output details (no output schema) and does not explain how results are returned.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description provides no explanation of the parameters. The complex 'target' parameter and optional 'specVersion' are entirely undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it applies heuristic annotation rules to advertised tools, using a specific verb and resource. It differentiates from siblings by focusing specifically on annotation rules.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool vs alternatives, no exclusions, and no context on prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runCapabilityIntrospectionA
Read-onlyIdempotent

Compare the capabilities the target advertises in initialize against what tools/list, resources/list, prompts/list actually return.

ParametersJSON Schema
NameRequiredDescriptionDefault
targetYes
specVersionNo2025-06-18

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive traits. The description adds the behavior of comparing advertised vs actual capabilities, which is consistent but not extensive beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with no filler, effectively front-loading the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, no parameter descriptions, and 0% schema coverage, the description omits critical details about parameter usage and return values, leaving the agent underinformed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, yet the description does not explain any parameters (target, specVersion). The agent must infer from the schema alone, which is insufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('Compare') and resources ('capabilities... initialize... tools/list, resources/list, prompts/list'), and it distinguishes from sibling tools that perform other tests or assertions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains what the tool does but does not explicitly state when to use it versus alternatives like runJsonRpcCompliance or runFullSuite. Usage context is implied but not clearly delineated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runFullSuiteB
Read-onlyIdempotent

Run all (or selected) suites and produce a FullReport with summary + per-suite + per-check status.

ParametersJSON Schema
NameRequiredDescriptionDefault
oauthNo
suiteNo
targetYes
specVersionYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnly, idempotent, and non-destructive behavior. The description adds that it produces a 'FullReport' with structure, but does not disclose details like error handling, timeout, or dependency on server. It is adequate but not thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is front-loaded and contains no unnecessary words. It efficiently conveys the main action and output.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (4 parameters, no output schema, no param descriptions), the description is insufficient. It fails to explain key parameters like target or specVersion, and does not mention the tool's read-only nature or any prerequisites. The description leaves critical gaps for correct usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage, yet the description provides no explanation of any parameters (target, specVersion, oauth, suite). It only loosely references 'suites' but adds no meaning beyond the schema names and enums. This is a severe gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool runs 'all (or selected) suites' and produces a 'FullReport', which distinguishes it from sibling tools that run specific suites (e.g., runJsonRpcCompliance, runTransportSuite). The verb 'Run' and resource 'suites' are specific, and the scope is clarified.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus its siblings. It implies it is for comprehensive testing via 'all (or selected) suites', but lacks guidance on when to choose specific suite tools or when this is appropriate (e.g., for CI vs debugging).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runJsonRpcComplianceB
Read-onlyIdempotent

Run JSON-RPC 2.0 compliance suite (parse-error, invalid-request, method-not-found, invalid-params, internal-error) against a target MCP server.

ParametersJSON Schema
NameRequiredDescriptionDefault
targetYes
specVersionYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, indicating safe operations. Description adds the list of error cases but does not disclose additional behavioral traits like execution duration, prerequisites (e.g., server must be running), or error handling beyond the listed cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that front-loads the main purpose, but it omits critical parameter details. It is efficient but incomplete, which reduces its helpfulness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having 2 parameters (one complex) and no output schema, the description provides no insight into return values, success/failure indicators, or side effects. It is insufficient for an agent to fully understand the tool's behavior without additional documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, yet the description provides no explanation of the complex 'target' parameter (stdio vs http connection) or the 'specVersion' enum. The agent must rely entirely on the JSON schema, which is detailed but lacks natural language guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly specifies the tool runs a JSON-RPC 2.0 compliance suite targeting an MCP server, listing specific error cases. It distinguishes itself from sibling tools (e.g., runTransportSuite, runToolSchemaValidation) which cover other aspects of testing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied by the tool name and description but no explicit guidance on when to use versus alternatives, nor any prerequisites or exclusions. The sibling tools are different test suites, so the purpose is clear but not comparative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runOauthPkceFlowB
Read-onlyIdempotent

Exercise OAuth 2.1 PKCE S256 flow against the target. Mock-AS by default; real-tenant when --oauth-mode=real.

ParametersJSON Schema
NameRequiredDescriptionDefault
scopesNo
targetYes
clientIdYes
redirectUriYes
specVersionNo2025-06-18
authorizationServerUrlNo

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds the behavioral nuance of default mock-AS mode and real-tenant mode, but does not elaborate on other traits like side effects or authentication needs beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise sentences, with the core purpose in the first sentence and a mode distinction in the second. There is no redundancy or superfluous text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite moderate complexity (6 parameters, nested objects, no output schema), the description covers only the high-level flow and a mode switch. It lacks information on return values, error handling, and parameter roles, making it insufficient for complete understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description must explain parameter meanings. However, it mentions no parameters at all; terms like 'target', 'clientId', and 'redirectUri' are left undefined. This is a critical gap for an agent to invoke the tool correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as exercising an OAuth 2.1 PKCE S256 flow against a target, with distinct modes (mock vs. real). This verb+resource+mode combination is specific and differentiates it from sibling tools like runJsonRpcCompliance or runTransportSuite.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description only mentions a flag to switch between mock and real tenant, but provides no explicit guidance on when to use this tool versus alternatives. There is no mention of prerequisites, contexts, or exclusions relative to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runRoundtripSmokeB
Read-onlyIdempotent

Perform one tools/call per advertised tool. Fails on JSON-RPC transport errors, warns on tool-level isError=true.

ParametersJSON Schema
NameRequiredDescriptionDefault
targetYes
sampleArgsNo
specVersionNo2025-06-18

TDQS

B3.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark it as read-only and idempotent. The description adds behavioral details: fails on transport errors, warns on tool-level errors. This goes beyond what annotations provide, giving the agent a clear picture of failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, no extraneous words. Excellent conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (3 params, nested objects, no output schema), the description omits important context: how sampleArgs affects tool calls, what the return value looks like, and that it requires a running server. Incomplete for effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so description must compensate for parameter explanation. It does not mention any parameters (target, sampleArgs, specVersion), leaving their meaning and usage entirely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool performs one tools/call per advertised tool, making it a smoke test. It distinguishes from sibling testing tools by specifying its lightweight roundtrip nature, though could be more explicit about its narrow scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like runFullSuite or runToolSchemaValidation. The description does not mention prerequisites, context, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runSpecVersionAssertionA
Read-onlyIdempotent

Drive an initialize handshake and assert the target advertises the expected MCP spec version.

ParametersJSON Schema
NameRequiredDescriptionDefault
targetYes
expectedVersionYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, indicating a safe read operation. The description adds that the tool performs a handshake (implying network interaction) and assertion, which provides some behavioral context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no redundancy or unnecessary detail. Every word contributes to the purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description should mention return values or result format. It also lacks context about prerequisites (e.g., target must be a running MCP server). While adequate for a simple test tool, there are gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description partially compensates by stating that expectedVersion is the version to assert and target is the server to test. However, it does not explain the target structure (stdio vs http) beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool drives an initialize handshake and asserts the target advertises the expected MCP spec version. This specific verb+resource structure distinguishes it from sibling test tools like runJsonRpcCompliance or runTransportSuite.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives (e.g., runJsonRpcCompliance or runToolSchemaValidation) is provided. The description implies use for spec version testing but lacks explicit context or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runToolSchemaValidationA
Read-onlyIdempotent

Validate that every advertised tool has a well-formed JSON-Schema inputSchema. Optionally diff against an expected manifest.

ParametersJSON Schema
NameRequiredDescriptionDefault
targetYes
specVersionNo2025-06-18
expectedManifestNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, indicating no side effects. The description ('validate', 'diff') is consistent but adds no new behavioral traits beyond what annotations provide. No contradiction, but no extra transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the main purpose, no redundant information. Highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the tool's complexity (3 parameters, nested objects, no output schema, no schema descriptions), the description is minimal. It fails to explain the critical target parameter or how the tool connects to a server, making it incomplete for an AI agent to use effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain the three parameters (target, specVersion, expectedManifest). It does not mention any of them, leaving the AI agent without guidance on what 'target' (a complex union), 'specVersion', or 'expectedManifest' mean or how to use them. This is a critical gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool's purpose: validating that advertised tools have well-formed JSON-Schema inputSchema, with an optional diff against an expected manifest. The verb 'validate' and resource 'tool inputSchemas' are explicit, and it distinguishes from sibling tools like runJsonRpcCompliance or runSpecVersionAssertion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the primary use case is schema validation and mentions the optional diff feature. However, it does not provide explicit guidance on when to use this tool versus alternative siblings, nor does it state when not to use it. The context is clear but lacks exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

runTransportSuiteC
Read-onlyIdempotent

Verify the target's transport layer (stdio line-delimited JSON or Streamable HTTP).

ParametersJSON Schema
NameRequiredDescriptionDefault
targetYes
transportNoboth
specVersionNo2025-06-18

TDQS

C2.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows it's a safe read operation. The description adds that it 'verifies' the transport layer, which is consistent. No additional behavioral details (e.g., side effects, authorization needs) are disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence and front-loaded, but it omits essential details about parameters and usage. This is under-specification, not genuine conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given three parameters (including a complex required object target with conditional properties and enum options) and no output schema, the description is severely incomplete. It fails to explain how to specify the target or choose transport/specVersion, leaving the agent without necessary context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description provides no explanation of any parameter (target, transport, specVersion). The agent must infer meaning solely from the schema, which is insufficient for proper invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool verifies the transport layer, specifying the two transports (stdio and HTTP). It is specific and actionable, but does not differentiate from sibling tools like runJsonRpcCompliance or runRoundtripSmoke.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No information is provided about when to use this tool versus alternatives (e.g., runJsonRpcCompliance for JSON-RPC compliance, runRoundtripSmoke for roundtrip tests). The agent receives no context for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 12 tool updatesv0.2.1
    • First observedassertNoBreakingChanges
    • First observedcompareManifests
    • First observedgenerateReport
    • First observedrunAnnotationsAudit
    • First observedrunCapabilityIntrospection
    • First observedrunFullSuite
    • First observedrunJsonRpcCompliance
    • First observedrunOauthPkceFlow
    • First observedrunRoundtripSmoke
    • First observedrunSpecVersionAssertion
    • First observedrunToolSchemaValidation
    • First observedrunTransportSuite

TDQS

A3.5/5.0
Disambiguation5/5

Every tool has a clearly distinct purpose targeting a specific conformance aspect or utility. There is no overlap between tools like runJsonRpcCompliance, runTransportSuite, and runOauthPkceFlow, as they each test different parts of the protocol.

Naming Consistency5/5

All tool names follow a consistent verbNoun pattern in camelCase, using descriptive verbs like run, generate, compare, and assert. This uniform convention makes the tool surface predictable and easy to navigate.

Tool Count5/5

With 12 tools, the server covers a comprehensive set of conformance testing areas without being overwhelming. Each tool serves a necessary function, from individual compliance checks to full suite execution and reporting.

Completeness5/5

The tool set covers all major MCP protocol conformance areas: JSON-RPC errors, spec version, transport, OAuth, tool schema, capability introspection, roundtrip testing, annotations, and regression comparison. No obvious gaps remain for the stated purpose.

Maintenance

ActivityActive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    A TypeScript-based test implementation for the Model Context Protocol (MCP) with example configurations and Jest test cases. Provides a development foundation for building and testing MCP server functionality.
    10
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    A comprehensive reference implementation demonstrating all features of the Model Context Protocol (MCP) specification, serving as documentation, learning resource, and testing tool for MCP implementations.
    1
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    A comprehensive testing server for validating MCP scanner tools and implementations, providing diverse tools, resources, and prompts that exercise various aspects of the Model Context Protocol specification.
    19
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/studiomeyer-io/mcp-protocol-conformance'

If you have feedback or need assistance with the MCP directory API, please join our Discord server