Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Kryve Agent Evaluation MCP Generate a test plan for our customer support agent."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Un serveur MCP local et open source pour concevoir des tests d’agents IA, noter une exécution et documenter les preuves avant d’accorder davantage d’autonomie.
Il expose trois outils en lecture seule :
generate_test_plan : construit des cas nominaux, ambigus et sensibles selon le niveau de risque ;
score_agent_run : calcule une auto-évaluation provisoire du résultat, des preuves déclarées, des validations et du coût des corrections ;
evaluation_scorecard : renvoie les colonnes d’une scorecard réutilisable.
npm install
npm run buildPuis lance le serveur sur stdio :
npm startConfiguration d’un client MCP :
{
"mcpServers": {
"kryve-agent-eval": {
"command": "node",
"args": ["/chemin/vers/kryve-agent-eval-mcp/dist/src/index.js"]
}
}
}Related MCP server: Coval MCP Server
npx @modelcontextprotocol/inspector node dist/src/index.jsUne démo réussie ne prouve pas qu’un agent est fiable. Le test doit mesurer un résultat attendu, conserver des preuves, détecter les actions hors permission et rendre visible le coût des corrections humaines.
Pour comprendre le protocole et choisir une infrastructure :
Le kit CSV complémentaire est disponible dans le dépôt Kryve Agent Evaluation Kit.
Le score est une grille opérationnelle simple, pas une certification. Les champs sont déclarés par l’appelant : chaque résultat porte donc le statut unverified_self_report, n’est jamais éligible à une décision d’autonomie et doit être recoupé avec des preuves indépendantes.
La similarité textuelle sert uniquement d’indicateur provisoire. Elle ne remplace ni des critères structurés, ni un journal d’actions, ni une validation humaine vérifiable.
MIT — Kryve Solutions.
Return a reusable scorecard structure for documenting AI-agent tests.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, which cover the safety profile. The description adds the clarifying context that this returns a reusable structure for documentation purposes. It does not contradict the annotations, but it does not reveal much beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, front-loaded with the action verb and resource. Every word earns its place, and there is no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless tool with safety annotations, the description is reasonably complete. However, there is no output schema, so the description should more clearly explain what the 'scorecard structure' contains or how it relates to the sibling tools like score_agent_run. Without that, an agent may not fully understand the return value.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there are no parameter semantics to document. The description correctly implies a parameterless invocation by simply describing the action and output. Baseline of 4 for no-parameter tools is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Return') and a clear resource ('reusable scorecard structure for documenting AI-agent tests'). It conveys that this tool provides a template rather than performing scoring, which distinguishes it from siblings like score_agent_run and generate_test_plan, though it does not explicitly name them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied: call this when you need a reusable scorecard structure. However, there is no explicit guidance on when to prefer this over the sibling tools or any exclusions. It would benefit from stating that it returns a template rather than executing or scoring anything.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate practical test cases and success criteria for an AI-agent use case.
| Name | Required | Description | Default |
|---|---|---|---|
| risk | Yes | Impact level if the agent makes a mistake | |
| tools | No | Tools the agent may use | |
| useCase | Yes | Real task the agent must complete |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations (readOnlyHint=true, idempotentHint=true) already cover the operation's safety profile, so the bar for the description is lower. The description adds that the output is 'practical test cases and success criteria,' but does not disclose additional behavioral traits such as output format, determinism, or any side effects beyond what annotations already imply. It does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence that front-loads the core purpose and avoids any filler or repetition. It is appropriately sized for a tool with a simple, well-documented schema and annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, the schema documents all parameters, and the annotations cover safety, so the description is nearly sufficient on its own. Since there is no output schema, the description helpfully states that the output is test cases and success criteria, though it could go one step further by mentioning the expected return format or how the generated plan is meant to be used.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear descriptions for all three parameters (useCase, risk, tools), so the schema already carries the parameter meaning. The description does not add any parameter-specific information beyond what the schema provides, and maps only loosely to the useCase parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('Generate') and resource ('test cases and success criteria') for an AI-agent use case, so an agent can understand what the tool produces. It does not explicitly distinguish itself from the sibling tools score_agent_run and evaluation_scorecard, but the generation purpose is clearly distinct from scoring.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used when test cases and success criteria are needed for an agent use case, but it gives no explicit guidance about when to choose this tool over the siblings or when not to use it. There are no exclusions or alternative routing, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Create an unverified self-assessment of outcome, evidence, control and correction cost. Never use it alone to grant autonomy.
| Name | Required | Description | Default |
|---|---|---|---|
| task | Yes | ||
| actualOutcome | Yes | ||
| expectedOutcome | Yes | ||
| evidenceProvided | Yes | ||
| correctionsNeeded | Yes | ||
| unauthorizedActions | Yes | ||
| humanApprovalObtained | Yes | ||
| requiredHumanApproval | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so the main behavioral risk (side effects) is covered. The description adds important trust context: the result is unverified and must not be the sole basis for autonomy decisions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the function and closed with a high-value caution. No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 8 required parameters, no output schema, and no parameter descriptions, the description is too thin: it omits how the score is computed or returned, what evidence/control data is expected, and how it relates to evaluation_scorecard. The 'never use alone' warning is valuable but does not make the tool fully callable without further context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description only groups parameters into broad categories ('outcome, evidence, control and correction cost') rather than explaining individual required inputs. With 8 required parameters and no property descriptions, the prose does not compensate enough for an agent to know parameter-specific meanings or formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a concrete action ('Create an unverified self-assessment') and a specific subject ('of outcome, evidence, control and correction cost'), so an agent can tell it is a scoring tool. It does not explicitly differentiate from sibling evaluation_scorecard, though 'unverified' hints at a provisional role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The explicit warning 'Never use it alone to grant autonomy' is a clear when-not-to-use signal and a behavioral boundary. No sibling alternative is named, and there is no positive trigger condition, so it falls short of a full routing guide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v1.1.0evaluation_scorecardgenerate_test_planscore_agent_runEach tool addresses a distinct phase of evaluation: generating test plans, scoring an agent run, and retrieving a scorecard structure. There is no functional overlap or ambiguity between them.
Two tools follow a clear verb_noun pattern (generate_test_plan, score_agent_run), but evaluation_scorecard is noun-led rather than verb-led. The inconsistency is minor and all names remain readable and predictable.
Three tools is an appropriate, focused scope for an evaluation-oriented MCP server. Each tool serves a core need without bloat, making the surface easy to navigate.
The set covers planning, assessment, and structure retrieval, but lacks an explicit verification or finalization tool. The 'unverified' caveat on score_agent_run hints at this gap, but the core workflow is still functional.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Test the voice agents you run: scored transcripts, pass/fail verdicts, latency and WER metrics.
Read-only, deterministic AI triage and readiness tools implementing Sophon's published rubrics.
130+ QA & dev tools for AI agents: prompt injection, RAG testing, VLM eval, guardrails. Free.
A read-only verified record of agent-operable GTM tools: search, fetch, compare, track changes.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/louislaurent1/kryve-agent-eval-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
