The groundcheck server evaluates RAG (Retrieval-Augmented Generation) outputs for faithfulness, hallucination detection, and retrieval quality — using your MCP client's own model as the judge, requiring no separate API keys.
Evaluate Faithfulness (groundcheck_evaluate_faithfulness): Score how well an answer is supported by source chunks, claim by claim. Returns a 0–1 score, counts of supported/unsupported/contradicted claims, and per-claim verdicts. Supports concise (problems only) or detailed (all claims) output.
Detect Hallucinations (groundcheck_detect_hallucinations): Identify only unsupported or contradicted claims, returning specific answer spans and reasons — ideal for fix-it loops. Returns an empty list if the answer is clean.
Evaluate Retrieval Quality (groundcheck_evaluate_retrieval): Compute precision@k, recall@k, MRR, and NDCG. Use gold relevance labels for instant, math-only calculation, or LLM-graded relevance (0–3 per chunk) when no labels are available.
Compare Two Answers (groundcheck_compare): A/B judge two candidate answers across criteria like faithfulness, completeness, and relevance. Mitigates position bias by running both orderings and reporting ties where verdicts flip.
Run Evaluation Suites (groundcheck_run_suite): Batch-evaluate multiple cases (inline or from a JSONL file), persisting a full report and returning a summary with mean faithfulness, mean NDCG, and the worst-performing cases.
Fetch Reports (groundcheck_get_report): Retrieve a previously saved report by ID in concise (summary) or detailed (all case results) format. No model calls required.
Generate Dashboards: Use the groundcheck-dashboard CLI to produce a self-contained HTML dashboard from persisted reports for visual analysis.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@groundcheck Check if this answer is faithful to the provided documents."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Playwright verifies your UI. Sentry verifies your runtime. groundcheck verifies your AI's answers.
An MCP server that lets any AI agent evaluate RAG outputs -- faithfulness scoring, hallucination detection, and retrieval quality metrics -- with zero API keys, using MCP sampling so the judge model is whatever the connected client is already running.
You've vibe-coded a RAG app. It answers, but sometimes it makes stuff up, and you can't tell when. There's no test for "did the model lie about what the documents say." groundcheck is that test.
Related MCP server: multivon-mcp
groundcheck-dashboard --out docs/dashboard.html generates a single
self-contained HTML file from your report store -- summary cards, a
faithfulness-by-report chart, a sortable per-case table, and worst-5-cases,
all with Chart.js loaded from a CDN and the data embedded inline. Opens
straight from disk, no server required.
docs/dashboard.html in this repo is generated from a real report: knowledge-assistant's 18-query eval (see its Evaluation section, cross-linked above) -- not synthetic smoke-test numbers. Enable GitHub Pages on this repo to get a live, clickable link instead of a clone-and-open file (see Roadmap).
Claude Code:
claude mcp add groundcheck -- uvx groundcheckClaude Desktop -- add to claude_desktop_config.json
(full example):
{ "mcpServers": { "groundcheck": { "command": "uvx", "args": ["groundcheck"] } } }Cursor -- same shape, in .cursor/mcp.json (see
examples/claude_code.md for the full config).
No API key needed if your client supports MCP sampling. Try it: ask your
agent to run groundcheck_detect_hallucinations on an answer and its
sources.
Tool | What it does | Judge? |
| Claim-by-claim faithfulness score (0-1) against sources | LLM (sampling) |
| Only the unsupported/contradicted claims, for a fix-it loop | LLM (sampling) |
| precision@k, recall@k, MRR, NDCG -- gold labels (instant) or LLM-graded | Optional |
| Judge which of two candidate answers is better, with position-bias mitigation | LLM (sampling) |
| Batch-evaluate a set of cases (inline or JSONL), persist a report | LLM (sampling) |
| Fetch a persisted report by id | None |
flowchart LR
subgraph Client["Your MCP client (Claude Desktop / Code / Cursor)"]
Model[("Your LLM")]
end
Client -- "tool call" --> Server["groundcheck MCP server"]
subgraph Server["groundcheck"]
Det["Deterministic tools\nmetrics.py -- pure Python\nprecision@k, recall@k, MRR, NDCG"]
Judged["LLM-judged tools\nclaim decomposition + verification"]
end
Judged -- "sampling/createMessage" --> Model
Model -- "judged verdicts" --> Judged
Server -- "result" --> ClientThe split matters: retrieval metrics with gold labels are pure math and run
instantly with zero model calls. Faithfulness, hallucination detection, and
compare need semantic judgment, so they call back into your own connected
model via MCP sampling
-- no separate API key, no separate bill. If your client doesn't support
sampling yet, set ANTHROPIC_API_KEY as a fallback; if neither is available,
you get a clear error naming both options.
Cost: deterministic metrics are free (no model calls). LLM-judged tools cost 1-2 model calls via your client's existing inference -- no API key required on top of what you're already paying your client for.
Not an observability platform. It doesn't collect traces, dashboards, or alerts over time -- it scores the RAG output you hand it, once, when you ask. For production observability, look at LangSmith or Arize Phoenix.
Not for agent-trajectory evals. It judges answers against sources, not whether an agent took the right sequence of actions.
Not enterprise-scale. Reports are local JSON files. If you need multi-tenant dashboards, RBAC, or dataset versioning at scale, LangSmith or Phoenix are the right tool.
evals/RESULTS.md tracks tool-selection and hallucination-detection accuracy before and after tuning tool docstrings -- the actual before/after numbers, not just the final descriptions.
Real RAG app: knowledge-assistant (a multi-tenant document Q&A app) uses groundcheck to score its own real pipeline output -- 18 real queries against a real document, 85% mean faithfulness, 94% mean NDCG@5. The interesting finding wasn't a perfect score: the app never hallucinated a fact, but groundcheck's strict literal-support judge flagged several accurate paraphrases as "unsupported" -- a useful data point about judge strictness, not just app quality. Full breakdown in its README Evaluation section.
Read-only, compute-only server: no shell access, no network egress except an
opt-in ANTHROPIC_API_KEY fallback, no filesystem writes outside the local
report store. dataset_path is validated against an allowlisted directory
to prevent path traversal. Full threat model in SECURITY.md.
MCP Tasks primitive for async run_suite on large datasets.
MCP Apps report UI (render a report inline instead of raw JSON).
Publish to PyPI and the MCP Registry (packaging is ready; not yet published).
Live FastAPI dashboard (v2) -- groundcheck-dashboard's static HTML file is
the v1 by design (zero deployment burden); a running app with live updates
is a reasonable next step once there's a reason to keep one up.
Enable GitHub Pages for docs/dashboard.html so there's a live, clickable
"see eval results" link with no clone required.
uv sync --all-extras
uv run pytest
uv run ruff check .MIT licensed. See LICENSE.
Judge which of two candidate answers to query is better.
Use this to A/B two RAG configurations (prompts, retrievers, models) on
the same query. Position bias is mitigated automatically: the judge sees
(A,B) and (B,A) in separate calls, and any criterion whose verdict flips
with presentation order is reported as a tie rather than a pick.
Args:
query: the shared query both answers respond to.
answer_a: first candidate answer.
answer_b: second candidate answer.
sources: optional list of {id, text} chunks, used to judge faithfulness.
criteria: judged criteria (default ["faithfulness", "completeness", "relevance"]).
Returns a winner ("a"/"b"/"tie/uncertain"), a verdict per criterion, and a
brief rationale. Costs 2 model calls via your client's sampling -- no API
key needed if your client supports sampling.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| sources | No | ||
| answer_a | Yes | ||
| answer_b | Yes | ||
| criteria | No |
| Name | Required | Description |
|---|---|---|
| winner | Yes | |
| criteria | Yes | |
| rationale | Yes | Brief overall rationale, notes disagreement if any. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond annotations by explaining the tie mechanism (double call to mitigate position bias), the return structure (winner, verdict per criterion, rationale), and resource usage (costs 2 model calls, no API key needed if sampling supported). Annotations already indicate read-only behavior, so no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a concise one-line summary followed by usage context and an Args list. Every sentence adds value, and it is appropriately sized without redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers most aspects: purpose, usage, parameter semantics, behavioral details, and return. However, it could mention potential error cases or edge scenarios (e.g., empty answers). Given the presence of an output schema, return values are adequately described. The tool's role among siblings is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description provides clear explanations for all parameters in the Args section: query, answer_a, answer_b, sources, and criteria (with default). This compensates fully for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool judges which of two candidate answers is better, using specific verbs ('judge', 'compare') and a defined resource (candidate answers to query). It distinguishes from sibling tools like groundcheck_detect_hallucinations by focusing on A/B comparison rather than detection of single issues.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly describes when to use this tool: 'Use this to A/B two RAG configurations (prompts, retrievers, models) on the same query.' It also explains position bias mitigation. However, it does not explicitly state when not to use it or mention alternatives, though sibling tools are available for other tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Find only the unsupported or contradicted claims in answer.
Use this in a fix-it loop where you only care about what's wrong, not a
full faithfulness report -- for the full picture with a score and every
claim's verdict, use groundcheck_evaluate_faithfulness instead.
Args:
answer: the RAG-generated answer text to check.
sources: list of {id, text} chunks the answer was generated from.
Returns an empty list if the answer is clean. Otherwise, each entry has
the exact answer span, the closest source passage that fails to support
it, and a one-line reason. Costs 2 model calls via your client's
sampling -- no API key needed if your client supports sampling.
| Name | Required | Description | Default |
|---|---|---|---|
| answer | Yes | ||
| sources | Yes |
| Name | Required | Description |
|---|---|---|
| hallucinations | Yes | Empty list means the answer is clean. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=true and destructiveHint=false, but the description goes beyond by explaining return behavior (empty list if clean), entry structure, and cost (2 model calls, no API key needed if client supports sampling). No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: brief purpose, usage distinction, parameter explanations, and return value description. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, the description sufficiently explains the return format (empty list or entries with span, closest source, reason) and usage context. Complete for the intended purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% description coverage, but the description's Args section provides clear definitions for both parameters: answer as 'RAG-generated answer text' and sources as 'list of {id, text} chunks'. This compensates for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool finds unsupported or contradicted claims in an answer. It distinguishes itself from the sibling tool groundcheck_evaluate_faithfulness by specifying this tool is for a fix-it loop focusing only on wrong claims.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool ('fix-it loop where you only care about what's wrong') and when to use the alternative ('for the full picture with a score and every claim's verdict, use groundcheck_evaluate_faithfulness').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Score how well answer is supported by sources, claim by claim.
Use this when you need a faithfulness score (0-1) and want to see every
claim's verdict, not just the problems -- for that, use
groundcheck_detect_hallucinations instead, which is tighter and cheaper.
Args:
answer: the RAG-generated answer text to check.
sources: list of {id, text} chunks the answer was generated from,
e.g. [{"id": "doc1#chunk3", "text": "..."}].
response_format: "concise" (default) returns the score, claim counts,
and only unsupported/contradicted claims. "detailed" returns
every claim's verdict.
Returns a score (supported/total claims), counts by verdict, and a claims
list (filtered per response_format). Costs 2 model calls via your client's
sampling (decompose, then verify) -- no API key needed if your client
supports sampling.
| Name | Required | Description | Default |
|---|---|---|---|
| answer | Yes | ||
| sources | Yes | ||
| response_format | No | concise |
| Name | Required | Description |
|---|---|---|
| score | Yes | supported claims / total claims, 0-1. |
| claims | Yes | Concise: only problem claims. Detailed: every claim. |
| supported | Yes | |
| unsupported | Yes | |
| contradicted | Yes | |
| total_claims | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=false, destructiveHint=false. The description adds valuable context: costs 2 model calls via sampling, no API key needed if client supports sampling, and explains response format behavior (concise vs detailed). It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear purpose sentence, usage guidance, then args list. It is slightly long but every part adds value. Front-loading the key purpose and sibling differentiation is effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, an output schema is referenced (not shown), and description covers return structure (score, counts, claims) and cost. It lacks details like max sources limit or async behavior, but overall is quite complete for a metadata evaluation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description carries full burden. It explains 'answer' as RAG-generated answer text, 'sources' as list of {id, text} chunks, and 'response_format' with values 'concise' (default) and 'detailed' plus their return behavior. This adds significant meaning beyond bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The tool description explicitly states it scores faithfulness of answer vs sources claim-by-claim. It clearly distinguishes from the sibling tool groundcheck_detect_hallucinations by noting it shows every claim's verdict instead of just problems.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use (need overall faithfulness score with all verdicts) and when-not-to-use (use groundcheck_detect_hallucinations for tighter/cheaper problem-only detection). This is excellent guidance with named alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Score retrieval quality for retrieved chunks against query.
Two modes, chosen automatically:
- Mode A (pass `relevant_ids`): precision@k, recall@k, MRR, NDCG computed
by pure math from your gold relevance labels. Instant, no model calls.
- Mode B (omit `relevant_ids`): no gold labels available, so each chunk
is graded 0-3 for relevance via one sampling call, then the same
metrics are computed from those grades. Use this when you don't have
a labeled relevant-docs set for this query.
Args:
query: the search query the chunks were retrieved for.
retrieved: ranked list of {id, text} chunks, in retrieval order (rank matters).
relevant_ids: ids of chunks known to be relevant. Supply this whenever
you have gold labels -- Mode A is free and exact.
k_values: cutoffs to compute metrics at (default [3, 5, 10]).
Output states which mode ran. Mode A: instant. Mode B: 1 model call, no
API key needed if your client supports sampling.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| k_values | No | ||
| retrieved | Yes | ||
| relevant_ids | No |
| Name | Required | Description |
|---|---|---|
| mrr | Yes | |
| mode | Yes | Which mode actually ran: gold_labels or llm_graded. |
| ndcg | Yes | NDCG@k for each requested k. |
| metrics | Yes | Precision/recall@k for each requested k. |
| graded_relevance | No | Per-chunk LLM grades, only present in llm_graded mode. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as read-only and non-destructive. The description adds valuable context: Mode A is instant with no model calls, Mode B uses one sampling call without requiring an API key. This helps the agent understand performance and dependency implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is approximately 150 words, well-organized into purpose, modes, and argument list. Every sentence serves a distinct purpose—no redundancy or fluff. Front-loading the core action and mode selection makes it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (two modes, multiple parameters, output schema exists), the description covers behavioral aspects, parameter semantics, and output indication ('states which mode ran'). It does not detail the output metrics, but that is delegated to the output schema. Minor gap: it could explicitly mention the metric names (precision, recall, MRR, NDCG) are computed, but they are listed earlier in the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite low schema description coverage (0%), the description fully explains all four parameters (query, retrieved, relevant_ids, k_values) with their roles and constraints. It clarifies the structure of 'retrieved' as a list of {id, text} objects and emphasizes that rank matters. This compensates entirely for the sparse schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear action ('Score retrieval quality') and specifies the resources ('retrieved' chunks, 'query'). It distinguishes itself from sibling tools (e.g., detect_hallucinations, evaluate_faithfulness) by focusing solely on retrieval quality. The two-mode breakdown further clarifies the tool's scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly guides when to use each mode: Mode A when relevant_ids are available ('free and exact'), Mode B when not. It does not directly contrast with sibling tools, but the purpose is distinct enough that an agent can infer usage. The guidance is clear and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch a previously persisted evaluation report by id.
Args:
report_id: the id returned by groundcheck_run_suite.
response_format: "concise" returns just the summary (scores,
aggregates, worst cases). "detailed" returns every case's full
faithfulness and retrieval results.
Raises an error listing available report ids if `report_id` is unknown.
No model calls -- reads from the local report store.
| Name | Required | Description | Default |
|---|---|---|---|
| report_id | Yes | ||
| response_format | No | concise |
| Name | Required | Description |
|---|---|---|
| cases | Yes | |
| summary | Yes | |
| report_id | No | |
| created_at | No | |
| prompt_version | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds valuable context: it will raise an error listing available report IDs if an unknown ID is provided, and it clarifies that it reads from a local store without making API calls. This goes beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is composed of four concise sentences, front-loaded with the main purpose. Every sentence adds valuable information with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (fetching a report by ID), the description covers all necessary aspects: purpose, parameter usage, error behavior, and performance characteristics (no model calls). The presence of an output schema further complements the documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description fully explains both parameters. For `report_id`, it specifies it is the id returned by `groundcheck_run_suite`. For `response_format`, it details the two enum values ('concise' for summary, 'detailed' for full results). Since the schema description coverage is 0%, the description carries the full burden and does so excellently.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Fetch a previously persisted evaluation report by id.' It clearly identifies the verb (fetch) and the resource (evaluation report). The distinction from sibling tools like `groundcheck_run_suite` (which runs suites) and `groundcheck_evaluate_*` (which perform evaluations) is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage after running a suite by referencing `groundcheck_run_suite` for the report_id origin. It also mentions that no model calls are made, which suggests it is safe to use without cost. However, it does not explicitly state when to use this tool over alternatives or provide exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run faithfulness (+ retrieval, where gold labels exist) over a batch of cases.
Use this to evaluate a whole RAG pipeline run rather than one answer at a
time. Supply exactly one of `cases` or `dataset_path`.
Args:
cases: inline list of {id, query, answer, sources, relevant_ids?}.
dataset_path: path to a JSONL file of the same case objects, one per
line. Must resolve inside the allowlisted data directory (env
GROUNDCHECK_DATA_DIR, default cwd) -- paths outside it are rejected.
k_values: retrieval cutoffs (default [3, 5, 10]).
Persists a full report and returns a summary (mean faithfulness, mean
NDCG, worst 5 cases, report_id). Fetch the full report with
groundcheck_get_report(report_id). Cost scales with case count: ~2 model
calls per case via your client's sampling.
| Name | Required | Description | Default |
|---|---|---|---|
| cases | No | ||
| k_values | No | ||
| dataset_path | No |
| Name | Required | Description |
|---|---|---|
| mean_ndcg | No | |
| report_id | Yes | |
| case_count | Yes | |
| worst_cases | Yes | Ids of the 5 lowest-faithfulness cases. |
| mean_faithfulness | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark as readOnly. Description adds cost info ('~2 model calls per case'), report persistence path, and how to retrieve the full report. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with first sentence stating purpose, then usage note, then parameter descriptions. Slightly verbose with the Args block but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, parameters, behavior, cost, and return structure. References related tool for full report. Lacks detail on output schema but given it's provided separately, it's sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description fully explains each parameter: cases as inline list format, dataset_path with directory restriction, k_values with defaults. Adds critical meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs faithfulness (+ retrieval) over a batch of cases, distinguishing it from siblings like groundcheck_evaluate_faithfulness which likely handles single cases. Explicitly says 'evaluate a whole RAG pipeline run rather than one answer at a time.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'Use this to evaluate a whole RAG pipeline run' and 'Supply exactly one of cases or dataset_path.' Doesn't explicitly mention when not to use or alternatives, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v0.1.0groundcheck_comparegroundcheck_detect_hallucinationsgroundcheck_evaluate_faithfulnessgroundcheck_evaluate_retrievalgroundcheck_get_reportgroundcheck_run_suiteThe tools are mostly distinct, covering comparison, hallucination detection, faithfulness evaluation, retrieval evaluation, batch runs, and report retrieval. However, there is overlap between groundcheck_detect_hallucinations and groundcheck_evaluate_faithfulness as both deal with unsupported claims, though descriptions differentiate by use case.
All tools follow a consistent pattern: 'groundcheck_' prefix followed by an imperative verb and noun (e.g., compare, detect_hallucinations, evaluate_faithfulness, get_report). Snake_case and verb-noun structure are uniform across the set.
Six tools is well-scoped for a server focused on RAG evaluation. Each tool has a clear role without redundancy, and the count is appropriate for the domain.
The tools cover the main evaluation needs: comparing answers, detecting hallucinations, scoring faithfulness, evaluating retrieval, batch execution, and report retrieval. Minor gaps exist, such as no dedicated tool for answer relevance beyond the compare function, but the surface is largely complete.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
Pay-per-call AI evaluation MCP server. Score LLM outputs against benchmark rubrics via Workers AI.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/offquestxo/groundcheck'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
