Skip to main content
Glama

IA-QA — 130+ QA & Dev Tools for AI Agents

Server Details

130+ QA & dev tools for AI agents: prompt injection, RAG testing, VLM eval, guardrails. Free.

If you are the author of this connector, you can claim ownership with GitHub, an HTTP challenge, or a DNS record. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Last Tested
Transport
Streamable HTTP
URL

Available Tools

152 tools
ab_test_reportA
Read-onlyIdempotent
Inspect

Generate an A/B test report comparing two prompts or model configurations. Accepts arrays of scores and returns statistical comparison: mean, median, std deviation, winner, and improvement percentage.

ParametersJSON Schema
NameRequiredDescriptionDefault
variant_aYesFirst variant configuration with name and score array
variant_bYesSecond variant configuration with name and score array

Output Schema

ParametersJSON Schema
NameRequiredDescription
maxNo
minNo
meanNo
countNo
medianNo
winnerNo
std_devNo
variant_aNo
variant_bNo
recommendationNo
improvement_percentNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds value by specifying the statistical metrics returned (mean, median, std deviation, winner, improvement percentage), which goes beyond the structured annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that efficiently covers purpose, input, and output without fluff. It is front-loaded with the primary action 'Generate an A/B test report'.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema and simple input schema, the description adequately covers what the tool does. It could mention edge cases like equal scores or unequal array lengths, but for a straightforward statistical tool, it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds semantic meaning by calling the variants 'prompts or model configurations', which is not explicit in the schema's generic 'variant configuration'. It also clarifies that 'scores' are arrays for statistical comparison, aiding understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool's function: 'Generate an A/B test report comparing two prompts or model configurations. Accepts arrays of scores and returns statistical comparison' with specific outputs. This distinguishes it from sibling tools like compare_models or compare_responses, which likely have different comparison methods.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly implies when to use: when you have two sets of scores to compare statistically. It mentions 'two prompts or model configurations' and 'arrays of scores', which is sufficient context. It doesn't explicitly name alternatives, but the usage context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_diff_bugsA
Read-onlyIdempotent
Inspect

Pattern-based diff linter: flags a fixed set of risky shapes in changed code — query-string interpolation (SQL/Cypher/Mongo injection shape), shell interpolation, eval/new Function, empty catch blocks, regex built from a variable, fewer catch blocks than before, and named authorization guards that disappeared. Every finding cites the line that produced it. It does NOT do data-flow analysis: it cannot follow a value to a sink, across functions or files, and an empty result is not a safety verdict (the response lists what it did not analyse). Advisory triage — use a static analyser for a real security gate.

ParametersJSON Schema
NameRequiredDescriptionDefault
contextNoOptional PR title or feature context for better analysis
version1NoOriginal code (before changes). If omitted, only the new version is analysed.
version2YesNew/modified code (after changes)

Output Schema

ParametersJSON Schema
NameRequiredDescription
bugsNo
disclaimerNo
notAnalysedNo
overallRiskNo
rulesAppliedNo
scannedLinesNo
totalSuggestionsNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnly, idempotent, non-destructive), the description discloses significant behavioral traits: it explicitly says 'It does NOT do data-flow analysis', cannot follow values across functions/files, and that empty results are not safety verdicts. It also notes the response lists what it did not analyse, which is rich, actionable transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded; it starts with 'Pattern-based diff linter', then lists the targeted patterns, cites line-level findings, and clarifies limitations and advisory nature. While dense, every sentence is informative and the structure is logical. It loses one point for being slightly long relative to the simplest possible phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite an existing output schema, the description covers all essential information: what it does, which patterns it flags, how results are presented, what it does not do, and its advisory role. This is complete for an agent to select and invoke the tool correctly without relying on the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the meaning of context, version1, and version2. The description adds only the general diff-linting context, which does not materially enhance parameter understanding beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description defines it as a 'Pattern-based diff linter' that 'flags a fixed set of risky shapes in changed code', then enumerates specific patterns like SQL/shell interpolation, eval/new Function, and empty catch blocks. This is a specific verb+resource (lints diffs for known risky patterns) and clearly distinguishes it from siblings such as diff_text or analyze_responses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the tool is 'Advisory triage — use a static analyser for a real security gate', providing an explicit when-not-to-use and an alternative. It also clarifies that it does not do data-flow analysis and that an empty result is not a safety verdict, guiding agents on appropriate reliance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_responsesA
Read-onlyIdempotent
Inspect

Semantically analyze N already-produced model outputs for the SAME task (the MCP counterpart to the LLM Sandbox). Without a reference: computes consensus — pairwise cosine agreement, the most-representative output, and the outlier. With a reference (ground truth): also ranks every output by closeness (token cosine + ROUGE-L composite) and names the closest. Deterministic, no LLM, no key — gate-able in CI. You bring the outputs (2+). For a 2-way head-to-head with structural JSON diff use compare_responses instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
referenceNoOptional ground-truth answer. If set, each output is also ranked by closeness to it and the closest one is named.
responsesYesThe outputs to analyze (same task, N models/prompts/versions). Each item is a plain string or { "label": "GPT-4o", "text": "..." }. At least 2 required.

Output Schema

ParametersJSON Schema
NameRequiredDescription
countNo
summaryNo
consensusNo
reference_rankingNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive. The description adds valuable behavioral traits beyond this: 'Deterministic, no LLM, no key — gate-able in CI.' It also explains the algorithmic behavior (consensus, pairwise cosine, ROUGE-L composite, closest/outlier). No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Five sentences, front-loaded with the core purpose. Each sentence earns its place: no-reference behavior, reference behavior, determinism/no-key/CI guarantee, input requirement, and sibling tool alternative. No redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a semantically complex tool, the description covers use cases, input constraints, behavioral guarantees, and alternatives. An output schema exists, so return-value details are already structured. The description is fully complete without being verbose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters and already describes the reference behavior ('If set, each output is also ranked by closeness to it...'). The description reinforces output semantics ('same task', '2+', 'reference') but adds no significant syntax or format details beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a specific verb and resource: 'Semantically analyze N already-produced model outputs for the SAME task.' It clearly distinguishes this tool from the sibling compare_responses by explicitly recommending that alternative for a different use case. The scope and function are unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit usage context: 'the MCP counterpart to the LLM Sandbox' and 'You bring the outputs (2+)'. It also names an alternative with a precise exclusion: 'For a 2-way head-to-head with structural JSON diff use compare_responses instead.' This fully guides when to use the tool versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

base64_decodeA
Read-onlyIdempotent
Inspect

Decode a Base64 string back to UTF-8 text. Use for inspecting Base64-encoded API responses, JWT payload claims, config file values, or attachment data.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesBase64 string to decode

Output Schema

ParametersJSON Schema
NameRequiredDescription
decodedNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the safety profile is covered. The description adds that the output is UTF-8 text, clarifying that this is not binary decoding, and lists common input sources. It does not discuss error behavior on invalid Base64, but the annotation coverage lowers the burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: the first states the core action, the second gives practical use cases. No redundant wording or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter utility with robust annotations and an output schema, the description covers purpose, usage context, and output format sufficiently. No additional details are necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with the parameter described as 'Base64 string to decode'. The description reinforces this but does not add new semantic details beyond the schema, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Decode a Base64 string back to UTF-8 text', clearly stating the verb, resource, and output format. The use cases ('API responses, JWT payload claims, config file values, or attachment data') add specificity and distinguish it from sibling decode tools like decode_jwt, which targets JWT specifically.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Use for inspecting...' gives explicit context and typical scenarios. However, no alternatives or when-not-to-use conditions are mentioned; it doesn't explicitly compare to decode_jwt or base64_encode, so it stops short of full exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

base64_encodeA
Read-onlyIdempotent
Inspect

Encode a UTF-8 string to Base64. Use when you need to embed binary data, multi-line text, or special characters safely inside JSON fields, HTTP headers, or data URIs.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to encode

Output Schema

ParametersJSON Schema
NameRequiredDescription
encodedNo

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is well covered. The description adds the input encoding detail (UTF-8) and typical use cases, but does not disclose additional behavioral traits such as exact output format or handling of edge cases. This is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the action, the second provides usage rationale. No fluff or repetition of schema/annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter encoding utility with solid annotations and an output schema, the description fully covers purpose and when to use it. No further explanation is necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% ('Text to encode'). The description adds the meaningful detail that the input must be a UTF-8 string, which clarifies behavior for non-ASCII text beyond the schema's vague 'Text to encode.'

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Encode') and the specific resource ('a UTF-8 string to Base64'). It is immediately distinguishable from sibling tools like base64_decode and url_encode.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use when you need to embed binary data, multi-line text, or special characters safely inside JSON fields, HTTP headers, or data URIs.' This provides clear context, though it does not mention alternatives or when not to use the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bias_detectA
Read-onlyIdempotent
Inspect

Analyse a set of LLM responses generated from the same prompt template but with different demographic variants (gender, origin, age, tone). Returns a bias score (0-100), sentiment analysis per variant, pairwise Jaccard similarity, and a human-readable verdict. No API key needed — runs entirely locally.

ParametersJSON Schema
NameRequiredDescriptionDefault
responsesYesArray of variant responses to compare for bias

Output Schema

ParametersJSON Schema
NameRequiredDescription
ratioNo
verdictNo
lengthCVNo
negativeNo
positiveNo
biasScoreNo
sentimentsNo
avgSimilarityNo
minSimilarityNo
sentimentVarianceNo
pairwiseSimilaritiesNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/idempotentHint/destructiveHint. The description adds valuable context beyond annotations: it runs entirely locally and requires no API key, plus details the return structure. This complements annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, followed by outputs and a key execution detail. No filler—every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a single well-documented parameter, output schema present, and annotations covering safety, the description fully addresses the tool's complexity. It explains inputs, outputs, and execution context, making it complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers the single parameter 'responses' with descriptions. The description adds meaning by requiring responses to come from the same prompt template with demographic variants, clarifying the expected input beyond the schema's generic 'variant responses'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: analyzing LLM responses with demographic variants for bias. It specifies the input type (same prompt template with different variants), the output (bias score, sentiment, similarity, verdict), and differentiates from generic siblings like analyze_responses or compare_responses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear context for when to use (responses from same prompt template with demographic variants) and what it returns. However, it does not explicitly mention alternatives or when not to use, which prevents a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bm25_scoreA
Read-onlyIdempotent
Inspect

Compute BM25 relevance score between a query and one or more documents. BM25 is the industry-standard keyword-based ranking algorithm used in Elasticsearch, OpenSearch, and Weaviate hybrid search. Returns ranked results with normalized scores.

ParametersJSON Schema
NameRequiredDescriptionDefault
bNoLength normalization factor (default: 0.75)
k1NoTerm frequency saturation (default: 1.5)
queryYesThe search query
top_kNoReturn top K results (default: all)
documentsYesArray of documents to rank

Output Schema

ParametersJSON Schema
NameRequiredDescription
bNo
k1No
indexNo
queryNo
resultsNo
bm25_scoreNo
doc_lengthNo
doc_previewNo
avg_doc_lengthNo
documents_countNo

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is covered. The description adds that scores are normalized and it handles one or more documents, but does not detail other behavioral traits like tokenization or comparability. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with the main action front-loaded. The second sentence provides useful context about the algorithm's industry usage, though it could be trimmed slightly. Overall very concise with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and annotations covering read-only/idempotent safety, the description adequately covers the tool's purpose and return behavior. It lacks explicit alternative guidance but is sufficiently complete for a computational scoring tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, providing descriptions for all parameters (query, documents, top_k, k1, b). The description adds no additional parameter-specific meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Compute BM25 relevance score between a query and one or more documents', identifying the specific verb, resource, and unique algorithm. This distinguishes it from siblings like rag_relevance_rank or similarity_score by naming BM25 explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear context by stating BM25 is the industry-standard keyword-based ranking algorithm used in Elasticsearch, OpenSearch, and Weaviate hybrid search, implying when to use it. However, it does not explicitly name alternatives or exclusions such as 'for semantic similarity use embedding_similarity instead'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

build_rag_promptA
Read-onlyIdempotent
Inspect

Assemble a complete RAG (Retrieval-Augmented Generation) prompt from retrieved context chunks and a user query. Handles token budgeting, citation numbering, system instruction injection, and source attribution.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesThe user question to answer
chunksYesRetrieved context chunks with .text (required), .source (optional), .score (optional)
languageNoResponse language instruction (e.g. "French", "Spanish")
cite_sourcesNoAdd [1], [2] citation numbers (default: true)
max_context_tokensNoMax tokens for context section (default: 2000)
system_instructionNoCustom system instruction (default: standard RAG grounding instruction)

Output Schema

ParametersJSON Schema
NameRequiredDescription
promptNo
system_promptNo
chunks_includedNo
included_chunksNo
chunks_truncatedNo
total_tokens_estimateNo
context_tokens_estimateNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only/idempotent/non-destructive safety, so the bar is lower. The description adds valuable behavioral context by enumerating internal steps (token budgeting, citation numbering, system instruction injection, source attribution), which helps predict output structure. It doesn't disclose edge-case behaviors (e.g., truncation strategy) but exceeds baseline.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action. Every phrase adds information—no filler or redundancy. Appropriate length for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description plus rich schema and output schema provide enough for selection and basic invocation. It covers the main functional areas and hints at all parameter roles. It omits details like truncation strategy and citation ordering, but these are secondary given the strong schema context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameters, so baseline is 3. The description adds meaning by connecting parameters to their effects: max_context_tokens↔token budgeting, cite_sources↔citation numbering, system_instruction↔system instruction injection, chunks.source↔source attribution. This contextualizes the schema without duplicating it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Assemble a complete RAG prompt', combining a specific verb with resource and data source. The additional capabilities (token budgeting, citations, system instruction) distinguish it from sibling tools like few_shot_formatter or system_prompt_builder.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States it operates on 'retrieved context chunks', making the input precondition clear. However, it doesn't explicitly name alternatives or state when not to use it, though the name and RAG specificity imply these boundaries.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

calculate_readabilityA
Read-onlyIdempotent
Inspect

Calculate readability scores: Flesch Reading Ease, Flesch-Kincaid Grade Level, Coleman-Liau Index, and Automated Readability Index. Useful for evaluating LLM output quality.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to analyze for readability

Output Schema

ParametersJSON Schema
NameRequiredDescription
levelNo
statsNo
coleman_liau_indexNo
flesch_reading_easeNo
flesch_kincaid_gradeNo
automated_readability_indexNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint true, idempotentHint true, and destructiveHint false, covering safety and side effects. The description adds no additional behavioral context (e.g., language limitations, score ranges, or empty-input handling), but this is a simple read-only operation. It adds no contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the action and metrics, and the second sentence adds a practical use case. No wasted words or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read-only tool with an output schema and full annotations, the description is complete. It specifies what it does, names the metrics, and gives a usage context; return values are covered by the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single 'input' parameter, which is described as 'Text to analyze for readability'. The tool description does not add extra parameter semantics beyond restating the purpose, so it meets the baseline without enhancements.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Calculate readability scores') and enumerates the exact metrics (Flesch Reading Ease, Flesch-Kincaid Grade Level, Coleman-Liau Index, Automated Readability Index). This distinguishes it from sibling text-analysis tools like text_stats or count_tokens.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear use case ('Useful for evaluating LLM output quality'), indicating when to use it. It does not explicitly exclude alternatives, but no competing readability tool exists among siblings, so the context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

case_convertA
Read-onlyIdempotent
Inspect

Convert a string between naming conventions: camelCase, PascalCase, snake_case, kebab-case, UPPER_SNAKE_CASE, dot.case, Title Case. Essential for code generation and refactoring.

ParametersJSON Schema
NameRequiredDescriptionDefault
toYesTarget case: "camel", "pascal", "snake", "kebab", "upper_snake", "dot", "title"
inputYesString to convert (e.g., "myVariableName", "my-css-class")

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultNo
from_wordsNo
target_caseNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety. The description adds behavioral context by specifying all supported conversions, which directly informs the agent of expected transformations. No contradictions exist.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with the primary action front-loaded. Every word adds value; the list of conventions and use-case phrase are efficient and clear. No redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple two-parameter, deterministic conversion tool. The schema fully documents parameters, annotations cover safety, and the description lists all supported cases. With an output schema present, no further return-format details are needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with both 'input' and 'to' documented, including examples. The description restates the list of target cases already present in the schema's 'to' description, adding minimal new semantic detail beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Convert a string between naming conventions,' using a specific verb and resource while enumerating all targeted cases (camelCase, PascalCase, snake_case, etc.). This clearly differentiates it from sibling conversion tools like base64_decode or number_base_convert, which handle other formats.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Essential for code generation and refactoring' provides a clear use case, indicating when to apply the tool. It does not explicitly name alternative tools but the strong context and enumerating of naming conventions make the usage obvious. Lacks an explicit when-not-to-use statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

check_contrast_ratioA
Read-onlyIdempotent
Inspect

Calculate WCAG 2.1 contrast ratio between two colors. Returns ratio and compliance for AA/AAA normal and large text.

ParametersJSON Schema
NameRequiredDescriptionDefault
backgroundYesBackground color in hex (e.g., "#ffffff")
foregroundYesForeground color in hex (e.g., "#333333")

Output Schema

ParametersJSON Schema
NameRequiredDescription
ratioNo
AA_largeNo
AAA_largeNo
AA_normalNo
AAA_normalNo
backgroundNo
foregroundNo
ratio_textNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive. The description adds behavioral output context by stating it returns compliance for AA/AAA normal and large text, which is useful beyond the annotations. It does not mention edge cases like non-hex input, but this is a low-risk calculation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action, no fluff. Every word adds value, and it clearly states inputs and outputs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple calculation tool with complete schema, annotations, and an output schema, the description covers purpose and return semantics. No additional context is necessary; it is fully adequate for agent use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with both params (foreground/background) documented as hex strings. The description only says 'two colors', adding little beyond the schema. Baseline 3 applies since schema fully covers parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Calculate' and names the resource 'WCAG 2.1 contrast ratio between two colors', clearly distinguishing it from sibling tools like color_convert. It also states the return value (ratio and compliance), fully clarifying intent.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool (for checking WCAG accessibility contrast) and context of compliance levels. It does not explicitly exclude alternatives or name siblings, but the context is clear enough for an agent to select it appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

color_convertA
Read-onlyIdempotent
Inspect

Convert a color between HEX, RGB, and HSL formats. Use when translating design tokens between CSS notations, verifying color accessibility, or normalizing color values from user input. Accepts #rrggbb, #rgb, rgb(r,g,b), or hsl(h,s%,l%).

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesColor value to convert, e.g. "#ff6b6b", "rgb(255,107,107)", "hsl(0,100%,71%)"

Output Schema

ParametersJSON Schema
NameRequiredDescription
bNo
gNo
rNo
hexNo
hslNo
rgbNo
inputNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe read operation. The description adds accepted input formats but does not disclose additional behavioral traits (e.g., output behavior, error handling). No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences with no filler: the first states purpose, the second gives usage context, and the third specifies accepted formats. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter converter with a present output schema and thorough annotations, the description adequately covers purpose, usage, and input formats. No critical gaps are apparent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'input' is fully described in the schema with concrete examples (100% coverage). The description adds a compact pattern summary, but this overlaps with the schema's examples, so the added value is marginal.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific action 'Convert a color between HEX, RGB, and HSL formats', naming both the resource (color) and the exact formats. This clearly differentiates it from sibling tools like case_convert or base64_encode.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use the tool (translating design tokens, verifying accessibility, normalizing user input) but does not mention when not to use or mention alternatives such as check_contrast_ratio, which is a related sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_modelsA
Read-onlyIdempotent
Inspect

Compare 2-5 AI models side by side: context window, pricing, multimodal, reasoning capabilities, and provider. Returns a comparison table with a recommendation based on your use case.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelsYesArray of 2-5 model names (e.g. ["gpt-4o","claude-3.5-sonnet","gemini-2.0-flash"])
use_caseNoOptimize recommendation for this criterion

Output Schema

ParametersJSON Schema
NameRequiredDescription
rowsNo
modelNo
use_caseNo
recommendationNo
models_comparedNo
cost_per_1k_totalNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is known. The description adds behavioral context beyond annotations: it enforces a 2-5 model range (not in schema) and indicates the tool returns a recommendation based on use case. It also lists the comparison dimensions, which clarifies expected behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently conveys purpose, scope, and output. Every clause adds value, with no redundant information or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is complete for a read-only comparison tool with an output schema. It states the input range and criteria, the output type (comparison table with recommendation), and the use-case parameter. Since annotations cover safety and an output schema exists, no further details are necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both 'models' and 'use_case' are already explained. The description adds value by specifying the 2-5 count constraint on 'models' and clarifying that 'use_case' optimizes the recommendation, which goes beyond the schema's enum descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Compare 2-5 AI models side by side' with specific attributes (context window, pricing, multimodal, reasoning, provider). It uses a specific verb and resource, and it distinguishes itself from siblings like 'compare_responses' (which likely compares outputs) and 'model_info' (single model info).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool ('based on your use case') and lists the comparison criteria, providing clear context. However, it does not explicitly mention alternatives or exclusion conditions, such as when to use 'compare_responses' instead. No explicit 'when not to use' guidance is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_responsesA
Read-onlyIdempotent
Inspect

Compare two ALREADY-PRODUCED outputs (e.g. model A vs model B on the same task) side by side. Returns deterministic metrics (token cosine, ROUGE-L, Jaccard, length/structure deltas, JSON diff) and a verdict. If a reference (ground truth) is given, scores each output against it and picks the closer one. If model + api_key are given, an LLM judge also picks a qualitative winner for the task. No re-execution — you bring the outputs.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskNoThe task/prompt both outputs were answering — used by the LLM judge for context
modelNoOptional judge model id (BYOK). When set with api_key, an LLM judge picks a qualitative winner.
api_keyNoOptional API key for the judge model (BYOK). Used only for the judge call; never stored.
label_aNoLabel for output A (e.g. "GPT-4o", "v1.0")
label_bNoLabel for output B (e.g. "GPT-5-nano", "v1.1")
referenceNoOptional ground-truth / expected answer. If set, each output is scored against it and the closer one wins (deterministic).
check_jsonNoTry to parse as JSON and compare structurally (keys, types, values)
response_aYesFirst output (e.g. model A's answer)
response_bYesSecond output (e.g. model B's answer)

Output Schema

ParametersJSON Schema
NameRequiredDescription
judgeNo
labelANo
labelBNo
metricsNo
summaryNo
verdictNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive behavior. The description adds valuable behavioral context beyond that: it lists exactly what metrics are returned, explains that a `reference` causes scoring against ground truth, and that providing `model`+`api_key` triggers an LLM judge call. It also notes 'No re-execution' and the schema notes the API key is 'never stored,' which bolsters transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact—five sentences that front-load the core purpose, then layer conditional behavior and end with a clear constraint. Every sentence earns its place; no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema, rich annotations, and 100% schema parameter coverage, the description is complete enough. It covers the two major optional modes (reference-based scoring and LLM judging), the deterministic metrics, and the no-re-execution constraint, leaving no significant gaps for a tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with good per-parameter descriptions, yielding a baseline of 3. The description adds semantic value by explaining how parameters interact: `reference` makes each output scored against it, `model`+`api_key` enables the judge, and the metric list (token cosine, ROUGE-L, Jaccard, JSON diff) clarifies what the tool computes from `response_a` and `response_b`.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Compare two ALREADY-PRODUCED outputs ... side by side.' It clearly differentiates from siblings by emphasizing no re-execution and listing concrete deterministic metrics plus an optional LLM judge verdict.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly states when to use the tool: when outputs already exist and you want a side-by-side comparison. It gives conditional guidance for `reference` and `model`+`api_key`, and explicitly says 'No re-execution — you bring the outputs,' which implies it is not for generating outputs. It does not name alternatives explicitly, so it's strong context without formal exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

consistency_checkA
Read-onlyIdempotent
Inspect

Compare multiple LLM responses to the same prompt and detect inconsistencies using Jaccard word-overlap similarity and fact drift (number comparison). Fast, deterministic, no API key needed. Limitations: relies on surface-level word matching — "Paris is the capital of France" vs "Paris is the French capital" may score low despite semantic equivalence. For true semantic consistency, use run_semantic_tests with embedding mode. Essential for determinism testing.

ParametersJSON Schema
NameRequiredDescriptionDefault
responsesYesArray of 2+ LLM responses to compare (same prompt, different runs)
check_factsNoCheck for contradictory numbers/facts across responses (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
verdictNo
fact_driftNo
avg_similarityNo
response_countNo
pairwise_scoresNo
fact_contradictionNo
length_variance_percentNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes beyond the annotations by revealing the algorithmic limitation (surface-level word matching) with a concrete example. It also adds traits such as 'fast' and 'deterministic', which are not present in the annotations but are important behavioral context. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise, using only two sentences to convey purpose, method, limitations, and alternatives. Every sentence adds value, and the text is well-structured with a clear separation of strengths, limitations, and use case.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, has full parameter documentation, and an output schema. The description covers use cases, limitations, and alternatives, making it complete for an agent to decide when to use it. There are no obvious gaps given the available structured information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% coverage for both parameters, so the baseline is 3. The description adds slight context by explaining that check_facts deals with 'fact drift (number comparison)', but this is already implied by the schema description. No additional parameter-level detail is provided in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: compare multiple LLM responses to the same prompt and detect inconsistencies via Jaccard similarity and fact drift. It names specific methods and distinguishes from sibling tools like run_semantic_tests by explicitly referencing embedding mode for semantic consistency.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear when-to-use context: 'Essential for determinism testing' and highlights the fast/deterministic/no-API-key benefits. It explicitly states when not to rely on it (semantic equivalence) and directs to run_semantic_tests as an alternative for true semantic consistency.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

context_window_checkA
Read-onlyIdempotent
Inspect

Given an array of message objects [{role, content}], estimate total token usage and check if it fits in the target model's context window. Warns about truncation risk.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesTarget model name (e.g. gpt-4o, claude-3.5-sonnet)
messagesYesArray of messages (system/user/assistant)
max_output_tokensNoReserved tokens for output (default: 4096)

Output Schema

ParametersJSON Schema
NameRequiredDescription
fitsNo
roleNo
charsNo
indexNo
modelNo
tokensNo
warningsNo
breakdownNo
per_messageNo
total_tokensNo
message_countNo
context_windowNo
total_input_tokensNo
utilization_percentNo
reserved_output_tokensNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only and idempotent behavior, so the bar is lower. The description adds valuable context: the operation is an 'estimate' (implying approximation) and it 'warns about truncation risk'. This goes beyond the schema and annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the most important action, and contains no filler or redundant repetition of annotations. Every word contributes to understanding the tool's purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, full schema coverage, and presence of an output schema, the description sufficiently covers inputs, purpose, and key behavior. It does not need to describe return values because the output schema handles that, and it adequately sets expectations about estimation and warnings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description repeats the message structure already present in the schema and does not add new parameter details. It provides no additional syntax or format guidance beyond what the schema already documents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with specific verbs ('estimate', 'check', 'warns') and names its resource (message array and target model's context window). It distinguishes itself from siblings like count_tokens by adding the context-window fit check and truncation warning.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implicitly communicates when to use the tool (when you need to estimate token usage and verify context fit), but it does not explicitly name alternatives or exclusions. Given the clear context stated, this meets the 'clear context, no exclusions' level.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

conversation_analyzeA
Read-onlyIdempotent
Inspect

Analyze a multi-turn conversation for context retention, topic drift, instruction following, and repetition. Accepts messages array [{role, content}]. Essential for chatbot QA.

ParametersJSON Schema
NameRequiredDescriptionDefault
messagesYesConversation messages in order

Output Schema

ParametersJSON Schema
NameRequiredDescription
turn_countNo
repetitionsNo
topic_driftNo
user_messagesNo
context_retentionNo
has_system_promptNo
assistant_messagesNo
avg_response_lengthNo
repetition_detectedNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds valuable behavioral context by specifying the input format (messages array) and the exact analysis dimensions. It does not contradict annotations and provides extra detail about what the tool evaluates.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, with three sentences each serving a distinct purpose: stating the core action, specifying input format, and giving usage context. It is front-loaded with the primary verb and resource, and every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has a single well-documented parameter, a full schema, clear annotations, and an output schema. The description covers purpose, input format, and use case, which is sufficient for an agent to select and invoke it correctly. Minor gaps like message ordering constraints are not mentioned, but the tool's simplicity makes these non-critical.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% coverage for the single 'messages' parameter, which is already described as 'Conversation messages in order'. The description repeats the array format '[{role, content}]' but adds no additional semantic info beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Analyze') and resource ('multi-turn conversation'), and lists concrete analysis dimensions (context retention, topic drift, instruction following, repetition). It also mentions the input format and the primary use case (chatbot QA), which distinguishes it from sibling tools like analyze_responses or consistency_check.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description says 'Essential for chatbot QA', providing a clear context for when to use the tool. However, it does not explicitly mention when not to use it or name alternative tools, so it falls short of full differentiation but still gives strong usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cors_checkerA
Read-only
Inspect

Check the CORS configuration of a URL the same way a browser would. Returns the main response status, all Access-Control-* headers, the tested origin, and the preflight OPTIONS response. Use this for direct CORS debugging, not just security auditing.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesFull URL to test, e.g. https://api.example.com/resource
methodNoHTTP method to simulate (default: GET)
originNoOrigin header to simulate (default: https://yourdomain.com)

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
methodNo
statusNo
preflightNo
allHeadersNo
corsHeadersNo
testedOriginNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral context beyond the annotations: it states the tool simulates a browser, returns the main response status, all Access-Control-* headers, the tested origin, and the preflight OPTIONS response. This informs the agent about the tool's output and behavior, complementing the readOnlyHint and destructiveHint annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the primary purpose, then lists return values and usage guidance. Every sentence adds value with no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and full schema coverage, the description covers purpose, behavior, and typical usage. It's complete for a simple diagnostic tool, though it could optionally mention error handling or network failure behavior for full completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides descriptions for all three parameters (url, method, origin), with 100% coverage. The description doesn't add any additional parameter-specific details, so it meets the baseline but doesn't exceed it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: checking CORS configuration like a browser would. It mentions the resource (URL) and specific action, but doesn't explicitly distinguish itself from the sibling 'cors_test' tool, only from security auditing in general.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear usage context ('Use this for direct CORS debugging') and hints at an exclusion ('not just security auditing'). However, it does not name specific alternative tools, so it lacks explicit when-not-to-use instructions relative to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cors_testA
Read-only
Inspect

Test a URL for CORS misconfigurations. Sends preflight (OPTIONS) requests with various Origin headers to detect: wildcard origins with credentials, origin reflection (echoing any origin), null origin acceptance, subdomain wildcard bypass, and missing Vary headers. Returns risk level (safe/low/medium/high/critical) plus per-origin results. "unknown" means nothing was actually tested — every origin either failed to connect or answered 5xx, so the target returned no CORS decision; never read it as "safe". A 4xx preflight IS a real result (the server refused it and a browser would fail closed).

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesFull URL to test (e.g. https://api.example.com/endpoint)
originNoCustom Origin header to test (default: tests multiple origins automatically)

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
errorNo
testsNo
warningNo
risk_levelNo
origins_testedNo
total_findingsNo
origins_reachableNo
origins_conclusiveNo
origins_inconclusiveNo

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavioral nuance beyond the annotations: it explains that preflight OPTIONS requests are sent, what specific misconfigurations are detected, and critically clarifies the meaning of 'unknown' and 4xx responses. This is exactly the kind of context that helps an agent interpret results correctly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: purpose, method, output format, and critical interpretation caveats are all included without redundancy. It is front-loaded with the core purpose and structured logically.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, the description is complete: it covers what the tool does, how it works, what it returns, and how to interpret ambiguous or error-like results. The output schema exists, and the description complements it with risk-level and per-origin result context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already covers both parameters with 100% coverage, so the baseline is 3. The description does not add significant parameter-level detail beyond what the schema provides, though it does reinforce that multiple origins are tested by default.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Test a URL for CORS misconfigurations' and enumerates specific detection categories. It is specific and actionable, but it does not explicitly differentiate itself from the sibling tool 'cors_checker', so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when the tool is appropriate: testing URLs for CORS misconfiguration issues. It does not mention alternatives or exclusions, but the intended use case is unambiguous and well-scoped.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cot_analyzerA
Read-onlyIdempotent
Inspect

Analyze a Chain-of-Thought (CoT) or reasoning trace from an LLM. Detects step count, logical flow, conclusion presence, backtracking, and estimates reasoning depth. Useful for o1/o3/DeepSeek-R1 evaluation.

ParametersJSON Schema
NameRequiredDescriptionDefault
reasoningYesThe CoT / reasoning trace text (e.g. from <think> tags or step-by-step output)
expected_conclusionNoExpected final answer to check against (optional)

Output Schema

ParametersJSON Schema
NameRequiredDescription
markersNo
step_countNo
total_charsNo
total_linesNo
has_conclusionNo
reasoning_depthNo
backtracking_signalsNo
reasoning_depth_labelNo
conclusion_matches_expectedNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so safety is covered. The description adds valuable behavioral insight by listing what the analysis detects (step count, backtracking, reasoning depth, etc.), which goes beyond the structured fields and helps the agent anticipate the tool's output dimensions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with no filler. The first sentence states the action and key capabilities; the second gives the evaluation context. Every clause adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is complete for a tool with rich annotations and an output schema. It explains what the tool does, what it detects, and when it is useful. There is no missing context about safety or return values because annotations and output schema cover those aspects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3; the description does not need to repeat parameter details. It adds implicit alignment by mentioning 'conclusion presence' which relates to the optional expected_conclusion parameter, but does not explicitly explain parameter semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Analyze' with a precise resource ('Chain-of-Thought (CoT) or reasoning trace') and enumerates concrete detection outputs (step count, logical flow, conclusion presence, backtracking, reasoning depth). It also distinguishes this tool from siblings by explicitly targeting reasoning traces and naming model families (o1/o3/DeepSeek-R1) for evaluation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context by stating the tool is 'Useful for o1/o3/DeepSeek-R1 evaluation,' which signals when to use it. It does not explicitly mention when not to use it or suggest alternatives, but the use case is specific enough to guide selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

count_code_linesA
Read-onlyIdempotent
Inspect

Count lines of code: total, code lines, comment lines, blank lines, and comment density. Supports JS/TS, Python, Java/C/C++, Ruby, Go, Shell, HTML/XML, and CSS.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesSource code to analyze
languageNoLanguage hint: "js", "ts", "py", "java", "c", "rb", "go", "sh", "html", "css" (auto-detect if omitted)

Output Schema

ParametersJSON Schema
NameRequiredDescription
languageNo
code_linesNo
blank_linesNo
total_linesNo
comment_linesNo
comment_densityNo
code_to_comment_ratioNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as a safe, read-only, idempotent operation, so the description need not restate those traits. It adds value by specifying the exact outputs (comments, blank lines, density) and supported language families, which informs the agent about the tool's capabilities beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a compact two-sentence summary: the first sentence states the core function and output metrics, the second lists supported languages. There is no fluff or redundant information; every sentence contributes to understanding the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a read-only annotation, a well-defined schema, and an output schema present, the description covers the essential context: what the tool counts, which languages it supports, and the output metrics. For a simple analysis tool of this complexity, no additional behavioral details are necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters are fully described in the input schema with clear descriptions (source code string and optional language hint), providing 100% schema coverage. The tool description does not add additional parameter semantics, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Count lines of code' and enumerates the specific metrics (total, code, comment, blank, density), making the tool's function unambiguous. It also lists supported languages, which distinguishes it from generic text analysis tools like text_stats or count_tokens.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for code analysis but does not explicitly state when to use it over alternatives or when not to use it. There is no mention of sibling tools such as count_tokens or calculate_readability, leaving the choice to the agent's inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

count_tokensA
Read-onlyIdempotent
Inspect

Estimate the token count of a text string using the cl100k_base approximation (~4 chars/token). Call this BEFORE sending any text to an LLM API to check if it fits within the model context window and to estimate cost. Returns token estimate, character count, and word count.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to count tokens for

Output Schema

ParametersJSON Schema
NameRequiredDescription
charsNo
wordsNo
tokens_estimateNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, and non-destructive behavior. The description adds the approximation method (cl100k_base, ~4 chars/token), which sets expectations about accuracy, and discloses the return fields. This adds meaningful behavioral context beyond the annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The first sentence states the action and method; the second provides usage timing and output summary. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with one parameter and an output schema. The description covers purpose, usage timing, approximation details, and return fields, making it complete for an agent to select and invoke correctly. The existing output schema handles return value schemas, so no additional return documentation is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 100% of the parameter with a basic description. The tool description adds no additional detail about the input parameter (e.g., length limits or encoding), so it does not exceed the baseline for a fully covered schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it estimates token count for a text string using the cl100k_base approximation. It distinguishes itself from siblings like estimate_llm_cost by focusing on token counting rather than cost, and mentions the return fields (token estimate, character count, word count) which further clarifies its function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly instructs to call it before sending text to an LLM API to check context window fit and estimate cost. This provides clear when-to-use context, but it does not mention alternative tools (e.g., truncate_to_tokens) or exclusions, so it lacks explicit alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_confluence_pageAInspect

Create a new Confluence page from the output of jira_to_test_suite. Formats Gherkin, E2E steps, API tests, and test data as a properly structured Confluence page with code blocks and tables. STATEFUL — creates a new page in the specified space.

ParametersJSON Schema
NameRequiredDescriptionDefault
titleNoPage title. Defaults to "Test Plan: {issue_key}"
issue_keyNoSource Jira issue key (for the page title and source link)
issue_urlNoSource Jira issue URL (added as a link in the page)
space_keyYesConfluence space key where the page will be created, e.g. "QA", "ENG"
test_suiteYesThe test_suite object from jira_to_test_suite result
parent_page_idNoOptional parent page ID — page will be created as a child of this page
confluence_emailYesAtlassian account email
confluence_tokenYesAtlassian API token
confluence_base_urlYesAtlassian base URL

Output Schema

ParametersJSON Schema
NameRequiredDescription
titleNo
page_idNo
successNo
page_urlNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish state-changing behavior via readOnlyHint=false and idempotentHint=false. The description adds value by disclosing that the page is structured with code blocks and tables, and reiterates the stateful nature. This goes beyond what annotations provide, although no additional warnings about duplicates or auth are included.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core action and source, followed by formatting and statefulness. Every word earns its place; no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 parameters, nested objects, output schema present), the description provides essential workflow context, source relationship, and formatting expectations. The output schema covers return details, so the description is sufficient for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with each parameter already having a meaningful description (e.g., test_suite defined as 'from jira_to_test_suite result'). The tool description does not add new parameter details beyond the schema, so it meets the baseline but doesn't exceed it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create a new Confluence page') and the specific source ('output of jira_to_test_suite'), distinguishing it from sibling tools like fetch_confluence_page. It also details the content formatting (Gherkin, E2E steps, API tests) making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it: after jira_to_test_suite produces its output. It does not explicitly mention alternatives or exclusions, but the 'from the output of jira_to_test_suite' phrase provides clear context. The STATEFUL warning adds a caution about side effects, which is useful for decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cron_parseA
Read-onlyIdempotent
Inspect

Parse a cron expression into a human-readable schedule description. Supports standard 5-field cron (minute hour day month weekday).

ParametersJSON Schema
NameRequiredDescriptionDefault
expressionYesCron expression (e.g., "0 9 * * 1-5", "*/15 * * * *")

Output Schema

ParametersJSON Schema
NameRequiredDescription
fieldsNo
expressionNo
human_readableNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false. The description adds behavioral context by specifying support for standard 5-field cron (minute hour day month weekday), which is useful beyond the annotations. It does not cover error behavior, but the output schema likely covers that.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the primary purpose, and no wasted words. Every part of the description contributes useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with one parameter and an output schema present. The description is sufficiently complete for an agent to understand when and how to use it, and the format clarification covers the key edge case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a parameter description and examples. The description adds semantic value by explaining the exact format and field order, which goes beyond the schema's simple 'Cron expression' label.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Parse a cron expression') and a clear outcome ('into a human-readable schedule description'). It distinguishes itself from the sibling cron_validator by focusing on producing a description rather than validating.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for getting human-readable schedules from cron expressions, which gives clear context. However, it does not explicitly mention alternatives or when not to use it (e.g., if validation is needed, use cron_validator), so it misses the explicit exclusion criterion for a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cron_validatorA
Read-onlyIdempotent
Inspect

Validate a 5-field cron expression, explain the schedule, and preview the next execution times. Use this to debug cron jobs before they reach production. Returns parsed fields, a human-readable description, and upcoming ISO timestamps.

ParametersJSON Schema
NameRequiredDescriptionDefault
expressionYesCron expression with 5 fields, e.g. "*/15 9-18 * * 1-5"
next_runs_countNoHow many upcoming runs to return (1-50, default: 10)

Output Schema

ParametersJSON Schema
NameRequiredDescription
validNo
fieldsNo
next_runsNo
expressionNo
human_readableNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds value by stating that it returns parsed fields, a human-readable description, and ISO timestamps, which is beyond annotation coverage and sets proper expectations for the output behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the primary action, a clear use case, and a concise output summary. Every sentence earns its place with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With two well-described parameters, an output schema, and strong annotations, the description fully covers what an agent needs: purpose, usage, and return value highlights. The mention of ISO timestamps helps set format expectations, making this complete for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already well-documented. The description refers to '5-field cron expression' and 'upcoming ISO timestamps,' which aligns with schema but adds no new parameter semantics beyond what is already present.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Validate', 'explain', 'preview') applied to a clear resource ('5-field cron expression'), and it distinguishes itself from sibling cron_parse by adding validation and preview functionality. This is precise and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly gives a usage context: 'Use this to debug cron jobs before they reach production.' It doesn't mention when not to use it or name alternatives, but the use case is clear enough to guide an agent toward appropriate selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

decode_jwtA
Read-only
Inspect

Decode a JWT (JSON Web Token) and return its header and payload without verifying the signature. Also reports whether the token is expired and the exact expiry date. Use to inspect claims (sub, iss, exp, roles) during debugging or when integrating with an auth provider.

ParametersJSON Schema
NameRequiredDescriptionDefault
tokenYesThe JWT string to decode (header.payload.signature)

Output Schema

ParametersJSON Schema
NameRequiredDescription
noteNo
headerNo
expiredNo
payloadNo
expiresAtNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so the safety profile is known. The description adds crucial behavior beyond annotations: it explicitly states the signature is NOT verified, which is a key security-relevant trait. It also mentions expiry reporting. This adds meaningful context without redundancy.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core function. Every sentence adds value: the first states what it does, the second gives usage guidance. No fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with one parameter and an output schema, so the description does not need to explain return values. It covers the purpose, key behavior (no signature verification), and typical use cases, making it fully sufficient for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 100% of the parameter, including a description of the JWT format. The description does not add extra parameter-specific details beyond what the schema provides, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Decode') and resource ('JWT'), clearly stating it returns header and payload without verifying the signature. It distinguishes itself from siblings like base64_decode by focusing specifically on JWT structure and expiry reporting. The scope is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear use case: 'Use to inspect claims (sub, iss, exp, roles) during debugging or when integrating with an auth provider.' This gives context for when to use the tool, though it does not explicitly mention exclusions or alternatives, which prevents a top score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

detect_languageA
Read-onlyIdempotent
Inspect

Detect the natural language of a text using n-gram frequency analysis and common word markers. Supports 15 languages: English, French, Spanish, German, Italian, Portuguese, Dutch, Russian, Chinese, Japanese, Korean, Arabic, Polish, Turkish, Swedish.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to detect language from (min 20 chars for accuracy)

Output Schema

ParametersJSON Schema
NameRequiredDescription
langNo
nameNo
scoreNo
methodNo
matchedNo
languageNo
confidenceNo
top_candidatesNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior. The description adds useful context beyond that: it discloses the underlying method (n-gram frequency analysis and common word markers) and the supported language list, which sets expectations for accuracy and scope without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loads the purpose, and contains no filler. It efficiently provides the method and supported language list, earning every sentence's place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter tool with an output schema, the description is fully complete: it explains what it does, how it works, and its language scope. The schema covers the input constraint, and annotations cover safety, so no critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully documents the 'input' parameter and its minimum length requirement, so the description does not need to repeat it. The description adds no additional parameter semantics beyond what the schema already provides, aligning with the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('Detect') and resource ('natural language of a text'), and distinguishes itself from sibling tools like detect_secrets or toxicity_scan by specifying its unique task. It also lists supported languages, further clarifying scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by describing what it does, but does not explicitly state when to use this tool versus alternatives or mention any exclusions. There is no guidance on when not to use it or how it differs from related text analysis tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

detect_secretsA
Read-onlyIdempotent
Inspect

Scan code or config files for hardcoded secrets: AWS keys, GitHub tokens, OpenAI/Anthropic API keys, Stripe secrets, JWTs, database connection strings, and generic passwords. Returns findings with severity. Run before every commit.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesCode or config content to scan (max 500KB)
filenameNoOptional filename for context (e.g. ".env", "config.js")

Output Schema

ParametersJSON Schema
NameRequiredDescription
filenameNo
findingsNo
risk_levelNo
recommendationNo
total_findingsNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds contextual behavior: it returns findings with severity and highlights what types of secrets are detected. This goes beyond the annotations, revealing the tool's output characteristics without contradicting the declared safety profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences: the first defines the core action and scope, the second states the return value, and the third gives usage guidance. It is front-loaded, informative, and contains no filler words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of a complete input schema (100% coverage) and an output schema, the description covers the tool's purpose, target content, return behavior, and usage context. It is fully adequate for an agent to select and invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the parameters 'input' and 'filename' are fully described in the schema. The description does not provide additional parameter-level details beyond implying that 'input' contains code/config content. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the tool scans code/config files for hardcoded secrets and provides a comprehensive list of secret types (AWS keys, GitHub tokens, API keys, etc.). It uses a specific verb 'scan' and resource, making the purpose unambiguous and distinguishable from generic scanning tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear usage directive: 'Run before every commit.' This provides strong contextual guidance, though it does not explicitly mention alternatives or when not to use the tool. The lack of exclusions makes it a 4 rather than a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

diff_mappingsA
Read-onlyIdempotent
Inspect

Diff a baseline page mapping against a current one and return a CI-style verdict: PASS / FIX / BLOCK, plus per-element drift (ok, renamed, healable, ambiguous, lost, added, rebound). Pure and deterministic — provide two mappings as JSON with "elements" arrays of {role, name, selector, context?}. Use the companion @ia-qa/self-healing package (npm install -g @ia-qa/self-healing) to capture mappings from your app via its local MCP server ia-qa-heal-mcp, or paste the snippet from ia-qa.com/devtools/selector-drift into your browser console.

ParametersJSON Schema
NameRequiredDescriptionDefault
afterYesCurrent page mapping: same shape as before, captured after the UI change.
beforeYesBaseline page mapping: { page, url, capturedAt, elements: [{role, name, selector, context?}] }. Captured before a UI change.

Output Schema

ParametersJSON Schema
NameRequiredDescription
rowsNo
addedNo
countsNo
verdictNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive, and the description adds 'Pure and deterministic' and explains the verdict and drift outputs. No contradictions; the extra context about output categories enhances transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three dense sentences, front-loaded with the core purpose and output, followed by practical acquisition instructions. Every sentence earns its place without fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich output schema and annotations, the description still adds necessary context about how to capture input mappings and the deterministic nature. It's complete for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the two required params, and the description adds meaningful detail beyond the schema by specifying the internal structure of 'elements' arrays and the optional 'context' field. This enriches understanding without redundancy.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool diffs a baseline page mapping against a current one and returns a CI-style verdict with per-element drift categories. This specific verb+resource combination distinguishes it from sibling diff tools like diff_text or json_diff.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool (comparing baseline vs current page mappings) and how to obtain the required inputs via the companion package or browser snippet. It doesn't explicitly mention alternatives or exclusions, but the use case is evident.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

diff_textA
Read-onlyIdempotent
Inspect

Compute a unified line-by-line diff between two text strings (LCS algorithm). Returns added/removed/unchanged line counts and formatted diff hunks with configurable context lines (0–20). Use to compare versions of prompts, configs, code snippets, or any text where you need to see exactly what changed.

ParametersJSON Schema
NameRequiredDescriptionDefault
aYesOriginal (before) text
bYesModified (after) text
contextNoContext lines around each change (0–20, default: 3)

Output Schema

ParametersJSON Schema
NameRequiredDescription
diffNo
addedNo
removedNo
unchangedNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds the LCS algorithm detail and specifies the return format (added/removed/unchanged counts and formatted hunks with configurable context lines), enriching behavioral understanding beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core operation, then output details, then usage guidance. Every sentence contributes meaningful information with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a straightforward 3-parameter tool with full schema coverage and an output schema, the description covers purpose, output, and usage sufficiently. Nothing critical is missing; the LCS algorithm mention and return value summary make it complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptive parameter definitions. The description reinforces the context line range but does not add substantial new semantic detail beyond what the schema already provides, matching the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource: 'Compute a unified line-by-line diff between two text strings (LCS algorithm).' This clearly distinguishes it from sibling tools like json_diff, diff_mappings, or levenshtein_distance by focusing on text diffing with line-level granularity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit use cases: 'Use to compare versions of prompts, configs, code snippets, or any text where you need to see exactly what changed.' This provides clear context, though it doesn't explicitly mention alternatives or when not to use, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

embedding_similarityA
Read-onlyIdempotent
Inspect

Compute text similarity using local algorithms (Bag of Words, TF-IDF, Character N-grams). No API key needed — runs entirely in-process. NOT real embeddings: for true semantic similarity with vector embeddings, use run_semantic_tests with mode="embeddings" and your OpenAI API key. Supports single pair or batch mode with pipe-separated pairs. Useful for RAG retrieval testing, semantic search evaluation, and text deduplication.

ParametersJSON Schema
NameRequiredDescriptionDefault
batchNoBatch mode: array of { text_a, text_b } pairs. Overrides text_a/text_b if provided.
text_aNoFirst text to compare (single-pair mode)
text_bNoSecond text to compare (single-pair mode)
methodsNoAlgorithms to use (default: all three). Options: "bow", "tfidf", "ngram"

Output Schema

ParametersJSON Schema
NameRequiredDescription
modeNo
countNo
scoresNo
text_aNo
text_bNo
resultsNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds useful context beyond annotations: runs entirely in-process, no API key, and the limitation that it's not real embeddings. However, the claim about 'pipe-separated pairs' in batch mode is inconsistent with the schema's array-of-objects definition, creating a minor ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences deliver substantial information without redundancy. The core function is front-loaded, and the caveat about real embeddings is essential context. Slightly dense but all sentences earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description adequately covers purpose, limitations, alternatives, and use cases. The only notable gap is the inconsistent batch-mode format description, which could confuse an agent about how to pass input.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers all parameters (100% coverage), so the description's mapping of 'bow', 'tfidf', 'ngram' to human-readable algorithm names adds value. The batch-mode explanation is helpful but the 'pipe-separated pairs' phrase conflicts with the schema's batch structure, slightly reducing clarity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb+resource ('Compute text similarity') and enumerates the exact local algorithms (Bag of Words, TF-IDF, Character N-grams). It explicitly distinguishes from true embeddings by clarifying 'NOT real embeddings' and pointing to run_semantic_tests, which differentiates it from sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit contrast with run_semantic_tests for embedding-based similarity, noting no API key needed and in-process execution. Also lists concrete use cases (RAG retrieval testing, semantic search evaluation, text deduplication), making when-to-use clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

env_parseA
Read-onlyIdempotent
Inspect

Parse a .env file content into a JSON object. Handles quoted values (single and double), inline comments, export prefix, and escaped sequences (\n, \t inside double quotes). Returns all key-value pairs. Use in CI/CD pipelines, agent config loaders, or when processing dotenv files programmatically.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYes.env file content to parse (e.g. the output of `cat .env`)

Output Schema

ParametersJSON Schema
NameRequiredDescription
varsNo
countNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only and idempotent behavior. The description supplements this with concrete parsing behaviors: quoted values, inline comments, export prefix, and escaped sequences, which adds useful context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences: first states purpose, second details parsing capabilities, third gives use cases. Compact, front-loaded, and every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a single parameter, complete annotations, and an output schema, the description covers all necessary context: what it does, how it handles edge cases, and when to use it. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the input parameter is well-described in the schema ('.env file content to parse (e.g. the output of cat .env)'). The description adds little beyond restating that it parses file content, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the verb (Parse) and resource (.env file content), and specifies the output (JSON object). It distinguishes itself from sibling tools like parse_csv and parse_http_headers by focusing on dotenv parsing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly lists target contexts: 'CI/CD pipelines, agent config loaders, or when processing dotenv files programmatically.' While it doesn't mention when not to use it or alternatives, the provided contexts offer clear usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

escape_htmlA
Read-onlyIdempotent
Inspect

Escape HTML special characters (&, <, >, ", ') to their safe HTML entities. ALWAYS call this before inserting any user-provided or LLM-generated content into an HTML template to prevent cross-site scripting (XSS) attacks.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesString to HTML-escape

Output Schema

ParametersJSON Schema
NameRequiredDescription
escapedNo
original_lengthNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false. The description adds the specific characters being escaped and the XSS-prevention purpose, going beyond the structural hints without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core operation. Every word earns its place—lists exact characters, states the mandatory usage, and explains the security benefit without fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with an output schema and clear annotations, the description fully covers purpose, usage, and effect. No gaps are evident; it's complete and actionable on its own.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (input described as 'String to HTML-escape'). The description adds meaning by specifying which characters get escaped and preemptively addressing the security context, enhancing the schema's basic parameter description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Escape') and resource ('HTML special characters') listing the exact characters (&, <, >, ", '). It clearly distinguishes the tool from its sibling 'unescape_html' and other escaping utilities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description says 'ALWAYS call this before inserting any user-provided or LLM-generated content into an HTML template to prevent XSS attacks', which is explicit about when to use. It does not mention when not to use or alternative tools, but the strong 'ALWAYS' directive provides clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

estimate_llm_costA
Read-onlyIdempotent
Inspect

Estimate the API cost in USD for a given model and token counts. Supports all major 2024–2026 models: GPT-4o, GPT-4.1, o3, o4-mini, Claude Opus 4, Claude Sonnet 4/4.5, Gemini 2.5 Pro/Flash, DeepSeek V3/R1, Grok 3, and legacy models.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesModel name, e.g. "gpt-4o", "claude-3.5-sonnet", "deepseek-v3"
input_tokensYesNumber of input/prompt tokens
output_tokensNoNumber of output/completion tokens (default: 0)

Output Schema

ParametersJSON Schema
NameRequiredDescription
modelNo
ratesNo
input_tokensNo
output_tokensNo
input_cost_usdNo
total_cost_usdNo
output_cost_usdNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive. The description adds useful context by listing the exact supported model families and that the result is a USD cost figure. It doesn't disclose edge cases like currency rounding or pricing data freshness, but the annotation coverage lowers the burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose and followed by a concise support list. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and simple inputs, the description covers the essential context: what is calculated, the currency, and the list of supported models. It could mention the source/timing of pricing data, but it is sufficient for selecting this tool among many siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and all parameters are described with meaningful examples. The description adds only the general 'token counts' phrasing and the model list, which does not materially improve on the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Estimate') and resource ('API cost in USD') and states the inputs (model and token counts). It is clear and distinct from siblings like count_tokens, though it does not explicitly differentiate from token_budget_calculator.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied: call this when you need a cost estimate for a given model and token counts. However, there is no explicit guidance on when not to use it or which sibling tool might be preferable (e.g., model_info, count_tokens).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_json_from_textA
Read-onlyIdempotent
Inspect

Extract the first valid JSON object or array embedded in chaotic LLM output (surrounded by markdown fences, prose, or explanatory text). Handles ```json blocks and inline JSON. Call this whenever an LLM returns structured data mixed with explanation text instead of raw JSON.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesRaw text (e.g., LLM output) that may contain a JSON object or array

Output Schema

ParametersJSON Schema
NameRequiredDescription
jsonNo
sourceNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, and non-destructive behavior. The description adds behavioral context beyond annotations by specifying that it extracts the *first* valid JSON and handles markdown-fenced and inline JSON, which is important for invocation. No contradiction with annotations found.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured: the first sentence states the core function, the second covers input format handling, and the third gives explicit usage context. Every sentence earns its place with no redundancy or unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one parameter, complete schema coverage, an output schema, and thorough annotations, the description provides ample context. It covers what the tool does, when to use it, and what input formats are acceptable, making it easy for an agent to select and invoke the tool correctly. The minor absence of edge-case behavior (e.g., no JSON found) is negligible given the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'input' is well-defined in the schema, with high coverage. The description enriches the parameter meaning by clarifying the input type ('chaotic LLM output', 'surrounded by markdown fences, prose, or explanatory text') and that the extraction targets the first valid JSON—details that go beyond the schema description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: extracting the first valid JSON object/array from chaotic LLM output. It specifies the resource (JSON in text) and the action (extract), and distinguishes itself from sibling tools like extract_json_path and format_json by focusing on embedded data surrounded by prose or markdown fences.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit when-to-use scenario: 'Call this whenever an LLM returns structured data mixed with explanation text instead of raw JSON.' This gives clear practical guidance. However, it does not name alternatives or explicitly state when not to use the tool, so it misses the upper boundary of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_json_pathA
Read-onlyIdempotent
Inspect

Extract a value from a JSON string using dot-notation path (e.g., "user.address.city", "items.0.name", "meta.tags"). Supports array index access via numeric path segments.

ParametersJSON Schema
NameRequiredDescriptionDefault
pathYesDot-notation path, e.g. "user.address.city" or "items.0.name"
inputYesThe JSON to traverse — a JSON string, or the object/array itself.

Output Schema

ParametersJSON Schema
NameRequiredDescription
pathNo
typeNo
valueNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe, non-mutating operation. The description adds clarity that it uses dot-notation and supports array index access. No contradictions with annotations. Could mention what happens when the path is not found (undefined/null) or if input is invalid, but annotations reduce the burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste. Front-loaded with core purpose and examples. Every word earns its place. A model of conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (2 params, both required), has full schema coverage, comprehensive annotations, and an output schema. The description adequately covers the core behavior and path syntax. Could briefly mention return type (any) or error case behavior, but with output schema present and low complexity, this is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both 'path' and 'input' parameters well-documented in the schema. The description reinforces the dot-notation format and gives examples, but adds no semantics beyond what the schema provides. Baseline 3 is appropriate given full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool extracts a value from a JSON string using dot-notation path, with concrete examples like 'user.address.city' and 'items.0.name'. It also notes array index access via numeric path segments. This is a specific verb+resource pairing that distinguishes it from sibling tools like 'flatten_json', 'transform_json_array', 'json_to_csv', or 'json_schema_validate'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage (reading values from JSON) but provides no explicit guidance on when to use this tool versus alternatives like 'extract_json_from_text', 'json_diff', or 'json_schema_validate'. No when-not-to-use conditions or alternative tool mentions. Minimal but functionally adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_todosA
Read-onlyIdempotent
Inspect

Extract TODO, FIXME, HACK, BUG, NOTE, OPTIMIZE, and custom tags from any source code or text. Returns line numbers, tag types, and message text. Essential for technical debt auditing.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoCustom tags to add (default set: TODO, FIXME, HACK, NOTE, BUG, OPTIMIZE, XXX)
inputYesCode or text to scan
include_contextNoInclude full line text (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
itemsNo
totalNo
countsNo
has_criticalNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is established. The description adds useful behavioral context by listing the output fields and the tag categories handled. This is consistent with annotations; no contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with the action and scope front-loaded. It avoids redundancy and every phrase carries meaningful information, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple, non-destructive tool with full schema coverage, a read-only annotation set, and an output schema. The description adds the essential purpose and a clear use case, so the agent has everything needed to select and invoke the tool correctly without ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides descriptions for all three parameters (100% coverage), so the schema carries the semantic load. The description mentions custom tags generically but doesn't add new details about the tags, include_context, or input parameter beyond what the schema already states. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('extract'), lists concrete tag types (TODO, FIXME, HACK, BUG, NOTE, OPTIMIZE), and states the return contents (line numbers, tag types, message text). This makes the tool's purpose unmistakable and differentiates it from sibling extractors like extract_links or extract_json.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly identifies a primary use case ('technical debt auditing') and says it works on 'any source code or text', giving the agent context for when to select this tool. However, it does not name alternative tools or specify when not to use it, so it stops short of explicit when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_confluence_pageA
Read-only
Inspect

Fetch a Confluence page and return its content as clean Markdown. Accepts a numeric page_id or a full page URL. Optionally lists direct child pages. BYOK — credentials transit in-memory only, never stored.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_idNoConfluence page ID (numeric string), e.g. "123456789"
page_urlNoFull Confluence page URL (alternative to page_id), e.g. "https://mycompany.atlassian.net/wiki/spaces/ENG/pages/123456789"
confluence_emailYesAtlassian account email (same credentials as Jira)
confluence_tokenYesAtlassian API token
include_childrenNoList direct child pages (id + title) (default: false)
confluence_base_urlYesAtlassian base URL, e.g. "https://mycompany.atlassian.net"

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
titleNo
page_idNo
childrenNo
markdownNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the description doesn't need to repeat that. It adds valuable behavioral context: credentials transit in-memory only and are never stored, output is clean Markdown, and the tool can optionally list child pages. This exceeds the annotation baseline.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the core purpose, and every clause adds value: input forms, optional behavior, and credential handling. No irrelevant details or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that an output schema exists, the description doesn't need to detail return values. It covers the essential invocation details (page_id/URL, credentials, optional children) and auth handling. The description is complete enough for an agent to use the tool safely and correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with detailed descriptions for every parameter, including the alternative relationship between page_id and page_url. The tool description adds little beyond restating those facts, though it does clarify 'optionality' of children listing. Since the schema is rich, the description earns the baseline score but no more.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Fetch' with a clear resource 'Confluence page' and specifies the output format 'clean Markdown'. It clearly distinguishes this from sibling tools like 'create_confluence_page' and 'fetch_jira_issue' by naming the exact system and action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states when to use the tool (to fetch a Confluence page) and mentions two input options (page_id or page_url) plus an optional children list. It does not explicitly name alternatives or exclusions, but the context is clear enough for an agent to select it over siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_jira_issueA
Read-only
Inspect

Fetch a complete Jira issue: summary, description converted to Markdown, status, assignee, priority, labels, custom fields, and optionally comments and attachment metadata. BYOK — credentials transit in-memory only, never stored on ia-qa.com.

ParametersJSON Schema
NameRequiredDescriptionDefault
fieldsNoSpecific Jira field names to return. Omit for all standard fields.
issue_keyYesJira issue key, e.g. "PROJ-123"
jira_emailYesAtlassian account email
jira_tokenYesAtlassian API token (from id.atlassian.com > Security > API tokens)
jira_base_urlYesAtlassian base URL, e.g. "https://mycompany.atlassian.net"
include_commentsNoInclude issue comments, up to 20 (default: true)
include_attachmentsNoInclude attachment metadata list (default: false)

Output Schema

ParametersJSON Schema
NameRequiredDescription
keyNo
urlNo
typeNo
labelsNo
statusNo
summaryNo
assigneeNo
priorityNo
reporterNo
descriptionNo

TDQS

A4.2/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful context beyond annotations: description is converted to Markdown, comments/attachments are optional, and BYOK credentials transit in-memory only and are never stored on ia-qa.com. This is consistent with readOnlyHint=true and non-destructive behavior, and it provides important security/formatting details not present in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose, followed by a concise security note. Every clause earns its place with no fluff or repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's purpose, key output transformations, optional inclusions, and credential handling. With an output schema present, it need not describe return structure. It is adequately complete for a 7-parameter tool with strong annotations, though it could briefly mention prerequisites like network access or Jira permissions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and all parameters have descriptions, so the baseline is 3. The description adds context about output contents (Markdown conversion, custom fields, optional comments/attachments) but does not materially extend parameter semantics beyond what the schema already documents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Fetch' and clearly identifies the resource as 'a complete Jira issue' with an explicit enumeration of returned contents (summary, description converted to Markdown, status, assignee, priority, labels, custom fields, optional comments/attachments). This distinguishes it from sibling tools like search_jira_issues and post_jira_comment.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for retrieving full Jira issue details and optional comments/attachment metadata, but it does not explicitly state when to prefer this over search_jira_issues or whether to use it before post_jira_comment. There is clear context but no exclusions or alternative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_veille_feedA
Read-only
Inspect

Fetch the latest QA & AI/LLM articles aggregated from curated RSS sources (Google Testing Blog, DEV.to Testing/QA/AI/LLM/Agents, Hugging Face Blog, Simon Willison). Perfect for agents monitoring the QA & AI landscape. Each article carries summary_source — the XML tag the summary was read from, or "none" when the feed publishes titles and links only; an empty summary with summary_source "none" is a property of that feed, not a parse failure.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax articles to return (default: 20, max: 50)
categoryNoFilter: "qa" (testing/quality), "ai" (AI/LLM/agents), "all" (default — both)

Output Schema

ParametersJSON Schema
NameRequiredDescription
articlesNo
categoryNo
total_foundNo
sources_queriedNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds significant behavioral context beyond the annotations: it explains the 'summary_source' field and clarifies that empty summaries with 'summary_source: none' are a property of the feed, not a parse failure. The annotations already declare readOnlyHint=true and openWorldHint=true, and the description doesn't contradict them. It also mentions the XML tag behavior, which is helpful for understanding the returned data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded: it states the main purpose in the first sentence, then adds context about sources and a helpful note about the summary_source field. Every sentence adds value; no waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that the tool has an output schema and the annotations provide safety hints, the description is complete. It explains the source list, the summary_source field, and the behavior. No significant gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers 100% of the parameters with descriptions (limit, category). The description doesn't add much beyond what the schema provides, but it doesn't need to since the schema is thorough. The description also mentions 'summary_source' but that's a response field, not a parameter. Score baseline 3 because the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool fetches articles from curated QA/AI RSS sources, listing the exact sources. It also distinguishes this tool from siblings by mentioning it aggregates from specific feeds, which is distinct from other fetch tools like fetch_confluence_page or fetch_jira_issue. The purpose is specific with a verb (fetch) and resource (QA & AI/LLM articles).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains this tool is 'perfect for agents monitoring the QA & AI landscape', providing clear context on when to use it. It doesn't explicitly mention alternatives like simple web fetching, but the context is clear enough for an agent to know when to use this tool. The mention of curated sources implies a specific use case compared to generic fetch tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

few_shot_formatterA
Read-onlyIdempotent
Inspect

Format few-shot examples for LLM prompts. Converts example pairs into formatted blocks. Supports chat format (User/Assistant), XML tags, Markdown, or plain text.

ParametersJSON Schema
NameRequiredDescriptionDefault
formatNoOutput format (default: chat)
examplesYesArray of {input, output} pairs
input_labelNoLabel for input (default: User / <input>)
output_labelNoLabel for output (default: Assistant / <output>)

Output Schema

ParametersJSON Schema
NameRequiredDescription
formatNo
formattedNo
example_countNo
token_estimateNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the tool's safe, non-mutating nature is well established. The description adds that it converts example pairs into formatted blocks, which is largely synonymous with its purpose and provides minimal extra behavioral context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, with two sentences that are front-loaded with the core purpose. It lists the supported formats without wasting words, making it easy to scan and understand.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's main inputs and outputs, and the schema and annotations provide the rest. Since the tool is relatively simple and has high schema coverage, the description is sufficiently complete for an agent to use it correctly, though it could theoretically include an example of the output format.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage, with descriptions for all four parameters including enum values for format. The description only repeats the format options and mentions example pairs, adding little beyond the schema. Per the baseline for high schema coverage, this is adequate but not exceptional.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: formatting few-shot examples for LLM prompts. It specifies the action (format/converts), the resource (few-shot examples), and the supported output formats (chat, XML, Markdown, plain text), which distinguishes it from sibling formatting tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context by stating it formats few-shot examples for LLM prompts, which is clear enough for most users. However, it doesn't explicitly mention alternatives or when not to use this tool, but the context is sufficient given the sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

find_toolA
Read-onlyIdempotent
Inspect

Search available MCP tools by keyword or category before calling them. Returns matching tool names, descriptions, and optionally their inputSchemas. Call this when you are unsure which tool to use or want to explore the catalogue. Categories: data, encoding, text, llm, qa, rag, dev, security, web.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesKeyword(s) to search in tool name and description (e.g. "cors", "token", "vector", "json")
categoryNoOptional: filter by category — data | encoding | text | llm | qa | rag | dev | security | web
max_resultsNoMaximum tools to return (default 10, max 50). Results are ranked by IDF-weighted relevance, so common words like "test" do not inflate the list.
with_schemaNoSet true to include inputSchema in results (default: false)

Output Schema

ParametersJSON Schema
NameRequiredDescription
hintNo
toolNo
countNo
queryNo
scoreNo
toolsNo
categoryNo
truncatedNo
total_matchesNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds behavioral context such as ranking by IDF-weighted relevance and that common words do not inflate results. It also notes the optional inclusion of inputSchemas in results, which is a useful behavioral detail beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, each earning its place: the first states the core action and timing, the second describes the return content, and the third provides the category list. There is no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema, full parameter documentation, and annotations, the description supplies the missing contextual information: when to invoke it and how results are ranked. It covers the tool's role in the larger catalogue and does so without needing to explain return structures.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so each parameter (query, category, max_results, with_schema) is already well-documented in the input schema. The description adds no additional parameter-specific meaning beyond what the schema provides. It does list the allowed categories, but this is also present in the schema description for the category parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Search' with a clear resource ('available MCP tools') and includes the scope ('by keyword or category before calling them'). It also distinguishes itself from siblings by emphasizing it is a catalogue exploration tool, not a functional tool. The return value (tool names, descriptions, optionally schemas) is stated directly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use the tool: 'Call this when you are unsure which tool to use or want to explore the catalogue.' This provides clear context for use. It does not explicitly mention when not to use it or list alternative tools, but given the meta-purpose, this is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fix_gherkinA
Read-only
Inspect

Fix Gherkin syntax warnings from a jira_to_test_suite result. Takes the current gherkin text and the _gherkin_warnings array, calls your LLM to fix ONLY the flagged issues (adds missing Given/When/Then steps, etc.), and returns the corrected Gherkin. Lightweight — uses ~300-500 tokens vs ~5k for a full regeneration. Requires BYOK LLM key.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesLLM model to use for the fix, e.g. "gpt-4o-mini". Must belong to the provider whose key you passed in api_key.
api_keyYesYour own LLM provider API key (BYOK) — OpenAI "sk-…", Anthropic "sk-ant-…", Google "AIzaSy…", or Groq "gsk_…". There is no server-side key for this tool: if you do not have one, do not call it and do not invent a value — placeholders like "configured", "your_api_key" or a masked "sk-…***…" are rejected. Used for this call only, never stored.
gherkinYesThe current Gherkin text from the jira_to_test_suite result (test_suite.gherkin).
warningsYesThe _gherkin_warnings array from the jira_to_test_suite result.

Output Schema

ParametersJSON Schema
NameRequiredDescription
latency_msNo
model_usedNo
fixed_gherkinNo
warnings_afterNo
warnings_beforeNo
remaining_warningsNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false; the description adds critical behavioral context: it calls an external LLM, uses 300-500 tokens, requires a BYOK key, and never stores the key. It also clarifies it only fixes flagged issues, providing context about the side effects and dependencies that annotations do not cover.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences: the first states the purpose, the second explains inputs/process/output, and the third adds cost and key requirements. Every sentence earns its place, and the key information is front-loaded. No redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (external LLM call, 4 params, output schema), the description covers the input source, process, output, token cost, and BYOK requirement. The output schema exists, so not detailing the return structure is fine. It is complete enough for an agent to invoke correctly, though it omits nuanced edge-case behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline 3 applies. The description adds meaning by linking 'gherkin' to 'test_suite.gherkin' and 'warnings' to '_gherkin_warnings', and by clarifying the fix scope ('ONLY the flagged issues') with an example ('adds missing Given/When/Then steps'). This enriches the parameter roles beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with 'Fix Gherkin syntax warnings from a jira_to_test_suite result', which is a specific verb+resource+source combination. It clearly distinguishes itself from sibling tools like jira_to_test_suite (generation) and generic LLM tools by focusing on post-hoc fixing of flagged warnings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It identifies when to use the tool: when there are _gherkin_warnings from a jira_to_test_suite result. The cost comparison '~300-500 tokens vs ~5k for a full regeneration' implies using this lightweight fix instead of regenerating the whole suite. The schema adds explicit 'do not call if you have no key', but the main description gives solid context without naming a specific alternative tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

flatten_jsonA
Read-onlyIdempotent
Inspect

Flatten a nested JSON object to single-level dot-notation keys (e.g. {"a":{"b":1}} → {"a.b":1}), or unflatten dot-notation keys back to a nested object. Supports custom separators.

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNo"flatten" (default) or "unflatten"
inputYesThe JSON to flatten or unflatten — a JSON string, or the object itself.
separatorNoKey separator (default: ".")

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultNo
key_countNo
max_depthNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the safety profile is clear. The description adds behavioral context about directional behavior (flatten vs. unflatten) and allows custom separators, which is helpful. It does not disclose edge cases (e.g., handling arrays, collisions) or performance traits. Given strong annotations, this is acceptable but not exceptional.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences cover purpose, both modes, and a key customization. Every sentence adds value with no redundancy or fluff. The example in parentheses is a nice touch that clarifies the output format without being verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is relatively simple with 3 fully documented parameters and a clear output schema. The description covers the bidirectional transformation and customization adequately. It could mention behavior with arrays or nested objects that aren't flat, but the description and schema provide sufficient information for an agent to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters. The description adds conceptual context about dot-notation transformation and separators but does not add concrete details beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool flattens nested JSON to dot-notation keys and can also unflatten them back. The specific verb 'Flatten' combined with the resource 'nested JSON object' and clear one-to-many directional capability differentiates it from sibling JSON tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains both directions (flatten/unflatten) and mentions support for custom separators, giving clear context. However, it does not explicitly state when to use this tool vs. other JSON transformation siblings like 'transform_json_array' or 'merge_json', nor does it mention when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

format_bytesA
Read-onlyIdempotent
Inspect

Convert raw byte counts to human-readable sizes in SI (KB=1000) or IEC (KiB=1024) units, or parse size strings back to bytes. Covers B, KB/KiB, MB/MiB, GB/GiB, TB/TiB, PB/PiB.

ParametersJSON Schema
NameRequiredDescriptionDefault
bytesNoNumber of bytes to format
standardNoOutput standard (default: both)
size_stringNoSize string to parse to bytes (e.g. "1.5 GB", "512 MiB")

Output Schema

ParametersJSON Schema
NameRequiredDescription
bytesNo
originalNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only and idempotent behavior. The description adds behavioral detail beyond annotations by stating the tool supports both formatting (bytes to string) and parsing (string to bytes), and specifies the exact unit systems. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences. The first sentence states the core transformation in both directions, and the second lists the covered units. No redundant or filler text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, the schema covers all parameters, annotations cover safety, and an output schema exists, so the description need not explain return values. The description fully captures the tool's scope and standards, making it complete for an agent to select and use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all three parameters. The description adds value by defining the SI and IEC standards explicitly (KB=1000, KiB=1024), which clarifies the 'standard' enum, and by explaining that 'size_string' is for parsing back to bytes—enhancing the schema's brief parameter comments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Convert raw byte counts to human-readable sizes... or parse size strings back to bytes.' It clearly identifies the tool's dual functionality and the unit standards (SI vs IEC), distinguishing it from generic formatting or conversion tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context by explaining the two output standards (KB=1000, KiB=1024) and the range of units covered, which implies when to use each. It does not explicitly exclude alternatives, but for a single-purpose utility this is sufficient context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

format_jsonA
Read-onlyIdempotent
Inspect

Validate and pretty-print a string that is ALREADY valid JSON. Strict by design — it is a validity gate: valid JSON comes back formatted, anything else is rejected with the exact parse error. It never repairs, completes, or guesses. NOT for: plain text or prose (will fail), JSON embedded in markdown/prose (use extract_json_from_text first), JS objects (JSON.stringify them first), YAML (use yaml_to_json).

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesA raw JSON string, e.g. '{"key":"value"}'. Must already parse as JSON — plain text or truncated JSON is rejected, not repaired.
indentNoIndent size (default: 2)

Output Schema

ParametersJSON Schema
NameRequiredDescription
validNo
formattedNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (readOnlyHint, idempotentHint), the description adds critical behavioral context: it is a validity gate, rejects invalid input with exact parse error, and never repairs, completes, or guesses. This gives clear expectations for edge cases without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured: front-loaded main purpose, followed by strictness details, then specific exclusions. Every sentence earns its place with actionable information, and the use of em-dashes and 'NOT for' makes it highly scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, complete schema, and available annotations, the description fully covers purpose, usage, exclusions, and behavior. The presence of an output schema means return values are already documented, so no further explanation is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description does not add significant parameter-level meaning beyond the schema, though it reinforces the strict parsing requirement. The schema already documents both parameters thoroughly, so no compensation is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: validate and pretty-print a string that is already valid JSON. It specifies the resource (JSON string), the action (validate/pretty-print), and the strict scope, distinguishing it from siblings like extract_json_from_text and yaml_to_json.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides when-to-use and when-not-to-use guidance, including specific alternatives for other cases: extract_json_from_text for JSON embedded in prose, JSON.stringify for JS objects, yaml_to_json for YAML, and notes that plain text will fail. This is exemplary usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

format_tableA
Read-onlyIdempotent
Inspect

Convert a JSON array of objects into a Markdown table. Automatically detects columns, aligns headers, and fills missing keys with empty cells. Use when an agent needs to present structured data — tool results, model comparisons, test reports — as a readable table in a response or document.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesThe array of objects to render — a JSON string, or the array itself.
columnsNoColumn names and order (default: all keys from first row)

Output Schema

ParametersJSON Schema
NameRequiredDescription
rowsNo
tableNo
columnsNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds value beyond annotations by explaining that the tool automatically detects columns, aligns headers, and fills missing keys. It also notes that the input can be a JSON string or the array itself. This is consistent with the readOnlyHint and idempotentHint annotations. The description does not repeat annotation info but adds behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences: first sentence states purpose, second describes behavior, third gives usage guidance. It is concise, front-loaded, and every sentence adds value. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (2 parameters, 1 required), the presence of an output schema, and annotations covering safety, the description is complete. It explains purpose, behavior, and usage. The output schema likely documents the return format, so the description does not need to repeat that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with both parameters having descriptions in the schema. The description adds context to the input parameter ('The array of objects to render') but does not provide additional semantics beyond what the schema already states. Per the guidelines, baseline 3 is appropriate when schema coverage is high.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'convert', the resource 'JSON array of objects', and the output 'Markdown table'. It also mentions automatic column detection, header alignment, and missing key handling, which distinguishes it from sibling tools like format_json or format_bytes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use when an agent needs to present structured data — tool results, model comparisons, test reports — as a readable table'. This provides clear usage context. However, it does not mention when not to use this tool or directly reference alternatives among the many sibling formatting tools, which would be a minor improvement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

function_call_validateA
Read-onlyIdempotent
Inspect

Validate an LLM function call / tool_use output: check that function name is in allowed list, arguments match expected schema, no extra/missing args. For OpenAI function calling & MCP tool_use testing.

ParametersJSON Schema
NameRequiredDescriptionDefault
function_callYesThe function call object from LLM (e.g. { "name": "get_weather", "arguments": {"city":"Paris"} })
allowed_functionsYesList of allowed function definitions

Output Schema

ParametersJSON Schema
NameRequiredDescription
validNo
errorsNo
error_countNo
function_nameNo
provided_argsNo
required_argsNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish safety (read-only, idempotent, non-destructive). The description adds the behavioral logic of the validation checks, which is beyond the annotations. It does not discuss error handling, but the presence of an output schema offsets that need.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core function, followed by context. No redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is well-described for its complexity. The output schema and annotations cover return values and safety. The description covers validations performed and target platforms. Minor ambiguity about what 'expected schema' means relative to the required/optional args format, but it's adequately complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description's reference to 'arguments match expected schema' adds little beyond the schema's own descriptions. It does not clarify the allowed_functions structure beyond what's in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Validate' with a clear resource 'LLM function call / tool_use output', and enumerates the exact validation checks (name in allowed list, arguments schema, extra/missing args). This clearly distinguishes it from generic validators like llm_output_validator or json_schema_validate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the use case: 'For OpenAI function calling & MCP tool_use testing.' This provides clear context for when to use the tool. It does not explicitly name alternatives or when not to use it, but the targeted scope is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_ci_workflowA
Read-onlyIdempotent
Inspect

Generate a ready-to-commit GitHub Actions workflow that gates a build on IA-QA. Two gate types, combinable: "eval_contract" runs a .ia-eval.yaml through ia-qa-com/eval-action@v1 (LLM quality gate, needs a provider API key as a repo secret), and "cli_checks" runs deterministic primitives via npx @ia-qa/cli (secret scan, prompt-injection scan, security headers…) whose exit code fails the build. Deterministic template — no LLM call, no API key, same inputs give the same file. Returns the YAML, the secrets to create, and the remaining steps. Pair with generate_eval_yaml to produce the contract itself.

ParametersJSON Schema
NameRequiredDescriptionDefault
cronNoCron expression when triggers include 'schedule' (default: '0 6 * * 1' — Mondays 06:00 UTC).
gateNoWhich gate to emit. eval_contract = LLM eval via the action (default). cli_checks = deterministic CLI assertions. selector_drift = an E2E selector-drift gate via @ia-qa/self-healing (boots the app, captures, diffs against the committed baseline, branches on exit code 0/1/2). both = CLI checks + eval. all = CLI checks, then drift, then eval.
providerNoLLM provider the contract runs against — decides which repository secret the workflow wires (default: groq).
triggersNoWorkflow triggers (default: push + pull_request).
cli_toolsNoIA-QA tool names to run as deterministic gates, e.g. ["secret_scan","prompt_injection_scan"]. Tools with no known CI recipe get a --stdin step flagged in notes.
min_scoreNoOverride the contract min_score (0-100). Omit to use the value in the contract.
app_base_urlNoURL the drift gate waits for before capturing (default: http://127.0.0.1:3000). Must match config.baseUrl in .ia-qa/config.json.
fail_on_failNoFail the build on a FAIL/PARTIAL verdict (default: true). Set false to report without gating.
node_versionNoNode version for the CLI steps (default: "20").
contract_pathNoPath to the .ia-eval.yaml contract, relative to the repo root (default: evals/smoke.ia-eval.yaml). Only used when the gate includes eval_contract.
workflow_nameNoWorkflow display name (default: "IA-QA Quality Gate").
app_start_commandNoCommand that boots the app for the selector_drift gate, e.g. "npm run start:ci". Cannot be guessed — omitted, the step carries an explicit TODO and the note says so, because a plausible default would silently map nothing.

Output Schema

ParametersJSON Schema
NameRequiredDescription
gateNo
pathNo
yamlNo
notesNo
next_stepsNo
secrets_requiredNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already signal readOnly/idempotent/non-destruct, and the description adds important behavior beyond them: it is deterministic, makes no LLM call, requires no API key despite supporting LLM-based eval, and returns the YAML plus secrets and remaining steps. It also discloses the provider-API-key prerequisite for eval_contract, giving the agent useful operational context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but economical: purpose first, then gate types, then determininism, then return payload, then the companion tool. Every sentence adds distinct value, and there is no filler or repetition of schema content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parametér tool, the description combined with a 100%-covered schema and an output schema is nearly complete. The one gap is that it says 'Two gate types' while the gate enum actually includes selector_drift as a third gate type plus combination values, so the prose slightly under-maps the tool's full capability.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Paraméter description coverage is 100%, so the schema already explains every parametér with defaults, enums, and nuances (e.g., app_start_command cannot be guessed). The description adds narrative around gate types but does not materially enhance parametér semantics beyond what the schema provides, which makes the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb, resource, and outcome: generate a ready-to-commit GitHub Actions workflow that gates a build on IA-QA. It also distinguishes the tool from sibling generators by naming generate_eval_yaml as complementary and by describing the exact gate types it emits.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when the tool applies and explicitly points to generate_eval_yaml as the companion tool for producing the contract. It does not enumerate exclusions or alternatives beyond that, so it stops short of fully routing an agent to or away from other generator siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_curlA
Read-onlyIdempotent
Inspect

Generate a curl command from request parameters. Supports GET/POST/PUT/DELETE, custom headers, JSON body, and form data. Useful for documentation, sharing, and debugging API calls.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesRequest URL (must be http/https)
bodyNoRaw request body string
methodNoHTTP method (default: GET)
headersNoRequest headers as key-value object
verboseNoAdd -v for verbose output (default: false)
body_jsonNoJSON body (auto-adds Content-Type: application/json)
follow_redirectsNoFollow redirects with -L flag (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
curlNo
methodNo
header_countNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, and non-destructive behavior. Description adds context about supported HTTP methods and output (curl command), which complements the annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with the primary purpose. Every clause adds value—methods, body types, and use cases are all included without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 7 parameters, an output schema, and a straightforward function, the description provides sufficient high-level context. It does not need to enumerate parameters or return format, as those are covered by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema fully documents all parameters. The description mentions JSON body and form data, which loosely map to body_json and body, but adds no new semantic information beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: generating a curl command from request parameters, with specific supported methods and body types. It distinguishes itself from sibling tools by being the only curl-focused generation tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides explicit use cases: documentation, sharing, and debugging API calls. No exclusions or alternative tools are mentioned, but this is sufficient given the tool's simplicity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_eval_yamlA
Read-only
Inspect

Generate a complete .ia-eval.yaml evaluation contract from a plain-language description of what your LLM should do. Uses Groq openai/gpt-oss-20b (server-side, no API key needed). Returns ready-to-run YAML for the LLM Test Runner (run_eval_contract). Picks appropriate evaluators (cosine_similarity, contains_check, hallucination_check, etc.) based on the task type.

ParametersJSON Schema
NameRequiredDescriptionDefault
task_typeNoOptional task type hint to guide evaluator selection.
descriptionYesPlain-language description of what the LLM under test should do. Be specific: describe inputs, expected behaviour, and constraints.
system_promptNoOptional system prompt of the LLM under test. Helps generate more accurate test cases.
scenario_countNoNumber of scenarios to generate (default: 5). Covers happy path + edge cases + adversarial.

Output Schema

ParametersJSON Schema
NameRequiredDescription
yamlNo
task_typeNo
model_usedNo
scenario_countNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true and openWorldHint=true, which align with the description's mention of server-side generation with no API key needed, adding useful context. The description also discloses that it picks appropriate evaluators based on task type, which is beyond annotations, but doesn't detail side effects or limitations like model latency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded: it states what it generates, the model used, and the output integration, all in three sentences. It avoids unnecessary detail and each sentence contributes to understanding the tool's purpose and behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, the description covers essential aspects: generation, model, output usage, and evaluator selection. With a high schema coverage and an output schema present, the description does not need to explain return values. It misses a bit on how the tool handles ambiguous task descriptions, but overall it is fairly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% coverage with descriptions for all parameters, including enums and hints. The description adds some value by explaining how task_type influences evaluator selection and mentions scenario_count covers happy path/edge cases/adversarial, but does not deeply elaborate beyond schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool generates a complete .ia-eval.yaml evaluation contract from a plain-language description, which is a specific verb and resource. It differentiates from siblings like generate_test_cases and prompt_test_suite by focusing on the evaluation contract format and integration with run_eval_contract.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it (when you need a ready-to-run evaluation contract) and mentions it pairs with run_eval_contract, but does not explicitly state when not to use it or name specific alternatives. However, the context is clear enough for an agent to decide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_hmacA
Read-onlyIdempotent
Inspect

Compute an HMAC signature for a message using a secret key. Supports SHA-256 (default), SHA-512, SHA-1, and MD5. Used for API request signing, webhook verification (GitHub, Stripe, Twilio), and JWT validation.

ParametersJSON Schema
NameRequiredDescriptionDefault
secretYesSecret key
messageYesMessage to sign
encodingNoOutput encoding (default: hex)
algorithmNoHash algorithm: sha256 (default), sha512, sha1, md5

Output Schema

ParametersJSON Schema
NameRequiredDescription
hmacNo
encodingNo
algorithmNo
message_lengthNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is known. The description adds algorithm support and default behavior, but it does not disclose output formatting details (e.g., lowercase hex) or any security nuances such as avoiding MD5 in production. With annotations provided, this level of added context is adequate but not exceptional.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loads the core function, and provides practical context in the second sentence. Every word earns its place with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, pure, read-only cryptographic utility with a full input schema, output schema, and strong annotations, the description is complete. It covers purpose, supported algorithms, defaults, and real-world use cases, leaving no meaningful gap for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully describes message, secret, encoding, and algorithm. The description repeats the algorithm choices and default but adds no new parameter-level meaning beyond what the schema provides, placing it at the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific action, 'Compute an HMAC signature for a message using a secret key,' clearly identifying the tool's resource and operation. It distinguishes itself from sibling tools like hash_text and base64_encode by explicitly naming HMAC and keyed signing, and it reinforces this with concrete use cases such as API signing and webhook verification.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists explicit usage contexts: 'API request signing, webhook verification (GitHub, Stripe, Twilio), and JWT validation.' While it does not mention when not to use the tool or name alternatives, the stated use cases are clear enough for an agent to select this tool among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_html_reportA
Read-onlyIdempotent
Inspect

Convert a run_eval_contract() LLM Test Runner JSON result into a fully self-contained dark-themed HTML report with Pass/Fail badges, side-by-side Input/Output/Ground-Truth panels, evaluator score bars, and a radar chart. Returns the HTML as a string.

ParametersJSON Schema
NameRequiredDescriptionDefault
resultsYesThe JSON object returned by run_eval_contract()

Output Schema

ParametersJSON Schema
NameRequiredDescription
htmlNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds valuable context by specifying the return type ('Returns the HTML as a string') and the self-contained nature, which are not in the annotations. No contradictions or hidden side effects are present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, information-dense sentence. It front-loads the core action and enumerates specific output components, with zero filler or redundant phrasing. Every clause adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the tool's complexity (rich HTML report generation), the description fully specifies input source, output characteristics, and return type. An output schema exists, so return values are already structured, and the description adds the necessary behavioral context without leaving major gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides 100% coverage with a description for the single 'results' parameter. The tool description repeats this source info without adding further parameter-level detail (e.g., edge cases, required fields inside the object). Baseline 3 applies since the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Convert') and resource ('run_eval_contract() LLM Test Runner JSON result') and details the output format (dark-themed HTML report with badges, panels, score bars, radar chart). This clearly differentiates it from siblings and states exactly what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use the tool: whenever a run_eval_contract() result needs to be turned into a visual report. It does not explicitly mention alternatives or exclusions, but the purpose is so specific that usage context is unambiguous. A named alternative would have earned a 5, but the absence doesn't hurt.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_json_ldA
Read-onlyIdempotent
Inspect

Generate a ready-to-paste snippet for GEO / structured data optimization. Supported types: WebSite, FAQPage, Article, Person, Organization, SoftwareApplication, HowTo.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeYesSchema @type: "WebSite", "FAQPage", "Article", "Person", "Organization", "SoftwareApplication", "HowTo"
fieldsNoSchema fields as key-value pairs (name, url, description, author, datePublished, etc.)
faq_itemsNoFor FAQPage/HowTo: array of { question, answer } objects

Output Schema

ParametersJSON Schema
NameRequiredDescription
nameNo
schemaNo
snippetNo
acceptedAnswerNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds useful behavior beyond the annotations: it specifies the output format (script tag) and the scope of supported types. The annotations already indicate read-only and idempotent behavior, and the description is consistent, contributing transparency about what the tool returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first states the primary function and output, the second lists supported types. Every word earns its place, with no redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of a rich input schema, output schema, and annotations, the description is adequate for a user to understand the tool's purpose and basic usage. It could mention the purpose of the 'fields' or 'faq_items' parameters, but the schema already covers that, so no significant gap remains.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters with descriptions. The tool description does not add extra semantic detail beyond the schema, fitting the baseline of 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('Generate') and resource ('a ready-to-paste <script type="application/ld+json"> snippet') and lists supported schema types, distinguishing it from sibling tools like 'score_geo_signals' and other data transformation utilities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear context for when to use the tool ('for GEO / structured data optimization') and enumerates supported types, but it does not explicitly mention alternatives or exclusions. The context is sufficiently clear given the focused purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_passwordA
Read-only
Inspect

Generate a cryptographically secure random password using crypto.randomBytes. Configurable length (4–128), uppercase letters, digits, and symbols. Use when resetting user passwords, seeding test accounts, or generating API secrets.

ParametersJSON Schema
NameRequiredDescriptionDefault
lengthNoPassword length (4–128, default: 16)
numbersNoInclude digits (default: true)
symbolsNoInclude symbols like !@#$ (default: false)
uppercaseNoInclude uppercase letters (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
lengthNo
passwordNo
charset_sizeNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as read-only and non-destructive. The description adds the cryptographically secure, random nature of the output and the length range, which is useful behavioral context beyond what annotations declare. It does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, front-loaded with the verb, and every sentence adds value: purpose, feature summary, and use cases. No fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With four optional parameters fully described in the schema and an output schema present (as indicated by context), the description provides sufficient context for selection and invocation. It includes purpose, capabilities, and appropriate use cases, making it complete for a generator tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the description does not need to repeat parameter details. The description mentions the configurable features (length, uppercase, digits, symbols) but adds no deeper semantics beyond what the schema already provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Generate'), a specific resource ('password'), and implementation detail ('using crypto.randomBytes'). It distinguishes from sibling tools like generate_uuid or generate_hmac by focusing on passwords.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases ('resetting user passwords, seeding test accounts, or generating API secrets'), but does not mention when NOT to use it or alternatives. Sibling tools are not directly referenced, but the guidance is clear enough for most agents.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_slugA
Read-onlyIdempotent
Inspect

Convert any string into a URL-friendly slug: lowercase, ASCII-normalized (é→e), special characters removed, spaces replaced with hyphens. Use for generating SEO-friendly URL paths, file names, or identifier keys from user-provided titles or labels.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesString to slugify
separatorNoSeparator character (default: "-")

Output Schema

ParametersJSON Schema
NameRequiredDescription
slugNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so safety is covered. The description adds behavioral details beyond annotations: the exact normalization behavior (é→e, lowercase, special chars removed) and the default separator via the schema. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the transformation rules, followed by usage guidance. Every word earns its place; no redundant phrasing or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, schema covers parameters, annotations cover safety, and an output schema exists (so return values need not be explained). The description fully covers what the tool does and when to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both 'input' and 'separator' parameters. The description adds no new parameter-specific details beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Convert' with a clear resource ('any string') and output ('URL-friendly slug'). It enumerates the transformation steps (lowercase, ASCII-normalized, special characters removed, spaces to hyphens), making it unmistakable from sibling tools like case_convert or url_encode.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states usage contexts: 'generating SEO-friendly URL paths, file names, or identifier keys from user-provided titles or labels.' It does not explicitly mention when NOT to use it or name alternative tools, but the context is clear enough for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_test_casesA
Read-onlyIdempotent
Inspect

Generate a set of test cases (valid, edge, invalid, pairwise) for a given feature description. Declared constraints drive the boundaries: a length or numeric bound ("[8-64]", "min 8 chars", "at least 8 characters" — read from inputs, and from the feature prose when a sentence names exactly one field) yields the last accepted value AND the first rejected one; a format (email/url/uuid, from the type, the field name or the wording) yields malformed-value cases. A bound nobody declared is labelled as this tool's assumption, not as expected behaviour. gherkinFormat renders every case (cap 200, stated in the output) and gherkinScenarioCount lets you check it against totalCases.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputsNoOptional: list of input parameters (one per line, e.g. "email: string [required]", "password: string [required, min 8 chars]", "age: number [18-99]")
featureYesFeature or function to test. Be specific: describe inputs, expected behaviour, context. Constraints stated here ("password must be at least 8 characters") are used when the sentence names exactly one field.

Output Schema

ParametersJSON Schema
NameRequiredDescription
featureNo
test_casesNo
parsedInputsNo
gherkinFormatNo
gherkinScenarioCountNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations already declaring readOnlyHint=true, idempotentHint=true, and destructiveHint=false, the description adds valuable behavioral context: it explains how constraints are interpreted (declared vs assumed boundaries), that Gherkin output is capped at 200 cases, and that assumptions are labeled. This goes beyond the annotations without contradicting them. It doesn't fully describe all possible behaviors (e.g., what happens with invalid input), but for a read-only generator, this is strong.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core purpose. The second sentence is dense but efficiently packs key behavioral rules (boundaries, assumptions, caps). It could be slightly more structured (e.g., separating input handling from output details), but it earns its place without fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema (which likely includes totalCases, gherkin, etc.), so the description needn't explain return values. It thoroughly covers parameter semantics, edge-case generation logic, and output constraints (cap 200). Given the tool's moderate complexity and the rich annotations/schema, the description is complete enough for an agent to select and invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds significant meaning: it explains that 'feature' prose constraints are used when a sentence names exactly one field, and it details how 'inputs' formats (e.g., 'min 8 chars') drive boundary generation. It also introduces parameters 'gherkinFormat' and 'gherkinScenarioCount' which are NOT in the provided input schema (likely from additional properties), and explains their purpose. This exceeds the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource ('Generate a set of test cases (valid, edge, invalid, pairwise) for a given feature description'). It specifies the output types and the source input (feature description), clearly distinguishing it from siblings like 'fix_gherkin' or 'jira_to_test_suite'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains when to use the tool (for a feature description with declared constraints) and how constraints drive boundary generation. It does not explicitly state when not to use it or name alternative tools, but the context is clear enough given the sibling set. The mention of 'gherkinFormat' and 'gherkinScenarioCount' parameters implicitly guides usage for Gherkin-related needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_uuidA
Read-only
Inspect

Generate one or more cryptographically random UUID v4 identifiers. Use this when you need unique IDs for test fixtures, database records, session tokens, or any scenario requiring a guaranteed-unique string. Returns up to 100 UUIDs in one call.

ParametersJSON Schema
NameRequiredDescriptionDefault
countNoNumber of UUIDs to generate (1–100, default: 1)

Output Schema

ParametersJSON Schema
NameRequiredDescription
countNo
uuidsNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate a safe read-only operation, and the description adds useful behavioral context: cryptographically random generation, support for multiple UUIDs, and the 100-UUID cap. This goes beyond the structured annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences deliver the core function, use cases, and output limit without any fluff. The most important information appears first, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple utility tool with one well-documented parameter, an output schema, and strong annotations, the description covers all necessary context: what it does, when to use it, and what output to expect. No significant gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already fully describes the single 'count' parameter (1–100, default 1), so the description's mention of 'up to 100 UUIDs in one call' adds minimal new meaning. Baseline of 3 is appropriate given 100% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool generates cryptographically random UUID v4 identifiers, which is a specific verb+resource combination. It also differentiates from sibling tools like generate_password or generate_hmac by focusing on UUIDs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases ('test fixtures, database records, session tokens') and states when to use it. It does not explicitly list when not to use it or mention alternative tools, but the guidance is clear and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_testing_guidelinesA
Read-onlyIdempotent
Inspect

Query the IA-QA methodology knowledge base. Returns structured testing guidelines, assertion strategies, thresholds, best practices, and relevant MCP tools for a given topic. Call without a topic to list all available topics. Topics: llm-unit-testing, rag-pipeline, prompt-stability, prompt-ab-testing, embedding-quality, eval-framework, semantic-testing, auto-testing, security, api-testing, ci-cd, multimodal, llm-data-security, agent-observability, pro-tips, learning-paths, golden-dataset.

ParametersJSON Schema
NameRequiredDescriptionDefault
topicNoThe testing topic to retrieve guidelines for. Omit to get the full list of available topics.

Output Schema

ParametersJSON Schema
NameRequiredDescription
tipNo
topicNo
usageNo
keywordsNo
available_topicsNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly/idempotent/non-destructive behavior. The description adds useful behavioral context beyond annotations: it is a knowledge-base query rather than an execution tool, returns structured rather than raw content, and supports a no-topic listing mode. There is no contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loads the core purpose and return content, and wastes no words. The topic list is useful but partly redundant with the schema enum and incomplete, which prevents a top score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-optional-parameter read-only knowledge-base tool with rich annotations and an output schema, the description covers invocation modes, return content, and available topics. The prose topic list missing selector-drift and the lack of explicit sibling-tool boundaries are the only notable gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema is fully documented with an explicit enum, so the baseline should be 3. However, the description's 'Topics:' list omits 'selector-drift', which is present in the schema enum. This can mislead an agent into believing that topic is unavailable, adding incorrect information beyond the schema rather than merely restating it. That inconsistency drops the score below baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource ('Query the IA-QA methodology knowledge base') and clearly enumerates the returned content: structured testing guidelines, assertion strategies, thresholds, best practices, and relevant MCP tools. It does not explicitly differentiate itself from sibling test-execution tools like run_semantic_tests or multimodal_eval_guide, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit invocation guidance: call with a topic to retrieve guidelines, and call without a topic to list all available topics. This makes the optional parameter behavior clear. It does not state when not to use this tool or name alternative tools, so it lacks the exclusionary guidance needed for a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

guardrail_testA
Read-onlyIdempotent
Inspect

Test an LLM response against a set of guardrail rules: must-include, must-not-include, max length, required format, language, forbidden patterns, and custom regex. Returns pass/fail per rule.

ParametersJSON Schema
NameRequiredDescriptionDefault
rulesYesArray of guardrail rules to check
responseYesThe LLM response to test

Output Schema

ParametersJSON Schema
NameRequiredDescription
passNo
ruleNo
labelNo
valueNo
detailNo
failedNo
passedNo
resultsNo
all_passedNo
total_rulesNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds behavioral context by enumerating supported rule types and stating that it returns pass/fail per rule, which is useful beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that front-loads the core purpose, lists rule examples for clarity, and ends with the return behavior. Every word earns its place; there is no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's modest complexity (two parameters) and the presence of an output schema, the description is adequately complete. It mentions the return format (pass/fail per rule) and enumerates rule types, though it does not address edge cases like invalid regex, which could be inferred from the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides descriptions for both parameters, with 100% coverage. The description's list of rule types echoes the enum values in the schema without adding new semantic detail, so it contributes little beyond the structured documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool does: tests an LLM response against guardrail rules, with a specific verb and resource. It lists the rule types, which distinguishes it from sibling tools like toxicity_scan or prompt_injection_scan that focus on specific safety dimensions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description establishes a clear context for use: validating LLM responses against custom guardrail rules. It does not explicitly name alternatives or exclusion criteria, but the specific rule categories imply a general-purpose guardrail testing role distinct from more specialized sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hallucination_checkA
Read-onlyIdempotent
Inspect

Lexical hallucination check: verifies an LLM answer's words, numbers and polarity against the provided source/context. Fast, deterministic, no API key needed. Each answer sentence is aligned to its best-matching source sentence, so a number only counts as support when it sits on the SAME statement ("founded in 1998" is not grounded by "sold 1998 units"), and a negation or antonym flip against that sentence returns verdict "contradicted" — the corrupted-fact hallucination that reuses source vocabulary. Limitations: still lexical — it cannot follow a paraphrase, a synonym, or multi-sentence reasoning, so a "well_grounded" verdict means "nothing lexical found", never "verified true". For entailment use run_semantic_tests (NLI/embedding) or a calibrated judge.

ParametersJSON Schema
NameRequiredDescriptionDefault
answerYesThe LLM-generated answer to verify
strictNoIf true, every sentence in the answer must be supported (default: false)
contextYesThe source/reference text that should ground the answer

Output Schema

ParametersJSON Schema
NameRequiredDescription
detailNo
messageNo
numbersNo
overlapNo
verdictNo
analysisNo
entitiesNo
groundedNo
sentenceNo
total_wordsNo
matched_wordsNo
contradictionsNo
grounded_countNo
unbound_claimsNo
grounding_scoreNo
total_sentencesNo
ungrounded_countNo
unsupported_claimsNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations include readOnlyHint true and idempotentHint true, and the description does not contradict them. It adds useful behavioral context: the alignment algorithm for sentences, the handling of negation/antonym flips, and the explicit caveat about lexical limitations ('cannot follow a paraphrase...'). This exceeds the baseline for a read-only tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is detailed yet well-structured, with a clear explanation, a concrete example, and a limitation note. It might be slightly long but each sentence earns its place; it's front-loaded with the core purpose and usage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity, the description covers the main use case, limitations, and provides an example for clarity. The schema covers parameters and output schema exists, so completeness is good for an LLM agent to invoke correctly, though it could add a bit more about the verdict values ('contradicted', 'well_grounded') but these are implied.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and does not contradict: it describes the behavior and the strict parameter's meaning briefly, but the schema already defines all parameters. The description adds explanation about sentence alignment for 'strict' mode behavior, which is helpful, so it meets the baseline with slight bonus.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the tool performs a 'lexical' hallucination check, explicitly verifying words, numbers, and polarity against a source. It distinguishes itself from sibling tools like semantic tests by emphasizing its fast, deterministic, and lexical nature, and even names the alternative (run_semantic_tests).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly explains when to use this tool: for quick, deterministic verification of factual alignment, and when not to use it: when entailment or deeper reasoning is needed ('For entailment use run_semantic_tests'). It also mentions limitations that guide appropriate usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hash_textA
Read-onlyIdempotent
Inspect

Compute a cryptographic hash of a text string. Use when you need to verify data integrity, generate content fingerprints, hash passwords (prefer SHA-256+), or produce a fixed-length digest of any input. Supports SHA-256 (default), SHA-512, SHA-1, and MD5.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to hash
algorithmNoHash algorithm: sha256 (default), sha512, sha1, md5

Output Schema

ParametersJSON Schema
NameRequiredDescription
hashNo
algorithmNo
input_lengthNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive. The description adds behavioral traits such as 'cryptographic' and 'fixed-length digest,' and explains algorithm support including the default (SHA-256) and a recommendation for password hashing. This enriches the agent's understanding of output characteristics without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two crisp sentences: the first states the core function, the second packs use cases and algorithm list. No filler words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with two parameters and an output schema, the description covers purpose, usage contexts, algorithm choices, and output nature (fixed-length digest). Combined with annotations and schema, the agent has everything needed to select and invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema describes both parameters with 100% coverage, so the description doesn't need to repeat them. It does provide a small addition by recommending 'prefer SHA-256+' for password hashing, which helps with algorithm selection, but otherwise adds little beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Compute' and resource 'cryptographic hash of a text string,' clearly distinguishing it from siblings like base64_encode (encoding) and generate_hmac (keyed hash). It also lists concrete use cases (data integrity, content fingerprints, password hashing), making the tool's purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use the tool: 'Use when you need to verify data integrity, generate content fingerprints, hash passwords (prefer SHA-256+), or produce a fixed-length digest of any input.' This provides clear usage context, though it does not explicitly mention alternative tools like generate_hmac for keyed hashes, so it lacks explicit exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

html_to_markdownA
Read-onlyIdempotent
Inspect

Convert HTML to clean Markdown. Strips scripts, styles, nav, ads, and comments. Converts headings, lists, links, images, code blocks. Ideal for preparing web content as LLM context.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesHTML string to convert
strip_linksNoStrip link URLs, keep text only (default: false)

Output Schema

ParametersJSON Schema
NameRequiredDescription
markdownNo
markdown_lengthNo
original_lengthNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite annotations already declaring readOnly, idempotent, and non-destructive, the description adds valuable behavioral detail: it strips scripts, styles, nav, ads, and comments, and converts specific HTML elements. This goes beyond annotations and gives the agent a clear mental model of the tool's output.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first states the primary action, the second lists what is stripped and converted, ending with a use case. Every word contributes, with no wasted space or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is moderate complexity, but with an output schema present and clear annotations, the description covers all essential aspects: input, transformations, removals, and ideal usage context. No critical gaps remain for an agent to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with both 'input' and 'strip_links' already described in the schema. The description mentions links in the conversion list, which slightly reinforces the strip_links option, but it adds no new parametric meaning beyond the schema. Baseline 3 is appropriate given the complete schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb+resource: 'Convert HTML to clean Markdown.' It then enumerates the transformations (headings, lists, links, images, code blocks) and removals (scripts, styles, nav, ads, comments), making the tool's function unmistakable and distinguishing it from siblings like strip_markdown or escape_html.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Ideal for preparing web content as LLM context' provides clear context for when to use the tool, but it does not explicitly mention exclusions or alternatives. Since siblings like strip_markdown or extract_links exist, a brief 'when-not' statement would elevate this, but the context is adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

http_status_lookupA
Read-onlyIdempotent
Inspect

Look up detailed information about any HTTP status code: class, name, description, cacheability, typical causes, and handling best practices. Covers every code in the IANA HTTP Status Code Registry (1xx-5xx, including 226, 425, 451, 508, 511 and the WebDAV codes) with its defining RFC; anything outside the registry is reported as registered: false rather than described. cacheable means heuristically cacheable by default per RFC 9110 §15.1.

ParametersJSON Schema
NameRequiredDescriptionDefault
codeYesHTTP status code (e.g. 200, 404, 429, 503)

Output Schema

ParametersJSON Schema
NameRequiredDescription
codeNo
descNo
nameNo
classNo
cacheableNo
registeredNo
descriptionNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly and idempotent, which the description reinforces with a clear non-destructive lookup nature. The description adds value by specifying the registry scope and reporting behavior for non-registered codes (registered: false), and clarifies the meaning of cacheability per RFC 9110. However, it doesn't mention response format details like pagination or error codes, but given the annotations and output schema, the marginal gap is acceptable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact, two sentences, and conveys all necessary details: purpose, scope, edge cases, and specific terminology. Zero fluff, front-loaded with the primary function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (one parameter, output schema present, no nested objects). The description covers the registry scope, non-registry behavior, and the meaning of cacheability. It might benefit from mentioning the output format or the returning of RFC references, but the output schema presumably covers that. Not enough to score 5, but solidly complete for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (code parameter fully described). The description does not add extra syntax or formatting details beyond what the schema provides, but the schema itself is sufficient given the single, self-explanatory parameter. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Look up') with a clear resource ('HTTP status code') and specifies the exact scope (IANA registry, 1xx-5xx including edge codes). It clearly distinguishes from sibling tools which are mostly unrelated (encoding, formatting, analysis of other domains).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly states the tool is for looking up detailed info about HTTP status codes, covering the entire IANA registry. It does not explicitly mention alternative tools or when not to use it, but the broad coverage and domain specificity effectively imply usage context, and it explicitly notes the behavior for non-registry codes.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

identify_callerA
Read-onlyIdempotent
Inspect

Returns what the server knows about the current MCP client: clientInfo captured during initialize, User-Agent, and any _meta fields sent with this request. Useful for debugging caller identification.

ParametersJSON Schema
NameRequiredDescriptionDefault
_metaNoOptional self-identification. Keys: agent (string), model (string), version (string).

Output Schema

ParametersJSON Schema
NameRequiredDescription
noteNo
sessionNo
meta_overrideNo
effective_agentNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, so the safety profile is covered. The description adds behavioral detail by specifying the exact data returned, including the optional _meta request field, without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main return value, and contains no redundant or speculative language. Every phrase contributes to understanding the tool's function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a low-complexity tool with rich annotations, an output schema, and a description that covers the return contents, the single optional parameter, and a typical use case. Nothing important is missing for an agent to decide whether to invoke it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of the parameter with descriptions for _meta and its sub-properties. The description adds meaning by clarifying that the _meta field is echoed in the response ('any _meta fields sent with this request'), tying the parameter to the tool's output.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Returns' and enumerates exactly what is returned: clientInfo captured during initialize, User-Agent, and any _meta fields. This clearly distinguishes it from sibling utilities, which address different functions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Useful for debugging caller identification' provides clear contextual guidance for when to use the tool. It does not name alternatives or explicitly say when not to use it, but the uniqueness of the tool makes this less critical.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

jira_to_test_suiteA
Read-only
Inspect

Transform a Jira ticket into a complete test suite: Gherkin scenarios, E2E steps, API test cases, test data matrix, and ambiguity detection. Accepts either Jira credentials (auto-fetch) or a pre-fetched issue object. The returned test_suite includes _gherkin_warnings (deterministic syntax validation — empty if clean). Requires BYOK LLM key (OpenAI, Anthropic, etc.).

ParametersJSON Schema
NameRequiredDescriptionDefault
issueNoPre-fetched issue object from fetch_jira_issue, OR a mock object with fields: key, summary, description (plain text or Markdown), status, issue_type, priority, labels, comments. Use this for offline/CI testing without Jira credentials.
modelYesLLM model to use, e.g. "gpt-4o-mini", "claude-3-5-haiku-20241022", "gemini-2.0-flash".
api_keyYesYour LLM provider API key (OpenAI sk-, Anthropic sk-ant-, Google AIzaSy-, etc.).
issue_keyNoJira issue key to fetch automatically, e.g. "PROJ-123". Required if issue is not provided.
jira_emailNoAtlassian account email. Required for auto-fetch mode.
jira_tokenNoAtlassian API token. Required for auto-fetch mode.
max_tokensNoMaximum tokens for the LLM response. Default: 8192. Increase for large tickets with many ACs; decrease to reduce cost on simple tickets.
jira_base_urlNoAtlassian base URL. Required for auto-fetch mode.
confluence_pagesNoOptional array of pre-fetched Confluence page objects from fetch_confluence_page, used as documentation context.

Output Schema

ParametersJSON Schema
NameRequiredDescription
summaryNo
issue_keyNo
issue_urlNo
latency_msNo
model_usedNo
test_suiteNo
tokens_usedNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so the description doesn't need to restate that. It adds valuable context: requires a BYOK LLM key, returns _gherkin_warnings with deterministic validation, and supports auto-fetch via credentials. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core transformation outcome, then modes, then key requirement and output caveat. Every sentence carries distinct information with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 params, two mutually exclusive input modes, nested objects), the description covers purpose, input modes, required external key, and an important output field (_gherkin_warnings). Output schema exists, so full return values aren't needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds the key relationship that credentials and pre-fetched issue are alternative modes, helping the agent decide whether to use issue or issue_key+jira_* parameters. This grouping insight goes beyond the schema's individual parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource: 'Transform a Jira ticket into a complete test suite' and lists concrete outputs (Gherkin scenarios, E2E steps, API test cases, test data matrix, ambiguity detection). This clearly distinguishes it from sibling tools like generate_test_cases or prompt_test_suite by anchoring to Jira ticket input.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly explains the two input modes: Jira credentials (auto-fetch) or a pre-fetched issue object, and the schema adds the offline/CI testing use case for the issue parameter. It does not compare against alternatives or state exclusions, so not a 5, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_diffA
Read-onlyIdempotent
Inspect

Compute a deep structural diff between two JSON values. Returns added, removed, and changed keys with dot-notation paths. Like git diff but for JSON objects — perfect for API response regression testing. Arrays are not compared blindly by position: the same elements in a different order collapse to a single "reordered" change, and an array of records sharing a stable identity field (id, uuid, key, name…) is matched by that field, so paths read [id=42] and a moved record is not reported as N rewrites.

ParametersJSON Schema
NameRequiredDescriptionDefault
afterYesThe modified JSON (after) — a JSON string, or the value itself.
beforeYesThe original JSON (before) — a JSON string, or the value itself.
max_depthNoMax nesting depth to recurse (default: 10)

Output Schema

ParametersJSON Schema
NameRequiredDescription
addedNo
changesNo
removedNo
modifiedNo
identicalNo
total_changesNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes beyond annotations by explaining key behavioral details: how arrays are handled (not by position, but by identity or order), how paths are formatted (dot notation, [id=42]), and that reordered elements are treated as a single change. This is significant added context beyond the readOnly/idempotent annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, directly to the point, and contains no filler or redundancy. It efficiently conveys the core purpose and a key behavior in a compact form.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is quite complete given that an output schema exists. It explains the main return semantics (added, removed, changed keys) and the special handling of arrays, which covers the essential aspects. It does not need to detail return format since that is presumably in the output schema. A minor gap is not explaining depth limit behavior, but that is likely covered by the max_depth parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all three parameters, and the description does not add substantially new info about them beyond what the schema already states. The description's mention that values can be strings or JSON objects is already present in the schema types and descriptions, so it provides minimal added parameter insight.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: computing a deep structural diff between two JSON values. It uses a specific verb (compute) and a specific resource (JSON values), and distinguishes it from sibling tools like diff_text or diff_mappings by focusing on structural JSON comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a concrete use case ('perfect for API response regression testing') which helps an agent decide when to employ this tool. However, it does not explicitly mention alternative tools or when not to use it, though the context is sufficiently implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_schema_generateA
Read-onlyIdempotent
Inspect

Infer a JSON Schema (draft-07) from a sample JSON value. Detects types, required fields, array item shapes, nested objects, and common string formats (email, uri, date, date-time, uuid). Returns a ready-to-use schema compatible with json_schema_validate. Use when you have a sample API response or LLM output and want to auto-generate a validation schema for CI/CD testing.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesThe sample JSON value to infer the schema from — a JSON string, or the value itself.
required_allNoMark all detected object properties as required (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeNo
itemsNo
formatNo
schemaNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint and idempotentHint, so the safety profile is clear. The description adds behavioral details: it detects types, required fields, array shapes, nested objects, common formats (email, uri, date, date-time, uuid), and returns a schema compatible with json_schema_validate. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences front-loaded with the core action, followed by capabilities and use case. No redundancy, every sentence earns its place. Highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity and the presence of an output schema, the description covers input nature, detection capabilities, and typical use cases. It lacks mention of error handling or format limitations but is overall adequate for an agent to understand and invoke.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds minimal parameter-specific meaning; it implies 'input' is a sample JSON value but does not elaborate on the 'required_all' parameter behavior. The schema itself already documents these.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool infers a JSON Schema from a sample JSON value, specifying verb and resource. It distinguishes from siblings like json_schema_validate and json_diff by focusing on generation rather than validation or diffing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use when you have a sample API response or LLM output and want to auto-generate a validation schema for CI/CD testing,' providing clear context. However, it does not contrast with alternatives or state when not to use, which would strengthen guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_schema_validateA
Read-onlyIdempotent
Inspect

Validate a JSON value against a JSON Schema (draft-07 subset). Supports type, required, properties, items, enum, const, pattern, format (email/uri/date), minimum/maximum, minLength/maxLength, minItems/maxItems, uniqueItems, additionalProperties, anyOf, allOf, oneOf. Returns all validation errors with dot-notation paths.

ParametersJSON Schema
NameRequiredDescriptionDefault
valueYesThe JSON value to validate — a JSON string, or the value itself.
schemaYesThe JSON Schema — a JSON string, or the schema object itself.

Output Schema

ParametersJSON Schema
NameRequiredDescription
validNo
errorsNo
error_countNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, making safety clear. The description adds useful behavioral context: it validates against a 'draft-07 subset' and lists supported keywords, which helps the agent understand limitations. However, it does not disclose performance considerations (e.g., large schemas) or error handling beyond returning errors with dot-notation paths.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (two sentences) and front-loaded with the core purpose. The second sentence lists supported features efficiently. It could be slightly more structured (e.g., bullet points) but is clear and free of fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (2 params, draft-07 subset) and the presence of an output schema, the description covers the key validation features and return format. It does not mention edge cases (e.g., cyclic schemas) or how to handle unknowns, but the annotations and schema provide sufficient context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters are fully described in the schema itself. The description adds minimal extra semantic meaning—only noting that value and schema can be JSON strings or objects. The return value format is covered by an output schema, so no additional param info is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The tool name 'json_schema_validate' is descriptive, and the description explicitly states 'Validate a JSON value against a JSON Schema (draft-07 subset)'. This clearly distinguishes it from siblings like json_schema_generate or json_diff, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions the supported schema features (e.g., type, required, properties) but does not provide explicit guidance on when to use this tool versus alternatives like llm_json_schema_check or function_call_validate. There is no mention of when not to use it or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_to_csvA
Read-onlyIdempotent
Inspect

Convert a JSON array of objects to CSV format. Automatically detects columns from all object keys. Handles quoting and escaping per RFC 4180.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesThe array of objects to convert — a JSON string, or the array itself.
headersNoInclude header row (default: true)
delimiterNoColumn delimiter (default: ",")

Output Schema

ParametersJSON Schema
NameRequiredDescription
csvNo
rowsNo
columnsNo
column_namesNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, idempotentHint=true, and destructiveHint=false, indicating safe non-destructive behavior. The description adds valuable behavioral context: automatic column detection and RFC 4180 quoting/escaping, which aids understanding beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the key purpose. Every sentence earns its place: first sentence states the primary function, second adds specific behaviors (column detection and quoting). No redundancy or unnecessary details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple conversion tool, the description covers the core aspects: input format, automatic column handling, and quoting standard. The output schema exists (not shown), so return format is covered externally. Minor gap: it does not address how nested objects are handled (e.g., stringified or flattened), but overall it is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (baseline 3). The description adds meaning by stating 'Automatically detects columns from all object keys,' which clarifies the input's structure and how headers relate to keys. It also mentions RFC 4180 compliance for quoting, adding depth to delimiter and escaping behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'Convert a JSON array of objects to CSV format,' which is a clear verb+resource. It distinguishes from siblings like parse_csv (reverse operation) and json_to_yaml (different format). The addition of 'Automatically detects columns from all object keys' further clarifies the behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when you need to convert a JSON array of objects to CSV, but it does not explicitly state when not to use it or mention alternatives. Given the many sibling tools, more guidance would be beneficial, but the purpose is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

json_to_yamlA
Read-onlyIdempotent
Inspect

Convert a JSON object to clean, human-readable YAML. Handles nested objects, arrays, multiline strings, and special characters. No external dependencies.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesThe JSON to convert to YAML — a JSON string, or the value itself.
indentNoIndentation size in spaces (default: 2)

Output Schema

ParametersJSON Schema
NameRequiredDescription
yamlNo
linesNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive, so the bar is lower. Description adds value by clarifying output is human-readable YAML, handles special cases (nested, arrays, multiline), and has no external dependencies. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two informative sentences with no waste. First sentence states core purpose, second adds behavioral details. Every part earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists to describe return values, so description doesn't need to explain output structure. The description, combined with annotations and schema, provides a complete understanding for a simple conversion tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. Description adds context for the input parameter (JSON string or value itself) and mentions clean human-readable output, which supplements schema. The indent parameter is well-described by schema, no extra needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool converts JSON to YAML, highlighting handling of nested objects, arrays, multiline strings, and special characters. It effectively distinguishes from sibling tools like yaml_to_json and other json-related conversions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by naming the conversion task, but doesn't guide when to use this tool versus alternatives (e.g., other format converters in sibling list). No explicit when-not or context for choosing this over similar tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

latency_benchmarkA
Read-only
Inspect

Measure response time of one or more HTTP endpoints (GET/POST). Runs N iterations and returns min/max/avg/p95 latency. Useful for API and MCP server benchmarking.

ParametersJSON Schema
NameRequiredDescriptionDefault
endpointsYesEndpoints to benchmark. Accepts a single URL string, an array of URL strings, or an array of {url, method?, body?, headers?, label?} objects.
iterationsNoNumber of iterations per endpoint (default: 3, max: 10)

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultsNo
iterationsNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as read-only and non-destructive. The description adds that it runs N iterations and returns latency statistics, which is beyond the hints. It also mentions support for GET/POST, but does not contradict the read-only annotation since the tool only measures timing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three concise sentences: purpose, behavior/return values, and use case. It is front-loaded and contains no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has a rich schema and output schema, and the description covers the main behavior and intended use. No additional details are needed for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with complete descriptions for 'endpoints' and 'iterations'. The description's mention of 'N iterations' and 'GET/POST' mirrors the schema without adding new semantic details, hence the baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'measure' and identifies the resource as 'response time of one or more HTTP endpoints (GET/POST)'. It clearly distinguishes from siblings like mcp_server_health_check by focusing on latency and benchmark scenarios.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states it is 'Useful for API and MCP server benchmarking', giving clear context for when to deploy it. It does not mention alternatives or exclusion criteria, so it stops short of a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

levenshtein_distanceA
Read-onlyIdempotent
Inspect

Compute the Levenshtein (edit) distance and normalized similarity ratio between two strings. Supports batch comparison. Useful for fuzzy string matching, deduplication, and test result comparison.

ParametersJSON Schema
NameRequiredDescriptionDefault
aNoFirst string (single-pair mode)
bNoSecond string (single-pair mode)
batchNoBatch of {a,b} pairs (max 50)
case_insensitiveNoIgnore case differences (default: false)

Output Schema

ParametersJSON Schema
NameRequiredDescription
aNo
bNo
modeNo
countNo
resultsNo
distanceNo
similarityNo
operations_neededNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds batch support and normalized ratio but does not discuss edge cases, normalization formula, or output structure. This matches the baseline for annotations providing the main behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with the primary function front-loaded and use cases following. Every word earns its place; there is no redundant or promotional language.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a straightforward calculation tool with an output schema and solid annotations. The description plus schema fully cover usage, including batch mode and case sensitivity. The only minor omission (explicit guidance that a/b vs batch are mutually exclusive) is already clear from schema descriptions like 'single-pair mode.'

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with each parameter ('a', 'b', 'batch', 'case_insensitive') already explaining its role. The description adds no new parameter-level details, so it earns the baseline 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Compute the Levenshtein (edit) distance and normalized similarity ratio between two strings.' This clearly distinguishes it from sibling tools like embedding_similarity or vector_similarity by naming the exact algorithm and output type. The mention of batch comparison further clarifies scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists concrete use cases ('fuzzy string matching, deduplication, and test result comparison'), providing clear context for when to apply the tool. However, it does not explicitly name alternative tools or exclude scenarios, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lint_commit_messageA
Read-onlyIdempotent
Inspect

Validate a git commit message against the Conventional Commits spec (feat, fix, docs, style, refactor, test, chore, ci, perf, build). Returns compliance score, breaking change detection, and actionable suggestions.

ParametersJSON Schema
NameRequiredDescriptionDefault
strictNoEnforce strict rules: max 72-char subject, imperative mood check (default: false)
messageYesGit commit message to validate

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeNo
scopeNo
scoreNo
validNo
checksNo
subjectNo
has_bodyNo
is_breaking_changeNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, establishing safety. The description adds useful outcome details, such as returning a compliance score, breaking change detection, and actionable suggestions, which goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the core purpose and then lists output details. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple validation tool with two well-documented parameters, a full output schema, and strong safety annotations. The description covers the purpose, spec, and expected output, making it complete for an agent to decide and invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both 'message' and 'strict' clearly documented. The description does not add parameter-level detail beyond the schema, but it also does not need to; the schema fully explains the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Validate a git commit message against the Conventional Commits spec,' and lists the allowed commit types. This clearly distinguishes it from sibling validation tools like validate_email or json_schema_validate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies the tool is used when you need to validate a git commit message against Conventional Commits. It does not explicitly mention exclusions or alternatives, but the context is unambiguous and sufficient for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_llm_modelsA
Read-onlyIdempotent
Inspect

List all LLM models available on ia-qa.com with their provider, API endpoint, and capabilities. Filter by provider name (e.g. "Groq", "HuggingFace", "OpenAI") or return the full catalog. Use this to discover which models are available before calling an LLM API, or to compare providers.

ParametersJSON Schema
NameRequiredDescriptionDefault
providerNoFilter by provider name (case-insensitive). E.g. "Groq", "HuggingFace", "OpenAI", "Anthropic", "Google", "DeepSeek", "xAI", "Ollama". Omit for full catalog.

Output Schema

ParametersJSON Schema
NameRequiredDescription
totalNo
filterNo
modelsNo
providersNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds that the tool returns provider, endpoint, and capabilities, and that it lists models from ia-qa.com. It does not disclose pagination or rate limits, but for such a simple tool this is adequate. No contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the primary purpose and output fields. The second sentence explains filtering and usage examples. Every phrase adds value, with no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (one optional parameter), strong annotations, and the existence of an output schema, the description is sufficiently complete. It covers the main behavior, output scope, and typical use cases without needing to explain return structures.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents the optional 'provider' parameter at 100% coverage, including examples. The description adds meaning by explaining the filtering effect and explicitly stating that omitting the parameter returns the full catalog, which is not apparent from the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List'), the resource ('LLM models available on ia-qa.com'), and the output fields (provider, API endpoint, capabilities). It accurately conveys the tool's purpose but does not explicitly differentiate it from closely related sibling tools like 'model_info' or 'compare_models', which also deal with LLM models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases: 'discover which models are available before calling an LLM API, or to compare providers.' This gives clear context for when to use the tool, though it does not explicitly mention alternatives or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_local_testsA
Read-onlyIdempotent
Inspect

Discover .ia-eval.yaml LLM test suite files in the project directory. Scans CWD and standard sub-directories (evals/, tests/, contracts/). Returns file paths ready to pass to run_eval_contract.

ParametersJSON Schema
NameRequiredDescriptionDefault
dirNoDirectory to scan (defaults to server CWD)

Output Schema

ParametersJSON Schema
NameRequiredDescription
dirNo
countNo
filesNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that the tool is read-only, idempotent, and non-destructive. The description adds behavioral context by specifying the scan locations (CWD, evals/, tests/, contracts/) and the output format (file paths ready for run_eval_contract). However, it does not detail edge-case behavior such as error handling, recursion depth, or how the optional 'dir' parameter interacts with the default scan paths, leaving some behavioral aspects undisclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences that are succinct and front-loaded. The first sentence states the primary purpose, and the second explains the output's usability. No unnecessary words or redundant content are present, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (one optional parameter), the annotations covering its safety, and the presence of an output schema, the description provides sufficient context. It informs the user of typical scan locations and that the results are directly consumable by run_eval_contract. It slightly lacks context about the behavior when 'dir' is provided versus the default CWD scan, but overall it is complete enough for its scope.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for the only parameter 'dir' with a clear description ('Directory to scan (defaults to server CWD)'). The tool description does not add extra parameter details beyond this, so it does not enhance what the schema already conveys. With high schema coverage, a baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Discover .ia-eval.yaml LLM test suite files in the project directory.' It specifies a concrete verb ('Discover'), a well-defined resource ('.ia-eval.yaml LLM test suite files'), and scopes the action to the CWD and standard subdirectories. It also distinguishes itself from the sibling run_eval_contract by noting that it returns file paths ready for that tool, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the usage context: you use this tool to locate local test suite files that can subsequently be passed to run_eval_contract. This provides clear context for when to use it, though it does not explicitly mention alternatives or exclusions. The connection to run_eval_contract gives a practical use case without explicitly saying 'use this before running tests.'

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_fit_finderA
Read-onlyIdempotent
Inspect

Find the best LLM for a given use case. Compares 30+ cloud API models and 12+ local models by cost, speed, benchmarks, features and VRAM requirements. Returns ranked recommendations with cost simulation. No API key needed.

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNocloud (API models) or local (Ollama/self-hosted). Default: cloud
top_nNoNumber of recommendations to return (default: 5)
vram_gbNoGPU VRAM in GB (only for mode=local). Default: 16
featuresNoRequired features: vision, function_calling, json_mode, streaming, reasoning
use_caseNoPrimary use case: chatbot | code | rag | summarization | classification | reasoning | agents | multilingual
max_budgetNoMaximum monthly budget in USD (based on tokens_per_day)
quantizationNoQuantization (only for mode=local): Q4_K_M | Q8_0 | FP16. Default: Q4_K_M
tokens_per_dayNoEstimated daily token volume (default: 100000)

Output Schema

ParametersJSON Schema
NameRequiredDescription
modeNo
scoreNo
resultsNo
vram_gbNo
use_caseNo
quantizationNo
tokens_per_dayNo
total_matchingNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate a safe, read-only, idempotent operation. The description adds valuable context: 'No API key needed' and the scope of models compared (30+ cloud, 12+ local). It also discloses that results are ranked with cost simulation, which is helpful beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the primary purpose, then provides essential details and a key differentiator ('No API key needed'). Every sentence contributes value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (8 optional params, output schema, clear annotations), the description covers the main use case, comparison dimensions, output format, and access requirements. The output schema also exists, so return values need not be detailed in the description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline of 3 applies. The description mentions relevant dimensions (cost, speed, VRAM) that relate to parameters like max_budget, tokens_per_day, and vram_gb, but it does not add significant new meaning beyond the detailed schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Find the best LLM for a given use case.' It specifies comparison criteria (cost, speed, benchmarks, features, VRAM) and output type (ranked recommendations with cost simulation), distinguishing it from generic comparison tools like compare_models or list_llm_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: use this tool when you need to select an LLM for a specific use case. It does not explicitly mention alternatives or state when not to use it, but the context is sufficiently clear to guide an agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_format_checkA
Read-onlyIdempotent
Inspect

Validate that an LLM output matches an expected format: JSON, Markdown, code block, bullet list, numbered list, table, YAML, XML, or custom regex. Essential for structured output testing.

ParametersJSON Schema
NameRequiredDescriptionDefault
outputYesThe LLM output to validate
regex_patternNoCustom regex pattern (only when expected_format is "regex")
expected_formatYesExpected format

Output Schema

ParametersJSON Schema
NameRequiredDescription
validNo
checksNo
failedNo
passedNo
total_checksNo
expected_formatNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the list of supported formats and the use case ('structured output testing'), which is helpful context. However, it does not disclose any additional behavioral traits such as error handling, return structure, or whether it simply returns a boolean or detailed results; the presence of an output schema lessens this burden, but the description itself adds no behavioral depth beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences, front-loaded with the core purpose ('Validate that an LLM output matches an expected format'). The list of formats is compact though comprehensive, and 'Essential for structured output testing' adds valuable context without bloat. Every sentence earns its place; no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple validation tool with strong annotations, a 100% schema-covered parameter set, and an output schema, the description is nearly complete: it states the tool's purpose, the formats it supports, and the context (structured output testing). The only notable gap is that it does not clarify its relationship to similar sibling tools, but given the rich structured data, the description itself is sufficient for most use cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters (output, expected_format, regex_pattern) are already fully documented in the schema. The description merely repeats the enum values from expected_format and reiterates that regex_pattern is for custom regex, which does not add new meaning beyond the structured field descriptions. The baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Validate') and resource ('LLM output...expected format'), and lists the supported formats (JSON, Markdown, code block, bullet list, numbered list, table, YAML, XML, regex). This clearly states what the tool does, but it does not explicitly differentiate it from sibling tools like 'llm_output_validator' or 'regex_test', so it misses the upper bound of full distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Essential for structured output testing' provides clear usage context, implying it should be used to verify LLM output formatting. However, it does not explain when to use this tool instead of alternatives (e.g., json_schema_validate for schema validation, regex_test for regex, or llm_output_validator for broader validation), nor does it state exclusions or prerequisites. This is implied usage rather than explicit guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_generateA
Read-only
Inspect

Generate text using open-source LLM models hosted on Groq (ultra-fast) or HuggingFace Inference (serverless). No API key required — the server provides its own keys. Supported models: Qwen3 32B, Gemma 4 27B, Gemma 3 27B, Llama 3.3 70B, Llama 4 Scout, DeepSeek R1, Mistral Small 24B, and more. Use list_llm_models to see the full catalog. Rate-limited to prevent abuse.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNoModel ID (default: "openai/gpt-oss-20b"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.6-27b; HuggingFace: Qwen/Qwen3-32B, meta-llama/Llama-3.3-70B-Instruct, deepseek-ai/DeepSeek-R1, google/gemma-3-27b-it, and more. Other ids from list_llm_models are BYOK-only and will be rejected.
promptYesThe user prompt / instruction to send to the model
systemNoOptional system prompt to set context or persona
max_tokensNoMaximum tokens to generate (default: 2048, max: 4096)
temperatureNoSampling temperature 0.0–1.5 (default: 0.7)

Output Schema

ParametersJSON Schema
NameRequiredDescription
modelNo
usageNo
contentNo
providerNo
latency_msNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds rate limiting and server-provided keys, which are beyond annotations. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences covering purpose, models, and rate limit. No fluff, but the model list is slightly verbose; could reference list_llm_models more directly. Still efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists, description covers key aspects: purpose, providers, no API key, rate limiting, and a sibling reference. Sufficient for a generation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so description adds little extra beyond the schema. It does mention using list_llm_models for the full catalog, but that's minor extra value. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Generate text using open-source LLM models' with specific providers (Groq, HuggingFace) and distinguishes from list_llm_models by instructing to use that sibling for the full catalog.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly points to list_llm_models for full model list and mentions providers and rate limit, giving context. Does not explicitly state when not to use, but purpose is clear enough among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_json_schema_checkA
Read-onlyIdempotent
Inspect

Validate that an LLM JSON output matches a JSON Schema definition. Tests required fields, types, enums, nested objects, and arrays. Critical for function-calling and structured output testing.

ParametersJSON Schema
NameRequiredDescriptionDefault
outputYesThe LLM JSON output (raw string, will be parsed)
schemaYesJSON Schema (draft-07 subset) to validate against

Output Schema

ParametersJSON Schema
NameRequiredDescription
validNo
errorsNo
error_countNo
parse_errorNo
parsed_typeNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, and non-destructive behavior. The description adds behavioral insight by enumerating the validation checks performed (required fields, types, enums, nested objects, arrays), which goes beyond the structured annotation data and gives the agent a clearer expectation of tool behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, each earning its place. The first sentence states the core purpose, the second details specific validation coverage, and the third provides contextual importance. It is front-loaded and free of unnecessary fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, the description is complete: it states the purpose, the validation details, and the primary use cases. Since an output schema exists (as indicated by context signals), the description does not need to explain return values. It adequately covers the necessary context for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides full descriptions for both parameters (100% coverage), which sets a baseline of 3. The description adds value by explaining the validation semantics (tests required fields, types, etc.), giving the agent a better understanding of how the parameters are used in practice.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Validate') and a clearly defined resource ('LLM JSON output' against 'a JSON Schema definition'). It explicitly calls out key validation aspects (required fields, types, enums, nested objects, arrays) and differentiates itself from generic validators by focusing on LLM outputs and structured output testing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool: 'Critical for function-calling and structured output testing.' While it does not explicitly name alternatives or exclusion criteria, the stated use cases are specific enough to guide an agent, especially given the sibling-tool context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

llm_output_validatorA
Read-onlyIdempotent
Inspect

Validate an LLM response against QA criteria: format checks (JSON, code, markdown), content rules (must-include, must-not-include), length constraints, language detection, and safety patterns. Essential for QA testing LLM-powered features.

ParametersJSON Schema
NameRequiredDescriptionDefault
outputYesThe LLM output text to validate
max_lengthNoMaximum character length for the output
min_lengthNoMinimum character length for the output
check_safetyNoCheck for PII patterns (emails, phones, SSN), profanity signals, and prompt leakage
must_includeNoComma-separated strings that MUST appear in the output
expected_formatNoExpected output format
must_not_includeNoComma-separated strings that must NOT appear (e.g. "TODO, FIXME, undefined, NaN")
check_json_schemaNoIf expected_format is JSON, provide required keys as comma-separated list to validate the structure
expected_languageNoExpected language of the output (en, fr, es, de…). Checks for common words.

Output Schema

ParametersJSON Schema
NameRequiredDescription
totalNo
checksNo
failedNo
passedNo
verdictNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, and the description's 'Validate' is consistent. It adds a list of validation categories, but these mostly mirror the schema parameters; no additional behavioral details like auth or return behavior. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence with a colon-separated list, front-loaded with verb and resource. Every clause earns its place and there is no filler or repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 9 parameters and a detailed schema, the description provides a solid high-level scope and use case. It doesn't need to explain return values because an output schema exists. A minor gap is not explicitly relating it to more specific sibling validators, but overall it is adequately complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed per-parameter descriptions, so baseline 3 applies. The description adds a high-level grouping of parameter categories (format, content, length, language, safety) that helps conceptualize the tool, but it doesn't add new semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool validates LLM responses against QA criteria, enumerating specific checks (format, content, length, language, safety). This distinguishes it from sibling tools like compare_responses or llm_format_check by positioning it as a comprehensive QA validation utility.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It says 'Essential for QA testing LLM-powered features', giving clear context for when to use it. It doesn't explicitly name alternatives or exclusions, so it stops short of full usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lorem_ipsumA
Read-only
Inspect

Generate Lorem Ipsum placeholder text for UI mockups, design prototypes, or test data population. Configurable paragraphs (1–10), sentences per paragraph (1–20), and approximate words per sentence (3–30).

ParametersJSON Schema
NameRequiredDescriptionDefault
paragraphsNoNumber of paragraphs to generate (1–10, default: 1)
words_per_sentenceNoApproximate words per sentence (3–30, default: 10)
sentences_per_paragraphNoSentences per paragraph (1–20, default: 5)

Output Schema

ParametersJSON Schema
NameRequiredDescription
paragraphsNo
paragraph_countNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true and destructiveHint=false, so the description does not need to restate the read-only nature. The description adds the fact that the text is configurable, but it duplicates the parameter ranges already present in the schema. It does not disclose optional behavioral traits such as randomness or output formatting, but with annotations covering the safety profile, the description is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core purpose and followed by parameter specifics. Every word earns its place; there is no fluff or repetition of information beyond the concise parameter enumeration. It is appropriately sized for a simple generation tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (3 optional parameters, no nested objects) and the presence of both an output schema and safety annotations, the description is complete. It covers the purpose and use cases, and the parameter details are fully provided in the schema. The description does not need to explain return values because an output schema exists.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% parameter description coverage, so the baseline is 3. The description repeats the parameter ranges ('paragraphs (1–10), sentences per paragraph (1–20), approximate words per sentence (3–30)') without adding any new semantics beyond what the schema already provides. It does not explain parameter interactions or edge-case behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Generate Lorem Ipsum placeholder text.' It clearly states the tool's function and distinguishes it from all siblings, as no other tool generates placeholder text. The use cases are specific and make the purpose immediately understandable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides concrete use cases ('for UI mockups, design prototypes, or test data population'), which gives clear context for when to use the tool. While it does not explicitly name alternatives or exclusions, the uniqueness of the tool among siblings makes this less critical. The guidance is sufficient for an AI agent to select it appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mcp_schema_lintA
Read-onlyIdempotent
Inspect

Lint an MCP tool definition for best practices: naming conventions, description quality, schema completeness, required fields consistency, description length. Returns actionable warnings.

ParametersJSON Schema
NameRequiredDescriptionDefault
tool_definitionYesMCP tool definition object with name, description, inputSchema

Output Schema

ParametersJSON Schema
NameRequiredDescription
gradeNo
errorsNo
warningsNo
error_countNo
quality_scoreNo
warning_countNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so no contradiction. The description adds 'Returns actionable warnings' and the list of lint checks, but it does not disclose edge cases, behavior on invalid input, or the exact structure of warnings, which the output schema may not fully convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one dense, front-loaded sentence that efficiently lists the tool's scope and output. Every phrase earns its place—no filler or repetition of schema content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (one parameter), rich annotations, and presence of an output schema, the description is largely complete. It covers purpose, checks, and return value. It slightly lacks explicit context about preconditions or alternative tool disambiguation, but these are minor given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single tool_definition parameter, so baseline is 3. The description repeats the parameter's purpose (linting that definition) but adds no extra semantic detail beyond the schema's 'object with name, description, inputSchema'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Lint') and resource ('MCP tool definition') and enumerates concrete aspects (naming conventions, description quality, schema completeness, required fields consistency, description length). This clearly distinguishes it from sibling tools like mcp_server_evaluate or validate_mcp_response.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage: when you need to validate an MCP tool definition against best practices. However, it does not explicitly state when to prefer this over alternatives, nor does it mention exclusions or prerequisites, leaving some ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mcp_server_evaluateA
Read-only
Inspect

Run a full compliance evaluation against a live MCP server URL. Tests: server reachability (ping), manifest discovery (GET /mcp), schema quality (snake_case names, descriptions, inputSchema), JSON-RPC 2.0 test call, and P50/P95 latency. Returns a PASS/FIX/BLOCK verdict with a 0-100 score and per-check details.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesBase URL of the MCP server (e.g. https://ia-qa.com or http://localhost:3001)
test_tool_nameNoSpecific tool name to use in the JSON-RPC test call (defaults to the first tool in the manifest)

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
scoreNo
checksNo
latencyNo
verdictNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, and the description adds meaningful behavioral context: it performs live network requests, runs multiple test categories, and returns a PASS/FIX/BLOCK verdict with a numeric score. This goes beyond the annotations by clarifying the tool's side effect of contacting an external server.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. It front-loads the main purpose, then efficiently lists the test components in parentheses and the output format. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has moderate complexity and an output schema, so the description does not need to reiterate return values. It covers the main inputs and outputs, and the annotations cover safety characteristics. Missing minor context like potential network prerequisites or timeout behavior, but not critical for invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for both parameters (url and test_tool_name), and the schema already provides clear descriptions including the default behavior for test_tool_name. The description adds no additional parameter-level detail beyond what the schema supplies, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Run a full compliance evaluation') and clearly identifies the resource ('a live MCP server URL'). It enumerates concrete checks (ping, manifest discovery, schema quality, JSON-RPC call, latency) that distinguish it from generic tools like mcp_server_health_check or mcp_schema_lint.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool (for comprehensive MCP server evaluation) and the phrase 'full compliance evaluation' suggests it is more thorough than a lightweight health check. However, it does not explicitly name alternatives or state when not to use it, so usage guidance remains implicit rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mcp_server_health_checkA
Read-onlyIdempotent
Inspect

Generate a health check report for an MCP server's tool manifest. Validates tool definitions, schema quality, naming conventions, and documentation completeness. Paste the server manifest JSON to audit.

ParametersJSON Schema
NameRequiredDescriptionDefault
strictNoEnable strict mode: also check for optional best practices (examples, default values, descriptions > 20 chars)
manifestYesMCP server manifest JSON (the response from GET /mcp or tools/list)

Output Schema

ParametersJSON Schema
NameRequiredDescription
statsNo
totalNo
checksNo
failedNo
passedNo
verdictNo
toolIssuesNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds value beyond annotations by detailing what the report validates (tool definitions, schema quality, naming conventions, documentation completeness), giving insight into the tool's functionality rather than just its side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three concise sentences: purpose, validation scope, and usage instruction. No redundant phrasing or unnecessary details, earning a top score for structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a strong output schema present and comprehensive annotations, the description sufficiently conveys the tool's purpose and usage. It omits mention of the `strict` parameter, but the schema covers that. The description is complete for a read-only report generator, though a brief note on output format could make it slightly richer.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for both parameters, so the schema already defines their meaning. The description only reinforces that `manifest` is the server manifest JSON ('Paste the server manifest JSON'), and does not add extra semantics for `strict`. Therefore a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Generate') and a distinct resource ('health check report for an MCP server's tool manifest'), enumerating the validation dimensions (tool definitions, schema quality, naming conventions, documentation completeness). This clearly distinguishes it from sibling tools like mcp_schema_lint or mcp_server_evaluate, which focus on other aspects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear usage context by instructing the user to 'Paste the server manifest JSON to audit', implying when to use it (i.e., to audit manifest health). However, it does not explicitly mention alternatives or when not to use the tool, so it stops short of full exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

merge_jsonA
Read-onlyIdempotent
Inspect

Deep merge two JSON objects. Supports three array strategies: replace (default), concat, or unique (dedup concat). Nested objects are recursively merged — override takes precedence for primitives.

ParametersJSON Schema
NameRequiredDescriptionDefault
baseYesThe base JSON object (merged into) — a JSON string, or the object itself.
overrideYesThe override JSON object (takes precedence) — a JSON string, or the object itself.
array_strategyNoArray merge strategy: replace (default), concat, or unique

Output Schema

ParametersJSON Schema
NameRequiredDescription
mergedNo
new_keysNo
total_keysNo
overridden_keysNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark it as read-only (readOnlyHint: true) and idempotent (idempotentHint: true), so the description's main behavioral contribution is the array strategy detail and recursive merging behavior. This adds valuable context beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. First sentence states purpose and array strategies. Second sentence explains nested object behavior and priority. Every sentence provides essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists (no need to describe return values) and schema covers all parameters, the description's focus on merge behavior and array strategies makes it fairly complete. Could mention that default strategy is replace, which is implied by 'replace (default)'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds high-level context (recursive merge, override precedence for primitives) but doesn't elaborate on parameter format or edge cases beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Deep merge two JSON objects', specifying the verb (merge), resource (JSON objects), and mode (deep). It distinguishes from siblings like json_diff or flatten_json by focusing specifically on recursive merge with array strategy options.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for merging JSON objects with control over array handling, but does not explicitly state when to prefer this over sibling tools like json_diff or transform_json_array. No guidance on prerequisites (e.g., valid JSON structure) or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

metamorphic_checkA
Read-onlyIdempotent
Inspect

Reference-free stability primitive: instead of comparing an answer to a ground truth, it checks that an assistant's answer stays INVARIANT when the QUESTION is transformed (typo, casing, paraphrase, reordering, translation). Catches the failure class no reference answer can expose — an assistant that handles one phrasing well and a trivial variant of it badly. You bring the outputs (no model is called), so it is deterministic and free in tfidf mode. Relations: case (θ .95), typo (.90), paraphrase (.80), reorder (.80), translation (.75, embeddings only), specialization (.60, ADVISORY — directional, never gated). Returns PASS / FAIL / INVALID, where INVALID means the BASE answer was a refusal or too short so invariance was never measurable — an assistant that refuses every variant would otherwise score a perfect 1.0. Use run_semantic_tests alongside it: invariance without a correctness floor is a green light for a broken assistant.

ParametersJSON Schema
NameRequiredDescriptionDefault
baseYesThe reference run: the original question and the answer your system produced for it.
modeNotfidf (default): free, lexical, deterministic — but a genuine paraphrase rarely reaches 0.80, so gate on case/typo and treat paraphrase as a trend. embeddings: OpenAI text-embedding-3-small, true semantic similarity, requires api_key. translation requires this mode.
api_keyNoOpenAI API key — required only when mode is embeddings.
variantsYesAnswers produced for transformed versions of the same question, each tagged with the relation that was applied.
thresholdsNoPer-relation threshold overrides. Calibrate on your own corpus before gating — the defaults are starting points, not measurements.
require_allNoIf true (default), every gated variant must pass. KEEP THE DEFAULT for any run you gate on. Setting it false is not a tolerance dial but an off switch: relations have asymmetric pass rates (a typo variant usually scores ~1.0 because the answer really is identical), so one trivial row is enough to hold the whole run at PASS while a paraphrase fails. When that happens the result carries an explicit warning naming the failed rows.
baseline_guardNoCorrectness floor applied to the BASE answer before anything is scored. Failing it returns INVALID, not FAIL.

Output Schema

ParametersJSON Schema
NameRequiredDescription
modeNo
summaryNo
verdictNo
weakestNo
baselineNo
variantsNo
warningsNo
thresholdsNo

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description adds substantial behavioral context beyond that: deterministic and free in tfidf mode, no model call, PASS/FAIL/INVALID semantics, INVALID meaning base refusal/too short, relation-specific thresholds, and the require_all off-switch warning. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense and long, but every sentence carries useful information for a complex 7-parameter tool. It is front-loaded with the core concept and then covers thresholds, return values, and guardrails. Slightly less structured than ideal, but not bloated or redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (nested objects, enums, output schema, 7 params), the description is remarkably complete: it explains return values, mode requirements, relation thresholds, the require_all warning, baseline_guard behavior, and the companion tool. It leaves no major behavioral gap for an agent to discover at runtime.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds meaning well beyond the schema: it explains mode differences (tfidf vs embeddings), relation thresholds and which are advisory, the require_all 'off switch' behavior with asymmetric pass rates, and the baseline_guard first-200-chars refusal matching. This materially helps an agent choose and set parameters correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Reference-free stability primitive' and immediately states the specific behavior: 'checks that an assistant's answer stays INVARIANT when the QUESTION is transformed.' This clearly distinguishes it from ground-truth comparison tools and from sibling tools like run_semantic_tests. The verb+resource+scope is precise and memorable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly frames when to use it ('instead of comparing an answer to a ground truth'), what the caller must bring ('You bring the outputs (no model is called)'), and names the companion tool: 'Use run_semantic_tests alongside it: invariance without a correctness floor is a green light for a broken assistant.' This gives clear context and an explicit alternative/complement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

minify_jsA
Read-onlyIdempotent
Inspect

Minify a JavaScript snippet, function, class, or module up to 50 KB using Terser. Returns minified code and byte savings. Use when embedding scripts in HTML templates, report payloads, or injecting inline code programmatically.

ParametersJSON Schema
NameRequiredDescriptionDefault
codeYesJavaScript code to minify (max 50kb)

Output Schema

ParametersJSON Schema
NameRequiredDescription
minifiedNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds the 50 KB limit, the Terser implementation, and the return value (minified code and byte savings), which are useful behavioral details beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: the first states what it does and returns, the second gives use cases. Every word earns its place, no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter utility with output schema present, the description covers purpose, constraints, and output. It provides enough context for the agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers 100% of the single parameter 'code' with a description. The tool description adds that it accepts a snippet, function, class, or module, enriching the semantic understanding of what can be passed, going beyond the schema's basic 'JavaScript code' label.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action (minify), the resource (JavaScript snippet/function/class/module), and the tool used (Terser). It also mentions the output (minified code and byte savings), making it distinct from sibling tools like base64_encode or count_tokens.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit use cases: embedding scripts in HTML templates, report payloads, or injecting inline code. It does not mention when not to use or alternatives, but the context is clear enough for an agent to decide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mock_from_schemaA
Read-only
Inspect

Generate realistic mock data from a JSON Schema. Supports all common types (string, number, integer, boolean, array, object, null), format hints (email, date, date-time, uri, uuid), enum, const, and nested schemas. Perfect for testing MCP tools with realistic data.

ParametersJSON Schema
NameRequiredDescriptionDefault
seedNoOptional seed string for deterministic output (uses first char codes)
countNoNumber of mock objects to generate (default: 1, max: 20)
schemaYesThe JSON Schema to generate from — a JSON string, or the schema object itself.

Output Schema

ParametersJSON Schema
NameRequiredDescription
countNo
resultsNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true, idempotentHint=false, destructiveHint=false, which already convey safety. The description adds behavioral detail: it supports common types, format hints, enum/const, and nested schemas. It does not contradict annotations. It could mention that the output is deterministic when a seed is provided, but that is partially covered by the schema description of 'seed'. Since annotations are already present, the description adds enough value about what the tool can process.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, both substantive. The first sentence states the core purpose, the second lists capabilities. Every sentence earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple input schema (3 params, 1 required), good annotation coverage, and presence of an output schema, the description is sufficiently complete. It explains what the tool does and its capabilities. A perfect score would require mention of output behavior (e.g., that it returns an array of mock objects), but the output schema likely covers that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter has a description in the input schema. The description adds context about what types and features are supported, beyond the individual parameter descriptions. The 'seed' and 'count' parameters are not further detailed, but the schema already covers them adequately. The description does not need to restate the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool generates realistic mock data from a JSON Schema, and enumerates supported features. There is no ambiguity about its purpose. However, the sibling list contains many other JSON-related utilities (json_schema_generate, json_schema_validate, etc.), and the description does not distinguish this tool as specifically for mock data generation vs. other schema operations like validation or generation of the schema itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this tool is for generating test data for MCP tools, which is a valid use case. However, it provides no guidance on when not to use it (e.g., when real data is needed, or when a specific format is required that mock data cannot satisfy). It also does not mention alternatives from the sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

model_infoA
Read-onlyIdempotent
Inspect

Get detailed specs for an AI model: context window, pricing per 1K tokens, knowledge cutoff, provider, multimodal support, reasoning capabilities, and feature list. Covers 30+ models from OpenAI, Anthropic, Google, DeepSeek, Meta, Mistral, Cohere, xAI.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesModel name (e.g. "gpt-4o", "claude-3.5-sonnet", "gemini-2.5-pro")

Output Schema

ParametersJSON Schema
NameRequiredDescription
modelNo
pricing_per_1kNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior. The description adds useful context about the specific data fields returned and the model coverage (30+ models, named providers), going beyond the annotation baseline.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, then detailed field list and coverage. No wasted words; every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter lookup tool with a documented output schema and strong annotations, the description fully covers what the tool returns and its scope. No significant gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a clear parameter description and examples. The tool description does not add significant meaning beyond the schema, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it retrieves detailed specs for an AI model, listing specific attributes (context window, pricing, knowledge cutoff, etc.) and provider coverage. This distinguishes it from siblings like list_llm_models or compare_models with a specific verb+resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use for single-model detail lookups, which is clear context. However, it does not explicitly mention alternatives or when not to use it, so it falls just short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

multimodal_eval_guideA
Read-onlyIdempotent
Inspect

Unified tool for multimodal AI evaluation: set action=guide for reference thresholds/interpretation (CLIP, FID, VQA), or set action=clip_score / fid_score / vqa_accuracy / pipeline to compute real metrics via HuggingFace Inference API and VLM BYOK calls. One tool for both reference and computation.

ParametersJSON Schema
NameRequiredDescriptionDefault
fidNo[pipeline] {real_images, generated_images} for FID.
vqaNo[pipeline] VQA config object (same inputs as vqa_accuracy).
clipNo[pipeline] {image_url, text} for CLIP.
textNo[clip_score only] Text description to compare against the image.
modelNo[vqa_accuracy] VLM model ID (default: gpt-4o).
scoreNo[guide only] Optional score value to interpret.
actionNoguide (default) = reference thresholds/interpretation. clip_score/fid_score/vqa_accuracy = compute that metric. pipeline = run all three.
metricNo[guide only] Metric to explain.
api_keyNo[vqa_accuracy] Your API key for the provider (BYOK).
image_urlNo[clip_score/vqa_accuracy] Public URL of the image.
test_casesNo[vqa_accuracy] Array of {question, accepted_answers} objects.
real_imagesNo[fid_score] Array of real image URLs.
image_base64No[clip_score/vqa_accuracy] Base64-encoded image data.
system_promptNo[vqa_accuracy] Optional system prompt.
image_mime_typeNo[clip_score/vqa_accuracy] MIME type for base64 image.
generated_imagesNo[fid_score] Array of generated image URLs.

Output Schema

ParametersJSON Schema
NameRequiredDescription
errorsNo
metricsNo
resultsNo
web_toolNo
best_practicesNo
comparison_tableNo
score_interpretationNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

It discloses that computation happens 'via HuggingFace Inference API and VLM BYOK calls', adding context about external dependencies and authentication beyond the readOnly/idempotent annotations. It also clarifies that guide is for reference only. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, key information front-loaded, and every clause adds value. It communicates the unified nature, action modes, reference vs. compute distinction, and external API dependencies with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the high schema coverage, output schema, and detailed annotation context, the description is sufficient at a high level. It could have elaborated on pipeline behavior or external API prerequisites, but those details are covered by the schema and descriptions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with each parameter tagged by action mode, so the description adds no detailed parameter semantics beyond the schema. It only restates action modes that the schema already enumerates.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description opens with 'Unified tool for multimodal AI evaluation', names specific metrics (CLIP, FID, VQA), and explicitly lists actions 'guide', 'clip_score', 'fid_score', 'vqa_accuracy', 'pipeline'. This clearly identifies the tool's function and distinguishes it from generic utility siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit internal guidance: set action=guide for reference thresholds, or set action to metric actions for computation, and states 'One tool for both reference and computation.' However, it does not mention sibling alternatives or state when not to use it, so explicit exclusions are missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

needle_haystack_generateA
Read-only
Inspect

Generate a "needle in a haystack" test: embeds a target fact into a large block of filler text at a specified position. Use this to test LLM context window retrieval accuracy. Returns the full haystack, the question to ask, and metadata. No API key needed.

ParametersJSON Schema
NameRequiredDescriptionDefault
needleYesThe fact to hide (e.g. "The secret code is ALPHA-42")
tokensNoTarget haystack size in tokens (default: 5000, max: 100000)
positionNoWhere to insert the needle: "start", "middle", "end", "random" (default: "middle")middle
questionYesThe question to ask the LLM (e.g. "What is the secret code?")

Output Schema

ParametersJSON Schema
NameRequiredDescription
needleNo
haystackNo
positionNo
questionNo
insert_blockNo
total_blocksNo
estimated_tokensNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds operational detail beyond the readOnlyHint=true annotation: it notes that no API key is needed and describes the return payload (full haystack, question, metadata). It also clarifies the generation behavior (embedding a target fact at a specified position). No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences cover purpose, behavior, use case, and operational requirements. No filler or redundant phrases.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers what the tool does, when to use it, what it returns, and auth requirements. With an output schema and full parameter schema present, there are no significant gaps in context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides full descriptions for all four parameters (100% coverage), including defaults and enum options. The description adds little parameter-specific detail beyond implying position and token size, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool generates a needle-in-a-haystack test by embedding a target fact into filler text, and explicitly identifies the purpose (testing LLM context window retrieval). This distinguishes it from sibling text-generation tools like lorem_ipsum or few_shot_formatter.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly recommends the tool for testing LLM context window retrieval accuracy, giving a clear context. It does not mention exclusions or alternative tools, so it stops short of full usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

normalize_vectorA
Read-onlyIdempotent
Inspect

L2-normalize a float vector (produce a unit vector with norm=1). Required by many vector DBs (Pinecone, Qdrant cosine). Supports batch normalization of up to 1000 vectors.

ParametersJSON Schema
NameRequiredDescriptionDefault
batchNoBatch of vectors to normalize (overrides vector)
vectorNoSingle vector to normalize

Output Schema

ParametersJSON Schema
NameRequiredDescription
modeNo
normNo
countNo
indexNo
vectorNo
resultsNo
dimensionNo
norm_afterNo
normalizedNo
norm_beforeNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false. The description adds valuable context about the output (unit vector) and batch limits, without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, followed by context and batch support. Every sentence earns its place with no waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool, the description covers the operation, use case, batch limit, and expected output. The output schema handles return details, so no gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds the batch normalization limit and confirms the override relationship between batch and vector, enhancing beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'L2-normalize a float vector (produce a unit vector with norm=1)', using a specific verb and resource. It distinguishes itself from vector-related siblings like vector_similarity and vector_quantize by focusing on normalization.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear context for when to use the tool ('Required by many vector DBs (Pinecone, Qdrant cosine)') and mentions batch normalization up to 1000 vectors. It lacks explicit exclusions or alternatives, but the use cases are well implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

normalize_whitespaceA
Read-onlyIdempotent
Inspect

Normalize whitespace: trim trailing spaces, collapse blank lines, normalize line endings (LF/CRLF), convert tabs to spaces. Useful for cleaning code, configs, and text before processing.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to normalize
trim_fileNoTrim leading/trailing blank lines (default: true)
trim_linesNoTrim trailing whitespace from each line (default: true)
line_endingNo"lf" (default), "crlf", or "cr"
tab_to_spacesNoConvert tabs to N spaces (omit to keep tabs)
collapse_blanksNoCollapse runs of blank lines down to max_blank_lines (default: true)
max_blank_linesNoBlank lines to keep when collapsing, 0-10 (default: 2)

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultNo
line_endingNo
original_lengthNo
normalized_lengthNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, and the description adds concrete transformation behavior. However, it states 'convert tabs to spaces' as a flat fact while the schema shows tab_to_spaces is optional and defaults to keeping tabs. This could mislead an agent about the default behavior, creating a transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with dense, front-loaded information. The first sentence enumerates all operations in a compact list; the second gives a practical use case. There is no filler, restating of the tool name, or redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich annotations, 100% parameter schema coverage, and presence of an output schema, the description covers the tool's main behaviors and usage contexts sufficiently. The only gap is the tab conversion default ambiguity, which prevents a perfect completeness score.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description's transformation list paraphrases parameters (trim_lines, collapse_blanks, line_ending, tab_to_spaces) but provides no additional default, format, or range details beyond what the schema already documents. It adds marginal semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear verb and resource, 'Normalize whitespace,' followed by a precise enumeration of transformations ('trim trailing spaces, collapse blank lines, normalize line endings (LF/CRLF), convert tabs to spaces'). This distinguishes it from sibling tools like sort_lines or format_json by specifying exactly what whitespace normalization does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence offers clear usage context: 'Useful for cleaning code, configs, and text before processing.' It tells the agent when to use it, but does not explicitly name alternatives or exclusion cases, so it falls short of the 5-level despite being a clear and practical guideline.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

number_base_convertA
Read-onlyIdempotent
Inspect

Convert numbers between bases: decimal, binary, octal, hexadecimal, or any base 2–36. Auto-detects 0x, 0b, 0o prefixes.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesNumber to convert (e.g., "255", "0xFF", "0b1010", "0o77")
to_baseNoTarget base 2–36 (omit to get all common bases)
from_baseNoSource base 2–36 (auto-detects prefix if omitted)

Output Schema

ParametersJSON Schema
NameRequiredDescription
octalNo
binaryNo
resultNo
decimalNo
to_baseNo
from_baseNo
hexadecimalNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds valuable behavioral details beyond annotations, such as auto-detecting 0x, 0b, and 0o prefixes. It does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one concise sentence, front-loaded with the main purpose and immediately followed by supported bases and prefix auto-detection. Every word adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simple nature, the presence of a rich schema, and an output schema, the description sufficiently covers the tool's core behavior. It could mention handling of invalid input, but this is not critical for a pure conversion utility.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and includes descriptions and examples for all parameters. The description adds no additional parameter-level detail, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool converts numbers between bases with specific supported bases (decimal, binary, octal, hexadecimal, any base 2–36), using a specific verb and resource. It also distinguishes itself from sibling conversion tools like base64_encode/decode by specifying 'numbers' and 'base 2–36'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes the intended use clear: number base conversion. It does not explicitly name alternatives or exclusions, but the phrase 'between bases' and the listed base range provide sufficient context to differentiate from other conversion tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

openapi_validateA
Read-onlyIdempotent
Inspect

Validate the structure of an OpenAPI 3.x specification (JSON or YAML). Checks required top-level fields (openapi, info.title, info.version, paths), validates each operation (responses, operationId uniqueness), detects undeclared $ref components, and flags missing 2xx responses. Returns a PASS/FAIL verdict, a 0–100 compliance score, and a list of errors and warnings with JSON-pointer locations. Use before publishing an API spec or generating SDK code.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesThe OpenAPI 3.x spec — a JSON string, a YAML string, or the already-parsed spec object.

Output Schema

ParametersJSON Schema
NameRequiredDescription
scoreNo
statsNo
errorsNo
verdictNo
warningsNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint=true, idempotentHint=true, destructiveHint=false) already declare safe, read-only behavior. The description adds rich behavioral details: specific checks performed (required fields, operationId uniqueness, missing 2xx responses), return structure (PASS/FAIL verdict, compliance score, errors/warnings with JSON-pointer locations). No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (about 4 sentences) and front-loaded with the purpose. Every sentence adds value. It could be slightly more structured (e.g., bullet points for checks) but is not verbose. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool is moderately complex and has an output schema (which covers return values), the description sufficiently covers input format, validation scope, and output summary. It does not need to detail return types. The description is complete for an agent to understand what the tool does and when to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter 'input', which already explains it accepts a JSON string, YAML string, or parsed object. The description does not add extra parameter semantics beyond what the schema provides. Baseline 3 is appropriate as the schema carries the load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'Validate the structure of an OpenAPI 3.x specification' with a detailed list of checks (required fields, operations, $ref, responses). This clearly distinguishes it from sibling tools like json_schema_validate or mcp_schema_lint, as it is specific to OpenAPI specs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description ends with 'Use before publishing an API spec or generating SDK code', providing clear when-to-use guidance. It does not explicitly mention when not to use or compare to alternatives, but the context makes it obvious that this tool is for OpenAPI 3.x validation. A minor omission: it doesn't caution against use for OpenAPI 2.x or other formats.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

optimize_prompt_tokensA
Read-onlyIdempotent
Inspect

Compress an LLM prompt by removing filler words, verbose phrases, duplicate sentences, and unnecessary whitespace. Returns optimized text with token savings breakdown. 100% deterministic, no API key needed.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYesThe prompt text to optimize
optionsNoToggle optimization steps (all true by default)

Output Schema

ParametersJSON Schema
NameRequiredDescription
stepsNo
optimizedNo
tokens_afterNo
tokens_savedNo
percent_savedNo
tokens_beforeNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a safe, idempotent, read-only operation. The description adds valuable context by guaranteeing deterministic output and noting no API key is required, which helps agents assess reliability and dependencies. It also discloses the return format (optimized text with token savings breakdown), enhancing behavioral understanding.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded with the primary action. Every sentence adds relevant detail, including deterministic behavior and the absence of an API key requirement. There is no redundant or unnecessary text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema, comprehensive annotations, and the tool's straightforward nature, the description covers all essential context for an agent to use it correctly. It could optionally mention alternative tools, but the description is otherwise complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides full documentation for both parameters (text and options), including defaults and descriptions. The tool description does not add significant parameter-level detail beyond what the schema already covers, but since schema coverage is 100%, the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description accurately states the tool's purpose with a specific verb ('Compress') and resource ('LLM prompt'), and clearly lists the compression techniques (removing filler words, verbose phrases, duplicate sentences, whitespace). It implicitly distinguishes from token counting or truncation tools, but does not explicitly name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by stating the function ('Compress an LLM prompt'), but provides no explicit guidance on when to choose this tool over related siblings like count_tokens or truncate_to_tokens. There are no exclusions or alternative recommendations, so the usage context is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

parse_csvA
Read-onlyIdempotent
Inspect

Parse a CSV string into a JSON array of objects (or raw arrays). Full RFC 4180: quoted fields may contain the delimiter, embedded newlines (the Excel/Sheets multi-line cell), and doubled quotes. Custom delimiters supported. An unterminated quote is rejected with its position rather than parsed into corrupted rows. Use when processing spreadsheet exports, data imports, or structured text pipelines where the source is CSV. Supports up to 200 KB.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesCSV content to parse
headerNoTreat the first row as headers (default: true)
delimiterNoField delimiter character (default: ",")

Output Schema

ParametersJSON Schema
NameRequiredDescription
rowsNo
columnsNo
headersNo
row_countNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds beyond annotations: RFC 4180 compliance, handling of quoted fields with delimiters/newlines/doubled quotes, rejection of unterminated quotes with position, and 200KB size limit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Concise and well-structured: covers purpose, capabilities, error handling, usage, and limit in a few sentences without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Provides a complete picture: input, output, error behavior, use cases, and constraint. No critical information is missing for a parser tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema already describes parameters; description adds context about custom delimiters and output format but does not significantly enhance parameter-specific understanding beyond what schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the function: parses CSV string into JSON array. Distinguishes from other tools by specifying output format (objects or raw arrays) and custom delimiter support.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit use cases: spreadsheet exports, data imports, structured text pipelines where source is CSV. Implicitly contrasts with other conversion tools, though alternatives are not named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

parse_http_headersA
Read-onlyIdempotent
Inspect

Parse a raw HTTP headers block into a structured JSON object. Detects multi-value headers, masks Authorization values, and optionally audits for missing security headers (HSTS, CSP, X-Frame-Options, etc.).

ParametersJSON Schema
NameRequiredDescriptionDefault
headersYesRaw HTTP headers (one "Name: Value" per line)
analyze_securityNoAudit for missing security headers (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
parsedNo
securityNo
header_countNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true. The description supplements this with valuable behavioral details: 'Detects multi-value headers, masks Authorization values, and optionally audits for missing security headers.' This adds context about data handling without contradicting the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the main verb action, and every clause adds substantive information. No wasted words or redundancy with schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity, comprehensive annotations, and presence of an output schema, the description covers purpose, key behaviors, and optional features. It does not need to describe return values because the output schema exists. The description is fully adequate for an agent to select and use the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description enriches the 'headers' parameter by explaining multi-value detection and Authorization masking, and adds security header examples (HSTS, CSP, X-Frame-Options) that clarify the 'analyze_security' parameter. This goes beyond the schema's static descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Parse a raw HTTP headers block into a structured JSON object,' which is a specific verb+resource+output. It distinguishes itself from siblings like parse_csv and security_headers_check by mentioning multi-value detection, Authorization masking, and optional security auditing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the use case: when you have a raw HTTP headers block and want structured JSON. It clearly notes the optional security audit feature, which hints at when to use this vs a dedicated security checker, but does not explicitly name alternatives or exclude cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

post_jira_commentAInspect

Post the output of jira_to_test_suite as a formatted comment on the source Jira ticket. Converts Gherkin, E2E steps, API tests, and ambiguities into Atlassian Document Format (ADF). STATEFUL — creates a comment on the issue.

ParametersJSON Schema
NameRequiredDescriptionDefault
issue_keyYesJira issue key, e.g. "PROJ-123"
jira_emailYesAtlassian account email
jira_tokenYesAtlassian API token
test_suiteYesThe test_suite object from jira_to_test_suite result
jira_base_urlYesAtlassian base URL

Output Schema

ParametersJSON Schema
NameRequiredDescription
successNo
comment_idNo
comment_urlNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description explicitly discloses the stateful nature of the operation ('STATEFUL — creates a comment on the issue') and explains the conversion to ADF. Annotations already indicate non-read-only and non-idempotent, so the description adds useful context about the side effect without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the primary purpose, the second adds key details about conversion and side effects. Every sentence earns its place with no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (5 parameters, nested test_suite object, output schema exists), the description is sufficiently complete. It explains the tool's role, side effects, and content transformation. It doesn't cover error scenarios, but that is not expected given the output schema and annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value by linking the test_suite parameter to the output of jira_to_test_suite and by mentioning the content (Gherkin, E2E steps, API tests, ambiguities) that will be converted, which helps the agent understand what to pass.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: posting the output of jira_to_test_suite as a formatted comment on a Jira ticket. It specifies the resource (output of jira_to_test_suite), verb (post), and target (source Jira ticket), distinguishing it from siblings like create_confluence_page or fetch_jira_issue.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a clear workflow: use this tool after jira_to_test_suite to post its output. It provides context on the intended scenario, though it does not explicitly state exclusions or alternatives, such as not using it for generic Jira comments.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pr_gatekeeperA
Read-onlyIdempotent
Inspect

Compound quality gate for pull requests. Runs three sequential checks: (1) secret detection — scans diff for API keys, tokens, passwords matching 16 regex patterns; (2) bug analysis — heuristic scan for eval(), innerHTML, empty catch, console.log, TODO/FIXME; (3) commit message linting against Conventional Commits spec. Returns gate verdict (PASS/WARN/BLOCK), blockers, and actionable warnings. Use before merging any code change.

ParametersJSON Schema
NameRequiredDescriptionDefault
diffYesUnified git diff (output of `git diff HEAD`)
contextNoOptional: PR title or description for richer bug analysis
commit_messageYesThe commit message to lint (e.g. "feat(auth): add OAuth2 login")

Output Schema

ParametersJSON Schema
NameRequiredDescription
flagsNo
scoreNo
checksNo
verdictNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so safety is covered. The description adds rich behavioral detail: 'three sequential checks', specific regex patterns (16), heuristic scan targets (eval(), innerHTML, etc.), Conventional Commits spec, and output types (PASS/WARN/BLOCK, blockers, warnings). This goes well beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with 'Compound quality gate for pull requests', then a structured list of checks and output. No filler, each sentence conveys essential information. Excellent density.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex (three sequential checks) but the description covers what it does, the checks, the output format, and when to use it. Output schema exists, so return values don't need elaboration. Given the annotations and schema, this is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% coverage with clear descriptions for all three parameters (diff, context, commit_message). The description adds little beyond mapping diff to secret/bug checks and commit_message to linting—context is not explicitly mentioned. Baseline 3 is appropriate since the schema already documents parameters well.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool is a 'compound quality gate for pull requests' and enumerates three specific checks (secret detection, bug analysis, commit message linting). This distinguishes it from sibling tools like analyze_diff_bugs, detect_secrets, and lint_commit_message by being an all-in-one gate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states 'Use before merging any code change', which gives a clear trigger for when to invoke the tool. It doesn't mention alternatives or when not to use it, but the compound nature and the presence of sibling tools imply it's the comprehensive option. Minor gap in not naming excluded alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

prompt_injection_scanA
Read-onlyIdempotent
Inspect

Scan user input or prompts for common prompt injection patterns. Detects system prompt overrides, jailbreak attempts, role manipulation, encoding tricks, delimiter attacks (chat-template tags <|im_start|>/[INST]/<<SYS>> AND fake role headers imitating markdown or chat separators: "### System:", "--- SYSTEM ---", "---BEGIN SYSTEM OVERRIDE---", "--- SYSTEM:"), template/interpolation injection ({{...}}, ${...}), and context-exfiltration attempts ("repeat everything above"). A match inside quoted or fenced text (documentation citing a payload) is reported one severity level lower and marked quoted — never suppressed, since an LLM reading the document as data can still follow a quoted instruction. A quote preceded by a live imperative ("output the following: ...") keeps its full severity.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesThe user input or prompt to scan for injection patterns
sensitivityNoDetection sensitivity (default: medium)

Output Schema

ParametersJSON Schema
NameRequiredDescription
notesNo
detectionsNo
risk_levelNo
sensitivityNo
input_lengthNo
detections_countNo
quoted_detectionsNo
injection_detectedNo

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnly, idempotent, non-destructive), the description discloses nuanced behavior: how quoted/fenced content is handled (severity reduction, never suppressed) and that live imperatives keep full severity, providing extra transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is detailed and informative but somewhat lengthy with examples. It is well-structured as a single paragraph explaining purpose and behavior, though it could be slightly more concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema is provided, but the description implies the output includes severity levels and detection results. It covers the essential context for using the tool, though it stops short of explicit return format details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides descriptions for both parameters ('input' and 'sensitivity'). The description does not add further parameter-level detail, but since schema coverage is 100%, baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: scanning user input for prompt injection patterns, and lists specific categories (system prompt overrides, jailbreak attempts, etc.). It is unambiguous and distinguishes the tool from typical text processing tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description indicates the tool is for security scanning of user-provided text, but does not explicitly state when not to use it or mention alternatives. However, the intent is clear enough for typical use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

prompt_template_fillA
Read-onlyIdempotent
Inspect

Fill a prompt template with variables. Supports {{variable}} syntax and {{#if key}}...{{/if}} conditional blocks. Returns the filled prompt and lists unfilled variables.

ParametersJSON Schema
NameRequiredDescriptionDefault
strictNoThrow error if any variable is not provided (default: false)
templateYesPrompt template with {{variable}} placeholders
variablesNoKey-value pairs to fill (e.g. {"name":"Alice","role":"engineer"})

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultNo
total_varsNo
filled_variablesNo
unfilled_variablesNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only and idempotent behavior. The description adds useful behavioral details beyond those annotations: support for conditional blocks, returning the filled prompt, and listing unfilled variables. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise, front-loaded sentences each contribute necessary information: the core action, supported syntax, and return behavior. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich schema, output schema, and annotations, the description fully covers the operation and its behavior. An agent can correctly invoke the tool without additional explicit details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters with descriptions for template, variables, and strict. The description does not add further parameter-level meaning, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Fill a prompt template with variables') and adds supported syntax and return behavior, making it clear and distinguishable from sibling prompt-related tools like few_shot_formatter or build_rag_prompt.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied by the description: use when you have a prompt template with variables. However, there is no explicit when-to-use guidance or comparison to alternative tools, so the description only partially addresses this dimension.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

prompt_test_suiteA
Read-onlyIdempotent
Inspect

Define a test suite for a prompt: provide the system prompt, user prompt, and expected output criteria. Returns a test plan with scored rubric — use this as input for manual or automated LLM evaluation.

ParametersJSON Schema
NameRequiredDescriptionDefault
max_tokensNoMax token budget for the test
temperatureNoTemperature to use
user_promptYesThe user prompt to send
check_safetyNoInclude safety/PII checks in the rubric
must_includeNoRequired content (comma-separated)
system_promptYesThe system prompt under test
expected_formatNoExpected output format
must_not_includeNoForbidden content (comma-separated)
expected_behaviorNoDescription of what the LLM should do (free text)
adversarial_promptsNoAuto-generate adversarial test variants (jailbreak, injection, edge cases)

Output Schema

ParametersJSON Schema
NameRequiredDescription
rubricNo
categoriesNo
total_testsNo
instructionsNo
test_suite_nameNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds that it returns a test plan with a scored rubric, which is useful but not extensive behavioral disclosure. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose and action, no wasted words. It efficiently states what the tool does and returns.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description orients the agent on the tool's role (test suite definition) and return value (test plan with rubric). Given the output schema and 100% parameter schema coverage, the description is sufficient for selection and invocation despite numerous parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description references 'system prompt, user prompt, and expected output criteria,' loosely mapping to several parameters but adds no deeper meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource ('Define a test suite for a prompt') and clearly distinguishes from siblings like run_semantic_tests by stating it produces an input for evaluation, not the evaluation itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'use this as input for manual or automated LLM evaluation' provides clear context for when to use the tool. It does not explicitly name alternatives or exclude other tools, but the intended workflow is evident.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rag_relevance_rankA
Read-onlyIdempotent
Inspect

Rank an array of text chunks by relevance to a query using TF-IDF scoring. Simulates retrieval ranking for RAG testing without needing embeddings or an API.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesThe user query
top_kNoReturn top K results (default: all)
chunksYesArray of text chunks to rank

Output Schema

ParametersJSON Schema
NameRequiredDescription
rankNo
indexNo
queryNo
scoreNo
resultsNo
returnedNo
total_chunksNo
chunk_previewNo
keyword_overlapNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, and non-destructive behavior. The description adds valuable context beyond annotations by disclosing the TF-IDF algorithm and the fact that it works without external resources, which is useful for understanding what the tool does under the hood.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences. The first sentence states the action and method; the second explains the purpose and avoids unneeded details. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity, full schema coverage, output schema presence, and rich annotations, the description provides enough context. It explains what the tool does, how it does it, and when it's useful, which is complete for a read-only utility.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all parameters are already described adequately in the schema. The description does not add extra parameter-level semantics, but the baseline of 3 applies because no additional meaning is needed beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool does: 'Rank an array of text chunks by relevance to a query using TF-IDF scoring.' It also distinguishes itself from sibling tools by specifying the algorithm and noting it 'Simulates retrieval ranking for RAG testing without needing embeddings or an API.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: it is for RAG testing scenarios where embeddings or an API are unavailable. It does not explicitly name alternative tools or provide exclusion criteria, but the use case is well implied and focused.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rate_toolAInspect

Give honest usage feedback on an IA-QA MCP tool. Provide a score (1-5) and a comment. Rate low (1-2) if the tool was wrong, irrelevant, or a poor fit; rate high (4-5) only if it genuinely solved your need. Ratings are aggregated on a public dashboard at /devtools/mcp-ratings. Skip rating routine successes — we want signal, not praise. Example: rate_tool({ tool_name: "format_json", score: 2, comment: "Tried to pretty-print a JSON5 file, it rejected trailing commas — not usable for my case." })

ParametersJSON Schema
NameRequiredDescriptionDefault
scoreYesRating from 1 (poor) to 5 (excellent)
commentNoStrongly encouraged — explain what you were trying to do and whether the tool got you there. Be specific about what was missing, wrong, or a poor fit. This is the most valuable part of the rating. Up to 2000 chars are stored; go over and the response says so (truncated: true) — send the remainder as a second call rather than assuming it landed.
tool_nameYesName of the MCP tool to rate (e.g. "format_json", "shield_analyze")

Output Schema

ParametersJSON Schema
NameRequiredDescription
okNo
scoreNo
commentNo
messageNo
rated_atNo
tool_nameNo
truncatedNo
stored_charsNo

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds context beyond annotations: ratings are aggregated on a public dashboard, comment truncation behavior is described, and the tool's write action is clear (no annotation contradiction).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single efficient paragraph that front-loads purpose, then moves through scoring guidelines, dashboard note, usage rule, and example—every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 3-param tool with output schema, the description covers what the tool does, when to use it, how to use parameters, behavioral quirks (truncation), and an example—no gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%; the description enriches each parameter with usage guidance (e.g., comment is 'the most valuable part', score scale explained, tool_name example) and provides a full invocation example.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ("Give") and resource ("usage feedback on an IA-QA MCP tool"), and clearly distinguishes this rating tool from the many sibling utilities by its unique role of collecting feedback.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit criteria for when to rate low (1-2) vs high (4-5), and a direct instruction to skip routine successes ('we want signal, not praise'), plus a concrete example showing how to call the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

redact_piiA
Read-onlyIdempotent
Inspect

Automatically detect and redact Personally Identifiable Information (PII) from text. Replaces emails, phone numbers, SSNs, credit cards, IP addresses, and JWT tokens with [REDACTED_TYPE] placeholders. Safe to use before logging or sending to an LLM.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to redact PII from
typesNoComma-separated types to redact (default: all). Options: email, phone, ssn, credit_card, ip_address, jwt
markerNoCustom replacement marker (default: "REDACTED"). Result: [REDACTED_EMAIL]

Output Schema

ParametersJSON Schema
NameRequiredDescription
cleanNo
pii_foundNo
replacementsNo
redacted_textNo
total_redactionsNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark it as read-only and idempotent, and the description adds behavioral detail: it replaces specific PII types with formatted [REDACTED_TYPE] placeholders. It also communicates safety for downstream operations, which supplements the annotations without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: action, replacement behavior, and recommended use case. No filler or redundant restatement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-input utility with a straightforward output, the description plus annotations and output schema fully cover operation. It explains what the tool does, what it redacts, and when to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with descriptions for input, types, and marker. The tool description merely lists the same entity types found in the schema and doesn't add new parameter-level semantics, so it meets the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb ('detect and redact') with a specific resource (PII from text) and enumerates the exact entity types covered. This distinguishes it from sibling text-analysis tools like detect_secrets or bias_detect, making the purpose immediately obvious.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly recommends use before logging or sending to an LLM, giving a concrete scenario for when to use the tool. It doesn't name alternatives or exclusions, but the context is clear enough for an agent to choose this over other text utilities.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

regex_testA
Read-onlyIdempotent
Inspect

Test a regular expression pattern against an input string and return all matches with their index positions and named capture groups. Use for validating user inputs, extracting structured data from text, or debugging regex patterns. Supports flags g, i, m, s, u, y. The match runs in an isolated thread with a 500 ms budget: a pattern that blows up (catastrophic backtracking, e.g. "(a+)+$") comes back as redos_detected:true — a real ReDoS verdict on your pattern — instead of hanging.

ParametersJSON Schema
NameRequiredDescriptionDefault
flagsNoRegex flags: g (global), i (case-insensitive), m (multiline), s (dotAll) — default: ""
inputYesThe string to test against (max 50 KB)
patternYesRegular expression pattern (without delimiters)

Output Schema

ParametersJSON Schema
NameRequiredDescription
noteNo
flagsNo
matchedNo
matchesNo
patternNo
verdictNo
elapsed_msNo
match_countNo
redos_detectedNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive hints. The description adds meaningful behavioral context beyond those: it runs in an isolated thread with a 500 ms budget, and catastrophic backtracking returns redos_detected:true instead of hanging. This provides a concrete safety guarantee and failure mode not encoded in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: the first states the purpose and output, the second gives usage context, and the third exposes the timeout/ReDoS behavior. Information is front-loaded and there is no redundant verbiage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description need not enumerate return fields. It covers purpose, applicable use cases, flag support, input size limits (via schema), and the critical safety/failure behavior. This is complete for a moderate-complexity regex testing tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for all three parameters, so the baseline is 3. The description adds value by listing the complete supported flag set (g, i, m, s, u, y) — the schema omits u and y — and by clarifying with an example that patterns use no delimiters. This enriches parameter understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource: 'Test a regular expression pattern against an input string and return all matches with their index positions and named capture groups.' This clearly differentiates the tool from sibling text-processing utilities and states its exact function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly names three use cases: 'validating user inputs, extracting structured data from text, or debugging regex patterns.' It provides clear context for when to use the tool, though it does not mention exclusions or alternative tools. Since no sibling tool directly competes with regex testing, this is adequate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rerank_evaluateA
Read-only
Inspect

Evaluate RAG retrieval quality: rank passages against a query and compute Precision@k / Recall@k plus a PASS/FAIL CI verdict from ground-truth relevance labels. Three modes, all keyless except the last. (1) BYO scores — give each passage the score your own reranker produced (Cohere, Jina, a self-hosted NIM, a cross-encoder): deterministic, offline, and it evaluates YOUR reranker rather than someone else's. This is the mode to gate CI on. (2) Default, no scores and no key — ranks with local BM25, a lexical keyword baseline: it answers "does a keyword floor already surface my relevant passages?", never "is my neural reranker good". (3) Live NVIDIA reranker — supply api_key for an NVIDIA account that still has reranking entitlement; NVIDIA retired its hosted reranking endpoints on 2026-05-18, so this one is for accounts that were grandfathered in.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesThe search query or question to rank against
top_kNok for Precision@k evaluation (default 3)
api_keyNoYour NVIDIA API key (BYOK), used only when no passage carries a score. Transits RAM for the single call, never stored.
passagesYesArray of passage objects to rank (min 2, max 20)
thresholdNoMinimum Precision@k to PASS (0-1, default 0.5)

Output Schema

ParametersJSON Schema
NameRequiredDescription
kNo
modeNo
modelNo
rankedNo
verdictNo
thresholdNo
latency_msNo
recall_at_kNo
precision_at_kNo
total_passagesNo
has_ground_truthNo

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite good annotations, the description adds substantial behavioral context: BYO mode is deterministic/offline, BM25 mode is a lexical baseline with no neural evaluation, the NVIDIA endpoint was retired on 2026-05-18, and the API key 'transits RAM for the single call, never stored.' This fully discloses side effects and state implications beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is structured as a numbered list with a front-loaded purpose. It is detailed but every sentence earns its place, including context like the NVIDIA retirement date and key-storage behavior. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has three distinct execution modes, CI-gating semantics, and a historical deprecation caveat, all fully explained. Output schema exists, so return-value documentation is not required here. The description is complete for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaningful semantic guidance: score must be present on every passage or none, api_key is used only when no passage carries a score, and higher scores mean more relevant. These details connect the parameters to mode selection in a way the schema alone does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Evaluate RAG retrieval quality: rank passages against a query and compute Precision@k / Recall@k plus a PASS/FAIL CI verdict.' It clearly differentiates three modes and is distinct from sibling tools like rag_relevance_rank or bm25_score by emphasizing ground-truth evaluation rather than mere ranking.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance for each mode: BYO scores is 'the mode to gate CI on', BM25 mode answers a specific keyword-floor question and 'never is my neural reranker good', and the live NVIDIA mode is restricted to grandfathered accounts with reranking entitlement. This is exemplary usage differentiation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

response_quality_scoreB
Read-onlyIdempotent
Inspect

Score an LLM response on multiple quality dimensions: relevance, completeness, clarity, conciseness, formatting. Returns a weighted 0-100 score with detailed breakdown.

ParametersJSON Schema
NameRequiredDescriptionDefault
questionYesThe original question/prompt
responseYesThe LLM response to score
max_lengthNoIdeal max character length (penalize if exceeded)
expected_keywordsNoKeywords that should appear in a good answer

Output Schema

ParametersJSON Schema
NameRequiredDescription
gradeNo
statsNo
breakdownNo
max_scoreNo
total_scoreNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds that scoring is 'weighted' and returns 'detailed breakdown', but offers no insight into scoring methodology, parameter influence, or edge cases. This meets the minimum bar given annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently conveys purpose and output. Every word contributes, with no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the existence of an output schema and full parameter documentation, the description sufficiently covers the core function. Gaps remain in usage context and behavioral specifics, but the tool is relatively simple and self-contained.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are already well-documented. The description does not add extra meaning to parameters, though its mention of 'quality dimensions' hints at scoring criteria. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as scoring an LLM response on specified quality dimensions (relevance, completeness, clarity, conciseness, formatting) and notes the output is a weighted 0-100 score. This distinguishes it from sibling tools like compare_responses, though it doesn't explicitly name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus similar scoring or evaluation tools. The description only states what the tool does, with no context for selection or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_eval_contractA
Read-only
Inspect

Parse a .ia-eval.yaml LLM test suite, call the specified LLM model for each scenario, run all configured scorers, and return a structured JSON report with per-scenario Pass/Fail verdicts and a Markdown summary. Use list_local_tests to discover available test files.

ParametersJSON Schema
NameRequiredDescriptionDefault
api_keysNoAPI keys to use for LLM generation (all optional — falls back to server env vars)
overridesNoOverride contract defaults
contract_pathNoAbsolute or relative path to a .ia-eval.yaml file (required unless inline_contract is provided)
inline_contractNoRaw contract object (alternative to contract_path). Must contain top-level "metadata" ({name, version, model?, provider?}), "expectations" ({min_score?}), and "scenarios" ([{id, input, ground_truth?}]) — scenarios alone are rejected. Use generate_eval_yaml to scaffold one.

Output Schema

ParametersJSON Schema
NameRequiredDescription
summaryNo
metadataNo
warningsNo
contract_pathNo
scenario_resultsNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is known. The description adds that the tool makes external LLM calls and generates a report, which aligns with openWorldHint. It doesn't disclose potential costs, rate limits, or auth requirements, but with annotation coverage, the added behavioral context is adequate but not extensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The first sentence front-loads the core purpose and workflow; the second provides a practical pointer to a sibling tool. Every part contributes to usability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (4 parameters, nested objects, output schema), the description covers the main workflow and points to list_local_tests for discovery. It doesn't dwell on edge cases, but the schema and output schema cover parameter constraints and return format. This is sufficient for a capable agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema already explains all parameters, including nested objects and fallback behavior for API keys. The description adds no parameter-level semantics, so a baseline of 3 is appropriate since the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear, specific action: parse a .ia-eval.yaml LLM test suite, call the LLM model per scenario, run scorers, and return a structured JSON report with per-scenario verdicts and a Markdown summary. It distinguishes itself from generic eval or test tools by naming the file format and the workflow, making its purpose unique among siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for running an existing .ia-eval.yaml test suite and explicitly directs the user to list_local_tests for discovering test files, providing useful workflow context. However, it doesn't explicitly exclude alternatives like run_semantic_tests or run_vlm_test_suite, so it stops short of full when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_pr_gate_pipelineA
Read-onlyIdempotent
Inspect

Review triage for a pull request. Takes a unified git diff (git diff HEAD) and returns: diff-lint findings with the lines that produced them, regression impact areas, a risk score 0–100 with the factors that built it (churn, files touched, sensitive paths, whether any test file changed, lint severities, impacted risk areas), generated test cases, and a PASS / CONDITIONAL / BLOCK recommendation. Advisory: the score measures properties of the diff, not the correctness of the change — it does not read the code semantically and does not replace a reviewer or a static analyser. See notAnalysed in the response.

ParametersJSON Schema
NameRequiredDescriptionDefault
contextNoOptional PR title or description for richer analysis
git_diffYesUnified git diff (output of `git diff HEAD` or copied from GitHub diff view)

Output Schema

ParametersJSON Schema
NameRequiredDescription
slaNo
highNo
topBugsNo
criticalNo
bugsFoundNo
riskLevelNo
riskScoreNo
disclaimerNo
impactAreasNo
inputFormatNo
notAnalysedNo
riskFactorsNo
changedFilesNo
severityLevelNo
testCasesGeneratedNo
mergeRecommendationNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds behavioral context: the risk score is based on diff properties, not semantic correctness, and it explicitly states what the tool does not do. This goes beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but well-structured: a one-sentence purpose, a list of outputs, and an advisory. Every sentence provides necessary information without padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool, the description covers inputs, outputs, risk factors, and limitations. It mentions the notAnalysed field, indicating awareness of what's not included. The presence of an output schema means it needn't detail return structure, and it doesn't.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for git_diff and context. The description repeats the git_diff input but adds no new parameter semantics beyond saying 'Takes a unified git diff'. The optional context parameter is not mentioned in the description. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Review triage for a pull request' and lists specific outputs (lint findings, regression areas, risk score, test cases, recommendation). This distinguishes it from siblings like pr_gatekeeper and analyze_diff_bugs by focusing on diff-property triage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The advisory explicitly scopes the tool: it measures diff properties, not correctness, and doesn't replace a reviewer or static analyser. This gives context on when not to use it, though it doesn't name alternative tools. 'See notAnalysed in the response' further hints at its limitations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_semantic_testsA
Read-onlyIdempotent
Inspect

Semantic assertion primitive: compare actual vs expected text pairs using cosine similarity + ROUGE-L. Two modes: tfidf (default, free, no API key) or embeddings (OpenAI text-embedding-3-small, BYOK, true semantic similarity). Returns per-case PASS/FAIL verdicts and an overall verdict. CI-ready: pipe the JSON verdict field to gate a build.

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNotfidf (default): fast, free, lexical. embeddings: OpenAI text-embedding-3-small, true semantic similarity, requires api_key.
casesYesArray of (actual, expected) pairs to evaluate.
api_keyNoOpenAI API key — required only when mode is embeddings.
thresholdsNoPass/fail thresholds (defaults: cosine 0.75, rouge_l 0.5).
require_allNoIf true (default), all cases must pass for overall PASS. If false, at least one case passing returns PASS.

Output Schema

ParametersJSON Schema
NameRequiredDescription
modeNo
totalNo
failedNo
passedNo
resultsNo
verdictNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description goes beyond annotations by explaining return values (verdicts) and the requirement for an API key in embeddings mode, adding valuable behavioral context without contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured, with three sentences that front-load the core purpose and then add mode details and CI relevance. Every sentence contributes useful information without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (5 params, nested objects, output schema), the description covers key aspects: modes, API key requirement, default thresholds, verdict output, and CI use case. The existing output schema handles return-value details, so the description is sufficiently complete for an agent to select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds some semantic context by mentioning 'tfidf (default)' and 'BYOK', but these are largely redundantly covered in the schema parameter descriptions. It does not meaningfully enhance parameter understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly defines the tool as a 'Semantic assertion primitive' that compares actual vs expected text pairs using cosine similarity and ROUGE-L. It distinguishes between tfidf and embeddings modes, and mentions returns of per-case and overall verdicts, which clearly differentiates it from generic similarity tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use each mode (tfidf is free/lexical, embeddings is true semantic with BYOK) and notes it is CI-ready. However, it does not explicitly name alternative sibling tools or state when not to use this tool, so it falls short of full exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_vlm_test_suiteA
Read-only
Inspect

Run a test suite against a Vision-Language Model (VLM) — send an image (URL or base64) + N test cases (each with a question + assertion) to GPT-4o, Claude 3.5, or Gemini. Returns per-case PASS/FAIL verdicts, a pass rate, an overall PASS/WARNING/FAIL verdict (customizable threshold), and latency stats. Assertion types: contains, not_contains, json_format, min_length, max_length, semantic_contains (TF-IDF cosine similarity ≥ 0.4). BYOK: requires your own API key for the target provider.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesVLM model to use.
api_keyYesAPI key for the model provider (OpenAI sk-, Anthropic sk-ant-, or Google AIzaSy...).
image_urlNoPublic URL of the image to evaluate (required unless image_base64 is provided).
thresholdNoPass rate threshold for overall verdict (default: 80, 0–100).
test_casesYesArray of test cases to run.
image_base64NoBase64-encoded image data (required unless image_url is provided).
system_promptNoOptional system prompt sent to the VLM.
image_mime_typeNoMIME type of the image if using image_base64 (default: image/jpeg).

Output Schema

ParametersJSON Schema
NameRequiredDescription
modelNo
totalNo
failedNo
passedNo
resultsNo
verdictNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so safety profile is covered. The description adds meaningful behavioral context: it requires the user's own API key (BYOK), returns latency stats, and supports a customizable threshold. It does not mention rate limits or costs, but given annotation coverage, the added detail is valuable and non-contradictory.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core purpose, and every sentence adds value: what it does, returns, assertion details, and BYOK requirement. No redundant or filler content; highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (8 params, output schema exists) and 100% schema coverage, the description fully covers the workflow: input (image + test cases), supported models, assertion types, and the BYOK requirement. The output schema handles return value details, so no gap exists.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for all 8 parameters, so the baseline is 3. The description repeats some parameter details (assertion types, image URL/base64, model names) but does not add meaning beyond the schema. It provides a concise summary but relies on the schema for full parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool's function: running a test suite against a VLM with image and test cases. It uses a specific verb+resource ('Run a test suite against a Vision-Language Model') and lists supported models. However, it does not explicitly differentiate from the sibling tool run_vlm_test_suite_batch, so it stops short of full sibling distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool — when you need to evaluate a VLM with custom test cases and assertions. It does not explicitly mention alternatives or exclusions, but the scenario is sufficiently defined to guide selection. No misleading guidance is present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_vlm_test_suite_batchA
Read-only
Inspect

Compare multiple VLMs on the same test suite in parallel — send an image (URL or base64) + N test cases to all models simultaneously. Returns per-model PASS/FAIL verdicts, pass rates, latency stats, and a comparison table. Assertion types: contains, not_contains, json_format, min_length, max_length, semantic_contains. BYOK: requires API keys for each provider.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelsYesArray of model IDs to compare (runs in parallel).
api_keysYesMap of model ID → API key. Example: { "gpt-4o": "sk-...", "claude-3-5-sonnet-20241022": "sk-ant-..." }
image_urlNoPublic URL of the image to evaluate (required unless image_base64 is provided).
thresholdNoPass rate threshold for overall verdict (default: 80, 0–100).
test_casesYesArray of test cases to run against every model.
image_base64NoBase64-encoded image data (required unless image_url is provided).
system_promptNoOptional system prompt sent to every VLM.
image_mime_typeNoMIME type of the image if using image_base64 (default: image/jpeg).

Output Schema

ParametersJSON Schema
NameRequiredDescription
suitesNo
verdictNo
total_failedNo
total_passedNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds valuable context beyond annotations: 'BYOK: requires API keys for each provider' and highlights parallel execution. It does not contradict annotations and provides extra operational detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, well-front-loaded with the core purpose first. Every sentence adds distinct value: what it does, what it returns, and key requirements (assertion types, API keys). No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 8 parameters, nested objects, and an output schema, the description covers the essential flow, output stats, and external dependency (API keys). It omits threshold defaults and more granular parameter details, but those are present in the schema. Overall adequate for an agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents each parameter. The description repeats some of this (image URL/base64, test cases, assertion types) but adds no new meaning beyond what's in the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb ('Compare') and clearly identifies the resource ('multiple VLMs on the same test suite') and mode ('in parallel'). It distinguishes itself from the sibling 'run_vlm_test_suite' (singular) by emphasizing multi-model batch comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clear context is given: send an image plus test cases to all models simultaneously. It implies the batch use case for multi-model comparison, and the sibling tool name differentiates it from the single-model variant. However, it does not explicitly state when NOT to use it or mention alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

sandbox_scenarioA
Read-onlyIdempotent
Inspect

Get a ready-made selector-drift test case with a known-correct answer, for testing this MCP server or an agent workflow end to end. Each scenario is a real DOM capture of a deliberately breakable app, taken before and after one specific UI change (a renamed label, two swapped buttons, a duplicated locator, an element moved behind a menu…), plus the verdict those two contracts MUST produce. Call with no arguments to list the scenarios; call with a scenario id to get "baseline" and "current" mappings. THE LOOP: pass baseline and current to diff_mappings, then compare its "verdict" and "counts" to this tool's "expected" — they must match exactly. A mismatch means this server's diff engine has drifted, not that your inputs are wrong. Deterministic and offline: the captures are committed fixtures, identical on every call. Try it live at https://www.ia-qa.com/devtools/sandbox

ParametersJSON Schema
NameRequiredDescriptionDefault
scenarioNoScenario id. Omit to list every available scenario with its expected verdict. Ids: no-change, swap-label, add-testid, duplicate-role-name, remove-element, insert-sibling, rename-label, counter-label, move-behind-menu, add-element
include_htmlNoInclude the generated HTML of the mutated page (default false). Only useful if you want to render or re-capture it yourself; the loop does not need it.

Output Schema

ParametersJSON Schema
NameRequiredDescription
htmlNo
pageNo
blurbNo
countNo
titleNo
currentNo
teachesNo
baselineNo
expectedNo
scenarioNo
mutationsNo
scenariosNo
how_to_run_the_loopNo

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive behavior, so the bar is lower, but the description goes well beyond that: it reveals outputs are deterministic committed fixtures, identical on every call, offline, and that a mismatch indicates server drift rather than user error. It also clarifies the structural shape of the response (baseline/current/expected) beyond what the output schema requires.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence earns its place: the purpose, the scenario content, the no-arg vs id calling convention, the integration loop, and the determinism guarantee are all packed in without redundancy. The critical workflow is highlighted with 'THE LOOP' and front-loaded before secondary details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool of this complexity, the description is complete: it explains what the tool returns, how to use it in combination with diff_mappings, the semantic meaning of each parameter, and the determinism/offline guarantees. The output schema and annotations cover the remaining structured details, leaving no critical gap for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, giving a baseline of 3, but the description adds meaningful guidance: it explains that omitting scenario lists all scenarios, and it adds extra context for include_html ('Only useful if you want to render or re-capture it yourself; the loop does not need it'), which helps the agent decide not to set it. This exceeds schema documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and object: "Get a ready-made selector-drift test case with a known-correct answer." It clearly identifies the resource (DOM captures of deliberately breakable apps with expected verdicts) and distinguishes it from siblings like diff_mappings by positioning itself as the fixture provider in a testing workflow.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage instructions: call with no arguments to list scenarios, or with a scenario id to get baseline/current mappings. It then gives a precise multi-tool workflow ('THE LOOP') telling the agent to pass baseline and current to diff_mappings and compare its verdict/counts to the expected result, which is exactly the kind of when-to-use guidance needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

score_geo_signalsA
Read-onlyIdempotent
Inspect

Analyze a webpage HTML (or full HTML) for GEO (Generative Engine Optimization) signals. Returns a score /60 with per-check results and improvement tips. GEO = optimizing pages for AI-powered search engines (ChatGPT Search, Perplexity, etc.).

ParametersJSON Schema
NameRequiredDescriptionDefault
head_htmlYesRaw HTML of the <head> section (or full page HTML) to analyze

Output Schema

ParametersJSON Schema
NameRequiredDescription
gradeNo
scoreNo
checksNo
passedNo
max_scoreNo
total_checksNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, non-destructive, and idempotent behavior. The description adds meaningful behavioral context beyond annotations by stating the output format ('score /60'), that it includes 'per-check results and improvement tips,' and that it accepts either head or full HTML. No contradictions exist between description and annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no fluff: the first states the action and input, the second explains the output format and defines the GEO acronym. Every sentence earns its place, and the structure is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a low-complexity tool with one parameter and an output schema, so the description sufficiently covers what, why, and expected output. It mentions the scoring scale, per-check results, tips, and input flexibility, leaving no critical gap for successful invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter is fully documented in the schema ('Raw HTML of the <head> section (or full page HTML) to analyze'), so the description need not repeat it. The tool description reinforces the input flexibility but adds no new parameter-level information beyond the schema. Baseline 3 applies due to high schema description coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Analyze') and resource ('webpage <head> HTML or full HTML') and clearly states the tool's purpose: detecting GEO signals for AI-powered search engines. It differentiates itself from sibling analysis tools by naming the unique GEO scoring focus and expanding the acronym.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool: whenever you need to evaluate a page's GEO readiness for AI search engines. It does not explicitly name alternatives or exclusions, but the clear purpose and context ('optimizing pages for AI-powered search engines') provide strong guidance. The lack of explicit 'use this instead of X' prevents a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_jira_issuesA
Read-only
Inspect

Search Jira using JQL (Jira Query Language). Returns matching issues with key fields. Ideal for finding open bugs, sprint tickets, or issues by label/assignee/component. BYOK — credentials transit in-memory only, never stored.

ParametersJSON Schema
NameRequiredDescriptionDefault
jqlYesJQL query string, e.g. "project = PROJ AND status = Open AND assignee = currentUser() ORDER BY priority DESC"
fieldsNoFields per issue. Default: summary, status, assignee, priority, issuetype, labels, created, updated
jira_emailYesAtlassian account email
jira_tokenYesAtlassian API token
max_resultsNoMax issues to return (default: 10, max: 50)
jira_base_urlYesAtlassian base URL, e.g. "https://mycompany.atlassian.net"

Output Schema

ParametersJSON Schema
NameRequiredDescription
jqlNo
totalNo
issuesNo
returnedNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true and destructiveHint=false, and the description adds valuable credential handling context: 'BYOK — credentials transit in-memory only, never stored.' This goes beyond the annotations and informs the agent about security behavior. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two well-structured sentences. The first defines purpose, the second provides usage examples and a security note. There is no wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich schema (all 6 parameters documented), output schema, and annotations, the description covers the essential context: purpose, usage scenarios, and credential handling. It doesn't mention pagination or error behavior, but that is not critical given the schema defaults. No major gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description mentions 'label/assignee/component' which relates to JQL usage, but does not add specific parameter semantics beyond what the schema already documents. The schema carries the full burden for parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool 'Search Jira using JQL (Jira Query Language)' and what it returns: 'matching issues with key fields.' This specific verb+resource+scope distinguishes it from siblings like fetch_jira_issue (which likely fetches a single issue) and post_jira_comment.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear when-to-use context: 'Ideal for finding open bugs, sprint tickets, or issues by label/assignee/component.' It gives scenarios but does not explicitly mention when not to use or name alternatives, so it's a 4 rather than a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

secret_scanA
Read-onlyIdempotent
Inspect

Scan text or code for leaked secrets: API keys (AWS, GCP, Azure, OpenAI, Anthropic, Stripe, GitHub, GitLab, Slack, Twilio, SendGrid, HuggingFace), private keys (RSA/EC/PGP), JWTs, database connection strings, Bearer tokens, and Basic auth headers. Returns a list of findings with type, severity, line number, and a redacted preview. Use before committing code, sharing logs, or sending text to an LLM. 100% regex-based, zero network calls.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText or code to scan for secrets
typesNoComma-separated families to scan (default: all): aws, gcp, azure, openai, anthropic, huggingface, github, gitlab, stripe, slack, twilio, sendgrid, jwt, private_key, connection_string, bearer, basic_auth, generic. Individual pattern names (e.g. "aws_access_key", "github_fine") are also accepted. An unknown value is rejected with an error — a scoped scan never silently returns "clean".

Output Schema

ParametersJSON Schema
NameRequiredDescription
summaryNo
findingsNo
risk_levelNo
input_linesNo
scanned_typesNo
secrets_foundNo
findings_countNo

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that it is 100% regex-based with zero network calls, which is not covered by annotations. Annotations already declare readOnlyHint=true and destructiveHint=false, but description adds that it's regex-based (not ML) and doesn't exfiltrate data—critical for trust. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff, front-loaded with purpose. Every word contributes: listing secret types, return info, usage timing, and behavioral caveat. Perfectly sized for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given simple input (string) and optional types, plus output schema exists, the description fully covers what's needed: return format, usage timing, and safety behavior. No gaps for an agent to misuse the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions, but description adds detail on the 'types' parameter: lists default 'all' and gives examples (e.g., 'aws_access_key'), and notes that unknown values are rejected. This goes beyond schema by clarifying behavior on invalid input.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it scans text/code for leaked secrets, lists specific types (API keys, private keys, JWTs, etc.) and return findings. Distinguishes from sibling 'detect_secrets' by specifying the comprehensive families and usage context, making it the go-to secret scanner.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs when to use: before committing code, sharing logs, or sending text to an LLM. This clearly delineates use cases and suggests this is the recommended tool for secret scanning, differentiating from siblings like 'detect_secrets' and 'redact_pii'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

security_headers_checkA
Read-only
Inspect

Analyse the HTTP security headers of a public URL OR of raw response headers you paste in. Grades each header (A–F) for: Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, X-XSS-Protection, Cross-Origin-Opener-Policy, Cross-Origin-Resource-Policy, and Cross-Origin-Embedder-Policy. Returns an overall score (0–100), per-header grades, missing headers, and fix snippets for Express, Nginx, and Apache. For localhost/private targets the remote server cannot reach, pass the headers parameter instead of url.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlNoOptional. Full public URL to check (e.g. https://example.com). Omit it entirely when using `headers`. The server cannot reach localhost/private IPs.
headersNoOptional, and sufficient on its own (no url needed). The response headers to grade, either as an object {"strict-transport-security": "max-age=...", ...} or as the raw header block pasted as a string (e.g. `curl -sI` output). Use this to audit a local server the remote MCP cannot reach.

Output Schema

ParametersJSON Schema
NameRequiredDescription
fixNo
keyNo
urlNo
weakNo
gradeNo
scoreNo
valueNo
headerNo
sourceNo
weightNo
detailsNo
missingNo
weak_countNo
missing_countNo
overall_gradeNo
headers_checkedNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, and the description adds substantial behavioral context: the A–F grading scale, the overall 0–100 score, missing header reporting, fix snippets, and the critical limitation that the server cannot reach localhost/private IPs. This goes well beyond what annotations alone convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense, well-organized sentences cover the tool's purpose, input modes, grading criteria, output details, and a usage caveat. Every sentence carries substantive information with no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description is not obligated to detail return values, but it still summarizes the overall score, per-header grades, missing headers, and fix snippets. The input modes and limitations are thoroughly explained, making the tool usable without needing to inspect the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers both parameters 100%, so the baseline is 3. The description adds meaning by framing the tool as accepting 'a public URL OR raw response headers', clarifying that one of the two is expected and providing a practical decision rule for which to use. This is useful guidance beyond the individual parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Analyse') and resource ('HTTP security headers') and clearly distinguishes two input modes (public URL or raw headers). It lists the exact headers graded and the output components, making it unmistakable what the tool does and how it differs from sibling tools like cookie_security_audit or cors_checker.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance on when to use the `headers` parameter instead of `url` (for localhost/private targets the server cannot reach), which is directly actionable. It doesn't explicitly name sibling tools as alternatives, but the domain and parameter guidance provide clear context for when this tool is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

shield_analyzeA
Read-only
Inspect

Run a comprehensive AI guardrail analysis on an LLM response. Orchestrates 7 deterministic safety checks plus an optional LLM-powered deep analysis in parallel: hallucination detection (grounding score), prompt injection scan on BOTH the response and the original prompt (reported separately as checks.injection / checks.injection_prompt, scored once at the worse of the two), toxicity scan, output validation (PII/safety), guardrail rules, response quality scoring, and AI verdict (via Qwen, Gemma, Llama, etc.). Returns a unified PASS/FIX/BLOCK verdict with a 0-100 safety score, per-check results, and actionable fix recommendations. PII gates the verdict: an SSN or credit card in the response forces BLOCK, an email forces FIX, phone/IP matches are flagged only (their regexes also match dates and version strings). Use this as a single-call safety gate before surfacing any LLM output to users.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNoLLM model for AI-powered deep analysis (default: "openai/gpt-oss-20b"). Set to "none" to skip LLM check. Supports any model from list_llm_models.
rulesNoOptional guardrail rules array (same format as guardrail_test tool)
promptNoOptional original prompt. Used for quality scoring AND scanned for prompt injection in its own right (checks.injection_prompt) — pass it whenever you have it, it is where the attack actually lands.
sourceNoOptional reference/source text for hallucination grounding check
responseYesThe LLM-generated response to analyze

Output Schema

ParametersJSON Schema
NameRequiredDescription
flagsNo
gradeNo
scoreNo
checksNo
verdictNo

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description reveals extensive behavioral details: it orchestrates 7 deterministic checks, includes an optional LLM deep analysis, explains how PII gates the verdict (SSN/credit card forces BLOCK, email forces FIX), and notes edge cases like regex matching pitfalls. This goes well beyond the annotations and provides high transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is relatively long but information-dense, covering a complex tool with multiple checks, scoring, and edge cases. It is well-structured with a clear main purpose up front and detailed specifics afterward, avoiding unnecessary fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description gives a good overview of the output (verdict, score, per-check results, recommendations) and covers key behaviors and edge cases. While the output schema is not explicitly shown, the description provides enough context for the caller to understand what to expect. It does not detail every possible scenario but is reasonably complete for the complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides detailed descriptions for each parameter (e.g., model, prompt, source). The tool description does not add significant new meaning beyond what is already in the schema; it focuses on tool behavior rather than parameter semantics. Since schema coverage is 100%, the baseline is 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's primary function as a comprehensive AI guardrail analysis on LLM responses, listing the specific checks it orchestrates. It distinguishes itself from individual sibling tools like toxicity_scan or prompt_injection_scan by positioning itself as a unified safety gate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly recommends using this as a single-call safety gate before surfacing LLM output, giving a clear usage scenario. However, it does not explicitly contrast it with the individual sibling tools or state when not to use it, so it falls short of perfect explicitness.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

similarity_scoreA
Read-onlyIdempotent
Inspect

Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU. No API key needed. Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking. Supports batch mode.

ParametersJSON Schema
NameRequiredDescriptionDefault
batchNoBatch mode: array of {reference, hypothesis} pairs.
metricsNoMetrics to compute (default: all). Options: "cosine_bow", "cosine_tfidf", "jaccard", "rouge1", "rouge2", "rougeL", "bleu"
referenceNoReference / expected text (ground truth)
thresholdNoOptional pass/fail threshold (0-1). Applies to ROUGE-L F1 score.
hypothesisNoHypothesis / actual text (LLM output)

Output Schema

ParametersJSON Schema
NameRequiredDescription
f1No
modeNo
countNo
recallNo
resultsNo
precisionNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive. The description adds that no API key is required and that batch mode is supported, giving useful operational context beyond the annotations. There is no contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences front-load the purpose and metric list, then add use cases and batch support. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, use cases, metrics, auth requirements, and batch mode. The threshold parameter's behavior is in the schema, and the output schema exists, so return values are covered. It is complete for a read-only scoring tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all five parameters. The description mentions the metric names and batch mode, but doesn't add syntax or format details beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Compute text similarity between reference and hypothesis' with a specific verb and resource, then enumerates six distinct metrics. This distinguishes it from sibling tools like embedding_similarity or levenshtein_distance by specifying the exact metric set and batch capability.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides explicit use cases: 'Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking.' It does not explicitly name alternatives or exclusion criteria, but the context is clear enough for an agent to know when to invoke this tool over similar ones.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

sort_linesA
Read-onlyIdempotent
Inspect

Sort, deduplicate, reverse, or filter lines of text. Useful for cleaning import lists, dependencies, log files, and config entries.

ParametersJSON Schema
NameRequiredDescriptionDefault
trimNoTrim whitespace from each line (default: true)
inputYesMulti-line text to process
filterNoFor "filter": keep lines containing this substring (case-insensitive)
operationNo"sort" (default), "sort_desc", "reverse", "deduplicate", "unique_sort", "filter"
remove_emptyNoRemove empty lines (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultNo
removedNo
line_countNo
original_countNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds value by explicitly listing the transformations performed (sort, deduplicate, reverse, filter) and the typical target data (import lists, dependencies, etc.). This extra context helps the agent understand the operational behavior without relying solely on the parameter schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with a clear verb phrase, and contains no filler. The second sentence adds practical usage context without redundancy. Every word earns its place, making it an exemplar of concise tool description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (5 parameters, multiple operations) and the availability of both a 100% explanatory schema and an output schema, the description is adequately complete. It communicates the core purpose and common use cases, while the schema handles operational details. It could optionally mention default behaviors (e.g., trim defaults to true), but the schema already covers these, so the description does not need to repeat them.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage for all five parameters, including a detailed description for the 'operation' parameter listing all possible values. The description's mention of 'sort, deduplicate, reverse, or filter' aligns with the schema but does not introduce new parameter-specific details, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool's function with a specific verb phrase: 'Sort, deduplicate, reverse, or filter lines of text.' This distinguishes it from sibling text utilities by explicitly enumerating distinct operations and naming the target resource (lines of text). It goes beyond a simple tautology and fully captures the tool's scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context by stating it is 'useful for cleaning import lists, dependencies, log files, and config entries.' This gives the agent a sense of when to select this tool, though it does not explicitly mention when not to use it or alternative tools. The absence of explicit exclusions is a minor gap, but the use-case list is sufficient for most selection scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

split_chunksA
Read-onlyIdempotent
Inspect

Split text into chunks of at most N tokens (cl100k_base: ~4 chars/token) with optional overlap. Designed for RAG ingestion pipelines.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to split into chunks
overlapNoToken overlap between consecutive chunks (default: 0)
chunk_tokensYesMaximum tokens per chunk (10–8000)

Output Schema

ParametersJSON Schema
NameRequiredDescription
chunksNo
chunk_countNo
overlap_tokensNo
tokens_per_chunkNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds the tokenizer detail ('cl100k_base: ~4 chars/token') and optional overlap, which helps agents predict output behavior without repeating annotation info.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action; no filler. The first sentence conveys the core behavior and the second adds context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity, the description plus the full input schema and output schema provide sufficient context. The only gap is no explicit alternative reference, but the use case is adequately scoped.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all parameters are documented. The description adds the cl100k_base tokenizer and char/token ratio, which clarifies how chunk_tokens is interpreted—beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Split text into chunks' – a specific verb and resource that clearly distinguishes from siblings like count_tokens and truncate_to_tokens. The mention of 'at most N tokens' and 'overlap' further specifies the tool's unique behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states 'Designed for RAG ingestion pipelines,' giving a clear context of when to use it. It does not explicitly name alternatives or exclusions, but the chunking use case is self-evident and no misleading guidance is present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ssl_certificate_checkA
Read-only
Inspect

Analyse the SSL/TLS certificate of any HTTPS host. Returns certificate subject, issuer, validity dates, days until expiry, protocol version, cipher suite, key exchange info, and an overall grade (A+, A, B, C, F). Detects expired, self-signed, and weak certificates. Use this to audit TLS posture before production deployment or during security reviews.

ParametersJSON Schema
NameRequiredDescriptionDefault
hostYesHostname to check (e.g. example.com). Do not include https:// prefix.
portNoPort number (default: 443)

Output Schema

ParametersJSON Schema
NameRequiredDescription
hostNo
gradeNo
cipherNo
issuerNo
issuesNo
subjectNo
protocolNo
valid_toNo
is_expiredNo
valid_fromNo
is_self_signedNo
days_until_expiryNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark readOnlyHint=true and destructiveHint=false, and the description further explains that it detects expired, self-signed, and weak certificates, and returns a grade. This adds useful behavioral context beyond the annotations, such as its network-based external host checking. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences that front-load the core function, then list key outputs and use cases. Every sentence adds value, and there's no redundancy with the schema or annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and clear annotations, the description need not restate return structures. It provides the essential behavioral and usage context, including what the tool checks, what it returns, and when to use it. This is complete for a tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both host and port having clear descriptions. The description reinforces the host parameter by saying 'any HTTPS host' and implies the default port behavior indirectly, but adds no extra parameter details beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action (analyse SSL/TLS certificate) and resource (any HTTPS host), and enumerates specific return values (subject, issuer, validity, grade). This distinguishes it from sibling security tools like security_headers_check or web_security_audit, which target other aspects of web security.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit use cases: 'audit TLS posture before production deployment or during security reviews.' It doesn't explicitly contrast with sibling tools, but the focused scope makes the intended usage clear. It would be stronger with an explicit 'for checking HTTP headers use security_headers_check,' but the context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

strip_markdownA
Read-onlyIdempotent
Inspect

Strip all Markdown formatting (headers, bold, italic, code fences, links, lists) from text and return clean plain text. Run this before injecting scraped documentation, README files, or user content into an LLM prompt to eliminate redundant markup tokens and reduce cost.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesMarkdown text to convert to plain text

Output Schema

ParametersJSON Schema
NameRequiredDescription
textNo
original_lengthNo
stripped_lengthNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the description needn't rehash safety. It adds value by enumerating the elements stripped (headers, bold, italic, code fences, links, lists) and explaining the cost-reduction benefit, which goes beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: first states the action and output, second gives the usage context and rationale. Every word earns its place; zero fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, read-only, idempotent tool with an output schema, this description covers purpose, usage, and behavior sufficiently. There are no unresolved questions about invocation or results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes the sole parameter as 'Markdown text to convert to plain text' with 100% coverage. The description repeats this concept without adding syntax or format details, so it meets the baseline but doesn't exceed it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Strip all Markdown formatting') and the expected output ('return clean plain text'), with explicit examples of formatting elements. This makes the tool's purpose unmistakable and distinguishes it from conversion tools like html_to_markdown.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly instructs when to use: 'Run this before injecting scraped documentation, README files, or user content into an LLM prompt.' This provides clear context and a concrete use case, though it doesn't contrast with alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

system_prompt_builderA
Read-onlyIdempotent
Inspect

Build a structured system prompt from components: role, task, constraints, output format, tone, language, and examples. Generates a production-ready system prompt with token estimate.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesRole/persona (e.g. "Senior QA Engineer", "JSON extraction assistant")
taskNoMain task or objective
toneNoCommunication tone
examplesNoBrief examples to include
languageNoResponse language (e.g. "French")
constraintsNoRules and constraints to follow
output_formatNoExpected output format description

Output Schema

ParametersJSON Schema
NameRequiredDescription
sectionsNo
system_promptNo
token_estimateNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is well covered. The description adds valuable context by stating it generates a production-ready system prompt with a token estimate, which goes beyond the annotations. No contradiction is present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exactly two sentences: the first states the action and components, the second states the output and token estimate. It is front-loaded with the verb 'Build' and contains no unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 7 parameters, full schema descriptions, a rich annotation set, and an output schema, the description adequately covers the overall behavior. It mentions the key output (structured system prompt) and the token estimate, which is sufficient for an agent to select and invoke the tool. It does not explain edge cases or prerequisites, but those are not essential here.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, and the description merely lists the parameter names in prose. It does contextualize them as 'components' but adds no additional meaning beyond what the schema already provides. This meets the baseline for full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the specific action 'Build a structured system prompt' and enumerates all component types (role, task, constraints, output format, tone, language, examples). This clearly distinguishes it from sibling prompt-related tools like build_rag_prompt or few_shot_formatter by emphasizing a structured, component-driven system prompt builder with a token estimate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear context for use: when you need to assemble a system prompt from the listed components. However, it does not explicitly mention when not to use this tool or suggest alternatives among the sibling tools, so it stops short of a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_skillA
Read-only
Inspect

Validate a SKILL.md definition (Cursor / GitHub Copilot / Windsurf) by auto-generating trigger-positive and trigger-negative scenarios, running each through the model with the skill injected as a system prompt, and scoring trigger accuracy + step adherence. Returns a PASS/FIX/BLOCK verdict with per-scenario breakdown. Uses Groq llama-3.3-70b by default (server key, no api_key needed). Pass api_key + model to use your own provider.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNoLLM model ID to use for both scenario generation and testing (e.g. gpt-4o-mini, claude-3-5-haiku-20241022). Defaults to openai/gpt-oss-20b (Groq, server key).
api_keyNoAPI key for the chosen model provider. Not required when using the default Groq model.
skill_mdYesFull content of the SKILL.md file to test. Must include a name, a "Use when:" trigger description, and at least one step.
scenario_countNoNumber of test scenarios to generate: half trigger-positive, half trigger-negative. Default: 6.

Output Schema

ParametersJSON Schema
NameRequiredDescription
scoreNo
verdictNo
scenariosNo
step_adherenceNo
trigger_accuracyNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as read-only and non-destructive. The description adds meaningful context beyond that: it reveals the testing process, the default provider (Groq), that no api_key is needed with the server key, and the option to supply api_key + model for a custom provider, plus the PASS/FIX/BLOCK verdict behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose, followed by process, output, and provider details. No fluff; each sentence earns its place. The only minor issue is the default-model mismatch with the schema, but structurally it is concise and well-organized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description explains the core workflow, return verdict, and auth model, and an output schema exists to cover return details. It does not describe scenario_count default or enumeration constraints, but the schema covers those. The description is complete enough for a moderate-complexity read-only tool, though the model default inconsistency is a clarity gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers all four parameters at 100%, so baseline is 3. The description adds provider-switching context but lacks extra param nuance. However, it contains a default-model contradiction: the description says 'Groq llama-3.3-70b' while the schema's model param says 'Defaults to openai/gpt-oss-20b (Groq, server key)', undermining the reliability of the added info.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description opens with a specific verb+resource: 'Validate a SKILL.md definition'. It details the process (auto-generating trigger-positive/negative scenarios, running each through the model with the skill injected as a system prompt) and distinguishes itself from generic test tools by focusing on SKILL.md files for Cursor/GitHub Copilot/Windsurf and scoring trigger accuracy + step adherence.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use case is explicit—validating a SKILL.md definition—though it does not name alternative tools or provide when-not-to-use guidance. The domain targeting (Cursor / GitHub Copilot / Windsurf) gives clear context for when this tool is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

text_statsA
Read-onlyIdempotent
Inspect

Compute comprehensive statistics for any text: character count (with and without spaces), word count, line count, sentence count, paragraph count, and estimated reading time in minutes. Sentence counting is abbreviation-aware — titles (Mr., Dr.), acronyms (U.S., i.e., p.m.), initials, decimals, URLs and emails do not end a sentence, and a text with no terminal punctuation still counts as one. Use for validating form field lengths, evaluating LLM output verbosity, or content auditing.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesThe text to analyse

Output Schema

ParametersJSON Schema
NameRequiredDescription
charsNo
linesNo
wordsNo
sentencesNo
paragraphsNo
chars_no_spaceNo
reading_time_minutesNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, which already convey safety. The description adds valuable behavioral context beyond annotations by detailing abbreviation-aware sentence counting, handling of titles, acronyms, URLs, etc., and the edge case of text without terminal punctuation. This enriches the agent's understanding of the tool's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, with each sentence adding unique value: the first lists metrics, the second explains sentence detection nuances, and the third lists use cases. No fluff, well-structured for quick reading.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter, full schema coverage, and detailed behavioral transparency on edge cases, the description is complete. It covers what statistics are returned, how edge cases are handled, and use cases, which fully equips an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with one parameter 'input' described as 'The text to analyse', which fully documents the parameter. The description adds context on what statistics are computed but doesn't need to add more parameter detail. Baseline 3 is appropriate since schema covers parameter meaning fully.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Compute') and clearly identifies the resource (text statistics) and enumerates exact metrics (character count, word count, line count, etc.). It distinguishes itself from siblings by detailing the comprehensive set of statistics and the reading time estimate, which is specific to this tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly lists use cases such as validating form field lengths, evaluating LLM output verbosity, and content auditing, providing clear context for when to use this tool. It does not explicitly name alternative tools or state when not to use it, but the uniqueness of the tool's purpose minimizes ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

timestamp_convertA
Read-only
Inspect

Convert between Unix timestamps (seconds or milliseconds) and ISO-8601 / UTC date strings. Auto-detects epoch vs. millisecond format. Omit input to get the current time. Returns iso, unix_s, unix_ms, utc, date, and time fields.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputNoUnix timestamp (number, seconds or ms) or ISO date string. Omit to get the current time.

Output Schema

ParametersJSON Schema
NameRequiredDescription
isoNo
utcNo
dateNo
timeNo
unix_sNo
unix_msNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnlyHint and destructiveHint annotations, the description discloses auto-detection of input format and the exact fields returned. It does not contradict annotations and adds useful behavioral context, though edge cases like invalid input are not mentioned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences: purpose, auto-detection, and output fields. Information is front-loaded with the primary action first, and every sentence earns its place without redundant padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool with an output schema and read-only annotation, the description covers all essential aspects: bidirectional conversion, input formats, current-time shortcut, and return fields. No critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% coverage for the single parameter, including the omit-for-current-time behavior. The description adds the auto-detection detail, which is helpful but marginal; the schema carries most of the semantic weight.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool converts between Unix timestamps and ISO-8601/UTC date strings, using a specific verb and resource. It distinguishes from sibling tools like base64_decode or json_to_csv, which handle other data types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context: auto-detects epoch vs. millisecond format, omit input for current time. Since no sibling tool handles timestamp conversion, exclusion is unnecessary, but the tool could benefit from an explicit 'use this when' statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

token_budget_calculatorA
Read-onlyIdempotent
Inspect

Plan token allocation across system prompt, user input, context/RAG chunks, and expected output. Warns if budget exceeds model context window. Supports 25+ models.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesModel name (e.g. gpt-4o, claude-3.5-sonnet, gemini-2.0-flash)
contextNoActual context text (will estimate tokens)
user_inputNoActual user input text (will estimate tokens)
system_promptNoActual system prompt text (will estimate tokens)
context_tokensNoToken count for RAG context / documents
user_input_tokensNoToken count for user message
system_prompt_tokensNoToken count for system prompt
expected_output_tokensNoExpected max output tokens

Output Schema

ParametersJSON Schema
NameRequiredDescription
modelNo
warningsNo
breakdownNo
context_windowNo
fits_in_windowNo
remaining_tokensNo
utilization_percentNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so no safety disclosure is needed. The description adds beyond annotations by specifying the warning behavior ('Warns if budget exceeds model context window') and model coverage ('Supports 25+ models'), which are useful behavioral traits not present in annotation fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, no redundancy. Every word earns its place, and the warning/model support details are valuable additions without bloat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 8 parameters and an output schema, so the description doesn't need to detail returns. It covers the core behavior and key constraints (warning, model support). However, it lacks explicit guidance on how it differs from closely related siblings (count_tokens, context_window_check, llm_fit_finder), which would make it fully complete for an agent operating in a crowded tool space.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so each parameter is already explained. The description adds semantic meaning by grouping parameters into logical categories ('system prompt, user input, context/RAG chunks, and expected output') and implicitly clarifying that the tool can accept both raw text (to estimate tokens) and manual token counts. This goes slightly beyond the schema's individual field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Plan token allocation across system prompt, user input, context/RAG chunks, and expected output.' It clearly distinguishes from siblings like count_tokens and contextualize by focusing on planning allocation across multiple components, not just counting or checking.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it (planning allocation across multiple segments) and mentions warning behavior, but does not explicitly name alternatives or exclusions. In a crowded sibling space (count_tokens, estimate_llm_cost, context_window_check), it would benefit from explicit 'use this when' guidance, though the purpose is clear enough to infer intended use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

toxicity_scanA
Read-onlyIdempotent
Inspect

Scan text for toxic language, hate speech, bias/stereotype framing, violence, sexual and self-harm content. Lexical + structural pattern matching (identity term + predicate), not a semantic classifier — returns per-category risk plus the named rules that fired, so every finding can be checked. Useful for LLM safety guardrail testing and triage; signal-only, not a calibrated CI gate.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYesText to scan
categoriesNoCategories to check (default: all)

Output Schema

ParametersJSON Schema
NameRequiredDescription
methodNo
resultsNo
text_lengthNo
overall_riskNo
categories_checkedNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the underlying methodology (lexical + structural pattern matching, not semantic classifier) and explains the output (per-category risk plus named rules). This goes beyond the basic annotations, providing substantial transparency about limitations and behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, tightly packed with purpose, method, output, and usage context. No unnecessary words or repetition, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers what the tool does, how it works, what it returns, and when to use it. It even clarifies limitations (not a calibrated CI gate) and provides an example of output ('named rules that fired'). This is comprehensive for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides descriptions for both parameters (text and categories) with the default for categories. The description adds little beyond listing the categories, which is already in the schema. Since schema coverage is 100%, the baseline is 3, and the description does not significantly enhance parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: scanning text for toxic language, hate speech, and other categories. It also distinctively identifies the tool as a toxicity scanner among many siblings, with no ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage context ('LLM safety guardrail testing and triage') and cautions that it is not a calibrated CI gate. While it doesn't name alternatives, it gives clear guidance on appropriate use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

transform_json_arrayA
Read-onlyIdempotent
Inspect

Transform a JSON array using common operations: pluck (extract specific fields), filter (by field value), sort_by (field), group_by (field), count_by (field), uniq_by (field). Useful for processing MCP tool results and LLM structured outputs.

ParametersJSON Schema
NameRequiredDescriptionDefault
nNoFor first_n / last_n: number of items
pathNoOptional dot-notation path to the array within the JSON object (e.g. "data.items")
fieldNoField to operate on (for sort_by, group_by, count_by, uniq_by, filter)
inputYesThe JSON containing an array (or an object with an array at `path`) — a JSON string, or the value itself.
fieldsNoComma-separated field list for "pluck" (e.g. "id,name,email")
filter_opNoFor "filter": "==" | "!=" | ">" | ">=" | "<" | "<=" | "contains" | "exists" | "!exists"
operationYesOperation: "pluck", "filter", "sort_by", "group_by", "count_by", "uniq_by", "reverse", "first_n", "last_n", "flatten"
sort_orderNoFor sort_by: "asc" (default) or "desc"
filter_valueNoFor "filter": value to compare against

Output Schema

ParametersJSON Schema
NameRequiredDescription
countNo
fieldNo
orderNo
totalNo
fieldsNo
resultNo
removedNo
operationNo
group_countNo
unique_valuesNo
removed_duplicatesNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety and idempotency. The description adds that the tool transforms JSON arrays through operations like filter and sort, which is consistent. It does not add behavioral traits beyond what annotations provide (e.g., no mention of side effects or error handling), so a baseline score is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no wasted words: the first sentence states the action and lists operations, the second provides use-case context. The verb 'Transform' is front-loaded and the structure is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 9 parameters (2 required) and a high-coverage schema, the description effectively communicates the tool's array-transformation purpose and common operations. It does not explicitly mention the return value (a transformed array) or give examples, but the output schema exists and the name implies an array result. Minor gap, but overall sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents each parameter's role. The description lists operations at a high level but does not add meaning beyond the schema (e.g., how 'field' or 'filter_value' apply to specific operations). A baseline score of 3 is warranted since the schema carries the full semantic burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Transform a JSON array using common operations' and lists specific operations (pluck, filter, sort_by, etc.), which distinguishes it from sibling tools like json_diff or json_to_csv that handle different aspects of JSON manipulation. The usage context ('processing MCP tool results and LLM structured outputs') further clarifies scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description indicates that the tool is 'useful for processing MCP tool results and LLM structured outputs,' providing a clear context for when to use it. However, it does not explicitly state when not to use it or mention alternatives (e.g., flatten_json for nested arrays), which would strengthen guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

truncate_to_tokensA
Read-onlyIdempotent
Inspect

Truncate text to at most N tokens (cl100k_base: ~4 chars/token) to avoid exceeding an LLM context window. Optionally keeps the end of the text instead of the start (useful for keeping recent conversation history). Reports whether truncation occurred and the estimated token count.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to truncate
from_endNoKeep the end of the text instead of the start (default: false)
max_tokensYesMaximum number of tokens to keep

Output Schema

ParametersJSON Schema
NameRequiredDescription
textNo
truncatedNo
tokens_estimateNo
original_tokens_estimateNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool read-only, idempotent, and non-destructive. The description adds meaningful behavioral detail: it uses cl100k_base (~4 chars/token), can optionally keep the end, and reports whether truncation occurred along with an estimated token count. This goes beyond the annotations and provides transparency about the operation's output semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences with no filler. It front-loads the main purpose, then covers the optional behavior, and finally states what the tool reports. Every sentence contributes useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's modest complexity and the presence of an output schema, the description covers all essential aspects: purpose, tokenization basis, optional direction, and output reporting. It is complete enough for an agent to select and invoke the tool correctly without further clarification.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already has 100% parameter coverage, providing descriptions for input, from_end, and max_tokens. The description adds extra meaning by explaining cl100k_base tokenization and the practical reason for using from_end (keeping recent history), which helps the agent understand how to use the parameters effectively.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Truncate text to at most N tokens' and clearly states the goal of avoiding exceeding an LLM context window. It also distinguishes itself from sibling tools like count_tokens and token_budget_calculator by focusing on the action of truncation rather than counting or budgeting.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly states the primary use case ('to avoid exceeding an LLM context window') and adds a practical scenario for the optional from_end flag ('useful for keeping recent conversation history'). It does not explicitly name alternatives or state when not to use the tool, but the context is clear enough for an agent to decide appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

unescape_htmlA
Read-onlyIdempotent
Inspect

Convert HTML entities (&amp;, &lt;, &gt;, &quot;, &#x27;, and numeric &#NNN;) back to plain characters. Use when processing HTML-encoded text from APIs, email content, or legacy database fields before passing to an LLM or displaying to users.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesHTML-encoded string to unescape

Output Schema

ParametersJSON Schema
NameRequiredDescription
unescapedNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already disclose read-only, idempotent, and non-destructive behavior. The description adds valuable context by listing supported entity types (named and numeric) and the conversion direction, which goes beyond what annotations provide. It does not mention edge cases like unknown entities, but this is acceptable for a simple conversion tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the action and entity list, the second gives usage context. No filler or redundancy; all information earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with a clear schema, annotations, and an output schema (as signaled), the description covers purpose, usage, and example entities. It lacks explicit mention of handling invalid HTML entities or recursion, but those are edge cases unlikely to block correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema fully describes the single parameter 'input' as 'HTML-encoded string to unescape'. The description does not add further parameter-specific details beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb 'Convert' and a clear resource: HTML entities. It enumerates the exact entities handled (&amp;, &lt;, etc.) and contrasts with counterpart tools like escape_html by stating it converts 'back to plain characters'. This clearly distinguishes it from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use context: 'when processing HTML-encoded text from APIs, email content, or legacy database fields before passing to an LLM or displaying to users.' It does not mention when-not-to-use, but the guidance is clear enough for selection among sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

url_decodeA
Read-onlyIdempotent
Inspect

Decode a percent-encoded URL string back to plain text. Use when parsing query parameters from raw URLs or when displaying encoded values to users.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesURL-encoded string to decode

Output Schema

ParametersJSON Schema
NameRequiredDescription
decodedNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this is read-only, idempotent, and non-destructive, so the description's safety burden is lowered. The description adds the behavior of converting percent-encoded input to plain text and the intended use context, but it does not disclose edge-case behaviors like handling of malformed input, plus signs, or non-UTF-8 encodings.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise, using just two sentences that state the purpose and usage context without any filler or redundant details. Every phrase earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (one parameter), full schema coverage, presence of an output schema, and comprehensive annotations, the description is complete. It covers the core function and primary use cases without needing to explain return values, which are presumably defined in the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage for the single parameter is 100%, and the schema description already says 'URL-encoded string to decode.' The description adds the qualifier 'percent-encoded' and clarifies the output is 'plain text,' which is marginal value but not substantial beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a strong verb phrase, 'Decode a percent-encoded URL string back to plain text,' which clearly names the specific operation and resource. It distinguishes itself from sibling tools like url_encode, base64_decode, and unescape_html by explicitly citing percent-encoding.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives concrete use cases: 'Use when parsing query parameters from raw URLs or when displaying encoded values to users.' This provides clear context for when to use the tool, though it does not mention explicit when-not-to-use scenarios or alternative tools, so it falls short of a perfect 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

url_encodeA
Read-onlyIdempotent
Inspect

Percent-encode a string for safe use in URLs. Call this before programmatically building query strings, path segments, or form-encoded bodies to prevent injection and malformed URLs.

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNo"component" (default) or "full" for encodeURI behavior
inputYesString to URL-encode

Output Schema

ParametersJSON Schema
NameRequiredDescription
encodedNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds rationale (prevent injection) but does not disclose deeper behavioral details like which characters are encoded, UTF-8 handling, or edge cases, though the mode parameter hints at encodeURI behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose, and every clause adds value. No fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity and the presence of an output schema, the description sufficiently covers purpose, usage, and safety. The mode distinction is left to the schema, which is acceptable. A brief example could push it to 5, but current coverage is solid.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both input and mode clearly described. The description adds usage context (e.g., what to encode) but does not introduce new parameter-level details beyond what the schema already provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Percent-encode a string for safe use in URLs.' It clearly distinguishes from sibling encoding tools like base64_encode by specifying URL percent-encoding, and even lists common use cases (query strings, path segments, form-encoded bodies).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to call the tool ('before programmatically building query strings, path segments, or form-encoded bodies') and why (to prevent injection and malformed URLs). It provides clear context but does not mention when not to use it or name alternatives like url_decode.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_agent_trajectoryA
Read-onlyIdempotent
Inspect

Run declarative assertions on an agent trace (OpenAI tool-call messages, Anthropic tool_use/tool_result blocks, LangChain run trees, or plain text ReAct logs). No LLM call — deterministic. Assertion types: order (tool A before B), must_call, must_not_call, max_calls, min_calls, no_error, recovery (agent completes a successful step after its last error). A step counts as errored when the trace says so — is_error/isError, status/state in {error,failed,exception,…}, an error field, a JSON body with error/success:false — or when its text payload STARTS with an error marker (Error:, Traceback, TypeError:). Model prose is never scanned for keywords, and every errored step reports error_signal naming what flagged it. Returns per-assertion PASS/FAIL, parsed steps, warnings (a trace parsing to 0 steps is flagged — assertions passing on emptiness prove nothing), and an overall verdict. Use this to gate CI/CD on agent behavior correctness.

ParametersJSON Schema
NameRequiredDescriptionDefault
traceYesAgent execution trace as JSON (OpenAI messages array, LangChain run tree) or plain text log (Thought/Action/Observation format).
formatNoTrace format. auto (default) detects automatically.
assertionsYesList of assertions to validate against the trace.

Output Schema

ParametersJSON Schema
NameRequiredDescription
stepsNo
resultsNo
summaryNo
verdictNo
warningsNo

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavioral detail beyond annotations: it states 'No LLM call — deterministic' (confirming annotations), explains error detection heuristics in depth (error markers, status fields, text prefixes), notes that 'Model prose is never scanned for keywords', and warns that a trace parsing to 0 steps is flagged. This exceeds the readOnly/idempotent hints and provides actionable transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense paragraph that efficiently packs key information: input formats, assertion types, error detection, outputs, and usage intent. It is not overly verbose for the complexity it covers, though it could be broken into bullet points for readability. It earns its place without fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complex input (trace) and seven assertion types, the description is comprehensive. It explains error detection behaviors, return values (per-assertion PASS/FAIL, parsed steps, warnings, verdict), and a caution about empty traces. With an output schema present and annotations covering safety, the description leaves no significant gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds meaning by explaining error detection logic and assertion semantics beyond the schema's brief field descriptions, such as how 'errored' is determined and that warnings are returned. This enhances understanding of the 'trace' and 'assertions' parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool runs declarative assertions on an agent trace, enumerates supported trace formats (OpenAI, Anthropic, LangChain, ReAct) and lists specific assertion types (order, must_call, etc.). It distinguishes from siblings by focusing on agent trajectory validation rather than generic validation tools like validate_email or validate_url.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use this to gate CI/CD on agent behavior correctness', providing a clear use case. It doesn't name specific alternatives, but the purpose is distinct from other validate* tools, and the context is sufficient for an agent to select it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_emailA
Read-onlyIdempotent
Inspect

Validate an email address against RFC 5322 syntax before storing it, sending a transactional email, or adding it to a mailing list. Returns { valid, email } — use this to avoid bounces and malformed data.

ParametersJSON Schema
NameRequiredDescriptionDefault
emailYesEmail address to validate

Output Schema

ParametersJSON Schema
NameRequiredDescription
emailNo
validNo
reasonNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds value beyond annotations by revealing the return shape `{ valid, email }` and by scoping the check to RFC 5322 syntax (vs. deliverability or MX-lookup), which sets accurate expectations about the tool's boundary. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences that each earn their place: the standard and use cases, the return shape as a code snippet, and the purpose. Front-loaded with the imperative verb 'Validate.' Zero wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity tool with 1 trivially-named parameter, 100% schema coverage, rich annotations, and an output schema, the description fully covers purpose, timing, return contract, and value. Nothing material is left undisclosed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage on a single self-evident parameter, the baseline is 3. The description adds marginal but real value by clarifying that 'validate' means syntactic RFC 5322 conformance and by disclosing the tool's output contract, which tells the agent what the single parameter gets checked against.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Uses a specific verb+resource+standard ('Validate an email address against RFC 5322 syntax') and grounds it in concrete use cases (storing, sending transactional email, mailing list). The RFC citation and email domain clearly distinguish it from siblings like validate_url, validate_agent_trajectory, and validate_mcp_response.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly states when to use the tool with three explicit scenarios ('before storing it, sending a transactional email, or adding it to a mailing list') and the intended outcome ('avoid bounces and malformed data'). However, it names no alternatives or when-not-to-use scenarios, despite validation siblings existing in the tool list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_mcp_responseA
Read-onlyIdempotent
Inspect

Validate that an MCP tool response conforms to expected format, schema, and content rules. Use this to QA-test any MCP server tool. Supply the tool's actual JSON result and a set of checks to perform.

ParametersJSON Schema
NameRequiredDescriptionDefault
responseYesThe MCP tool result as a JSON string to validate
min_itemsNoIf response is an array, minimum number of items expected
expected_typeNoExpected top-level type: "object", "array", "string", "number"
required_keysNoComma-separated list of keys that MUST exist in the response (dot-notation for nested: "data.id, data.name")
actual_latencyNoActual measured latency in ms (from the call)
forbidden_keysNoComma-separated list of keys that MUST NOT exist (e.g. "password, secret, token")
max_size_bytesNoMaximum acceptable response size in bytes
max_response_msNoMaximum acceptable latency in ms (will be compared if provided)

Output Schema

ParametersJSON Schema
NameRequiredDescription
totalNo
checksNo
failedNo
passedNo
verdictNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe read-only operation. The description adds that it performs checks on a provided response, which is useful but does not elaborate on error handling or output behavior beyond what annotations imply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the core purpose, and every word earns its place. No filler or redundant detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists and parameter coverage is 100%, the description sufficiently covers the tool's purpose and usage. It could briefly mention that the tool does not call the MCP server itself, but this is not essential for selection or invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all parameters. The description only generically references 'checks to perform,' adding no meaningful detail beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool validates MCP tool responses against format, schema, and content rules. It names the specific resource (MCP tool response) and the action (validate), but does not explicitly distinguish it from sibling validation tools like json_schema_validate or mcp_schema_lint.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear usage context: 'Use this to QA-test any MCP server tool.' This tells the agent when to use the tool, but it does not mention exclusions or explicitly compare to alternatives, falling short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_urlA
Read-onlyIdempotent
Inspect

Parse and validate a URL. Returns decomposed components: protocol, hostname, port, path, query parameters, hash, and origin.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesURL to validate and parse

Output Schema

ParametersJSON Schema
NameRequiredDescription
fullNo
hashNo
portNo
validNo
originNo
searchNo
hostnameNo
pathnameNo
protocolNo
query_paramsNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is known. The description adds the return component breakdown but does not discuss edge cases like invalid URLs or normalization behavior, which would add extra value beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short, front-loaded sentences cover purpose, validation aspect, and return contents. Every word earns its place with no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter signature, existing output schema, and full annotations, the description is sufficient for an agent to select and invoke the tool correctly. It even mentions the return structure, making the tool's behavior immediately clear.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (the parameter is already described as 'URL to validate and parse'). The description essentially restates the input purpose and does not add format or behavior details beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb phrase 'Parse and validate a URL' and clearly identifies the resource and output (decomposed components). This distinguishes it from sibling tools like url_encode/url_decode or validate_email.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly implies use for parsing/validating URLs and describes what will be returned, giving clear context. However, it does not explicitly mention when not to use it or point to an alternative sibling tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vector_quantizeA
Read-onlyIdempotent
Inspect

Simulate int8 or int4 quantization of float32 embedding vectors. Reduces storage by 4x (int8) or 8x (int4). Returns quantized values, scale factor, and precision loss (MSE). Useful for understanding vector DB compression trade-offs.

ParametersJSON Schema
NameRequiredDescriptionDefault
bitsNoQuantization bits: 8 (int8, default) or 4 (int4)
vectorYesFloat32 vector to quantize

Output Schema

ParametersJSON Schema
NameRequiredDescription
mseNo
bitsNo
offsetNo
dimensionNo
quantizedNo
scale_factorNo
compression_ratioNo
storage_bytes_float32No
storage_bytes_quantizedNo

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (readOnly, idempotent, not destructive), the description adds valuable behavioral context: it simulates quantization (not actual), explains storage reduction ratios (4x/8x), and discloses the return contents (quantized values, scale, MSE). No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core function, followed by key details and use case. Every sentence conveys distinct information without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With simple parameters, a full schema, annotations, and an output schema, the description still adds essential context: the simulation nature, trade-off purpose, and return elements. It is complete for an agent to decide and invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds some context about bits (int8/int4) and float32 vectors, but it does not meaningfully augment the schema's parameter descriptions. The mapping to storage reduction is a minor addition.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Simulate') and resource ('int8 or int4 quantization of float32 embedding vectors'), clearly distinguishing it from siblings like normalize_vector or vector_stats. It also explicitly mentions the output and purpose, making the tool's function unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear use case: 'Useful for understanding vector DB compression trade-offs.' It implies when to use it but does not explicitly state when not to use it or name alternatives, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vector_similarityA
Read-onlyIdempotent
Inspect

Compute similarity/distance between two float vectors: cosine similarity, dot product, Euclidean and Manhattan distance. Essential for vector DB relevance scoring, embedding evaluation, and nearest-neighbor testing.

ParametersJSON Schema
NameRequiredDescriptionDefault
metricNoDistance metric (default: all)
vector_aYesFirst vector as array of floats
vector_bYesSecond vector as array of floats

Output Schema

ParametersJSON Schema
NameRequiredDescription
norm_aNo
norm_bNo
dimensionNo
dot_productNo
interpretationNo
cosine_distanceNo
cosine_similarityNo
euclidean_distanceNo
manhattan_distanceNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (read-only, idempotent, non-destructive). The description adds the supported metrics and typical applications, but it doesn't disclose edge-case behavior (e.g., dimension mismatch, zero vectors) or return formats beyond what the output schema would cover.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two clean sentences: first identifies the core function, second gives use cases. No wasted words, proper front-loading.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a pure computational tool with strong annotations and an output schema, the description covers the essentials: operation, metrics, and use cases. It is sufficiently complete without needing to detail return types or edge cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and each parameter has a meaningful description. The tool description repeats metric names from the enum but does not add new parameter-level semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the action ('Compute similarity/distance'), the input type ('two float vectors'), and the specific metrics (cosine, dot product, Euclidean, Manhattan). It also lists use cases, distinguishing it from similar sibling tools like similarity_score or embedding_similarity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit use cases ('vector DB relevance scoring, embedding evaluation, and nearest-neighbor testing'), giving clear context for when to use. However, it doesn't mention alternatives or exclusions relative to sibling tools, so it's not a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vector_statsA
Read-onlyIdempotent
Inspect

Compute statistics for a float vector or matrix of vectors: mean, std, L2 norm, min, max, sparsity, top-K indices. Useful for debugging embedding quality and analyzing vector distributions in a vector DB.

ParametersJSON Schema
NameRequiredDescriptionDefault
top_kNoReturn indices of top K absolute values (default: 5)
matrixNoMatrix of vectors (overrides vector). Returns per-vector + matrix-level stats. Required unless `vector` is given.
vectorNoSingle vector to analyze. Required unless `matrix` is given.

Output Schema

ParametersJSON Schema
NameRequiredDescription
maxNo
minNo
stdNo
meanNo
l2_normNo
sparsityNo
dimensionNo
per_vectorNo
matrix_shapeNo
matrix_statsNo
top_k_indicesNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=true, idempotentHint=true, destructiveHint=false – the description aligns, stating computation (no mutation). It adds context that the tool handles both single vectors and matrices, and returns per-vector plus matrix-level stats. No contradiction. With annotations already covering safety, the description adds meaningful behavioral context about the data structure handling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. Front-loaded with verb and resource, then outcomes, then use cases. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the full schema coverage, clear annotations, and presence of an output schema (which presumably describes return format), the description is complete. It explains what the tool does, what data structures it accepts (vector or matrix), what statistics are computed, and when to use it. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter has a clear description. The description adds value beyond the schema by listing computed statistics (not in schema) and clarifying that matrix overrides vector. The top_k default (5) is mentioned in schema but the description reaffirms it. Slight extra context about what statistics are returned improves usability.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Compute') and resource ('float vector or matrix of vectors'), lists the computed statistics (mean, std, L2 norm, min, max, sparsity, top-K indices), and distinguishes itself from sibling tools like normalize_vector (which normalizes but does not compute stats) and vector_similarity (which computes similarity). The purpose is clear and distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases: 'debugging embedding quality' and 'analyzing vector distributions in a vector DB'. It implicitly distinguishes from siblings like vector_quantize, normalize_vector, and vector_similarity by focusing on statistics computation. However, it lacks explicit when-not-to-use guidance or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webhook_endpoint_createAInspect

Create a temporary webhook endpoint that captures incoming HTTP requests for one hour. Returns the webhook id, public URL, expiration timestamp, and current request count. Use together with webhook_endpoint_requests to inspect captured payloads.

ParametersJSON Schema
NameRequiredDescriptionDefault
base_urlNoOptional public base URL. Default: https://www.ia-qa.com/mcp/webhook (the apex ia-qa.com answers 301 and a redirected POST loses its body, so an apex base_url is normalized to www)

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
urlNo
expires_atNo
request_countNo
retention_minutesNo

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare no safety hints (readOnlyHint=false, etc.), but the description adds important behavioral context beyond those annotations: the endpoint is temporary (one hour lifetime), it returns a public URL, expiration timestamp, and request count. It also notes a normalization nuance about apex base_url redirect behavior, which is valuable for correct usage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, each earning its place: the first defines the function and primary outputs; the second gives usage guidance with the sibling tool. No fluff, front-loaded with purpose, and efficient in communication.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This tool is relatively simple (one optional param, output schema exists) and the description covers the key elements: temporary nature, public URL, expiration, request count, and the pairing with the requests inspector. However, it does not specify allowed HTTP methods or payload size limits, which could be relevant for a webhook endpoint but are minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Though schema coverage is 100% and the parameter (base_url) is well-documented in the schema, the description adds operational meaning: it clarifies default behavior, webhook lifecycle, and the returned fields. However, the description doesn't add much beyond the schema for the single parameter itself, as the schema already includes the normalization note.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: creating a temporary webhook endpoint that captures incoming HTTP requests for one hour. It specifies the resource (webhook endpoint), the action (create), the constraint (temporary, one hour), and the primary capability (capturing HTTP requests). This clearly distinguishes it from sibling tools like webhook_endpoint_requests, which is for inspecting captured payloads.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool: 'Use together with webhook_endpoint_requests to inspect captured payloads.' This provides clear guidance on the complementary relationship with a sibling tool, effectively communicating the typical workflow sequence.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webhook_endpoint_requestsA
Read-only
Inspect

Fetch the requests captured by a webhook created with webhook_endpoint_create. Returns the newest requests first with method, headers, query params, body payload, and timestamps.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesWebhook id returned by webhook_endpoint_create
limitNoMaximum number of requests to return (1-100, default: 20)

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
requestsNo
expires_atNo
request_countNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is known. The description adds useful behavioral detail: newest-first ordering and the specific fields returned (method, headers, query params, body, timestamps).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, focused sentence that front-loads the verb and resource, with no unnecessary words. It conveys purpose, source, ordering, and return contents efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description does not need to detail return shapes. It covers the purpose, relationship to webhook_endpoint_create, ordering, and data fields, which is sufficient for correct selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema covers 100% of parameters with clear descriptions, so the baseline is 3. The description only reinforces the source of the id and does not add significant parameter-specific semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Fetch') and resource ('requests captured by a webhook created with webhook_endpoint_create'), clearly distinguishing this tool from sibling tools. It names the exact object and its source.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description references webhook_endpoint_create, establishing the prerequisite and making it clear this is the retrieval counterpart. It does not explicitly list alternatives or when-not-to-use, but the context is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

web_security_auditA
Read-only
Inspect

Run a comprehensive web security audit combining headers, SSL, CORS, and cookies checks — then use an LLM to produce a prioritised remediation plan. Orchestrates security_headers_check + ssl_certificate_check + cors_test + cookie_security_audit in parallel, merges all findings, then asks an AI model to: (1) rank vulnerabilities by real-world exploitability, (2) generate a remediation roadmap, (3) produce fix code snippets for the detected stack. Returns both raw audit data and the AI analysis. Use this as a one-click security posture assessment.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesFull URL to audit (e.g. https://example.com)
modelNoLLM model for AI analysis (default: "openai/gpt-oss-20b"). Set to "none" to skip AI analysis.
api_keyNoYour Groq or HuggingFace API key. Required to enable AI analysis.

Output Schema

ParametersJSON Schema
NameRequiredDescription
fixNo
keyNo
urlNo
nameNo
weakNo
gradeNo
scoreNo
testsNo
valueNo
headerNo
issuesNo
secureNo
weightNo
cookiesNo
detailsNo
messageNo
missingNo
httpOnlyNo
sameSiteNo
risk_levelNo
weak_countNo
cookies_foundNo
missing_countNo
overall_gradeNo
origins_testedNo
total_findingsNo
headers_checkedNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that it runs the four checks in parallel, merges findings, invokes an LLM to rank vulnerabilities, generate a remediation roadmap, and produce fix snippets, and returns both raw audit data and AI analysis. Annotations already cover read-only/non-destructive behavior, and the description adds valuable orchestration and LLM behavior without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences with a front-loaded main purpose and a structured enumerated list of the AI analysis outputs. The length is justified for an orchestrator of this complexity, though it could be slightly tightened without losing value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a high-complexity orchestrator, it covers the sub-checks performed, the parallel execution model, the AI analysis outputs, and the combined return payload; the output schema handles return-value details. It does not mention failure behavior, timing, or cost, but it is sufficiently complete for tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the schema already documents url, model, and api_key with defaults and the 'none' option for skipping AI analysis. The description reinforces the purpose of the model/api_key parameters but does not add semantics beyond what the schema provides, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it runs a comprehensive web security audit combining headers, SSL, CORS, and cookies checks, then produces an LLM-generated remediation plan. It explicitly names the exact sub-tools it orchestrates (security_headers_check, ssl_certificate_check, cors_test, cookie_security_audit), distinguishing it from those siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context with 'Use this as a one-click security posture assessment,' which signals when the bundled orchestration is appropriate versus running individual checks. It does not explicitly state when not to use it or name alternatives, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

word_frequencyA
Read-onlyIdempotent
Inspect

Analyze word frequency in text. Returns top N words with counts and percentages. Supports English stopword filtering. Useful for content analysis, keyword extraction, and LLM output analysis.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesText to analyze
top_nNoReturn top N words (default: 20, max: 200)
min_lengthNoMinimum word length to include (default: 3)
remove_stopwordsNoRemove common English stopwords (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
top_wordsNo
total_wordsNo
unique_wordsNo
stopwords_removedNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds useful behavioral details: it returns percentages, supports stopword filtering, and honors a top N limit. It does not go into edge cases like punctuation handling or case sensitivity, but the added context is meaningful beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: the first sentence states the core purpose, the second defines the output, and the third adds filtering behavior and use cases. Every sentence contributes value with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simple nature and the presence of an output schema (which explains return structure), the description covers the essentials: purpose, output, stopword behavior, and typical use cases. It does not explicitly address language scope or normalization details, but these are not critical gaps given the schema and tool simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with each parameter already described (input, top_n, min_length, remove_stopwords). The description reiterates the top N and stopword concepts but does not add new parameter-specific details beyond what the schema provides. Baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb ('Analyze') and resource ('word frequency'), and explicitly describes the output ('Returns top N words with counts and percentages'). It distinguishes itself from sibling tools like text_stats and count_tokens by focusing on word-level frequency analysis.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear use cases ('content analysis, keyword extraction, and LLM output analysis') that help an agent decide when to invoke it. However, it does not explicitly mention when not to use it or name alternative tools, so it stops short of full exclusionary guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

xml_to_jsonA
Read-onlyIdempotent
Inspect

Convert an XML string to a JSON object. Supports attributes, nested elements, arrays, CDATA, and namespaces. Options: parse numbers, parse booleans, ignore attributes.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesXML string to convert
attr_prefixNoPrefix for attribute keys (default: "@_")
ignore_attrsNoIgnore XML attributes (default: false)
parse_valuesNoAuto-parse numbers and booleans (default: true)

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultNo
key_countNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the description does not need to cover safety. It adds useful behavioral context by explaining parsing options (numbers, booleans) and attribute handling, which are non-obvious conversion behaviors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loads the core purpose, and every clause adds value. It is concise and well-structured with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the annotations, output schema, and full parameter coverage, the description is nearly complete. It covers supported XML features and configurable behaviors. It lacks explicit edge-case handling (e.g., malformed XML), but that is acceptable with the output schema present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description mostly restates parameter meanings already present in the schema (e.g., parse_values is paraphrased as 'parse numbers, parse booleans'). It does not add meaningful new semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a clear verb+resource statement: 'Convert an XML string to a JSON object.' It then lists supported features (attributes, nested elements, arrays, CDATA, namespaces) which further clarifies its scope and distinguishes it from sibling conversion tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool by listing supported XML features, but it does not explicitly state when to use it versus alternatives like yaml_to_json or json_to_csv. There are no exclusions or alternative tool references, so guidance is only implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

yaml_to_jsonA
Read-onlyIdempotent
Inspect

Parse a YAML string and return the equivalent JSON value. The reverse of json_to_yaml. Supports nested objects, arrays, anchors, aliases, multi-document streams, and all scalar types. Use when processing config files, CI/CD pipeline definitions, or OpenAPI specs authored in YAML.

ParametersJSON Schema
NameRequiredDescriptionDefault
inputYesYAML string to parse
multiNoIf true, parse all documents in a multi-document stream and return an array (default: false)

Output Schema

ParametersJSON Schema
NameRequiredDescription
jsonNo
countNo
documentsNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool as read-only, idempotent, and non-destructive, so the description does not need to repeat that. It adds behavioral detail about supported YAML features (anchors, aliases, multi-document streams, scalar types), which goes beyond schema information. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no filler. The first sentence states the core action, the second gives relational context, and the third lists use cases and capabilities. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with only two parameters and an output schema present, so the description does not need to explain return values. It covers purpose, usage scenarios, and supported YAML features, making it complete for an agent to decide and invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds general input capabilities (nested objects, arrays, etc.) but does not provide additional parameter-specific meaning beyond what the schema already states for 'input' and 'multi'. There is no need for extra compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Parse') and resource ('YAML string') with a clear output ('equivalent JSON value'). It explicitly names the reverse sibling ('json_to_yaml'), distinguishing itself from related tools like json_to_yaml and xml_to_json.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear use cases ('config files, CI/CD pipeline definitions, or OpenAPI specs authored in YAML') and indicates the tool's relationship to its reverse. While it does not explicitly state when NOT to use it, the reverse reference gives implicit exclusion, which is sufficient for a straightforward conversion tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 1 tool update
    • Changedgenerate_ci_workflow4 fields changed
      • addedInput schema / properties / app_base_url
        Added value: +{
        +  "description": "URL the drift gate waits for before capturing (default: http://127.0.0.1:3000). Must match config.baseUrl in .ia-qa/config.json.",
        +  "type": "string"
        +}
      • addedInput schema / properties / app_start_command
        Added value: +{
        +  "description": "Command that boots the app for the selector_drift gate, e.g. \"npm run start:ci\". Cannot be guessed — omitted, the step carries an explicit TODO and the note says so, because a plausible default would silently map nothing.",
        +  "type": "string"
        +}
      • changedInput schema / properties / gate / description
        Previous value: -"Which gate to emit. eval_contract = LLM eval via the action (default). cli_checks = deterministic CLI assertions. both = CLI checks first, eval last."New value: +"Which gate to emit. eval_contract = LLM eval via the action (default). cli_checks = deterministic CLI assertions. selector_drift = an E2E selector-drift gate via @ia-qa/self-healing (boots the app, captures, diffs against the committed baseline, branches on exit code 0/1/2). both = CLI checks + eval. all = CLI checks, then drift, then eval."
      • changedInput schema / properties / gate / enum
        Previous value: -[
        -  "eval_contract",
        -  "cli_checks",
        -  "both"
        -]New value: +[
        +  "eval_contract",
        +  "cli_checks",
        +  "both",
        +  "selector_drift",
        +  "all"
        +]
  2. 2 tool updates
    • Changedget_testing_guidelines1 field changed
      • changedInput schema / properties / topic / enum
        Previous value: -[
        -  "llm-unit-testing",
        -  "rag-pipeline",
        -  "prompt-stability",
        -  "prompt-ab-testing",
        -  "embedding-quality",
        -  "eval-framework",
        -  "semantic-testing",
        -  "auto-testing",
        -  "security",
        -  "api-testing",
        -  "ci-cd",
        -  "multimodal",
        -  "llm-data-security",
        -  "agent-observability",
        -  "pro-tips",
        -  "learning-paths",
        -  "golden-dataset"
        -]New value: +[
        +  "llm-unit-testing",
        +  "rag-pipeline",
        +  "prompt-stability",
        +  "prompt-ab-testing",
        +  "embedding-quality",
        +  "eval-framework",
        +  "semantic-testing",
        +  "auto-testing",
        +  "security",
        +  "api-testing",
        +  "ci-cd",
        +  "multimodal",
        +  "llm-data-security",
        +  "agent-observability",
        +  "pro-tips",
        +  "learning-paths",
        +  "golden-dataset",
        +  "selector-drift"
        +]
    • Addedsandbox_scenario
  3. 1 tool update
    • Changednormalize_whitespace2 fields changed
      • changedInput schema / properties / collapse_blanks / description
        Previous value: -"Collapse 3+ consecutive blank lines to 2 (default: true)"New value: +"Collapse runs of blank lines down to max_blank_lines (default: true)"
      • addedInput schema / properties / max_blank_lines
        Added value: +{
        +  "description": "Blank lines to keep when collapsing, 0-10 (default: 2)",
        +  "type": "number"
        +}
  4. 1 tool update
    • Changedmultimodal_eval_guide1 field changed
      • addedOutput schema / properties / score_interpretation
        Added value: +{}
  5. 3 tool updates
    • Changedgenerate_test_cases5 fields changed
      • changedInput schema / properties / feature / description
        Previous value: -"Feature or function to test. Be specific: describe inputs, expected behaviour, context."New value: +"Feature or function to test. Be specific: describe inputs, expected behaviour, context. Constraints stated here (\"password must be at least 8 characters\") are used when the sentence names exactly one field."
      • changedInput schema / properties / inputs / description
        Previous value: -"Optional: list of input parameters (one per line, e.g. \"email: string [required]\")"New value: +"Optional: list of input parameters (one per line, e.g. \"email: string [required]\", \"password: string [required, min 8 chars]\", \"age: number [18-99]\")"
      • addedOutput schema / properties / gherkinFormat
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / gherkinScenarioCount
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / parsedInputs
        Added value: +{
        +  "items": {
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Changedshield_analyze1 field changed
      • changedInput schema / properties / prompt / description
        Previous value: -"Optional original prompt (used for quality scoring and injection detection)"New value: +"Optional original prompt. Used for quality scoring AND scanned for prompt injection in its own right (checks.injection_prompt) — pass it whenever you have it, it is where the attack actually lands."
    • Changedwebhook_endpoint_create1 field changed
      • changedInput schema / properties / base_url / description
        Previous value: -"Optional public base URL. Default: https://ia-qa.com/mcp/webhook"New value: +"Optional public base URL. Default: https://www.ia-qa.com/mcp/webhook (the apex ia-qa.com answers 301 and a redirected POST loses its body, so an apex base_url is normalized to www)"
  6. 6 tool updates
    • Changedllm_generate1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Model ID (default: \"qwen/qwen3-32b\"). Server-keyed whitelist only — Groq: qwen/qwen3-32b, llama-3.3-70b-versatile, meta-llama/llama-4-scout-17b-16e-instruct, llama-3.1-8b-instant; HuggingFace: Qwen/Qwen3-32B, meta-llama/Llama-3.3-70B-Instruct, deepseek-ai/DeepSeek-R1, google/gemma-3-27b-it, and more. Other ids from list_llm_models are BYOK-only and will be rejected."New value: +"Model ID (default: \"openai/gpt-oss-20b\"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.6-27b; HuggingFace: Qwen/Qwen3-32B, meta-llama/Llama-3.3-70B-Instruct, deepseek-ai/DeepSeek-R1, google/gemma-3-27b-it, and more. Other ids from list_llm_models are BYOK-only and will be rejected."
    • Changedrerank_evaluate15 fields changed
      • addedInput schema / properties / api_key
        Added value: +{
        +  "description": "Your NVIDIA API key (BYOK), used only when no passage carries a score. Transits RAM for the single call, never stored.",
        +  "type": "string"
        +}
      • addedInput schema / properties / passages / items / properties / score
        Added value: +{
        +  "description": "Relevance score from YOUR reranker. Present on every passage → ranking is done from these (offline, no key). Higher = more relevant. Score all passages or none.",
        +  "type": "number"
        +}
      • addedOutput schema / properties / has_ground_truth
        Added value: +{
        +  "type": "boolean"
        +}
      • addedOutput schema / properties / k
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / latency_ms
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / mode
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / precision_at_k
        Added value: +{
        +  "type": "number"
        +}
      • removedOutput schema / properties / query
        Removed value: -{
        -  "type": "string"
        -}
      • addedOutput schema / properties / ranked
        Added value: +{
        +  "items": {
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • addedOutput schema / properties / recall_at_k
        Added value: +{
        +  "type": "number"
        +}
      • removedOutput schema / properties / results
        Removed value: -{
        -  "items": {
        -    "type": "object"
        -  },
        -  "type": "array"
        -}
      • addedOutput schema / properties / threshold
        Added value: +{
        +  "type": "number"
        +}
      • removedOutput schema / properties / top_n
        Removed value: -{
        -  "type": "number"
        -}
      • addedOutput schema / properties / total_passages
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / verdict
        Added value: +{
        +  "type": "string"
        +}
    • Changedshield_analyze1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"LLM model for AI-powered deep analysis (default: \"qwen/qwen3-32b\"). Set to \"none\" to skip LLM check. Supports any model from list_llm_models."New value: +"LLM model for AI-powered deep analysis (default: \"openai/gpt-oss-20b\"). Set to \"none\" to skip LLM check. Supports any model from list_llm_models."
    • Changedtest_skill1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"LLM model ID to use for both scenario generation and testing (e.g. gpt-4o-mini, claude-3-5-haiku-20241022). Defaults to llama-3.3-70b-versatile (Groq, server key)."New value: +"LLM model ID to use for both scenario generation and testing (e.g. gpt-4o-mini, claude-3-5-haiku-20241022). Defaults to openai/gpt-oss-20b (Groq, server key)."
    • Changedvalidate_agent_trajectory7 fields changed
      • removedOutput schema / properties / assertions
        Removed value: -{
        -  "items": {
        -    "type": "object"
        -  },
        -  "type": "array"
        -}
      • removedOutput schema / properties / failed
        Removed value: -{
        -  "type": "number"
        -}
      • removedOutput schema / properties / passed
        Removed value: -{
        -  "type": "number"
        -}
      • addedOutput schema / properties / results
        Added value: +{
        +  "items": {
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
      • addedOutput schema / properties / summary
        Added value: +{
        +  "additionalProperties": true,
        +  "type": "object"
        +}
      • removedOutput schema / properties / total
        Removed value: -{
        -  "type": "number"
        -}
      • addedOutput schema / properties / warnings
        Added value: +{
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
    • Changedweb_security_audit1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"LLM model for AI analysis (default: \"qwen/qwen3-32b\"). Set to \"none\" to skip AI analysis."New value: +"LLM model for AI analysis (default: \"openai/gpt-oss-20b\"). Set to \"none\" to skip AI analysis."
  7. 1 tool update
    • Changedhttp_status_lookup1 field changed
      • addedOutput schema / properties / registered
        Added value: +{
        +  "type": "boolean"
        +}
  8. 1 tool update
    • Changedtoxicity_scan1 field changed
      • addedOutput schema / properties / method
        Added value: +{
        +  "type": "string"
        +}
  9. 2 tool updates
    • Changedcors_test6 fields changed
      • addedOutput schema / properties / error
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / origins_conclusive
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / origins_inconclusive
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / origins_reachable
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / risk_level / enum
        Added value: +[
        +  "safe",
        +  "low",
        +  "medium",
        +  "high",
        +  "critical",
        +  "unknown"
        +]
      • addedOutput schema / properties / warning
        Added value: +{
        +  "type": "string"
        +}
    • Changedregex_test3 fields changed
      • addedOutput schema / properties / elapsed_ms
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / redos_detected
        Added value: +{
        +  "type": "boolean"
        +}
      • addedOutput schema / properties / verdict
        Added value: +{}
  10. 4 tool updates
    • Changedhallucination_check2 fields changed
      • addedOutput schema / properties / contradictions
        Added value: +{}
      • addedOutput schema / properties / unbound_claims
        Added value: +{}
    • Changedprompt_injection_scan2 fields changed
      • addedOutput schema / properties / notes
        Added value: +{}
      • addedOutput schema / properties / quoted_detections
        Added value: +{
        +  "type": "number"
        +}
    • Changedsecret_scan2 fields changed
      • changedInput schema / properties / types / description
        Previous value: -"Comma-separated types to scan (default: all). Options: aws, gcp, azure, openai, anthropic, stripe, github, gitlab, slack, twilio, sendgrid, huggingface, jwt, private_key, connection_string, bearer, basic_auth"New value: +"Comma-separated families to scan (default: all): aws, gcp, azure, openai, anthropic, huggingface, github, gitlab, stripe, slack, twilio, sendgrid, jwt, private_key, connection_string, bearer, basic_auth, generic. Individual pattern names (e.g. \"aws_access_key\", \"github_fine\") are also accepted. An unknown value is rejected with an error — a scoped scan never silently returns \"clean\"."
      • addedOutput schema / properties / scanned_types
        Added value: +{}
    • Changedvalidate_email1 field changed
      • addedOutput schema / properties / reason
        Added value: +{
        +  "type": "string"
        +}
  11. 1 tool update
    • Changedmetamorphic_check1 field changed
      • changedInput schema / properties / require_all / description
        Previous value: -"If true (default), every gated variant must pass. If false, one passing gated variant is enough."New value: +"If true (default), every gated variant must pass. KEEP THE DEFAULT for any run you gate on. Setting it false is not a tolerance dial but an off switch: relations have asymmetric pass rates (a typo variant usually scores ~1.0 because the answer really is identical), so one trivial row is enough to hold the whole run at PASS while a paraphrase fails. When that happens the result carries an explicit warning naming the failed rows."
  12. 1 tool update
    • Addedmetamorphic_check
  13. 1 tool update
    • Changedrate_tool3 fields changed
      • changedInput schema / properties / comment / description
        Previous value: -"Strongly encouraged — explain what you were trying to do and whether the tool got you there. Be specific about what was missing, wrong, or a poor fit. This is the most valuable part of the rating (max 500 chars)."New value: +"Strongly encouraged — explain what you were trying to do and whether the tool got you there. Be specific about what was missing, wrong, or a poor fit. This is the most valuable part of the rating. Up to 2000 chars are stored; go over and the response says so (truncated: true) — send the remainder as a second call rather than assuming it landed."
      • addedOutput schema / properties / stored_chars
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / truncated
        Added value: +{
        +  "type": "boolean"
        +}
  14. 13 tool updates
    • Changedextract_json_path2 fields changed
      • changedInput schema / properties / input / description
        Previous value: -"A valid JSON string to traverse"New value: +"The JSON to traverse — a JSON string, or the object/array itself."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedflatten_json2 fields changed
      • changedInput schema / properties / input / description
        Previous value: -"JSON string to flatten or unflatten"New value: +"The JSON to flatten or unflatten — a JSON string, or the object itself."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedformat_table2 fields changed
      • changedInput schema / properties / input / description
        Previous value: -"JSON array of objects to convert to a Markdown table"New value: +"The array of objects to render — a JSON string, or the array itself."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedjson_diff4 fields changed
      • changedInput schema / properties / after / description
        Previous value: -"Modified JSON string (after)"New value: +"The modified JSON (after) — a JSON string, or the value itself."
      • changedInput schema / properties / after / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
      • changedInput schema / properties / before / description
        Previous value: -"Original JSON string (before)"New value: +"The original JSON (before) — a JSON string, or the value itself."
      • changedInput schema / properties / before / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedjson_schema_generate2 fields changed
      • changedInput schema / properties / input / description
        Previous value: -"Sample JSON value (object, array, or scalar) to infer the schema from"New value: +"The sample JSON value to infer the schema from — a JSON string, or the value itself."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedjson_schema_validate4 fields changed
      • changedInput schema / properties / schema / description
        Previous value: -"JSON Schema as a JSON string"New value: +"The JSON Schema — a JSON string, or the schema object itself."
      • changedInput schema / properties / schema / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
      • changedInput schema / properties / value / description
        Previous value: -"JSON string to validate"New value: +"The JSON value to validate — a JSON string, or the value itself."
      • changedInput schema / properties / value / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedjson_to_csv2 fields changed
      • changedInput schema / properties / input / description
        Previous value: -"JSON string containing an array of objects"New value: +"The array of objects to convert — a JSON string, or the array itself."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedjson_to_yaml2 fields changed
      • changedInput schema / properties / input / description
        Previous value: -"JSON string to convert to YAML"New value: +"The JSON to convert to YAML — a JSON string, or the value itself."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedmerge_json4 fields changed
      • changedInput schema / properties / base / description
        Previous value: -"Base JSON object (will be merged into)"New value: +"The base JSON object (merged into) — a JSON string, or the object itself."
      • changedInput schema / properties / base / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
      • changedInput schema / properties / override / description
        Previous value: -"Override JSON object (takes precedence)"New value: +"The override JSON object (takes precedence) — a JSON string, or the object itself."
      • changedInput schema / properties / override / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedmock_from_schema2 fields changed
      • changedInput schema / properties / schema / description
        Previous value: -"JSON Schema as a JSON string"New value: +"The JSON Schema to generate from — a JSON string, or the schema object itself."
      • changedInput schema / properties / schema / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedopenapi_validate2 fields changed
      • changedInput schema / properties / input / description
        Previous value: -"OpenAPI 3.x specification as a JSON or YAML string"New value: +"The OpenAPI 3.x spec — a JSON string, a YAML string, or the already-parsed spec object."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedtransform_json_array2 fields changed
      • changedInput schema / properties / input / description
        Previous value: -"JSON string containing an array (or object with an array at path)"New value: +"The JSON containing an array (or an object with an array at `path`) — a JSON string, or the value itself."
      • changedInput schema / properties / input / type
        Previous value: -"string"New value: +[
        +  "string",
        +  "object",
        +  "array"
        +]
    • Changedvector_stats3 fields changed
      • addedInput schema / anyOf
        Added value: +[
        +  {
        +    "required": [
        +      "vector"
        +    ]
        +  },
        +  {
        +    "required": [
        +      "matrix"
        +    ]
        +  }
        +]
      • changedInput schema / properties / matrix / description
        Previous value: -"Matrix of vectors (overrides vector). Returns per-vector + matrix-level stats."New value: +"Matrix of vectors (overrides vector). Returns per-vector + matrix-level stats. Required unless `vector` is given."
      • changedInput schema / properties / vector / description
        Previous value: -"Single vector to analyze"New value: +"Single vector to analyze. Required unless `matrix` is given."
  15. 1 tool update
    • Changedfix_gherkin2 fields changed
      • changedInput schema / properties / api_key / description
        Previous value: -"Your LLM provider API key."New value: +"Your own LLM provider API key (BYOK) — OpenAI \"sk-…\", Anthropic \"sk-ant-…\", Google \"AIzaSy…\", or Groq \"gsk_…\". There is no server-side key for this tool: if you do not have one, do not call it and do not invent a value — placeholders like \"configured\", \"your_api_key\" or a masked \"sk-…***…\" are rejected. Used for this call only, never stored."
      • changedInput schema / properties / model / description
        Previous value: -"LLM model to use for the fix, e.g. \"gpt-4o-mini\"."New value: +"LLM model to use for the fix, e.g. \"gpt-4o-mini\". Must belong to the provider whose key you passed in api_key."
  16. 1 tool update
    • Addedgenerate_ci_workflow
  17. 3 tool updates
    • Changedanalyze_diff_bugs7 fields changed
      • addedOutput schema / properties / disclaimer
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / notAnalysed
        Added value: +{
        +  "items": {
        +    "type": "string"
        +  },
        +  "type": "array"
        +}
      • addedOutput schema / properties / overallRisk
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / rulesApplied
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / scannedLines
        Added value: +{
        +  "type": "number"
        +}
      • removedOutput schema / properties / summary
        Removed value: -{
        -  "type": "string"
        -}
      • addedOutput schema / properties / totalSuggestions
        Added value: +{
        +  "type": "number"
        +}
    • Changedfind_tool3 fields changed
      • addedInput schema / properties / max_results
        Added value: +{
        +  "description": "Maximum tools to return (default 10, max 50). Results are ranked by IDF-weighted relevance, so common words like \"test\" do not inflate the list.",
        +  "type": "number"
        +}
      • addedOutput schema / properties / total_matches
        Added value: +{
        +  "type": "number"
        +}
      • addedOutput schema / properties / truncated
        Added value: +{
        +  "type": "boolean"
        +}
    • Changedrun_pr_gate_pipeline4 fields changed
      • addedOutput schema / properties / disclaimer
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / inputFormat
        Added value: +{
        +  "type": "string"
        +}
      • addedOutput schema / properties / notAnalysed
        Added value: +{
        +  "type": "array"
        +}
      • addedOutput schema / properties / riskFactors
        Added value: +{
        +  "type": "array"
        +}
  18. 4 tool updates
    • Changedformat_json1 field changed
      • changedInput schema / properties / input / description
        Previous value: -"Raw JSON string to format"New value: +"A raw JSON string, e.g. '{\"key\":\"value\"}'. Must already parse as JSON — plain text or truncated JSON is rejected, not repaired."
    • Changedlatency_benchmark1 field changed
      • changedInput schema / properties / endpoints / type
        Previous value: -"array"New value: +[
        +  "string",
        +  "array"
        +]
    • Changedllm_generate1 field changed
      • changedInput schema / properties / model / description
        Previous value: -"Model ID (default: \"qwen/qwen3-32b\"). Use list_llm_models tool with provider \"Groq\" or \"HuggingFace\" to see available models."New value: +"Model ID (default: \"qwen/qwen3-32b\"). Server-keyed whitelist only — Groq: qwen/qwen3-32b, llama-3.3-70b-versatile, meta-llama/llama-4-scout-17b-16e-instruct, llama-3.1-8b-instant; HuggingFace: Qwen/Qwen3-32B, meta-llama/Llama-3.3-70B-Instruct, deepseek-ai/DeepSeek-R1, google/gemma-3-27b-it, and more. Other ids from list_llm_models are BYOK-only and will be rejected."
    • Changedrun_eval_contract1 field changed
      • changedInput schema / properties / inline_contract / description
        Previous value: -"Raw contract object (alternative to contract_path)"New value: +"Raw contract object (alternative to contract_path). Must contain top-level \"metadata\" ({name, version, model?, provider?}), \"expectations\" ({min_score?}), and \"scenarios\" ([{id, input, ground_truth?}]) — scenarios alone are rejected. Use generate_eval_yaml to scaffold one."

Frequently Asked Questions

Discussions

No comments yet. Be the first to start the discussion!

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    Not graded
    maintenance
    Search and discover 500+ tools, APIs, and services for AI agents. Browse 15 categories, get recommendations, and access structured metadata including auth methods, free tiers, and example calls.
    1
    -
  • A
    license
    Not graded
    quality
    A
    maintenance
    Zero-setup safety toolkit for AI coding agents with 16 built-in tools for context gathering, safe file editing, validation, and session memory. Features rollback, circuit breaker, sandbox, timeout, and dangerous pattern blocking — no config, no API key, no database required.
    83
    2
    MIT
Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A3.6/5.0
Disambiguation2/5

Multiple tools overlap significantly: compare_models/llm_fit_finder/model_info/list_llm_models all compare models; similarity_score/embedding_similarity/run_semantic_tests all measure text similarity; detect_secrets/secret_scan/analyze_diff_bugs/pr_gatekeeper all scan for secrets. Descriptions attempt to differentiate, but the boundaries between many tools are unclear, making selection error-prone.

Naming Consistency4/5

The vast majority of tools follow a snake_case verb_noun pattern (validate_email, generate_uuid, parse_csv), making the set mostly predictable. A few notable deviations exist (pr_gatekeeper, llm_fit_finder, cot_analyzer, jira_to_test_suite, needle_haystack_generate) but they are the exception rather than the rule.

Tool Count1/5

With 149 tools, this set is far beyond the 50+ threshold for an extreme mismatch. Even as a general-purpose QA & Dev toolkit, the sheer number overwhelms and exceeds any reasonable scope, making discovery and selection impractical.

Completeness4/5

The toolkit covers an impressively broad range: text processing, LLM evaluation, security auditing, web checks, MCP validation, Jira/Confluence integration, and more. Minor gaps exist, such as missing delete/update for webhooks and Confluence pages, and no create/update for Jira issues, but these are workable around.

Resources