M00N Report MCP
OfficialServer Quality Checklist
Latest release: v1.5.4
- Disambiguation4/5
Tools are grouped around distinct resources—cases, folders, collections, executions, launches, releases, external links, and analytics—and the descriptions explicitly call out near-misses like test collections vs executions and launch statistics vs launch trends. A few pairs such as add_cases_to_collection vs add_cases_to_execution still need careful reading, but every tool has a distinct purpose.
Naming Consistency5/5Naming is consistently verb-first snake_case with standard CRUD verbs: get/list/create/update/delete/add/remove/link/link. Even compound tools like bulk_update_cases, run_health_check, and report_case_result follow the same readable pattern with no style mixing.
Tool Count2/555 tools is far beyond the 3–15 well-scoped range and clearly falls into the too-many category, even though the underlying domain is broad. The count will overwhelm agents during tool selection and makes the server hard to use effectively.
Completeness4/5Major lifecycle coverage is strong: cases, folders, suites, collections, executions, releases, launches, external links, and reports all have meaningful create/read/update/delete or equivalent workflows. Minor reverse-operation gaps exist—such as no unlink for autotest-to-case and no way to remove a single loose case from an execution group—but agents can generally work around them.
Average 4.7/5 across 55 of 55 tools scored. Lowest: 4/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 26 commits in the last 12 weeks
- Last stable release on
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
Tools from this server were used 2 times in the last 30 days.
This repository includes a glama.json configuration file.
This server has been verified by its author.
How to sync the server with GitHub?
Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.
To manually sync the server, click the "Sync Server" button in the MCP server admin interface.
How is the quality score calculated?
The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).
Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.
Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).
Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.
Tool Scores
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already establish that this is read-only, idempotent, and non-destructive. The description adds only 'given entity' and 'optionally filtered by provider', which are also visible in the schema. It does not mention result shape, pagination, or other runtime behavior beyond what the call itself implies.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the verb, object, scope, and optional filter. There is no filler and no repetition of obvious schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
It is largely sufficient for a simple read-only listing tool, especially with strong annotations and a small well-understood parameter set. However, there is no output schema and the description doesn't describe the shape of the links returned or any pagination/filtering behavior, so it leaves a small gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%, with entityId and provider already described and entityType constrained by enum. The description loosely mirrors these with 'given entity' and 'filtered by provider', but it adds no substantive semantic detail beyond the schema's existing documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and names a specific resource: 'external ticket links for a given entity'. It also notes the optional provider filter, making the tool's scope immediately clear and distinguishing it from add_external_link and remove_external_link.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: to use this tool when they want to display external ticket links for an entity, optionally narrowed by provider. It doesn't explicitly name alternatives or state when this tool should be avoided, so it falls just short of full guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses behavioral traits beyond the annotations: duplicate cases are skipped, and the cases must exist and belong to the same project. These are valuable operational details that align with, and enrich, the idempotentHint=true annotation. It does not specify failure behavior for invalid cases or collection, but the core behavior is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with no filler. The core operation is front-loaded, and the caveats are stated briefly and precisely. Every sentence provides useful action-oriented information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter mutating operation, the description is largely sufficient: it gives the target, the precondition, and the duplicate-skipping behavior. With idempotentHint=true and destructiveHint=false already supplied, the description covers most operational concerns. It does not describe what happens if a case belongs to another project or if the collection is missing, but that is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema coverage is 100%, so the schema already explains parameter formats and meanings. The description adds operational meaning such as duplicate handling and project membership, but it does not need to restate parameter semantics. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action (add), the object (test cases), and the target (an existing test collection). This differentiates it from related tools like add_cases_to_execution and remove_cases_from_collection without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states when to use this tool: when adding test cases to an existing collection. The preconditions that cases must exist in the same project help clarify correct use. It does not explicitly exclude alternatives or name sibling tools, but the target resource is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already provide the safety profile (read-only false, destructive false, idempotent true), and the description mostly reinforces that moves stay within the same project. The 'different folder' and 'same project' scope adds a small amount of behavioral context, but doesn't disclose anything beyond the schema constraints or annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one concise, front-loaded sentence that states the action, the scope, and the exception. It does not repeat the schema or add filler; every words earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter move operation with no output schema, the description effectively captures what the tool does and its main boundary condition. A bit more detail about expected result or failure behavior would be nice, but the annotations already carry the idempotent and destructive-profile information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents both parameters completely (caseId and targetFolderId, with their types and the same-project requirement). The description just restates the folder constraint and adds no extra detail about parameter values, formats, or derived behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Move'), a clear resource ('test case'), and an explicit target ('a different folder within the same project'). It also names the key limitation ('Cross-project moves are not supported'), which distinguishes this tool from any generic update or move operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It states exactly when to use the tool: moving a test case to another folder within the same project. It also explicitly says when not to use it: cross-project moves are not supported. No alternative sibling exists for moving a test case, so no further routing guidance is needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the idempotent and non-destructive annotations, the description discloses an important behavioral trait: 'Only provided fields are updated; omitted fields remain unchanged.' This is critical for correctly invoking a partial update and prevents unintended overwrites. It also implies a stateless merge behavior, adding value over the annotations alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences with no filler. The primary action is front-loaded, followed by the update semantics and a helpful get-first instruction, making every sentence valuable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter update tool with 100% schema coverage and meaning-bearing annotations, the description supplies the missing execution model: partial update, preserving omitted values, and for prospective state. It does not describe the response shape, but no output schema is defined and the description already gives enough to invoke the tool safely. A slight esponse-type note would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter already has a clear description, including enum restrictions for status and priority. The description adds that fields are updated only if present, but it does not need to go into parameter details. Given the 100% schema coverage, the baseline is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Update test collection metadata.' It clearly distinguishes from sibling tools like update_folder and update_test_case by honoring collection metadata. It is not a tautology and gives an immediate sense of the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool and a key precondition: 'Use get_test_collection first to see current values.' It also clarifies that only provided fields are updated, so an agent knows to send only the fields that need changing. It does not explicitly exclude alternatives, but the guidance is sufficient for a read-before-patch workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description adds valuable behavioral details: the operation is a soft delete, the case can be restored via the UI, and connected autotests are unlinked. These side effects are not visible from annotations alone and help the agent predict the full impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences with no wasted words. The core operation is front-loaded, followed by the restore path and the important side effect of unlinking autotests.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter mutation with annotations, this description is sufficient. It explains what happens, how it can be undone, and what additional effects occur, so an agent can safely predict the behavior without an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the only parameter, caseId, is already documented as 'Numeric test case ID to archive'. The tool description adds no additional parameter meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the resource ('a test case') and the specific behavior ('Soft delete (archive)'). It also separates it from sibling deletion tools by emphasizing archiving and restoration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use case is implied: call this when you want to archive a test case, and the restoration note implies the operation is reversible. However, it does not explicitly mention alternatives or when not to use it, such as removing a case from a collection instead of deleting it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond the annotations. Although destructiveHint and idempotentHint already mark the operation as destructive and repeatable, the description clarifies the exact nature: it is an archive/soft delete, the collection is recoverable via UI, and test executions remain intact. This materially refines the meaning of the destructive hint and helps the agent avoid assuming irreversible deletion.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences deliver high-value information with no redundant words. The most important fact, 'soft delete', is front-loaded, and the following sentences add critical behavioral nuance about restoration and preserved executions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a simple one-parameter mutation. It explains the operation type, reversibility, and impact on test executions. There is no output schema, so the absence of return-format details is acceptable. The core behavior an agent needs to invoke this tool safely and correctly is fully covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is only one parameter and the schema description coverage is 100%. The schema already explains that collectionId is the test collection UUID to archive. The tool description does not add further parameter-specific detail, which is acceptable because the schema carries that burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific operation: 'Archive (soft delete) a test collection'. It identifies both the verb and the resource, and explicitly clarifies it is a soft delete. This distinguishes it from the sibling delete_* tools, especially delete_test_case and delete_execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly name alternative tools or state when not to use it, but it gives clear behavioral context: restoration via the UI and preservation of test executions. For an agent deciding whether to use this tool, the 'soft delete' framing and reversibility strongly imply this is the safer deletion pathway.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already establish a non-read-only, idempotent, non-destructive mutation, so the description's added value is clarifying that launches are linked rather than copied. This gives the agent a meaningful model of what operation occurs without being deep about partial failures or return values.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main action, and the second sentence earns its place by explaining the reference-as-copy distinction. There is no filler or repeated schema content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple mutation with two fully documented parameters and annotations covering idempotence and destructiveness, the description supplies the essential semantic distinction needed for correct invocation. Nothing import is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already explains both releaseId and launchIds, including where launchIds should come from. The description adds no parameter-specific detail, so it meets the baseline without needing to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Link launches to a release.' It further separates itself from Test Collections by explaining that launches are linked by reference rather than copied, so an agent can select this over related collection or copy workflows.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context that a reference link is being created, and explicitly contrasts this with Test Collection copy semantics. It does not explicitly name alternative tools or state 'when not to use', but the reference-versus-copy distinction is enough to guide tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description's 'Remove' matches the annotations' destructiveHint=true and readOnlyHint=false, so there is no contradiction. It adds that the link is an external ticket link and that valid identifiers are the UUIDs produced by specific companion tools. However, it does not add much behavioral detail beyond what the annotations already express, such as irreversibility or failure behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
One clear, front-loaded sentence communicates the operation, the target, and the source of the required identifier. There is no filler or redundant restating of the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter destructive tool with an informative annotation set and no output schema, this is sufficiently complete. The annotations establish that it is destructive and non-read-only, and the description supplies the remaining practical context: what kind of link and how to obtain its UUID.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description compensates by explaining that linkId is a link UUID and, more usefully, that it comes from add_external_link or list_external_links. This gives an agent concrete guidance on where to find a valid value rather than simply naming the field.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Remove'), a clear resource ('external ticket link'), and the exact key needed to identify it (link UUID). It also names the source tools that produce the UUID (add_external_link or list_external_links), which disambiguates it from related link-based siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description effectively tells the agent when this tool is appropriate: when there is an external link UUID, and it explains how to obtain that UUID. It does not explicitly state when not to use it or name alternatives for non-removal operations, but the single-job scope is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description reveals important behavioral characteristics not present in the annotations: this is pattern matching, not model output, and the results are scaffolding that require human/AI refinement before persistence. The readOnlyHint already signals safety, but the description adds the expectation that results are provisional and not final artifacts. This meaningfully helps the agent decide how to use the output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a compact two-sentence definition that front-loads the tool's purpose and then adds just enough behavioral and next-step context. Every clause serves a clear function: output expectation, pattern-matching caveat, and handoff to its sibling. There is no padding or reiteration of schema fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the fully documented input schema, read-only annotation, and simple output shape stated in the description, the tool is fully scoped. The description even covers what to do with the returned scaffolding, which is essential for a suggestion-only tool. Nothing critical is missing for an agent to correctly invoke it and handle the output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all four parameters with 100% coverage, including defaults and constraints. The description adds minimal extra parameter semantics, mostly reinforcing that featureDescription is the source input and projectId makes suggestions project-aware. That is helpful but only tangentially, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as returning template-based test case scaffolding for a feature description, and specifies the output format (titles, steps, priorities). It explicitly contrasts itself with model output and create_test_case, which helps an agent understand its distinct role among siblings. The verb 'Return' plus the resource 'test case scaffolding' makes the core purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage context: use this tool to generate project-aware starting suggestions, refine them yourself, then persist them with create_test_case. It implicitly communicates that create_test_case is the follow-on tool for actual creation, but it does not enumerate situations where this tool should be avoided or other alternatives preferred. This is strong but not fully exhaustive guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true. The description adds useful behavioral transparency by stating that only the link is removed and the launch itself remains unaffected, which prevents an agent from assuming the launch is deleted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no filler or redundant phrasing. The core action is front-loaded and the clarifying caveat about the launch being unaffected is valuable and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, two-parameter unlink operation, the description covers the action, the object being modified, and the critical non-effect on the launch itself. The annotations and schema carry the remaining safety and input details, so nothing essential appears to be missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully documents both releaseId and launchId with clear descriptions, so the description does not need to add parameter-level meaning. The description adds nothing beyond what the schema provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action: removing an automated launch from a release. It also clarifies a key distinction by noting that the launch itself is not affected, which helps separate this from destructive launch deletion or removal tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context about the operation's effect, removing the association while preserving the launch. It does not explicitly name alternatives such as link_launches_to_release, but the inverse relationship is strongly implied by the phrasing and sibling tool list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
It discloses the partial-update behavior and states that the full updated case object including shareUrl is returned, which is useful beyond the annotations. The annotations already mark it as non-read-only, idempotent, and potentially destructive, and the description does not contradict those hints. It adds context without requiring the agent to infer behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences convey purpose, behavioral nuance, return value, and a useful practical tip. There is no filler or repetition. The format is efficiently front-loaded with the essential action and semantics.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 9 parameters and no output schema, the description includes the key guidance: partial update behavior, current value retrieval, and return shape. Since all parameters are fully documented in the schema, the description does not need to restate them. Nothing critical about calling this tool is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the input schema already documents each parameter clearly. The description adds no per-parameter details, which is acceptable since the schema is rich. The partial-update note adds some context for all optional fields but does not reduce the need to rely on the schema for param-specific semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the operation ('Update an existing test case') with a specific verb and resource. 'Existing' distinguishes it from create_test_case and delete_test_case, and the scope is unambiguous. No ambiguity remains regarding what the tool acts on.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives practical usage context, especially the tip to call get_test_case first to see current values and the partial-update semantics ('only provided fields are updated'). It does not explicitly contrast with bulk_update_cases or say when to use create instead, but the guidance is clear enough for common use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as read-only, and the description adds the useful behavioral detail that the returned vocabularies are authoritative, project-specific, and that callers must use the VALUE rather than the label elsewhere. It also explains isDefault semantics, which helps other tools behave correctly after this call.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is somewhat verbose but every sentence contributes meaningful guidance. It is appropriately front-loaded with a clear summary, then details, usage warning, and return shape. The only minor downside is that it lists many field names inline, making the text denser than necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 1-parameter read-only configuration tool, the description is complete: it states the tool's purpose, when to use it, what the valid-looking fields mean, how defaults behave, and the full return structure. No additional details such as auth, side effects, or prerequisites are needed for this operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is only one required parameter (the project ID), and schema description coverage is 100%, so the schema mostly says what is needed. The description focuses on the output rather than how to provide the project ID, so it adds limited parameter meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Get a project's configuration') and its full scope: metadata, system-field vocabularies, and custom-field definitions. It clearly differentiates this discovery tool from the many sibling get_* and list_* tools by focusing on valid settings rather than a single entity's contents.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use it: before setting any status or priority value. It warns not to hardcode values and explains that these vocabularies are project-customizable, which prevents misuse. This gives an agent clear selection and invocation guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds meaningful behavioral context beyond the annotations: the deletion is soft, restoration is possible through the UI, and linked Test Collections and launches are preserved. This is exactly the kind of non-obvious behavior an agent needs to know and is not captured in the structured annotation fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences deliver all essential information with no redundancy or filler. The primary action is front-loaded, and subsequent sentences provide valuable non-obvious details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, no-output-schema tool, the description is complete. It states the action, the restoration path, and the preservation of related resources, giving an agent enough to invoke it correctly without missing important consequences.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents releaseId as 'Release UUID to archive' with 100% coverage. The description adds little beyond the schema, but it is consistent with the parameter's meaning. Given full schema coverage, the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Soft delete (archive) a release.' It immediately distinguishes this tool from sibling delete tools by naming the target resource and clarifying the action is a soft delete/archive rather than a permanent purge.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear context for when to use the tool: when you need to archive a release. It also indirectly sets expectations by noting restoration is done via the UI, implying this is not a permanent delete. It does not explicitly name alternatives, but the resource-specific purpose makes the usage context sufficiently clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds substantial behavior beyond the annotations: runs counts launches and does not add up correctly, passRate counts flaky as a pass, passRate is floored rather than rounded, and release reports ignore days. These are non-obvious behavioral details that the schema and annotations do not provide. There is no contradiction with the readOnly, idempotent, non-destructive annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The core purpose is front-loaded, and almost every detail contributes important behavioral context. It is dense and long, but not padded. A little restructuring with clearer separation between report-type notes would improve scannability, so it is strong but not a perfect 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a report tool with no output schema, the description explains that the return value is a string and details both summary report fields and release report fields. The failures and trends report contents are left to inferrence from the reportType enum, which is acceptable but slightly incomplete. Overall, the description is robust enough for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All five parameters are already described in the schema, so the baseline is 3. The description adds meaningful semantics around reportType, the release/releaseId relationship, and the difference between summary and release report contents. That goes beyond the schema's short field descriptions, earning a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific action and resource: 'Generate a formatted report for a project.' It then enumerates the supported report types and output formats, making it easy to understand both what the tool does and how it differs from sibling tools that return raw metrics such as get_launches_statistics or get_launches_trends.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives concrete usage conditions: reportType 'release' requires releaseId from list_releases, and days is ignored in that mode because release reports cover the entire release. It also references get_launches_statistics to clarify shared semantics. It does not explicitly say when to choose this tool over sibling reporting/data tools, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description adds behavioral context beyond them: there is NO updatedAt on executions, startDate/endDate are human-typed planning dates usually null, assignedTo is a bare unresolved user UUID, and the progress.passRate denominator divides by total cases rather than executed ones. It openly discloses these semantic traps and other gotchas an agent must know before calling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: no redundant filler, and the opening sentence identifies what the tool returns. However, it is a ~200-word wall of prose; a bulleted breakdown per field would be much more scannable, and the density requires careful parsing by the agent. Efficient but not optimized for scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates fully: it enumerates the returned fields and the shape of progress (five buckets that sum to total, pass`Completeness note: passRate/completenessRate). It also enumerates the confusing fields (updatedAt absence, planning dates, unresolved UUID). It omits straightforward context such as default ordering and whether the list is newest-first or how paging responds, but those are secondary for the tool(to-find-id) core use case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 60%: projectId and status are richly documented, with status getting substantial guidance (case-sensitive vocabulary, rejection, label-vs-content distinction), while limit and offset have no descriptions in the schema nor any in the tool description. The description itself adds little about parameters directly — most of its detail concerns output values and the status caveat is in the schema. Paging params remain underspecified for an unannotated numeric name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and object: 'List Manual Test Executions for a project.' It names the exact return columns, scopes the tool to manual executions, and differentiates itself from get_execution and get_execution_case by explaining what it is not (no updatedAt, no per-execution extended details). An agent can distinguish this from all 50+ siblings without opening other schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing is given: 'Use this to find an existing execution id before adding anything sources.' It also gives an explicit when-not and alternative: when you need to know the last time an execution was worked, 'read get_execution_cases and take the newest executedAt'; and it warns the passRate here must not be compared with get_case_execution_history's. This is unambiguous selection guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description adds important partial-update semantics: only provided fields are updated and omitted fields remain unchanged. This helps prevent destructive accidental field resets. It does not describe return values, but annotations already cover idempotency and non-destructiveness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two compact sentences with no filler. It front-loads the purpose, follows with the most important behavioral rule, and ends with a directly actionable prerequisite.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the full 100% schema coverage, the idempotentHint annotation, and the simple update semantics, this description is complete enough. It covers what the tool does, the partial-update behavior, and the recommended sequencing with get_release.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter is already documented. The description adds critical cross-parameter semantics: omitted fields are left untouched, which is essential for correctly forming an update request. This goes beyond what individual parameter descriptions convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'Update' with the resource 'release metadata', making the operation unambiguous. It distinguishes itself from sibling tools like create_release, delete_release, and get_release by stating exactly what it modifies.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: it is for updating release metadata and explicitly instructs agents to call get_release first to fetch current values. It explains the partial-update behavior, but it does not explicitly list exclusions or name alternative tools for similar actions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description thoroughly explains partial failure behavior, all-or-nothing failure modes, and the fact that invalid IDs do not abort the batch. It also warns about no undo and recommends snapshotting with list_test_cases first. This adds substantial behavioral detail well beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose and limits, then follows a logical sequence: output, failure modes, batch behavior, and undo safety. Every sentence adds information; there is little redundancy for a tool with this much edge behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
It covers edge cases, return shape, project constraint, id limits, and practical guidance. The main missing detail is the exact meaning of the `skipped` return member, which is listed but never defined.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema by explaining that each field is replace-all across the batch, that project span matters, and that the response should be read via `updated` rather than treated as a single success/failure result.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with a clear verb and resource: 'Update many test cases in one call,' and distinguishes it from single-case updates by focusing on bulk, batch, and same-project constraints. It is specific about what fields can be updated: status, priority, and tags.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives strong context for when to use it: bulk operations up to 100 cases, all belonging to the same project. It also suggests preconditions by referencing list_test_cases for snapshotting values and for reading available statuses. It does not explicitly say 'for a single case, use update_test_case,' but the bulk framing makes the intended scope clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the annotations by disclosing side effects: tags are auto-created, folders/suites are auto-created when needed, and status/priority silently fall back to project defaults if omitted. It also states that the full created case object including shareUrl is returned. This gives the agent a strong model of what will happen at runtime.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long, but the tool is complex and almost every sentence earns its place: the suite constraint, the two creation paths, auto-creation behaviors, return value, tag behavior, and project defaults. It is organized with parenthetical labels and an 'IMPORTANT' callout, making the density digestible.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 13 parameters, no output schema, and no behavioral annotations, the description is remarkably complete: it explains return shape, required placement, alternate creation modes, auto-creation side effects, default-value behavior, and tells the agent where to look for project-specific vocabulary. The external link rollback behavior is also covered in the schema's nested parameter description, so nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all parameters at 100% coverage, so the baseline is 3. The description adds decision-level meaning on top: which parameter branch to choose, what gets auto-created, and the important behavior of omitting status/priority. This is meaningful guidance beyond the schema, though the schema itself is already quite descriptive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb and resource: 'Create a new test case in a test suite.' It clearly distinguishes creation from related utilities like update_test_case, delete_test_case, and move_test_case. The nuance that test cases must live inside a suite is front and center.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly explains the two mutually exclusive ways to call it: via folderId or via projectId with folderName/suiteName. It also tells the agent when to consult get_project for project-specific status/priority defaults. It does not explicitly contrast with update_test_case, but the create vs. update semantics are clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already carry destructiveHint=true and idempotentHint=true, but the description adds the soft-delete semantics (only result rows are removed; collections/suites/cases survive), the reversibility path (support or database only), the 'Execution not found' trap after deletion, and the return contract ('returns deleted: true'). Nothing in the description contradicts the annotations — the idempotentHint is consistent with a re-call best trouble behavior — so no annotation contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The key verb, scope, and warning are front-loaded; every sentence in the long description earns its keep, particularly the ordering warning and the explicit alternative. It loses only a point to mild filler ('use with care') and a slight redundancy between the two sentences about the remove_source consequence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter destructive tool with no output schema, this is complete: it states the return value, what is destroyed, what survives, the ordering precondition, and the post-condition failure mode. The idempotence trait is already covered by annotations. An agent needs nothing else to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With a single parameter at 100% schema-description coverage ('Execution UUID'), the schema already fully documents executionId, so the description carries no obligation to re-explain it. The description does not add syntax or formatting detail beyond the schema, correctly leaving parameter semantics to the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action ('soft-delete an entire Manual Test Execution') and a concrete side effect ('unlink any Test Collections that pointed to it'). It scopes what is destroyed versus preserved and explicitly distinguishes itself from the sibling remove_source_from_execution, so an agent can tell them apart without perusing the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly names the alternative that selects it away ('To remove just one source group instead of the whole execution, use remove_source_from_execution') and hard-codes a sequencing prerequisite ('do any per-source cleanup FIRST'). It also warns about the consequence of ignoring that ordering, so when-to-use and when-not-to-use are both given from inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark readOnlyHint/idempotentHint/destructiveHint, and the description is fully consistent — yet it adds a lot of disclosure beyond the annotations: the exact return field set, that 'nothing is changed', the history caveats (50-entry cap with no truncation flag, oldValue and newValue are often identical), the no-time-window execution semantics ('empty array is DEFINITIVE'), and the rich-text round-trip trap where sending a read step back corrupts the write. No contraction found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely information-dense; every sentence carries real operational value — the sibling routing, the definitive empty-array semantics, the marker-bound-line caveat, the replace-all steps warning. The most critical usage note ('caseId is the internal numeric id… not a TC-N') is appropriately prominent at the end of the first part. It loses one point for being a single long monolithic paragraph — splitting the sibling routing, the link nuances, and the write-back warning would make it easier to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a high-complexity tool — 4 params, 3 optional flags with surprising semantics (no time win dow, 50-entry cap, explicit-versus-marker links), one required non-obvious id, read-only with a write-back footgun, and no output schema. The description covers all of these: the shared return list, the suffix-share-difference between includeHistory and includeRecentExecutions, the exotic-autotest binding discrepancy, the rich text versus markdown asymmetry, and the page ambiguity of the id. Nothing of consequence is left to inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline of 3 applies; the schema already picks up each parameter in detail (caseId as internal id vs TC-N, defaults, semantics of includeHistory/includeRecentExecutions/includeLinkedAutotests). The main description only repeats this framing ('Optional flags add…') and points the id semantics, without adding anything beyond what the schema provides. It does not compensate further nor does it need to.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ('Get one manual test case in full') and an explicit field list (id, caseNumber, title, steps, folderId/folderName, shareUrl, etc.) that precisely bounds the operation. It also differentiates itself from the most confusable siblings — list_test_cases, get_case_execution_history and search_tests — within the same description, so there is no ambiguity about which tool is which.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit decision rules with alternatives: 'Choose list_test_cases instead to browse, filter or count many cases, or to turn a TC-N into an id', 'Choose get_case_execution_history instead for how the case has fared in Manual Test Executions over time', and flags that search_tests covers a separate record type. The 'IMPORTANT if you plan to write back' warning further conditions the behavior expected of the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true and destructiveHint=false, and the description does not contradict them. It adds behavioral context by specifying that status is validated against project test_run_status values, listing defaults, and clarifying the write operates on the group level rather than the case level.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with the core purpose, followed by lookup guidance, status validation details, and a sibling distinction. Every sentence carries useful information with little redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a three-parameter write operation with full schema coverage and helpful annotations, the description provides all needed context: what the resource is, where to get the ID, how status is validated, and the critical boundary against the sibling case-level tool. Nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema by explaining where to obtain runId (from get_execution sources[].id or get_execution_cases groupId) and what the status values typically are, including project-custom possibilities.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Set the status of one source group' inside an execution. It clarifies what a source group is and explicitly distinguishes this from setting an individual case's status by naming report_case_result. The agent can tell this apart from the broad sibling set.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear when-to-use guidance and deliberately differentiates from the per-case tool: 'This is the GROUP status; use report_case_result for an individual case's status.' It also tells the agent where to fetch the runId and how to confirm valid status values, making the correct use unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations carry no behavioral signal beyond non-read-only, but the description discloses key behaviors: snapshots collections and keeps identity, skips collections from another project, skips shadow collections, skips duplicates instead of double-snapshotting, and returns {added, skipped}. This fully covers likely agent concerns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise yet information-dense; every sentence adds value. The main purpose and selection guidance are front-loaded, and the edge-case behaviors are summarized compactly at the end.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite lacking an output schema, the description states the return shape {added, skipped}, covers selection guidance, explains snapshot identity, and enumerates all skip cases. It gives an agent everything needed to invoke this tool successfully and predict results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents both parameters with descriptions and 100% coverage, so the baseline is 3. The description adds contextual meaning about collections being snapshotted but does not add new per-parameter syntax or format details beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Add whole Test Collections to a Manual Test Execution') and clearly distinguishes it from sibling tools by contrasting snapshotting whole collections with adding suite cases or loose cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly provides the selection cue: 'Pick this when the work is already gathered into a Collection.' It also names alternatives add_suites_to_execution and add_cases_to_execution with their conditions, so an agent can choose correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key behavioral details beyond the annotations: cases are snapshotted at their current state, suites keep their own identity, cross-project suites are skipped, and duplicate suites are skipped rather than snapshotted twice. It also states the return shape, {added, skipped}, which compensates for the lack of an output schema. No contradiction with the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place. The description front-loads the core behavior, then covers alternatives, edge cases, and return shape without unnecessary filler. It is concise yet information-dense.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given only two schema-covered parameters, no output schema, and clear annotations, this description is complete enough for an agent to select and invoke the tool correctly. It explains behavior, selection criteria, exceptions, and expected output in a compact form.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and both executionId and suiteIds already have clear schema descriptions. The tool description reinforces the meaning of suiteIds with 'whole Suites' and snapshotting context, but it does not add additional parameter-level detail beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Add whole Suites to a Manual Test Execution.' It clearly explains what happens to the cases inside the suites and how suite identity is preserved, making the tool's function obvious. The description also differentiates it from related collection-based and case-based tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says when to use this tool: 'Pick this when the work is organised by suite.' It also names the alternatives, add_collections_to_execution and add_cases_to_execution, and states the conditions for choosing them instead. This is strong routing guidance with no ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a write operation, and the description adds valuable behavioral detail: the created execution starts empty, and the response embeds the entire execution under an 'execution' key. It also flags the critical quirk that execution.id is the useful identifier, not a top-level id, which agents need to know for subsequent calls.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: it states the purpose, clarifies the boundary vs. create_test_collection, explains the response shape and key id, and closes with actionable next steps. No filler or redundant restating of the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with 10 parameters and no output schema, this description is unusually complete. It covers what the tool creates, what the response contains, how to identify the created resource, and which subsequent tools to invoke. Nothing critical is missing for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameters are already well documented in the input schema. The description does not explain parameter specifics beyond noting the required projectId and name, but it compensates slightly by explaining what happens after creation. This meets the baseline for full schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific action and resource ('Create a Manual Test Execution') and defines it as the top-level container on the Executions page. It explicitly distinguishes this tool from create_test_collection and notes the intended composite scope, so an agent can clearly tell it apart from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives direct usage direction: use this tool for assembling an execution from several collections/suites, use create_test_collection for a single grouping of cases. It also tells the agent what to do next after creation, listing add_suites_to_execution, add_collections_to_execution, and add_cases_to_execution, leaving no ambiguity about when and how to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (which already indicate this is not read-only), the description explains important behavioral nuances: omitted type defaults to 'folder' at root but 'suite' when nested, suites cannot be parents, and suites always require parentId. It also discloses the return value shape. This adds meaningful context without contradicting the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and information-dense. Every sentence adds actionable detail: creation action, type semantics, default behavior, parent constraint, return value, and reading/list alternatives. No filler or repetition of the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is no output schema, this description sufficiently covers the return values and key constraints. It explains core creation rules, the parentId requirement for suites, the role of 'folder' vs 'suite', and directs users to get_folder/list_folders for reads. This gives an agent what it needs to invoke the tool correctly in different scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all four parameters with high coverage, so the baseline is 3. However, the description adds useful semantic clarity on top, particularly around the 'type' default behavior and the requirement that a suite needs parentId. This helps an agent reason about valid parameter combinations beyond simple field definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a concrete verb and resource: 'Create a folder or a suite in a project.' It distinguishes the purpose of suites vs folders and differentiates itself from sibling tools by name ('get_folder', 'list_folders'). An agent can tell exactly what this tool does and how it differs from related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit guidance on when to use 'suite' vs 'folder', explains the default type behavior at root vs nested, and states when parentId is required. It also names sibling tools for reading and listing existing folders, making the appropriate context for this tool unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already flag destructiveHint and idempotentHint, but the description adds essential behavior beyond these: the deletion is soft/archival, recursive across nested folders and test cases, and reversible through the UI. That is concrete, useful information that the annotations alone do not provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences front-load the essential behavior ('soft delete', 'all its contents', 'recursive', 'restorable'). Every sentence carries necessary information and there is no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple destructive tool with one fully-documented parameter and no output schema, the description fully covers the operation's scope, side effects, and reversibility. There is nothing critical missing for an agent to safely decide whether to invoke this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Since schema description coverage is 100%, the baseline is 3. The description adds meaning to folderId by explaining that identifying that folder also archives all nested folders and test cases, not just the folder itself. This reinforces the subtree-wide implication of the single parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb and resource ('Soft delete (archive) a folder') and immediately clarifies scope ('all its contents'). It distinguishes itself from sibling tools such as delete_test_case, delete_test_collection, and update_folder by emphasizing recursive archiving of nested folders and test cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use this tool: when a folder and all nested content should be archived, and notes that restoration is only via the UI. It does not explicitly say 'don't use this if you want permanent deletion' or point to a non-recursive alternative, but the recursive wording implies this.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description's existing annotations declare readonly and non-destructive, and the description goes far beyond them by revealing the exact scenarios: summary figures are nested rather than top-level, passRate divides by executedCount while list_executions's progress.passRate divides by total case count, source is a STRING whose .type is undefined and filter silently to nothing, and totalExecutions counts memberships including never-run entries. It also discloses that limit bounds the summary aggregates, not just the list, and that no inverse join exists from runId/executionCaseId back to the owning execution — all non-obvious behavior that an agent would otherwise be unaware of.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The content is dense and earns its place, but the description is a single wall of text of roughly 400 words with no paragraph breaks, no bulleted structure, and several compressed explanations repeated (the source-is-a-string caveat is described twice, the passRate denominator difference appears both in the middle and near the end). The opening sentence is front-loaded and strong, but the mid-portion is very hard to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the full burden of explaining the return shape — and it does so thoroughly for both the row fields (id, status, runId, runName, executedBy, executedAt, notes, source with its standalone/release variants) and the summary fields — totalExecutions, executedCount, passed, failed, blocked, inTesting, notExecuted and passRate. It also documents edge cases (never-run memberships, inTesting shortfall, 365-day ceiling, both join paths and the lack of a reverse lookup) and identifies which sibling covers what, so nothing that an agent needs to invoke and interpret the tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for all three parameters, so the baseline is 3. The description contributes additional semantics beyond schema: days at its default of 30 can produce a near-empty result that falsely reads as 'never tested,' 365 is a hard ceiling with no way to query all history at once, and limit bounds the summary aggregates rather than just page size — none of which the schema states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence names a specific verb ('get'), a precise resource ('how one manual test case has fared over time'), and an explicit scope ('across every Manual Test Execution and release it appeared in, newest first'). It further differentiates itself from siblings by stating that only MANUAL results are covered, so an agent can distinguish it from get_test_history and get_execution_cases without opening their schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly routes to alternatives: 'use get_test_history for an automated test across launches, and get_execution_cases for the rows inside one execution.' It also warns against the tempting workaround of passing a releaseId to list_executions, since that tool has no releaseId filter and silently ignores unrecognized arguments, and instructs the caller to pass days explicitly because the 30-day default makes older cases read as never tested.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and the description reinforces 'Read-only, nothing is written.' Even with that coverage, the description adds substantial non-obvious context: LIVE-only counting with archived/stubs inference, two mutually exclusive definitions of 'linked' active in the same response, a hard 50-row cap on items, a recentLaunches window, and a precise disambiguation path via get_test_case. This is far beyond what annotations convey, and nothing contradicts them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose is front-loaded and almost every sentence carries unique information, but the middle section is one dense paragraph mixing output shapes, caps, windows, and linkage definitions, with contorted phrasing like 'since limit sizes unlinkedCases and does not reach it, and it is drawn only from the last recentLaunches launches.' Bulleted structure would meaningfully reduce parse effort for an agent. It is bloated rather than merely long.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the full burden of describing return values — and it fully carries it: it names all four output blocks (summary, foldersWithLowCoverage, unlinkedCases, unlinkedAutotests), lists the fields of each, explains the two linkage conventions, the 50-row cap, the window scoping, and the exact procedure for settling a single title with get_test_case. For a tool this intricate, even the edge cases (missing from items ≠ linked) are documented.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description genuinely adds meaning on top by linking parameters to output behavior: limit sizes unlinkedCases but does not reach the hard-capped unlinkedAutotests items, recentLaunches (default 10) scopes both totalUnlinked and items, and automationThreshold drives foldersWithLowCoverage aggregation. These semantics make the parameters' effects explicitly discoverable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence states a specific verb + object + scope: 'Identify test coverage gaps across a whole project.' It also differentiates from siblings later by contrasting the project-wide scope with the single-Jira-issue scope of get_feature_scan, so an agent can tell it apart without opening other schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Contains an explicit routing block: 'Pick this when the question is which cases or folders lack automation project-wide,' followed by named alternatives with their conditions (get_feature_scan for a single Jira issue, run_health_check for a scored assessment, suggest_test_cases for gaps that are nonexistent cases, search_tests for known autotests). Nothing is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Even though annotations already declare readOnlyHint and idempotentHint, the description adds substantial non-obvious behavior: statuses are manually set and not inferable from counts, execution.status and sources[].status use different vocabularies, and passRate is divided by total rather than by completed cases. It also warns that list_executions and get_release repeat these labels unchanged, extending the caution beyond this single tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place: it fronts the return structure, then packages the traps, formulas, and sibling routing in clearly labeled units. The length is justified by the actual semantic complexity of the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, one-parameter tool with no output schema, this description is exceptionally complete. It covers return shape, field semantics, status vocabulary, count-to-status relationship, percentage formulas, and the exact way to obtain the required id. Nothing needed for safe invocation is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers the single parameter fully with 'Execution UUID', so the description does not need to restate it. It does add a useful hint to use list_executions to find an id, but that is more usage guidance than expanded parameter semantics. Baseline 3 is appropriate because the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb-resource pair, 'get one Manual Test Execution', and itemizes exactly what the tool returns: execution, sources, releases, externalLinks, and progress. It also differentiates itself from siblings by explicitly pointing to get_execution_cases for individual case rows and list_executions for finding an id.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit routing guidance: use get_execution_cases for individual case rows, use list_executions to find an id, and warns that sources[].id is what remove_source_from_execution and set_execution_source_status consume. This is exactly the when-to-use-this-vs-alternatives information an agent needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Even with annotations already marking this as read-only, idempotent, and non-destructive, the description adds substantial behavioral detail: it documents unit differences versus siblings, explains that automationRatio is a conservative floor, reveals the dual meaning of 'linked' between explicit links and title-marker links, and states the aggregation depth. Nothing in the description contradicts the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every section earns its place: the purpose is front-loaded, and the UNITS and IMPORTANT sections are clearly labeled warnings that prevent real misuse. It reads like a structured briefing rather than a rambling explanation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite missing an output schema, the description is unusually complete: it lists the main return categories, explains units, describes the nuanced interpretation of linked test cases, notes hierarchy aggregation, and states the Jira connection prerequisite. An agent has enough context to invoke this tool and interpret its results accurately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already describes jiraKey fully at 100% coverage, so the baseline is 3. The description adds value beyond the schema by specifying the aggregation depth ('2 levels deep: epic→story→subtask') and by making clear that the single key drives the whole feature health result, including linked cases and autotest trends.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Get a comprehensive feature health scan by Jira issue key.' It then enumerates concrete return contents such as Jira context, coverage stats, linked test cases, and autotest trends. It also clarifies how this tool differs from other health-related tools by highlighting its unique unit convention and its aggregation of child issues.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when this tool is appropriate, especially for 'comprehensive' feature health and multi-level issue aggregation. However, it does not explicitly say when to prefer siblings like get_coverage_gaps or run_health_check, aside from noting they return percentages rather than fractions. There are no explicitly stated exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Even with annotations present, the description adds substantial behavior context: the link is forward-only, historical results are not backfilled, and the returned testExists/wasCreated fields do not mean history was attached. It also explains the concrete get_test_case side effect with an empty includeRecentExecutions until the next report, which is genuinely useful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place: the core action is first, followed by the forward-only warning, the meaning of the return fields, and the relevant alternative tool. There is no filler or redundant repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter tool with no output schema, the description gives the agent what it needs: the side-effect model, the return-field semantics, the important no-backfill behavior, and a pointer to get_test_history for existing results. No critical operational gap remains.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already covers both parameters with descriptions and 100% coverage, so the baseline applies: the description does not add much new parameter-specific format or syntax detail. It reinforces that linking happens testTitle and can be for an autotest that does not exist, but those are behavioral clarifications rather than parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Link an automated test, by title, to a manual test case for coverage tracking.' This clearly distinguishes it from sibling linking tools like link_launches_to_release and add_external_link, and makes the exact entity relationship explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use context: it supports pre-linking since the autotest does not need to exist yet. It also warns that the link is forward-only and no past results are backfilled, and directs agents to 'Use get_test_history to read the results that already exist for that title,' which is a clear alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly flag, the description discloses return shape ('eras array', 'pagination holding real total', no top-level count or data flag) and flags an important domain nuance about release status being SET BY A PERSON and potentially inconsistent with aggregate stats. This directly prevents an agent from misinterpreting the aggregated data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded; the first sentence states the core purpose, the second sets expectations about the return shape, and the third adds routing and filtering context. Every sentence supplies meaningful information with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema, the description explains the top-level return structure and the pagination format, so the agent knows how to parse results. Combined with the detailed input schema, it is sufficient to call the tool correctly and understand how its results differ from related tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% description coverage for every parameter, including status vocabulary and rejection behavior. The description adds little about the parameters themselves beyond saying filtering by status and search is supported, so it stays at the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description states a clear verb ('List') and target ('releases for a project'), and specifies it returns aggregated stats for manual tests and automated rseats. It also explicitly differentiates itself from get_release by noting get_release is for when a single flag id exists and linked executions or launches are needed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The final sentence gives an explicit rule: use get_flag when you have one id and need its linked executions and launches, implying list_releases is for the broader project-level list. It also names possible returns/filtering ('by status and search') so an flag can decide this is the right entry point.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already mark the tool as read-only, idempotent, and non-destructive, and the description adds meaningful behavioral context: shadow collections are excluded, archived collections require includeDeleted, and the response includes a pagination block. These are details an agent cannot infer from annotations alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but not padded; it front-loads the purpose and return contents, then handles sibling distinctions and behavioral caveats. Every sentence adds information an agent needs before calling the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with no output schema, the description adequately enumerates the returned fields, mentions pagination, names the filter dimensions, and differentiates it from nearby siblings. Nothing essential to selecting and invoking the tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description reinforces that folderId, releaseId, and status are filters and that the tool is meant for finding a collection UUID, but it does not need to repeat what the schema already documents in full.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: "List Test Collections in a project." It also distinguishes this tool from list_executions, list_test_cases, and get_test_collection, making its scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly tells the agent when to use this tool: to find a collection UUID filtered by folder, release, or status, then call get_test_collection. It also gives exclusion rules: shadow collections are never listed, archived ones need includeDeleted, and executions/case libraries belong to other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes beyond annotations by explaining idempotency behavior, Jira-specific validation order, canonical URL fetching, remote-link push-back with remoteLinkWarning, and failure semantics. Annotations already mark readOnlyHint=false and idempotentHint=true; the description adds concrete operational detail that an agent needs to anticipate side effects and error outcomes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but organized: it starts with the tool's purpose and return value, then addresses idempotency and Jira-specific behavior. It is longer than typical, but every clause adds operational guidance that affects how the agent invokes the tool. The only minor inefficiency is the length, though it is mostly earned.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description explicitly enumerates the return fields, which is essential. It also covers entrance criteria (required parameters, Jira lookup), side effects (remote link push-back), and alternatives (siblings). An agent has enough to call this correctly and predict the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is high (86%), and the description adds meaningful context over the schema: it explains idempotency keyed on provider+externalId, specifies that externalUrl is required for non-Jira providers and ignored for Jira, and states which fields come from the Jira API. This goes beyond the schema property descriptions, though a few parameters like linkType are not deeply expanded.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Link any test entity to an external ticket' and explicitly enumerates the supported entity types and external systems (Jira, Linear, GitHub, GitLab). It clearly differentiates from sibling remove_external_link and list_external_links by naming them and their roles.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool versus alternatives: 'Call list_external_links to see what an entity is already linked to and to get link ids, and remove_external_link to delete one by its link UUID.' It also gives conditional usage guidance, such as the Jira issue lookup requirement and when externalUrl is ignored.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint and idempotentHint are true, and the description goes well beyond that: it explains nullability of caseId and durationMinutes, the UUID-vs-email executionBy discrepancy, conditional externalLinks based on integrations:read scope, and that report_case_result writes are visible here. This gives an agent a strong behavioral model of the tool without contradicting the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense, but nearly every sentence carries actionable information about return fields, nullability, or caveats. It is not a model of brevity, but because there is no output schema, the field-level detail is justified and the core workflow guidance is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only listing endpoint with no output schema, the description covers the returned fields, their types/nullability, permissions, workflow placement, and how this tool differs from get_case_execution_history. No important usage dimension appears to be missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents search, status, groupId, and executionId, while limit and offset rely on defaults and bounds rather than descriptive text. The description compensates by explaining that pagination and filtering by group/status/title are supported and by establishing groupId as the significant identifier for report_case_result, which is exactly the kind of semantic weight an agent needs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb and resource: 'List the individual case-run rows of a Manual Test Execution' and adds a clear purpose ('so you can report results against them'). It also distinguishes itself from get_case_execution_history by pointing out the title field naming difference and the user UUID vs email difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states where this tool fits in a workflow: use it after add_suites_to_execution, add_collections_to_execution, or add_cases_to_execution, and before report_case_result. It also directs the agent to get_case_execution_history when a user UUID needs to be resolved to an email address, giving a clear alternative-selection rule.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes far beyond annotations by exposing non-obvious behavior: rows are per retry attempt, retry is 0-based but a retry:0 row may not exist, testId is per-row rather than per-test, previousStatus 'superseded' is explicitly unreliable, and launch stats exclude skipped tests. It also documents authentication requirements for artifact URLs, which is valuable operational behavior not visible in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long, but it is front-loaded with a clear one-sentence summary followed by essential caveats. Nearly every sentence earns its place because the retry-attempt semantics are genuinely non-obvious and easy to get wrong. It is slightly verbose in explaining examples and traps, but the structure stays readable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, yet the description fully compensates by listing row fields (status, previousStatus, durationMs, tags, errors) and explaining the attachments array with url, name, id, contentType, size. It also covers pagination semantics indirectly through limit/offset in the schema and covers retry counting pitfalls, test-identity semantics, launch-stats conventions, and artifact authentication. An agent has enough context to invoke the tool and interpret results correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is already strong. The description adds real semantic value beyond the schema by explaining that statuses filters by ATTEMPT-ROW status rather than final test outcome, and by noting where launchId comes from (get_launches_statistics or get_launches_trends). It does not re-explain limit, offset, or search, which is appropriate since the schema covers them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Get every test inside one launch, with results, error messages and stack traces for the failures.' It also distinguishes this tool from related siblings by noting get_test_history is for following a test across launches and that get_launches_statistics is the source of the launch id. The caveat about one row per retry attempt further clarifies exactly what the tool returns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: 'Use this once you have a launch id, which get_launches_statistics gives you' and provides the alternative for a different need: 'use get_test_history to follow one test across launches instead.' It also warns against using row-level filters for test-level counts, giving clear guidance on what this tool should not be used for.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly/idempotent annotations, the description adds crucial behavioral context: the executionId is the row's own ID and not a Manual Test Execution ID, so passing it to get_execution would fail. It also discloses the conditional nature of the tests array based on includeTestDetails. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but dense with useful distinctions. It is front-loaded with the core action and resource, and every sentence conveys a worthwhile fact, though a few phrases are slightly redundant with the annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description fully compensates by listing metadata, stats, and the tests array structure, including the crucial executionId caveat. The agent has everything needed to invoke the tool and interpret the result correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description enriches parameter understanding by stating where collectionId comes from and clarifying that includeTestDetails controls the tests array. This adds value beyond the schema's property descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Get one Test Collection by UUID') and enumerates exactly what is returned. It explicitly distinguishes this tool from get_execution and list_test_collections, so an agent can reliably tell them apart.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit routing guidance: use list_test_collections to find collections, use get_execution/get_execution_cases for Manual Test Executions, and clarifies that executionId rows are not execution IDs. This is exemplary when-to-use versus alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this as read-only and idempotent, so no safety contradiction. The description adds valuable behavioral context: casesCount only counts direct cases, childFolderCount and casesCount ignore archived data unless includeDeleted is passed, and the output contains enough fields to rebuild the tree. These are non-obvious behaviors an agent needs to know.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every sentence adds useful information: output shape, field list, use cases, caveats about counting, and routing to siblings. It is front-loaded with the core purpose and then covers the two most important behavioral caveats. A slightly shorter version would be possible, but the density of useful guidance justifies the length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description must stand in for return-value documentation. It does this thoroughly by enumerating all key fields, indicating the response structure as flat array under folders, and covering edge cases around archived content and hierarchical counting. It also integrates sibling routing, so detail can be retrieved with get_folder and test cases with list_test_cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already describes the two parameters, so the baseline is 3. The description earns an extra point by explaining the practical effect of includeDeleted: 'archived cases and archived subfolders are invisible here unless you pass includeDeleted.' It clarifies that the tree representation includes enough fields to derive paths and descendants from parentId, going beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'List the folder and suite tree of a project as a flat array under folders, with a total count.' It clearly identifies what the tool does, the kind of data returned, and how it differs from get_folder and list_test_cases by explicitly naming those alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit guidance on when to use this tool versus alternatives: 'Use this to browse or to find an id, then get_folder for one folder in detail' and 'use list_test_cases for the cases themselves.' It also explains when this tool is NOT appropriate, such as for size ranking, and names the better alternative with get_folder includeStats.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate a destructive, non-read-only operation, and the description adds valuable behavioral context beyond them: the cascade deletion of case-run rows and step results, the fact that the execution remains intact, and the exact return value { removed: true }. This meaningfully informs the agent about side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each carrying essential information: what is removed, the destructive cascade effect, how to find the required id, and the return value. There is no filler or redundant restatement of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive two-parameter operation with no output schema, the description covers the removal target, how to obtain the id, the cascade impact, the boundary (execution not deleted), and the response shape. Combined with a thorough parameter schema and annotations, nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 100% with detailed runId explanation; the description further reinforces how to get runId and emphasizes that runId is the source group id, not the original suite/collection id. This adds practical lookup guidance beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (Remove), the resource (a source group from an execution), and precisely defines what a source group is: a suite, collection, or named case-group represented by one execution_test_run. It also distinguishes itself from deleting the whole execution, so an agent can correctly separate it from sibling tools like delete_execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives actionable guidance on how to obtain the runId: from get_execution (sources[].id) or get_execution_cases (groupId). It states that the execution itself is not deleted, which hints at the difference from delete_execution, but it does not explicitly name the alternative or specify when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so the description does not need to repeat that. It goes well beyond annotations by explaining the non-comparability of healthScore across projects, the renormalisation over available dimensions, and the exact meaning of the three different failure counts. These behavioral traits are not deducible from annotations or schema, and they materially affect how an agent interprets results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately front-loaded with a crisp summary before diving into warnings. It is, however, very dense and somewhat sprawling, with several examples and parentheticals that could be condensed. Given the genuinely subtle behavior being described, the length earns its place, but a clearer structure or shorter phrasing would improve it.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema, the description carries full responsibility for the return values. It does this well by enumerating the return sections (automation, testAssets, coverage, manualExecution, releases) and explaining the key subScores and edge cases. It also covers parameter interactions and cross-tool context. No critical behavioral corner required for correct invocation is left unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% coverage for projectId, days, and include, including defaults and enum values. The description adds a crucial semantic layer beyond the schema: it explains that narrow include[] changes the renormalisation denominator and therefore shifts the composite score, and that days defaults to 14 here but to 30 in get_launches_trends. This is useful but not needed for every parameter, so a 4 is fair rather than a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence states a clear, specific action: 'Get a holistic health assessment for a project: a composite score from 0 to 100.' It names the resource (project) and the narrow output focus. It also explicitly separates itself from siblings: get_coverage_gaps and get_feature_scan are described as alternatives for different use cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the agent when to prefer alternatives: 'Reach for get_coverage_gaps when you want the specific cases and folders that lack automation rather than one score, and get_feature_scan when the scope is a single Jira issue.' It also warns about comparing days with get_launches_trends, and explains how the include parameter changes what is computed. This is strong, scenario-based guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations only provide basic readOnly/destructive hints, so the description adds meaningful behavioral context: it creates a named group, returns groupId, skips cases from other projects, and does not create a group when the result is empty. This goes well beyond the schema and gives the agent a clear model of side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose and every sentence adds value: sibling differentiation, numeric ID clarification, alternative routing, and edge-case behavior. No filler or repetition of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema present, the description properly states the return shape as {added, skipped, groupId}. All three required parameters are semantically covered, sibling alternatives are addressed, and important non-obvious behaviors are disclosed. This is a complete description for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers all three parameters, but the description adds important nuance beyond the schema: caseIds are numeric IDs and 'never TC-numbers', and name is required because loose cases have no identity of their own. It does not add detail about executionId beyond what the schema says, but the added semantic guidance is substantive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Add individual test cases to a Manual Test Execution as a new named group') and clearly differentiates this tool from add_suites_to_execution and add_collections_to_execution by explaining the distinction between loose cases and gathered suites/collections. An agent can immediately tell what this tool is for relative to its siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names alternatives and the conditions under which they should be used: 'Use add_suites_to_execution or add_collections_to_execution when the cases are already gathered into one of those.' It also clarifies an edge case behavior ('empty result creates no group'), helping the agent decide whether this is the right tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already signal a read/write safety profile, and the description meaningfully extends this by declaring the call is not idempotent, that duplicate names create separate releases, and that the new release starts empty without modifying anything else. It also states what is returned, including the fields and why keeping the id matters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place: the main purpose comes first, then defaults, duplicate behavior, alternatives, and downstream workflow. It gives a lot of needed context without repeating schema content or adding filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters, no output schema, and no enums, the description closes the important gaps itself: return shape, id retention for sibling tools, duplicate pitfalls, property updates route, and the relationship to related execution and launch tools. The only omissions, such as error conditions, are not critical enough to hurt an agent correctly invoking this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the structured field descriptions already explain each parameter. The description adds useful extra meaning, such as sourcing projectId from list_projects, the status default behavior, and where to resolve project-specific release statuses via get_project. This is a strong complement to the schema, though some parameters still rely mostly on their own schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Create a new release in a project') on a specific resource and explains what releases are for. It clearly differentiates this tool from update_release and delete_release, which is enough for an agent to select it correctly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives concrete before-you-call guidance: get projectId from list_projects, check list_releases first because the call is not idempotent, and use update_release when only modifying an existing release. It also explains how manual executions should be linked on the execution side, so the tool is placed in the correct workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses important non-obvious behaviors: newly created collections always start at status not_started, no status argument is accepted, priority defaults to 'medium', and priority matching is case-insensitive but stored lowercase. It also states return behavior (created collection with count of added tests) and clarifies the distinction from an execution container. These details go well beyond the minimal annotation hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is information-dense but efficiently organized, front-loading the core purpose and the critical disambiguation from executions. Every sentence adds value, and there is no filler or repetition of schema basics.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 9-parameter creation tool, the description covers the essential call context: what it returns, how defaults work, what it cannot do, and how it relates to adjacent sibling operations. The absence of an output schema is adequately mitigated by explicitly stating the created collection and added test count.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Though the schema already documents all nine parameters, the description adds meaningful semantic nuance such as the default for priority, the lowercase-only constraint, the fact that collection priority cannot be project-customized, and that caseIds are optional. It does not restate every parameter, but it enriches the most behaviorally important one.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the action ('Create a Test Collection') and the resource, and immediately distinguishes it from the top-level Executions-page container ('NOT the top-level Executions-page container - use create_execution'). This makes the tool's purpose unambiguous and separates it from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly explains when to use this tool versus create_execution and update_test_collection, including routing to add_collections_to_execution for attaching collections to executions. This gives an agent clear decision guidance with named alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations establish read-only, idempotent, and non-destructive behavior. The description adds a wealth of nuanced behavior: casesCount only counts direct children, statistics cover nested items, null status/priority are excluded, and archived rows are excluded from all counts. This goes far beyond the annotations and is essential for interpreting results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but each sentence and clause carries a distinct fact about scope, lifetime, or counting semantics. It front-loads the core summary of returned fields and then systematically explains the flags and edge conditions. No filler or repetition exists.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This tool has no output schema, so the description fully compensates by enumerating returned fields and the caveats of archived data, direct versus nested case counts, and shortcut behavior. It also gives alternatives for cases requiring archived data or the full tree. An agent has enough information to call and interpret this tool reliably.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all four parameters, so the baseline is 3. The description provides meaningful extra semantics: includeStats yields a statistics object over the whole subtree, includeCases returns recent cases under recentCases rather than cases, and casesLimit caps that list. It even clarifies that folderId comes from list_folders.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Get') and resource ('folder or suite by id') and explicitly lists the returned fields. It distinguishes get_folder from list_folders, and the name and description clearly frame it as a detail-fetch tool rather than a tree/navigation tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use list_folders to see the tree or find an id; use this when you have one id and want detail.' It also notes when archived data is needed, pointing to list_test_cases with includeDeleted, giving the agent clear routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as readOnly, idempotent, and safe. The description goes well beyond these by exposing crucial statistical behaviors: statuses are reported by the CI job, not derived from the numbers; total excludes skipped; retries counts attempts; runPassRate and testPassRate have different denominators; rates are null when there is no data; and pagination has a real total with no top-level total. None of this contradicts the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is unusually long, but it is packed with distinct near-miss pitfalls that are directly relevant to correct invocation and interpretation. It front-loads the purpose and row schema, then expands into statistical pitfalls that an agent would otherwise never guess. A small amount of trimming and structural formatting could improve scannability, but no sentence felt purely filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Even though there is no output schema, the description reveals the entire launch object field set, the aggregated summary object's exact fields, the pagination behavior, the meaning of null rates, and the empty-list interpretation. It also covers how to distinguish this tool from related alternatives. For a statistical listing tool with this level of edge-case nuance, nothing needed for a careful call is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Though the schema already describes every parameter with 100% coverage, the description adds meaningful extra semantics: days means a lookback window and an empty list can mislead; search looks at the launch field, id, or tags and carries no trigger metadata; statuses have a fixed enum set; pagination through limit/offset is explained in context; statuses alone do not sum to totalRuns. This far exceeds the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'List the launches of a project, newest first', then enumerates the fields returned (id, launch, status, startedAt, endedAt, durationSeconds, tags, stats). It also explicitly differentiates from siblings by telling the agent when to use get_launches_trends and get_launch_tests, so the purpose is unmistakable even among many similar tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance: get_launches_trends for questions about the project over time, get_launch_tests for a single launch's tests. It also explains under what conditions an empty list means no data in the window (days defaults to 30) and warns that a name search matching nothing should be reported as 'cannot be identified' rather than assuming the newest launch. This is far beyond a generic read tool description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes substantially beyond the readOnly/idempotent/destructiveHint annotations, revealing critical behavior: status is a person-applied label and not a derived verdict; manual rollups count executions only, result rows rather than surviving cases, and no deduplication; deleted-case artifacts show as null-caseId; and the launches rollup has special flaky/skipped semantics inconsistent with get_launches_statistics. This is rich, non-obvious, and fully consistent with the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but remarkably dense—every sentence adds a distinct caveat, envelope alternative, or semantic clarification that an agent needs before trusting the output. It fronts the core detail, then progressively deepens into data-quality warnings and finally routing guidance, with no filler to trim.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema, the description names the return payload, explains rollup semantics, warns about misleading counts, provides the exact lookup method to troubleshoot archived releases, and routes agents to list_releases for browsing. It is difficult to think of a required input-output or edge-case detail missing for successful invocation and interpretation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, and the description adds meaningful context on top: releaseId should be obtained from list_releases, includeLinkedExecutions/includeLaunches control only the payload arrays while the stats rollup still comes back either way. This is more than simple schema restatement, though less than exhaustive since nested response shape is covered only narratively.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and target resource—'Get one release by releaseId with full detail'—and immediately enumerates the returned fields. It also distinguishes itself from list_releases by noting that list_releases browses, searches, and filters while get_release fetches a single already-identified release.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly instructs when to use list_releases instead (browse, search, filter, obtain releaseId) and when to call get_release once the release is known. It also explains when to pass includeLinkedExecutions and includeLaunches false to keep the response small, and warns about archived releases versus list_releases requiring includeDeleted.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, and the description adds meaningful behavioral context beyond those: role-based visibility, the meaning of an empty list, and the display-order-then-created order. It also states the return fields (id, name, metadata), which helps the agent know what to expect before calling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence carries intent: the START HERE hook, access scope, empty-list interpretation, ordering caveat, return fields, and pointer to get_project. It is dense but not wasteful, and the most actionable guidance is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless listing tool, the description covers purpose, access behavior, ordering, return shape, and the correct next-step tool. There is no missing information that would prevent the agent from invoking it correctly or interpreting its results properly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is no parameter ambiguity for the description to resolve. The description instead clarifies the output fields and access semantics, which is useful given that no granular schema or output schema exists.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('List'), the resource ('projects'), and scopes it to what the current key can reach. It clearly distinguishes itself from get_project, which provides detailed single-project information, and its START HERE framing establishes it as the entry point for obtaining project UUIDs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the agent to start here because almost every other tool needs a project UUID. It also gives concrete interpretation guidance: an empty list means no access rather than no projects, and the ordering should not be over-interpreted. It routes to get_project for detail and for the actual status/priority vocabularies before setting status.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite strong annotations (readOnly, idempotent, non-destructive), the description adds substantial behavioral context: it reveals that counts are attempts (retries inflate counts), explains passRate and reliability computation, highlights the asymmetry between failed/passed filters, and clarifies what lastStatus means. This goes far beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense and front-loaded with the core purpose, then covers edge cases and alternatives. Every sentence serves a purpose; there is no fluff. It is long because the tool's behavior is complex. A slight structural improvement could be grouping related concepts, but the existing organization is functional and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema and 5 parameters plus complex semantics, the description fully compensates: enumerates every row field (titlePath, lastStatus, reliability, etc.), explains absence semantics, distinguishes attempts from test cases, and explicitly states what the tool cannot answer (linkage, current failing status) while providing the correct tools to route those cases. Nothing needed for correct use is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema already covers all params at 100%, the description adds deep semantics: days is the shortest window and must be extended unless the question is about the current week; query should be a single titlePath segment rather than a joined path; status filtering is window-based and asymmetric, with failed selected by at least one failure while passed requires all attempts passed. This is meaningful value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a precise action ('Search AUTOMATED tests'), the search dimensions (title, file path, tag), and what results include (pass rate, reliability, etc.). It explicitly names sibling tools it is not (get_test_case, get_test_history, get_launch_tests), so an agent can disambiguate immediately.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use and when-not-to-use guidance: it notes the default 7-day window, warns that absence means 'not run' rather than 'nonexistent', and routes to get_test_case for linkage, get_test_history for one test in detail, and get_launch_tests for a full launch. This leaves no ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description adds meaningful behavioral context: status values are project-configurable, setting status follows a during/after pattern, and releaseIds is a replace-all operation. It also warns that this tool does not modify case results, which is important behavioral information not captured by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: it opens with the tool's core purpose, immediately gives the key status workflow, and then clarifies alternative tools. Every sentence carries operational value, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 10 parameters, no output schema, and meaningful behavioral traps like replace-all release links and project-custom statuses, the description covers all critical operational needs. It also references the exact sibling tools an agent might confuse it with and provides a concrete fallback for discovering valid status values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already covers 70% of parameters with descriptions, and the description adds practical semantics on top: status should be in_testing while running and passed/failed/blocked when done, and releaseIds replaces the entire release-link set. It does not elaborate on every param interpretation, but the added guidance is valuable.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Update'), a specific resource ('a Manual Test Execution's own fields'), and distinguishes the tool from sibling operations by explicitly saying it does not change case results. It is instantly clear what this tool is for and what it is not for.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: use this for execution-level fields and status, use report_case_result for case results, and use set_execution_source_status for source group status. It also instructs calling get_project to confirm project-specific status values rather than hardcoding.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations, the description explains meaningful behavioral consequences: moving carries all nested children and changes their paths, it never changes type, and it never creates or destroys anything. It also clarifies that re-parenting to root changes depth/path. This is exactly the context an agent needs to understand what happens to the rest of the tree.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place: core action, parentId behavior, return payload, move side effects, exclusions, and sibling routing. It is front-loaded with the primary purpose and avoids any redundant restatement of the tool name or schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is no output schema, the description properly documents the return value fields (id, name, type, parentId, path, depth, casesCount, childFolderCount). It also covers key edge cases and sibling boundaries. For a mutation tool with three simple parameters, nothing an agent needs to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the schema already documents all three parameters. The description adds value by explaining the practical meaning of setting parentId to null versus omitting it, and by noting the same-project requirement for the new parent. The name parameter has no additional semantics beyond the schema, so it doesn't get a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise verb and resource: 'Rename a folder or suite and/or re-parent it.' It clearly distinguishes itself from sibling tools by explicitly naming create_folder, delete_folder, and move_test_case as alternatives for different operations. An agent can tell exactly what this tool does without opening other definitions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use and when-not-to-use guidance: use create_folder for new folders/suites, delete_folder for archiving, and move_test_case for moving a single case. It also adds operational constraints like 'the new parent must be a folder in the same project' and 'a folder cannot move into itself or into one of its own descendants.' This leaves no ambiguity about when to invoke it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes far beyond the readOnly/idempotent annotations by exposing critical behavioral caveats: passRate is a per-day series requiring manual summation, rates are over green launches not passed tests, failure_rate counts attempt rows rather than launches, rankings describe the whole window and never current status, flakiness uses flip_count over attempts, skipped/flaky rows are excluded, and numeric values arrive as strings. This is extensive, non-obvious behavior that an agent must know to use results correctly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every sentence delivers a distinct, non-redundant warning or instruction. It opens with a scannable summary of what is computed, then front-loads the most dangerous misinterpretation (no single pass rate), and systematically proceeds through failure, flakiness, bounds, types, options, and alternatives. No filler exists despite the length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex aggregation tool with no output schema, the description covers the shape of the response, the meaning of each major section, the units of measurement, known limitations, silent behaviors (quiet projects return empty series), and exactly how to interpret values for decision-making. Given the tool's complexity, this description is unusually complete and leaves little to chance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Even though schema already covers all parameters, the description adds crucial semantics: daily series with quiet-day bias, attempts versus launches, 90-day caps regardless of days for some sections while the full range applies elsewhere, and string coercion for counts and rates. It also contextualizes days, include, and topN by explaining their downstream effects on the response, which the schema does not provide.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description begins with specific verb+resource: 'Aggregate a project over a window' and lists distinct deliverables (pass rate, failing tests, flaky tests, duration trends, slowest tests). It clearly distinguishes itself from siblings by naming get_launches_statistics and get_launch_tests as alternatives, so an agent can immediately tell what this tool uniquely does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is provided for when to use this tool vs alternatives: paginate individual launches with get_launches_statistics, read newest launch with get_launch_tests, per-launch flipping with get_test_history. It also instructs on include[] and topN to control which sections are computed, leaving no ambiguity about scope.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite readOnlyHint=true, the description adds substantial behavioral detail not inferable from annotations: one row per retry attempt, 0-based retry field, no history key, summary nesting, matchedTests count, matchedTitlePath selection, and reliabilityStatus limitations. It fully discloses edge cases and response semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence conveys a distinct, necessary caveat or fact. It opens with the core purpose, then systematically addresses response structure, common pitfalls, retry semantics, and alternative tools. No redundant phrases, and the warnings are grouped logically.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema, the description thoroughly explains the return shape, nested summary fields, edge cases, and interpretation guidance. It covers all four parameters, identifies behavioral pitfalls, and routes to sibling tools where appropriate. Despite length, it leaves little room for an agent to misinvoke.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description goes far beyond schema text by explaining titlePath is a single segment, matching is case-insensitive substring, how substring ambiguity works, and how limit and days affect the response. It clarifies the relationship between matchedTests and matchedTitlePath, which is essential for correct parameter use.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Get the history of one AUTOMATED test across launches, newest first.' It clearly distinguishes from related tools by defining the exact scope (one test across launches) versus get_case_execution_history for manual cases and get_launch_tests for all tests in one launch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use and when-not-to-use guidance, naming alternatives: 'Use get_case_execution_history for how a MANUAL case has fared, and get_launch_tests for every test inside one launch rather than one test across many.' It also warns when multiple tests match and advises narrowing the substring.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Even though annotations already declare the operation read-only and idempotent, the description adds extensive behavioral detail: return shape, updatedAt semantics, absence of lastExecutedAt, the meaning of linkedAutotestCount, the custom priority marker, and the no-top-level-total quirk. This goes well beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every sentence earns its place; it front-loads the primary purpose and then packs edge cases, return-shape details, and exclusions without padding. The structure moves from what the tool does, to return shape, to caveats, to alternatives — a clear logical flow.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list endpoint with no output schema, the description is extraordinarily complete: it covers return fields, pagination shape, field semantics, filter behavior, deletion handling, automation interpretation, and alternatives for missing data. There is no meaningful gap left for an agent to discover by trial and error.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 100%, the description adds crucial semantics beyond the schema: search accepts TC- prefix, folderId does not recurse and a zero result can look like an empty folder, automationStatus uses the same narrow definition as linkedAutotests, and availablePriorities.custom is a marker rather than a real priority. These details materially improve correct tool usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource — 'List test cases in a project with filtering and pagination' — and immediately communicates the tool's scope. It clearly distinguishes listing cases from related sibling tools like get_execution_cases or list_test_collections.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly directs agents to alternatives when appropriate, such as using get_folder for subtree counts, get_case_execution_history for manual execution history, and get_test_case with includeRecentExecutions for automated runs. It also explains how to resolve display IDs with search, giving concrete when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations, it discloses that action is destructive and not reversible, deletes status/step results for these cases in this collection, ignores ids not present instead of raising an error, and preserves the test cases themselves in the project library. This aligns with destructiveHint=true and idempotentHint=true and adds concrete consequences, so no contradiction exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The definition opens with a crisp one-summary action, then uses each following sentence for a high-value detail: return value, edge-case behavior, destructive consequence set, and sibling routing. No filler or restatement of schema-aware items; the text is dense but fully purposeful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by naming the return values (collectionId, collectionName, removed) and the count semantics. It covers error behavior, idempotency, and the scope of damage, and it routes all nearby sibling cases. An agent has enough to decide when to call it and to establish expectation about the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Even though the schema covers both parameters, the description adds meaningful provenance: collectionId comes from list_test_collections, and caseIds are the numeric internal ids from list_test_cases, explicitly warning against TC-numbers. That disambiguates a common source error for this API family and goes well beyond the field descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description leads with a specific verb+resource pair: 'Remove test cases from a test collection.' It also disambiguates from nearby tools by explaining that test cases themselves stay in the project while the collection membership is removed, and it positions delete_test_case as the tool for project-wide case archival.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly names the inverse operation (add_cases_to_collection) and gives when-to-use alternatives for related but distinct scenarios: delete_test_collection for retiring the whole collection, remove_source_from_execution for source groups on an Executions page, and delete_test_case for archiving the case project-wide. It also tells the agent where IDs come from, which is a definitive selection and parameter-prefill hint.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the idempotentHint and non-destructive annotations, the description discloses atomicity, the auto-derivation rules, how group status rolls up, that execution status does not roll, and that set_execution_source_status overrides last only until the next report recomputes. This gives the agent substantial behavioral transparency beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every section earns its place: targeting, the Collection trap, status semantics, roll-up rules, overrides, and return value each address a failure mode an agent would hit. It is front-loaded with the main action and then builds outward into edge cases.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 9-parameter tool with no output schema, the description covers targeting, required/preferred parameters, status vocabularies, idempotency, atomicity, roll-up behavior, side effects, and even what is returned. It is complete enough for an agent to select and invoke the tool correctly without further research.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds critical semantic meaning beyond it: executionCaseId is preferred, executionId+caseId is the alternative, executionId from get_test_collection is rejected, and status 'auto' has a precise derivation order. It also clarifies the step-status vocabulary is separate from the case-status vocabulary.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific action and resource: 'Record the result of running one case inside a Manual Test Execution' and clarifies the exact scope by setting per-step pass/fail and/or case status. It also explicitly warns that 'A Test Collection is NOT an Execution', which separates it from collection-related siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the agent how to target the case, including the preferred executionCaseId and the alternative executionId + caseId, and when to use groupId. It also gives a concrete migration path for Collections, 'add it to an Execution with add_collections_to_execution', and points to update_execution for moving execution-level status.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
GitHub Badge
Glama performs regular codebase and documentation scans to:
- Confirm that the MCP server is working as expected.
- Confirm that there are no obvious security issues.
- Evaluate tool definition quality.
Our badge communicates server capabilities, safety, and installation instructions.
Card Badge
Copy to your README.md:
Score Badge
Copy to your README.md:
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/m00nreport/mcp-server'
If you have feedback or need assistance with the MCP directory API, please join our Discord server