TestAtlas

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and reveals meaningful behavior: matching is by substring containment, and the response includes full detail and ordered steps with doc-string/data-table flags. It also signals read-only intent via 'Use to read.' It omits minor behaviors like result ordering or default limit, but those are partially covered by the schema's limit parameter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight, information-dense sentences with no filler. The first front-loads the exact contents of the returned detail; the second adds a clear usage context. Every phrase earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 2-parameter read tool with full schema coverage, the description covers return content, matching behavior, and a use case. It could be improved by explicitly naming sibling tools to avoid confusion with search_scenarios, but nothing critical is missing for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with both parameters documented in the schema. The description's mention that name is matched by substring effectively restates the schema and adds no new semantic detail. Limit is not elaborated in the description, but the schema already covers its default and meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('get' + 'scenario') and enumerates the returned fields (feature, tags, kind, example-row count, file:line, ordered steps). It is clear, but it does not explicitly distinguish get_scenario from the sibling search_scenarios or state selection criteria relative to those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The sentence 'Use to read an existing scenario before writing a similar one' provides a clear and concrete use case. However, it does not explicitly mention when not to use this tool or point to alternatives such as search_scenarios for lighter-weight discovery.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It clearly lists what the returned data includes, which is valuable, but it does not describe matching behavior beyond 'contains the given text,' absence-of-results behavior, or any side effects. The 'inspect before reusing or changing' phrasing implies a read-only operation but does not state it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one dense, front-loaded sentence that lists the output fields and provides a usage recommendation. Every part contributes useful information, and there is no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a lookup tool with no output schema, the description covers the tool's purpose, result contents, and appropriate use context. Minor gaps remain, such as behavior when no definitions match and whether results are sorted or paginated, but these do not seriously hinder a correct call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with both 'query' and 'limit' documented in the input schema. The description adds little parameter meaning beyond restating that text matches the query, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb-resource pair ('get step definition') and enumerates the returned details: keyword, expression kind, method parameters, class/signature, file:line, and usage count. It is clear what the tool does, though it does not explicitly distinguish itself from siblings like search_steps or resolve_step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states when to use the tool: 'Use to inspect a step before reusing or changing it.' This gives a clear context for use, but it does not mention when not to use it or point to a specific alternative among the sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations available, the description carries the burden of behavioral disclosure. It does explain the return shape — affected scenarios with feature and connecting step text, plus step-definition and feature counts — which is useful. It does not explicitly state that the operation is read-only or describe matching nuances, but the language strongly implies a non-mutating analysis tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences front-load the tool's purpose and then specify what is returned. Every word earns its place, and there is no redundant restatement of the tool name or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter tool with no output schema, the description gives enough to understand the result payload and the kinds of targets. It could add an example or an edge case note, but the essential information for invoking it correctly is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description maps the target types to the schema enum and clarifies the purpose of the value parameter, but it adds little beyond the schema's own parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool's function: computing the blast radius of a change by listing affected test scenarios. It names the entity types (class, method, step definition, endpoint) and differentiates itself from sibling search/list tools by focusing on impact rather than generic lookup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool: when you need to understand which scenarios are affected by a change. However, it does not explicitly state when not to use it or name alternatives such as search_scenarios or resolve_step for different lookup needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden and provides useful context: results include real path where known, scenario blast radius, and are sorted by highest reach. This clarifies what the agent should expect beyond a bare list, though it does not address pagination or response shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

One dense sentence conveys the resource, output fields, a caveat, and ordering. Every clause contributes information, and there is no filler or repetition of the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, has no output schema, and the description adequately defines the return contents and sort order. It is slightly incomplete because it does not explain what 'scenario blast radius' means or how limit interacts with the listing, but overall the description gives enough for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the single 'limit' parameter is already documented as max rows with a default of 50. The description adds no additional meaning for this parameter, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states exactly what the tool returns: the HTTP endpoints/operations the suite calls, including verb, route, and scenario blast radius. It also specifies ordering (highest-reach first), which makes the tool's purpose concrete and distinct from sibling tools like search_steps or impact.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage: use this when you need an inventory of all endpoints and their reach. It does not explicitly state when to prefer this over alternatives, nor does it mention exclusions or related tools, so guidance is present only by implication.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses what is returned—tag names, counts, and ordering—which is valuable behavioral context. It does not explicitly state that the operation is read-only, but the verb in the tool name and the absence of side-effect language make that reasonably clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no filler. The first sentence packs in the output format, examples, and ordering; the second gives the usage context. It is slightly dense due to the em-dash style and parenthetical examples, but remains appropriately sized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description's explanation of 'every scenario tag ... with the number of scenarios carrying it, most-used first' reasonably conveys the return shape. It also explains the intended use case. Minor gaps like explicit JSON structure and default-limit behavior are not critical given the simple one-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents the only parameter, 'limit', with a full description ('Max tags (default 200)'), so schema coverage is 100%. The tool description adds no additional parameter semantics, meeting the baseline but not exceeding it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource—scenario tags—and specifies the output structure: each tag with its scenario count, sorted most-used first. It differentiates from siblings by focusing on tags rather than steps, endpoints, or statistics, but it lacks an explicit verb like 'List' and instead opens with a noun phrase.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides a clear use case: 'Use to tag new scenarios consistently with what already exists.' This tells an agent when to invoke it, though it does not mention exclusions or explicitly name alternative tools for other scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral disclosure burden. It explains that the graph is derived from cross-project binds_to/uses_type/inherits edges and that edges are weighted, which is useful beyond the schema. It does not detail output formatting, but the core behavior is transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and well-structured, with the core purpose front-loaded and a helpful example question. Each sentence adds relevant information without unnecessary filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description explains what the graph contains, how it is derived, and gives an example use case. It lacks explicit output-format details, but the absence of an output schema is partially mitigated by the clear high-level return semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The description only reiterates the optional project filter without adding new semantics, keeping this at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as providing a project dependency graph, specifying what it contains and giving a concrete example question it answers. It is distinct from the sibling tools, which focus on stats, impact, steps, scenarios, endpoints, and tags.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear use context: it answers dependency-related questions like 'what depends on the Party project?' and mentions the optional project filter. It does not explicitly contrast with alternatives, but the intended use case is evident from the description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden and does so thoroughly. It discloses the three status outcomes, how matches are ranked in the 'none' case, the exact fields returned per match, and that matching ignores Given/When/Then keywords.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but well-structured, front-loading the core purpose before explaining statuses and return contents. Every sentence adds value, though it is slightly long and could be tightened without losing meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is exceptionally complete. It explains what the tool returns, how statuses should be interpreted, and why the agent should use it, leaving no critical gaps for correct invocation and interpretation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The description reinforces that the keyword parameter is informational and keyword-agnostic, but it does not add substantial per-parameter semantics beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's verb and resource: it resolves a Gherkin step phrase to the existing step definition(s) that would bind it. It also differentiates itself from authoring a new step and explains its keyword-agnostic matching behavior, making its purpose distinct from sibling search tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit timing guidance: use this BEFORE writing a new step so the agent reuses existing definitions instead of duplicating. It does not explicitly name sibling alternatives or state when not to use it, but the usage context is clear and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does disclose the search scope and that it returns matching scenarios, but it does not explicitly state that the operation is read-only, nor does it mention result limits, pagination, or ordering. These are meaningful gaps for a search tool without annotation support.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the core operation and then specifies searchable fields and return type. There is no redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one parameter, no output schema, and no annotations, the description covers the main essentials: what is searched and what is returned. Some detail about the returned scenario shape or result handling would make it fully complete, but it is adequate for straightforward invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema only describes 'query' as 'Search terms,' but the description adds significant meaning by specifying exactly which text is searched: feature names, scenario names, step text, and tags. This extra context helps the agent formulate effective queries and goes beyond the schema's minimal description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a full-text search operation with a specific resource (scenarios) and enumerates the fields searched: feature name, scenario name, step text, and tags. This distinguishes it from sibling tools like search_steps and get_scenario without needing to inspect schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it: when a free-text search across scenario content is needed. However, it does not explicitly state when not to use it or name alternatives such as search_steps, so the routing to the correct sibling is left mostly to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral burden. It discloses that the tool returns matching step definitions, but does not cover ordering, pagination, matching semantics, or error behavior. For a straightforward search tool this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single focused statement that front-loads the core purpose, specifies the searchable fields, and states the return value with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter search tool with no output schema, the description covers the main requirement: what is searched and what is returned. Minor gaps like result limits or ordering are present, but the tool's simplicity reduces the impact of those omissions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the query parameter, and the description adds meaning by clarifying that the query applies to expression text, method, and class name. This goes beyond the schema's generic 'Search terms' description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Full-text search') and a specific resource ('step definitions'), and identifies the exact fields being searched: expression text, method, and class name. This distinguishes it from sibling tools like search_scenarios, which targets scenarios rather than step definitions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: use this tool to full-text search step definitions by expression, method, or class name. It does not explicitly call out when not to use it or name alternatives, but the scope is specific enough for an agent to infer appropriate use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It communicates a non-mutating reporting operation by using 'Summary' and specifying the categories returned. It does not describe response shape, freshness, computation cost, or any other behavioral nuance, but for a simple zero-parameter stats tool the basic behavior is reasonably clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, tightly packed sentence that leads with the resource ('Summary of the test map') and then lists the specific included metrics. There is no filler, and every phrase contributes useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no annotations and no output schema, the description is mostly sufficient: it tells the agent what kind of data the summary contains. It could be more explicit about the exact output format and how it differs from sibling tools, but those are not critical for basic invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and schema description coverage is 100%, so the 0-parameter baseline of 4 applies. No parameter documentation is needed, and the description does not need to compensate for any schema gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource ('the test map') and enumerates the exact contents of the summary: project/class/method counts, class-kind breakdown, endpoints, and edge tallies. It is not a tautology and leaves little doubt about what the tool does, though it lacks an explicit verb and does not differentiate from sibling tools like list_endpoints or project_dependencies.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The term 'Summary' and the focus on aggregate counts strongly imply use when a high-level overview is needed rather than detailed step/endpoint listings. However, the description does not explicitly state when to use this tool versus its siblings, nor does it mention any exclusions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose a key non-obvious behavior: allowed values are 'best-effort' and derived via specific rules (cucumber types, regex alternations, free parameters). This gives the agent realistic expectations about output quality. It does not mention output structure or side effects, but the catalog/resource framing implies a read-only operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded, with the core definition first and the usage context second. The final fragment about optional filters is somewhat redundant with the schema, but it is short and does not distract from the main message.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Without an output schema, the description conveys the main return content (step definitions, placeholders, allowed values) and the intended use case. It does not detail the exact response structure or pagination, but the limit default is in the schema and the catalog tool is simple enough that this is likely sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All three parameters have schema descriptions (100% coverage), so the baseline is 3. The description adds no new parameter-level meaning beyond summarizing 'Optional keyword/query filters'; the explanation of value extraction pertains to the output, not the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource as a catalog of reusable step definitions and explains what it provides: placeholders and best-effort allowed values derived from the expression. It also states a usage goal ('compose new scenarios'), which distinguishes it from a generic search tool. However, it does not explicitly contrast with sibling tools like search_steps, so differentiation is implied rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use to compose new scenarios from steps and values that already exist,' giving a clear context for when to use this tool. It does not name alternatives or state when not to use it, so exclusion guidance is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.