Server Quality Checklist

Tool Scores

• qql_helpA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish readOnly/idempotent/non-destructive, and the description adds value beyond them: no API call, static text, ~2ms latency, and free invocations. It also discloses the failure mode it guards against (rejected queries from nonexistent attributes) and that field names diverge from write tools. No contradiction with annotations; only the output format of the returned reference text is left unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence earns its place: purpose, critical topics, rationale, session cadence, and cost. The inline topic list partially duplicates the schema enum, and the most actionable instruction ('once per session') sits mid-description rather than front-loaded, but the density of useful information justifies the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter, static-reference tool with fully documented enum values and strong annotations, the description covers everything essential: when to call, which topics exist, which matter most, why, and cost. The only gap is the unspecified shape of the returned reference text, which is minor given the low complexity and absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with rich per-enum-value descriptions, so the baseline is 3. The description adds meaning by highlighting which topics matter most (entities, enumValues) and why — field names differ from write tools and unknown attributes cause rejection rather than being ignored. This elevates understanding beyond what the schema alone provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action — read the QQL reference — and enumerates the eight available topics, making the tool's function clear. It distinguishes itself from the qql_search sibling by framing this as the prerequisite reference lookup. Slight deduction: the phrasing is advisory ('Read the QQL reference before writing a query') rather than declarative about what the tool returns, so an agent must infer that the deliverable is the reference text for the requested topic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly names the alternative (qql_search), prescribes cadence ('once per session before the first qql_search'), and discourages the anti-pattern ('rather than guessing and retrying'). It also justifies the rule by disclosing the consequence — QQL rejects queries naming nonexistent attributes instead of ignoring them, and field names differ from the write tools. Nothing about when or when not to call is left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_regression_runA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish that the operation is not read-only, not idempotent, and not destructive. The description adds useful context beyond annotations: it resolves the source into cases, makes two API calls, and opens a run without closing one, including a rough latency estimate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but every sentence earns its place: purpose, alternative routing, and behavioral/cost notes. The core purpose is front-loaded, and there is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 8 parameters and an output schema, the description covers the main invocation decision, the source-resolution behavior, and sibling alternatives. It is slightly incomplete only in not explaining the optional milestone/environment/description parameters or whether source selectors are mutually exclusive, but the schema and output schema mitigate the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 63%, so the description needs to add moderate value. It clarifies the semantics of the three source selectors (plan_id, suite_ids, include_cases) by explaining the tool resolves cases itself, but optional parameters such as milestone_id, environment_id, and description remain undocumented in both schema and description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb phrase, 'Build and start a test run from a suite, a test plan, or an explicit list of case IDs,' naming the resource and supported sources. It also explicitly differentiates the tool from qase_run_upsert and qase_ci_report, so an agent can select it without opening sibling schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear when-to-use guidance: launch a regression cycle from a source without first querying cases. It names alternatives and the conditions that select them — qase_run_upsert for hand-assembled runs, qase_ci_report for already-finished pipelines — and clarifies that this tool opens but does not close a run.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_result_recordA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as a non-read-only, non-idempotent write. The description adds useful behavior: results require an existing run, results are sent together in one API call, status must be a string label rather than a numeric ID, and cost grows with payload size. Repeated-call semantics (append vs. update) are not disclosed, but core behavior is well conveyed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but each sentence carries informational weight: definition, batching, alternative tool, status constraints, and cost. There is slight redundancy between 'Pass several results in one call' and 'takes a list and sends them together,' but the structure remains purposeful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description does not explain return values, but that is not critical for invoking the tool correctly. It covers the run prerequisite, batching behavior, status format, cost expectation, and the main sibling alternative. It could explicitly mention qase_run_upsert for non-CI run creation, but the given guidance is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful guidance by clarifying the status enum usage, explicitly rejecting numeric IDs unlike case enums, and stressing that the results argument is a batchable list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states a specific verb and resource: 'Record one or more results into an existing run.' The case-vs-result distinction and the explicit alternative to qase_ci_report make its scope unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs batching multiple results into one call instead of calling per test, and names qase_ci_report as the correct alternative for a finished CI job without an existing run. It does not state what to do when a run is missing outside a CI context, but the prerequisite is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_ci_reportA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds rich context beyond annotations, including the create/record/complete sequence, the no-half-open-run guarantee, and performance trade-offs. However, it states that the run is always completed, while the schema's `complete` parameter defaults to true but can be false, making that guarantee conditional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core behavior, then provides explicit routing rules and a useful cost/performance note. Every sentence contributes to selection or invocation, and the structure makes it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a multi-step tool with six parameters and an output schema, the description is nearly complete: it explains the workflow, result shape, and alternatives. The only gap is the unconditional 'completes it' language, which should acknowledge the `complete` parameter's ability to leave the run open.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (83%), so the baseline is 3. The description adds only a compact restatement of result requirements like numeric `case_id`, status, and optional attachment hashes; it does not add meaningful semantics for top-level parameters such as `code`, `title`, `environment_id`, or `complete` beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: 'Report a whole CI run in one call: creates the run, records every result, and completes it.' It clearly identifies the operation and distinguishes it from the related split tools without requiring schema inspection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit selection guidance: use this after a pipeline finishes, use qase_result_record when the run already exists and results arrive in stages, and use qase_run_upsert when the run must stay open. These conditional rules fully route an agent to the correct sibling tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_getA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the operation read-only, idempotent, and non-destructive, so the description doesn't need to repeat that. It adds useful behavioral context: one API call, measured latency, and a concrete performance penalty for doing this in a loop instead of using qql_search. It doesn't cover error/not-found behavior, but that is a minor gap given the strong annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the operation and entity list, followed by code requirements, then usage boundaries, then cost guidance. Every sentence earns its place, and the measured performance comparison directly supports the recommended routing to qql_search.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-record fetch tool, the description covers scope, code requirements, field projection, and alternative routing, which is enough for correct selection and invocation. It could mention response/error shape and the include parameter more explicitly, but the schema covers include and annotations cover safety.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful nuance beyond the schema: code is required for project-scoped entities but can be omitted for global ones, and fields can be narrowed or set to ['*']. It does not add much about the include parameter, but the schema already documents that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: 'Fetch one known record by type and ID', and enumerates all 16 supported entity types. This clearly distinguishes it from qql_search, which is for batched/filtered lookup, without requiring an agent to open the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit selection rule: 'Use this only when you already know the ID and want a single record.' It names the alternative, qql_search, and lists the exact conditions that should route an agent away from this tool: several records, filtered or cross-project queries, or loops.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_attachment_uploadA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already signal a non-read-only, non-idempotent operation, and the description adds meaningful context: uploading alone attaches nothing, the hash must be passed on, file_path will fail against remote servers, and each upload costs one API call. This goes well beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average, but every sentence earns its place: purpose, parameter choice, failure mode, downstream usage, and cost. It is front-loaded with the core purpose and the hash contract, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description clearly states the return contract: the tool returns a hash to use in the attachments field of related tools. It also covers the critical failure mode and reuse recommendation, so an agent has enough context to call and use this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description reinforces the choice between file_base64 and file_path and confirms filename is always required, but most of this is already present in the schema. It adds little genuinely new per-parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Upload a file and get back the hash that other tools reference it by.' It also distinguishes this tool from qase_api by noting it is the only tool that sends multipart/form-data, so an agent can differentiate it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: use file_base64 unless the server is on the same machine, file_path only works for local stdio servers, and the returned hash must be passed to specific sibling tools' attachments fields. It even explains why qase_api cannot handle these uploads, covering the main alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_case_upsertA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations mark this as a write operation (readOnlyHint=false, destructiveHint=false without idempotency), but the description adds meaningful behavior: enums normalize labels vs numeric IDs, steps can be classic or Gherkin, shared-step references resolve to shared_step_hash on read, and the review workflow may intercept direct writes. It doesn't spell out error semantics or whether update is partial or full, but the write path is well disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact for 21 parameters, front-loaded with the create/update decision and alternatives. Every sentence adds useful guidance. The only waste is the measured timing detail ('Ten sequential calls measured 5.6s...') — it supports the bulk-create recommendation but is more evidence than need for an agent decision.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 21 parameters, 52% schema coverage, nested objects, no output schema, and a complex upsert flow, this description covers all critical aspects: the id branch, enum values, shared steps, review-mode caveat, and the bulk alternative. Nothing an agent needs to invoke it correctly is missing from the core decision path.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 52%, and this description compensates for the largest gap by explaining the core branching parameter semantics: id selects update vs create. It also gives meaning to enum fields collectively, and clarifies the steps object structure including the alias shared_step_hash for shared. That's exactly the kind of meaning the raw schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the exact verb-resource relationship: 'Create or update a single test case' and immediately clarifies the id-based branching. This differentiates it from qase_case_bulk_create by name and by role. The first sentence alone gives an agent everything needed to decide whether this tool applies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use qase_case_bulk_create instead: 'Writing more than one case? Use qase_case_bulk_create instead.' It also instructs running qase_discover_tools with 'review' when the project has Test case review enabled, directing the agent to alternatives when direct writes may be blocked. That's excellent routing guidance, not just description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_defect_upsertA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnly=false, openWorld=true, idempotent=false, destructive=false), the description discloses create/update branching, mandatory-field rejection, server-side severity label mapping, the dedicated resolve endpoint for status='resolved', and the lack of run/result attachment. No statement contradicts the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well front-loaded and every major sentence adds operational value, but the final cost sentence repeats 'about 0.5s' three times, introducing unnecessary redundancy. Apart from that, the structure is clear and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 9-parameter upsert with nested objects and no output schema, the description covers the critical invocation decisions: create vs update, required fields, label handling, resolve behavior, alternatives, duplicate avoidance, and cost. It does not describe return values or custom_field semantics, but those gaps do not block correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema coverage at 56%, the description compensates by explaining the functional meaning of id, title/actual_result/severity requirements, and the enum labels for severity and status. It also clarifies how severity labels map to workspace IDs, which the schema does not convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource ('Create or update a defect') and explains the create/update distinction by id, making the operation unambiguous. It also names the nearby sibling qase_triage_defect, so the agent can tell this tool apart from the test-failure-specific alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use qase_triage_defect when the defect stems from a specific test failure, and instructs the agent to search with qql_search before filing duplicates. It also warns that runs/results cannot be attached to a defect, outlining when not to expect linking behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_run_upsertA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description carries the behavioral burden well despite minimal annotations. It reveals the create/update branch, the ordering dependency with qase_result_record, the role of a run as a results container, and the operational cost of one API call. This goes beyond what the annotations provide and contains no contradiction with them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core behavior, and every clause earns its place. It packs branching behavior, ordering, scoping options, alternative-tool routing, and cost into a few tight sentences with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 13-parameter tool with no output schema, the description provides essential context: sequencing, scoping options, and tool-selection rules. A small gap is that it does not warn about the limited update support mentioned in the schema, but overall the description gives an agent enough to call this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 54%, so the description meaningfully compensates by explaining the defining behavior of 'id' (new vs update) and summarizing how milestone, environment, plan, and case IDs scope the run. It does not explain every optional parameter like custom_field or timestamps, but the schema already provides basic descriptions for those.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: 'Create or update a test run.' It immediately clarifies the create-vs-update branching based on 'id' and distinguishes itself from nearby siblings like qase_result_record, qase_regression_run, and qase_ci_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage guidance is explicit and actionable: open the run before calling qase_result_record; use qase_regression_run when building from a suite or plan; use qase_ci_report when the CI job has already finished. It gives clear selection criteria and even explains why the alternative avoids leaving a half-finished run.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_apiA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations, the description discloses JSON-only request bodies, inability to upload files, raw API response passthrough, DELETE confirmation, typical latency/cost, and the absence of caching, pagination help, and extra retries. These are concrete behavioral traits that help an agent predict side effects. There is no contradiction with the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose and usage, then moves through exclusions, safety behavior, and operational limits. Every sentence carries decision-relevant information for a generic passthrough tool, with no filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a deliberately generic tool with no output schema, the description covers everything needed to invoke it correctly: HTTP method, path convention, optional body/query, reference link, JSON-only constraint, file upload exception, DELETE confirmation, cost/latency, and limitations. It sets correct expectations for response passthrough and lack of pagination/retries.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameters are already documented at baseline. The description adds practical semantics: the path must start with /v1/, body and query are optional, and the API accepts JSON only. This goes beyond what the schema alone provides and justifies a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence, 'Call any Qase REST endpoint directly, for the few things no dedicated tool covers,' gives a specific verb, resource, and role as a fallback. It explicitly distinguishes itself from siblings by positioning dedicated tools as the preferred route and this tool as the catch-all.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to prefer a dedicated tool wherever one exists and gives the reason (enum normalization, validation, shaped responses). It also names qase_attachment_upload as the alternative for multipart uploads, and notes DELETE confirmation behavior. This is clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_discover_toolsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  It discloses behaviors beyond the annotations: hidden-by-default state, in-memory matching, no API call, ~3ms cost, and that matching tools become activated and callable. These details complement the readOnlyHint, openWorldHint, and idempotentHint annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence earns its place: purpose, hidden capabilities, usage trigger, query constraints, and cost. The description is dense but well-organized and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a discovery tool with readOnly, openWorld, and idempotent annotations plus an output schema, the description covers all essential information: what the tool does, when to use it, how to search, side effects, and cost. Nothing needed for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by explaining query matching semantics and giving concrete example queries, though it does not add detail for the activate or category parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence names a specific verb ('Find and switch on') and resource ('tools that are hidden by default'), and the description lists concrete hidden capabilities such as deletes, attachments, and custom-field management. This makes it clearly distinguishable from sibling tools like qase_get or qase_api.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent when to call this tool: never conclude a capability is missing without searching here first. It also provides query strategy, examples, and the rule that every word in the query must appear in a tool name or description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_project_contextA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral context beyond the annotations: 100-entity pagination limits per collection, the coverage field shape, truncation semantics, full:true behavior, cold vs. cached latency, response size, and cost in API calls. It also confirms read-only/idempotent behavior through 'Cached for 5 minutes' and 'repeat calls... return in about 5ms.'

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: purpose, first-call guidance, pagination behavior, alternatives, cost, latency, caching, and full-mode tradeoffs are all relevant for correct tool selection and invocation. The core purpose is front-loaded before performance and alternative details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with this complexity, the description covers what the tool returns, pagination limits, truncation detection, when to pass full:true, cost implications, alternatives, and caching behavior. The output schema handles the detailed return structure, so nothing essential is missing from the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters are already documented. The description adds extra meaning for the full parameter by explaining the cost model ('one extra call per 100 entities') and directing users toward targeted tools when only a subset is needed. That goes beyond the schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description gives a specific verb and resource with an explicit inventory: 'Seed everything about a project in one call: project details, the full suite tree, milestones, environments, custom fields, and users.' It also clearly distinguishes itself from siblings by naming qase_get and qql_search as alternatives for different query patterns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage context is explicit: 'This is the first call to make when starting work on a project' and it 'replaces six separate list calls.' It also states when not to use it: 'For a single record you already have the ID for, qase_get is cheaper; for filtered or cross-project questions, use qql_search.' This gives the agent clear decision rules.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qase_triage_defectA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations by disclosing API rejection behavior for missing fields, the inability to attach existing runs/results, the lack of a link in the UI, cost/latency, and per-defect call behavior. It does not contradict the readOnly/idempotent/destructive annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence carries operational information: purpose, requirements, API limitation, alternative tool, latency, and batching guidance. It is dense but not bloated, and the most important facts are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for an agent deciding whether and how to call this tool: it explains purpose, required fields, API limitations, expected behavior, cost, and the right alternative. With an output schema present, describing return values is unnecessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful parameter guidance: title, actual_result, and severity are required by the API, and failing results should be referenced inside actual_result. Schema coverage is 63%, so the description partially compensates, though optional params like tags, description, and custom_field remain unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Create a defect from a test failure,' giving a specific verb, resource, and trigger context. It also explicitly names qase_defect_upsert as the alternative for non-failure defects, making sibling differentiation clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states exactly when to use this tool (test failures) and when not to (unrelated defects, use qase_defect_upsert). It also provides operational guidance: cluster identical failures, file one defect per distinct cause, and expect one API call per defect.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qql_searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even though annotations already mark this as read-only, idempotent, and non-destructive, the description adds valuable behavioral context: a query naming an unknown attribute is rejected outright, this costs one API call, latency ranges are provided for different page sizes, and one search is preferred over N single fetches. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose, followed by concrete use cases, prerequisite guidance, alternative selection, and performance details. Every sentence earns its place, and the examples make the QQL query style concrete without unnecessary padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complex query language, rich annotations, complete input schema, and presence of an output schema, the description covers everything an agent needs: when to use it, when not to use it, how to get syntax help, failure behavior, and cost/performance trade-offs. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema by explaining that query fields and enums require qql_help, unknown attributes are rejected, and the tool is meant for batch retrieval rather than point lookups. This adds meaning to the query parameter beyond the inline examples in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Search any entity with Qase Query Language' and enumerates capabilities (filtering, cross-project queries, sorting, aggregation). It also explicitly contrasts with qase_get by saying it is not for 'give me this one record by ID', which makes it easy to distinguish from its sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: it is 'the right tool for every question that is not "give me this one record by ID"' and the right way to fetch many records instead of looping over qase_get. It also tells the agent to call qql_help first for syntax and enum values, and to use qase_get instead when the entity and ID are already known.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.