Skip to main content
Glama
SmartBear

SmartBear MCP server

Official
by SmartBear

A Model Context Protocol (MCP) server which provides AI assistants with seamless access to SmartBear's suite of testing and monitoring tools, including BearQ, BugSnag, Reflect, Swagger, PactFlow, Pact Broker, QMetry, QTM4J, Zephyr and Collaborator.

What is MCP?

The Model Context Protocol (MCP) is an open standard that enables AI assistants to securely connect to external data sources and tools. This server exposes SmartBear's APIs through natural language interfaces, allowing you to query your testing data, analyze performance metrics, and manage test automation directly from your AI workflow.

Related MCP server: Sauce Labs MCP Server

Supported Tools

See individual guides for suggested prompts and supported tools and resources:

  • BearQ - AI-powered QA test management and execution capabilities

  • BugSnag - Comprehensive error monitoring and debugging capabilities

  • Reflect - Test management and execution capabilities

  • Swagger

    • Portal - Portal and product management capabilities

    • Studio - API and Domain management capabilities, including AI-powered API generation from prompts and automatic standardization

    • Contract Testing (PactFlow) - Contract testing capabilities

    • Functional Testing - API test discovery capabilities

  • QMetry - QMetry Test Management capabilities

  • Zephyr - Zephyr Test Management capabilities

  • Collaborator - Review and Remote System Configuration management capabilities

  • QTM4J - QTM4J Test Management for Jira capabilities

Remote MCP Servers

For BugSnag, Swagger, and Zephyr, SmartBear hosts Remote MCP Servers that you can connect to directly from your MCP client via a URL β€” no installation, Node.js, or API tokens required. Authentication is handled through an OAuth browser flow.

Product

Server URL

Swagger

https://swagger.mcp.smartbear.com/mcp

BugSnag

https://bugsnag.mcp.smartbear.com/mcp

Zephyr

https://zephyr.mcp.smartbear.com/mcp

See the Remote MCP Servers guide for per-client setup instructions. You can connect to multiple remote servers at the same time.

Need BearQ, Reflect, QMetry, QTM4J, PactFlow, Collaborator, or Functional Testing? These products are only available via the local npm package below, which bundles all products into a single MCP server.

Prerequisites

  • Node.js 22+ and npm

  • Access to SmartBear products (BugSnag, Reflect, Swagger, QMetry, QTM4J or Zephyr)

  • Valid API tokens for the products you want to integrate

Local MCP Server Installation (npm)

For all products β€” or if you prefer running the server locally β€” the MCP server is distributed as an npm package @smartbear/mcp, making it easy to integrate into your development workflow.

The server is started with the API key or auth token that you use with your SmartBear product(s). They are optional and can be removed from your configuration if you aren't using the product. For BugSnag, if you provide a project API key it will narrow down all searches to a single project in your BugSnag dashboard. Leave this field blank if you wish to interact across multiple projects at a time.

VS Code with Copilot

For the quickest setup, use the "MCP: Add server…" command in the Command Palette to add the @smartbear/mcp npm package.

Alternatively, you can use npx (or globally install) the @smartbear/mcp package to run the server and add the following to your .vscode/mcp.json file:

{
  "servers": {
    "smartbear": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@smartbear/mcp@latest"
      ],
      "env": {
        "BEARQ_API_TOKEN": "${input:bearq_api_token}",
        "BEARQ_API_BASE_URL": "${input:bearq_api_base_url}",
        "BUGSNAG_AUTH_TOKEN": "${input:bugsnag_auth_token}",
        "BUGSNAG_PROJECT_API_KEY": "${input:bugsnag_project_api_key}",
        "REFLECT_API_TOKEN": "${input:reflect_api_token}",
        "SWAGGER_API_KEY": "${input:swagger_api_key}",
        "SWAGGER_PORTAL_BASE_PATH": "${input:swagger_portal_base_path}",
        "SWAGGER_REGISTRY_BASE_PATH": "${input:swagger_registry_base_path}",
        "SWAGGER_UI_BASE_PATH": "${input:swagger_ui_base_path}",
        "PACT_BROKER_BASE_URL": "${input:pact_broker_base_url}",
        "PACT_BROKER_TOKEN": "${input:pact_broker_token}",
        "PACT_BROKER_USERNAME": "${input:pact_broker_username}",
        "PACT_BROKER_PASSWORD": "${input:pact_broker_password}",
        "QMETRY_API_KEY": "${input:qmetry_api_key}",
        "QMETRY_BASE_URL": "${input:qmetry_base_url}",
        "ZEPHYR_API_TOKEN": "${input:zephyr_api_token}",
        "ZEPHYR_BASE_URL": "${input:zephyr_base_url}",
        "COLLABORATOR_BASE_URL": "${input:collab_base_url}",
        "COLLABORATOR_USERNAME": "${input:collab_username}",
        "COLLABORATOR_LOGIN_TICKET": "${input:collab_login_ticket}",
        "QTM4J_API_KEY": "${input:qtm4j_api_key}",
        "QTM4J_BASE_URL": "${input:qtm4j_base_url}",
        "QTM4J_AUTOMATION_API_KEY": "${input:qtm4j_automation_api_key}",
        "SWAGGER_FUNCTIONAL_TESTING_API_TOKEN": "${input:swagger_functional_testing_api_token}",
        "SWAGGER_FUNCTIONAL_TESTING_BASE_PATH": "${input:swagger_functional_testing_base_path}"
      }
    }
  },
  "inputs": [
      {
         "id": "bearq_api_token",
         "type": "promptString",
         "description": "BearQ workspace API token - leave blank to disable BearQ tools",
         "password": true
      },
      {
         "id": "bearq_api_base_url",
         "type": "promptString",
         "description": "BearQ API base URL - leave blank to use the default (https://api.bearq.smartbear.com)",
         "password": false
      },
      {
         "id": "bugsnag_auth_token",
         "type": "promptString",
         "description": "BugSnag Auth Token - leave blank to disable BugSnag tools",
         "password": true
      },
      {
         "id": "bugsnag_project_api_key",
         "type": "promptString",
         "description": "BugSnag Project API Key - for single project interactions",
         "password": false
      },
      {
         "id": "reflect_api_token",
         "type": "promptString",
         "description": "Reflect API Token - leave blank to disable Reflect tools",
         "password": true
      },
      {
         "id": "swagger_api_key",
         "type": "promptString",
         "description": "Swagger API Key - leave blank to disable Swagger tools",
         "password": true
      },
      {
         "id": "swagger_portal_base_path",
         "type": "promptString",
         "description": "Swagger Portal API base path - only needed for on-premise installations (leave blank for cloud)",
         "password": false
      },
      {
         "id": "swagger_registry_base_path",
         "type": "promptString",
         "description": "Swagger Registry API base path - only needed for on-premise installations (leave blank for cloud)",
         "password": false
      },
      {
         "id": "swagger_ui_base_path",
         "type": "promptString",
         "description": "Swagger UI base path - only needed for on-premise installations (leave blank for cloud)",
         "password": false
      },
      {
         "id": "pact_broker_base_url",
         "type": "promptString",
         "description": "PactFlow or Pact Broker base url - leave blank to disable the tools",
         "password": true
      },
      {
         "id": "pact_broker_token",
         "type": "promptString",
         "description": "PactFlow Authentication Token",
         "password": true
      },
      {
         "id": "pact_broker_username",
         "type": "promptString",
         "description": "Pact Broker Username",
         "password": true
      },
      {
         "id": "pact_broker_password",
         "type": "promptString",
         "description": "Pact Broker Password",
         "password": true
      },
      {
          "id": "qmetry_api_key",
          "type": "promptString",
          "description": "QMetry Open API Key",
          "password": true
      },
      {
          "id": "qmetry_base_url",
          "type": "promptString",
          "description": "By default, connects to https://testmanagement.qmetry.com. Change to a custom QMetry server URL or a region-specific endpoint if needed.",
          "password": false
      },
      {
          "id": "zephyr_api_token",
          "type": "promptString",
          "description": "Zephyr API token - leave blank to disable Zephyr tools",
          "password": true
      },
      {
          "id": "zephyr_base_url",
          "type": "promptString",
          "description": "Zephyr API base URL. By default, connects to https://api.zephyrscale.smartbear.com/v2. Change to region-specific endpoint if needed.",
          "password": false
      },
      {
          "id": "collab_base_url",
          "type": "promptString",
          "description": "Collab base url",
          "password": true
      },
      {
          "id": "collab_username",
          "type": "promptString",
          "description": "Collab username",
          "password": true
      },
      {
          "id": "collab_login_ticket",
          "type": "promptString",
          "description": "Collab login ticket",
          "password": true
      },
      {
          "id": "qtm4j_api_key",
          "type": "promptString",
          "description": "QTM4J API Key",
          "password": true
    },
    {
          "id": "qtm4j_base_url",
          "type": "promptString",
          "description": "US region (default): https://qtmcloud.qmetry.com. Australia region: https://syd-qtmcloud.qmetry.com.",
          "password": false
    },
    {
          "id": "qtm4j_automation_api_key",
          "type": "promptString",
          "description": "QTM4J Automation API Key - required for automation tools, leave blank to disable them",
          "password": true
    },
    {
          "id": "swagger_functional_testing_api_token",
          "type": "promptString",
          "description": "Swagger Functional Testing API Token - leave blank to disable Functional Testing tools",
          "password": true
    },
    {
          "id": "swagger_functional_testing_base_path",
          "type": "promptString",
          "description": "Swagger Functional Testing API Base URL - leave blank to use the default (https://api.reflect.run/v1)",
          "password": false
    }
  ]
}

Claude Desktop

Add the following configuration to your claude_desktop_config.json to launch the MCP server via npx:

{
  "mcpServers": {
    "smartbear": {
      "command": "npx",
      "args": [
        "-y",
        "@smartbear/mcp@latest"
      ],
      "env": {
        "BEARQ_API_TOKEN": "your_bearq_api_token",
        "BUGSNAG_AUTH_TOKEN": "your_personal_auth_token",
        "BUGSNAG_PROJECT_API_KEY": "your_project_api_key",
        "REFLECT_API_TOKEN": "your_reflect_token",
        "SWAGGER_API_KEY": "your_swagger_key",
        "SWAGGER_PORTAL_BASE_PATH": "https://api.portal.swaggerhub.com/v1",
        "SWAGGER_REGISTRY_BASE_PATH": "https://api.swaggerhub.com",
        "SWAGGER_UI_BASE_PATH": "https://app.swaggerhub.com",
        "PACT_BROKER_BASE_URL": "your_pactflow_or_pactbroker_base_url",
        "PACT_BROKER_TOKEN": "your_pactflow_token",
        "PACT_BROKER_USERNAME": "your_pact_broker_username",
        "PACT_BROKER_PASSWORD": "your_pact_broker_password",
        "QMETRY_API_KEY": "your_qmetry_api_key",
        "QMETRY_BASE_URL": "https://testmanagement.qmetry.com",
        "ZEPHYR_API_TOKEN": "your_zephyr_api_token",
        "ZEPHYR_BASE_URL": "https://api.zephyrscale.smartbear.com/v2",
        "COLLABORATOR_BASE_URL": "your collab base url",
        "COLLABORATOR_USERNAME": "your collab user name",
        "COLLABORATOR_LOGIN_TICKET": "your collab login ticket",
        "QTM4J_API_KEY": "your_qtm4j_key",
        "QTM4J_BASE_URL": "https://qtmcloud.qmetry.com",
        "QTM4J_AUTOMATION_API_KEY": "your_qtm4j_automation_api_key",
        "SWAGGER_FUNCTIONAL_TESTING_API_TOKEN": "your_swagger_functional_testing_api_token",
        "SWAGGER_FUNCTIONAL_TESTING_BASE_PATH": "https://api.reflect.run/v1"
      }
    }
  }
}

Documentation

For detailed introduction, examples, and advanced configuration visit our πŸ“– Full Documentation

Local Development

For developers who want to contribute to the SmartBear MCP server, please see the CONTRIBUTING.md guide.

License

This MCP server is licensed under the MIT License. This means you are free to use, modify, and distribute the software, subject to the terms and conditions of the MIT License. For more details, please see the LICENSE file in the project repository.

Support


SmartBear MCP Server - Bringing the power of SmartBear's testing and monitoring ecosystem to your AI-powered development workflow.

Available Tools

308 tools
bearq_chat_with_qa_leadBearQ: Chat with QA LeadA
Read-onlyIdempotent
Inspect

Sends an open-ended instruction to BearQ's QA lead agent. Use this when no other BearQ tool fits β€” the QA lead can list, create, and update test cases, manage functional areas, and read the application model, and acts as a general-purpose escape hatch.

Toolset: Tasks

Parameters:

  • instruction (string) required: Natural language instruction to send to the BearQ QA lead agent.

ParametersJSON Schema
NameRequiredDescriptionDefault
instructionYesNatural language instruction to send to the BearQ QA lead agent.

TDQS

A3.5/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description says the QA lead 'can list, create, and update test cases, manage functional areas, and read the application model,' implying potentially mutating behavior. However, annotations declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. This directly contradicts the described create/update capabilities, so the behavioral disclosure is not just incomplete but inconsistent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the action and main usage directive. The capabilities sentence is informative, and the structured parameter section is easy to scan. Minor duplication exists because the parameter line repeats the schema, but overall it is appropriately sized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description does not explain what happens after the instruction is sent, whether the work is performed synchronously, or what the return value is. The 'Toolset: Tasks' hint and sibling tools like bearq_get_task and bearq_wait_for_task suggest an asynchronous task flow, but the description leaves that to inference. Combined with the annotation contradiction, this is incomplete for an open-ended, potentially effectful tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description repeats the same parameter meaning: 'Natural language instruction to send to the BearQ QA lead agent.' It adds no extra detail about format, length, examples, or what kinds of instructions work best. With full schema coverage, baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Sends an open-ended instruction to BearQ's QA lead agent.' It then clarifies the tool's role as a general-purpose escape hatch when no other BearQ tool fits, which clearly distinguishes it from the many sibling BearQ tools. The list of capabilities (list/create/update test cases, manage functional areas, read the application model) adds concreteness.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool: 'Use this when no other BearQ tool fits.' This also implies a when-not: if a more specific BearQ tool applies, use that instead. The capability list gives the agent a sense of what kinds of requests are appropriate, making the usage guidance strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_delete_test_casesBearQ: Delete Test CasesA
DestructiveIdempotent
Inspect

Deletes specific BearQ test cases by ID. Deletion is reversible β€” tests are archived and can be restored. Works on any unprotected test; protected tests are rejected.

Toolset: Tasks

Parameters:

  • testCaseIds (array) required: IDs of BearQ test cases to delete.

ParametersJSON Schema
NameRequiredDescriptionDefault
testCaseIdsYesIDs of BearQ test cases to delete.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the operation as destructive and non-read-only, but the description adds meaningful nuance: deletion is reversible, test cases are archived and restorable, and protected tests are rejected. This goes well beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: first the action, then the reversibility nuance, then the constraint. Every sentence carries useful information with no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with rich annotations and full schema coverage, the description provides everything needed to invoke it correctly. No output schema exists, but return details are not essential for a delete operation with clearly stated behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters and fully describes testCaseIds. The description only paraphrases the schema by saying deletion is by ID, so it adds no meaningful new parameter-level meaning. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Deletes'), a specific resource ('BearQ test cases'), and a specific selection method ('by ID'). It also clarifies the behavior for protected versus unprotected tests, making the tool's purpose unambiguous and distinct from sibling run/expand tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly indicates that this tool works on unprotected tests and that protected tests are rejected, which gives an explicit when-to-use condition. It does not name an alternative tool, but none of the siblings appear to offer the same delete operation, so this is sufficient context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_expand_application_modelBearQ: Expand Application ModelC
Read-onlyIdempotent
Inspect

Explores the live application to discover or update its pages and elements in BearQ's application model. Optionally scope to a single functional area.

Toolset: Tasks

Parameters:

  • functionalArea (union): Functional area to scope the exploration to, by ID or name. Omit to explore the entire application.

ParametersJSON Schema
NameRequiredDescriptionDefault
functionalAreaNoFunctional area to scope the exploration to, by ID or name. Omit to explore the entire application.

TDQS

C2.7/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description says the tool will 'discover or update' pages and elements, implying a state change, while annotations declare readOnlyHint=true. This is a direct contradiction unless 'update' refers only to an internal cache, which the description does not clarify.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core purpose, followed by a concise parameter note. The 'Toolset: Tasks' line adds marginal context but does not make the description bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description does not mention return values, whether the tool starts an asynchronous task, or how to monitor progressβ€”especially relevant given the 'Toolset: Tasks' hint and sibling task-status tools. The read/update ambiguity also leaves an important side-effect question unresolved.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%; the schema already documents that functionalArea is an optional ID or name and that omitting it explores the entire application. The description restates this without adding meaningful detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies a specific action and resource: exploring the live application to discover or update pages and elements in BearQ's application model. This is clearly distinct from the test-execution and test-management siblings, though it does not explicitly contrast with any sibling tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives such as bearq_run_test_cases or bearq_get_task. The only usage hint is parameter-scoping ('Omit to explore the entire application'), which does not help an agent choose this tool over others.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_get_taskBearQ: Get TaskA
Read-onlyIdempotent
Inspect

Retrieves a task's current state, metadata, and activity log. Returns immediately with whatever's available β€” does not block on the task completing.

Toolset: Tasks

Parameters:

  • taskId (number) required: BearQ task ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskIdYesBearQ task ID.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior. The description adds valuable behavioral context by disclosing that the call returns immediately and does not block on task completion, which is important for agents deciding between this and a waiting/blocking tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is concise, front-loaded, and contains only useful information. The separate 'Parameters' section adds minor redundancy since the schema already documents taskId, but the overall structure is clean and quick to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, read-only, single-parameter tool, the description is sufficiently complete: it states what is returned, highlights non-blocking behavior, and the schema covers the parameter. No output schema exists, but the description gives a reasonable high-level view of the response contents.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter taskId is fully described in the schema. The description repeats this information without adding new meaning, so it stays at the baseline 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieves') with a clear resource: a task's current state, metadata, and activity log. It also differentiates itself from sibling tools by explicitly stating it returns immediately and does not block on completion, distinguishing it from bearq_wait_for_task and bearq_get_task_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use this tool: when you want the currently available snapshot of a task without waiting. However, it does not explicitly name alternatives or state when not to use them, so the guidance is clear but not fully explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_get_task_statusBearQ: Get Task StatusA
Read-onlyIdempotent
Inspect

Retrieves the status of a task (running / complete / error / cancelled). Cheaper than fetching full task details.

Toolset: Tasks

Parameters:

  • taskId (number) required: BearQ task ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskIdYesBearQ task ID.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior. The description adds value by disclosing the set of possible statuses (running/complete/error/cancelled) and signaling a lightweight response relative to full task details. There is no contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core message is two front-loaded sentences with statuses and the cost trade-off first. Some space is spent restating the parameter in a manual Parameters section that duplicates the schema, so it is not maximally lean.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only lookup, the annotations carry the safety profile and the description supplies the meaningful output domain (statuses) and a cost rationale. An explicit note on the exact return shape or a pointer to bearq_get_task when full details are needed would make it more complete, but nothing essential for a basic call is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema already covers taskId with description and constraints, and schema_description_coverage is 100%. The description only repeats the parameter name and 'BearQ task ID', adding no new semantics or format guidance, which fits the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ('Retrieves the status of a task') and enumerates the possible status values. The phrase 'Cheaper than fetching full task details' distinguishes it from the full-task retrieval sibling without relying on the title alone. This clearly separates it from run/stop/wait sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The cost note ('Cheaper than fetching full task details') tells an agent to prefer this tool when only status is needed. It does not explicitly name the alternative tool (e.g., bearq_get_task) or spell out when not to use it, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_list_environmentsBearQ: List EnvironmentsA
Read-onlyIdempotent
Inspect

Lists the environments configured in the workspace. Use this to discover valid environment names to pass to the test-running tools, and to identify the workspace default.

Toolset: Environments

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description only needs to add non-obvious context. It does so by explaining the output's intended role: supplying valid environment names to test-running tools and revealing the workspace default. There is no contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two front-loaded sentences deliver the core message with no filler. The optional 'Toolset' and 'Parameters' sections are compact and don't repeat schema details excessively. Every sentence in the description earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only, idempotent, parameterless listing tool, this description is complete. It states what is listed, how the information should be used, and that the workspace default is included. No output schema exists, but the description's promise of 'valid environment names' and 'workspace default' gives the agent enough to call and interpret the result correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the input schema is an empty object with 100% coverage, so there is no parameter meaning for the description to add. The description explicitly states 'Parameters: None,' which is accurate and harmless. The baseline of 4 for parameterless tools applies here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb-resource pair: 'Lists the environments configured in the workspace.' This clearly identifies what the tool does and distinguishes it from the many BearQ test-running and task-management siblings. The added detail about discovering environment names and the workspace default reinforces its unique role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent when to use this tool: 'Use this to discover valid environment names to pass to the test-running tools, and to identify the workspace default.' This provides clear context and a concrete purpose. It does not list alternatives or state when not to use it, but for a unique zero-parameter listing tool, the guidance is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_run_regression_testsBearQ: Run Regression TestsA
Read-onlyIdempotent
Inspect

Runs the full BearQ regression suite β€” every regression-ready test case in the workspace. Use for CI/CD or pre-release smoke.

Toolset: Tasks

Parameters:

  • environment (string): Target environment name to run tests against. Omit to use the workspace default.

ParametersJSON Schema
NameRequiredDescriptionDefault
environmentNoTarget environment name to run tests against. Omit to use the workspace default.

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the scope of 'every regression-ready test case' and the CI/CD context, but it does not disclose async behavior, how results are returned, or whether a task object is createdβ€”information an agent may need for long-running suites.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded in one clear sentence, followed by a concise usage hint. The 'Parameters' section duplicates the schema and could be trimmed, but overall the description is compact and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does not explain what the tool returns, whether the run is synchronous or asynchronous, or how to retrieve test results. Sibling tools like bearq_get_task_status and bearq_wait_for_task imply a task-based flow, but the description leaves this important behavioral detail implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single optional environment parameter. The description repeats the same parameter text verbatim without adding syntax, defaults, or examples beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description uses a specific verb and resource: 'Runs the full BearQ regression suite β€” every regression-ready test case in the workspace.' This clearly distinguishes it from sibling tools like bearq_run_test_cases and bearq_run_tests_in_functional_areas by emphasizing the full suite scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States clear usage context: 'Use for CI/CD or pre-release smoke.' This tells the agent when to invoke the tool, though it does not explicitly mention alternatives or when not to use it, such as pointing to bearq_run_test_cases for targeted runs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_run_test_casesBearQ: Run Test CasesA
Read-onlyIdempotent
Inspect

Runs specific BearQ regression test cases by ID. Targets only regression-ready cases β€” drafts will be rejected.

Toolset: Tasks

Parameters:

  • testCaseIds (array) required: IDs of BearQ regression test cases to run.

  • environment (string): Target environment name to run tests against. Omit to use the workspace default.

ParametersJSON Schema
NameRequiredDescriptionDefault
environmentNoTarget environment name to run tests against. Omit to use the workspace default.
testCaseIdsYesIDs of BearQ regression test cases to run.

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal read-only, idempotent, non-destructive behavior, lowering the burden on the description. It adds the draft-rejection constraint and workspace-default environment context, but it doesn't disclose whether execution is asynchronous, whether results are returned immediately, or whether re-running the same IDs has side effects. 'Runs' sits somewhat uneasily with readOnlyHint but is not a direct contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main action is front-loaded and the important draft-rejection behavior is prominent. The explicit Parameters block repeats schema descriptions, creating minor redundancy, but the overall length remains compact and scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 2-parameter tool with strong annotations and fully documented parameters, an agent has enough information to select and invoke it correctly. It could be more complete by noting how results or task status are retrieved and how it compares with nearby bearq_run_* siblings, but these are not blocking for correct use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the Description's Parameters block largely duplicates the schema text verbatim ('IDs of BearQ regression test cases to run', 'Omit to use the workspace default'). The description adds no meaningful semantic value beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ('Runs specific BearQ regression test cases') with the resource (test cases) and a clear selection mechanism (by ID). The qualifiers 'specific... by ID' and 'regression-ready cases β€” drafts will be rejected' distinguish it from sibling run tools like bearq_run_regression_tests and bearq_run_tests_in_functional_areas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context: use for specific BearQ regression test cases by ID, and do not pass drafts since they will be rejected. It stops short of a 5 because it does not explicitly name sibling alternatives or state when those should be used instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_run_tests_in_functional_areasBearQ: Run Tests in Functional AreasB
Read-onlyIdempotent
Inspect

Runs every regression test case tagged with one or more functional areas. Functional areas can be given by ID or name.

Toolset: Tasks

Parameters:

  • functionalAreas (array) required: Functional areas to target, by ID or name.

  • environment (string): Target environment name to run tests against. Omit to use the workspace default.

ParametersJSON Schema
NameRequiredDescriptionDefault
environmentNoTarget environment name to run tests against. Omit to use the workspace default.
functionalAreasYesFunctional areas to target, by ID or name.

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only/idempotent/destructive safety, so the description's burden is mainly behavior beyond that. It does not mention that this appears to create or run a task, return a task identifier, or require polling with sibling task tools. 'Runs every regression test case' communicates the core action but adds no execution-lifecycle context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is efficient and front-loaded. However, the parameter list duplicates the schema descriptions and the 'Toolset: Tasks' line contributes little functional value, so not every line earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no return-value description, and no mention of the task lifecycle despite sibling get_task_status/wait_for_task/stop_task tools. An agent can invoke it with functionalAreas, but cannot know what to do with the result or how to monitor completion. With no output schema, the description should bridge that gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%: both functionalAreas and environment already have descriptions in the schema. The description repeats them verbatim, adding no new meaning beyond saying that functional areas may be IDs or names, which the schema's anyOf already implies. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific action and scope: it 'Runs every regression test case tagged with one or more functional areas.' This clearly identifies the resource and distinguishes it from broader runners like bearq_run_regression_tests, though it does not explicitly name siblings or say what it is not.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied by scope: choose this tool when regression tests need to be filtered by functional areas. There is no explicit when-to-use/when-not-to-use guidance or comparison with bearq_run_regression_tests and bearq_run_test_cases, leaving some routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_stop_taskBearQ: Stop TaskC
Read-onlyIdempotent
Inspect

Cancels a running task.

Toolset: Tasks

Parameters:

  • taskId (number) required: BearQ task ID to cancel.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskIdYesBearQ task ID to cancel.

TDQS

C2.9/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description says 'Cancels a running task,' which is a state-changing operation, while annotations declare readOnlyHint=true and destructiveHint=false. This directly contradicts the tool's actual semantics and gives an agent conflicting safety signals.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is short and front-loaded with the core action. The 'Toolset' and 'Parameters' sections are mildly redundant with metadata already present in the schema, but they do not significantly bloat the definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool without an output schema, complete coverage should be easy, but the description omits side effects, reversibility, and behavior on already-finished tasks. More importantly, the readOnlyHint contradiction makes the overall context unreliable for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema already documents taskId with type, bounds, required flag, and the same description as the tool description's parameter line. The description adds no meaning beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Cancels') and a clear object ('a running task'), making the operation unmistakable. It also differentiates this tool from siblings like bearq_get_task, bearq_get_task_status, and bearq_wait_for_task, which are read/wait operations, and bearq_run_* tools, which start tasks.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for running tasks, but it gives no explicit guidance on when to use it versus alternatives, whether completed tasks can be cancelled, or whether a status check should precede cancellation. No exclusions or preconditions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bearq_wait_for_taskBearQ: Wait For TaskA
Read-onlyIdempotent
Inspect

Blocks until a BearQ task reaches a terminal state (completed / failed / cancelled) or the stream times out, then returns the full ordered sequence of SSE events from the public API (metadata, activityLogEntries, and a terminal done or timeout event) verbatim. Blocks for the lifetime of the task β€” for a quick check use bearq_get_task_status instead.

Toolset: Tasks

Parameters:

  • taskId (number) required: BearQ task ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskIdYesBearQ task ID.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/idempotent annotations, the description reveals the blocking nature, the terminal-state conditions, the timeout possibility, and the verbatim SSE event sequence returned. This is meaningful behavioral context an agent needs to decide if calling this tool is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly packed sentences deliver the core behavior, the trigger conditions, the return shape, and the sibling alternative. The parameter listing is minimal and redundant but harmless. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a single parameter, clear blocking semantics, stated return content, and a sibling pointer, the description covers everything an agent needs to decide whether to invoke it and what to expect in response. The lack of an output schema is mitigated by the explicit SSE event description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the description repeats the schema's parameter text ('BearQ task ID') without adding new meaning. No additional format, constraints, or usage nuance is provided beyond what the JSON schema already documents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Blocks') and resource ('BearQ task'), plus the exact return payload ('full ordered sequence of SSE events... verbatim'). It clearly distinguishes this from sibling bearq_get_task_status by framing it as the blocking/long-polling counterpart.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly explains when to use the tool ('Blocks for the lifetime of the task') and when not to, recommending 'bearq_get_task_status' for a quick check. This gives an agent actionable routing guidance without ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_buildBugSnag: Get BuildA
Read-onlyIdempotent
Inspect

Get more details for a specific build by its ID

Toolset: Releases

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • buildId (string) required: Unique identifier of the app build

Output Description: JSON object containing build details along with stability metrics such as user and session stability, and whether it meets project targets

Use Cases: 1. View build metadata such as version, source control info, and error counts 2. Analyze a specific build to correlate with error spikes or deployments 3. See the stability targets for a project and if the build meets them

Examples:

  1. Get details for a specific build

{
  "buildId": "5f8d0d55c9e77c0017a1b2c3"
}

Expected Output: JSON object with build details including version, source control info, error counts and stability data.

Hints: 1. Build IDs can be found using the List builds tool

ParametersJSON Schema
NameRequiredDescriptionDefault
buildIdYesUnique identifier of the app build
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), and the description adds useful output expectations: a JSON object with build details, stability metrics, and whether project targets are met. This goes beyond the schema and enriches the behavioral picture without contradicting the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections for Parameters, Output Description, Use Cases, Examples, and Hints. While it is somewhat verbose and duplicates schema parameter text, every section earns its place with actionable guidance (e.g., examples, expected output, hints). It is not as terse as ideal but remains effective.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only get-by-ID tool, the definition covers the required and optional params, provides output shape, and offers concrete use cases plus a hint on locating build IDs. It lacks error-handling details, but given the annotations, simple interface, and 100% schema coverage, it is sufficiently complete for an agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters have thorough descriptions in the input schema (100% schema description coverage). The tool description repeats these parameter descriptions but does not add new semantic details such as formats, allowed values, or default behaviors, so it stays at the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Get more details for a specific build by its ID' – a clear verb+object statement that names the resource (build) and the lookup mechanism (by ID). It distinguishes itself from sibling tools like bugsnag_get_release by focusing on builds, and the title reinforces the purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases section gives explicit scenarios (view build metadata, analyze error spikes, check stability targets) and the hint references the 'List builds' tool as a prerequisite for obtaining build IDs. However, it does not explicitly state when not to use this tool or compare it to alternatives like get_release, so it stops short of a perfect score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_current_projectBugSnag: Get Current ProjectA
Read-onlyIdempotent
Inspect

Retrieve the 'current' project on which tools should operate by default. This allows BugSnag tools to be called with no projectId parameter.

Toolset: Projects

Parameters: None

Use Cases: 1. Understand if a current project has been set

Hints: 1. If a project is returned, it can be assumed that the user expects interactions with BugSnag tools to refer to this project 2. If this tool returns no current project then other BugSnag tools will require an explicit project ID parameter 3. Call the List Projects tool to see all projects that the user has access to. Get the project ID from this list either by asking the user for the project name or slug 4. You might find a BugSnag API key in the user's code where they configure the BugSnag SDK that can be matched to a project 'apiKey' field from the project list

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool read-only and idempotent. The description adds valuable context beyond those annotations: the tool allows other BugSnag tools to be called without projectId, and the distinction between returning a project vs. returning nothing changes how subsequent tools must be invoked.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and then organized into Toolset, Parameters, Use Cases, and Hints. It is longer than strictly necessary, but each section earns its place by helping an agent decide when and how to use the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless tool with comprehensive hover annotations and no output schema, the description fully covers what the agent needs: what the tool does, what return scenarios mean, and what alternative to call if no current project is set. Nothing important is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With zero parameters and 100% schema description coverage, there is nothing for the description to add about parameter meaning. The description confirms 'Parameters: None' and focuses on behavioral context, which is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Retrieve the current project on which tools should operate by default.' It clearly distinguishes this from the sibling bugsnag_list_projects by focusing on the 'current' default project rather than all accessible projects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases and Hints sections give explicit guidance: call this to understand whether a current project has been set, expect other BugSnag tools to use that project if one is returned, and fall back to listing projects if none is returned. It also explains when an explicit project ID will be required instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_errorBugSnag: Get ErrorA
Read-onlyIdempotent
Inspect

Get full details on an error, including aggregated and summarized data across all events (occurrences) and details of the latest event (occurrence), such as breadcrumbs, metadata and the stacktrace. Use the filters parameter to narrow down the summaries further.

Toolset: Errors

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • errorId (string) required: Unique identifier of the error to retrieve

  • filters (record<string, array>): Apply filters to narrow down the error list. Use the List Project Event Filters tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h). (default: {"event.since":[{"type":"eq","value":"30d"}],"error.status":[{"type":"eq","value":"open"}]})

Output Description: JSON object containing: - error_details: Aggregated data about the error, including first and last seen occurrence - latest_event: Detailed information about the most recent occurrence of the error, including stacktrace, breadcrumbs, user and context - pivots: List of pivots (summaries) for the error, which can be used to analyze patterns in occurrences - url: A link to the error in the dashboard - this should be shown to the user for them to perform further analysis

Use Cases: 1. Investigate a specific error found through the List Project Errors tool 2. Understand which types of user are affected by the error using summarized event data 3. Get error details for debugging and root cause analysis 4. Retrieve error metadata for incident reports and documentation

Examples:

  1. Get details for a specific error

{
  "errorId": "6863e2af8c857c0a5023b411"
}

Expected Output: JSON object with error details including message, stack trace, occurrence count, and metadata

Hints: 1. Error IDs can be found using the List Project Errors tool 2. Use this after filtering errors to get detailed information about specific errors 3. Use Get Event Details tool if you need detailed information about a specific event (occurrence) rather than the aggregated error 4. If you used a filter to get this error, you can pass the same filters here to restrict the results or apply further filters 5. The URL provided in the response points should be shown to the user in all cases as it allows them to view the error in the dashboard and perform further analysis

ParametersJSON Schema
NameRequiredDescriptionDefault
errorIdYesUnique identifier of the error to retrieve
filtersNoApply filters to narrow down the error list. Use the List Project Event Filters tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h).
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context: it specifies the returned fields (error_details, latest_event, pivots, url), notes that filters narrow summaries further, and instructs the agent to show the dashboard URL to the user. No annotation contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a clear purpose sentence and uses well-labeled sections for parameters, output, use cases, examples, and hints. It is longer than strictly necessary because the parameter section largely duplicates the input schema and the example is generic, but the extra sections provide actionable agent guidance overall.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is complete for a read-only retrieval tool: it enumerates output fields despite no output schema, gives concrete workflows for error investigation, explains how to get error IDs, names the event-detail alternative, and tells the agent to surface the dashboard URL. Nothing critical for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema coverage is 100%, and the description largely restates schema text, which establishes the baseline of 3. It goes slightly further by clarifying that filters narrow 'summaries further', explaining when to reuse filters, directing users to the List Project Event Filters tool for available filter fields, and documenting time format options.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource β€” 'Get full details on an error' β€” and clarifies the scope: aggregated data across all events plus details of the latest event. This clearly distinguishes it from event-level or error-list siblings like bugsnag_get_event and bugsnag_list_project_errors.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use guidance: investigate errors found via List Project Errors, use after filtering, and pass the same filters to restrict results. It also names the alternative β€” 'Use Get Event Details tool if you need detailed information about a specific event rather than the aggregated error' β€” making the choice unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_eventBugSnag: Get EventA
Read-onlyIdempotent
Inspect

Get detailed information about a specific event

Toolset: Events

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • eventId (string) required: Unique identifier of the event

Use Cases: 1. Get the full details of an event, including any thread stack traces

Examples:

  1. Get event details of an event

{
  "eventId": "6863e2af012caf1d5c320000"
}

Expected Output: JSON object with complete event details including stack trace (error trace and other threads, if present), metadata, and context

ParametersJSON Schema
NameRequiredDescriptionDefault
eventIdYesUnique identifier of the event
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safe read-only, idempotent, non-destructive profile, and the description is consistent with that profile. It adds some return-content context (stack traces, metadata, context) but does not disclose anything else such as authentication requirements, rate limits, or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the purpose and uses concise sections and a JSON example. The explicit parameter list is redundant with the schema, but the overall size is acceptable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter read with no output schema, the description supplies the essential details: required eventId, optional projectId behavior, an example, and expected response contents. It is nearly complete, only lacking guidance about how to discover eventId or how this tool differs from similar event/error tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the parameter section mostly duplicates the schema. The example adds a concrete eventId format ('6863e2af012caf1d5c320000') and the expected-output description clarifies what the response contains, which goes beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific verb and resource ('Get detailed information about a specific event'), and the Use Cases section clarifies that this returns full event details including stack traces. It is clear, but it does not explicitly differentiate itself from sibling tools like bugsnag_get_event_details_from_dashboard_url or bugsnag_get_events_on_an_error.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The use case implies the tool is for retrieving a single event's full details, and the example encodes the expected input. However, there is no guidance on when to prefer this over nearby siblings, no exclusions, and no relationship to list-events or error-detail tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_event_details_from_dashboard_urlBugSnag: Get Event Details From Dashboard URLA
Read-onlyIdempotent
Inspect

Get detailed information about a specific event using its dashboard URL

Toolset: Events

Parameters:

  • link (string) required: Full URL to the event details page in the BugSnag dashboard (web interface), containing project slug and event_id parameter.

Use Cases: 1. Get event details when given a dashboard URL from a user or notification 2. Extract event information from shared links or browser URLs 3. Quick lookup of event details without needing separate project and event IDs

Examples:

  1. Get event details from a dashboard URL

{
  "link": "https://app.bugsnag.com/my-org/my-project/errors/6863e2af8c857c0a5023b411?event_id=6863e2af012caf1d5c320000"
}

Expected Output: JSON object with complete event details including stack trace, metadata, and context

Hints: 1. The URL must contain both project slug in the path and event_id in query parameters 2. This is useful when users share BugSnag dashboard URLs and you need to extract the event data

ParametersJSON Schema
NameRequiredDescriptionDefault
linkYesFull URL to the event details page in the BugSnag dashboard (web interface), containing project slug and event_id parameter.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds value beyond annotations by specifying the required URL structure (project slug in path, event_id in query) and what the expected output contains (stack trace, metadata, context). No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections (Toolset, Parameters, Use Cases, Examples, Hints) and the core purpose is front-loaded. Some redundancy exists between Use Cases and Hints, and the parameter section repeats the schema verbatim, but overall it remains focused and scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity tool with one parameter and strong annotations, the description is sufficiently complete. It explains expected output, input requirements, and common use cases. It could be more thorough about error behavior or exact parsing rules, but nothing critical is missing for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single 'link' parameter, so baseline is 3. The description adds meaning beyond the schema with a concrete example URL and hints about the necessary components, which helps an agent construct a valid input. This extra guidance justifies a score above baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get detailed information') and a precise resource ('specific event using its dashboard URL'). It clearly differentiates from sibling tools like bugsnag_get_event by emphasizing the URL-based access path and explicitly noting this avoids needing separate project/event IDs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context through Use Cases and Hints: it is intended for dashboard URLs from users/notifications, shared links, or quick lookups. It implies alternatives (separate project/event ID tools) by saying this works 'without needing separate project and event IDs', but does not explicitly name when-not-to-use alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_events_on_an_errorBugSnag: Get Events on an ErrorA
Read-onlyIdempotent
Inspect

Gets a list of events that have grouped into the specified error

Toolset: Events

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • errorId (string) required: Unique identifier of the error

  • filters (record<string, array>): Apply filters to narrow down the event list. Use the List Project Event Filters tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h). (default: {"event.since":[{"type":"eq","value":"30d"}],"error.status":[{"type":"eq","value":"open"}]})

  • direction (enum): Sort direction for ordering results (default: "desc")

  • perPage (number): How many results to return per page. (default: 30)

  • nextUrl (string): URL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.

Use Cases: 1. Retrieving all the events for comparison to find commonalities or differences in stack traces, breadcrumbs and metadata

Examples:

  1. Get events of an error

{
  "projectId": "1234567890abcdef12345678",
  "errorId": "6863e2af012caf1d5c320000"
}

Expected Output: A list of events, ordered by timestamp, with complete details including stack trace, breadcrumbs, metadata, and context

ParametersJSON Schema
NameRequiredDescriptionDefault
errorIdYesUnique identifier of the error
filtersNoApply filters to narrow down the event list. Use the List Project Event Filters tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h).
nextUrlNoURL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.
perPageNoHow many results to return per page.
directionNoSort direction for ordering resultsdesc
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral details beyond the annotations: default filters (event.since=30d, error.status=open), pagination via nextUrl with a warning not to construct it manually, and the expected output (ordered by timestamp with stack traces, breadcrumbs, metadata, and context).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is direct and useful, and the Use Cases/Expected Output sections add value. However, the Parameters section duplicates the schema descriptions almost verbatim, making the definition longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite lacking an output schema, the description provides expected output format, default filters, pagination guidance, filter discovery advice, and a realistic example. It is largely complete for correct invocation, though it could better differentiate this list endpoint from single-event retrieval tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The Parameters section largely repeats the schema's descriptions verbatim and does not add substantial new meaning beyond the example call and expected output.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Gets a list of events that have grouped into the specified error.' This clearly distinguishes it from tools like bugsnag_get_error or bugsnag_get_event, though it does not explicitly name those siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases section provides a concrete scenario: retrieving all events for comparison of stack traces, breadcrumbs, and metadata. This gives clear context for when the tool is appropriate, though it does not explicitly describe when-not-to-use it or name alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_network_endpoint_groupingsBugSnag: Get Network Endpoint GroupingsA
Read-onlyIdempotent
Inspect

Get the network endpoint grouping rules for a project

Toolset: Performance

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

Use Cases: 1. View current network endpoint grouping configuration 2. Understand how network requests are being grouped in performance monitoring 3. Check grouping patterns before making updates

Examples:

  1. Get network grouping rules for a project

{}

Expected Output: Array of endpoint URL patterns

Hints: 1. Network grouping patterns help consolidate similar requests into single span groups 2. Patterns use OpenAPI path templating syntax with curly braces for path parameters (e.g., /users/{userId}) 3. Wildcards () can be used in domains to match multiple subdomains (e.g., https://.example.com)

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is known. The description adds useful behavioral context: projectId is used to set the current project for BugSnag tools, the expected output is an array of endpoint URL patterns, and hints explain OpenAPI templating and wildcard syntax. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized into sections with the core purpose front-loaded. The parameter section duplicates the schema but the use cases, example, and hints all add value. Slightly verbose, but every section serves a clear function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-optional-parameter read tool with no output schema, the description covers the essentials: an example call, expected output shape, and interpretation hints. The only notable gap is what happens if no current project is set and no projectId is supplied, but the optionality note partially addresses this.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the parameter description in the tool mirrors the schema description exactly. No additional parameter-level semantics are provided, so the baseline of 3 applies. The grouping-pattern hints relate to the returned data, not to the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Get') and resource ('network endpoint grouping rules'), scoped to 'a project'. It clearly differentiates from the sibling bugsnag_set_network_endpoint_groupings by verb and from other BugSnag getters by resource, so an agent can tell what it does without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases section gives three explicit scenarios: viewing current configuration, understanding grouping, and checking patterns before making updates. This clearly implies when to use the tool, e.g. before calling the set tool, but it does not explicitly name alternatives or state when not to use it. That prevents a top score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_releaseBugSnag: Get ReleaseA
Read-onlyIdempotent
Inspect

Get more details for a specific release by its ID, including source control information and associated builds

Toolset: Releases

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • releaseId (string) required: Unique identifier of the app release

Output Description: JSON object containing release details along with stability metrics such as user and session stability, and whether it meets project targets

Use Cases: 1. View release metadata such as version, source control info, and error counts 2. Analyze the stability data and targets for a release 3. See the builds that make up the release

Examples:

  1. Get details for a specific release

{
  "releaseId": "5f8d0d55c9e77c0017a1b2c3"
}

Expected Output: JSON object with release details including version, source control info, error counts and stability data.

Hints: 1. Release IDs can be found using the List releases tool

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.
releaseIdYesUnique identifier of the app release

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the description does not need to restate safety. It adds value by describing the output contents (stability metrics, project targets) and notes that projectId can set the current project for BugSnag tools, which is useful behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections and front-loads the core purpose. It is somewhat redundant, repeating the JSON output description in both the Output Description and Expected Output, but the overall organization helps an agent parse the information efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get-by-ID tool with one required parameter and no output schema, the description is complete: it explains what the output contains, gives concrete use cases, provides an example, and tells the agent where to find release IDs. Nothing essential for calling this tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter descriptions in the tool description are near-verbatim copies of the schema descriptions. The example and hint add usability value, but the description does not meaningfully expand on the schema's parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: "Get more details for a specific release by its ID," and names what is included (source control information, associated builds). This clearly distinguishes it from siblings like bugsnag_list_releases and bugsnag_get_build.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides use cases and explicitly hints that "Release IDs can be found using the List releases tool," which gives clear routing context. It does not explicitly state when not to use this tool versus siblings, but the ID-based scope and sibling naming make the intended usage clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_span_groupBugSnag: Get Span GroupA
Read-onlyIdempotent
Inspect

Get detailed performance metrics for a specific span group

Toolset: Performance

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • spanGroupId (string) required: ID of the span group

  • filters (record<string, array>): Apply filters to narrow down the span group list. Use the List Trace Fields tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h). (default: {"span.since":[{"type":"eq","value":"7d"}]})

Use Cases: 1. View detailed statistics (p50, p75, p90, p95, p99) for an operation 2. Check if performance targets are configured 3. Monitor span count to understand operation volume

Examples:

  1. Get details for an API endpoint span group

{
  "spanGroupId": "[HttpClient]GET-api.example.com"
}

Expected Output: Statistics, category, and performance target info

  1. Get span group details with device filtering

{
  "spanGroupId": "[HttpClient]GET-api.example.com",
  "filters": {
    "device.browser_name": [
      {
        "type": "eq",
        "value": "Chrome"
      }
    ]
  }
}

Expected Output: Statistics filtered for Chrome browser only

Hints: 1. Use List Span Groups first to discover available span group IDs 2. IDs are automatically URL-encoded - provide the raw ID 3. Statistics include p50, p75, p90, p95, p99 percentiles

ParametersJSON Schema
NameRequiredDescriptionDefault
filtersNoApply filters to narrow down the span group list. Use the List Trace Fields tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h).
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.
spanGroupIdYesID of the span group

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly and idempotent, and the description adds meaningful behavioral detail: it specifies that statistics include p50/p75/p90/p95/p99, that the response provides category and performance target info, and that span group IDs are automatically URL-encoded. This goes beyond the annotation safety profile without contradicting it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded: purpose first, then parameters, use cases, examples, and hints. It is somewhat long and repeats some schema parameter descriptions, but each section serves a practical purpose and the examples are directly actionable. It earns its length for a tool with no output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema, the description does an excellent job of explaining what the agent will receive: statistics, category, performance target info, and filtered results. It also covers the workflow (discover span groups first), filter discovery, parameter defaults, and URL-encoding behavior. Nothing critical is missing for successful invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds genuine value with the URL-encoding hint for spanGroupId, a concrete default filter value, an example of device.browser_name filtering, and a pointer to List Trace Fields for discovering filter fields. These details help agents use the parameters correctly beyond the schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource: 'Get detailed performance metrics for a specific span group.' The word 'specific' distinguishes it from bugsnag_list_span_groups, and the performance-metrics focus separates it from other get tools like bugsnag_get_trace or bugsnag_get_error. This is clear and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section clearly states when to use the tool (view percentiles, check performance targets, monitor span count). Hints provide routing guidance: use List Span Groups first to discover IDs and List Trace Fields to discover filters. It does not explicitly enumerate when-not-to-use alternatives, but the context is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_get_traceBugSnag: Get TraceA
Read-onlyIdempotent
Inspect

Get all spans within a specific trace

Toolset: Performance

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • traceId (string) required: Trace ID

  • from (string) required: Start time (ISO 8601 format)

  • to (string) required: End time (ISO 8601 format)

  • targetSpanId (string): Optional target span ID to focus on

  • perPage (number): How many results to return per page. (default: 30)

  • nextUrl (string): URL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.

Use Cases: 1. Debug slow requests by viewing all operations in the trace 2. Understand the flow of a request through the system 3. Identify bottlenecks in distributed systems

Examples:

  1. Get all spans for a trace

{
  "traceId": "abc123",
  "from": "2024-01-01T00:00:00Z",
  "to": "2024-01-01T23:59:59Z"
}

Expected Output: Array of all spans in the trace with timing and hierarchy

  1. Get spans for a trace with pagination and target span

{
  "traceId": "def456",
  "from": "2024-01-01T00:00:00Z",
  "to": "2024-01-01T23:59:59Z",
  "targetSpanId": "span-789",
  "perPage": 50
}

Expected Output: Array of up to 50 spans focused around the target span

Hints: 1. Traces show the complete execution path of a request 2. Use from/to parameters to narrow the time window 3. targetSpanId can be used to focus on a specific span in the trace

ParametersJSON Schema
NameRequiredDescriptionDefault
toYesEnd time (ISO 8601 format)
fromYesStart time (ISO 8601 format)
nextUrlNoURL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.
perPageNoHow many results to return per page.
traceIdYesTrace ID
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.
targetSpanIdNoOptional target span ID to focus on

TDQS

A4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal read-only, idempotent, and non-destructive behavior. The description adds meaningful behavioral detail beyond that: pagination via nextUrl with an explicit warning not to construct the URL manually, default page size, target-span focusing, and the return shape (array of spans with timing and hierarchy). This exceeds what the annotations alone provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with headings, use cases, examples, and hints, and the core action is front-loaded. However, it contains a full parameter list that duplicates the input schema and several hints that restate the examples, making it longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema, the description provides enough context: required parameters, optional pagination and target-span behavior, example invocations, and expected response shape. It could be more precise about the response structure, but an agent has the information needed to select and call the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter. The description repeats those definitions and adds illustrative examples, but it does not materially expand on the schema's parameter explanations.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line 'Get all spans within a specific trace' names a specific verb and resource, so it is immediately clear what the tool returns. It does not explicitly distinguish itself from sibling BugSnag span tools such as bugsnag_list_spans, though 'specific trace' narrows the scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear use cases (debugging slow requests, understanding request flow, identifying bottlenecks) and hints about narrowing the time window and focusing on a target span. It does not name sibling alternatives or state when not to use it, but the context is strong enough for an agent to decide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_list_project_errorsBugSnag: List Project ErrorsA
Read-onlyIdempotent
Inspect

List and search errors in a project using customizable filters and pagination

Toolset: Errors

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • filters (record<string, array>): Apply filters to narrow down the error list. Use the List Project Event Filters tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h). (default: {"event.since":[{"type":"eq","value":"30d"}],"error.status":[{"type":"eq","value":"open"}]})

  • sort (enum): Field to sort the errors by (default: "last_seen")

  • direction (enum): Sort direction for ordering results (default: "desc")

  • perPage (number): How many results to return per page. (default: 30)

  • nextUrl (string): URL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.

Use Cases: 1. Debug recent application errors by filtering for open errors in the last 7 days 2. Generate error reports for stakeholders by filtering specific error types or severity levels 3. Monitor error trends over time using date range filters 4. Find errors affecting specific users or environments using metadata filters

Examples:

  1. Find errors affecting a specific user in the last 24 hours

{
  "filters": {
    "user.email": [
      {
        "type": "eq",
        "value": "user@example.com"
      }
    ],
    "event.since": [
      {
        "type": "eq",
        "value": "24h"
      }
    ]
  }
}

Expected Output: JSON object with a list of errors in the 'data' field, a count of the current page of results in the 'count' field, and a total count of all results in the 'total' field

  1. Get the 10 open errors with the most users affected in the last 30 days

{
  "filters": {
    "event.since": [
      {
        "type": "eq",
        "value": "30d"
      }
    ],
    "error.status": [
      {
        "type": "eq",
        "value": "open"
      }
    ]
  },
  "sort": "users",
  "direction": "desc",
  "perPage": 10
}

Expected Output: JSON object with a list of errors in the 'data' field, a count of the current page of results in the 'count' field, and a total count of all results in the 'total' field

  1. Get the next 50 results

{
  "nextUrl": "https://api.bugsnag.com/projects/515fb9337c1074f6fd000003/errors?base=2025-08-29T13%3A11%3A37Z&direction=desc&filters%5Berror.status%5D%5B%5D%5Btype%5D=eq&filters%5Berror.status%5D%5B%5D%5Bvalue%5D=open&offset=10&per_page=10&sort=users",
  "perPage": 50
}

Expected Output: JSON object with a list of errors, with a URL to the next page if more results are available and a total count of all errors matched

Hints: 1. Use List Project Event Filters tool first to discover valid filter field names for your project 2. Combine multiple filters to narrow results - filters are applied with AND logic 3. For time filters: use relative format (7d, 24h) for recent periods or ISO 8601 UTC format (2018-05-20T00:00:00Z) for specific dates 4. Common time filters: event.since (from this time), event.before (until this time) 5. The 'event.since' filter and 'error.status' filters are always applied and if not specified are set to '30d' and 'open' respectively 6. There may not be any errors matching the filters - this is not a problem with the tool, in fact it might be a good thing that the user's application had no errors 7. This tool returns paged results. The 'page_error_count' field indicates the number of results returned in the current page, and the 'total_error_count' field indicates the total number of results across all pages. 8. If the output contains a 'next_url' value, there are more results available - call this tool again supplying the next URL as a parameter to retrieve the next page. 9. Do not modify the next URL as this can cause incorrect results. The only other parameter that can be used with 'next' is 'per_page' to control the page size.

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoField to sort the errors bylast_seen
filtersNoApply filters to narrow down the error list. Use the List Project Event Filters tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h).
nextUrlNoURL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.
perPageNoHow many results to return per page.
directionNoSort direction for ordering resultsdesc
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds substantial behavioral detail: default filters, pagination mechanics, nextUrl usage rules, the requirement not to modify the URL, the meaning of output fields, and the possibility of an empty result set. This goes well beyond the annotations and schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured into Parameters, Use Cases, Examples, and Hints. Each section earns its place: examples illustrate complex filter syntax, hints cover pagination pitfalls and empty results, and the structure is scannable. There is minor duplication of schema details, but it is purposeful reinforcement rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 6 parameters, nested filter objects, no output schema, and non-obvious pagination behavior, the description is remarkably complete. It covers all parameters, defaults, filter discovery, time format options, AND logic, output structure, next-page handling, and even the no-results case. An agent has everything needed to invoke and paginate correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description adds real semantic value to every parameter: projectId's optionality and current-project behavior, filter format and time syntax, sort/direction/perPage defaults, and the strict rules around nextUrl. The examples demonstrate exact filter JSON structures, making parameter usage far clearer than the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb-resource pair: 'List and search errors in a project using customizable filters and pagination.' It clearly identifies what the tool does, but it does not explicitly differentiate itself from sibling tools like bugsnag_get_error or bugsnag_list_events_on_an_error, so the distinction is implied rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides concrete use cases (debugging recent errors, generating reports, monitoring trends) and explicitly instructs to use the List Project Event Filters tool first to discover valid filters. It does not state when to prefer another error-related tool instead, so no explicit exclusions or sibling selection guidance is present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_list_project_event_filtersBugSnag: List Project Event FiltersA
Read-onlyIdempotent
Inspect

Get available event filter fields for a project

Toolset: Projects

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

Use Cases: 1. Discover what filter fields are available before searching for errors 2. Find the correct field names for filtering by user, environment, or custom metadata 3. Understand filter options and data types for building complex queries

Examples:

  1. Get all available filter fields

{}

Expected Output: JSON array of EventField objects containing display_id, custom flag, and filter/pivot options

Hints: 1. Use this tool before the List Errors or Get Error tools to understand available filters 2. Look for display_id field in the response - these are the field names to use in filters

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, and the description is consistent with them. It adds useful behavioral context beyond annotations by noting that an optional projectId is 'used to set the current project for BugSnag tools' and by describing the expected EventField response shape with display_id and filter/pivot options.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The one-sentence summary is front-loaded and the use cases, example, expected output, and hints are organized in scannable sections. The Parameters section duplicates the schema's projectId description, which adds some redundancy, but the overall structure remains efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description compensates by stating the expected response ('JSON array of EventField objects') and the key output fields. It also explains the optional parameter semantics, provides an empty-input example, and connects the result to the next step via the display_id hint, making the tool fully actionable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the description's Parameters section restates the same projectId explanation already present in the input schema without adding new meaning. With full schema coverage, the baseline of 3 applies because the description offers no additional parameter nuance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Get available event filter fields for a project,' a specific verb-object-resource statement that matches the tool's name and makes its scope obvious. It further distinguishes itself by framing the output as filter fields to be used before querying errors, which separates it from sibling error/event query tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance: 'Use this tool before the List Errors or Get Error tools to understand available filters' and lists discovery use cases. It does not state when not to use it or mention direct alternatives such as bugsnag_list_trace_fields, so it stops short of a full when/when-not matrix.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_list_projectsBugSnag: List ProjectsA
Read-onlyIdempotent
Inspect

List all projects in the organization that the current user has access to, or find a project matching an API key.

Toolset: Projects

Parameters:

  • apiKey (string): The API key of the BugSnag project, if known.

Use Cases: 1. Get an overview of all projects in the organization 2. Locate a project by its API key if known from the user's code

Hints: 1. Project IDs from this list can be used with other tools when no project API key is configured

ParametersJSON Schema
NameRequiredDescriptionDefault
apiKeyNoThe API key of the BugSnag project, if known.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds useful behavior beyond that: access is scoped to projects the current user can see, the apiKey acts as a lookup filter, and returned project IDs can be passed to other BugSnag tools. Pagination and response shape are not disclosed, but this is a minor gap for a simple read-only list.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main sentence is front-loaded and the rest is organized into short, scannable sections. The 'Parameters' bullet mostly duplicates the schema, and 'Toolset' adds little, but the overall length is reasonable and each remaining section contributes useful context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, read-only list tool with strong annotations, the description is largely complete: it states scope, the optional filter, use cases, and how results can be consumed by other tools. It does not describe pagination or the exact response fields, but the hint about project IDs covers the most important downstream need.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents the optional apiKey parameter. The description adds minor context by linking the key to 'the user's code,' but it repeats the schema text and does not introduce format, constraints, or examples.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'List all projects in the organization that the current user has access to.' It also clearly distinguishes the alternate API-key-based lookup behavior, so an agent understands both the default action and the optional filter.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section gives explicit scenarios: getting an overview of all projects and locating a project by API key from user code. The 'Hints' section adds downstream guidance that project IDs can be reused with other tools, but it does not explicitly contrast this tool with siblings like bugsnag_get_current_project.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_list_releasesBugSnag: List ReleasesA
Read-onlyIdempotent
Inspect

List releases for a project

Toolset: Releases

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • releaseStage (string): Filter releases by this stage (e.g. production, staging), defaults to 'production' (default: "production")

  • visibleOnly (boolean): Whether to only include releases that are marked as visible in the dashboard (default: false)

  • perPage (number): How many results to return per page. (default: 30)

  • nextUrl (string): URL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.

Output Description: JSON array of release summary objects with metadata, with a URL to the next page if more results are available

Use Cases: 1. View recent releases to correlate with error spikes 2. Filter releases by stage (e.g. production, staging) for targeted analysis

Examples:

  1. List production releases for a project

{}

Expected Output: JSON array of release objects in the production stage

  1. List staging releases for a project

{
  "releaseStage": "staging"
}

Expected Output: JSON array of release objects in the staging stage

  1. Get the next page of results

{
  "nextUrl": "/projects/515fb9337c1074f6fd000003/releases?offset=30&per_page=30"
}

Expected Output: JSON array of release objects with metadata from the next page

Hints: 1. Use the Get Release tool to get more details on a specific release, including the builds it contains 2. The release stage defaults to 'production' if not specified 3. Use visibleOnly to filter out releases that have been marked as hidden in the dashboard

ParametersJSON Schema
NameRequiredDescriptionDefault
nextUrlNoURL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.
perPageNoHow many results to return per page.
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.
visibleOnlyNoWhether to only include releases that are marked as visible in the dashboard
releaseStageNoFilter releases by this stage (e.g. production, staging), defaults to 'production'production

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds valuable behavioral context beyond these: pagination semantics for nextUrl, default values for releaseStage and visibleOnly, the projectId side effect of setting the current project, and the output shape with next-page URL. All behavior is disclosed consistently.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with clear sections (Toolset, Parameters, Output, Use Cases, Examples, Hints). It is longer than necessary, but each section earns its place and the key information is front-loaded in the first sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only listing tool with five all-optional parameters, no output schema, and rich annotations, the description is fully sufficient. It documents every parameter, provides three worked examples, describes the output format, and gives hints for follow-up toolsβ€”nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description goes beyond the schema by providing concrete examples (staging filter, nextUrl usage) and restating defaults and constraints in context, which helps an agent apply parameters correctly without opening the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List releases for a project') with a specific verb and resource. The 'Toolset: Releases' label and the hint pointing to the 'Get Release tool' for more details help distinguish this listing tool from its sibling bugsnag_get_release.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases ('View recent releases to correlate with error spikes', 'Filter releases by stage') and a hint to use the Get Release tool when more detail on a specific release is needed. This gives clear when-to-use guidance and names an alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_list_span_groupsBugSnag: List Span GroupsA
Read-onlyIdempotent
Inspect

List span groups (operations) tracked for performance monitoring

Toolset: Performance

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • sort (enum): Field to sort by

  • direction (enum): Sort direction for ordering results (default: "desc")

  • perPage (number): How many results to return per page. (default: 30)

  • starredOnly (boolean): Show only starred span groups

  • nextUrl (string): URL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.

  • filters (record<string, array>): Apply filters to narrow down the span group list. Use the List Trace Fields tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h). (default: {"span.since":[{"type":"eq","value":"7d"}]})

Use Cases: 1. View all operations being tracked for performance 2. Find slow operations by sorting by duration metrics 3. Filter to starred/important span groups

Examples:

  1. List slowest operations

{
  "sort": "duration_p95",
  "direction": "desc",
  "perPage": 10
}

Expected Output: Array of span groups sorted by 95th percentile duration

  1. List starred span groups with filtering

{
  "starredOnly": true,
  "filters": {
    "span_group.category": [
      {
        "type": "eq",
        "value": "full_page_load"
      }
    ]
  }
}

Expected Output: Array of starred span groups filtered by category

Hints: 1. Span groups represent different operation types (page loads, API calls, etc.) 2. Use sort by duration_p95 or duration_p99 to find the slowest operations 3. Star important span groups for quick access 4. Use nextUrl for pagination

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoField to sort by
filtersNoApply filters to narrow down the span group list. Use the List Trace Fields tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h).
nextUrlNoURL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.
perPageNoHow many results to return per page.
directionNoSort direction for ordering resultsdesc
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.
starredOnlyNoShow only starred span groups

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as read-only, idempotent, and non-destructive, so the description mainly adds value with behavioral caveats: the default span.since=7d filter, the nextUrl restriction that only prior output values are valid and must not be hand-constructed, and the projectId side effect of setting the current BugSnag project. These go beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well organized with headings, a front-loaded purpose, use cases, examples, and hints, making it easy to scan. However, the Parameters section duplicates the schema verbatim, adding length without new information; the same content could be tighter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with no output schema, the description covers invocation-relevant context well: defaults, pagination through nextUrl, filter format, example payloads, and expected array output. It does not enumerate response fields, but it provides enough for an agent to call the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description largely restates the schema's parameter descriptions for sort, direction, perPage, filters, and nextUrl rather than adding new semantic detail. It does add working examples and sorting hints, which are useful, but they are supplements rather than necessary compensation for missing schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'List span groups (operations) tracked for performance monitoring'. This clearly identifies the tool as a list operation over span groups and distinguishes it from sibling tools like bugsnag_get_span_group or bugsnag_list_spans.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases enumerate concrete scenarios: viewing all operations, finding slow operations via duration sorts, and filtering starred groups. Hints recommend duration_p95/duration_p99 for slowest operations, and the description points to List Trace Fields as the place to discover filter fields. It does not explicitly contrast with bugsnag_list_spans or bugsnag_get_span_group, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_list_spansBugSnag: List SpansA
Read-onlyIdempotent
Inspect

Get individual spans belonging to a span group

Toolset: Performance

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • spanGroupId (string) required: ID of the span group

  • sort (enum): Field to sort by

  • direction (enum): Sort direction for ordering results (default: "desc")

  • perPage (number): How many results to return per page. (default: 30)

  • nextUrl (string): URL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.

  • filters (record<string, array>): Apply filters to narrow down the span group list. Use the List Trace Fields tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h). (default: {"span.since":[{"type":"eq","value":"7d"}]})

Use Cases: 1. Analyze individual slow operations 2. Debug performance issues by examining specific traces 3. Find patterns in operation attributes

Examples:

  1. Get slowest spans for an operation

{
  "spanGroupId": "[HttpClient]GET-api.example.com",
  "sort": "duration",
  "direction": "desc",
  "perPage": 10
}

Expected Output: Array of the 10 slowest span instances

  1. Get spans filtered by OS with pagination

{
  "spanGroupId": "[HttpClient]GET-api.example.com",
  "sort": "timestamp",
  "filters": {
    "os.name": [
      {
        "type": "eq",
        "value": "iOS"
      }
    ]
  },
  "nextUrl": "/projects/123/spans?offset=30&per_page=30"
}

Expected Output: Array of spans from iOS devices with next page navigation

Hints: 1. Sort by duration descending to find the slowest instances 2. Each span includes trace ID for further investigation

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoField to sort by
filtersNoApply filters to narrow down the span group list. Use the List Trace Fields tool to discover available filter fields. Time filters support extended ISO 8601 format (e.g. 2018-05-20T00:00:00Z) or relative format (e.g. 7d, 24h).
nextUrlNoURL for retrieving the next page of results. Use the value in the previous response to get the next page when more results are available. Only values provided in the output from this tool can be used. Do not attempt to construct it manually.
perPageNoHow many results to return per page.
directionNoSort direction for ordering resultsdesc
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.
spanGroupIdYesID of the span group

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, and non-destructive behavior. The description adds useful behavioral context beyond the annotations, such as how nextUrl pagination works, the default span.since=7d filter, and expected output examples. There is no contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections: Toolset, Parameters, Use Cases, Examples, and Hints. It is somewhat longer than necessary because the Parameters section largely repeats the input schema, but each remaining section contributes practical value for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 7 parameters, nested filter objects, and no output schema, the description provides sufficient guidance for correct invocation: concrete examples, expected output descriptions, pagination behavior, and a pointer to List Trace Fields for filter discovery. Nothing that would block selection or calling is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds meaning through concrete example payloads, explains the pagination contract for nextUrl, and offers practical hints like sorting by duration descending to find the slowest instances. This goes beyond simply restating parameter names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence, 'Get individual spans belonging to a span group,' names a specific verb and resource, and the modifier 'individual' distinguishes this from sibling tools like bugsnag_list_span_groups and bugsnag_get_span_group. An agent can identify the target resource without needing to inspect other tool definitions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases section provides explicit scenarios for when this tool is appropriate: analyzing individual slow operations, debugging performance issues via specific traces, and finding patterns in operation attributes. It does not explicitly state exclusions or alternatives, so it falls just short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_list_trace_fieldsBugSnag: List Trace FieldsA
Read-onlyIdempotent
Inspect

Get available trace fields/attributes for filtering

Toolset: Performance

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

Use Cases: 1. Find available custom attributes for performance filtering 2. Understand what metadata is attached to traces 3. Build dynamic filters based on available fields

Examples:

  1. Get all trace fields

{}

Expected Output: Array of field names and types available for filtering

Hints: 1. Trace fields are custom attributes added to spans 2. Use these fields for filtering other performance queries

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds useful behavioral context by specifying the expected output as an array of field names and types and clarifying that trace fields are custom attributes added to spans. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with sections and a front-loaded summary, making it scannable. However, it contains redundancy: the parameter bullet duplicates the schema, and the use cases and hints overlap with the opening sentence. It could be tighter without losing value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple optional-parameter, read-only list tool with no output schema, the description covers the core purpose, usage scenarios, and expected output format. It is complete enough for an agent to select and call the tool correctly, though it does not discuss edge cases such as behavior when no project is set.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's Parameters section essentially restates the same projectId text already present in the schema. The description adds no new meaning about the parameter, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Get available trace fields/attributes for filtering', which names a specific verb and resource and states the purpose. It is distinguishable from sibling list tools because it targets trace fields specifically, though it does not explicitly contrast itself with alternatives like bugsnag_list_project_event_filters.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use Cases explicitly state when to use the tool: finding custom attributes for performance filtering, understanding trace metadata, and building dynamic filters. The Hints section reinforces that the returned fields should be reused in other performance queries, but it does not mention when not to use this tool or name an alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_set_network_endpoint_groupingsBugSnag: Set Network Endpoint GroupingsA
Idempotent
Inspect

Set the network endpoint grouping rules for a project

Toolset: Performance

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • endpoints (array) required: Array of URL patterns by which network spans are grouped. Endpoints follow OpenAPI path templating syntax (https://swagger.io/specification/#path-templating) where path parameters use curly braces (e.g., /users/{id}). If you encounter colon-prefixed parameters (e.g., :userId from Express/React Router), convert them to curly braces (e.g., {userId}). Wildcards () can be used in domains (e.g., https://.example.com) to match multiple subdomains.

Use Cases: 1. Consolidate similar API endpoints into single span groups 2. Group dynamic URLs using path parameters (e.g., /api/users/{userId} groups /api/users/123, /api/users/456) 3. Match multiple subdomains using wildcards (e.g., https://*.example.com groups api.example.com, cdn.example.com) 4. Simplify performance monitoring by reducing span group clutter

Examples:

  1. Group API endpoints with path parameters

{
  "endpoints": [
    "/api/users/{userId}",
    "/api/products/{productId}",
    "/api/orders/{orderId}/items/{itemId}"
  ]
}

Expected Output: Success response confirming the update

  1. Group endpoints with domain wildcards and path parameters

{
  "endpoints": [
    "https://*.example.com/api/v1/{resourceId}",
    "https://api.example.com/v2/users/{userId}",
    "/graphql"
  ]
}

Expected Output: Success response confirming the update

  1. Convert colon-prefixed parameters to curly braces (e.g., from Express/React Router)

{
  "endpoints": [
    "/{organizationSlug}/{projectSlug}/performance/view-load",
    "/api/{version}/items/{itemId}"
  ]
}

Expected Output: Success response confirming the update

Hints: 1. Use Get Network Grouping first to see current patterns 2. Use OpenAPI path templating with curly braces for path parameters: /users/{userId}, /orders/{orderId}/items/{itemId} 3. Convert colon-prefixed parameters to curly braces: :organizationSlug becomes {organizationSlug}, :projectSlug becomes {projectSlug} 4. Wildcards () can be used in domains to match subdomains: https://.example.com/api 5. This replaces all existing patterns - include all patterns you want to keep 6. Well-designed patterns reduce noise in performance monitoring

ParametersJSON Schema
NameRequiredDescriptionDefault
endpointsYesArray of URL patterns by which network spans are grouped. Endpoints follow OpenAPI path templating syntax (https://swagger.io/specification/#path-templating) where path parameters use curly braces (e.g., /users/{id}). If you encounter colon-prefixed parameters (e.g., :userId from Express/React Router), convert them to curly braces (e.g., {userId}). Wildcards (*) can be used in domains (e.g., https://*.example.com) to match multiple subdomains.
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description explicitly states a critical behavioral trait: 'This replaces all existing patterns - include all patterns you want to keep.' This warns the agent about the overwrite semantic, which is exactly the kind of behavioral context that matters at invocation time. The idempotentHint and destructiveHint annotations are not contradicted; the description strengthens the agent's understanding of side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured with clear sections: Toolset, Parameters, Use Cases, Examples, and Hints. The front-loaded summary sentence immediately conveys the core action. Some content is redundant with the schema and the repeated 'Expected Output' lines add bulk, but given the syntax complexity, the length is justified.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema, the description provides expected outputs in examples and covers the essential operational concern of pattern replacement. It also includes syntax rules, examples, and workflow hints. It stops short of specifying error cases or exact response structure, but the agent has enough context to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters fully. The description adds value by providing concrete examples, colon-to-brace conversion guidance, wildcard usage, and hints about project context. It does not introduce new parameter semantics beyond the schema, but the examples significantly improve practical understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear, specific operation: 'Set the network endpoint grouping rules for a project.' This immediately differentiates the tool from the sibling bugsnag_get_network_endpoint_groupings and clarifies the resource being acted upon. The title and Toolset label additionally anchor it in BugSnag performance monitoring.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases and hints that explain when to apply the tool, such as consolidating API endpoints and grouping dynamic URLs. It also advises using 'Get Network Grouping' first, which orients the agent toward the read-before-write workflow. It does not give formal when-not-to-use guidance, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bugsnag_update_errorBugSnag: Update ErrorAInspect

Update the status of an error

Toolset: Errors

Parameters:

  • projectId (string): Unique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.

  • errorId (string) required: Unique identifier of the error

  • operation (enum) required: The operation to apply to the error

  • issue_url (string): The URL of the issue to link to the error - required when operation is 'link_issue'

  • reopenRules (object): Reopen rules for snooze operation - required when operation is 'snooze'

Use Cases: 1. Mark an error as open, fixed or ignored 2. Discard or un-discard an error 3. Update the severity of an error 4. Snooze an error with defined conditions for when it should be reopened

Examples:

  1. Mark an error as fixed

{
  "errorId": "6863e2af8c857c0a5023b411",
  "operation": "fix"
}

Expected Output: Success response indicating the error was marked as fixed

  1. Snooze an error for 1 hour

{
  "errorId": "6863e2af8c857c0a5023b411",
  "operation": "snooze",
  "reopenRules": {
    "reopenIf": "occurs_after",
    "seconds": 3600
  }
}

Expected Output: Success response indicating the error was snoozed for 1 hour

  1. Snooze an error until 5 additional users are affected

{
  "errorId": "6863e2af8c857c0a5023b411",
  "operation": "snooze",
  "reopenRules": {
    "reopenIf": "n_additional_users",
    "additionalUsers": 5
  }
}

Expected Output: Success response indicating the error was snoozed until 5 additional users are affected

  1. Snooze an error until 10 occurrences in 24 hours

{
  "errorId": "6863e2af8c857c0a5023b411",
  "operation": "snooze",
  "reopenRules": {
    "reopenIf": "n_occurrences_in_m_hours",
    "occurrences": 10,
    "hours": 24
  }
}

Expected Output: Success response indicating the error was snoozed until 10 occurrences in 24 hours

  1. Link a Jira issue to an error

{
  "errorId": "6863e2af8c857c0a5023b411",
  "operation": "link_issue",
  "issue_url": "https://smartbear.atlassian.net/browse/PIPE-9547"
}

Expected Output: Success response indicating the Jira issue was linked to the error

  1. Unlink a Jira issue from an error

{
  "errorId": "6863e2af8c857c0a5023b411",
  "operation": "unlink_issue"
}

Expected Output: Success response indicating the Jira issue was unlinked from the error

Hints: 1. Only use valid operations - BugSnag may reject invalid values 2. When using 'snooze' operation, reopenRules parameter is required 3. When using 'link_issue' operation, issue_url parameter is required 4. Use 'unlink_issue' to remove the link between an error and its issue 5. For 'occurs_after' reopen rules, specify 'seconds' parameter 6. For 'n_additional_users' reopen rules, specify 'additionalUsers' parameter (max 100,000) 7. For 'n_occurrences_in_m_hours' reopen rules, specify both 'occurrences' and 'hours' parameters 8. For 'n_additional_occurrences' reopen rules, specify 'additionalOccurrences' parameter 9. Snoozing temporarily silences an error until the specified reopen condition is met

ParametersJSON Schema
NameRequiredDescriptionDefault
errorIdYesUnique identifier of the error
issue_urlNoThe URL of the issue to link to the error - required when operation is 'link_issue'
operationYesThe operation to apply to the error
projectIdNoUnique identifier of the project. This is optional if a current project is set and is used to set the current project for BugSnag tools.
reopenRulesNoReopen rules for snooze operation - required when operation is 'snooze'

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal readOnlyHint=false; the description adds meaningful behavior such as 'Snoozing temporarily silences an error until the specified reopen condition is met' and warns that invalid operations may be rejected. It doesn't cover auth/rate limits, but the mutation semantics are disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured with a one-line summary, Use Cases, Examples, and Hints, and front-loaded purpose. The Parameters section largely duplicates the schema, adding some redundancy, but the examples and hints justify the length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a nested-parameter, nine-operation mutation tool with no output schema, the description is thorough: every operation is covered by an example, conditional requirements are spelled out, and expected outputs are stated. An agent has enough to construct valid calls.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3; the description pulls ahead with examples showing exact JSON payloads for every operation class and Hints that map each reopenIf variant to its required fields (seconds, additionalUsers, occurrences+hours, additionalOccurrences). This adds compositional knowledge beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Update the status of an error') and immediately enumerates exact operations in Use Cases (open/fixed/ignored, discard, severity, snooze, link issue). The tool name and BugSnag sibling set make it the clear mutation counterpart to read-only tools like bugsnag_get_error and bugsnag_list_project_errors.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit 'Use Cases' list gives concrete scenarios (mark fixed, snooze, discard) that tell an agent when to invoke it. It does not name alternatives or state when not to use it, which would be the next step up.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_create_remote_system_configurationCollaborator: Create Remote System ConfigurationC
Read-onlyIdempotent
Inspect

Creates a remote system configuration in Collaborator (e.g., Bitbucket, GitHub, etc).

Toolset: Remote System Configuration Management

Parameters:

  • token (string) required: Remote system token, e.g., BITBUCKET, GITHUB, etc.

  • title (string) required: Remote system title.

  • config (string) required: JSON string containing configuration parameters for the remote system.

  • reviewTemplateId (string): Optional review template ID used by this remote system.

ParametersJSON Schema
NameRequiredDescriptionDefault
titleYesRemote system title.
tokenYesRemote system token, e.g., BITBUCKET, GITHUB, etc.
configYesJSON string containing configuration parameters for the remote system.
reviewTemplateIdNoOptional review template ID used by this remote system.

TDQS

C2.8/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description says the tool 'Creates' a configuration, while the annotations declare readOnlyHint: true. This is a direct contradiction about whether the operation mutates state. The description also provides no additional behavioral context such as authorization requirements, idempotency implications, or response behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose sentence is front-loaded and concise, but the parameter section duplicates the input schema's descriptions, which is redundant. The toolset line provides marginal context, yet several lines add little information beyond what an agent already has from the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a create operation with four parameters and no output schema, the description leaves out when to use it, what effects occur, and any details about the JSON config structure or token format beyond the schema. The readOnlyHint contradiction further undermines an agent's ability to understand the operation's true behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter list only repeats the schema's own descriptions without adding syntax, format examples, or constraints. The description adds no semantic value beyond the structured schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Creates a remote system configuration in Collaborator', a specific verb plus resource, and gives concrete examples (Bitbucket, GitHub). The create action is implicitly distinct from sibling tools like edit, delete, update, and test-connection for remote system configurations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to choose this tool over alternatives such as collaborator_edit_remote_system_configuration or collaborator_test_remote_system_configuration_connection. The 'Toolset' label provides weak context but no conditions, prerequisites, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_create_reviewCollaborator: Create ReviewB
Read-onlyIdempotent
Inspect

Creates a new review in Collaborator. All parameters are optional.

Toolset: Review Management

Parameters:

  • creator (string): Collaborator username of the review creator. Optional. Default: currently logged in user.

  • title (string): Title of the review. Optional. Default: null.

  • templateName (string): Review template name. Optional. Default: system default template.

  • accessPolicy (string): Access policy for the review. Optional. Default: ANYONE.

ParametersJSON Schema
NameRequiredDescriptionDefault
titleNoTitle of the review. Optional. Default: null.
creatorNoCollaborator username of the review creator. Optional. Default: currently logged in user.
accessPolicyNoAccess policy for the review. Optional. Default: ANYONE.
templateNameNoReview template name. Optional. Default: system default template.

TDQS

B3.1/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description states a state-changing action ('Creates a new review'), while annotations declare readOnlyHint=true and idempotentHint=true. This is a direct contradiction: readOnlyHint indicates no side effects, but creation is a side effect. The description also does not explain creation results, return values, or required permissions, but the contradiction alone forces a minimum score.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose statement is front-loaded and the layout is scannable, but the Parameter section unnecessarily repeats the full schema content, including each default. This duplication adds bulk without informational value, though the overall length is still modest. Around half the sentences could be removed without loss.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with zero required parameters and simple string inputs, the description covers the invocation surface: purpose, toolset, and all parameter defaults. However, it omits any return/output description and, due to the annotation conflict, the side-effect profile is ambiguous. This is minimally viable but not fully complete for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter list duplicates the schema's names, descriptions, optionality, and defaults almost verbatim. It adds no new meaning beyond the schema; 'All parameters are optional' is already encoded by required: [] and each field's Optional note. Therefore baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a concrete verb-object phrase, 'Creates a new review in Collaborator,' which directly states the tool's action and resource. The toolset tag and optional-parameter note reinforce that this is a creation operation, distinguishing it from sibling read/reject tools like collaborator_get_reviews or collaborator_reject_review.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or when-not-to-use guidance and no named alternatives. The phrase 'Creates a new review' implies the intended use, and 'All parameters are optional' clarifies there are no prerequisites, but the description never tells the agent when to prefer this over collaborator_reviewservice_action or find/get operations. This is implied usage rather than explicit direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_delete_remote_system_configurationCollaborator: Delete Remote System ConfigurationC
Read-onlyIdempotent
Inspect

Deletes a remote system configuration in Collaborator by its ID.

Toolset: Remote System Configuration Management

Parameters:

  • id (union) required: ID of the remote system Configuration to delete.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesID of the remote system Configuration to delete.

TDQS

C2.9/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description states the tool 'Deletes' a configuration, which is a mutating and destructive operation, but the annotations declare readOnlyHint=true and destructiveHint=false. This is a direct contradiction that makes the tool's actual behavior unreliable for an agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core action is stated in one clear, front-loaded sentence. The additional Toolset and Parameters blocks are somewhat redundant with the structured data but do not make the description excessively long.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Even though the tool has only one parameter, the description omits important behavioral context such as whether deletion is reversible, what happens to dependent data, or required permissions. The annotations further mislabel the operation as read-only, making the overall definition unreliable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description merely repeats the schema text: 'ID of the remote system Configuration to delete.' The parameter is already fully documented in the schema, so the description adds little semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Deletes' and identifies the exact resource, 'remote system configuration in Collaborator', with the key selector 'by its ID'. This clearly differentiates it from sibling tools like create, edit, or test remote system configuration.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. The 'Toolset' label only names a category without explaining when deletion is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_edit_remote_system_configurationCollaborator: Edit Remote System ConfigurationC
Read-onlyIdempotent
Inspect

Edits parameters of an existing remote system configuration in Collaborator. Only title and config are editable after creation.

Toolset: Remote System Configuration Management

Parameters:

  • id (string) required: ID of the remote system Configuration to edit.

  • title (string): Remote system title.

  • config (string): JSON string containing configuration parameters for the remote system.

  • reviewTemplateId (string): Optional review template ID used by this remote system.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesID of the remote system Configuration to edit.
titleNoRemote system title.
configNoJSON string containing configuration parameters for the remote system.
reviewTemplateIdNoOptional review template ID used by this remote system.

TDQS

C2.6/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotation Contradiction: the description says 'Edits parameters' and the tool is an edit operation, but the annotations declare readOnlyHint=true, implying the tool does not modify state. This is a direct and serious inconsistency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is clear and front-loaded, and the post-creation constraint is useful. But the description then duplicates the entire parameter schema verbatim, which adds length without adding information and contributes to the reviewTemplateId inconsistency.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The definition omits return behavior and gives no guidance on what the response looks like. More importantly, the contradictory 'only title and config are editable' note versus the reviewTemplateId parameter leaves a critical ambiguity, and no detail is given about what the config JSON should contain. This is not sufficient for reliable agent invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters. However, the description's statement that 'Only title and config are editable after creation' directly conflicts with its own parameter list, which includes reviewTemplateId as an editable parameter, leaving the agent uncertain about valid inputs. It also provides no extra detail about the JSON config format.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Edits') and identifies the exact resource ('parameters of an existing remote system configuration in Collaborator'). It is clearly distinct from creating, deleting, or testing a configuration, though it does not explicitly name sibling alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description establishes that this tool is for editing an existing configuration and notes a key constraint: only title and config are editable after creation. However, it does not explicitly guide the agent to create/delete/test siblings when those operations are more appropriate, relying instead on the tool name and context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_find_review_by_idCollaborator: Find Review By IDA
Read-onlyIdempotent
Inspect

Finds a review in Collaborator by its review ID.

Toolset: Review Management

Parameters:

  • reviewId (string) required: The Collaborator review ID to find.

ParametersJSON Schema
NameRequiredDescriptionDefault
reviewIdYesThe Collaborator review ID to find.

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds no behavioral context beyond that: it does not mention return format, not-found behavior, or any operational caveats, so it provides little transparency beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded, with a single clear purpose statement followed by a small toolset label and parameter list. The parameter list is somewhat redundant with the schema, but it is compact and does not introduce bloat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter lookup with rich safety annotations, the description is mostly complete. It lacks an explicit statement of the return value (relevant since there is no output schema) and any error/not-found behavior, but given the tool's simplicity, these omissions are minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for reviewId, and the description repeats the exact phrase 'The Collaborator review ID to find.' No additional meaning is added, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Finds') and resource ('a review in Collaborator') and clearly identifies the selection criterion ('by its review ID'). This is immediately distinct from the list-oriented sibling collaborator_get_reviews and other review actions, so an agent can tell what it does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'by its review ID' phrasing implies this is for looking up a single known review, but the description never explicitly contrasts it with collaborator_get_reviews or states that it should be used when an ID is already known. Usage context is implied, not spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_get_reviewsCollaborator: Get ReviewsB
Read-onlyIdempotent
Inspect

Retrieves reviews from Collaborator using ReviewService.getReviews. All parameters are optional and only provided ones are sent.

Toolset: Review Management

Parameters:

  • login (string): Collaborator username to filter reviews.

  • role (string): Role to filter reviews (e.g., AUTHOR).

  • creator (boolean): Whether to filter by creator.

  • reviewPhase (string): Review phase to filter (e.g., PLANNING).

  • fullInfo (boolean): Whether to retrieve full review info.

  • fromDate (string): Minimal creation date in format "yyyy-MM-dd"

  • toDate (string): Maximal creation date in format "yyyy-MM-dd"

ParametersJSON Schema
NameRequiredDescriptionDefault
roleNoRole to filter reviews (e.g., AUTHOR).
loginNoCollaborator username to filter reviews.
toDateNoMaximal creation date in format "yyyy-MM-dd"
creatorNoWhether to filter by creator.
fromDateNoMinimal creation date in format "yyyy-MM-dd"
fullInfoNoWhether to retrieve full review info.
reviewPhaseNoReview phase to filter (e.g., PLANNING).

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this is read-only, idempotent, and non-destructive, so the safety profile is covered. The description adds one useful behavioral nuance: all parameters are optional and only provided ones are sent. It does not disclose return shape, pagination, default behavior, or authentication requirements, but the annotation coverage lowers the burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is clear and front-loaded, but the description then duplicates every parameter already present in the input schema. The 'Toolset: Review Management' line adds slight structure, but the repeated parameter listing is redundant and makes the description longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list/filter tool with 7 optional parameters and no output schema, the description covers the filtering semantics adequately via schema descriptions. However, it does not explain what fullInfo changes, what values reviewPhase accepts, how the filters combine, or what the returned review objects look like. This incomplete context may leave an agent uncertain about invocation side effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema description coverage is 100%, so the schema already documents every parameter. The description repeats the parameter list but adds little beyond the sentence that all parameters are optional and only provided ones are sent, which is already implied by zero required parameters. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Retrieves reviews from Collaborator') and names the underlying service method. The plural 'reviews' and tool name distinguish it from the sibling find_review_by_id, but it does not explicitly contrast itself with that sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus collaborator_find_review_by_id, collaborator_reviewservice_action, or any other sibling. The description does not mention exclusions or alternatives, leaving the agent to infer selection criteria from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_reject_reviewCollaborator: Reject ReviewC
Read-onlyIdempotent
Inspect

Rejects a review in Collaborator by its review ID and reason.

Toolset: Review Management

Parameters:

  • reviewId (union) required: The Collaborator review ID to reject.

  • reason (string) required: Reason for rejecting the review.

ParametersJSON Schema
NameRequiredDescriptionDefault
reasonYesReason for rejecting the review.
reviewIdYesThe Collaborator review ID to reject.

TDQS

C2.9/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description states 'Rejects a review,' which implies a state-changing action, but the annotations declare readOnlyHint=true and idempotentHint=true. Rejecting a review is not read-only, so the description directly contradicts the annotation metadata. No additional behavioral context such as side effects, permissions, or irreversibility is provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is one concise, front-loaded sentence that clearly states the action and key inputs. The subsequent parameter block and toolset line add some redundancy with the schema but do not make the description bloated or hard to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a mutation-like operation with no output schema and no explanation of the result or side effects of rejecting a review. The annotation contradiction makes the behavioral model unreliable, leaving an agent unsure whether the tool modifies state. A simple 2-parameter schema does not compensate for the missing usage and behavioral context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter list simply duplicates the schema's field names and descriptions. It adds no new semantic detail such as value formats, constraints, or relationships between reviewId and reason. The baseline of 3 is appropriate because the schema already documents both parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Rejects a review in Collaborator'. It also names the required inputs, 'by its review ID and reason', making the operation unambiguous. Among many sibling tools, this is clearly the reject-review operation rather than create, find, or get reviews.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives such as collaborator_reviewservice_action, collaborator_get_reviews, or collaborator_find_review_by_id. The 'Toolset: Review Management' line only categorizes the tool; it does not explain conditions for use or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_reviewservice_actionCollaborator: ReviewService ActionD
Read-onlyIdempotent
Inspect

Invoke any ReviewService method by name and arguments. For finishReviewPhase and waitOnPhase, provide reviewId (required) and until (optional, defaults to 'ANY').

Toolset: Review Management

Parameters:

  • action (enum) required

  • args (record<string, any>) required

ParametersJSON Schema
NameRequiredDescriptionDefault
argsYes
actionYes

TDQS

D1.8/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description contradicts the annotations: readOnlyHint=true while the described generic invocation plus schema actions such as cancel, reopen, and uncancel imply state-changing behavior. It also fails to disclose side effects, preconditions, or failures for the supported action enum.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The text is short and the core purpose is front-loaded, but it wastes space repeating parameter metadata from the schema and the Toolset line, and it front-loads an inaccurate sentence. The structure is neat but the content is not reliable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a generic dispatcher with a nested free-form args object and no output schema, the description is severely incomplete: it omits valid action semantics, return values, error behavior, and examples. An agent cannot determine how to construct args for any of the four actual enum values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description does not compensate for the actual enum actions. It only documents reviewId/until for finishReviewPhase and waitOnPhase, which are not valid action values, leaving args for moveReviewToAnnotatePhase, cancel, reopen, and uncancel completely unspecified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description says it can 'Invoke any ReviewService method by name and arguments,' which identifies a generic dispatch tool on a specific service, but 'any method' is contradicted by the action enum and by mentioning finishReviewPhase/waitOnPhase that are not in that enum. It does not distinguish this generic action from the dedicated collaborator_* review tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use or alternative selection guidance is provided. The only usage detail is for two methods absent from the schema enum, and there is no indication that dedicated tools should be preferred for common operations such as creating or rejecting a review.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_test_remote_system_configuration_connectionCollaborator: Test Remote System Configuration ConnectionA
Read-onlyIdempotent
Inspect

Tests the connection for a remote system configuration in Collaborator by its ID.

Toolset: Remote System Configuration Management

Parameters:

  • id (union) required: ID of the remote system Configuration to test connection for.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesID of the remote system Configuration to test connection for.

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds only the connection-testing action and does not disclose response/outcome behavior, potential network dependencies, or whether credentials are validated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core definition is one clear sentence, and the toolset/parameter sections are compact. The parameter block is redundant with the schema but does not bloat the text enough to reduce usability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, safely annotated operation this is largely sufficient to invoke correctly. Still, with no output schema, the description does not explain what a successful/failed connection test returns or how the result is expressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the description repeats the id meaning verbatim without adding additional type, format, or source guidance. Baseline 3 is appropriate because the schema does all the explanatory work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Tests') and precise resource ('the connection for a remote system configuration in Collaborator by its ID'). This clearly differentiates it from sibling create/edit/delete/config-webhook operations even though it does not name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Remote System Configuration Management' toolset heading loosely positions this as the connectivity-check operation among related configuration tools. However, there is no explicit statement of when to use this versus creating, editing, or deleting a configuration, and no alternative tools are named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

collaborator_update_remote_system_configuration_webhookCollaborator: Update Remote System Configuration WebhookC
Read-onlyIdempotent
Inspect

Updates the webhook for a remote system configuration in Collaborator by its ID.

Toolset: Remote System Configuration Management

Parameters:

  • id (union) required: ID of the remote system Configuration to update the webhook for.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesID of the remote system Configuration to update the webhook for.

TDQS

C2.6/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description says 'Updates the webhook,' which is a write operation, while the annotations declare readOnlyHint: true. This directly contradicts the annotation and could cause an agent to treat this as a non-mutating call. No additional behavioral context is provided beyond the conflicting action verb.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is clear and front-loaded, but the description includes a 'Parameters:' block that duplicates the schema verbatim. The 'Toolset' line adds mild context, so the description is short but contains redundant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with one parameter and no output schema, the description provides the minimum callable information. However, it leaves the annotation contradiction unaddressed and omits any detail about the webhook payload, required permissions, or response, making it insufficient for fully reliable invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description repeats the id parameter's purpose without adding new details such as ID format, source, or prerequisites. The baseline of 3 applies because the schema already fully documents the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Updates the webhook for a remote system configuration in Collaborator by its ID.' This distinguishes it from sibling configuration-management tools like collaborator_edit_remote_system_configuration, though it does not clarify what webhook fields can be updated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as collaborator_edit_remote_system_configuration or webhook-related operations. There are no prerequisites, exclusions, or conditions that would help an agent choose between this and competing tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_add_label_to_pacticipantContract Testing: Add Label to PacticipantBInspect

Apply a label to a pacticipant.

Toolset: Labels

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • labelName (string) required: Name of the label

ParametersJSON Schema
NameRequiredDescriptionDefault
labelNameYesName of the label
pacticipantNameYesName of the pacticipant

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a mutation (readOnlyHint=false), but the description adds no behavioral context beyond the basic action. It does not disclose whether applying an existing label is idempotent, whether duplicate labels can occur, or what side effects or errors should be expected.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is compact and front-loaded with the core action. The parameter list is somewhat redundant because it duplicates the schema, but it does not significantly bloat the description and the overall structure is clear.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter mutation, the description plus schema is generally sufficient to invoke the tool. However, key operational context is missing, such as whether adding the same label twice has any effect, whether the label must already exist, and any constraints around the pacticipant name.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents both parameters. The description repeats the schema's parameter names and descriptions without adding extra semantics such as value formats, label existence requirements, or relationships between the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Apply') and a clear resource ('label to a pacticipant'), making the operation unambiguous. It also distinguishes naturally from the sibling tool contract-testing_remove_label_from_pacticipant, so an agent can tell addition from removal without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives, nor are prerequisites mentioned such as whether the pacticipant or label must already exist. The 'Toolset: Labels' label provides minimal grouping context but no explicit decision guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_add_role_to_userContract Testing: Admin Add Role to UserBInspect

Add a single role to a user (admin).

Toolset: Admin

Parameters:

  • userId (string) required: UUID of the user

  • roleId (string) required: UUID of the role

ParametersJSON Schema
NameRequiredDescriptionDefault
roleIdYesUUID of the role
userIdYesUUID of the user

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish the tool is not read-only, not idempotent, and not destructive. The description adds the admin context and single-role semantics, but it does not disclose what happens on duplicate assignment, whether the user or role must already exist, or any side effects beyond the role addition.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action. However, the 'Parameters' block duplicates the schema verbatim, and 'Toolset: Admin' repeats title information, so some lines do not add new value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter mutation with annotations and full schema coverage, the description is nearly sufficient: an agent can determine the operation, required parameters, and admin context. The main gap is explicit routing guidance versus admin_set_user_roles, but that is more a selection concern than an invocation blocker.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for both parameters, and the description restates the exact schema text for userId and roleId. It adds no new meaning beyond the parameter names, types, required flags, and UUID format already present in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Add'), resource ('a single role'), and target ('to a user'), plus the admin qualifier. The phrase 'single role' helps distinguish it from sibling admin_set_user_roles, which would set or replace multiple roles.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus admin_set_user_roles or admin_remove_role_from_user. No preconditions, exclusions, or alternative-selection hints are given; only 'admin' is mentioned, which already appears in the title.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_create_roleContract Testing: Admin Create RoleCInspect

Create a new role with specific permissions (admin).

Toolset: Admin

Parameters:

  • name (string) required: Name of the role

  • permissions (array) required: Permissions granted by this role

  • description (string): Description of the role

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the role
descriptionNoDescription of the role
permissionsYesPermissions granted by this role

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description only restates the create action, which is already captured by annotations (readOnlyHint=false). It adds no unique behavioral contextβ€”no mention of idempotency, required admin privileges, uniqueness constraints, or what happens after creation. With annotations present, the description contributes no additional disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core first sentence is concise and front-loaded with the operation. However, the following parameter list fully duplicates the input schema, adding redundancy without new information. The structure is adequate but not tightly edited.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a create operation with no output schema, the description does not state response details, name uniqueness, or admin-level authorization requirements. The full parameter schema does allow an agent to construct a valid request, so it is minimally adequate, but several contextual gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including the permissions item's scope property. The description repeats the parameter names and descriptions verbatim without adding examples, constraints, or relationships beyond the schema. This meets the baseline for high schema coverage but adds no extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the operation: 'Create a new role with specific permissions.' The verb 'Create' and resource 'role' are unambiguous, and the sibling tools (admin_update_role, admin_delete_role) are distinguished by verb. However, the parenthetical '(admin)' is ambiguousβ€”it likely refers to the toolset but could be misread as a permission modifier.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like admin_update_role, admin_set_user_roles, or admin_add_role_to_user. The 'Toolset: Admin' label gives only a category, not selection criteria. There are no stated exclusions or conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_create_system_accountContract Testing: Admin Create System AccountBInspect

Create a new system account (admin).

Toolset: Admin

Parameters:

  • name (string) required: Name of the system account

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the system account

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate a mutating, non-idempotent operation (readOnlyHint=false, idempotentHint=false), so the description's 'Create' confirms the action type rather than contradicting it. It adds little beyond thatβ€”no mention of auth requirements, uniqueness, side effects, or return behaviorβ€”though 'Toolset: Admin' hints at privilege.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loads the primary action, and the Toolset/Parameters layout is scannable. However, the Parameters block duplicates the schema, so not every part of the description earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter creation tool, the minimum required to invoke itβ€”action and required fieldβ€”is present. Missing context about output, naming constraints, or admin permissions keeps it from being complete, but the low complexity and annotation coverage make it adequately usable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already fully documents the single required `name` parameter with 100% coverage. The description's Parameters section repeats that information without adding constraints, defaults, or examples, so it meets the baseline but adds no extra semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Create') and identifies the resource ('a new system account'), and the 'Toolset: Admin' heading clarifies the administrative scope. The parenthetical '(admin)' is slightly ambiguous, but the action and object are clear enough to distinguish this from most sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or alternative guidance; an agent must infer that this tool is for creating system accounts. 'Toolset: Admin' provides loose context, but it doesn't differentiate this from admin_create_user or admin_create_team or state prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_create_teamContract Testing: Admin Create TeamBInspect

Create a new team (admin).

Toolset: Admin

Parameters:

  • name (string) required: Name of the team

  • administratorUuids (array): UUIDs of team administrators

  • environmentUuids (array): UUIDs of environments assigned to this team

  • pacticipantNames (array): Names of pacticipants assigned to this team

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the team
environmentUuidsNoUUIDs of environments assigned to this team
pacticipantNamesNoNames of pacticipants assigned to this team
administratorUuidsNoUUIDs of team administrators

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=false, idempotentHint=false, and destructiveHint=false, so the description is consistent with a non-read-only, non-idempotent mutation. The description adds the 'admin' scope, which hints at privilege requirements, but does not disclose side effects, return values, or any pitfalls. This is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core statement is front-loaded and clear: 'Create a new team (admin).' However, the subsequent parameter list duplicates information already present in the input schema, adding unneeded length. The structure is clean and readable, but not every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple create operation with a fully documented schema, the description provides enough to invoke the tool with required parameters. However, there is no output schema and the description does not mention what response to expect, whether referenced UUIDs must already exist, or how this fits into team management workflows. Additional behavioral context would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents name, administratorUuids, environmentUuids, and pacticipantNames. The description merely duplicates these parameter definitions without adding new semantics like relationships, validation rules, or usage examples. Baseline 3 is appropriate because the schema carries the descriptive burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Create a new team (admin).' The action is unambiguous and the 'admin' qualifier distinguishes this from non-admin team operations. Among siblings like admin_update_team, admin_delete_team, and admin_list_teams, the 'create' verb clearly identifies this tool's role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives. It does not mention that admin_update_team should be used for modifying existing teams, admin_delete_team for removal, or that this requires admin privileges beyond the word '(admin)' in the description. The usage context is only implied by the action itself, not explicitly stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_create_userContract Testing: Admin Create UserCInspect

Create a new user account (admin).

Toolset: Admin

Parameters:

  • email (string) required: Email address of the new user

  • name (string) required: Display name of the new user

  • firstName (string): First name

  • lastName (string): Last name

  • externalIdpId (string): External identity provider ID (for SAML/SSO)

  • externalIdpUsername (string): External IdP username (for SAML/SSO)

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name of the new user
emailYesEmail address of the new user
lastNameNoLast name
firstNameNoFirst name
externalIdpIdNoExternal identity provider ID (for SAML/SSO)
externalIdpUsernameNoExternal IdP username (for SAML/SSO)

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is not read-only, not idempotent, and not destructive, so the agent knows it is a write operation. However, the description adds little beyond the tool name: it does not disclose whether the created user receives an invitation, whether it can be used with external IdP accounts, or how conflicts or duplicate emails are handled.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and fronts the core purpose, but it spends several lines repeating parameter information already present in the input schema. The 'Toolset: Admin' line and the full parameter list add little marginal value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutating admin operation with no output schema and no behavioral disclaimers, the description is thin. An agent is left without knowing what the response will be, whether an email is sent, whether the user is active immediately, or what admin permissions are required.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all six parameters. The description duplicates those same descriptions without adding additional meaning or clarifying relationships between fields such as externalIdpId and externalIdpUsername.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Create a new user account') on a clear resource and scopes it as an admin operation. It is distinct from nearby siblings like admin_update_user or admin_invite_users, though it does not explicitly call out those distinctions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus alternatives like admin_invite_users, admin_create_system_account, or admin_set_user_roles. The 'Toolset: Admin' label is generic and does not help an agent choose among the many admin siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_delete_roleContract Testing: Admin Delete RoleB
DestructiveIdempotent
Inspect

Delete a role (admin).

Toolset: Admin

Parameters:

  • roleId (string) required: UUID of the role

ParametersJSON Schema
NameRequiredDescriptionDefault
roleIdYesUUID of the role

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true. The description adds no additional behavioral context such as irreversibility, cascading effects, permissions, or what happens to related data. It merely restates the delete action already obvious from the name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action. The toolset and parameter listing are useful, though the parameter block is partially redundant with the input schema, preventing a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with one parameter and annotations cover its destructive nature. However, the description omits any details about deletion consequences, success response, or required permissions, leaving some ambiguity for a mutating operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter section simply duplicates the schema's own roleId description. No extra meaning, format constraints, or relationships are added, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise action ('Delete') and resource ('role'), with '(admin)' clarifying the toolset. It is unambiguous and distinguishable from sibling tools like admin_delete_user or admin_update_role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like admin_reset_roles, admin_update_role, or other admin deletes. The toolset label is helpful but does not convey usage context, prerequisites, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_delete_teamContract Testing: Admin Delete TeamB
DestructiveIdempotent
Inspect

Delete a team (admin).

Toolset: Admin

Parameters:

  • teamId (string) required: UUID of the team

ParametersJSON Schema
NameRequiredDescriptionDefault
teamIdYesUUID of the team

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, readOnlyHint=false, and idempotentHint=true, so the core safety profile is known. The description adds the 'admin' scope but does not disclose whether deletion is permanent, whether it cascades to related resources, or what side effects may occur. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core purpose. The parameter section is redundant with the input schema but is minimal and does not significantly harm readability. A slightly leaner version could drop the repeated parameter details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter and existing annotations, the description is mostly adequate for invoking the tool. However, for a destructive admin operation, it does not explicitly state that the deletion is irreversible, whether it affects team members or dependencies, or what success/failure looks like since there is no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description only repeats exactly what the schema already says: teamId is a required UUID string. No additional meaning, formatting guidance, or value constraints are provided beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Delete a team (admin).' It clearly identifies the operation and distinguishes it from sibling admin team operations by the delete action. However, it largely restates the tool name and does not add any differentiating context beyond the action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like admin_update_team or admin_get_team. The description implies the obvious use caseβ€”deleting a teamβ€”but provides no exclusions, prerequisites, or decision context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_delete_userContract Testing: Admin Delete UserB
DestructiveIdempotent
Inspect

Delete a user account (admin).

Toolset: Admin

Parameters:

  • userId (string) required: UUID of the user

ParametersJSON Schema
NameRequiredDescriptionDefault
userIdYesUUID of the user

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, readOnlyHint=false, and idempotentHint=true, so the description adds little beyond what is already structured. It does not disclose irreversibility, cascading effects on related data, or any security/permission requirements beyond the already present '(admin)' label.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action, followed by a minimal parameter list. It contains no filler, though the parameter list duplicates the schema; still, it remains appropriately sized for a simple one-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter admin delete with no output schema, the description covers the essential action and parameter. However, it omits any mention of error cases, what happens to the deleted user's data, or how this interacts with team memberships and roles, leaving some ambiguity in a destructive Admin operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with a single parameter userId, and the schema itself provides the description 'UUID of the user'. The description only repeats this information without adding meaning such as format, examples, or how to obtain the UUID.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Delete a user account (admin).' This clearly distinguishes it from sibling tools like admin_update_user, admin_create_user, and admin_invite_users, so an agent can identify the correct operation without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance about when to use this tool versus alternatives, nor does it mention prerequisites or conditions. The 'admin' qualifier hints at scope but does not say, for example, when deletion is preferred over updating or deactivating a user.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_get_roleContract Testing: Admin Get RoleA
Read-onlyIdempotent
Inspect

Retrieve details for a specific role by UUID (admin).

Toolset: Admin

Parameters:

  • roleId (string) required: UUID of the role

ParametersJSON Schema
NameRequiredDescriptionDefault
roleIdYesUUID of the role

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description identifies this as an admin-level read operation, adding an authorization signal not present in the annotations. It is consistent with readOnlyHint, idempotentHint, and destructiveHint=false, and does not contradict them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main sentence is concise and front-loaded, with a short toolset label and parameter list. The parameter list is redundant with the schema but does not make the description overly long or difficult to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, read-only, idempotent retrieval by UUID, the description provides enough information to invoke the tool correctly. It does not describe response shape or not-found behavior, but the operation is simple and no output schema is provided.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description simply restates roleId/UUID with no additional semantic details such as format constraints, lookup behavior, or error conditions. The schema fully documents the only parameter, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve') and resource ('details for a specific role by UUID'), so an agent knows exactly what the tool does. It distinguishes from list-style tools by emphasizing 'specific role by UUID', but it does not explicitly name a sibling alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternatives such as admin_list_roles. 'Toolset: Admin' provides context but no explicit when-to-use or when-not-to-use direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_get_system_account_tokensContract Testing: Admin Get System Account TokensB
Read-onlyIdempotent
Inspect

Retrieve API tokens for a system account (admin).

Toolset: Admin

Parameters:

  • accountId (string) required: UUID of the system account

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
accountIdYesUUID of the system account
pageNumberNoPage number

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description's 'Retrieve' aligns with those. The description adds the system-account scope but not additional behavioral context such as authorization requirements or response/pagination behavior beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The summary sentence is front-loaded and clear. However, the parameter list duplicates the input schema almost verbatim, adding redundancy rather than new guidance for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only, paginated lookup with 3 fully documented parameters and safety annotations, the description is largely complete. The main gap is that there is no output schema and the description does not characterize the returned token objects or paginated response format.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description repeats the same parameter meanings provided by the schema without adding new details. Since the structured schema already documents accountId, pageNumber, and pageSize, the description provides no additional parameter value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description uses a specific verb ('Retrieve') and identifies the resource ('API tokens for a system account'), which clearly differentiates this from generic token listing. However, it does not explicitly contrast with the sibling contract-testing_list_api_tokens, so the agent must infer the discriminator from the system-account qualifier.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Admin' toolset label and 'system account' qualifier imply this is for administrative retrieval, giving some context. There is no explicit statement of when to prefer this tool over contract-testing_list_api_tokens or any exclusion of non-admin contexts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_get_teamContract Testing: Admin Get TeamA
Read-onlyIdempotent
Inspect

Retrieve details for a specific team by UUID (admin).

Toolset: Admin

Parameters:

  • teamId (string) required: UUID of the team

ParametersJSON Schema
NameRequiredDescriptionDefault
teamIdYesUUID of the team

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the main safety profile. The description adds only the admin toolset and single-team scope; it does not add auth, rate-limit, or response-behavior context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The operation is front-loaded and the overall description is short. However, the parameters block largely duplicates the input schema, so not every line earns its place, though the redundancy is minor.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, read-only, idempotent admin lookup, the description is sufficient for invocation. It does not specify the exact team fields returned or explicitly state admin credential requirements, but the low complexity and rich annotations make that gap minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the description simply restates the teamId parameter's type, requiredness, and UUID meaning. No additional parameter semantics or format guidance is provided beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific operation ('Retrieve details'), a specific resource ('a specific team'), and the identifying key ('by UUID'). It also marks the admin scope, which distinguishes it from sibling tools like admin_list_teams or team-user lookups.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly conveys that this is the admin-level single-team lookup by UUID, which is the selecting context for when to call it. It does not explicitly name alternatives or state when not to use it, but the context is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_get_team_userContract Testing: Admin Get Team UserA
Read-onlyIdempotent
Inspect

Check if a specific user is a member of a team (admin).

Toolset: Admin

Parameters:

  • teamId (string) required: UUID of the team

  • userId (string) required: UUID of the user

ParametersJSON Schema
NameRequiredDescriptionDefault
teamIdYesUUID of the team
userIdYesUUID of the user

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint, idempotentHint, and non-destructive behavior. The description adds the admin authorization context and confirms the operation is a read-only membership check, but it does not disclose return shape, error behavior, or behavior when the team or user does not exist. There is no contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The key sentence is front-loaded and easy to parse, and the overall description is compact. However, the 'Toolset: Admin' line and the full Parameters section duplicate the name and input schema, creating minor redundancy that prevents a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only, two-parameter membership check, the description provides the essential information: the operation, both required UUIDs, and the admin scope. The lack of an output schema and any mention of the return value (e.g., boolean) is a small gap, but not severe for this tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter list repeats the schema's teamId and userId descriptions almost verbatim. It adds no additional semantic meaning beyond what the input schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete operation: checking whether a specific user is a member of a team, with admin scope. This is clear and specific, but it does not explicitly differentiate itself from nearby sibling tools such as contract-testing_admin_get_user, contract-testing_admin_get_team, or contract-testing_admin_list_team_users.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The use case is implied by 'Check if a specific user is a member of a team' and the 'Toolset: Admin' label, but there is no explicit when-to-use guidance, no exclusions, and no mention of when a different admin tool would be more appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_get_userContract Testing: Admin Get UserA
Read-onlyIdempotent
Inspect

Retrieve details for a specific user by UUID (admin).

Toolset: Admin

Parameters:

  • userId (string) required: UUID of the user

ParametersJSON Schema
NameRequiredDescriptionDefault
userIdYesUUID of the user

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds only 'admin' scope and UUID lookup, which is mild context but no meaningful behavioral disclosure beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded with the essential operation and scope. The Toolset and Parameters block is somewhat redundant with the input schema, but the overall size remains compact and scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-required-parameter, read-only lookup tool, the description is nearly sufficient. It does not list what fields are returned and there is no output schema, so an agent cannot know the exact shape of the response, but the gap is minor for such a simple get-by-UUID operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully documents userId as a required string UUID, with 100% schema description coverage. The description repeats 'UUID' and 'required' but adds no additional constraints, examples, or error behavior, so it only meets the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ('Retrieve details'), a specific resource ('user'), a lookup key ('by UUID'), and the scope ('admin'). This is distinguishable from sibling tools like contract-testing_admin_list_users or contract-testing_get_current_user without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'admin' scope implies this is for administrator-facing lookup of an arbitrary user, so usage context is somewhat clear. However, it does not explicitly say when to use this tool versus sibling tools such as admin_list_users, get_current_user, or admin_get_team_user.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_invite_usersContract Testing: Admin Invite UsersBInspect

Send invitations to new users (admin).

Toolset: Admin

Parameters:

  • users (array) required: List of users to invite

ParametersJSON Schema
NameRequiredDescriptionDefault
usersYesList of users to invite

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a non-read-only, non-idempotent, non-destructive operation with open-world side effects. The description adds minimal behavioral context beyond 'send invitations,' and does not explain what happens after invitations are sent, such as email delivery or whether duplicate invites are allowed. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the action sentence, but the 'Toolset: Admin' line duplicates the title, and the 'Parameters' section repeats what the input schema already provides. Some redundancy prevents a higher conciseness score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a low-complexity tool with one well-documented required parameter and annotations covering idempotency, destructiveness, and side effects. The description is nearly sufficient for an agent to invoke it correctly, though it could be more complete by clarifying invitation behavior and expected results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema already documents the users array, its required email/name fields, and minItems. The description's parameter section merely restates the schema without adding extra meaning or usage detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Send invitations to new users (admin).' This clearly identifies the action and distinguishes it from listing, updating, or deleting users. It does not explicitly contrast with the sibling admin_create_user, but 'invite' versus 'create' is still a meaningful distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives such as contract-testing_admin_create_user, nor does it explain whether users must already exist or what conditions apply. The '(admin)' tag signals scope, but not usage context or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_list_permissionsContract Testing: Admin List PermissionsB
Read-onlyIdempotent
Inspect

List all available permission scopes (admin).

Toolset: Admin

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the admin-only context and pagination parameters, but it does not disclose behavior such as sort order, response format, or pagination limits beyond the default values.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loads the core purpose, but it includes redundant content: 'Toolset: Admin' duplicates the parenthetical admin qualifier, and the parameter section repeats the schema exactly. It is not bloated, but not every line earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only paginated list with two optional, well-documented parameters and strong safety annotations, the description is mostly sufficient. However, there is no output schema and the description does not explain the response shape or whether permission scope names/IDs are returned, leaving some ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both pageNumber and pageSize already documented with defaults. The description simply repeats the schema parameters without adding extra meaning, so it neither compensates for a schema gap nor provides additional semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: 'List all available permission scopes (admin).' This clearly identifies the tool's purpose and distinguishes it from sibling admin tools that concern users, roles, or teams, though it does not explicitly contrast them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternative admin list tools such as admin_list_roles or admin_list_users. The '(admin)' qualifier implies a prerequisite but no explicit usage context, exclusions, or decision guidance is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_list_rolesContract Testing: Admin List RolesA
Read-onlyIdempotent
Inspect

List all roles defined in the workspace (admin).

Toolset: Admin

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is well established. The description adds workspace scoping and pagination defaults, which is useful, but it does not disclose return format, ordering, or admin permission requirements beyond what annotations imply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the main purpose, and the parameter list is compact. However, 'Toolset: Admin' and the repeated parameter list duplicate the title and schema, so not every line provides unique value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, paginated list operation with rich annotations and full schema coverage, the description supplies enough information to invoke the tool correctly. There is no output schema, so the return structure is unstated, but the tool's simple nature makes this a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents both parameters. The description repeats the two parameters with their defaults, adding no extra semantic meaning such as maximum page size, filtering, or ordering behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states an explicit verb ('List'), a concrete resource ('all roles defined in the workspace'), and clarifies it is an admin operation. This clearly distinguishes it from role-specific siblings like admin_get_role or admin_create_role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for the main use case but never mentions alternatives or when not to use it. It does not route the agent to admin_get_role for single-role lookup or admin_create_role for creation, so selection must be inferred from the tool name and sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_list_teamsContract Testing: Admin List TeamsB
Read-onlyIdempotent
Inspect

List all teams in the workspace (admin).

Toolset: Admin

Parameters:

  • q (string): Filter teams by name

  • page (number): Page number (default: 1)

  • size (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
qNoFilter teams by name
pageNoPage number
sizeNoResults per page

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered by structured data. The description adds workspace scope and the admin requirement, but it does not clarify how pagination behaves or whether 'all teams' means all records across pages or just the default page's results. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and reasonably organized, but the 'Toolset: Admin' line largely duplicates the title/name, and the parameter section duplicates the input schema exactly. The core purpose sentence is front-loaded and clear, but some content does not add value beyond structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list operation with fully documented parameters and safety annotations, the description is mostly adequate. The main gap is the potential contradiction between 'List all teams' and the default page/size behavior, which is important for an agent to know whether a single call returns all teams or only a page. There is also no indication of what the response shape will be, and no output schema is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents q, page, and size with defaults and meanings. The description simply repeats the parameter list and defaults without adding extra semantics such as filtering behavior, maximum page size, or how q is matched against team names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb-resource pair: 'List all teams in the workspace (admin)'. It distinguishes the operation from related siblings like admin_get_team (single team) or admin_list_team_users (team members) by indicating the resource is teams and the scope is all teams. However, it lacks an explicit boundary like 'no filtering' and relies partly on the title/name for differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as admin_get_team, admin_create_team, or admin_list_team_users. The phrase '(admin)' hints at privilege requirements but does not state usage conditions, prerequisites, or when another team-related admin tool should be preferred.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_list_team_usersContract Testing: Admin List Team UsersA
Read-onlyIdempotent
Inspect

List all users in a specific team (admin).

Toolset: Admin

Parameters:

  • teamId (string) required: UUID of the team

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
teamIdYesUUID of the team
pageSizeNoResults per page
pageNumberNoPage number

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds little behavioral context beyond the list operation, such as pagination behavior or admin permission requirements, which keeps this at a baseline score.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loads the core purpose in the first sentence. The parameter list and toolset label add some organizational clarity, though the parameter bullets are redundant with the schema. Overall it is well-structured and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple listing operation with complete schema documentation, clear pagination parameters, and annotations covering the read-only/idempotent behavior. There is no output schema, so some return-format detail could be missing, but for a straightforward team-user list tool the description is sufficient for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description simply repeats the parameter names, defaults, and requirements already present in the schema. It adds no additional meaning or format details beyond what the structured input definition provides, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear, specific verb and resource: 'List all users in a specific team (admin)'. This distinguishes it from the sibling admin_list_users, which would list users across teams, and from admin_get_team, which focuses on team details rather than membership.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly indicates the target scope: list users belonging to a specific team, and the required teamId parameter reinforces when this tool is appropriate. It does not explicitly name alternatives or exclusion conditions, but the context is clear enough for an agent to select it over the closest siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_list_usersContract Testing: Admin List UsersA
Read-onlyIdempotent
Inspect

List all users in the workspace (admin).

Toolset: Admin

Parameters:

  • active (boolean): Filter by active/inactive status

  • q (string): Filter by name or email

  • userType (number): 0 = regular users, 1 = system accounts

  • page (number): Page number (default: 1)

  • size (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
qNoFilter by name or email
pageNoPage number
sizeNoResults per page
activeNoFilter by active/inactive status
userTypeNo0 = regular users, 1 = system accounts

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds no behavioral detail beyond the workspace scope; no rate limits, response shape, or pagination semantics are disclosed, but the annotation coverage keeps this at a baseline 3.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the action and scope, and the parameter list is clearly formatted. However, it redundantly repeats the schema's parameter descriptions, which takes up space without adding new information. Still compact and readable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple paginated list tool with all parameters documented and safety annotations covering the read-only behavior, the description is largely sufficient. It lacks explicit information about response format or pagination behavior beyond defaults, but those are minor gaps given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, and the description repeats those same parameter descriptions verbatim without adding extra meaning. The only slight addition is the userType label, but it is already present in the schema, so description adds no net value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise verb and resource: 'List all users in the workspace (admin).' It clearly differentiates from sibling tools by scope ('workspace') and admin context, leaving no ambiguity about what the tool returns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage through its clear list operation, but it does not explicitly state when to prefer this over related tools like admin_get_user or admin_list_team_users. No alternative conditions or exclusions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_patch_team_usersContract Testing: Admin Patch Team UsersBInspect

Add or remove individual users from a team using JSON Patch (admin).

Toolset: Admin

Parameters:

  • teamId (string) required: UUID of the team

  • operations (array) required: JSON Patch operations to apply

ParametersJSON Schema
NameRequiredDescriptionDefault
teamIdYesUUID of the team
operationsYesJSON Patch operations to apply

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal a non-read-only mutation, and the description adds the JSON Patch mechanism and admin-level scope, which provides some behavioral context. However, it does not disclose consequences like partial application, duplicate removals, required permissions, or response behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short, front-loaded with the core action, and easy to scan. The 'Parameters' section duplicates schema content unnecessarily, but overall the structure is efficient and readable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The schema covers the parameter constraints well, and annotations cover the basic safety profile. Still, the description omits when to use this over closely related admin team-user tools and does not describe the response or edge-case behavior, leaving noticeable gaps for an admin mutation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents both teamId and operations. The description repeats the parameter names and descriptions without adding meaning beyond the schema, so it meets the baseline but does not go further.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Add or remove individual users from a team using JSON Patch.' The phrase 'individual users' plus 'JSON Patch' distinguishes it from sibling tools like admin_set_team_users or admin_remove_user_from_team, and the admin qualifier clarifies scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states what the tool does but gives no guidance on when to prefer it over alternatives such as contract-testing_admin_set_team_users or contract-testing_admin_remove_user_from_team. There is no explicit 'use this when' or exclusionary context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_remove_role_from_userContract Testing: Admin Remove Role from UserB
DestructiveIdempotent
Inspect

Remove a single role from a user (admin).

Toolset: Admin

Parameters:

  • userId (string) required: UUID of the user

  • roleId (string) required: UUID of the role

ParametersJSON Schema
NameRequiredDescriptionDefault
roleIdYesUUID of the role
userIdYesUUID of the user

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already carry the key safety information: destructiveHint=true and idempotentHint=true. The description adds only the 'Toolset: Admin' context and the granularity of removing a single role, but it does not describe permissions, reversibility, or behavior when the user does not currently have the role. It is aligned with the annotations and minimally adequate, but not richly transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening action is front-loaded and clear, which is good. However, the description includes a full Parameters section that duplicates the input schema verbatim, making the definition less economical than it should be. It remains short, but not every element earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter admin mutation, the annotated destructive/idempotent hints and fully covered schema provide a reasonable baseline. Still, since there is no output schema, a brief note about the expected outcome, return value, or failure mode would make the tool easier to invoke correctly. The description is minimally sufficient but leaves these gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both userId and roleId are fully documented in the input schema, which has 100% schema description coverage. The description's parameter list simply repeats the schema text and adds no new constraints, format details, or relationships between the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the explicit verb 'Remove' with a specific objectβ€”a single role from a userβ€”which clearly states the action. The word 'single' helps distinguish this from bulk role operations like admin_set_user_roles and from the opposite operation admin_add_role_to_user. The admin scope is also reinforced by title and toolset.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to choose this tool over admin_add_role_to_user or admin_set_user_roles, and no prerequisites are stated. The 'Toolset: Admin' label provides some context but does not explain conditions for use or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_remove_user_from_teamContract Testing: Admin Remove User from TeamB
DestructiveIdempotent
Inspect

Remove a specific user from a team (admin).

Toolset: Admin

Parameters:

  • teamId (string) required: UUID of the team

  • userId (string) required: UUID of the user

ParametersJSON Schema
NameRequiredDescriptionDefault
teamIdYesUUID of the team
userIdYesUUID of the user

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry destructiveHint=true and idempotentHint=true, and the description does not contradict them. It adds the auth context '(admin)' and clarifies the scope is team membership rather than user deletion, but it does not disclose further side effects, reversibility, or error behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The operation sentence is front-loaded and the description is compact. The parameter block and 'Toolset: Admin' line repeat information already available from the schema and title, so not every piece truly earns its place, but there is no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter mutation with full schema coverage and clear destructive/idempotent annotations, the description is minimally sufficient to identify required UUIDs and the operation. It lacks guidance on side effects, return value, or failure modes, but these are less critical for such a simple admin removal.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents teamId and userId as UUIDs. The description's Parameters section merely repeats the schema and adds no additional meaning, constraints, or relationships between the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence uses a specific verb and resource: 'Remove a specific user from a team', which clearly identifies the operation. It stops short of explicitly contrasting itself with sibling admin tools like delete_user or set_team_users, so the differentiation is implicit rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Aside from the 'Toolset: Admin' label, the description provides no when-to-use guidance and no exclusions. With many similar admin user/team siblings, an agent is not told when to prefer this tool over removing a role, deleting a user, or bulk-setting team members.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_reset_rolesContract Testing: Admin Reset RolesA
DestructiveIdempotent
Inspect

Reset all roles to their factory defaults (admin).

Toolset: Admin

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as destructive and non-read-only, and the description aligns by saying it resets all roles to factory defaults. It adds the global 'all roles' scope and the 'defaults' outcome, but it does not warn about the loss of custom role definitions or permissions beyond what the annotations imply. This is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is captured in one clear sentence, and the layout is easy to scan. The Toolset and Parameters lines add mild redundancy, but there is no meaningful bloat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With zero parameters, the calling contract is simple, and the destructive hint is already present in annotations. Still, the description does not explain what 'factory defaults' means in practice, which roles/permissions are affected, or what the response looks like, leaving some ambiguity about side effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters, so the schema fully covers the interface and leaves nothing to document. The description's 'Parameters: None' line is accurate and redundant, which is acceptable for a no-argument tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Reset'), a specific resource ('all roles'), and a precise outcome ('factory defaults'). The parenthetical 'admin' plus the toolset line distinguish this from granular role-admin operations like admin_set_user_roles or admin_create_role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given for when to use this reset tool versus the many sibling role-management tools, nor are exclusions or prerequisites described. The 'Toolset: Admin' line only labels the family and does not help an agent decide when a full reset is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_set_team_usersContract Testing: Admin Set Team UsersA
Idempotent
Inspect

Replace all members of a team (admin).

Toolset: Admin

Parameters:

  • teamId (string) required: UUID of the team

  • uuids (array) required: UUIDs of users to set as team members (replaces existing)

ParametersJSON Schema
NameRequiredDescriptionDefault
uuidsYesUUIDs of users to set as team members (replaces existing)
teamIdYesUUID of the team

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is read-write, idempotent, and not destructive, so the bar is lower. The description adds the key behavioral fact that existing team memberships are replaced, not appended, and labels the operation as admin-scoped. This is consistent with the annotations, including the idempotentHint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded, but the description then restates both parameters with text identical to the input schema. That duplication adds length without adding information; a shorter description that simply emphasized replacement semantics would be more disciplined.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter mutation with no output schema, the description and annotations together cover the essential context: what is replaced, the admin scope, the required parameters, and the idempotent behavior. It could state what an empty uuids array does, but that is a minor gap given the clear 'replaces existing' language.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter descriptions in the tool description are essentially identical to those in the input schema. The description adds no new semantic detail beyond what the schema already provides, so it sits at the schema-covered baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with the specific verb 'Replace' and clearly names the resource: 'all members of a team (admin).' This immediately distinguishes it from sibling team tools like add_role_to_user, remove_user_from_team, and patch_team_users, which handle partial membership changes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Replace all members' and the parenthetical 'replaces existing' give clear context for when to use this tool: when a caller wants to overwrite the full membership set. It does not explicitly name alternatives, but the wholesale-replacement semantics are clear enough to steer an agent away from partial-update siblings like admin_patch_team_users.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_set_user_rolesContract Testing: Admin Set User RolesA
Idempotent
Inspect

Replace all roles assigned to a user (admin).

Toolset: Admin

Parameters:

  • userId (string) required: UUID of the user

  • roles (array) required: Array of role UUIDs to assign

ParametersJSON Schema
NameRequiredDescriptionDefault
rolesYesArray of role UUIDs to assign
userIdYesUUID of the user

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that the operation replaces all roles, which is key behavioral context beyond the annotations. Annotations already indicate idempotent, non-read-only, and non-destructive, so the description adds moderate value but does not discuss side effects, permissions, or response behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The key behavioral information is front-loaded in the first sentence. The parameter list is redundant with the schema, which prevents a perfect score, but the overall description is short, scannable, and not bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter admin operation with clear schema coverage and annotations, the description provides enough information to understand the operation. It lacks return-value details, but no output schema exists and the behavioral scope is straightforward.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description repeats the parameter information almost verbatim. It adds no new semantic detail beyond what the schema already provides, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Replace all roles assigned to a user') and clearly identifies the resource (user roles) and scope (admin). It distinguishes this tool from siblings like admin_add_role_to_user and admin_remove_role_from_user by emphasizing 'replace all' semantics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Replace all roles assigned to a user' conveys the primary use case and differentiates it from incremental role modification tools. However, it does not explicitly mention when not to use it or name alternatives, so it falls short of fully explicit routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_update_roleContract Testing: Admin Update RoleB
Idempotent
Inspect

Update an existing role's name and permissions (admin).

Toolset: Admin

Parameters:

  • roleId (string) required: UUID of the role to update

  • name (string) required: Name of the role

  • permissions (array) required: Permissions granted by this role

  • description (string): Description of the role

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the role
roleIdYesUUID of the role to update
descriptionNoDescription of the role
permissionsYesPermissions granted by this role

TDQS

B3.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this is non-read-only, idempotent, and non-destructive. The description adds meaningful context by specifying that the role must already exist ('existing role') and labeling the operation as admin-level, which helps the agent understand authorization scope. There is no contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is efficient and front-loaded with the core purpose. However, the 'Toolset: Admin' line and the full parameter list duplicate information already present in the annotations and schema, adding unnecessary redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple and all four parameters are fully documented in the schema, with annotations covering mutation, idempotence, and safety. However, there is no output or response information, and no explanation of side effects or when to prefer this over related role operations, which leaves some contextual gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter block in the description merely restates the schema's definitions for roleId, name, permissions, and description. No additional semantic detail, value formats, or behavioral constraints are provided beyond what the structured schema already offers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action and resource: 'Update an existing role's name and permissions (admin).' The word 'existing' distinguishes it from role creation, and the resource and updatable fields are clear. It does not explicitly name sibling alternatives, so it falls just short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus admin_create_role, admin_delete_role, or role-assignment tools. The description only restates the operation and the toolset; there are no exclusions, prerequisites, or alternative routing cues.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_update_teamContract Testing: Admin Update TeamB
Idempotent
Inspect

Replace a team's configuration (admin).

Toolset: Admin

Parameters:

  • teamId (string) required: UUID of the team to update

  • name (string) required: Name of the team

  • administratorUuids (array): UUIDs of team administrators

  • environmentUuids (array): UUIDs of environments assigned to this team

  • pacticipantNames (array): Names of pacticipants assigned to this team

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the team
teamIdYesUUID of the team to update
environmentUuidsNoUUIDs of environments assigned to this team
pacticipantNamesNoNames of pacticipants assigned to this team
administratorUuidsNoUUIDs of team administrators

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already supply readOnlyHint=false, destructiveHint=false, and idempotentHint=true, so the safety profile is known. The description adds the meaningful word 'replace', implying the provided fields define the new whole configuration rather than a partial update. It does not clarify whether omitted optional arrays are cleared or preserved, which is important for a replace operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The one-sentence core description is front-loaded and compact. However, the Description field then repeats the entire parameter list that already exists in the input schema, which is redundant and does not earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is minimally sufficient for a team config replacement with schema-covered parameters and annotations that clarify idempotence and destructiveness. It is incomplete regarding behavior of omitted optional arrays and the expected response, and no output schema exists to fill that gap. Given the sibling admin_patch_team_users, a usage distinction would also improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already fully documented in the schema; the baseline applies. The description's parameter list adds no extra semantics beyond the schema, such as effects of omission or relationships between fields. It is not misleading but also not additive.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific action and resource: 'Replace a team's configuration (admin).' This clearly identifies the tool as an administrative full replacement of team settings, and the verb 'replace' helps distinguish it from read/list/create/delete team siblings. It does not explicitly distinguish from admin_patch_team_users or admin_set_team_users, but the phrase 'team's configuration' conveys a broader scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to choose this tool over alternatives. It does not mention that this is a full replacement rather than a partial/patch-style update, and it gives no preconditions such as requiring an existing team. The 'Admin' label adds little because it is already implied by the tool name and annotations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_admin_update_userContract Testing: Admin Update UserA
Idempotent
Inspect

Update a user's profile or active status (admin).

Toolset: Admin

Parameters:

  • userId (string) required: UUID of the user to update

  • active (boolean): Whether the user is active

  • email (string): New email address

  • firstName (string): First name

  • lastName (string): Last name

  • name (string): Display name

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoDisplay name
emailNoNew email address
activeNoWhether the user is active
userIdYesUUID of the user to update
lastNameNoLast name
firstNameNoFirst name

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already communicate that this is a write operation (readOnlyHint=false), idempotent (idempotentHint=true), and non-destructive (destructiveHint=false). The description's 'Update' phrasing is consistent with these annotations and adds the scope of what can be changed, but it adds no deeper behavioral context such as auth requirements, response behavior, or side effects. No contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded, clearly stating the operation. However, the entire 'Parameters' section redundantly repeats schema descriptions, lengthening the description without adding value. It is not bloated but is not as tight as it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a six-parameter mutation tool with no output schema, the description covers purpose and parameters but omits expected return/response behavior and does not clarify whether the update is partial or full. Annotations cover safety traits, but the description alone leaves meaningful gaps an agent might need to infer.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents every parameter. The parameter list in the description exactly duplicates the schema descriptions without adding extra meaning, such as explaining partial-update behavior, validations, or interactions between fields. This keeps it at the baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with 'Update a user's profile or active status (admin),' which clearly identifies the verb, resource, and scope. It distinguishes this from sibling admin tools like admin_create_user, admin_delete_user, and admin_set_user_roles without needing to open the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Toolset: Admin' label and the action language imply when this should be used, but the description provides no explicit guidance on when to prefer this tool over alternatives or when not to use it. It does not name sibling tools as exclusions, leaving the usage context to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_can_i_deployContract Testing: Can I DeployA
Read-onlyIdempotent
Inspect

Performs a comprehensive compatibility check to determine whether a specific version of a service (pacticipant) can be safely deployed into a given environment. It analyzes the complete contract matrix of consumer-provider relationships to confirm that all required integrations are verified and compatible.

Toolset: Can I Deploy

Parameters:

  • pacticipant (string) required: The name of the pacticipant (application/service) being evaluated for deployment

  • version (string) required: The version of the pacticipant that you want to check if it's safe to deploy

  • environment (string) required: The target environment where the pacticipant version will be deployed (e.g., 'production', 'staging', 'test')

ParametersJSON Schema
NameRequiredDescriptionDefault
versionYesThe version of the pacticipant that you want to check if it's safe to deploy
environmentYesThe target environment where the pacticipant version will be deployed (e.g., 'production', 'staging', 'test')
pacticipantYesThe name of the pacticipant (application/service) being evaluated for deployment

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds meaningful behavioral context by explaining that it analyzes the complete contract matrix of consumer-provider relationships and confirms all required integrations are verified. This goes beyond the annotations, though it does not disclose the exact result format or whether the tool returns a boolean, report, or matrix.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening paragraph is focused and communicates the core purpose efficiently. However, the parameters section duplicates what the input schema already documents, and the 'Toolset: Can I Deploy' line adds little value. It is not overly long, but it contains redundant material.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description explains the operation's intent and required inputs well, but there is no output schema and no description of what the tool returns. The meaning of 'safe to deploy' is only implied, and an agent might need to know whether the result is a boolean, a detailed compatibility report, or something else. The annotations cover read-only and idempotent behavior but not the result shape.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter descriptions in the tool description are essentially identical to those in the input schema. The description adds no new semantics for the parameters, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the operation: performing a compatibility check to determine whether a specific pacticipant version can be safely deployed to a given environment. It also specifies the resource (pacticipant) and the analysis of the contract matrix. However, it does not explicitly differentiate this from siblings like contract-testing_matrix or contract-testing_get_currently_deployed_versions, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the usage scenario: checking deploy safety before deployment. It gives the required inputs and mentions the target environment, so an agent can infer when to invoke it. However, it provides no explicit guidance about when not to use it, nor does it mention alternatives such as contract-testing_matrix or related verification tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_check_pactflow_ai_entitlementsContract Testing: Check PactFlow AI EntitlementsA
Read-onlyIdempotent
Inspect

Check your PactFlow AI entitlements and credit balance if you encounter 401 Unauthorized errors or permission/credit issues when using PactFlow AI features.

Toolset: PactFlow AI Status

Use Cases: 1. Diagnose 401 unauthorized errors when attempting to use PactFlow AI features 2. Check remaining AI credits when PactFlow AI operations are rejected due to insufficient credits 3. Verify account entitlements when users receive permission denied errors for PactFlow AI functionality 4. Troubleshoot PactFlow AI access issues by retrieving current entitlement status and credit balance 5. Provide detailed error context when PactFlow AI features are unavailable due to account limitations

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that the tool is read-only, idempotent, and non-destructive. The description adds useful behavioral context about diagnosing account limitations and retrieving entitlement/credit status, but it does not describe the exact response shape or how the underlying PactFlow API behaves. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loads the core purpose. However, the five enumerated use cases largely restate the same trigger conditions in slightly different wording, making the description more repetitive than necessary. It is not excessively long, but it could be tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, zero-parameter, read-only diagnostic tool, the description adequately explains what the tool does, when to invoke it, and what information it retrieves (entitlement status and credit balance). It does not specify the exact return format, but the intended use and outputs are clear enough for an agent to call it appropriately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the schema has full coverage, so there are no parameter semantics for the description to clarify. Per the rubric baseline for zero-parameter tools, a 4 is appropriate; the description makes no misleading parameter claims.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Check') and a precise resource ('PactFlow AI entitlements and credit balance'), and ties it to concrete troubleshooting scenarios like 401 Unauthorized errors and credit/permission issues. This clearly differentiates it from the many unrelated sibling tool names in the contract-testing family.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use the tool: after encountering 401 Unauthorized errors, permission denied errors, insufficient credits, or other PactFlow AI access issues. It provides clear contextual triggers and enumerates use cases, though it does not explicitly mention when not to use it or name alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_create_environmentContract Testing: Create EnvironmentAInspect

Create a new deployment environment in PactFlow.

Toolset: Environments and Deployments

Parameters:

  • name (string) required: Unique name for the environment (e.g. 'production', 'staging')

  • production (boolean) required: Whether this is a production environment

  • displayName (string): Human-readable display name

  • teamUuids (array): UUIDs of teams that own this environment

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesUnique name for the environment (e.g. 'production', 'staging')
teamUuidsNoUUIDs of teams that own this environment
productionYesWhether this is a production environment
displayNameNoHuman-readable display name

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag this as a mutating (readOnlyHint=false), non-idempotent operation, and the description's 'Create' aligns with that. However, the description adds no further behavioral context, such as what happens on duplicate names, permission requirements for production environments, or side effects on deployments.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loads the purpose in the first sentence, followed by a clear toolset heading and a short parameter list. The parameter list is redundant with the schema, but the overall structure is clean and does not contain fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a straightforward create operation with four simple parameters, no nested objects, no output schema, and clear annotations, the description is mostly sufficient. It could add notes about duplicate behavior or return values, but these are not essential for basic use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter list simply repeats what is already in the input schema, including the same examples. It does not add any additional meaning beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Create a new deployment environment in PactFlow.' The toolset heading 'Environments and Deployments' and the word 'Create' clearly differentiate this from sibling tools like contract-testing_update_environment, contract-testing_delete_environment, and contract-testing_list_environments.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this tool is for creating new environments, but it does not explicitly state when to use it versus alternatives such as update_environment or record_deployment. It also does not mention prerequisites like whether the environment name must be unique or whether teams must already exist.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_create_pacticipantContract Testing: Create PacticipantAInspect

Register a new application/service (pacticipant) in the workspace.

Toolset: Pacticipants

Parameters:

  • name (string) required: Name of the pacticipant (cannot be changed after creation)

  • displayName (string): Human-readable display name

  • mainBranch (string): Name of the main/trunk branch (e.g. 'main')

  • repositoryName (string): Repository name

  • repositoryNamespace (string): Repository namespace/organisation

  • repositoryUrl (string): URL of the source repository

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the pacticipant (cannot be changed after creation)
mainBranchNoName of the main/trunk branch (e.g. 'main')
displayNameNoHuman-readable display name
repositoryUrlNoURL of the source repository
repositoryNameNoRepository name
repositoryNamespaceNoRepository namespace/organisation

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare non-readOnly and non-idempotent behavior. The description adds the irreversible property that 'name cannot be changed after creation', which is valuable. It does not cover other behavioral aspects such as duplicate handling, permissions, or response shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded and the toolset categorization is helpful. However, the full parameter list duplicates the schema, making the description longer than necessary without adding informational value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a flat six-parameter creation tool with complete schema coverage and annotations, the description covers the essential registration scope and the immutable name constraint. It lacks explicit alternative usage and post-creation behavior, but these are non-critical gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema describes all six parameters with 100% coverage. The description's parameter list is a near-verbatim repeat of the schema and adds no extra meaning beyond what is already structured.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Register'), a resource ('pacticipant'), and the scope ('in the workspace'). It clearly distinguishes this creation tool from its update/patch/delete siblings through the word 'new'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Register a new application/service' gives a clear creation trigger and implies the tool is for new resources, not modifications. However, it does not explicitly name alternatives like contract-testing_update_pacticipant for existing resources.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_create_secretContract Testing: Create SecretAInspect

Create a new secret for use in webhook authentication.

Toolset: Secrets

Parameters:

  • name (string) required: Name of the secret

  • value (string) required: Value of the secret

  • description (string): Description of the secret

  • teamUuid (string): UUID of the owning team (cannot be changed after creation)

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the secret
valueYesValue of the secret
teamUuidNoUUID of the owning team (cannot be changed after creation)
descriptionNoDescription of the secret

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a mutating, non-idempotent, non-destructive operation, and the description does not contradict that. It adds the useful constraint that teamUuid cannot be changed after creation, though that is also present in the schema; it does not disclose response behavior or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core purpose. However, the parameter list duplicates the input schema, so a meaningful portion of the description does not add value beyond the structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple create operation, the description covers the main intent and required fields, but with no output schema it does not clarify what the response contains or whether the secret value is shown only once. Permission requirements and default team behavior are also not addressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description repeats the schema's parameter descriptions almost verbatim. It adds minimal extra meaning beyond restating that name and value are required and that teamUuid is immutable after creation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Create a new secret for use in webhook authentication.' It clearly identifies what the tool does and distinguishes it from sibling secret tools like list_secrets, get_secret, update_secret, and delete_secret.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The purpose statement implies the tool should be used when a new secret is needed for webhook authentication, and the 'Toolset: Secrets' label provides some context. However, it does not explicitly state when to prefer this over related tools or mention any prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_create_webhookContract Testing: Create WebhookBInspect

Create a new webhook to trigger on contract events.

Toolset: Webhooks

Parameters:

  • description (string) required: Human-readable description of the webhook

  • events (array) required: Events that trigger this webhook

  • request (object) required: HTTP request to send when triggered

  • consumer (object): Restrict to a specific consumer (omit for all)

  • provider (object): Restrict to a specific provider (omit for all)

  • enabled (boolean): Whether the webhook is enabled (default: true)

  • teamUuid (string): UUID of the owning team (null for global)

ParametersJSON Schema
NameRequiredDescriptionDefault
eventsYesEvents that trigger this webhook
enabledNoWhether the webhook is enabled
requestYesHTTP request to send when triggered
consumerNoRestrict to a specific consumer (omit for all)
providerNoRestrict to a specific provider (omit for all)
teamUuidNoUUID of the owning team (null for global)
descriptionYesHuman-readable description of the webhook

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as a write operation (readOnlyHint=false), non-idempotent, and non-destructive. The description adds a small behavioral detail by saying the webhook 'trigger[s] on contract events,' but it does not disclose permissions, rate limits, duplicate behavior, or what happens after creation. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is front-loaded and clear, and the parameter bullets are cleanly formatted. However, the bullet list duplicates information already present in the input schema, so it adds length without contributing new value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description plus schema provide enough to construct a valid create request, including required fields and nested request options. However, with no output schema, the description omits return values, response shape, and error/edge-case behavior, and it gives no practical context about webhook activation or scope.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all seven parameters. The description's parameter list largely mirrors schema text (e.g., 'Human-readable description of the webhook') and adds no new constraints, examples, or relationships beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb and resource: 'Create a new webhook to trigger on contract events.' This distinguishes it from sibling tools like update_webhook, delete_webhook, list_webhooks, and execute_webhook without needing to inspect schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to choose this tool over contract-testing_update_webhook, list_webhooks, execute_webhook, or test_execute_webhooks. There are no prerequisites, exclusions, or alternative conditions mentioned; the only usage signal is the verb 'create.'

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_delete_all_integrationsContract Testing: Delete All IntegrationsA
DestructiveIdempotent
Inspect

Delete ALL consumer-provider integrations in the workspace.

Toolset: Integrations and Network

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey destructiveHint=true and idempotentHint=true, so the bar is lower. The description adds the workspace-wide 'ALL' scope but does not mention irreversibility, potential impact on dependent data, or any confirmation safeguards, which would strengthen it further.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core behavior is front-loaded in a single clear sentence. The 'Toolset' and 'Parameters: None' lines are somewhat redundant but short, and the overall description is appropriately minimal for a 0-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, 0-parameter destructive operation, the description plus annotations cover the essential facts: it deletes all consumer-provider integrations in the workspace, and it is destructive. A note about irreversibility would be nice, but nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters and schema coverage is 100%, so the schema fully defines the call surface. The description explicitly notes 'Parameters: None,' which adds a small but useful confirmation for the agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Delete'), a precise resource ('ALL consumer-provider integrations'), and the scope ('in the workspace'). It clearly differentiates this from the sibling contract-testing_delete_integration, which targets a single integration.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to choose this bulk-delete tool versus alternatives like delete_integration or list_integrations. The word 'ALL' implies a bulk operation, but the description never states conditions, warnings, or alternative routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_delete_branchContract Testing: Delete BranchC
DestructiveIdempotent
Inspect

Delete a branch from a pacticipant.

Toolset: Branches and Versions

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • branchName (string) required: Name of the branch to delete

ParametersJSON Schema
NameRequiredDescriptionDefault
branchNameYesName of the branch to delete
pacticipantNameYesName of the pacticipant

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true, so the description adds no behavioral context beyond the bare action. It does not mention permanence, cascading effects on branch versions, required permissions, or any side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short, front-loaded with the key action, and includes a clean parameter list. No unnecessary prose is present, though the parameter section is redundant with the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter delete operation, the annotations and schema cover the basics. However, there is no mention of prerequisites, return behavior, or consequences beyond deletion, leaving the description merely adequate rather than complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already documented. The description merely repeats the same parameter names and descriptions without adding new semantic detail, which meets the baseline but adds no extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: 'Delete a branch from a pacticipant.' It is unambiguous and distinct from many sibling tools like list_branches or get_branch, though it does not explicitly contrast itself with those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool instead of related branch or delete operations. The 'Toolset: Branches and Versions' label provides weak context, but there are no stated alternatives, exclusions, or conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_delete_environmentContract Testing: Delete EnvironmentB
DestructiveIdempotent
Inspect

Delete an environment by UUID.

Toolset: Environments and Deployments

Parameters:

  • environmentId (string) required: UUID of the environment

ParametersJSON Schema
NameRequiredDescriptionDefault
environmentIdYesUUID of the environment

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, idempotentHint=true, and readOnlyHint=false, so the description adds no new behavioral information. It does not disclose whether deletion is permanent, whether associated deployments or releases are removed, or whether there are any side effects beyond removing the environment itself.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action, and the toolset label aids orientation. The parameter section duplicates schema information, but this is minor given the tool's simplicity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter deletion tool, the description is minimally sufficient: the agent knows what to delete and by what identifier. However, given that this is a destructive operation and there is no output schema, some mention of consequences or the irreversibility of the deletion would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the sole parameter environmentId is fully described in both the schema and the description as 'UUID of the environment'. The description adds no meaning beyond the schema, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Delete'), a concrete resource ('an environment'), and the identifying key ('by UUID'). This clearly distinguishes it from sibling tools like contract-testing_create_environment, contract-testing_update_environment, and contract-testing_get_environment without requiring schema inspection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives such as contract-testing_update_environment or contract-testing_list_environments. It also does not mention any prerequisites, safety checks, or conditions under which deletion is appropriate. Usage is only implied by the tool name and title.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_delete_integrationContract Testing: Delete IntegrationC
DestructiveIdempotent
Inspect

Delete a specific consumer-provider integration.

Toolset: Integrations and Network

Parameters:

  • providerName (string) required: Name of the provider

  • consumerName (string) required: Name of the consumer

ParametersJSON Schema
NameRequiredDescriptionDefault
consumerNameYesName of the consumer
providerNameYesName of the provider

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already carry destructiveHint=true and readOnlyHint=false, so an agent knows this is a mutation. The description adds no behavioral context beyond the word 'Delete'β€”it does not state whether deletion is permanent, whether it cascades to related contracts/webhooks, or what permissions are required. With no extra behavioral disclosure beyond annotations, this is a gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose sentence is front-loaded and concise, but the 'Parameters' section merely duplicates the schema's parameter names and descriptions. The 'Toolset' line adds marginal context. The overall size is small, yet some content is redundant rather than earning its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter delete operation with annotations covering destructiveness, the basics are present: what resource is deleted and which identifiers to pass. However, there is no output schema and no description of post-conditions, irreversibility, or relationship to delete_all_integrations, so an agent's understanding of consequences is incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema fully documents both providerName and consumerName. The description only repeats that information verbatim, adding no further meaning about formats, allowed values, or how the parameters identify the integration. Baseline 3 applies because the schema carries the load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence uses a specific verb ('Delete') and resource ('specific consumer-provider integration'), and the word 'specific' distinguishes it from the sibling contract-testing_delete_all_integrations. It does not, however, explicitly contrast with the other delete_* siblings (e.g., delete_environment, delete_webhook), so it stops short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance is given on when to use this tool versus alternatives. The word 'specific' implies it is for deleting one integration rather than all, but the description never names the alternative (delete_all_integrations) or states when not to use it. There are no context signals such as prerequisites or side conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_delete_pacticipantContract Testing: Delete PacticipantA
DestructiveIdempotent
Inspect

Delete a pacticipant and all its associated data.

Toolset: Pacticipants

Parameters:

  • pacticipantName (string) required: Name of the pacticipant to delete

ParametersJSON Schema
NameRequiredDescriptionDefault
pacticipantNameYesName of the pacticipant to delete

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal destructive behavior, and the description usefully adds that deletion cascades to 'all its associated data.' This is important behavioral context beyond the structured annotations, even though permissions or irreversibility are not explicitly mentioned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action and its destructive scope. The parameter list is clear, though it largely duplicates the schema and the 'Toolset: Pacticipants' line adds limited value, keeping it just below a top score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter destructive operation, the description plus annotations covers the essential facts: what is deleted, that the operation is destructive, and that it is idempotent. Lack of an output schema is not a major gap for a simple delete operation, though more detail on permissions or irreversibility would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description essentially repeats the schema's parameter documentation. It adds no new semantic meaning beyond what the input schema already provides, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Delete') and resource ('pacticipant') and clarifies scope with 'all its associated data.' This distinguishes it from other deletion tools like delete_branch or delete_webhook and makes the tool's exact purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the name and description: use this when the pacticipant should be permanently removed. However, there is no explicit guidance about when not to use it, such as preferring update_pacticipant or considering downstream impacts before deletion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_delete_secretContract Testing: Delete SecretC
DestructiveIdempotent
Inspect

Delete a secret by UUID.

Toolset: Secrets

Parameters:

  • secretId (string) required: UUID of the secret

ParametersJSON Schema
NameRequiredDescriptionDefault
secretIdYesUUID of the secret

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description simply restates 'Delete' and gives the parameter, which adds nothing beyond the annotations' destructiveHint=true and idempotentHint=true. It does not disclose whether deletion is permanent, whether it affects dependent resources like webhooks, or whether any special authorization is required. With annotations already covering the safety profile, the description contributes little behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action. The separate 'Toolset' and 'Parameters' sections are mildly redundant with the schema, but they do not create meaningful bloat. Overall, it is appropriately concise for a single-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter destructive operation, the description plus annotations cover the basic call correctly. However, it omits useful context such as irreversibility, behavior when the secret does not exist, or any relation to other secret operations. It is adequate but not rich.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes secretId as a required UUID with 100% coverage. The description repeats this information without adding further semantic meaning, so it meets the baseline but does not exceed what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the operation and resource: 'Delete a secret by UUID.' This is specific enough for an agent to understand what the tool does and to distinguish it from sibling operations like get_secret, create_secret, and update_secret, though it does not explicitly name those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus other secret-related or deletion tools. There are no stated prerequisites, exclusions, or references to alternatives such as contract-testing_delete_webhook or contract-testing_delete_environment. The operation is inferable but not explicitly contextualized.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_delete_webhookContract Testing: Delete WebhookB
DestructiveIdempotent
Inspect

Delete a webhook by UUID.

Toolset: Webhooks

Parameters:

  • webhookId (string) required: UUID of the webhook

ParametersJSON Schema
NameRequiredDescriptionDefault
webhookIdYesUUID of the webhook

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already mark this as destructive and idempotent, and the description merely repeats the delete action without adding consequences, irreversibility, or side effects. It adds no behavioral context beyond what the annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is a single clear sentence, and the parameter list is appropriately minimal. The 'Toolset: Webhooks' label adds light context, though the parameter block duplicates schema content unnecessarily.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive operation with no output schema, the description provides enough to invoke the tool correctly. It lacks minor context such as side effects or feedback behavior, but the annotations cover the destructive nature.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents webhookId as the UUID of the webhook. The description repeats this information without adding additional meaning, which matches the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Delete') and resource ('a webhook') with the identifier type ('by UUID'). This clearly differentiates it from sibling webhook tools like list_webhooks, get_webhook, create_webhook, and update_webhook.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives, such as when to choose delete over update or when a webhook should not be deleted. The usage context is only implied by the verb 'Delete', not explicitly stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_execute_webhookContract Testing: Execute WebhookBInspect

Trigger a test execution of a specific webhook.

Toolset: Webhooks

Parameters:

  • webhookId (string) required: UUID of the webhook

ParametersJSON Schema
NameRequiredDescriptionDefault
webhookIdYesUUID of the webhook

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already communicate that this is non-read-only, non-idempotent, and non-destructive. The description adds that this is a 'test execution,' implying a manual trigger with side effects but no data destruction. Still, it does not explain whether the webhook call is asynchronous, what external request is fired, or what a successful execution returns, and there is no output schema to fill that gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short, front-loaded with the primary action, and logically structured into action, toolset, and parameters. The Parameters block is redundant with the schema but harmless, and the overall length is appropriate for the tool's simplicity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool, the essential invocation data is fully specified, and annotations cover the safety profile. However, with no output schema, the description leaves open what happens when the webhook is executed, whether it is synchronous, and how the agent should verify success. These are meaningful gaps, but minor given the simplicity of the call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already fully documents webhookId with the same description ('UUID of the webhook'), and the tool description repeats it verbatim without adding format, source, or lookup guidance. Since schema description coverage is 100%, the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Trigger a test execution'), the resource ('a specific webhook'), and the toolset context ('Webhooks'). It is not a tautology and the singular scope is evident from the wording and parameter. However, it does not explicitly differentiate itself from the sibling tool contract-testing_test_execute_webhooks, so it falls short of full distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or when-not-to-use guidance. The description does not mention alternatives, prerequisites, or when this should be chosen over other webhook-related tools such as contract-testing_list_webhooks or contract-testing_test_execute_webhooks. The 'Toolset: Webhooks' label provides context but no selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_generate_pact_testsContract Testing: Generate Pact TestsA
Read-onlyIdempotent
Inspect

Generate Pact tests using PactFlow AI. You can provide one or more of the following input types: (1) request/response pairs for specific interactions, (2) code files to analyze and extract interactions from, and/or (3) OpenAPI document to generate tests for specific endpoints. When providing an OpenAPI document, a matcher is required to specify which endpoints to generate tests for.

Toolset: Generate Pact Tests

Parameters:

  • language (enum): Target language for the generated Pact tests. If not provided, will be inferred from other inputs.

  • requestResponse (object): Direct request/response pair for a specific interaction. Use this when you have concrete examples of API requests and responses

  • code (array): Collection of source code files to analyze and extract API interactions from. Include client code, data models, existing tests, or any code that makes API calls

  • openapi (any)

  • additionalInstructions (string): Optional free-form instructions to guide the generation process (e.g., 'Focus on error scenarios', 'Include authentication headers', 'Use specific test framework patterns')

  • testTemplate (object): Optional test template to use as a basis for generation. Helps ensure generated tests follow your specific patterns, frameworks, and coding standards

ParametersJSON Schema
NameRequiredDescriptionDefault
codeNoCollection of source code files to analyze and extract API interactions from. Include client code, data models, existing tests, or any code that makes API calls
openapiNoIf provided, the OpenAPI document which describes the API being tested and is accompanied by a matcher which will be used to identify the interactions in the OpenAPI document which are relevant to the Pact refinement process.
languageNoTarget language for the generated Pact tests. If not provided, will be inferred from other inputs.
testTemplateNoOptional test template to use as a basis for generation. Helps ensure generated tests follow your specific patterns, frameworks, and coding standards
requestResponseNoDirect request/response pair for a specific interaction. Use this when you have concrete examples of API requests and responses
additionalInstructionsNoOptional free-form instructions to guide the generation process (e.g., 'Focus on error scenarios', 'Include authentication headers', 'Use specific test framework patterns')

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds useful input-related context: language inference ('If not provided, will be inferred from other inputs') and the OpenAPI matcher requirement. However, it does not disclose what generation returns or whether any state is persisted, leaving behavioral transparency partial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening paragraph is concise and useful, but the description then includes a full 'Parameters:' section that duplicates the schema almost verbatim. The 'Toolset: Generate Pact Tests' heading is redundant. Much of the added length does not earn its place in an MCP description that is meant to complement, not repeat, structured schema fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description handles the complex input surface well: three input modes, the OpenAPI matcher requirement, and the distinction between inline and remote OpenAPI documents. However, with no output schema, the description does not state what a successful generation returns or how an agent should consume the result. This is a clear gap for such a complex tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description mostly restates parameter meanings that already appear in the schema (e.g., additionalInstructions, testTemplate, code). It adds little semantic value beyond mapping the three top-level input modes to parameter names. The matcher requirement is also already documented in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Generate Pact tests using PactFlow AI.' It enumerates the three accepted input types, making the tool's purpose unmistakable. It is also clearly distinguished from the adjacent sibling contract-testing_review_pact_tests by the generate action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains that one or more input types may be supplied and states a hard prerequisite: 'When providing an OpenAPI document, a matcher is required.' It also gives scenario guidance for requestResponse ('Use this when you have concrete examples of API requests and responses'). It does not explicitly contrast with alternatives like review_pact_tests, but the intended usage context is largely clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_audit_logContract Testing: Get Audit LogA
Read-onlyIdempotent
Inspect

Retrieve the audit log of events in the workspace.

Toolset: Audit

Parameters:

  • since (string): Only include events at or after this ISO 8601 timestamp

  • userUuid (string): Filter events by PactFlow user UUID

  • type (string): Filter events by type (e.g. 'pact_publication')

  • sort (string): Sort order: '+timestamp' (asc, default) or '-timestamp' (desc)

  • from (string): Start result set from this audit event UUID (keyset pagination)

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (max 100) (default: 5)

Use Cases: 1. Review recent changes to pacticipants, webhooks, or secrets 2. Investigate who published a specific pact or verification 3. Filter events by user or event type for compliance reporting 4. Track deployment recording activity across environments

ParametersJSON Schema
NameRequiredDescriptionDefault
fromNoStart result set from this audit event UUID (keyset pagination)
sortNoSort order: '+timestamp' (asc, default) or '-timestamp' (desc)
typeNoFilter events by type (e.g. 'pact_publication')
sinceNoOnly include events at or after this ISO 8601 timestamp
pageSizeNoResults per page (max 100)
userUuidNoFilter events by PactFlow user UUID
pageNumberNoPage number

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, lowering the burden. The description adds useful behavioral context: workspace scope, default sort order, keyset pagination mechanism, and default page size. It does not mention rate limits or auth, but the annotation coverage makes this gap minor.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a front-loaded purpose, a clear Toolset label, and an organized use-case list. However, the Parameters block largely duplicates the input schema, adding unnecessary length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

It covers purpose, workspace scope, all seven parameters, defaults, and realistic use cases. The only noticeable gap is the absence of any description of the audit event record shape, which matters more because there is no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all seven parameters. The description largely restates those schema meanings and adds only an explicit 'keyset pagination' label and use-case hints, so it stays at the schema baseline without significant added value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The one-sentence description 'Retrieve the audit log of events in the workspace' uses a specific verb and resource, and the use-case list clarifies the tool's role. Among the many siblings, no other tool targets audit logs, so it is inherently differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section lists four concrete scenarios (reviewing changes, investigating pact publications, compliance filtering, tracking deployments), telling an agent exactly when to call it. No alternative audit-log tool exists among siblings, so no when-not-to-use instruction is needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_consumer_by_consumer_versionContract Testing: Get BDCT Consumer by Consumer VersionB
Read-onlyIdempotent
Inspect

Fetch the consumer Pact contract for a specific consumer-provider version pair in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • consumerName (string) required: Name of the consumer

  • consumerVersionNumber (string) required: Consumer version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
consumerNameYesName of the consumer
providerNameYesName of the provider
consumerVersionNumberYesConsumer version number
providerVersionNumberYesProvider version number

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds no behavioral context beyond identifying the resource, such as pagination behavior, return format, or what happens if no matching contract exists. It does not contradict the annotations, so the score stays at a baseline 3.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded with the core purpose. However, the 'Parameters' section duplicates information already present in the input schema, adding length without value. A shorter description relying on the schema would have been more efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read operation with all parameters documented in the schema, the description provides enough to invoke the tool. Yet it omits details about the response shape (no output schema) and how pageNumber/pageSize affect results, and it doesn't clarify the difference from the many similar sibling tools. This leaves the agent with some ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameters section in the description simply repeats the schema's property descriptions. It adds no extra meaning about formats, constraints, or how the parameters interact. Baseline 3 is appropriate because the schema already documents all parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: 'Fetch the consumer Pact contract for a specific consumer-provider version pair in Bi-Directional Contract Testing.' This clearly identifies what the tool returns and the precise selection criteria. However, it does not explicitly distinguish itself from closely named siblings like get_bdct_consumer_contracts or get_bdct_provider_by_consumer_version, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions 'Toolset: Bi-Directional Contract Testing' but provides no guidance on when to choose this tool over the many related get_bdct_* siblings. The phrase 'specific consumer-provider version pair' implies when it is appropriate, but no explicit exclusions or alternative tool references are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_consumer_contractsContract Testing: Get BDCT Consumer ContractsB
Read-onlyIdempotent
Inspect

Fetch all consumer Pact contracts relevant to a given provider version in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
providerNameYesName of the provider
providerVersionNumberYesProvider version number

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, and the description's 'Fetch' is consistent with those. The description adds little beyond thatβ€”no notes on pagination behavior, response content, or provider-version semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main sentence is appropriately terse and front-loaded. The repeated Parameters section is redundant with the provided input schema and adds length without value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only fetch with only four flat parameters, the description plus schema is sufficient to determine required inputs and defaults. It doesn't describe return structure, but the purpose statement makes the expected result clear enough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all four parameters are already documented. The description's Parameters block simply repeats the schema text and adds no additional semantic meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a concrete operation ('Fetch all consumer Pact contracts') and scopes it to a provider version in Bi-Directional Contract Testing. It is clear but does not explicitly contrast with sibling tools such as get_bdct_consumer_contract_verification_results, relying on the word 'contracts' to differentiate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when BDCT consumer contracts for a provider version are needed, and the Toolset label adds context. It provides no exclusion criteria or explicit alternatives among the many sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_consumer_contract_verification_resultsContract Testing: Get BDCT Consumer Contract Verification ResultsB
Read-onlyIdempotent
Inspect

Fetch the consumer contract verification results for a given provider version in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
providerNameYesName of the provider
providerVersionNumberYesProvider version number

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, and the description's 'Fetch' is consistent with those. The description adds the scoping detail of a provider version but does not disclose pagination behavior, response shape, or what happens when no results exist. With annotations covering the safety profile, this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short, front-loaded with the purpose, and then lists parameters in a readable format. The parameter list is somewhat redundant with the schema, but the overall length is appropriate and every major piece is covered without excessive detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only operation with full parameter schema coverage and annotations, the description covers the basics needed to call the tool. However, there is no output schema and the description does not clarify what the returned verification results contain or how this tool differs from the many related BDCT verification-result siblings, leaving some ambiguity for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description repeats the parameter names, requiredness, and defaults already present in the input schema. It adds no additional semantic meaning beyond what the schema provides, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb ('Fetch') and a specific resource ('consumer contract verification results') scoped to a given provider version. It distinguishes itself from sibling tools through the resource phrase, but does not explicitly contrast with similarly named tools like get_bdct_provider_contract_verification_results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context β€” this is for retrieving consumer contract verification results in Bi-Directional Contract Testing for a specific provider version. However, it gives no explicit guidance on when to choose this tool over sibling verification-result tools or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_consumer_pact_test_results_by_consumerContract Testing: Get BDCT Consumer Pact Test Results by ConsumerB
Read-onlyIdempotent
Inspect

Fetch the consumer contract verification results for a specific consumer-provider version pair in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • consumerName (string) required: Name of the consumer

  • consumerVersionNumber (string) required: Consumer version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
consumerNameYesName of the consumer
providerNameYesName of the provider
consumerVersionNumberYesConsumer version number
providerVersionNumberYesProvider version number

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description's 'Fetch' aligns with a non-mutating operation. The description adds no further behavioral detail such as pagination behavior, result shape, or empty-result semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is clear and front-loaded, but the description then redundantly repeats the full parameter schema. It is not badly organized, but the duplicated parameter list adds noise without providing new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only fetch tool with six simple parameters, required inputs are clear and annotations cover the safety profile. However, there is no output schema and the description does not describe what the returned verification results contain or how pagination affects them, leaving a notable gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline applies. The description's parameter list exactly duplicates the schema field names and descriptions and adds no additional semantics such as value formats, parameter relationships, or pagination interpretation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action (fetch verification results) and identifies the exact scope: a consumer-provider version pair in Bi-Directional Contract Testing. It is generally clear, though it does not explicitly distinguish itself from closely named siblings such as get_bdct_consumer_contract_verification_results or get_bdct_provider_check_results_by_consumer.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool over alternatives. The only context is the 'Bi-Directional Contract Testing' toolset label, with no exclusions, alternatives, or prerequisites mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_cross-contract_verification_resultsContract Testing: Get BDCT Cross-Contract Verification ResultsC
Read-onlyIdempotent
Inspect

Fetch the cross-contract verification results for a given provider version in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
providerNameYesName of the provider
providerVersionNumberYesProvider version number

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint and idempotentHint, and the description adds no behavioral context beyond that. It does not explain pagination behavior, result shape, or any caveats about how the provider version maps to results, so the description adds little beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is clear and front-loaded, but the subsequent parameter list duplicates the input schema almost verbatim. The description is not overly long, yet it contains redundant content that does not add value beyond the structured schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only fetch with fully documented parameters and safety annotations, the description is minimally adequate. However, with no output schema and many similar sibling tools, it would benefit from stating what the results contain or how this tool differs from other verification-result fetchers.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description simply repeats the parameter names, defaults, and short descriptions already present in the input schema. It adds no additional meaning or usage nuance, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Fetch'), a resource ('cross-contract verification results'), and a scope ('for a given provider version') in Bi-Directional Contract Testing. It is clear and matches the tool name, though it does not explicitly distinguish itself from sibling verification-result tools like get_bdct_provider_contract_verification_results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus the many similar BDCT verification-result tools in the sibling list. It only labels the toolset and provides parameters, with no mention of alternatives, exclusions, or selection criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_provider_by_consumer_versionContract Testing: Get BDCT Provider by Consumer VersionB
Read-onlyIdempotent
Inspect

Fetch the provider OpenAPI contract for a specific consumer-provider version pair in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • consumerName (string) required: Name of the consumer

  • consumerVersionNumber (string) required: Consumer version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
consumerNameYesName of the consumer
providerNameYesName of the provider
consumerVersionNumberYesConsumer version number
providerVersionNumberYesProvider version number

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this is read-only, idempotent, and non-destructive, so the description only needs to add artifact-level behavior. It does say the result is a provider OpenAPI contract scoped to a version pair, which is useful. However, it does not mention pagination behavior or response shape despite pageNumber/pageSize parameters, so it only moderately supplements the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is sharp and front-loaded. But the following Parameters section duplicates the input schema nearly word-for-word, adding length without new information; this redundancy depresses the score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The four required parameters are fully captured and the core purpose is clear enough for a basic call. With no output schema, though, the description should say more about the return structure and pagination behavior; right now the singular 'provider OpenAPI contract' conflicts with pagination parameters, and the many sibling tools are not disambiguated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents every parameter. The description's parameter bullets repeat the schema verbatim without adding semantics such as value formats, source, or interactions. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a concrete verb and resource: 'Fetch the provider OpenAPI contract for a specific consumer-provider version pair in Bi-Directional Contract Testing.' This tells an agent what the tool returns and at what scope, and aligns with the tool name. It does not explicitly compare with the similar sibling get_bdct_provider_contract, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description contains no guidance about when to choose this tool over alternatives; there is no 'use this when...' or 'prefer X instead' statement. Only the toolset label and name signal its niche, which is not enough to route an agent among the many BDCT sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_provider_check_results_by_consumerContract Testing: Get BDCT Provider Check Results by ConsumerB
Read-onlyIdempotent
Inspect

Fetch the provider contract self-verification results for a specific consumer-provider version pair in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • consumerName (string) required: Name of the consumer

  • consumerVersionNumber (string) required: Consumer version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
consumerNameYesName of the consumer
providerNameYesName of the provider
consumerVersionNumberYesConsumer version number
providerVersionNumberYesProvider version number

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful context by specifying 'self-verification results' and the consumer-provider pairing, but it does not go beyond that to explain response shape, pagination behavior, or any other runtime characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and reasonably concise: it leads with a clear purpose, identifies the toolset, and lists parameters with defaults. There is no fluff, though the parameter list slightly duplicates the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The invocation requirements are covered because all parameters are listed with defaults and the schema is complete. However, there is no explanation of the return format, and given the large number of similar sibling tools, more contextual detail about this specific result type would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents all parameters. The description restates the parameter names and defaults but adds no meaning beyond what the schema provides, which meets the baseline expectation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Fetch') and a specific resource ('provider contract self-verification results for a specific consumer-provider version pair'). It clearly identifies the subject matter, though it does not explicitly differentiate itself from closely related siblings like get_bdct_provider_contract_verification_results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus the many similar contract-testing tools in the sibling list. It does not state conditions, exclusions, or alternatives, leaving the agent to infer the appropriate context from the name and parameter list alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_provider_contractContract Testing: Get BDCT Provider ContractA
Read-onlyIdempotent
Inspect

Fetch the provider OpenAPI contract for a given provider version in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
providerNameYesName of the provider
providerVersionNumberYesProvider version number

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description's 'Fetch' is consistent with those. The description adds the fact that the returned artifact is an OpenAPI contract scoped to a provider version, but it does not disclose pagination semantics, response format details, or edge cases. This is adequate but not rich behavioral context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The action sentence is front-loaded and immediately informative, which is good. However, the 'Toolset:' line and the full parameter list duplicate what is already in the input schema and tool name, so part of the description does not earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Since there is no output schema, the description usefully states the return artifact (provider OpenAPI contract for a provider version), which is the key missing context. The required parameters are clear, the annotations cover the read-only/idempotent nature of the call, and no complex nested objects or enum choices exist. Pagination is only minimally contextualized, but the schema defaults are self-explanatory.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters, their types, required status, and defaults. The description's parameter block only restates that information and adds no new meaning such as value constraints, formats, or inter-parameter relationships.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific verb ('Fetch'), a precise resource ('provider OpenAPI contract'), and the scoping condition ('for a given provider version'), making the tool unambiguous. This also distinguishes it from sibling tools like contract-testing_get_bdct_provider_contract_verification_results, which target verification results rather than the contract itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use this tool: when an agent needs a provider OpenAPI contract for a specific provider version in Bi-Directional Contract Testing. It provides clear context via required parameters and toolset labeling, though it does not explicitly name alternatives or state exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_provider_contract_verification_resultsContract Testing: Get BDCT Provider Contract Verification ResultsB
Read-onlyIdempotent
Inspect

Fetch the self-verification results for a provider contract version in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
providerNameYesName of the provider
providerVersionNumberYesProvider version number

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is established. The description adds useful scoping context by specifying 'self-verification results for a provider contract version,' which helps distinguish this from consumer or cross-contract verification outcomes. It does not disclose further behavioral details like authentication requirements, rate limits, or return envelope, but the annotations lower the bar and the description is not contradictory.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is reasonably compact, opens with the core action and resource, and then lists parameters in a scannable format. The parameter section is largely redundant with the input schema, which is minor waste, but the overall structure is clear and front-loaded. No unnecessary prose or tangential context appears.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The core required inputs and pagination controls are present, and annotations cover the operational safety profile. However, without an output schema, the description is vague about what the 'verification results' actually contain, how they are structured, or what a successful/unsuccessful self-verification looks like. Given the dense ecosystem of similar contract-testing tools, a bit more domain context would improve completeness, but the description is minimally viable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema fully documents all four parameters and their defaults. The description repeats the parameter list almost verbatim without adding deeper meaning such as value formats, allowed identifiers, or how pagination affects results. This sits at the baseline for high schema coverage, meeting the minimum viability but adding no extra semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Fetch') and a specific resource ('self-verification results for a provider contract version'), clearly identifying the object of the operation. However, it does not explicitly distinguish itself from closely related siblings like get_bdct_consumer_contract_verification_results or get_bdct_cross-contract_verification_results, relying on the 'provider' and 'self-verification' wording to imply the distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus the many similar contract-testing verification tools. There are no explicit alternatives, exclusions, or conditions that would help an agent decide between this and get_bdct_consumer_contract_verification_results or get_bdct_cross-contract_verification_results. The usage context is only implied by the tool name and resource wording.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_bdct_x-contract_test_results_by_consumerContract Testing: Get BDCT X-Contract Test Results by ConsumerB
Read-onlyIdempotent
Inspect

Fetch the cross-contract verification results for a specific consumer-provider version pair in Bi-Directional Contract Testing.

Toolset: Bi-Directional Contract Testing

Parameters:

  • providerName (string) required: Name of the provider

  • providerVersionNumber (string) required: Provider version number

  • consumerName (string) required: Name of the consumer

  • consumerVersionNumber (string) required: Consumer version number

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
consumerNameYesName of the consumer
providerNameYesName of the provider
consumerVersionNumberYesConsumer version number
providerVersionNumberYesProvider version number

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description's word 'Fetch' is consistent with those. The description adds the scoping constraint 'specific consumer-provider version pair' but does not disclose result shape, pagination behavior, or potential error conditions, so it contributes only modest context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence is concise and informative. However, the 'Toolset' line is redundant with the title, and the explicit Parameters block duplicates the input schema almost verbatim, adding bulk without new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only fetch tool with full parameter schema coverage and safety annotations, the description is minimally viable: it specifies the resource and all required identifiers. Yet there is no output schema and no description of what the returned verification results contain or how pagination behaves, which leaves some ambiguity when an agent interprets the response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all six parameters with names, types, defaults, and descriptions. The description's Parameters section merely repeats that information and does not add semantic value such as value formats, parameter relationships, or interpretation of the pagination fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific verb ('Fetch'), a clear resource ('cross-contract verification results'), and the scope ('specific consumer-provider version pair'). It is not vague or tautological, but it does not differentiate this tool from near-siblings like contract-testing_get_bdct_cross-contract_verification_results or the other 'by consumer' BDCT getters, so it does not earn a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to choose this tool over the many similar BDCT result-fetching siblings. No alternatives are named, no exclusions are stated, and no 'use this when...' context is provided beyond the fact that a consumer and provider version pair is required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_branchContract Testing: Get BranchA
Read-onlyIdempotent
Inspect

Retrieve details for a specific branch of a pacticipant.

Toolset: Branches and Versions

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • branchName (string) required: Name of the branch

ParametersJSON Schema
NameRequiredDescriptionDefault
branchNameYesName of the branch
pacticipantNameYesName of the pacticipant

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe read operation. The description's 'Retrieve' matches those annotations but adds no additional behavioral context beyond what is already encoded, such as how branch details are returned or whether any special conditions apply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded with the tool's purpose. However, the parameter list duplicates the input schema exactly, adding bulk without new information. The 'Toolset' line provides useful grouping but is the only non-redundant extra content beyond the first sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only lookup with two required scalar parameters and no nested objects, this description is adequate. The annotations cover safety and idempotency, and the description names the target resource clearly. The word 'details' is somewhat vague, but the lack of an output schema is offset by the low complexity of the operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description simply restates the same parameter names and descriptions ('Name of the pacticipant', 'Name of the branch'). It does not add extra semantic detail such as allowed formats, examples, or the relationship between the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Retrieve details') and resource ('a specific branch of a pacticipant'). It distinguishes this tool from related siblings such as contract-testing_list_branches, contract-testing_get_branch_versions, and contract-testing_delete_branch by emphasizing the singular, read-only lookup intent.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'specific branch of a pacticipant' implies this is the right choice when the agent has a known branch name and pacticipant name, but it gives no explicit when-to-use guidance or mention of alternatives like list_branches or get_branch_versions. The 'Toolset: Branches and Versions' label adds mild context but does not resolve routing choices.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_branch_versionsContract Testing: Get Branch VersionsA
Read-onlyIdempotent
Inspect

Retrieve all versions published from a specific branch of a pacticipant.

Toolset: Branches and Versions

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • branchName (string) required: Name of the branch

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
branchNameYesName of the branch
pageNumberNoPage number
pacticipantNameYesName of the pacticipant

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description only adds the branch scope and restates pagination parameters; it does not disclose iteration behavior, response format, or limits beyond what the schema already provides.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose sentence is front-loaded and clear, but the toolset label and the parameter block are redundant with the input schema. The definition is compact yet contains repeated information that does not earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Required parameters and defaults are fully covered, and the annotations carry the safety information. However, because there is no output schema and the description says 'all versions' while pageSize defaults to 5, an agent is not explicitly told that multiple pages may need to be requested.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all four parameters. The description's parameter list duplicates the schema without adding semantic detail such as accepted formats, constraints, or richer meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence uses a specific verb ('Retrieve') and resource ('all versions published from a specific branch of a pacticipant'), making the operation distinct from sibling tools such as contract-testing_get_branch or contract-testing_list_pacticipant_versions. The branch scope is precise and immediately understandable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool (when branch-filtered versions are needed) but gives no explicit when-to-use/when-not-to-use guidance or named alternatives. The 'Toolset: Branches and Versions' label provides context but does not help an agent choose between closely related sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_currently_deployed_versionsContract Testing: Get Currently Deployed VersionsB
Read-onlyIdempotent
Inspect

Retrieve all versions currently deployed to a given environment.

Toolset: Environments and Deployments

Parameters:

  • environmentId (string) required: UUID of the environment

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
environmentIdYesUUID of the environment

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the 'currently deployed' state aspect but does not explain pagination behavior or what the returned payload contains; no contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is short and front-loaded, opening with the purpose statement. The toolset label and parameter block are compact, though the parameter details largely duplicate the input schema, preventing a top score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple paginated read operation, the description plus rich annotations is mostly sufficient, but it does not clarify whether pagination means 'all versions' are returned at once, and it does not distinguish this from closely named sibling tools. There is no output schema, so a note on the expected response would strengthen completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents environmentId, pageNumber, and pageSize. The description merely repeats those parameter descriptions and adds no meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve') and resource ('all versions currently deployed to a given environment') and clearly identifies the required environment scope. It does not explicitly contrast itself with sibling tools such as get_deployed_versions_for_version or get_currently_supported_versions, so it falls short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to choose this tool over its siblings, no exclusions, and no mention of alternatives like get_deployed_versions_for_version or list_environments. The only implied context is that an environmentId is needed, which is already in the schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_currently_supported_versionsContract Testing: Get Currently Supported VersionsB
Read-onlyIdempotent
Inspect

Retrieve all versions currently released and supported in a given environment.

Toolset: Environments and Deployments

Parameters:

  • environmentId (string) required: UUID of the environment

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
environmentIdYesUUID of the environment

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully carried by structured data and the description need not restate it. The description adds modest scoping context ('released and supported' vs. merely existing) but nothing about error behavior, whether the environment must exist, or pagination semantics. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose sentence is front-loaded and the one-line Toolset label adds useful grouping without bloating the description. The parameter list is somewhat redundant with the input schema, but the whole description remains compact and well-ordered.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only query with a fully documented schema and annotations carrying the safety profile, everything needed to invoke it correctly is present: the required environmentId and pagination defaults. The notable gap is that the nearly identically named sibling get_currently_deployed_versions is never addressed, so an agent gets no help distinguishing 'supported' from 'deployed' versions. With no output schema, one clarifying sentence about the result or the sibling distinction would make this complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The inline parameter list exactly duplicates the schema's descriptions ('UUID of the environment', 'Page number', 'Results per page', with defaults) and adds no meaning beyond the structured field documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve'), names the resource ('all versions currently released and supported'), and scopes it to 'a given environment.' The phrase 'released and supported' implicitly separates this from the near-twin sibling contract-testing_get_currently_deployed_versions, though that sibling is never named explicitly, which keeps this just short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Toolset: Environments and Deployments' label and the 'in a given environment' scope imply this is the tool for querying what is supported in an environment, which gives some usage context. However, there is no explicit when-to-use guidance, no exclusions, and no mention of closely related alternatives like contract-testing_get_currently_deployed_versions, contract-testing_get_environment, or contract-testing_list_environments.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_current_userContract Testing: Get Current UserA
Read-onlyIdempotent
Inspect

Retrieve the profile of the currently authenticated user.

Toolset: User, Tokens and Preferences

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already define the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), so the burden on the description is lower. It adds useful context that the profile belongs to the authenticated user, which clarifies the operation's scope. It doesn't contradict annotations and adds a small but meaningful behavioral detail beyond them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is one concise, front-loaded sentence that states the purpose. The 'Toolset' line adds organizational context, and 'Parameters: None' is redundant with the empty schema but not distracting. It is appropriately sized with no wasted wording, though not as tight as possible.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter read-only tool with annotations covering safety, the description is essentially complete. The absence of an output schema is acceptable because the description names the resource ('profile'), and the operation is a simple retrieval. Nothing needed to invoke the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the schema already fully covers everything (100% coverage). The description adds no parameter information beyond stating 'Parameters: None,' which is redundant but harmless. With no parameters, the baseline of 4 is appropriate β€” nothing is missing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve') and resource ('profile of the currently authenticated user'), clearly distinguishing it from siblings like contract-testing_admin_get_user (admin-scoped) and contract-testing_get_user_preferences (preferences). An agent can infer exactly what this tool does from the first sentence.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by stating 'currently authenticated user' β€” it is obvious this is for retrieving the caller's own profile. However, it does not explicitly mention when to choose this over alternatives like get_user_preferences or get_system_preferences, nor does it provide any exclusions. Context is clear but not explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_deployed_versions_for_versionContract Testing: Get Deployed Versions for VersionB
Read-onlyIdempotent
Inspect

Retrieve deployment records for a specific pacticipant version in a specific environment.

Toolset: Environments and Deployments

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • versionNumber (string) required: Version number

  • environmentId (string) required: UUID of the environment

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
environmentIdYesUUID of the environment
versionNumberYesVersion number
pacticipantNameYesName of the pacticipant

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description doesn't need to restate that this is a safe read operation. It does add that the result is deployment records for a specific version and environment, but it adds no behavioral context such as pagination handling, response format, or ordering. The description is consistent with the annotations and does not contradict them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose sentence is front-loaded and clear, and the parameter list is formatted cleanly. The list is somewhat redundant given the input schema, but the description remains short and readable. No extraneous marketing language or filler is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only retrieval tool with fully documented parameters and safety annotations, the description is mostly adequate. However, without an output schema, it would have been helpful to clarify that results are paginated and whether 'deployment records' means historical records as opposed to currently deployed versions, especially given the sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter, including defaults for pageNumber and pageSize. The description simply repeats the parameter names, types, requiredness, and defaults already in the schema, adding no new semantic meaning beyond what structured data provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve'), a clear resource ('deployment records'), and narrows the scope to 'a specific pacticipant version in a specific environment.' This distinguishes it from related tools like get_currently_deployed_versions, though it doesn't explicitly contrast itself with get_released_versions_for_version or mention alternates by name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to choose this tool over sibling tools such as get_currently_deployed_versions or get_released_versions_for_version. The 'Toolset: Environments and Deployments' line is categorization, not usage direction. An agent would have to infer the appropriate context from the tool name and first sentence alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_environmentContract Testing: Get EnvironmentB
Read-onlyIdempotent
Inspect

Retrieve details for a specific environment by UUID.

Toolset: Environments and Deployments

Parameters:

  • environmentId (string) required: UUID of the environment

ParametersJSON Schema
NameRequiredDescriptionDefault
environmentIdYesUUID of the environment

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description's 'Retrieve' is fully consistent with those. The description adds no extra behavioral context such as response shape, error behavior, or prerequisites, but it also introduces no contradiction or hidden surprise.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core sentence is short, direct, and front-loaded, immediately stating what the tool does. The Parameters section is redundant with the schema, but the overall description remains compact and easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read-only getter with no output schema, the description is mostly adequate, but 'details' is vague and does not indicate what fields the response will contain. With no output schema to fill that gap, a brief enumeration of the returned properties would make the tool more actionable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema documents the only parameter, environmentId, at 100% coverage including the UUID description, so the description's parameter block is pure duplication. No additional semantic meaning, format constraints, or usage nuances are added beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Retrieve details for a specific environment by UUID,' which clearly identifies the operation and its scope. The 'specific' and 'by UUID' phrasing distinguishes it from a list-style operation, though it does not name any sibling tool explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the right context: use this when you need details for one environment and already have its UUID. However, it does not mention alternatives such as contract-testing_list_environments, nor does it state when not to use the tool, leaving the guidance mostly implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_integrations_by_teamContract Testing: Get Integrations by TeamA
Read-onlyIdempotent
Inspect

Retrieve all consumer-provider integrations belonging to a specific team.

Toolset: Integrations and Network

Parameters:

  • teamId (string) required: UUID of the team

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
teamIdYesUUID of the team
pageSizeNoResults per page
pageNumberNoPage number

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already disclose readOnlyHint=true, idempotentHint=true, openWorldHint=false, and destructiveHint=false, so the safety profile is assured. The description adds the team-filtering scope and 'all' qualifier, but does not disclose response shape, sorting, or pagination semantics beyond the schema defaults; this adds some context but not deep behavioral detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is crisp and front-loaded, but the Toolset label and parameter bullet list largely repeat information already available in the schema and title. The extra format/headers are not redundant enough to hurt readability, but also do not earn their place under the value-beyond-structured-fields standard.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only, paginated list operation, the description plus annotations provide enough for an agent to call it correctly: purpose, required teamId, and pagination defaults. It is not a 5 because it never contrasts with the similar list_integrations sibling and says nothing about the returned integration structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the description's parameter list exactly mirrors the schema, including UUID, defaults, and requiredness. It adds no semantic meaning beyond the structured input schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific action ('Retrieve'), a resource ('consumer-provider integrations'), and a scope ('belonging to a specific team'), making it immediately clear what this tool does. The team-scoped wording also distinguishes it from siblings such as contract-testing_list_integrations, even though it does not name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for team-scoped integration retrieval, and the pagination parameters hint at list-style use, but it never says when to prefer this tool over contract-testing_list_integrations or other integration/network tools. No explicit when/when-not guidance or alternatives are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_latest_pacticipant_versionContract Testing: Get Latest Pacticipant VersionB
Read-onlyIdempotent
Inspect

Retrieve the latest version of a pacticipant, optionally filtered by tag.

Toolset: Branches and Versions

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • tag (string): Tag to filter by. If omitted, returns the overall latest version.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagNoTag to filter by. If omitted, returns the overall latest version.
pacticipantNameYesName of the pacticipant

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the useful behavioral detail that omitting the tag returns the overall latest version, but it does not clarify return shape, ordering definition of 'latest', or not-found behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded. However, the 'Parameters' section duplicates what the input schema already states exactly, and the 'Toolset' label adds only marginal context. Some content does not earn its place given the schema coverage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only tool with complete schema descriptions and strong annotations, the description is adequate. Still, with no output schema, it could more clearly describe what a 'version' response contains and what 'latest' means operationally, and it does not help disambiguate between the many contract-testing version-related siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The description repeats the schema's parameter descriptions without adding extra constraints, formats, or examples, meeting the baseline but adding no additional semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve') and a clear resource ('latest version of a pacticipant'), and notes the optional tag filter. It does not explicitly name sibling tools like contract-testing_get_pacticipant_version or contract-testing_list_pacticipant_versions, but 'latest version' makes the focus reasonably distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for fetching the newest version of a pacticipant, optionally filtered by tag, but it does not state when to prefer this over the many sibling get/list/version tools. No explicit alternatives or when-not-to-use guidance is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_metricsContract Testing: Get MetricsA
Read-onlyIdempotent
Inspect

Fetch metrics across the entire workspace

Toolset: Metrics

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, non-destructive behavior, and the description adds the workspace-wide scope, which is useful context. However, it does not disclose return format, metric types, or any aggregation behavior; the annotation coverage keeps this acceptable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Description is short, front-loaded, and easy to scan. 'Toolset: Metrics' and 'Parameters: None' are somewhat redundant with the title and schema, but they are not verbose or harmful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless, read-only metrics fetch, this description gives enough scope and operation context for an agent to invoke it safely. There is no output schema, so a bit more detail about what metrics are returned would improve completeness, but the simplicity of the tool keeps the gap small.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4 under the rubric. The empty input schema fully documents that no parameters are needed, and the description's 'Parameters: None' adds no further semantics but is accurate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states a specific verb ('Fetch'), a resource ('metrics'), and a clear scope ('across the entire workspace'). This scope directly distinguishes it from the sibling contract-testing_get_team_metrics, so an agent can identify which metrics tool is relevant.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'entire workspace' implies this is for workspace-wide metrics, but the description does not explicitly say when to use this instead of contract-testing_get_team_metrics or other metric-related tools. There is no mention of exclusions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_pacticipantContract Testing: Get PacticipantA
Read-onlyIdempotent
Inspect

Retrieve details for a specific pacticipant by name.

Toolset: Pacticipants

Parameters:

  • pacticipantName (string) required: Name of the pacticipant (application or service)

ParametersJSON Schema
NameRequiredDescriptionDefault
pacticipantNameYesName of the pacticipant (application or service)

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, establishing a safe read operation. The description's 'Retrieve' wording is consistent with those annotations but adds no behavioral context beyond themβ€”such as return shape, error conditions, or permission requirementsβ€”so it does not go beyond what structured metadata already conveys.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core sentence is concise, front-loaded, and immediately clear about the operation and target. The additional Toolset heading and Parameter list are redundant with schema metadata and do not add information, but the overall size is still economical.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter read operation with strong safety annotations, the description is largely sufficient: it identifies the target resource and lookup key. The main gap is that 'details' is somewhat vague and there is no output schema or mention of what fields will be returned, but this is minor for a tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter description in the schema ('Name of the pacticipant (application or service)') is repeated verbatim in the tool description. The description therefore adds no meaning beyond the schema, so it meets the baseline but does not enhance agent understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Retrieve'), a resource ('details for a specific pacticipant'), and the lookup key ('by name'). This clearly distinguishes it from siblings like list_pacticipants (list all) and get_pacticipant_version (fetch by version), even though those siblings are not named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by name' implies this tool is appropriate when the agent already knows the pacticipant name and needs its details. However, the description does not explicitly mention when to prefer this over list_pacticipants, get_latest_pacticipant_version, or other version-specific tools, nor does it provide any exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_pacticipant_labelContract Testing: Get Pacticipant LabelA
Read-onlyIdempotent
Inspect

Check whether a specific label is applied to a pacticipant.

Toolset: Labels

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • labelName (string) required: Name of the label

ParametersJSON Schema
NameRequiredDescriptionDefault
labelNameYesName of the label
pacticipantNameYesName of the pacticipant

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear and the description does not contradict it. The description adds the semantic behavior of checking an association, but provides no additional detail about return format, error cases, or authentication needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is one front-loaded sentence that immediately states the purpose. The parameter list is redundant with the schema but short and harmless, so it does not meaningfully impair usability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter read-only lookup, the description is largely sufficient: the annotations cover mutation safety and the wording 'Check whether' implies a boolean-style outcome. Still, because there is no output schema, an explicit statement of the returned value or error semantics would be more complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already documented in the schema. The description merely repeats the parameter names and descriptions without adding constraints, formats, defaults, or relationships beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Check whether') and clearly names the resource and relationship: a specific label applied to a pacticipant. It is clear enough to separate this from listing all labels, though it does not explicitly name or differentiate sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it: when an agent needs to check whether a specific label exists on a specific pacticipant. However, it gives no explicit guidance about alternatives such as list_labels or list_pacticipants_by_label, and no exclusions or when-not-to-use conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_pacticipant_networkContract Testing: Get Pacticipant NetworkB
Read-onlyIdempotent
Inspect

Retrieve the integration network graph for a specific pacticipant.

Toolset: Integrations and Network

Parameters:

  • pacticipantName (string) required: Name of the pacticipant to get network for

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
pacticipantNameYesName of the pacticipant to get network for

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is known. The description adds little behavioral context beyond the word 'Retrieve' and does not explain pagination behavior or the structure of the returned graph, but the annotations cover the key behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is a single clear sentence, but the parameter list is fully redundant with the schema, and the 'Toolset' line adds limited value. The overall size is acceptable, but not every section earns its place given the duplicate parameter documentation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should clarify what the 'integration network graph' contains, but it only names the resource without describing its structure. Invocation requirements are clear, with one required parameter and optional pagination, but the return value is underspecified, leaving a moderate gap in completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters with descriptions and defaults. The description repeats the parameter information verbatim, adding no additional meaning about formats, constraints, or relationships. A baseline score of 3 is appropriate because the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Retrieve' and identifies the resource as 'integration network graph for a specific pacticipant', which clearly conveys what the tool does. It does not explicitly name sibling tools to differentiate itself, but the unique 'network graph' scope distinguishes it from related tools like list_pacticipants and list_integrations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to choose this tool over alternatives, nor does it mention any exclusions or prerequisites. The 'Toolset: Integrations and Network' label is a category tag rather than actionable usage direction, leaving the agent to infer when this tool is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_pacticipant_versionContract Testing: Get Pacticipant VersionA
Read-onlyIdempotent
Inspect

Retrieve details for a specific version of a pacticipant.

Toolset: Branches and Versions

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • versionNumber (string) required: Version number to retrieve

ParametersJSON Schema
NameRequiredDescriptionDefault
versionNumberYesVersion number to retrieve
pacticipantNameYesName of the pacticipant

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint and idempotentHint, and the description's 'Retrieve details' aligns with a safe read. It adds a small scoping point by targeting a specific version rather than latest/all, but does not describe response contents, version format, or any auth/rate considerations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loads the purpose, but the 'Parameters' bullets largely duplicate the input schema, so not every line adds net-new information. Still, the layout is easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read-only tool with complete schema documentation and safety annotations, the description is sufficient for invocation. It could be more complete by noting when to prefer latest/list siblings, but nothing essential for this call is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%: both parameters are documented identically in the schema and description. The description adds no extra semantic meaning such as format constraints or how versionNumber is matched.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with 'Retrieve details for a specific version of a pacticipant', which names a concrete verb and resource and clearly distinguishes this from sibling list/latest tools like contract-testing_list_pacticipant_versions and contract-testing_get_latest_pacticipant_version. The two required parameters make the exact target unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance or exclusions are given. It doesn't say 'for the latest version, use get_latest_pacticipant_version' or contrast with listing versions, so an agent must infer selection from the name and purpose alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_pacts_for_verificationContract Testing: Get Pacts for VerificationB
Read-onlyIdempotent
Inspect

Retrieve the pacts that a provider should verify, based on consumer version selectors and WIP/pending pact configuration.

Toolset: Contracts

Parameters:

  • providerName (string) required: Name of the provider to get pacts for

  • consumerVersionSelectors (array): Selectors specifying which consumer versions to include

  • includePendingStatus (boolean): Include the pending status in the results

  • includeWipPactsSince (string): Include WIP pacts published since this date (ISO 8601)

  • providerVersionBranch (string): Branch of the provider version being verified

  • providerVersionTags (array): Tags for the provider version being verified

ParametersJSON Schema
NameRequiredDescriptionDefault
providerNameYesName of the provider to get pacts for
providerVersionTagsNoTags for the provider version being verified
includePendingStatusNoInclude the pending status in the results
includeWipPactsSinceNoInclude WIP pacts published since this date (ISO 8601)
providerVersionBranchNoBranch of the provider version being verified
consumerVersionSelectorsNoSelectors specifying which consumer versions to include

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description does not need to cover safety. It adds little behavioral detail beyond the selection criteria; it does not mention pagination, response shape, rate limits, or auth requirements. This is acceptable but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is front-loaded and clear, but the description includes a 'Toolset' line and a full parameter list that duplicate schema information, so not every element earns its place. It is reasonably sized but not as tight as it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The schema and annotations handle parameter meaning and safety well, but the tool has a fairly complex nested selector structure and no output schema. The description does not explain the verification workflow context, how consumerVersionSelectors interact with WIP/pending flags, or what the returned data looks like. It is minimally viable with clear gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline applies. The parameter list in the description is a near-verbatim duplicate of the schema and adds no semantic nuance or examples beyond the structured definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the operation ('Retrieve the pacts...') and the resource scope ('that a provider should verify'), with extra specificity about selectors and WIP/pending configuration. It does not explicitly distinguish itself from related contract-testing tools, so it cannot be a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'pacts that a provider should verify' implies this belongs in a provider-verification workflow, and the selector/configuration language hints at the intended use case. However, there is no explicit guidance about when to prefer this tool over related siblings such as matrix, can_i_deploy, or publish_consumer_contracts, and no exclusions are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_provider_statesContract Testing: Get Provider StatesC
Read-onlyIdempotent
Inspect

Retrieve the states of a specific provider

Toolset: Fetch Provider States

Parameters:

  • provider (string) required: name of the provider to retrieve states for

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
providerYesname of the provider to retrieve states for
pageNumberNoPage number

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds no extra behavioral context beyond the parameters themselves, such as whether an unknown provider causes an error, what 'states' means, or what the returned list contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core descriptive sentence is concise, but the parameter section duplicates what is already in the input schema, adding no new information. The structure is clear enough, yet the redundancy keeps it from being tightly economical.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only fetch with one required parameter, the description is minimally sufficient. However, there is no output schema and the description does not describe the return shape, the meaning of 'states' in this domain, or edge cases, so an agent may not fully understand what to expect from the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description essentially repeats the schema's parameter names, types, defaults, and descriptions without adding extra meaning, format constraints, or relationships between parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb ('Retrieve') and specific resource ('the states of a specific provider'), so an agent can tell what the tool operates on. It does not explicitly differentiate it from sibling contract-testing get_* tools, but the resource name is reasonably distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool instead of alternatives, no prerequisites, and no exclusions. It simply restates the action and parameters, leaving the agent to infer the appropriate usage context from the name and sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_released_versions_for_versionContract Testing: Get Released Versions for VersionA
Read-onlyIdempotent
Inspect

Retrieve release records for a specific pacticipant version in a specific environment.

Toolset: Environments and Deployments

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • versionNumber (string) required: Version number

  • environmentId (string) required: UUID of the environment

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
environmentIdYesUUID of the environment
versionNumberYesVersion number
pacticipantNameYesName of the pacticipant

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the environment-scoped resource context but does not disclose additional behavioral details such as response format, error conditions, or pagination behavior beyond the schema defaults. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded, but the Toolset label and the full parameter list duplicate information already present in the input schema. This redundancy prevents the description from being maximally efficient, though it remains reasonably sized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple filtered-list read operation, the combination of annotations, schema, and description covers the required parameters, defaults, and safety profile. The description could clarify what a 'release record' represents versus a deployment record, but this is not essential for invoking the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are already documented. The description's parameter list repeats the schema's names, types, required flags, and defaults without adding extra meaning or usage nuance beyond what the input schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Retrieve'), resource ('release records'), and scope ('for a specific pacticipant version in a specific environment'). This clearly separates it from siblings like get_deployed_versions_for_version, making the tool's function distinct without needing to open schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: use this when you need release records for a specific pacticipant version in a specific environment. It does not explicitly name alternatives or exclusion criteria, but the scoping is explicit enough for basic selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_secretContract Testing: Get SecretA
Read-onlyIdempotent
Inspect

Retrieve metadata for a specific secret by UUID.

Toolset: Secrets

Parameters:

  • secretId (string) required: UUID of the secret

ParametersJSON Schema
NameRequiredDescriptionDefault
secretIdYesUUID of the secret

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint=true, idempotentHint=true, destructiveHint=false). The description adds meaningful value by clarifying that this returns 'metadata' for the secret rather than the secret value itself, which is an important behavioral nuance for a tool named get_secret. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is concise and front-loaded, but the explicit 'Parameters' section duplicates the input schema. The 'Toolset: Secrets' heading and the parameter block are somewhat redundant, though they do not make the description bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only tool with no output schema and rich annotations, the description is sufficiently complete. It states the operation, the identifier type, and the return scope (metadata). Slightly more detail about what metadata fields are returned would be useful, but it is not essential for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter text merely repeats the schema's 'UUID of the secret' without adding format, source, or usage nuance. This meets the baseline but adds no additional semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise verb ('Retrieve'), a specific resource ('metadata for a specific secret'), and the required identifier ('by UUID'). This clearly distinguishes it from siblings like list_secrets (retrieving multiple secrets) and create/update/delete_secret (mutating secrets).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended usage is implied by the description: use this when you have a secret's UUID and need its metadata. However, it does not explicitly mention when to prefer this over alternatives like list_secrets, nor any exclusion conditions, so the guidance is only implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_system_preferencesContract Testing: Get System PreferencesA
Read-onlyIdempotent
Inspect

Retrieve system-wide preferences.

Toolset: User, Tokens and Preferences

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds no behavioral context beyond the basic operation β€” no return format, authentication needs, or side-effect caveats. With annotations present, the description does little to enhance behavioral transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short and front-loaded with the core action. The 'Toolset' and 'Parameters' lines add minor context but are somewhat redundant with the schema and name. Overall it is concise and clear, though not every line is strictly necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter read-only tool, the description is mostly adequate. However, there is no output schema, so the description should provide some hint about what the returned preferences look like. It also does not clarify how system preferences relate to user preferences, which would help an agent choose correctly among siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and schema coverage is 100%, so the schema already fully describes what is needed. With no parameters, the baseline is 4, and the description's 'Parameters: None' note is redundant but harmless.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Retrieve') and a specific resource ('system-wide preferences'). It clearly distinguishes itself from the sibling contract-testing_get_user_preferences through the 'system-wide' qualifier, so an agent can immediately tell what this tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this tool is for system preferences rather than user preferences, and the toolset label 'User, Tokens and Preferences' provides context. However, it does not explicitly state when to use this tool versus alternatives like contract-testing_get_user_preferences or contract-testing_get_current_user.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_team_metricsContract Testing: Get Team MetricsA
Read-onlyIdempotent
Inspect

Fetch metrics for all teams

Toolset: Metrics

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds only the 'all teams' scope and no additional behavioral detail such as response shape, aggregation behavior, or authorization context, but this is acceptable for a simple read operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded, with the core action in the first sentence. The 'Toolset' and 'Parameters' lines are somewhat redundant with the title and schema, but they do not bloat the description significantly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter read-only tool, the description is mostly sufficient, especially with the annotations covering safety. However, there is no output schema and the description does not clarify what metrics are returned or how they are grouped, which could matter when choosing between this and contract-testing_get_metrics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema is empty and the description explicitly states 'Parameters: None,' so there are no parameter semantics to add. The 0-parameter baseline of 4 applies here because the schema already fully describes the absence of inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Fetch') and resource ('metrics for all teams'), making the tool's purpose immediately clear. The scope qualifier 'all teams' also distinguishes it from the sibling contract-testing_get_metrics, which appears to be the more general metrics tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool should be used when team-level metrics are needed, but it provides no explicit guidance on when to prefer this over contract-testing_get_metrics or other related tools. There is no when-not-to-use or alternative-routing information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_user_preferencesContract Testing: Get User PreferencesA
Read-onlyIdempotent
Inspect

Retrieve the current user's preferences.

Toolset: User, Tokens and Preferences

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds useful context by specifying that the preferences belong to the 'current user,' which clarifies auth-scoped behavior without requiring parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is highly concise: one clear opening sentence followed by minimal structured metadata. Every element is short and front-loaded, with no wasted prose or redundant explanation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, annotation-covered read operation, this description is complete. There is no output schema, but the tool's return value ('user preferences') is stated directly in the first sentence, and no hidden inputs or side effects are possible with an empty schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters and the schema is empty, so schema coverage is complete. The description correctly states 'Parameters: None,' and no additional parameter meaning could be added. Baseline 4 is appropriate for a parameterless tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Retrieve the current user's preferences' uses a specific verb and resource, making the tool's purpose immediately clear. The 'current user' qualifier distinguishes it from the sibling contract-testing_get_system_preferences, and 'preferences' distinguishes it from contract-testing_get_current_user.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied by the zero-parameter read operation, but the description provides no explicit guidance about when to choose this tool over alternatives such as contract-testing_get_system_preferences. It is adequate for a trivial getter, but it does not name exclusions or sibling routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_get_webhookContract Testing: Get WebhookA
Read-onlyIdempotent
Inspect

Retrieve details for a specific webhook by UUID.

Toolset: Webhooks

Parameters:

  • webhookId (string) required: UUID of the webhook

ParametersJSON Schema
NameRequiredDescriptionDefault
webhookIdYesUUID of the webhook

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds no behavioral context beyond the operation itself, such as response shape, error/404 behavior, or relationship to other webhook tools. It simply restates the retrieval action without enriching it with non-obvious traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core purpose, followed by a minimal parameter list. The 'Toolset: Webhooks' line adds mild navigational context, and the parameter block is slightly redundant with the schema, but there is no unnecessary prose or padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter get-by-UUID operation with annotations covering the read-only, idempotent, non-destructive profile, the description provides enough information for an agent to select and invoke the tool. It does not detail the return shape, but the phrase 'Retrieve details' gives a reasonable expectation, and no output schema exists to provide more structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents webhookId as a required string UUID. The description repeats the same parameter information without adding extra semantics, such as how to obtain a valid webhookId or any format edge cases. Baseline 3 applies because the schema carries the parameter documentation burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ('Retrieve details'), a specific resource ('webhook'), and the required selector ('by UUID'). This clearly distinguishes the tool from siblings like contract-testing_list_webhooks or contract-testing_execute_webhook without needing to inspect their schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly establishes when to use the tool: when you need details for one specific webhook and have its UUID. It does not explicitly compare against list_webhooks or note that list_webhooks should be used when the UUID is unknown, but the UUID-based scoping makes the usage context clear enough for a simple get-by-id tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_api_tokensContract Testing: List API TokensB
Read-onlyIdempotent
Inspect

Retrieve API tokens for the current user.

Toolset: User, Tokens and Preferences

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the 'current user' scope but says nothing about response format, token masking, pagination behavior, or authentication requirements. It is not misleading, so a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action. The 'Toolset' line is low-value, and the parameter list duplicates the schema, but the overall length is acceptable and there is no verbose filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list operation with two optional pagination params and annotations covering safety, the description is mostly adequate. However, there is no output schema and the description does not hint at the response shape (e.g., token metadata, masking, pagination totals), leaving the agent to guess about the returned payload.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% β€” both pageNumber and pageSize have descriptions and defaults in the schema. The description merely repeats this information without adding semantics like range limits, indexing style, or maximum page size, so it adds no value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Retrieve API tokens for the current user.' This clearly differentiates the tool from token creation/regeneration and from admin/system-account token listing by scoping to the current user. It does not explicitly name a sibling alternative, but the scope is a meaningful differentiator.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance or exclusions are provided. The description does not mention that this lists only the user's own tokens, nor does it contrast with related tools like contract-testing_admin_get_system_account_tokens or contract-testing_regenerate_api_token. An agent must infer usage from the name and the 'current user' scope.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_branchesContract Testing: List BranchesA
Read-onlyIdempotent
Inspect

Retrieve all branches for a given pacticipant, with optional filtering and pagination.

Toolset: Branches and Versions

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • q (string): Filter branches by name

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
qNoFilter branches by name
pageSizeNoResults per page
pageNumberNoPage number
pacticipantNameYesName of the pacticipant

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the facts about filtering and pagination, but does not disclose behaviors such as default page size implications, sort order, or how to retrieve all items beyond the first page. This is minor added value beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main description is front-loaded in a single clear sentence, and the parameter list is compact. The Toolset line is short. The parameter block is redundant with the schema but not excessively verbose, so overall it is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple schema and safety annotations, the description is mostly adequate for a list operation. However, there is no output schema, and the description does not explain what a branch object contains or how pagination works in practice, which an agent may need to know to fully consume the results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all four parameters. The description repeats the parameter list almost verbatim without adding further semantic value or clarifying edge cases, so it stays at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve') and a specific resource ('all branches for a given pacticipant'), which distinguishes it from sibling tools like get_branch, get_branch_versions, and list_pacticipants. It also mentions optional filtering and pagination, further clarifying the exact operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this tool is for retrieving all branches of a pacticipant, which provides clear context. However, it does not explicitly state when to use this over alternatives such as get_branch or get_branch_versions, nor does it mention any exclusions or absence of filtering capabilities.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_environmentsContract Testing: List EnvironmentsB
Read-onlyIdempotent
Inspect

Retrieve all environments configured in the Pact Broker or PactFlow workspace.

Toolset: Environments and Deployments

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the operation read-only, idempotent, and non-destructive. The description adds a mild scoping statement ('all environments configured in ... workspace') but does not disclose additional behavioral traits such as pagination behavior, response format, ordering, or any workspace-specific constraints. The bar is lower with annotations, but this description contributes only marginal context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose sentence is clear and front-loaded, but the subsequent 'Parameters' section duplicates the schema descriptions, and the 'Toolset' line is low-value filler. The description is short, but not every element earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple two-parameter read-only operation with complete annotation coverage and fully documented parameters, so the definition is mostly sufficient. The main gap is the lack of return-value or pagination details, especially given there is no output schema, but the operation is straightforward enough that an agent can correctly invoke it from the provided information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%; both pageNumber and pageSize already carry descriptions and defaults in the input schema. The 'Parameters' section in the description simply restates those schema details, adding no new meaning or guidance such as maximum page size or pagination behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description's opening sentence 'Retrieve all environments configured in the Pact Broker or PactFlow workspace' uses a specific verb and resource, with a clear scope. The plural 'all environments' distinguishes it from the singular sibling contract-testing_get_environment, and the Pact Broker/PactFlow scope separates it from bearq_list_environments.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit when-to-use guidance, exclusions, or alternative tool routing. The 'Toolset: Environments and Deployments' heading is merely a category label, not a selection criterion. An agent choosing between contract-testing_list_environments, contract-testing_get_environment, and bearq_list_environments receives no help in deciding which to call.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_integrationsContract Testing: List IntegrationsA
Read-onlyIdempotent
Inspect

Retrieve all consumer-provider integrations registered in the workspace.

Toolset: Integrations and Network

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as read-only, idempotent, and non-destructive. The description adds useful context beyond those annotations: it retrieves workspace-level integrations, not a team-filtered subset. There is no contradiction with the annotations, and the safety profile is well covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded in a single clear sentence, and the parameter list is scannable. The parameter block duplicates the input schema somewhat, which is mild redundancy, but the overall definition is compact and well-organized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a straightforward, read-only listing operation with two optional and fully documented pagination parameters, the definition is largely complete. The main gaps are that there is no output schema and the description does not explain the response shape or reconcile 'all' with the paginated pageSize default, but these are moderate gaps given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% coverage for both parameters, including types, defaults, and descriptions. The description repeats the schema information without adding deeper semantics, such as range constraints, ordering, or pagination envelope details. This matches the baseline for fully schema-documented parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Retrieve all consumer-provider integrations registered in the workspace.' The workspace-level scope is clear. However, it does not explicitly differentiate itself from sibling tools like contract-testing_get_integrations_by_team, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use itβ€”when the agent needs all consumer-provider integrations in the workspaceβ€”and the pagination parameters suggest batch retrieval. It does not provide explicit guidance about alternatives, exclusions, or when a more specific endpoint like get_integrations_by_team should be preferred.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_labelsContract Testing: List LabelsA
Read-onlyIdempotent
Inspect

Retrieve all labels used across the workspace.

Toolset: Labels

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds workspace-wide scope, but it does not clarify the pagination behavior implied by pageNumber/pageSize while claiming to retrieve 'all' labels, nor does it describe the response shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening purpose sentence is clear, front-loaded, and brief. However, the embedded 'Parameters' section duplicates the input schema, adding mild redundancy that prevents a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list operation with two optional parameters, the description is mostly sufficient. The main gap is the absence of any indication of what the returned label objects contain, especially since there is no output schema, but the resource and pagination intent are clear enough for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema fully documents both parameters. The description merely restates the defaults without adding deeper semantics like maximum page size or how pagination interacts with the 'all labels' claim.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

'Retrieve all labels used across the workspace' uses a specific verb ('retrieve'), names the resource ('labels'), and scopes it ('across workspace'). This clearly distinguishes it from sibling tools like contract-testing_get_pacticipant_label and contract-testing_list_pacticipants_by_label, which operate on labels differently.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no explicit when-to-use guidance or alternatives. It does not mention when to prefer this over contract-testing_list_pacticipants_by_label or contract-testing_get_pacticipant_label, so the agent must infer usage solely from the purpose statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_pacticipantsContract Testing: List PacticipantsB
Read-onlyIdempotent
Inspect

Retrieve all pacticipants (applications/services) registered in the Pact Broker or PactFlow workspace.

Toolset: Pacticipants

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Number of results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoNumber of results per page
pageNumberNoPage number

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already communicate read-only, idempotent, non-destructive behavior. The description adds useful scope context (Pact Broker or PactFlow workspace, 'all' pacticipants), but does not disclose pagination behavior, ordering, or any access requirements beyond what the schema defaults imply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is direct and front-loaded, and the overall length is short. The separate 'Parameters' section duplicates schema information and the 'Toolset' line adds little, which prevents a top score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only listing tool with two optional pagination parameters, the essential facts are present. However, with no output schema, the description does not describe the shape of the returned pacticipant objects, and 'all' could be misinterpreted as a single non-paginated response despite the pageSize/pageNumber defaults.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes both parameters fully with defaults, so schema coverage is 100%. The description merely repeats the same parameter text and adds no extra constraints, formatting details, or interaction notes such as maximum page size.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Retrieve'), a specific resource ('pacticipants'), and clarifies what they are ('applications/services'), plus the workspace scope. It does not explicitly differentiate this from sibling tools like list_pacticipants_by_label or get_pacticipant, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The purpose statement implies this is the tool to use when you need the full set of pacticipants, but there is no explicit guidance on when to use it versus alternatives or when not to use it. The 'Toolset: Pacticipants' line is organizational rather than usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_pacticipants_by_labelContract Testing: List Pacticipants by LabelA
Read-onlyIdempotent
Inspect

Retrieve all pacticipants that have a specific label applied.

Toolset: Labels

Parameters:

  • labelName (string) required: Label name to filter by

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
labelNameYesLabel name to filter by
pageNumberNoPage number

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds little behavioral context beyond the filter and pagination parameters, but it does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loads the main purpose. The parameter list and 'Toolset: Labels' line add minor redundancy with the schema but keep the description self-contained without being verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list operation with well-documented parameters and safety annotations, the description is sufficient. It lacks output-shape details, but no output schema exists and the request is straightforward enough that this is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema fully documents all three parameters. The description repeats the parameter list without adding extra meaning, defaults, or usage nuance beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve') with a clear resource ('all pacticipants') and a precise filter ('that have a specific label applied'). This distinguishes it from siblings like contract-testing_list_pacticipants (no label filter) and contract-testing_list_labels (returns labels, not pacticipants).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not state when to choose this tool over alternatives, nor does it mention any exclusions or related tools. While the label-filtered purpose is implicit, there is no explicit guidance for an agent navigating the large sibling toolset.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_pacticipant_versionsContract Testing: List Pacticipant VersionsB
Read-onlyIdempotent
Inspect

Retrieve all versions for a given pacticipant.

Toolset: Branches and Versions

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number
pacticipantNameYesName of the pacticipant

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds no behavioral details beyond restating the purpose and parameter listβ€”no mention of result ordering, pagination limits, response shape, or other operational behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded, leading with the purpose and then organizing parameters under a toolset label. The parameter block mostly repeats schema content, so some lines are redundant, but the overall size is still appropriate and scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only, paginated list operation with one required parameter, the description plus schema and annotations provides enough information for an agent to select and invoke it correctly. The absence of an output schema and lack of return-value detail are not critical for this straightforward retrieval task.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters with defaults. The description's parameter section largely duplicates the schema and adds no richer meaning about formats, constraints, or how pagination interacts with 'all versions.'

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Retrieve all versions for a given pacticipant.' This clearly states the tool's function and the word 'all' distinguishes it from single-version siblings like get_pacticipant_version and get_latest_pacticipant_version, though it does not explicitly name those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is only implied: 'Retrieve all versions for a given pacticipant' suggests it is for listing all historical versions rather than fetching a specific or latest version. The 'Toolset: Branches and Versions' label adds categorization but no explicit when-to-use guidance or exclusions versus closely related tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_secretsContract Testing: List SecretsA
Read-onlyIdempotent
Inspect

Retrieve all secrets stored in the workspace.

Toolset: Secrets

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the read-only safety profile is covered by structured data. The description adds minimal behavioral context beyond the workspace scope; it does not mention pagination semantics beyond parameter defaults or any masking/redaction of secret values.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The one-sentence purpose is front-loaded and clear, but the following 'Toolset: Secrets' line and the entire Parameters block duplicate information already present in the title and input schema. This makes it compact but not every line earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list operation with two optional pagination parameters and no output schema, the description plus annotations provide enough information to invoke it correctly. It could be richer by stating the response shape or that secrets are only a listing of metadata, but these are not severe gaps at this complexity level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the schema already provides the same descriptions, types, and defaults for pageNumber and pageSize. The description repeats these details rather than adding extra meaning such as maximum page size, ordering, or response format.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description uses a specific verbβ€”'Retrieve all secrets stored in the workspace'β€”and clearly identifies the resource (secrets) and scope (workspace). This distinguishes the plural listing operation from siblings such as contract-testing_get_secret, create_secret, and delete_secret.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied by 'all secrets': the agent can infer this is the right tool for enumerating workspace secrets. However, there is no explicit guidance contrasting it with get_secret for retrieving a single secret or with secret mutations, leaving the alternatives to be inferred.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_list_webhooksContract Testing: List WebhooksA
Read-onlyIdempotent
Inspect

Retrieve all webhooks configured in the workspace.

Toolset: Webhooks

Parameters:

  • pageNumber (number): Page number (default: 1)

  • pageSize (number): Results per page (default: 5)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageSizeNoResults per page
pageNumberNoPage number

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds only the workspace-level scope and 'all webhooks', which is useful but does not disclose additional behaviors such as pagination semantics or return shape. With annotations present, this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core purpose. The 'Toolset: Webhooks' label and parameter list are slightly redundant with the title and schema, but the overall size is appropriate and no information is buried or verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, read-only, paginated list operation with fully documented optional parameters and strong annotations, the description is mostly complete. It does not describe the output format, but for a list-everything endpoint this is a minor gap given there is no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully documents both parameters with names, defaults, and descriptions, so the baseline is 3. The description merely repeats the parameters without adding new semantics like interaction between pageNumber and pageSize or result ordering. It provides no value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Retrieve') and resource ('all webhooks configured in the workspace'), clearly distinguishing this collection operation from sibling tools like contract-testing_get_webhook and contract-testing_execute_webhook. Scope and resource are both explicit, leaving no ambiguity about what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: call this when you need the full list of webhooks in the workspace. However, the description does not explicitly contrast it with contract-testing_get_webhook for retrieving a single webhook, nor does it state when not to use it. There is no explicit alternatives guidance beyond the verb 'list'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_matrixContract Testing: MatrixA
Read-onlyIdempotent
Inspect

Retrieve the comprehensive contract verification matrix that shows the relationship between consumer and provider versions, their associated pact files, and verification results stored in the Pact Broker or Pactflow. The matrix provides detailed visibility into which consumer and provider versions have been successfully verified against each other, and highlights failures with detailed information about the cause.

Toolset: Matrix

Parameters:

  • latestby (string): This property removes the rows for the overridden pacts/verifications from the results. The options are cvp (show only the latest row for each consumer version and provider) and cvpv (show only the latest row each consumer version and provider version). For a can-i-deploy query with one selector, it should be set to cvp. For a can-i-deploy query with two selectors, it should be set to cvpv.

  • limit (number): The limit on the number of results to return (1-1000, default: 100) (default: 100)

  • q (array) required

Use Cases: 1. Quickly identify which consumer and provider version combinations have passed or failed verification. 2. Diagnose and investigate why a particular consumer-provider verification failed. 3. Visualize the overall contract compatibility across two pacticipants / services. 4. Perform advanced queries using selectors to understand compatibility within specific branches, environments, or version ranges. 5. Support informed deployment decisions by answering 'can I deploy version X of this service to production?' 6. Expose contract verification details to non-frequent API users in a more accessible format.

ParametersJSON Schema
NameRequiredDescriptionDefault
qYes
limitNoThe limit on the number of results to return (1-1000, default: 100)
latestbyNoThis property removes the rows for the overridden pacts/verifications from the results. The options are cvp (show only the latest row for each consumer version and provider) and cvpv (show only the latest row each consumer version and provider version). For a can-i-deploy query with one selector, it should be set to cvp. For a can-i-deploy query with two selectors, it should be set to cvpv.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that the tool is read-only, idempotent, and non-destructive. The description adds useful behavioral detail beyond that: it describes exactly what the returned matrix exposes, the relationship between versions, and that failures come with detailed cause information. No contradiction with annotations exists. It omits exact response formatting, but the annotations and descriptive return semantics reduce the impact of that gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening paragraph is strong and front-loaded, and the bulleted use cases make the intended scenarios easy to scan. However, the Parameters section duplicates what the input schema already documents, and the use-case list is somewhat verbose and overlapping. The structure is clear but not every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only tool with strong annotations and no output schema, the description provides enough context to understand what will be returned and why an agent would call it: version relationships, pact files, verification outcomes, and failure details. It covers parameter selection, use cases, and system context (Pact Broker or Pactflow). It is not exhaustive about the exact response shape, but it is complete enough for successful invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67%, so the description should add value beyond the schema. It does add a useful rule of thumb for latestby (cvp for one-selector can-i-deploy queries, cvpv for two-selector queries), but it largely repeats the limit and latestby descriptions already present in the schema. The required q parameter is only marked as required, leaving its selector semantics to the schema entries; this is a meaningful gap because q is the core query parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the operation ('Retrieve the comprehensive contract verification matrix') and the resource domain: consumer/provider versions, pact files, and verification results. It is specific about what the matrix contains and that it highlights failures. It does not explicitly distinguish itself from the sibling contract-testing_can_i_deploy, even though it references can-i-deploy in its use cases, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section provides explicit contexts for using the tool, such as identifying passed/failed combinations, diagnosing failures, visualizing compatibility, and advanced selector queries. This gives an agent clear situational guidance. However, there is no explicit 'when not to use' or comparison to the closely related can_i_deploy sibling, so the guidance is not complete enough for a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_patch_pacticipantContract Testing: Patch PacticipantAInspect

Partially update a pacticipant's metadata β€” only fields provided are changed.

Toolset: Pacticipants

Parameters:

  • pacticipantName (string) required: Name of the pacticipant to update

  • displayName (string): Human-readable display name

  • mainBranch (string): Name of the main/trunk branch (e.g. 'main')

  • repositoryName (string): Repository name

  • repositoryNamespace (string): Repository namespace/organisation

  • repositoryUrl (string): URL of the source repository

ParametersJSON Schema
NameRequiredDescriptionDefault
mainBranchNoName of the main/trunk branch (e.g. 'main')
displayNameNoHuman-readable display name
repositoryUrlNoURL of the source repository
repositoryNameNoRepository name
pacticipantNameYesName of the pacticipant to update
repositoryNamespaceNoRepository namespace/organisation

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=false and destructiveHint=false, so the description doesn't need to cover mutation safety. It does add useful PATCH-specific behavior ('only fields provided are changed'), but doesn't mention outcomes like 404s, return values, or side effects beyond the metadata change.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is effective and front-loaded, but the Toolset label and the full parameter list duplicate structured information already present in the schema. The description is reasonably short but not maximally economical.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple patch operation with fully documented parameters, this is minimally adequate. However, there is no output schema and the description does not address what the response contains, how errors are surfaced, or how this differs from the sibling update tool. An agent could invoke it correctly but is not fully equipped to handle edge cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter. The description repeats the parameter list with identical wording, adding no meaning beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states the specific verb 'Partially update' and the resource 'pacticipant's metadata', and clarifies the PATCH semantics with 'only fields provided are changed.' This distinguishes it clearly from a full update operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it (partial update) but never names the alternative contract-testing_update_pacticipant or states when a full update would be preferred. Usage context is implicit rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_publish_consumer_contractsContract Testing: Publish Consumer ContractsB
Idempotent
Inspect

Publish one or more consumer Pact contracts to the Pact Broker or PactFlow, with branch and tag metadata.

Toolset: Contracts

Parameters:

  • pacticipantName (string) required: Name of the consumer application

  • pacticipantVersionNumber (string) required: Version number of the consumer

  • contracts (array) required: Contracts to publish

  • tags (array): Version tags (e.g. 'main', 'staging')

  • branch (string): Branch name of the consumer

  • buildUrl (string): URL of the CI build that produced these contracts

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoVersion tags (e.g. 'main', 'staging')
branchNoBranch name of the consumer
buildUrlNoURL of the CI build that produced these contracts
contractsYesContracts to publish
pacticipantNameYesName of the consumer application
pacticipantVersionNumberYesVersion number of the consumer

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the destination (Pact Broker/PactFlow) and metadata capabilities, but it does not disclose overwrite behavior, authentication needs, or other side effects. This is enough to slightly raise it above baseline, but not by much.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is crisp and front-loaded, but a large parameter block repeats the schema almost word-for-word. Given that a complete schema is already provided, this duplication is unnecessary and makes the description longer than it should be. The parameter list does not earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description plus schema provide enough to call the tool correctly, including required parameters, types, and nested contract object structure. Minor gaps remain: no mention of authentication, return values, or the relationship between pacticipantName and the contracts' consumerName. These omissions are not severe given the annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all six parameters are fully documented in the input schema. The description's 'Parameters' section is essentially a verbatim copy of the schema's descriptions and adds no new meaning beyond human-readable formatting. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Publish') and names the exact resource ('consumer Pact contracts') and destination ('Pact Broker or PactFlow'). It also mentions branch and tag metadata, clearly distinguishing it from provider-side contract publishing by explicitly saying 'consumer'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus alternatives. It does not mention the sibling contract-testing_publish_provider_contract or any other Pact tools, and there are no conditions or exclusions. The only context is the generic 'Toolset: Contracts' label, which is too weak to route an agent effectively.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_publish_provider_contractContract Testing: Publish Provider ContractA
Idempotent
Inspect

Publish a provider OpenAPI or AsyncAPI contract and self-verification results to PactFlow (Bi-Directional Contract Testing).

Toolset: Contracts

Parameters:

  • providerName (string) required: Name of the provider application

  • pacticipantVersionNumber (string) required: Version number of the provider

  • contract (object) required: Provider contract (OpenAPI or AsyncAPI spec) and verification details

  • tags (array): Version tags

  • branch (string): Branch name of the provider

  • buildUrl (string): URL of the CI build

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoVersion tags
branchNoBranch name of the provider
buildUrlNoURL of the CI build
contractYesProvider contract (OpenAPI or AsyncAPI spec) and verification details
providerNameYesName of the provider application
pacticipantVersionNumberYesVersion number of the provider

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide the key behavioral profile: not read-only, idempotent, and non-destructive. The description adds context about PactFlow and bi-directional contract testing, but it does not describe side effects such as whether an existing contract is replaced or whether the provider must already exist. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose sentence is strong and front-loaded, but the parameter list largely duplicates the input schema and adds limited value. The 'Toolset: Contracts' line is minor filler. The description is not bloated, but it could be shorter and more additive.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a six-parameter tool with a nested required contract object, the schema plus description cover all necessary input semantics, and the annotations cover the safety profile. It does not describe response or error behavior, and no output schema exists, but nothing critical is missing for invoking the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% parameter coverage with detailed descriptions, including nested fields like selfVerificationResults. The description repeats definitions such as 'contract (object): Provider contract...' without adding new constraints, examples, or clarifications beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence is explicit: 'Publish a provider OpenAPI or AsyncAPI contract and self-verification results to PactFlow.' This gives a specific verb, resource, and target system. The 'provider' wording also distinguishes it from the sibling publish_consumer_contracts tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes the intended use clear: publish a provider contract with self-verification details to PactFlow. It does not explicitly name alternatives or say when not to use it, but the provider-focused wording is enough to route an agent to the correct sibling tool in most cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_record_deploymentContract Testing: Record DeploymentBInspect

Record that a version of a pacticipant has been deployed to an environment.

Toolset: Environments and Deployments

Parameters:

  • pacticipantName (string) required: Name of the pacticipant that was deployed

  • versionNumber (string) required: Version number that was deployed

  • environmentId (string) required: UUID of the target environment

  • applicationInstance (string): Identifies a specific instance when multiple instances of the same application are deployed to the same environment (e.g. 'blue', 'green')

ParametersJSON Schema
NameRequiredDescriptionDefault
environmentIdYesUUID of the target environment
versionNumberYesVersion number that was deployed
pacticipantNameYesName of the pacticipant that was deployed
applicationInstanceNoIdentifies a specific instance when multiple instances of the same application are deployed to the same environment (e.g. 'blue', 'green')

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The phrase 'Record that... has been deployed' usefully clarifies that this tool records a deployment event rather than performing an actual deployment. Annotations already cover readOnly=false and destructive=false; the description adds only modest behavioral context and no side-effect or idempotency detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main purpose sentence is front-loaded and clear, and the format is readable. However, the inline parameter list largely duplicates the input schema, so it adds redundancy rather than genuinely new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple write operation with a fully documented schema, the required call shape is present and all parameters are described. It omits operational consequences, output/return behavior, and any note about how this relates to other environment/deployment tools, leaving the description only partially complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter list in the description exactly mirrors the schema descriptions, adding no new semantic value. Baseline 3 applies because the structured schema carries the parameter-documentation burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence uses a specific verb and object β€” 'Record that a version of a pacticipant has been deployed to an environment' β€” making the core operation clear. It does not explicitly contrast with sibling tools like contract-testing_record_release, so it stops short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no when-to-use guidance or exclusions. It doesn't tell the agent when to prefer this over contract-testing_record_release or mention prerequisites such as the environment already existing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_record_releaseContract Testing: Record ReleaseAInspect

Record that a version of a pacticipant has been released to an environment (for mobile/library release workflows).

Toolset: Environments and Deployments

Parameters:

  • pacticipantName (string) required: Name of the pacticipant that was released

  • versionNumber (string) required: Version number that was released

  • environmentId (string) required: UUID of the target environment

ParametersJSON Schema
NameRequiredDescriptionDefault
environmentIdYesUUID of the target environment
versionNumberYesVersion number that was released
pacticipantNameYesName of the pacticipant that was released

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal a non-read-only, non-idempotent mutation; the description's 'Record' phrase aligns with that. It adds release-workflow context but does not disclose side effects, response behavior, or what happens if the release already exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The key purpose is front-loaded and the organization is scannable. The parameter list is redundant with the schema, but the overall size is still appropriate for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

All three required parameters are documented and the workflow context helps an agent choose and invoke the tool. No output schema exists, but for a simple record operation the description is sufficient; explicit sibling routing would make it complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the description repeats the same parameter meanings verbatim. It adds no semantic detail beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: records a released pacticipant version to an environment. The parenthetical '(for mobile/library release workflows)' differentiates this from the sibling record_deployment tool without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context that this is for release workflows, especially mobile/library releases. It does not explicitly name record_deployment as the alternative or give when-not-to-use guidance, so it misses the top bar.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_regenerate_api_tokenContract Testing: Regenerate API TokenA
Destructive
Inspect

Regenerate (rotate) an API token by ID.

Toolset: User, Tokens and Preferences

Parameters:

  • tokenId (string) required: ID of the token to regenerate

ParametersJSON Schema
NameRequiredDescriptionDefault
tokenIdYesID of the token to regenerate

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool destructive and non-idempotent. The description's 'rotate' clarifies that this is a token rotation rather than a general update, but it doesn't disclose that the old token stops working, whether special permissions are required, or what the response contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loads the purpose in one sentence. The 'Toolset' line and parameter list add minor context but largely duplicate the name and schema, so it is efficient but not maximally tight.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive action it is mostly usable, but with no output schema it omits important detail: whether the new token value is returned, and what consequences occur after rotation. It also doesn't point to a sibling like list_api_tokens for finding tokenId.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter section only repeats the schema's tokenId description. It adds no new semantics (e.g., where to find token IDs or format expectations), so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Regenerate (rotate)') and a concrete resource ('API token by ID'), so an agent knows exactly what operation is performed. It adds 'by ID' to disambiguate selection from list/create/delete token tools, and is not a mere restatement of the title.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no explicit when-to-use guidance, such as 'use when a token is compromised/expired' or 'call list_api_tokens to obtain tokenId.' The use case is only implied by the verb 'regenerate'; there are no exclusions or alternatives named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_remove_label_from_pacticipantContract Testing: Remove Label from PacticipantA
DestructiveIdempotent
Inspect

Remove a label from a pacticipant.

Toolset: Labels

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • labelName (string) required: Name of the label

ParametersJSON Schema
NameRequiredDescriptionDefault
labelNameYesName of the label
pacticipantNameYesName of the pacticipant

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey destructiveHint=true and idempotentHint=true, and the description simply restates the removal action without adding new behavioral context such as side effects, error behavior when the label does not exist, or permanence of removal. No contradiction exists between description and annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action. The parameter list is somewhat redundant with the schema but the overall structure is clean and easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter removal operation with no output schema, the description plus annotations are largely sufficient. It could mention the outcome of removing a missing label, but the idempotentHint partially covers that concern.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter section only repeats the schema's parameter names and descriptions. It adds no additional meaning about types, formats, or constraints beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Remove') with a clear resource ('a label from a pacticipant'), making the operation unambiguous. It also distinguishes itself from sibling tools like add_label_to_pacticipant and list_labels by stating the exact reverse action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the use case: when a label should be removed from a pacticipant. However, it does not explicitly state when not to use it or point to alternatives such as add_label_to_pacticipant for creating the relationship.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_review_pact_testsContract Testing: Review Pact TestsC
Read-onlyIdempotent
Inspect

Review Pact tests using PactFlow AI. You can provide the following inputs: (1) Pact tests to be reviewed along with metadata

Toolset: Review Pact Tests

Parameters:

  • pactTests (object) required: Primary pact tests that needs to be refined.

  • code (array): Collection of source code files to analyze and extract API interactions from. Include client code, data models, existing tests, or any code that makes API calls

  • userInstructions (string): Optional free-form instructions that provide additional context or specify areas of focus during the refinement process of the Pact test.

  • errorMessages (array): Optional error output from failed contract test runs. These can be used to better understand the context or failures observed and guide the recommendations toward resolving specific issues.

  • openapi (any)

ParametersJSON Schema
NameRequiredDescriptionDefault
codeNoCollection of source code files to analyze and extract API interactions from. Include client code, data models, existing tests, or any code that makes API calls
openapiNoIf provided, the OpenAPI document which describes the API being tested and is accompanied by a matcher which will be used to identify the interactions in the OpenAPI document which are relevant to the Pact refinement process.
pactTestsYesPrimary pact tests that needs to be refined.
errorMessagesNoOptional error output from failed contract test runs. These can be used to better understand the context or failures observed and guide the recommendations toward resolving specific issues.
userInstructionsNoOptional free-form instructions that provide additional context or specify areas of focus during the refinement process of the Pact test.

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds that this is an AI-assisted review and hints at producing recommendations, but it does not describe what the tool returns, whether results are immediate, or how the review is structured.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise, but the description includes a redundant 'Toolset: Review Pact Tests' heading and a full parameter list that duplicates the input schema. These parts do not earn their place and add unnecessary length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The input parameters are well documented in the schema, but there is no output schema and the description does not explain what the agent should expect in response. With five parameters, nested objects, and no return-value information, the description is incomplete for confidently using the tool's result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters and nested fields. The description's parameter list largely duplicates the schema without adding meaningful new meaning or clarifying the optional openapi structure.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action ('Review Pact tests') and the subject ('Pact tests using PactFlow AI'), so an agent can understand what the tool does at a glance. It does not explicitly differentiate from the closely related sibling 'contract-testing_generate_pact_tests', so it misses the top score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives such as contract-testing_generate_pact_tests, nor any statement of when not to use it. The description lists inputs but does not explain the decision context for choosing this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_test_execute_webhooksContract Testing: Test Execute WebhooksAInspect

Trigger a test execution of all matching webhooks without a real event.

Toolset: Webhooks

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate the operation is not read-only or idempotent; the description adds that this is a test execution without a real event, which is useful side-effect context. It does not disclose whether outbound webhook calls are actually made, whether the operation is asynchronous, or how results are returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core behavior is stated in a single front-loaded sentence. The Toolset and Parameters labels are minimal and do not add meaningful clutter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless trigger, the description is mostly adequate and annotations cover the basic operation type. Still, 'all matching webhooks' is ambiguous and there is no mention of execution results or asynchronous behavior, which an agent would need to know what to expect after invoking this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the schema is effectively complete, so there is no parameter meaning for the description to add. The 'Parameters: None' line is redundant but harmless.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Trigger a test execution') and resource ('all matching webhooks'), and the qualifier 'without a real event' distinguishes it from real webhook execution. However, 'matching' is left undefined and no sibling is named, so the exact scope is somewhat inferred.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'without a real event' implies the intended use case of testing webhook configuration without triggering a real event. There is no explicit when-to-use versus alternatives guidance, especially against the sibling contract-testing_execute_webhook, so selection relies on inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_update_environmentContract Testing: Update EnvironmentB
Idempotent
Inspect

Update an existing environment's metadata.

Toolset: Environments and Deployments

Parameters:

  • environmentId (string) required: UUID of the environment to update

  • name (string) required: Unique name for the environment

  • production (boolean) required: Whether this is a production environment

  • displayName (string): Human-readable display name

  • teamUuids (array): UUIDs of teams that own this environment

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesUnique name for the environment
teamUuidsNoUUIDs of teams that own this environment
productionYesWhether this is a production environment
displayNameNoHuman-readable display name
environmentIdYesUUID of the environment to update

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already indicate that this is a write operation (readOnlyHint=false) but non-destructive and idempotent. The description adds little behavioral context beyond restating that it updates metadata, such as what happens to unspecified fields, whether displayName and teamUuids are replaced in bulk, or what the API returns on success.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and informative, but the parameter list duplicates the input schema almost verbatim. This redundancy means not every line earns its place, though the overall structure is readable and not excessive.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter update tool with full schema coverage and annotations covering idempotency and non-destructiveness, the description is minimally adequate. However, it does not explain the effect of omitting optional fields, whether the update is partial or full replacement, or what result the caller should expect, which are relevant gaps given there is no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all five parameters. The description repeats the parameter names and required flags without adding new meaning, which meets the baseline but does not go beyond what structured data provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Update'), the resource ('an existing environment'), and the scope ('metadata'), making it easy to distinguish from sibling tools like create_environment and delete_environment. The verb-resource pairing is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus create_environment, delete_environment, or list_environments. The description implies the user already knows they want to update an existing environment, but it does not state conditions, prerequisites, or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_update_pacticipantContract Testing: Update PacticipantA
Idempotent
Inspect

Fully replace a pacticipant's metadata (display name, main branch, repository URL, etc.).

Toolset: Pacticipants

Parameters:

  • pacticipantName (string) required: Name of the pacticipant to update

  • displayName (string): Human-readable display name

  • mainBranch (string): Name of the main/trunk branch (e.g. 'main')

  • repositoryName (string): Repository name

  • repositoryNamespace (string): Repository namespace/organisation

  • repositoryUrl (string): URL of the source repository

ParametersJSON Schema
NameRequiredDescriptionDefault
mainBranchNoName of the main/trunk branch (e.g. 'main')
displayNameNoHuman-readable display name
repositoryUrlNoURL of the source repository
repositoryNameNoRepository name
pacticipantNameYesName of the pacticipant to update
repositoryNamespaceNoRepository namespace/organisation

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish idempotent, non-read-only, and non-destructive behavior. The description adds the meaningful behavioral trait that this is a full replacement rather than a merge, which is exactly the kind of context an agent needs to avoid overwriting metadata unintentionally.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is well organized: a front-loaded purpose sentence, a short toolset line, and a scannable parameter list. It is slightly redundant with the input schema, but every element is relevant and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With all six parameters documented in the schema, annotations covering idempotency/safety, and no output schema to explain, the description covers the essential semantics of the operation. The only notable gap is that it doesn't contrast with the patch sibling or clarify the effect of omitting optional fields, but this is a minor omission rather than a blocking one.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the description's parameter list mostly repeats the schema's own descriptions verbatim. It adds no new format, default, or side-effect information about individual parameters, so it stays at the baseline for a fully documented schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening phrase 'Fully replace a pacticipant's metadata' uses a specific verb ('replace'), names the resource ('pacticipant'), and distinguishes this from the sibling patch tool by emphasizing full replacement. It states the exact fields involved, leaving no ambiguity about what the tool acts on.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Fully replace' wording implies this tool is appropriate when the caller wants to overwrite the whole metadata set, and the Toolset label provides loose grouping context. However, it never names the obvious sibling contract-testing_patch_pacticipant or says to prefer that tool for partial updates, so the when-to-use guidance is implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_update_pacticipant_versionContract Testing: Update Pacticipant VersionC
Idempotent
Inspect

Update metadata for a specific pacticipant version.

Toolset: Branches and Versions

Parameters:

  • pacticipantName (string) required: Name of the pacticipant

  • versionNumber (string) required: Version number to update

  • buildUrl (string): URL of the CI build that produced this version

ParametersJSON Schema
NameRequiredDescriptionDefault
buildUrlNoURL of the CI build that produced this version
versionNumberYesVersion number to update
pacticipantNameYesName of the pacticipant

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds no behavioral detail beyond the title: it does not state whether the update replaces existing metadata, requires an existing version, has side effects, or what the response contains. Annotations already cover idempotence and non-destructiveness, so the description itself contributes little transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded, and the toolset line adds useful context. However, the bulleted parameter section simply repeats the input schema, which is redundant in an MCP description and does not earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple three-parameter tool with annotations covering safety and idempotence, the definition is minimally adequate for invoking with correct arguments. But it omits when-to-use guidance, return behavior (no output schema), and any note about required existence of the version, leaving the overall context incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all three parameters. The description duplicates those definitions without adding new meaning such as formats, constraints, or relationships between parameters, so it meets the baseline but does not exceed it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action and resource: 'Update metadata for a specific pacticipant version.' This distinguishes it from pacticipant-level tools like update_pacticipant and from read-only version tools. However, 'metadata' is generic and the actual updatable field (buildUrl) is only revealed later in the parameter list, so the purpose is clear but not maximally specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus alternatives such as update_pacticipant, patch_pacticipant, or get_pacticipant_version. The 'Toolset: Branches and Versions' line provides weak grouping context but no explicit conditions, exclusions, or alternative tool routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_update_secretContract Testing: Update SecretA
Idempotent
Inspect

Update an existing secret's name, value, or description.

Toolset: Secrets

Parameters:

  • secretId (string) required: UUID of the secret to update

  • name (string): New name for the secret

  • value (string): New value for the secret

  • description (string): New description

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the secret
valueNoNew value for the secret
secretIdYesUUID of the secret to update
descriptionNoNew description

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide idempotentHint=true and destructiveHint=false, and the description does not contradict them. The description adds that this is an update of existing secret fields, but it does not disclose whether omitted fields are preserved or reset, nor any authentication or failure behavior. With annotations present, this is acceptable but not enriched.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is crisp and front-loaded, and the Toolset label adds helpful organization. However, the full parameter list duplicates the input schema almost verbatim, so not every line earns its place; the description would be stronger without this redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple: all four parameters are fully documented in the schema, required fields are clear, and annotations cover idempotency and safety. The main gaps are the lack of any note about what happens when secretId does not exist or what the update returns, but for a straightforward update this is not critical.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description's parameter list repeats the same names and descriptions as the schema and adds no deeper meaning about formats, constraints, or relationships between fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear action and resource: "Update an existing secret's name, value, or description." This distinguishes it from sibling tools like create_secret, delete_secret, get_secret, and list_secrets by emphasizing "existing" and naming the exact updatable fields.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase "existing secret" implies the tool should be used when a secret already exists and needs modification, and not for creating new secrets. However, it never explicitly names alternatives like contract-testing_create_secret or explains when to choose this over delete or create, so guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

contract-testing_update_webhookContract Testing: Update WebhookB
Idempotent
Inspect

Update an existing webhook's configuration.

Toolset: Webhooks

Parameters:

  • webhookId (string) required: UUID of the webhook to update

  • description (string): Human-readable description

  • events (array): Events that trigger this webhook

  • request (object): HTTP request to send when triggered

  • consumer (object): Restrict to a specific consumer

  • provider (object): Restrict to a specific provider

  • enabled (boolean): Whether the webhook is enabled

  • teamUuid (string): UUID of the owning team

ParametersJSON Schema
NameRequiredDescriptionDefault
eventsNoEvents that trigger this webhook
enabledNoWhether the webhook is enabled
requestNoHTTP request to send when triggered
consumerNoRestrict to a specific consumer
providerNoRestrict to a specific provider
teamUuidNoUUID of the owning team
webhookIdYesUUID of the webhook to update
descriptionNoHuman-readable description

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a mutating, non-destructive, idempotent operation, and the description adds little beyond that: it only says 'update existing webhook's configuration.' It does not clarify whether omitted optional fields are preserved or reset, what happens if the webhook does not exist, or what the response contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and clear, but the description then duplicates the entire parameter list that already exists in the schema. The bullet list adds length without value, so not every part of the description earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with eight parameters and nested objects, the description is too thin: it does not state whether the update is a full replacement or partial update, does not describe the result or errors, and does not connect to sibling tools like get_webhook or create_webhook. The schema covers parameter semantics but not operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents every parameter and its meaning. The description repeats these parameter names and short descriptions but adds no new semantics, validation detail, or guidance beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: 'Update an existing webhook's configuration.' The word 'existing' distinguishes it from creation, and the tool name plus sibling names like create_webhook and delete_webhook reinforce the intended operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'existing webhook' implies this tool is for modifying an already-created webhook rather than creating a new one, but it does not explicitly state when to choose this over alternatives or mention prerequisites such as fetching the webhook first. Usage guidance is implied, not stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_bulk_update_test_case_execution_statusQMetry: Bulk Update Test Case Execution StatusAInspect

Update execution status for individual or multiple test case runs in bulk

Toolset: Test Suites

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • entityIDs (string) required: Comma-separated IDs of Test Case Runs to update. CRITICAL: the parameter name is 'entityIDs' β€” do NOT use 'tcRunIDs', 'testCaseRunIds', 'runIds', or other variants. Accepts a number or string (e.g., 66095087 or '66095087' for single, '66095069,66095075' for bulk). To get the entityIDs - Call API 'Execution/Fetch Testcase Run ID'. From the response, get value -> data[].tcRunID

  • entityType (enum): Type of Entity to Execute: 'TCR' (Test Case Run) or 'TCSR' (Test Case Step Run) (default: "TCR")

  • qmTsRunId (string) required: Id of Test Suite Run to execute (required). CRITICAL: the parameter name is 'qmTsRunId' β€” do NOT use 'tsrunID', 'testSuiteRunId', 'tsRunID', or other variants. Accepts a number or string. To get the qmTsRunId - Call API 'Execution/Fetch Executions'. From the response, get value -> data[].tsRunID

  • runStatusID (number) required: Id of the execution status to set (required). To get the runStatusID - Call API 'Admin/Project GET info Service' From the response, get value of following attribute -> allstatus[].id Common statuses: Pass, Fail, Not Run, Blocked, WIP, etc.

  • dropID (union): Unique identifier of drop/build on which execution is to be performed (optional). To get the dropID - Call API 'Fetch Build/List' From the response, get value of following attribute -> data[].dropID

  • isAutoExecuted (enum): Set '1' for automated and '0' for manual Execution Type

  • isBulkOperation (boolean): Set true for bulk operations (multiple entityIDs), false for single execution update. Default: true if multiple comma-separated entityIDs, false otherwise

  • comments (string): Optional comments for the execution status update

  • username (string): If Part 11 Compliance is active then required for authentication

  • password (string): If Part 11 Compliance is active then required for authentication

  • qmRunObj (string): Internal QMetry run object (optional, usually empty string)

  • type (enum): Type of Entity - same as entityType (for backwards compatibility)

Output Description: JSON object with success status, updated execution details, and confirmation message

Use Cases: 1. Update single test case run status to Pass, Fail, Blocked, or Not Run 2. Bulk update multiple test case run statuses in a single operation 3. Mark all selected test case runs as Not Run for re-execution 4. Update execution status after manual test execution 5. Set execution status based on automated test results 6. Update test execution status across different test environments 7. Track test execution progress and completion 8. Manage test execution status for compliance and reporting

Examples:

  1. Update single test case run status to Failed (single execution)

{
  "entityIDs": "66095087",
  "entityType": "TCR",
  "qmTsRunId": "2720260",
  "runStatusID": 123266,
  "isBulkOperation": false
}

Expected Output: Test case run 66095087 status updated to Failed successfully

  1. Bulk update two test case runs to Pass status (bulk execution)

{
  "entityIDs": "66095069,66095075",
  "entityType": "TCR",
  "qmTsRunId": "2720260",
  "runStatusID": 123268,
  "isBulkOperation": true,
  "comments": "All test cases passed successfully"
}

Expected Output: Test case runs 66095069 and 66095075 updated to Pass status successfully

  1. Bulk update all selected test case runs to Not Run status

{
  "entityIDs": "66095069,66095075,66095081,66095087,66095093,66095099,66095105",
  "entityType": "TCR",
  "qmTsRunId": "2720260",
  "runStatusID": 123269,
  "isBulkOperation": true
}

Expected Output: 7 test case runs updated to Not Run status successfully for re-execution

  1. Update test case run with build/drop information

{
  "entityIDs": "66095087",
  "entityType": "TCR",
  "qmTsRunId": "2720260",
  "runStatusID": 123266,
  "dropID": 947,
  "isBulkOperation": false
}

Expected Output: Test case run updated with execution status and build information

  1. Update automated test execution status with automation flag

{
  "entityIDs": "66095069,66095075",
  "entityType": "TCR",
  "qmTsRunId": "2720260",
  "runStatusID": 123268,
  "isAutoExecuted": "1",
  "isBulkOperation": true,
  "comments": "Automated test execution completed"
}

Expected Output: Automated test case runs updated to Pass status with automation flag

  1. Update test case run status with Part 11 Compliance authentication

{
  "entityIDs": "66095087",
  "entityType": "TCR",
  "qmTsRunId": "2720260",
  "runStatusID": 123266,
  "username": "test.user",
  "password": "password",
  "isBulkOperation": false
}

Expected Output: Test case run status updated with Part 11 Compliance authentication

  1. Update ALL executions of test suite VKMC-TS-20 to Failed (MULTI-CALL OPERATION)

{
  "entityIDs": "66341841,66342887,66342893,66342899",
  "entityType": "TCR",
  "qmTsRunId": "2733104",
  "runStatusID": 123269,
  "isBulkOperation": true
}

Expected Output: Execution 1/4 updated. The MCP Agent will automatically repeat this operation for executions 2733205, 2733306, and 2733407 using their corresponding entityIDs.

Hints: 1. CRITICAL: entityIDs, entityType, qmTsRunId, and runStatusID are REQUIRED parameters 2. 3. CRITICAL - ALWAYS FETCH STATUS IDs FROM PROJECT INFO: 4. NEVER use hardcoded or memorized status IDs. Status IDs are PROJECT-SPECIFIC and must be fetched dynamically. 5. MANDATORY WORKFLOW BEFORE USING runStatusID: 6. 1. Call mcp_smartbear_qmetry_fetch_qmetry_project_info with the current projectKey 7. 2. Extract the 'allstatus' array from the response 8. 3. Match the desired status NAME to find its corresponding ID 9. 4. Use the fetched ID in the runStatusID parameter 10. 11. EXAMPLE STATUS ID RESOLUTION: 12. User says: 'Update status to Failed' 13. Step 1: Call FETCH_PROJECT_INFO β†’ Get allstatus array 14. Step 2: Find status where name='Failed' β†’ Extract its id property 15. Step 3: Use that id as runStatusID (e.g., 123269 for 'Failed') 16. 17. COMMON STATUS NAMES (IDs vary by project - MUST VALIDATE): 18. - 'Passed' / 'Pass' - Test case executed successfully 19. - 'Failed' / 'Fail' - Test case failed with errors 20. - 'Blocked' - Test case cannot be executed due to blockers 21. - 'Not Run' - Test case not yet executed or needs re-execution 22. - 'WIP' / 'Work In Progress' - Test case execution in progress 23. - 'Not Applicable' - Test case not applicable for this execution 24. 25. WHY THIS IS CRITICAL: 26. - Status IDs are assigned per QMetry project and are NOT universal 27. - Using wrong status ID will update tests with incorrect status 28. - Example: ID 123268 might be 'Blocked' in one project but 'Passed' in another 29. - The allstatus array is the AUTHORITATIVE source for all status mappings 30. 31. HOW TO GET entityIDs (Test Case Run IDs): 32. 1. Call API 'Execution/Fetch Testcase Run ID' (FETCH_TESTCASE_RUNS_BY_TESTSUITE_RUN tool) 33. 2. From the response, get value of following attribute -> data[].tcRunID 34. 3. Example: Single ID '66095087' or Multiple IDs '66095069,66095075,66095081' 35. 4. For bulk operations, provide comma-separated IDs without spaces 36. HOW TO GET qmTsRunId (Test Suite Run ID): 37. 1. Call API 'Execution/Fetch Executions' (FETCH_EXECUTIONS_BY_TESTSUITE tool) 38. 2. From the response, get value of following attribute -> data[].tsRunID 39. 3. Example: Test Suite Run ID might be '2720260' 40. HOW TO GET runStatusID (Execution Status ID) - DETAILED PROCESS: 41. 1. Call API 'Admin/Get info Service' (FETCH_PROJECT_INFO tool) with projectKey 42. 2. From the response, locate the 'allstatus' array 43. 3. Search for the status object where name matches your desired status (case-insensitive) 44. 4. Extract the 'id' property from the matching status object 45. 5. NEVER use example IDs from documentation - they are project-specific 46. 47. EXAMPLE allstatus ARRAY STRUCTURE: 48. allstatus: [ 49. { name: 'Passed', defaultName: 'passed', id: 123266, color: '#14892C|#FFFFFF' }, 50. { name: 'Failed', defaultName: 'failed', id: 123269, color: '#FF6666|#FFFFFF' }, 51. { name: 'Blocked', defaultName: 'blocked', id: 123268, color: '#CCCCCC|#FFFFFF' }, 52. { name: 'Not Run', defaultName: 'notrun', id: 123270, color: '#205081|#FFFFFF', isdefault: true }, 53. { name: 'Not Applicable', defaultName: 'empty', id: 123267, color: '#59AFE1|#FFFFFF' } 54. ] 55. Note: Above IDs are EXAMPLES ONLY - fetch actual IDs from your project 56. HOW TO GET dropID (Build/Drop ID) - OPTIONAL: 57. 1. Call API 'Build/List' (FETCH_BUILDS tool) 58. 2. From the response, get value of following attribute -> data[].dropID 59. 3. Example: Build/Drop ID might be 947 60. ENTITY TYPES: 61. - 'TCR' = Test Case Run (most common use case) 62. - 'TCSR' = Test Case Step Run (for step-level execution updates) 63. BULK OPERATION FLAG: 64. - isBulkOperation=false: Single test case run update (one entityID) 65. - isBulkOperation=true: Multiple test case runs update (comma-separated entityIDs) 66. - Auto-detected: If entityIDs contains comma, defaults to true; otherwise false 67. AUTOMATION FLAG (isAutoExecuted) - OPTIONAL: 68. - '1' = Automated execution (test run by automation framework) 69. - '0' = Manual execution (test run by human tester) 70. - Used for execution tracking and reporting purposes 71. PART 11 COMPLIANCE (username & password) - CONDITIONAL: 72. - Required ONLY if Part 11 Compliance is active in your QMetry instance 73. - Used for regulatory compliance and audit trail purposes 74. - Not needed for standard QMetry installations 75. COMMENTS FIELD - OPTIONAL: 76. - Add execution notes, failure reasons, or status change context 77. - Useful for tracking why status was changed 78. - Appears in execution history and audit logs 79. COMMON EXECUTION STATUS NAMES: 80. - Pass: Test case executed successfully 81. - Fail: Test case failed with errors 82. - Blocked: Test case cannot be executed due to blockers 83. - Not Run: Test case not yet executed or needs re-execution 84. - WIP: Work In Progress - test case execution in progress 85. WORKFLOW FOR USER PROMPTS: 86. 1. If user says 'execute test case run by id to failed' or 'update status to fail': 87. - Fetch test case runs to get tcRunID (entityIDs) 88. - Fetch project info to get 'Fail' status ID (runStatusID) 89. - Set isBulkOperation=false for single ID 90. 2. If user says 'bulk update test case run status to pass' or 'update all to passed': 91. - Fetch test case runs to get multiple tcRunIDs 92. - Fetch project info to get 'Pass' status ID 93. - Set isBulkOperation=true 94. - Join multiple IDs with commas (no spaces) 95. 3. If user says 'execute status to not run of given test case run ids': 96. - Use provided IDs or fetch if needed 97. - Fetch project info to get 'Not Run' status ID 98. - Set isBulkOperation based on ID count 99. 4. If the user requests updating status for ALL executions of a test suite, the agent must: 100. 1. Call FETCH_EXECUTIONS_BY_TESTSUITE to get all qmTsRunIds. 101. 2. For each qmTsRunId: 102. - Call FETCH_TESTCASE_RUNS_BY_TESTSUITE_RUN to get tcRunID (entityIDs) 103. - Fetch project info to get 'Fail' status ID (runStatusID) 104. - Call BULK_UPDATE_EXECUTION_STATUS with the corresponding qmTsRunId + tcRunID + desired runStatusID 105. 3. Repeat until all executions are updated. 106. This tool is intended to be invoked multiple times in sequence for multi-execution updates. 107. FIELD MAPPING CRITICAL NOTES: 108. - entityIDs must be comma-separated STRING (e.g., '66095069,66095075') 109. - qmTsRunId must be STRING format (e.g., '2720260') 110. - runStatusID must be NUMERIC (e.g., 123268) 111. - dropID can be numeric or string (flexible) 112. API ENDPOINT: PUT /rest/execution/runstatus/bulkupdate 113. This tool is essential for test execution management and status tracking 114. Critical for maintaining accurate test execution records and reporting 115. Use for manual test execution updates and automated test result integration 116. Essential for test execution audit trails and compliance requirements

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoType of Entity - same as entityType (for backwards compatibility)
dropIDNoUnique identifier of drop/build on which execution is to be performed (optional). To get the dropID - Call API 'Fetch Build/List' From the response, get value of following attribute -> data[<index>].dropID
commentsNoOptional comments for the execution status update
passwordNoIf Part 11 Compliance is active then required for authentication
qmRunObjNoInternal QMetry run object (optional, usually empty string)
usernameNoIf Part 11 Compliance is active then required for authentication
entityIDsYesComma-separated IDs of Test Case Runs to update. CRITICAL: the parameter name is 'entityIDs' β€” do NOT use 'tcRunIDs', 'testCaseRunIds', 'runIds', or other variants. Accepts a number or string (e.g., 66095087 or '66095087' for single, '66095069,66095075' for bulk). To get the entityIDs - Call API 'Execution/Fetch Testcase Run ID'. From the response, get value -> data[<index>].tcRunID
qmTsRunIdYesId of Test Suite Run to execute (required). CRITICAL: the parameter name is 'qmTsRunId' β€” do NOT use 'tsrunID', 'testSuiteRunId', 'tsRunID', or other variants. Accepts a number or string. To get the qmTsRunId - Call API 'Execution/Fetch Executions'. From the response, get value -> data[<index>].tsRunID
entityTypeNoType of Entity to Execute: 'TCR' (Test Case Run) or 'TCSR' (Test Case Step Run)TCR
projectKeyNoProject key - unique identifier for the projectdefault
runStatusIDYesId of the execution status to set (required). To get the runStatusID - Call API 'Admin/Project GET info Service' From the response, get value of following attribute -> allstatus[<index>].id Common statuses: Pass, Fail, Not Run, Blocked, WIP, etc.
isAutoExecutedNoSet '1' for automated and '0' for manual Execution Type
isBulkOperationNoSet true for bulk operations (multiple entityIDs), false for single execution update. Default: true if multiple comma-separated entityIDs, false otherwise

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavioral context beyond the annotations: it reveals the PUT endpoint, warns that status IDs are project-specific and must never be hardcoded, explains that incorrect IDs will update tests with wrong statuses, and documents conditional authentication. It also explains the multi-call behavior required for updating all executions of a test suite.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is heavily over-developed: parameter semantics are repeated in the schema, prose, examples, and hints; the status ID resolution guidance appears multiple times; and there are seven use cases plus seven examples, many of which are redundant. It is organized with headers and front-loaded intent, but far from appropriately sized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 13-parameter mutation tool with no output schema, the description is remarkably complete. It covers all required and optional parameters, external data dependencies, conditional auth, common status names, exact ID sourcing methods, and expected output shape, so an agent can invoke and sequence this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, but the description still adds considerable value by clarifying exact parameter naming requirements (entityIDs vs tcRunIDs, qmTsRunId vs tsRunID), value formats, defaults, how to source IDs from other APIs, and which parameters are required. The examples map parameters to realistic payloads.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence clearly identifies the action ('Update execution status') and the resource ('test case runs') with a scope qualifier ('individual or multiple... in bulk'). The use cases reinforce this by enumerating single, bulk, and automation-driven updates, making it easy to distinguish from sibling fetch/create tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit workflows for when to use this tool, including how to fetch entityIDs, qmTsRunId, and runStatusID from sibling tools. It also explains when isBulkOperation should be true/false, when Part 11 credentials are needed, and how to handle multi-execution updates by repeating calls.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_bulk_update_test_run_udfsQMetry: Bulk Update Test Run UDFsAInspect

Bulk update User Defined Field (UDF) values for one or more Test Case Runs in a test execution. Runs asynchronously in the background.

Toolset: UDF

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tcRunIDs (array) required: Array of Test Case Run IDs to update UDF values for. To get tcRunIDs β€” Call 'Fetch Test Case Runs by Test Suite Run' tool. From the response, get value of data[].tcRunID. Example: [41572006, 41572009, 41572013]

  • UDF (record<string, object>) required: Object mapping UDF field names to their new values. Each key is the UDF field name (e.g. 'test_env', 'priority_field'). Each value is an object with fieldID and value (and optionally multiSelectAction for multi-select fields). All UDF fields are optional β€” include only the fields you want to update.

Output Description: JSON object with success status, code 'CO.BULK_TC_EXECUTION_UDF_UPDATE_STARTED', and message confirming the background job was queued.

Use Cases: 1. Bulk update a string UDF (e.g. build version, environment name) for multiple test runs 2. Set a date UDF field (e.g. execution date) across multiple test case runs 3. Update a numeric UDF field (e.g. story points, priority score) in bulk 4. Set a single-select lookup UDF to a new value for multiple runs 5. Append new values to a multi-select UDF field across multiple test runs 6. Replace all existing selections in a multi-select UDF with new values 7. Update a cascading list UDF (parent + child) for multiple test runs 8. Update multiple UDF fields of different types in a single bulk operation 9. Reset a UDF field value for all runs in a test suite execution 10. Sync automated test result metadata (environment, build, platform) into UDF fields after execution

Examples:

  1. Bulk update a STRING UDF for multiple test runs

{
  "tcRunIDs": [
    41572006,
    41572009,
    41572013
  ],
  "UDF": {
    "8190_String": {
      "fieldID": 229241,
      "value": "regression-v2.1"
    }
  }
}

Expected Output: Bulk updates to execution UDF values will run in the background. Go to 'Scheduled Task' to track the process.

  1. Bulk update a DATE UDF field for multiple test runs (MM-DD-YYYY format)

{
  "tcRunIDs": [
    41572006,
    41572009
  ],
  "UDF": {
    "KN_DATE": {
      "fieldID": 229255,
      "value": "06-20-2026"
    }
  }
}

Expected Output: Bulk updates to execution UDF values will run in the background.

  1. Bulk update a NUMBER UDF field for multiple test runs

{
  "tcRunIDs": [
    41572006,
    41572009,
    41572013,
    41572015
  ],
  "UDF": {
    "defaultNum": {
      "fieldID": 229003,
      "value": 5
    }
  }
}

Expected Output: Bulk updates to execution UDF values will run in the background.

  1. Bulk update a single-select LOOKUPLIST UDF for multiple test runs

{
  "tcRunIDs": [
    41572006,
    41572009
  ],
  "UDF": {
    "8260LUP": {
      "fieldID": 228563,
      "value": 5108697
    }
  }
}

Expected Output: Bulk updates to execution UDF values will run in the background.

  1. Bulk update a MULTILOOKUPLIST UDF β€” APPEND new values to existing selections

{
  "tcRunIDs": [
    41572006,
    41572009,
    41572013
  ],
  "UDF": {
    "m_selections": {
      "fieldID": 229223,
      "value": [
        5158524,
        5158525
      ],
      "multiSelectAction": "append"
    }
  }
}

Expected Output: Bulk updates to execution UDF values will run in the background.

  1. Bulk update a MULTILOOKUPLIST UDF β€” REPLACE existing selections with new values

{
  "tcRunIDs": [
    41572006,
    41572009
  ],
  "UDF": {
    "mullt_env": {
      "fieldID": 229425,
      "value": [
        5108697,
        5108698
      ],
      "multiSelectAction": "replace"
    }
  }
}

Expected Output: Bulk updates to execution UDF values will run in the background.

  1. Bulk update a CASCADINGLIST UDF (parent + child) for multiple test runs

{
  "tcRunIDs": [
    41572006,
    41572009,
    41572013
  ],
  "UDF": {
    "cascade_mcp": {
      "fieldID": 229426,
      "value": {
        "parent": 5126498,
        "child": 5126499
      }
    }
  }
}

Expected Output: Bulk updates to execution UDF values will run in the background.

  1. Bulk update multiple UDF fields of different types in a single operation

{
  "tcRunIDs": [
    41572006,
    41572009,
    41572013,
    41572015,
    41579875
  ],
  "UDF": {
    "8190_String": {
      "fieldID": 229241,
      "value": "smoke-test"
    },
    "KN_DATE": {
      "fieldID": 229255,
      "value": "06-20-2026"
    },
    "defaultNum": {
      "fieldID": 229003,
      "value": 3
    },
    "8260LUP": {
      "fieldID": 228563,
      "value": 5108697
    },
    "m_selections": {
      "fieldID": 229223,
      "value": [
        5158524,
        5158525
      ],
      "multiSelectAction": "append"
    },
    "mullt_env": {
      "fieldID": 229425,
      "value": [
        5108697,
        5108698
      ],
      "multiSelectAction": "replace"
    },
    "cascade_mcp": {
      "fieldID": 229426,
      "value": {
        "parent": 5126498,
        "child": 5126499
      }
    }
  }
}

Expected Output: Bulk updates to execution UDF values will run in the background. Go to 'Scheduled Task' to track the process.

Hints: 1. REQUIRED: 'tcRunIDs' must be a non-empty array of numeric Test Case Run IDs. Get IDs from 'Fetch Test Case Runs by Test Suite Run' tool β†’ data[].tcRunID. 2. REQUIRED: 'UDF' must be an object with at least one field entry. Each key is the UDF field name; each value has 'fieldID' and 'value'. 3. VALUE FORMATS by field type: STRING: plain string, e.g. 'regression-build' NUMBER: number, e.g. 3 DATETIMEPICKER: date string in MM-DD-YYYY format, e.g. '06-20-2026' LOOKUPLIST (single select): numeric item ID, e.g. 5108697 MULTILOOKUPLIST (multi-select): array of item IDs, e.g. [5158524, 5158525] CASCADINGLIST: object with parent and child keys, e.g. {parent: 5126498, child: 5126499}. To get valid child IDs for a CASCADINGLIST field β€” first call 'Fetch Test Run UDF Metadata' to get the parent item IDs from lookupOptions, then call 'Fetch Cascade Child Values' with a parent item ID to get the available child IDs, then use {parent: , child: } as the value here. 4. MULTILOOKUPLIST β€” multiSelectAction rules: 'append' (default): new values are ADDED to existing selections. Use when user says 'add', 'include', 'append'. 'replace': existing selections are CLEARED and replaced with only the new values. Use when user says 'replace', 'set to', 'overwrite', 'change to'. If user does not specify, ALWAYS default to 'append'. Never assume 'replace'. 5. MULTILOOKUPLIST β€” apply multiSelectAction per field individually. Different multi-select fields in the same request can have different multiSelectAction values. 6. DATE FORMAT: Always use MM-DD-YYYY format for DATETIMEPICKER fields (e.g. '06-20-2026', not '2026-06-20'). Convert from any user-supplied date format before calling the tool. 7. ASYNC OPERATION: This API runs in the background. The success response means the job was queued, not that it completed. Tell the user to check 'Scheduled Task' in QMetry UI to track completion. 8. CRITICAL β€” FIELD IDs: The 'fieldID' for each UDF entry MUST be the exact numeric ID from QMetry's UDF definition β€” do NOT guess, infer, or fabricate fieldIDs. If the user has not provided a fieldID, ask the user to supply it or look it up in QMetry admin settings before calling this tool. Using a wrong fieldID will silently fail or update the wrong field. 9. CRITICAL β€” WORKFLOW: When user asks to bulk-update a UDF across all executions of a test suite run (e.g. tsRunID 731600), ALWAYS call 'Fetch Test Case Runs by Test Suite Run' first with that tsRunID to collect ALL tcRunIDs from the response (data[].tcRunID), THEN call this tool. Never skip the fetch step or hard-code tcRunIDs. 10. ALL UDF FIELDS ARE OPTIONAL: Only include the UDF fields the user wants to update. Do not include fields with no change. 11. tcRunIDs vs entityIDs: This tool uses 'tcRunIDs' (array of numbers). Do NOT confuse with 'Bulk Update Test Case Execution Status' which uses 'entityIDs' (comma-separated string).

ParametersJSON Schema
NameRequiredDescriptionDefault
UDFYesObject mapping UDF field names to their new values. Each key is the UDF field name (e.g. 'test_env', 'priority_field'). Each value is an object with fieldID and value (and optionally multiSelectAction for multi-select fields). All UDF fields are optional β€” include only the fields you want to update.
tcRunIDsYesArray of Test Case Run IDs to update UDF values for. To get tcRunIDs β€” Call 'Fetch Test Case Runs by Test Suite Run' tool. From the response, get value of data[<index>].tcRunID. Example: [41572006, 41572009, 41572013]
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (readOnlyHint=false), the description discloses critical runtime behavior: the operation is asynchronous and success only means the job was queued, not completed (Hints 1 and 7). It also warns that wrong fieldIDs "will silently fail or update the wrong field," describes append-vs-replace semantics including the default behavior, and explains that replace clears existing selections. This is rich behavioral context annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The tool is well-structured with headers, examples, and hints, and it front-loads the core purpose. However, it is substantially bloated: the Parameters section repeats schema descriptions almost verbatim, the 10 Use Cases largely restate the same idea with minor variations, and examples 2-8 duplicate the same call pattern with different types. It would earn a higher score with tighter editing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex nested-parameter tool with no output schema, the description is remarkably complete. It covers how to obtain tcRunIDs, how to obtain fieldIDs, per-type value formats, multiSelectAction rules, cascade child lookup workflow, async tracking via 'Scheduled Task', and even the response code 'CO.BULK_TC_EXECUTION_UDF_UPDATE_STARTED'. An agent has everything needed to invoke the tool correctly and set correct user expectations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema itself is already detailed with type-specific value formats and multiSelectAction semantics. The description adds meaningful operational meaning beyond the schema: concrete examples for each UDF type, explicit default of 'append' with instruction to never assume 'replace', date format conversion requirements, and the warning not to fabricate fieldIDs. It elevates the schema's syntactically valid payloads into safe, correct calls.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: "Bulk update User Defined Field (UDF) values for one or more Test Case Runs in a test execution." This clearly distinguishes the tool from siblings like qmetry_bulk_update_test_case_execution_status, which updates status rather than UDF values. The title, toolset label, and description all align precisely.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage guidance is explicit and actionable. Hint 9 mandates the prerequisite workflow: always call 'Fetch Test Case Runs by Test Suite Run' first to collect tcRunIDs. Hint 11 explicitly differentiates from 'Bulk Update Test Case Execution Status' (entityIDs vs tcRunIDs), and hints throughout name supporting tools like 'Fetch Test Run UDF Metadata' and 'Fetch Cascade Child Values' for obtaining required IDs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_create_cycleQMetry: Create CycleAInspect

Create a new cycle within an existing release in QMetry for test execution planning

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • cycle (object) required

Output Description: JSON object containing the created cycle ID, cycle details, and association with the release

Use Cases: 1. Create a new test cycle for a sprint within an existing release 2. Add additional testing phases to an existing release 3. Set up regression testing cycles for a specific release 4. Organize test execution by sprints, phases, or iterations 5. Create cycles with specific date ranges for milestone tracking 6. Establish test execution phases within release planning

Examples:

  1. Create a basic cycle with just a name in a release

{
  "cycle": {
    "name": "Sprint 2",
    "releaseID": 12345
  }
}

Expected Output: Cycle 'Sprint 2' created successfully in release ID 12345

  1. Create a cycle with description and dates

{
  "cycle": {
    "name": "Regression Testing Cycle",
    "description": "Full regression testing for release 2.0",
    "startDate": "15-01-2024",
    "targetDate": "31-01-2024",
    "releaseID": 12345
  }
}

Expected Output: Cycle 'Regression Testing Cycle' created with start date 15-01-2024 and target date 31-01-2024 in release 12345

  1. Create a locked cycle to prevent modifications

{
  "cycle": {
    "name": "Final QA Cycle",
    "description": "Locked cycle for final QA testing",
    "isLocked": true,
    "isArchived": false,
    "releaseID": 12345
  }
}

Expected Output: Locked cycle 'Final QA Cycle' created in release 12345 to prevent modifications

  1. Create a cycle with all details including project ID and dates

{
  "cycle": {
    "name": "Sprint 3 - Feature Testing",
    "description": "Testing new features for Sprint 3",
    "startDate": "01-02-2024",
    "targetDate": "15-02-2024",
    "isLocked": false,
    "isArchived": false,
    "projectID": 67890,
    "releaseID": 12345
  }
}

Expected Output: Cycle 'Sprint 3 - Feature Testing' created with dates and project context in release 12345

Hints: 1. CRITICAL: cycle.releaseID is REQUIRED - must provide the release ID to associate this cycle with 2. CRITICAL: cycle.name is REQUIRED - must provide a name for the cycle 3. HOW TO GET releaseID: 4. 1. Call FETCH_RELEASES_CYCLES tool to get all releases and their IDs 5. 2. From the response, get value from projects.releases[].releaseID 6. 3. Use that numeric releaseID in the cycle.releaseID parameter 7. Example: Release 'Q1 2024' might have releaseID: 12345 8. CRITICAL WORKFLOW - IF USER PROVIDES RELEASE NAME: 9. 1. User says: 'Create cycle Sprint 2 in release Q1 2024' 10. 2. You MUST first call FETCH_RELEASES_CYCLES tool to get all releases 11. 3. Search the response for release with name 'Q1 2024' 12. 4. Extract projects.releases[].releaseID from matching release 13. 5. Use that releaseID in cycle.releaseID parameter 14. 6. If release name not found, inform user and list available releases 15. Example workflow: 16. - User request: 'Create cycle Sprint 2 in Release 2.0' 17. - Step 1: Call FETCH_RELEASES_CYCLES 18. - Step 2: Find release where name = 'Release 2.0', get its releaseID (e.g., 12345) 19. - Step 3: Call CREATE_CYCLE with cycle.releaseID = 12345 20. RELEASE NAME RESOLUTION: 21. - NEVER assume or guess release IDs - always fetch from API 22. - Release names are user-defined strings (e.g., 'Q1 2024', 'Release 2.0', 'Sprint 15') 23. - Release IDs are numeric identifiers assigned by QMetry (e.g., 12345, 67890) 24. - Match release names case-insensitively when searching 25. - If multiple releases match the name, ask user to clarify or use the most recent one 26. - FETCH_RELEASES_CYCLES returns: projects.releases[] array with name and releaseID fields 27. Date format depends on QMetry instance configuration: DD-MM-YYYY or MM-DD-YYYY 28. Check your QMetry instance settings to determine the correct date format 29. If dates are in wrong format, QMetry will return an error - verify format with admin 30. projectID is optional in the cycle object - it will be auto-resolved from the project key if not provided 31. To explicitly set projectID, first call FETCH_PROJECT_INFO to get the numeric project ID 32. cycle.isLocked defaults to false if not provided - set to true to prevent modifications 33. cycle.isArchived defaults to false if not provided - set to true to archive immediately (rare) 34. Use descriptive cycle names like 'Sprint 2', 'Regression Cycle', 'Alpha Testing' for better organization 35. startDate and targetDate help with sprint planning and milestone tracking 36. Cycle hierarchy: Project β†’ Release β†’ Cycle β†’ Test Execution 37. After creating a cycle, you can associate test suites and test cases with it 38. Use FETCH_RELEASES_CYCLES tool after creation to verify the cycle was created successfully 39. DIFFERENCE FROM CREATE_RELEASE: This tool creates a cycle in an EXISTING release, while CREATE_RELEASE can create a release with an optional cycle 40. If you need to create both a release and a cycle together, use CREATE_RELEASE tool instead 41. If release doesn't exist yet, create it first with CREATE_RELEASE, then add more cycles with this tool

ParametersJSON Schema
NameRequiredDescriptionDefault
cycleYes
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only indicate a non-readonly, non-destructive write. The description adds critical behavioral context: releaseID and name are required, date format depends on instance configuration, isLocked/isArchived defaults, projectID auto-resolution, and verification via FETCH_RELEASES_CYCLES. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is well-structured with headings, examples, and numbered hints, but is verbose and repetitious: the release-name resolution workflow is restated multiple times. It could be edited down without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a nested-object creation tool with no output schema, the description covers required fields, ID resolution workflow, date format pitfalls, defaults, output shape, and sibling-tool routing. An agent has enough context to invoke it correctly in realistic release/cycle workflows.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description compensates for the schema's missing top-level cycle documentation with detailed examples and hints on resolving releaseID, auto-resolving projectID, and applying defaults. However, examples and hints use fields (description, isLocked, isArchived) that are absent from the input schema, which could confuse strict schema-validating agents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific action and resource: 'Create a new cycle within an existing release in QMetry'. It is distinguished from the sibling CREATE_RELEASE tool in the hints, so the agent can immediately identify the operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use cases enumerate concrete scenarios, and hints explicitly say when to prefer CREATE_RELEASE, when to create the release first, and when to call FETCH_RELEASES_CYCLES to resolve a release name. This is explicit when/when-not guidance with named alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_create_defect_or_issueQMetry: Create Defect or IssueAInspect

Create a new defect/issue internally in QMetry.

Toolset: Issues

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • issueType (number) required: Issue type ID (e.g. Bug, Enhancement, etc.)

  • issuePriority (number) required: Issue priority ID (e.g. High, Medium, Low, etc.)

  • summary (string) required: Summary or title of the defect/issue

  • description (string): Detailed description of the defect/issue

  • sync_with (string): External system to sync with (e.g. JIRA, QMetry, etc.)

  • issueOwner (number): Owner/user ID for the issue

  • component (array): Component IDs associated with the issue

  • affectedRelease (array): Release IDs affected by this issue

  • affectedCycles (array): Cycle IDs affected by this issue

  • tcRunID (number): Test Case Run ID to link this defect/issue to a test execution (optional)

  • environment (string): Environment where the issue was found (e.g. 'Chrome', 'Firefox', 'Production'). Free-text string β€” no ID lookup needed.

  • issueState (number): Issue status ID. Optional by default β€” QMetry allows admins to make this mandatory at the project level. Get valid IDs from project info β†’ customListObjs.issueState[index].id. Common values: Open, Reopened, Resolved, Closed.

  • udfFields (record<string, union>): Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.

Output Description: JSON object containing the new create issue with id, dfid(defectID).

Use Cases: 1. Create a basic defect/issue with just a summary 2. Set issueType, issueOwner, component (labels), environment, and affectedRelease using valid IDs from project info 3. Create defects/issues with UDF values (hobby, destination, custom fields) 4. Link defects/issues to specific test case runs using tcRunID

Examples:

  1. Create an issue with summary 'Login Issue'

{
  "name": "Login Issue",
  "issuePriority": 2231988,
  "issueType": 2231983
}

Expected Output: Issue created in summary details

  1. Create an issue with Major priority and Bug type to Bug with summary 'Login Issue'

{
  "name": "Login Issue",
  "issuePriority": 2231988,
  "issueType": 2231983
}

Expected Output: Issue created in summary details with priority and Bug type

  1. Create an issue with summary 'Login Issue' and set issueOwner to 'John Doe'

{
  "name": "Login Issue",
  "issueOwner": 15112,
  "issuePriority": 2231988,
  "issueType": 2231983
}

Expected Output: Issue created in summary details with owner, priority and Bug type

  1. Create an issue with summary 'Login Issue' and link it to test case run ID 567890

{
  "name": "Login Issue",
  "issueOwner": 15112,
  "issuePriority": 2231988,
  "issueType": 2231983,
  "tcRunID": 567890
}

Expected Output: Issue created in summary details and linked to test case run ID 567890

  1. Create an issue with summary 'Login Issue' and set description to 'User is unable to login' and owner to 'John Doe' and link it to test case run ID 567890

{
  "name": "Login Issue",
  "issueOwner": 15112,
  "issuePriority": 2231988,
  "issueType": 2231983,
  "tcRunID": 567890,
  "description": "User is unable to login"
}

Expected Output: Issue created in summary details with description, owner, priority, Bug type and linked to test case run ID 567890

  1. Create an issue with summary 'Login Issue' and set release to 'Release 1.0' and its associated all cycles and owner to 'John Doe'

{
  "name": "Login Issue",
  "issueOwner": 15112,
  "issuePriority": 2231988,
  "issueType": 2231983,
  "affectedRelease": [
    111840
  ],
  "affectedCycles": [
    112345,
    112346
  ]
}

Expected Output: Issue created in summary details with release and associated all cycles, owner

  1. Create an issue with summary 'Login Issue' and set release to 'Release 1.0' and its associated all cycle 'Cycle 1.0.1', 'Cycle 1.0.2'

{
  "name": "Login Issue",
  "issuePriority": 2231988,
  "issueType": 2231983,
  "affectedRelease": [
    111840
  ],
  "affectedCycles": [
    112345,
    112346
  ]
}

Expected Output: Issue created in summary details with release and cycles

Hints: 1. ╔══════════════════════════════════════════════════════════════════╗ 2. β•‘ STEP 0 β€” NON-NEGOTIABLE: Call 'Fetch UDF Layout' BEFORE create β•‘ 3. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 4. NEVER call 'Create Defect or Issue' without first calling 'Fetch UDF Layout' with entityType='IS', pageName='ADD'. 5. Skipping this step WILL cause 400 errors (CO.MANDATORY_FIELDS_MISSING) because mandatory fields and defaults are unknown. 6. This rule has NO exceptions β€” not even when the user only provided a name and nothing else. 7. 8. === MANDATORY PRE-CREATE CHECK (ALWAYS DO THIS FIRST) === 9. Before creating any issue, call 'Fetch UDF Layout' with entityType='IS', pageName='ADD'. 10. Response keys use 'IS' β€” same structure as TC but scoped to issue module. 11. 12. SYSTEM FIELDS mandatory check β€” use 'systemFields' array (from qmSDF.IS in newlayout): 13. Each entry: { name, label, fieldTypeName, isMandatory } 14. isMandatory=true (allowBlank=false) means field MUST have a value before creating. 15. 16. UDF FIELDS mandatory check β€” use 'fields' array (from qmUDF.IS in newlayout): 17. Each entry: { name, label, fieldTypeName, isMandatory, listName? } 18. isMandatory=true means field MUST have a value. 19. 20. DEFAULT VALUES β€” use 'defaultValues' object (from qmDefaultValue.IS in newlayout): 21. Shape: { fieldName: defaultValueId } e.g. { 'str1': 'Tony Stark', 'lookup19': 5232630, 'component': 5232632 } 22. IMPORTANT: defaultValues can contain BOTH system field defaults AND UDF field defaults β€” handle each differently: 23. β†’ SYSTEM field defaults (fields in qmSDF.IS, e.g. component/Labels, issueType, issuePriority): route to TOP-LEVEL params, NOT to udfFields. 24. - MULTILOOKUPLIST system fields (e.g. component): default is a single ID β€” wrap in array: component: [5232632]. 25. - LOOKUPLIST system fields (e.g. issueType, issuePriority): default is a single ID β€” use directly: issueType: 5232517. 26. β†’ UDF field defaults (fields in qmUDF.IS, e.g. str1, lookup19, age19): route to udfFields param. 27. IMPORTANT: QMetry's API sometimes returns 'defaultValues: {}' (empty) even when defaults exist in QMetry settings. 28. If 'defaultValues' is empty, you cannot auto-apply β€” ask user for mandatory fields without defaults. 29. RULE: isMandatory=true AND defaultValues[field.name] exists β†’ auto-use default, do NOT ask user. 30. RULE: isMandatory=true AND no defaultValues entry β†’ MUST ask user before creating. 31. RULE: isMandatory=false AND defaultValues entry exists β†’ auto-apply if user didn't specify. 32. 33. SYSTEM FIELDS mandatory check β€” additional caveat: 34. IMPORTANT: QMetry's API sometimes returns 'systemFields: []' (empty) even when system fields ARE mandatory. 35. If 'systemFields' is empty, treat the following as always mandatory: summary, issueType, issuePriority. 36. For issueType/issuePriority defaults: check 'customListObjs.issueType' / 'customListObjs.issuePriority' from Fetch Project Info β€” use first non-archived entry as fallback. 37. 38. ╔══════════════════════════════════════════════════════════════════════════╗ 39. β•‘ PRE-FLIGHT DEFAULT SWEEP β€” MANDATORY STEP BEFORE EVERY CREATE CALL β•‘ 40. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 41. After resolving mandatory fields, sweep ALL defaultValues entries: 42. For EACH key in defaultValues: 43. IF the user did not explicitly provide that field β†’ add it to the payload using the default value. 44. This applies regardless of isMandatory β€” non-mandatory defaults MUST also be auto-applied. 45. Route correctly: system fields β†’ top-level params, UDF fields β†’ udfFields. 46. MULTILOOKUPLIST system fields (e.g. component): wrap single default ID in array β†’ component: [id]. 47. Example: defaultValues = { str1: 'Tony Stark', lookup19: 5232630, component: 5232632 } 48. β†’ user only said 'create an issue named X' 49. β†’ payload MUST include: udfFields.str1='Tony Stark', udfFields.lookup19=5232630, component=[5232632] 50. β†’ WRONG to put component in udfFields β€” it is a system field and must be a top-level array param. 51. β†’ WRONG to omit component just because it is not mandatory β€” it has a default. 52. Skipping this sweep = missing fields in the created record = user-visible data loss. 53. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 54. 55. DECISION MATRIX: 56. | isMandatory | Has default | Action | 57. |-------------|-------------|-------------------------------------| 58. | true | YES | Auto-fill with default, no ask | 59. | true | NO | Ask user before creating | 60. | false | YES | Auto-fill with default β€” REQUIRED | 61. | false | NO | Skip if user didn't provide | 62. Only after ALL mandatory fields resolved AND default sweep complete β†’ proceed with create. 63. === END MANDATORY PRE-CREATE CHECK === 64. 65. === ERROR RECOVERY: CO.MANDATORY_FIELDS_MISSING === 66. If create fails with error code 'CO.MANDATORY_FIELDS_MISSING', DO NOT give up. Auto-recover: 67. 1. Parse the 'MISSING_FIELDS' list from the error response (comma-separated field labels). 68. 2. Match each label against 'fields[].label' and 'systemFields[].label' from the Fetch UDF Layout response. 69. 3. For matched UDF fields: check 'listOptions[field.listName]' for valid option IDs. 70. 4. For matched system fields (e.g. 'Priority'): check 'customListObjs.issuePriority' from project info. 71. 5. If the field has a 'defaultValues' entry: auto-fill it silently. 72. 6. If no default exists: ask the user ONLY for the missing fields by label. 73. 7. Retry create with the resolved values added to the payload. 74. NEVER ask user to 'try again' manually β€” resolve and retry automatically. 75. === END ERROR RECOVERY === 76. 77. === DATE FORMAT CHECK (MANDATORY β€” EVERY CREATE REQUEST) === 78. ALWAYS call 'Fetch QMetry Project Info' before every create request β€” not only when the user explicitly mentions a date. 79. Any UDF field could be a DATETIMEPICKER. Wrong format causes QMetry to silently discard the field value (API returns success but value is NOT stored β€” no error). 80. STEP 1: From project info, read dateTimeFormatID (e.g. 3). 81. STEP 2: Find entry in dateTimeFormatNew where id === dateTimeFormatID β†’ read its unique_value (e.g. 'yyyy-MM-dd'). 82. STEP 3: unique_value pattern: yyyy=4-digit year, MM=2-digit month (01-12), dd=2-digit day, MMM=3-letter month (Jan/Feb/...). 83. Example: id=1 β†’ MM-dd-yyyy β†’ '10-25-2000' | id=2 β†’ dd-MM-yyyy β†’ '25-10-2000' | id=3 β†’ yyyy-MM-dd β†’ '2000-10-25' | id=4 β†’ dd-MMM-yyyy β†’ '25-Oct-2000' 84. STEP 4: For EVERY DATETIMEPICKER field in the payload: parse any user-provided date and re-format it using the active unique_value pattern before sending. 85. NEVER assume a date format β€” always derive it from dateTimeFormatID. Wrong format = silent data loss. 86. === END DATE FORMAT CHECK === 87. 88. CRITICAL: summary, issueType, issuePriority are REQUIRED fields to create an issue 89. OPTIONAL SYSTEM FIELDS: issueOwner, component, affectedRelease, affectedCycles, description, environment, tcRunID 90. SYSTEM FIELD ID RESOLUTION β€” fetch project info, then use these mappings: 91. - issueType: customListObjs.issueType[].id 92. - issuePriority: customListObjs.issuePriority[].id 93. - issueOwner / owner: customListObjs.users[].id (match by name) 94. - component / labels: customListObjs.component[].id (component acts as labels β€” pass array of IDs) 95. - environment: free-text string (e.g. 'Chrome', 'Firefox', 'Production') β€” pass directly as top-level field, no ID lookup needed 96. - sync_with: customListObjs.component[].igConfigurationID or internalTrackerId 97. - tcRunID: data[].tcRunID (from 'Execution/Fetch Testcase Run ID') 98. If the user provides a issuePriority name (e.g. 'Blocker'), fetch project info, find the matching priority in customListObjs.issuePriority[index].name, and use its ID in the payload. If the name is not found, skip the issuePriority field and show a user-friendly message: 'Defect/issue created without issuePriority, as given issuePriority is not available in the current project.' 99. If the user provides an issueOwner name, fetch project info, find the matching issueOwner in customListObjs.users[index].name, and use its ID in the payload as issueOwner. If the name is not found, skip the issueOwner field and show a user-friendly message: 'Defect/issue created without issueOwner, as given issueOwner is not available in the current project.' 100. If the user provides an issue type name, fetch project info, find the matching type in customListObjs.issueType[index].name, and use its ID in the payload as issueType. If the name is not found, skip the issueType field and show a user-friendly message: 'Defect/issue created without issue type, as given type is not available in the current project.' 101. 102. === RELEASE/CYCLE ID RESOLUTION (MANDATORY WHEN USER PROVIDES RELEASE OR CYCLE) === 103. ALWAYS call 'Fetch Releases and Cycles' tool (FETCH_RELEASES_AND_CYCLES) to resolve release and cycle IDs β€” do NOT guess IDs from project info. 104. Fetch Releases and Cycles response structure: 105. releases[].releaseID β†’ use as affectedRelease value (wrap in array: [releaseID]) 106. releases[].name β†’ release display name to match against user input 107. releases[].builds[].buildID β†’ use as affectedCycles value (wrap in array: [buildID]) 108. releases[].builds[].name β†’ cycle display name to match against user input 109. PAYLOAD FORMAT: both affectedRelease and affectedCycles MUST be arrays of numeric IDs: 110. affectedRelease: [releaseID] e.g. affectedRelease: [92112] 111. affectedCycles: [buildID] e.g. affectedCycles: [130831] 112. affectedCycles: [buildID1, buildID2] multiple cycles allowed 113. WORKFLOW when user provides release/cycle name or ID: 114. 1. Call FETCH_RELEASES_AND_CYCLES to get all releases and their nested cycles (builds). 115. 2. Match user's release name/ID β†’ extract releases[].releaseID. 116. 3. Match user's cycle name/ID within that release β†’ extract releases[].builds[].buildID. 117. 4. Set affectedRelease: [releaseID] and affectedCycles: [buildID] in payload. 118. VALIDATION: If the release or cycle name/ID is not found in FETCH_RELEASES_AND_CYCLES response, skip both fields and show: 'Issue created without release/cycle association, as given release/cycle is not available in the current project.' 119. NEVER pass a single number for affectedRelease or affectedCycles β€” always wrap in array even for one ID. 120. === END RELEASE/CYCLE ID RESOLUTION === 121. 122. Ensure all IDs used are valid for the current QMetry project context 123. This tool is essential for defect management and test execution linkage 124. Helps maintain traceability between test executions and reported issues 125. Critical for quality assurance and defect lifecycle management 126. Use for creating issues directly from test execution contexts 127. 128. UDF (User Defined Fields) WORKFLOW FOR CREATE: 129. 1. Call 'Fetch UDF Layout' with entityType='IS', pageName='ADD' to discover field names, types, and list option IDs. 130. IF listOptions[field.listName] is empty after Fetch UDF Layout, the tool already tried a metadata fallback. If STILL empty, ask the user to provide the option ID from the QMetry UI β€” do NOT guess numeric IDs. 131. 2. For LOOKUPLIST fields: pick one ID from listOptions[field.listName][].id. 132. 3. For MULTILOOKUPLIST fields: pick an array of IDs. 133. 4. For CASCADINGLIST fields: pick parent ID, then call 'Fetch Cascade Child Values' for child ID. Pass { parent: parentId, child: childId }. 134. 5. Pass all UDF values via 'udfFields' param: { fieldName: value }. 135. 6. Mandatory UDF fields (isMandatory=true) MUST be included or create will fail.

ParametersJSON Schema
NameRequiredDescriptionDefault
summaryYesSummary or title of the defect/issue
tcRunIDNoTest Case Run ID to link this defect/issue to a test execution (optional)
componentNoComponent IDs associated with the issue
issueTypeYesIssue type ID (e.g. Bug, Enhancement, etc.)
sync_withNoExternal system to sync with (e.g. JIRA, QMetry, etc.)
udfFieldsNoFlat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.
issueOwnerNoOwner/user ID for the issue
issueStateNoIssue status ID. Optional by default β€” QMetry allows admins to make this mandatory at the project level. Get valid IDs from project info β†’ customListObjs.issueState[index].id. Common values: Open, Reopened, Resolved, Closed.
projectKeyNoProject key - unique identifier for the projectdefault
descriptionNoDetailed description of the defect/issue
environmentNoEnvironment where the issue was found (e.g. 'Chrome', 'Firefox', 'Production'). Free-text string β€” no ID lookup needed.
issuePriorityYesIssue priority ID (e.g. High, Medium, Low, etc.)
affectedCyclesNoCycle IDs affected by this issue
affectedReleaseNoRelease IDs affected by this issue

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only convey readOnly=false and destructive=false. The description goes far beyond by disclosing the mandatory pre-create 'Fetch UDF Layout' step, the CO.MANDATORY_FIELDS_MISSING auto-recovery protocol, silent date-format data loss, and the default-value sweep behavior. These behavioral traits are not derivable from the annotations or schema and are critical for correct invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a very long, repetitive wall of text. Warnings about 'Fetch UDF Layout' appear multiple times, examples 1 and 2 are almost identical, and ASCII-box headers add visual noise. While some length is justified for a complex tool, the redundancy and excessive formatting make it significantly less concise than it should be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 14-parameter creation tool with no output schema, the description is remarkably complete: it covers prerequisites, mandatory-field decision rules, default-value handling, error recovery, release/cycle ID resolution, date formatting, UDF workflows, and the return shape (id, dfid). An agent can correctly execute the tool with this information, despite the readability issues.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema description coverage is 100%, the description adds substantial meaning beyond the schema: environment is explicitly 'free-text' with no ID lookup, affectedRelease/affectedCycles must always be wrapped in arrays, issueType/issuePriority/issueOwner IDs come from customListObjs, and udfFields formats are fully enumerated by fieldTypeName. The examples also demonstrate real payload shapes, including name-to-ID resolution.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Create a new defect/issue internally in QMetry.' The 'Toolset: Issues' label and the use-case list reinforce that this tool creates issues, clearly distinguishing it from sibling tools like qmetry_update_issue or qmetry_fetch_defects_or_issues. The name alone is also highly descriptive.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear contextual guidance, including 'Use for creating issues directly from test execution contexts' and a detailed use-case list ranging from basic creation to UDF-linked and test-run-linked defects. However, it never explicitly tells the agent when NOT to use this tool or when to prefer an alternative like qmetry_update_issue, so it stops short of a full exclusion-based usage guide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_create_releaseQMetry: Create ReleaseAInspect

Create a new release in QMetry with optional cycle for test planning and execution tracking

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • release (object) required

  • cycle (object): Optional cycle to create within the release

Output Description: JSON object containing the created release ID, release details, and cycle information if provided

Use Cases: 1. Create a new release for a major product version (e.g., v2.0, Q1 Release) 2. Create a release with an initial cycle for immediate test planning 3. Set up release dates for sprint planning and milestone tracking 4. Organize test execution by product versions and cycles 5. Create release hierarchy for better test planning and reporting 6. Establish test execution phases with releases and cycles

Examples:

  1. Create a basic release with just a name

{
  "release": {
    "name": "Release 2.0"
  }
}

Expected Output: Release 'Release 2.0' created successfully with generated release ID

  1. Create a release with description and dates

{
  "release": {
    "name": "Q1 2024 Release",
    "description": "First quarter release for 2024",
    "startDate": "01-01-2024",
    "targetDate": "31-03-2024"
  }
}

Expected Output: Release 'Q1 2024 Release' created with start date 01-01-2024 and target date 31-03-2024

  1. Create a release with an initial cycle

{
  "release": {
    "name": "Release 3.0",
    "description": "Major product update"
  },
  "cycle": {
    "name": "Sprint 1",
    "isLocked": false,
    "isArchived": false
  }
}

Expected Output: Release 'Release 3.0' created with cycle 'Sprint 1' for test execution planning

  1. Create a release with all details

{
  "release": {
    "name": "Summer 2024 Release",
    "description": "Summer product release with new features",
    "startDate": "01-06-2024",
    "targetDate": "31-08-2024"
  },
  "cycle": {
    "name": "Beta Testing Cycle",
    "isLocked": false
  }
}

Expected Output: Release 'Summer 2024 Release' created with dates and 'Beta Testing Cycle' for test execution

Hints: 1. CRITICAL: release.name is REQUIRED - must provide a name for the release 2. Date format depends on QMetry instance configuration: DD-MM-YYYY or MM-DD-YYYY 3. Check your QMetry instance settings to determine the correct date format 4. If dates are in wrong format, QMetry will return an error - verify format with admin 5. projectID is optional in the release object - it will be auto-resolved from the project key if not provided 6. To explicitly set projectID, first call FETCH_PROJECT_INFO to get the numeric project ID 7. cycle parameter is completely optional - omit it if you only want to create a release 8. If providing cycle, cycle.name is REQUIRED 9. cycle.isLocked defaults to false if not provided - set to true to prevent modifications 10. cycle.isArchived defaults to false if not provided - set to true to archive immediately (rare) 11. Releases can have multiple cycles added later using other tools 12. Use descriptive release names like 'Release 2.0', 'Q1 2024', 'Sprint 15' for better organization 13. startDate and targetDate help with sprint planning and milestone tracking 14. Creating a release with a cycle is useful for immediate test planning after release creation 15. Release hierarchy: Project β†’ Release β†’ Cycle β†’ Test Execution 16. After creating a release, you can associate test suites and test cases with it 17. Use FETCH_RELEASES_CYCLES tool after creation to verify the release was created successfully

ParametersJSON Schema
NameRequiredDescriptionDefault
cycleNoOptional cycle to create within the release
releaseYes
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal this is a mutating, non-idempotent operation. The description adds useful behavioral context beyond the annotations: date format may depend on QMetry instance configuration, invalid dates will produce an error, projectID can be auto-resolved from projectKey, and cycle defaults are false. It does not cover duplicate names or permission requirements, but it adds meaningful operational detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded with the core purpose, but it is excessively long: six use cases, four examples, and seventeen hints contain repetition and generic advice. Some sections, such as the repeated "created successfully" expected outputs, could be condensed without losing value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the nested release/cycle objects, ambiguous date formats, and no output schema, the description supplies everything needed: output shape, required fields, defaults, error behavior, hierarchy context, and a verification step. The only minor gap is not naming qmetry_create_cycle explicitly for adding cycles later, but the description still gives enough guidance to invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 67%, so the description carries extra responsibility for parameter meaning. The hints compensate by clarifying required release.name, the date format ambiguity, optional projectID resolution, cycle.name being required when cycle is provided, and default values for isLocked/isArchived. The examples further demonstrate real payload shapes, going beyond the schema's field-level descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: "Create a new release in QMetry," and clarifies the optional nested cycle. It clearly communicates the tool's primary purpose, though it does not explicitly differentiate it from the sibling qmetry_create_cycle, so the distinction is left to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use cases and hints provide clear context for when to use this tool, such as creating a release for a product version or creating a release with an initial cycle. It also gives conditional guidance like "omit [cycle] if you only want to create a release" and notes that cycles can be added later with other tools, but it never names qmetry_create_cycle as the explicit alternative, so exclusions are slightly underspecified.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_create_requirementQMetry: Create RequirementAInspect

Create a new requirement in QMetry with metadata and release/cycle mapping.

Toolset: Requirements

Parameters:

  • name (string) required: Requirement name (required).

  • priority (number): Priority ID of the requirement.

  • component (array): Component (Label) IDs associated with the requirement.

  • requirementOwner (number): Owner ID of the requirement.

  • requirementState (number): State ID of the requirement.

  • releaseCycleMapping (array): Release/cycle mapping for the requirement.

  • description (string): Description of the requirement.

  • associateRelCyc (boolean): Whether to associate the release/cycle mapping.

  • rqFolderId (string): Requirement folder ID - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific folder ID. System will fetch project info using the projectKey and extract rootFolders.RQ.id automatically. Manual folder ID only needed if you want to target a specific sub-folder.

  • scope (string): Scope of the requirement, usually 'project'. (default: "project")

  • udfFields (record<string, union>): Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.

Output Description: JSON object containing the new requirement ID, summary, and creation metadata.

Use Cases: 1. Create a basic requirement with just a name 2. Add detailed metadata like priority, component, and description to a requirement 3. Associate a requirement with a specific release/cycle for planning 4. Set requirementOwner and requirementState using valid IDs from project info 5. Create a requirement in a specific folder using rqFolderId 6. Set custom fields (UDFs) values on the new requirement

Examples:

  1. Create a requirement in the root folder (auto-resolved)

{
  "name": "New login requirement"
}

Expected Output: Requirement created in the root requirement folder with ID and summary details

  1. Create a requirement with metadata

{
  "name": "New login requirement",
  "priority": 688864,
  "component": [
    689030
  ],
  "requirementOwner": 8,
  "requirementState": 688912,
  "description": "Users must be able to log in with SSO.",
  "associateRelCyc": true,
  "releaseCycleMapping": [
    {
      "release": 1628,
      "cycle": [
        1839,
        1840
      ],
      "version": 1
    }
  ]
}

Expected Output: Requirement created with metadata. Example uses: priority=688864, component=[689030], requirementOwner=8, requirementState=688912 β€” resolve these IDs from project info's customListObjs before use.

Hints: 1. 'name' is the only mandatory field β€” sets the requirement's title/summary. Every other field is optional. 2. 'description' is a free-text field (supports HTML) for detailed requirement information. 3. 'component' is an array of Component (Label) IDs β€” resolve names to IDs from project info before use. 4. If rqFolderId is not provided, it will be auto-resolved to the root requirement folder using project info (rootFolders.RQ.id). 5. To get valid values for priority, component, requirementOwner (owner), requirementState (state), call the 'Admin/Get info Service' API (FETCH_PROJECT_INFO tool) and use the returned customListObjs IDs. 6. If the user provides a name instead of an ID for owner/state/priority/component, fetch project info, find the matching entry by name in the relevant customListObjs list, and use its ID. If not found, skip that field and tell the user it was omitted because the value wasn't available in the current project. 7. Release/cycle mapping is optional. If the user wants to associate a release and cycle, set associateRelCyc: true and provide releaseCycleMapping. 8. HARD GATE (not configurable): before creating, this tool checks project info for isExtTrackerConfigured, extTrackerType, and isRQConfigured. If the project has an external tracker configured (isExtTrackerConfigured=true) and isRQConfigured=true, and the tracker is Jira (extTrackerType=1) or Azure (extTrackerType=3), the create is refused with an error β€” do NOT retry or work around this. Tell the user requirements for this project must be created directly in the external system (Jira or Azure), then synced into QMetry. 9. 10. UDF (User Defined Fields) WORKFLOW FOR CREATE: 11. 1. Call 'Fetch UDF Layout' with entityType='RQ', pageName='ADD' to discover field names, types, and list option IDs. 12. IF listOptions[field.listName] is empty after Fetch UDF Layout, the tool already tried a metadata fallback. If STILL empty, ask the user to provide the option ID from the QMetry UI β€” do NOT guess numeric IDs. 13. 2. For LOOKUPLIST fields: pick one ID from listOptions[field.listName][].id. 14. 3. For MULTILOOKUPLIST fields: pick an array of IDs. 15. 4. For CASCADINGLIST fields: pick parent ID, then call 'Fetch Cascade Child Values' for child ID. Pass { parent: parentId, child: childId }. 16. 5. Pass all UDF values via 'udfFields' param: { fieldName: value }. 17. 6. Mandatory UDF fields (isMandatory=true) MUST be included or create will fail.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesRequirement name (required).
scopeNoScope of the requirement, usually 'project'.project
priorityNoPriority ID of the requirement.
componentNoComponent (Label) IDs associated with the requirement.
udfFieldsNoFlat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.
rqFolderIdNoRequirement folder ID - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific folder ID. System will fetch project info using the projectKey and extract rootFolders.RQ.id automatically. Manual folder ID only needed if you want to target a specific sub-folder.
descriptionNoDescription of the requirement.
associateRelCycNoWhether to associate the release/cycle mapping.
requirementOwnerNoOwner ID of the requirement.
requirementStateNoState ID of the requirement.
releaseCycleMappingNoRelease/cycle mapping for the requirement.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description reveals important runtime behaviors: rqFolderId is auto-resolved unless a specific sub-folder is given, mandatory UDF fields will cause failure if omitted, and an external-tracker hard gate refuses creation for Jira/Azure and says not to retry. No stated behavior contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The content is front-loaded with a clear purpose and organized into useful sections, but it is verbose and repetitive: the Parameters section largely duplicates the schema, hints overlap with examples, and the numbering becomes malformed ('9.', '10.', '11.', then nested '11. 1.' etc.). Dense information is valuable, but this could be trimmed substantially.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 11-parameter create operation with nested objects, UDF handling, ID-resolution requirements, and an external-tracker gate, the description covers everything needed: required fields, optional metadata, prerequisite data sources, UDF discovery, expected output shape, and hard failure conditions. No critical invocation detail is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description substantially enriches parameter understanding: rqFolderId gets an auto-resolution explanation, udfFields gets fieldType-specific value shapes and a step-by-step discovery workflow, and releaseCycleMapping has a concrete example. This goes well beyond the schema's terse property descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific action and resource: 'Create a new requirement in QMetry with metadata and release/cycle mapping.' The 'Toolset: Requirements' label further disambiguates it from the many fetching/updating sibling tools, and the name itself differentiates it from qmetry_update_requirement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases section lists six concrete situations where this tool is appropriate, and the HARD GATE explicitly tells the agent when NOT to create in QMetry and to direct the user to Jira/Azure instead. Hints also specify prerequisite tool calls like FETCH_PROJECT_INFO and Fetch UDF Layout, plus behavior when a provided name cannot be resolved to an ID.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_create_test_caseQMetry: Create Test CaseAInspect

Create a new test case in QMetry with steps, metadata, and release/cycle mapping.

Toolset: Test Cases

Parameters:

  • tcFolderID (string): Test Case folder ID - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific folder ID. System will fetch project info using the projectKey and extract rootFolders.TC.id automatically. Manual folder ID only needed if you want to target a specific sub-folder.

  • skipSteps (boolean): Set to true ONLY when the user explicitly says they do NOT want steps created (e.g. 'create test case without steps', 'no steps', 'skip steps'). When true, the 'steps' field must be omitted entirely. When false or absent (the default), steps MUST always be included β€” auto-generate them from context if the user did not provide them.

  • steps (array): STEPS RULE β€” include this field unless the user explicitly says NOT to create steps.

NEVER send steps: [] (empty array) β€” always send at least 1 valid step object. Omit this field entirely (and set skipSteps: true) ONLY when the user explicitly asks to skip steps.

HOW TO POPULATE:

  • If user explicitly provides steps: parse each step into { orderId, description, inputData?, expectedOutcome? }.

  • If user does NOT provide steps (and did not say to skip them): auto-generate meaningful steps based on the test case name, description, and context. Use your knowledge to infer 2-5 logical, realistic steps for the feature or flow being tested. Example: name='Login Test Case' β†’ [{orderId:1, description:'Navigate to login page'}, {orderId:2, description:'Enter credentials'}, {orderId:3, description:'Submit and verify success'}]

  • If user explicitly said NOT to create steps: omit this field and set skipSteps: true.

STEP DEFAULT VALUES: After building the steps array, check 'stepDefaultValues' from Fetch UDF Layout. stepDefaultValues shape: { fieldName: defaultValue } For each step: for each key in stepDefaultValues, if the user did NOT explicitly provide a value for that field β†’ add it to step.UDF with the default value. Auto-apply silently β€” do NOT ask the user.

Step object fields: orderId (required): sequential integer starting at 1 description (required): step action text inputData (optional): test data for this step expectedOutcome (optional): what should happen after this step UDF (optional): step-level custom fields β€” auto-fill defaults from stepDefaultValues tcStepID (omit on create β€” only used when updating existing steps)

  • name (string) required

  • priority (number)

  • component (array)

  • testcaseOwner (number)

  • testCaseState (number)

  • testCaseType (number)

  • estimatedTime (number): Estimated execution time in SECONDS (e.g. 3600 = 1 hour, 36000 = 10 hours). NOT minutes.

  • testingType (number)

  • description (string)

  • associateRelCyc (boolean)

  • releaseCycleMapping (array): Release/cycle mapping. Set associateRelCyc=true when providing this. version field defaults to 1 if not specified.

  • udfFields (record<string, union>): Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.

Output Description: JSON object containing the new test case ID, summary, and creation metadata.

Use Cases: 1. Create a basic test case with just a name and folder 2. Add detailed steps with custom fields (UDFs) to a test case 3. Associate test case with specific release/cycle for planning 4. Set priority, owner, component, and other metadata using valid IDs from project info 5. Create test cases for automation or manual testing types 6. Add test case to a specific folder using tcFolderID 7. Include estimated execution time and description 8. Map test case to multiple cycles/releases

Examples:

  1. MOST COMMON: User provides only a name β€” steps are auto-generated from context.

{
  "name": "Login Test Case",
  "steps": [
    {
      "orderId": 1,
      "description": "Navigate to the login page"
    },
    {
      "orderId": 2,
      "description": "Enter valid username and password"
    },
    {
      "orderId": 3,
      "description": "Click the Login button"
    },
    {
      "orderId": 4,
      "description": "Verify successful login and dashboard is shown"
    }
  ]
}

Expected Output: Test case created with auto-generated steps inferred from the test case name 'Login Test Case'.

  1. Create test case with metadata only β€” steps auto-generated from context

{
  "tcFolderID": "102653",
  "name": "Login Test Case",
  "priority": 2025268,
  "testCaseState": 2025271,
  "estimatedTime": 3600,
  "description": "Verifies login flow",
  "steps": [
    {
      "orderId": 1,
      "description": "Navigate to the login page"
    },
    {
      "orderId": 2,
      "description": "Enter valid credentials"
    },
    {
      "orderId": 3,
      "description": "Submit the login form"
    },
    {
      "orderId": 4,
      "description": "Verify redirection to the home page"
    }
  ]
}

Expected Output: Test case created with metadata and auto-generated steps based on description 'Verifies login flow'.

  1. SCENARIO 1: User explicitly asked for steps β€” 'create test case with step 1 - Go to login page, step 2 - enter credentials'

{
  "tcFolderID": "102653",
  "name": "Login Flow Test",
  "steps": [
    {
      "orderId": 1,
      "description": "Go to login page"
    },
    {
      "orderId": 2,
      "description": "Enter credentials"
    }
  ]
}

Expected Output: Test case created with 2 steps because user explicitly mentioned steps in prompt.

  1. SCENARIO 1: User provided steps with full metadata (steps explicitly mentioned in prompt)

{
  "tcFolderID": "102653",
  "name": "Test Case 1",
  "steps": [
    {
      "orderId": 1,
      "description": "First Step",
      "inputData": "First Data",
      "expectedOutcome": "First Outcome",
      "UDF": {
        "customField1": "Custom Field Data A",
        "customField2": "Custom Field Data B"
      }
    }
  ],
  "priority": 2025268,
  "component": [
    2025328
  ],
  "testcaseOwner": 1467,
  "testCaseState": 2025271,
  "testCaseType": 2025282,
  "estimatedTime": 10,
  "description": "Description",
  "testingType": 2025275,
  "associateRelCyc": true,
  "releaseCycleMapping": [
    {
      "release": 14239,
      "cycle": [
        21395
      ],
      "version": 1
    }
  ]
}

Expected Output: Test case created with steps because user explicitly requested steps. All metadata populated.

  1. SCENARIO 3: User explicitly says no steps β€” 'create a test case without any steps'

{
  "name": "Login Test Case",
  "skipSteps": true
}

Expected Output: Test case created with no steps because user explicitly asked to skip them.

  1. User provides name only β€” steps auto-generated from test case name context

{
  "name": "Password Reset Test Case",
  "steps": [
    {
      "orderId": 1,
      "description": "Navigate to the login page"
    },
    {
      "orderId": 2,
      "description": "Click on 'Forgot Password' link"
    },
    {
      "orderId": 3,
      "description": "Enter registered email address and submit the reset form"
    },
    {
      "orderId": 4,
      "description": "Click reset link from email and set a new password"
    },
    {
      "orderId": 5,
      "description": "Verify login succeeds with the new password"
    }
  ]
}

Expected Output: Test case created with 5 auto-generated steps inferred from the name 'Password Reset Test Case'. No user-provided steps β€” LLM generated them from context.

Hints: 1. ╔══════════════════════════════════════════════════════════════════════════════╗ 2. β•‘ STEPS RULE β€” INCLUDE STEPS UNLESS USER EXPLICITLY ASKS TO SKIP THEM β•‘ 3. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 4. 5. Include 'steps' in the payload unless the user explicitly says NOT to create steps. 6. NEVER send steps: [] (empty array) β€” either send at least 1 valid step object or omit steps with skipSteps: true. 7. 8. HOW TO POPULATE STEPS: 9. 10. SCENARIO 1 β€” User explicitly provides steps in their prompt. 11. Trigger phrases: 'with steps', 'step 1 -', 'add steps', 'include steps', 'following steps', 'these steps'. 12. Action: Parse the user's step text into { orderId, description, inputData?, expectedOutcome? } objects. 13. Example: 'create test case, step 1 - open browser, step 2 - click login' 14. β†’ steps: [{ orderId: 1, description: 'open browser' }, { orderId: 2, description: 'click login' }] 15. 16. SCENARIO 2 β€” User does NOT mention steps at all. 17. Action: Generate meaningful steps based on the test case name, description, and all other context provided. 18. Use your knowledge of the feature/flow being tested to infer logical, realistic test steps. 19. Always include 2-5 steps that make sense for the test case. 20. Example: name='Login Test Case' β†’ steps: [{orderId:1, description:'Navigate to login page'}, {orderId:2, description:'Enter credentials'}, {orderId:3, description:'Submit form'}, {orderId:4, description:'Verify login success'}] 21. 22. SCENARIO 3 β€” User explicitly says NOT to create steps. 23. Trigger phrases: 'no steps', 'without steps', 'skip steps', 'don't add steps', 'without any steps'. 24. Action: Set skipSteps: true and omit the 'steps' field entirely from the payload. 25. Example: 'create test case Login Test Case without steps' β†’ { name: 'Login Test Case', skipSteps: true } 26. 27. STEP DEFAULT VALUES β€” apply after step generation (scenarios 1 and 2 only): 28. After building the steps array (from user input OR auto-generated), check 'stepDefaultValues' from Fetch UDF Layout. 29. stepDefaultValues shape: { fieldName: defaultValue } β€” e.g. { 'lookup19': 5232630 } 30. For EACH step, for EACH key in stepDefaultValues: 31. IF the user has NOT explicitly specified a value for that field in the step β†’ set it to the default value. 32. This applies to step UDF fields (step.UDF) that have defaults configured. 33. Auto-apply silently β€” do NOT ask the user. 34. Example: stepDefaultValues = { status: 5232630 } β†’ every step's UDF.status = 5232630 unless user gave a different value. 35. 36. DECISION TABLE: 37. | Situation | Action | 38. |------------------------------------|-----------------------------------------------------------------------------| 39. | User provided steps | Use steps from user's prompt; fill step UDF defaults from stepDefaultValues | 40. | User did not mention steps | Auto-generate steps from context; fill step UDF defaults from stepDefaultValues | 41. | User explicitly said NO steps | Set skipSteps: true, omit 'steps' field entirely | 42. 43. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 44. 45. ╔══════════════════════════════════════════════════════════════════╗ 46. β•‘ STEP 0 β€” NON-NEGOTIABLE: Call 'Fetch UDF Layout' BEFORE create β•‘ 47. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 48. NEVER call 'Create Test Case' without first calling 'Fetch UDF Layout' with entityType='TC', pageName='ADD'. 49. Skipping this step WILL cause 400 errors (CO.MANDATORY_FIELDS_MISSING) because mandatory fields and defaults are unknown. 50. This rule has NO exceptions β€” not even when the user only provided a name and nothing else. 51. 52. === MANDATORY PRE-CREATE CHECK === 53. 54. SYSTEM FIELDS mandatory check β€” use 'systemFields' array from Fetch UDF Layout: 55. Each entry: { name, label, fieldTypeName, isMandatory } 56. isMandatory=true (allowBlank=false in QMetry) means the field MUST have a value. 57. IMPORTANT: QMetry's API sometimes returns 'systemFields: []' (empty) even when system fields ARE mandatory. 58. If 'systemFields' is empty, treat the following as always mandatory: name (Summary), testCaseState (Status). 59. For testCaseState default: check 'customListObjs.testCaseState' from Fetch Project Info β€” use first non-archived entry as fallback. 60. 61. UDF FIELDS mandatory check β€” use 'fields' array from Fetch UDF Layout: 62. Each entry: { name, label, fieldTypeName, isMandatory, listName? } 63. IMPORTANT: QMetry's API sometimes returns isMandatory=false for fields that ARE enforced as mandatory. 64. The 'isMandatory' flag is a hint, not a guarantee. Trust the actual API error over this flag. 65. When isMandatory=true: field MUST have a value. 66. 67. DEFAULT VALUES β€” use 'defaultValues' object from Fetch UDF Layout: 68. Shape: { fieldName: defaultValueId } e.g. { 'lookup19': 5232630, 'estimatedTime': 18305.0, 'priority': 5232497 } 69. These are pre-configured QMetry defaults. ALWAYS auto-apply them β€” even when user did not mention the field. 70. IMPORTANT: QMetry's API sometimes returns 'defaultValues: {}' (empty) even when defaults exist in QMetry settings. 71. If 'defaultValues' is empty, you cannot auto-apply β€” ask user for mandatory fields without defaults. 72. RULE: if isMandatory=true AND defaultValues[field.name] exists β†’ use default, do NOT ask user. 73. RULE: if isMandatory=true AND NO defaultValues[field.name] β†’ MUST ask user before creating. 74. RULE: if isMandatory=false AND defaultValues[field.name] exists β†’ auto-apply default if user didn't specify. 75. RULE: if isMandatory=false AND no default β†’ skip if user didn't provide. 76. 77. ╔══════════════════════════════════════════════════════════════════════════╗ 78. β•‘ PRE-FLIGHT DEFAULT SWEEP β€” MANDATORY STEP BEFORE EVERY CREATE CALL β•‘ 79. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 80. After resolving mandatory fields, do a full sweep of ALL defaultValues entries: 81. For EACH key in defaultValues: 82. IF the user did not explicitly provide that field β†’ add it to the payload using the default value. 83. This applies regardless of isMandatory β€” non-mandatory defaults MUST also be auto-applied. 84. Example: defaultValues = { lookup19: 5232630, estimatedTime: 18305, priority: 5232497 } 85. β†’ user only said 'create a test case named X' 86. β†’ payload MUST include: lookup19=5232630, estimatedTime=18305, priority=5232497 87. β†’ WRONG to omit priority/estimatedTime just because they are not mandatory β€” they have defaults. 88. Skipping this sweep = missing fields in the created record = user-visible data loss. 89. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 90. 91. TEST CASE STEPS β€” always required (see STEPS RULE above): 92. stepSystemFields: built-in step fields { name, label, fieldTypeName, isMandatory } 93. stepFields: step-level UDF fields { name, label, fieldTypeName, isMandatory, listName? } 94. Mandatory stepFields UDFs (isMandatory=true) must be filled in step.UDF β€” use stepDefaultValues for defaults, else a placeholder value. 95. STEP DEFAULT VALUES: use 'stepDefaultValues' object from Fetch UDF Layout β€” same auto-fill logic as defaultValues. 96. For EACH key in stepDefaultValues: if user has NOT explicitly provided a value for that step field β†’ auto-apply the default silently. 97. This applies to all steps (both user-provided and auto-generated). 98. 99. DECISION MATRIX: 100. | isMandatory | Has default | Action | 101. |-------------|-------------|-------------------------------------| 102. | true | YES | Auto-fill with default, no ask | 103. | true | NO | Ask user before creating | 104. | false | YES | Auto-fill with default β€” REQUIRED | 105. | false | NO | Skip if user didn't provide | 106. 107. Only after ALL mandatory fields are resolved AND default sweep is complete β†’ proceed with create. 108. === END MANDATORY PRE-CREATE CHECK === 109. 110. === ERROR RECOVERY: CO.MANDATORY_FIELDS_MISSING === 111. If create fails with error code 'CO.MANDATORY_FIELDS_MISSING', DO NOT give up. Auto-recover: 112. 1. Parse the 'MISSING_FIELDS' list from the error response (comma-separated field labels). 113. 2. Match each label against 'fields[].label' and 'systemFields[].label' from the Fetch UDF Layout response. 114. 3. For matched UDF fields: check 'listOptions[field.listName]' for valid option IDs. 115. 4. For matched system fields (e.g. 'Status'): check 'customListObjs.testCaseState' from project info. 116. 5. If the field has a 'defaultValues' entry: auto-fill it silently. 117. 6. If no default exists: ask the user ONLY for the missing fields by label. 118. 7. Retry create with the resolved values added to the payload. 119. NEVER ask user to 'try again' manually β€” resolve and retry automatically. 120. === END ERROR RECOVERY === 121. 122. === DATE FORMAT CHECK (MANDATORY β€” EVERY CREATE REQUEST) === 123. ALWAYS call 'Fetch QMetry Project Info' before every create request β€” not only when the user explicitly mentions a date. 124. Any UDF field could be a DATETIMEPICKER. Wrong format causes QMetry to silently discard the field value (API returns success but value is NOT stored β€” no error). 125. Project info response contains 'dateTimeFormatID' and 'dateTimeFormatNew' array. 126. STEP 1: From project info, read dateTimeFormatID (e.g. 3). 127. STEP 2: Find entry in dateTimeFormatNew where id === dateTimeFormatID β†’ read its unique_value (e.g. 'yyyy-MM-dd'). 128. STEP 3: unique_value is the Java/QMetry format pattern. Mapping: 129. yyyy = 4-digit year | MM = 2-digit month (01-12) | dd = 2-digit day | MMM = 3-letter month (Jan/Feb/...) 130. Example: id=1 β†’ MM-dd-yyyy β†’ '10-25-2000' | id=2 β†’ dd-MM-yyyy β†’ '25-10-2000' | id=3 β†’ yyyy-MM-dd β†’ '2000-10-25' | id=4 β†’ dd-MMM-yyyy β†’ '25-Oct-2000' 131. STEP 4: For EVERY DATETIMEPICKER field in the payload (user-provided OR from defaultValues): 132. - Parse the date regardless of what format the user typed 133. - Re-format it using the active unique_value pattern 134. - Send the re-formatted string to the API 135. Examples with unique_value='yyyy-MM-dd': 136. User says '25 Dec 2024' β†’ send '2024-12-25' 137. User says '12/25/2024' β†’ send '2024-12-25' 138. Default value is a date string '2024-12-25' β†’ already correct, keep it 139. NEVER send a date in a format different from the project's active dateTimeFormatID format. 140. Fetch project info ONCE per create/update operation and reuse dateTimeFormatID for all date fields. 141. === END DATE FORMAT CHECK === 142. 143. If tcFolderID is not provided, it will be auto-resolved to the root test case folder using project info (rootFolders.TC.id). 144. To get valid values for priority, owner, component, etc., call the project info tool and use the returned customListObjs IDs. 145. STALE / NOT-FOUND ID RECOVERY (applies to ALL system fields β€” priority, component/label, owner, status, testCaseType, testingType, release, cycle): If the user references a value by name and it is NOT found in your current cached project info data, DO NOT give up or skip the field immediately. Instead: call 'Fetch QMetry Project Info' fresh (no arguments needed) to get the latest snapshot, then re-scan the relevant customListObjs list. This is mandatory when: (a) the user just added a new label/priority/status/user in QMetry UI, or (b) the cached info is from an earlier turn. Only skip + show a friendly message if the value is still missing AFTER the fresh fetch. 146. FOLDER ID RESOLUTION (tcFolderID): Project info only exposes the ROOT folder ID (rootFolders.TC.id). Sub-folder IDs are NOT returned by project info. If the user specifies a sub-folder (e.g. 'Folder 1'), use this resolution order: 1. Check if the user already provided the numeric folder ID β€” use it directly. 2. Try fetching test cases with folderPath='' and scope='folder' β€” if a TC exists there, its folder context confirms the path, but the ID is still needed from the UI. 3. If still unresolved, ask the user: 'Please provide the numeric folder ID for "". You can find it in the QMetry URL when browsing that folder (look for folderId=XXXXX).' NEVER silently fall back to root folder when the user explicitly named a sub-folder β€” always ask first. 147. If the user provides a priority name (e.g. 'Blocker'), fetch project info, find the matching priority in customListObjs.priority[index].name, and use its ID in the payload. If the name is not found after a fresh fetch, skip the priority field (it is not required) and show a user-friendly message: 'Test case created without priority, as given priority is not available in the current project.' 148. If the user provides a component name, fetch project info, find the matching component in customListObjs.component[index].name, and use its ID in the payload. If the name is not found, skip the component field (it is not required) and show a user-friendly message: 'Test case created without component, as given component is not available in the current project.' 149. If the user provides an owner name, fetch project info, find the matching owner in customListObjs.owner[index].name, and use its ID in the payload as testcaseOwner. If the name is not found, skip the testcaseOwner field (it is not required) and show a user-friendly message: 'Test case created without owner, as given owner is not available in the current project.' 150. If the user provides a test case state name, fetch project info, find the matching state in customListObjs.testCaseState[index].name, and use its ID in the payload as testCaseState. If the name is not found, skip the testCaseState field (it is not required) and show a user-friendly message: 'Test case created without test case state, as given state is not available in the current project.' 151. If the user provides a test case type name, fetch project info, find the matching type in customListObjs.testCaseType[index].name, and use its ID in the payload as testCaseType. If the name is not found, skip the testCaseType field (it is not required) and show a user-friendly message: 'Test case created without test case type, as given type is not available in the current project.' 152. If the user provides a testing type name, fetch project info, find the matching type in customListObjs.testingType[index].name, and use its ID in the payload as testingType. If the name is not found, skip the testingType field (it is not required) and show a user-friendly message: 'Test case created without testing type, as given testing type is not available in the current project.' 153. Example: If user says 'Create test case with title "High priority test case" and set priority to "Blocker"', first call project info, map 'Blocker' to its ID, and use that ID for the priority field in the create payload. If user says 'set priority to "Urgent"' and 'Urgent' is not found, skip the priority field and show: 'Test case created without priority, as given priority is not available in the current project.' 154. tcFolderID is required; use the root folder ID from project info or a specific folder. 155. STEPS: ALWAYS include steps in the payload. If user did not provide steps, auto-generate them from the test case name and context. See STEPS RULE at top of hints. 156. If the user provides a prompt like 'create test case with steps as step 1 - Go to login page, step 2 - give credential, step 3 - go to test case page, step 4 - create test case', LLM should parse each step and convert it into the steps payload array, mapping each step to an object with orderId, description, and optionally inputData and expectedOutcome. 157. Example mapping: 'step 1 - Go to login page' β†’ { orderId: 1, description: 'Go to login page' }. 158. LLM should increment orderId for each step, use the step text as description, and optionally infer inputData/expectedOutcome if provided in the prompt. 159. Demo steps payload: steps: [ { orderId: 1, description: 'First Step', inputData: 'First Data', expectedOutcome: 'First Outcome', UDF: { customField1: 'Custom Field Data A', customField2: 'Custom Field Data B' } }, ... ] 160. UDF fields in steps must match your QMetry custom field configuration. 161. Release/cycle mapping is optional but useful for planning. 162. If the user wants to link or associate a release and cycle to the test case, set associateRelCyc: true in the payload. 163. If the user provides a release ID, map it from projects.releases[index].releaseID in the project info response, and use that ID in releaseCycleMapping. 164. If the user provides both release and cycle IDs, validate both against the current project's releases and cycles; if valid, use them in releaseCycleMapping. 165. When adding releaseCycleMapping, always include the 'version' field (usually set to 1) in each mapping object. The correct format is: { release: , cycle: [], version: 1 }. If 'version' is missing, the request will fail. 166. If the user provides a release name, map it to its ID from project info; if a cycle name is provided, map it to its ID from the associated release's builds list. 167. Example payload: releaseCycleMapping: [ { release: , cycle: [], version: 1 } ] 168. LLM should ensure that provided release/cycle names or IDs exist in the current project before using them in the payload. If not found, skip and show a user-friendly message: 'Test case created without release/cycle association, as given release/cycle is not available in the current project.' 169. All IDs (priority, owner, etc.) must be valid for your QMetry instance. 170. If a custom field is mandatory, include it in the UDF object. 171. estimatedTime is in SECONDS (e.g. 3600 = 1 hour, 36000 = 10 hours). NOT minutes. 172. Description and testingType are optional but recommended for clarity. 173. 174. UDF (User Defined Fields) WORKFLOW FOR CREATE: 175. 1. Call 'Fetch UDF Layout' with entityType='TC', pageName='ADD' to discover field names, types, list option IDs, and udfmID (projectUserFieldID). 176. IF listOptions[field.listName] is empty after Fetch UDF Layout, the tool already tried a metadata fallback. If STILL empty, ask the user to provide the option ID from the QMetry UI β€” do NOT guess numeric IDs. 177. 2. For LOOKUPLIST fields: pick one ID from listOptions[field.listName][].id. 178. 3. For MULTILOOKUPLIST fields: pick an array of IDs from listOptions[field.listName][].id. 179. 4. For CASCADINGLIST fields (ROOT-LEVEL UDF β€” MANDATORY STEPS): 180. a. MUST call 'Fetch Cascade Child Values' with parentId to get available child options (do NOT skip this step). 181. b. Pass the cascade value as: { parent: parentId, child: childId } in udfFields. 182. Example: udfFields: { project19: { parent: 5232623, child: 5232625 } } 183. 5. For STRING/LARGETEXT/NUMBER/DATETIMEPICKER: pass value directly. 184. 6. Pass all UDF values via 'udfFields' param: { fieldName: value }. 185. 7. Mandatory UDF fields (isMandatory=true) MUST be included or create will fail. 186. 187. STEP UDFs: Pass step UDF values in each step's 'UDF' object. 188. Call 'Fetch UDF Layout' for stepFields to discover field names, types, and udfmID (projectUserFieldID). 189. Step UDF field types follow same rules as root UDF EXCEPT for CASCADINGLIST β€” step cascade requires a DIFFERENT format: 190. 191. STEP CASCADINGLIST UDF FORMAT (critical β€” different from root cascade): 192. For a cascade field named 'project19' with udfmID=2637584, parent={id:5232626, value:'React'}, child={id:5232628, value:'Redux'}: 193. You MUST include THREE keys inside the step's UDF object: 194. 1. fieldName: { parent: parentId, child: childId } 195. e.g. project19: { parent: 5232626, child: 5232628 } 196. 2. fieldName_value: [{ FieldID: 'fieldName', FieldValue: [{ id: parentId, value: 'parentLabel', child: { id: childId, value: 'childLabel' } }], type: 'CASCADINGLIST' }] 197. e.g. project19_value: [{ FieldID: 'project19', FieldValue: [{ id: 5232626, value: 'React', child: { id: 5232628, value: 'Redux' } }], type: 'CASCADINGLIST' }] 198. 3. fieldName_selectedList: { id: udfmID, name: 'fieldName', type: 'CASCADINGLIST' } 199. e.g. project19_selectedList: { id: 2637584, name: 'project19', type: 'CASCADINGLIST' } 200. To get parentLabel and childLabel: call 'Fetch Cascade Child Values' β€” it returns option labels alongside IDs. 201. udfmID comes from Fetch UDF Layout stepFields[].projectUserFieldID. 202. NEVER omit _value or _selectedList for step cascade fields β€” the API silently ignores cascade data without them.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
stepsNoSTEPS RULE β€” include this field unless the user explicitly says NOT to create steps. NEVER send steps: [] (empty array) β€” always send at least 1 valid step object. Omit this field entirely (and set skipSteps: true) ONLY when the user explicitly asks to skip steps. HOW TO POPULATE: - If user explicitly provides steps: parse each step into { orderId, description, inputData?, expectedOutcome? }. - If user does NOT provide steps (and did not say to skip them): auto-generate meaningful steps based on the test case name, description, and context. Use your knowledge to infer 2-5 logical, realistic steps for the feature or flow being tested. Example: name='Login Test Case' β†’ [{orderId:1, description:'Navigate to login page'}, {orderId:2, description:'Enter credentials'}, {orderId:3, description:'Submit and verify success'}] - If user explicitly said NOT to create steps: omit this field and set skipSteps: true. STEP DEFAULT VALUES: After building the steps array, check 'stepDefaultValues' from Fetch UDF Layout. stepDefaultValues shape: { fieldName: defaultValue } For each step: for each key in stepDefaultValues, if the user did NOT explicitly provide a value for that field β†’ add it to step.UDF with the default value. Auto-apply silently β€” do NOT ask the user. Step object fields: orderId (required): sequential integer starting at 1 description (required): step action text inputData (optional): test data for this step expectedOutcome (optional): what should happen after this step UDF (optional): step-level custom fields β€” auto-fill defaults from stepDefaultValues tcStepID (omit on create β€” only used when updating existing steps)
priorityNo
componentNo
skipStepsNoSet to true ONLY when the user explicitly says they do NOT want steps created (e.g. 'create test case without steps', 'no steps', 'skip steps'). When true, the 'steps' field must be omitted entirely. When false or absent (the default), steps MUST always be included β€” auto-generate them from context if the user did not provide them.
udfFieldsNoFlat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.
tcFolderIDNoTest Case folder ID - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific folder ID. System will fetch project info using the projectKey and extract rootFolders.TC.id automatically. Manual folder ID only needed if you want to target a specific sub-folder.
descriptionNo
testingTypeNo
testCaseTypeNo
estimatedTimeNoEstimated execution time in SECONDS (e.g. 3600 = 1 hour, 36000 = 10 hours). NOT minutes.
testCaseStateNo
testcaseOwnerNo
associateRelCycNo
releaseCycleMappingNoRelease/cycle mapping. Set associateRelCyc=true when providing this. version field defaults to 1 if not specified.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the sparse annotations, the description discloses many behavioral traits: mandatory pre-flight calls, auto-application of defaults, silent data-loss risks when date formats are wrong, API behavior on mandatory-field errors, and automatic error recovery. It also details side effects like auto-created steps and default sweeps. Nothing here contradicts the readOnlyHint=false / destructiveHint=false annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely long and heavily repetitive: the STEPS RULE appears in the description, the schema, and multiple hint sections, and examples 1, 2, and 6 are near-duplicates. Box-drawing characters and hundreds of lines bury key facts. While headings and decision tables help structure it, many sentences do not earn their place, so it is not concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 15-parameter tool with nested objects and no output schema, the description is exceptionally complete. It covers mandatory pre-checks, default application, date-format conversion, folder/sub-folder resolution, name-to-ID mapping for system fields, UDF cascade formats, error recovery, and output shape. The only blemish is an internal inconsistency where one hint line says tcFolderID is 'required' while the parameter description says it is auto-resolved, but overall nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is low (~40%), but the description compensates thoroughly. It explains tcFolderID auto-resolution, the skipSteps/steps contract, estimatedTime in seconds, releaseCycleMapping version defaulting, UDF value formats per field type, and step cascade format requirements. Multiple worked examples show exactly how to populate parameters in different user scenarios.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Create a new test case in QMetry with steps, metadata, and release/cycle mapping.' It is clearly distinguished from fetch/update/delete siblings by emphasizing 'new test case' and by describing creation output (new test case ID). The title and name reinforce the same purpose without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives strong usage context: a 'Use Cases' list, a 'MOST COMMON' scenario, mandatory pre-create calls to 'Fetch UDF Layout' and 'Fetch QMetry Project Info', and explicit rules for when to auto-generate steps vs. skip them. It does not explicitly name an alternative like qmetry_update_test_case for existing cases, though the tcStepID note ('omit on create β€” only used when updating existing steps') hints at the create/update boundary.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_create_test_suiteQMetry: Create Test SuiteAInspect

Create a new test suite in QMetry with metadata and release/cycle mapping.

Toolset: Test Suites

Parameters:

  • parentFolderId (string): Test Suite parent folder ID - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific folder ID. System will fetch project info using the projectKey and extract rootFolders.TS.id automatically. Manual folder ID only needed if you want to target a specific sub-folder.

  • name (string) required

  • isAutomatedFlag (boolean)

  • description (string)

  • testsuiteOwner (number)

  • testSuiteState (number)

  • associateRelCyc (boolean)

  • releaseCycleMapping (array): Release/cycle association for the test suite. CRITICAL SHAPE DIFFERENCE vs Test Case mapping: Test Suite uses: { releaseId: number, buildID: number } Test Case uses: { release: number, cycle: number[], version: number } β€” DO NOT use TC shape here. Set associateRelCyc=true when providing this array. Example: [{ releaseId: 92112, buildID: 130831 }]

  • udfFields (record<string, union>): Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.

Output Description: JSON object containing the new test suite ID, summary, and creation metadata.

Use Cases: 1. Create a basic test suite with just a name and folder 2. Add detailed metadata like description to a test suite 3. Associate test suite with specific release/cycle for planning 4. Set testsuiteOwner, testSuiteState, and other metadata using valid IDs from project info 5. Create test suites for isAutomatedFlag true or false for automated or manual types, default is false 6. Add test suite to a specific folder using parentFolderId 7. Map test suite to multiple cycles/releases and build ID

Examples:

  1. Create a test suite in the root folder (auto-resolved)

{
  "name": "Demo Test Suite"
}

Expected Output: Test suite created in the root test suite folder with ID and summary details

  1. Create a simple test suite in folder 102653

{
  "parentFolderId": "102653",
  "name": "Login Test Suite"
}

Expected Output: Test suite created with ID and summary details

  1. Create a test suite with some details and metadata

{
  "parentFolderId": "113557",
  "isAutomatedFlag": false,
  "name": "Testsuite Summary",
  "description": "desc",
  "testsuiteOwner": 6963,
  "testSuiteState": 505035,
  "associateRelCyc": true,
  "releaseCycleMapping": [
    {
      "buildID": 18411,
      "releaseId": 10286
    }
  ]
}

Expected Output: Test suite created with details and metadata. Example uses: parentFolderId=113557 (MAC root TS folder from rootFolders.TS.id), testsuiteOwner=6963 (umang.savaliya from customListObjs.owner[index].id), testSuiteState=505035 (New from customListObjs.testSuiteState[index].id), releaseId=10286 (Air release from projects[index].releases[index].releaseID), buildID=18411 (Air Q1-19 cycle from projects[index].releases[index].builds[index].buildID)

Hints: 1. ╔══════════════════════════════════════════════════════════════════╗ 2. β•‘ STEP 0 β€” NON-NEGOTIABLE: Call 'Fetch UDF Layout' BEFORE create β•‘ 3. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 4. NEVER call 'Create Test Suite' without first calling 'Fetch UDF Layout' with entityType='TS', pageName='ADD'. 5. Skipping this step WILL cause 400 errors (CO.MANDATORY_FIELDS_MISSING) because mandatory fields and defaults are unknown. 6. This rule has NO exceptions β€” not even when the user only provided a name and nothing else. 7. 8. === MANDATORY PRE-CREATE CHECK (ALWAYS DO THIS FIRST) === 9. Before creating any test suite, call 'Fetch UDF Layout' with entityType='TS', pageName='ADD'. 10. Response keys use 'TS' β€” same structure as TC but scoped to test suite module. 11. 12. SYSTEM FIELDS mandatory check β€” use 'systemFields' array (from qmSDF.TS in newlayout): 13. Each entry: { name, label, fieldTypeName, isMandatory } 14. isMandatory=true (allowBlank=false) means field MUST have a value before creating. 15. 16. UDF FIELDS mandatory check β€” use 'fields' array (from qmUDF.TS in newlayout): 17. Each entry: { name, label, fieldTypeName, isMandatory, listName? } 18. isMandatory=true means field MUST have a value. 19. 20. DEFAULT VALUES β€” use 'defaultValues' object (from qmDefaultValue.TS in newlayout): 21. Shape: { fieldName: defaultValueId } e.g. { 'priority': 5232497, 'estimatedTime': 18305 } 22. RULE: isMandatory=true AND defaultValues[field.name] exists β†’ auto-use default, do NOT ask user. 23. RULE: isMandatory=true AND no defaultValues entry β†’ MUST ask user before creating. 24. RULE: isMandatory=false AND defaultValues entry exists β†’ auto-apply if user didn't specify. 25. 26. ╔══════════════════════════════════════════════════════════════════════════╗ 27. β•‘ PRE-FLIGHT DEFAULT SWEEP β€” MANDATORY STEP BEFORE EVERY CREATE CALL β•‘ 28. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 29. After resolving mandatory fields, sweep ALL defaultValues entries: 30. For EACH key in defaultValues: 31. IF the user did not explicitly provide that field β†’ add it to the payload using the default value. 32. This applies regardless of isMandatory β€” non-mandatory defaults MUST also be auto-applied. 33. Example: defaultValues = { priority: 5232497, estimatedTime: 18305 } 34. β†’ user only said 'create a test suite named X' 35. β†’ payload MUST include: priority=5232497, estimatedTime=18305 36. β†’ WRONG to omit priority/estimatedTime just because they are not mandatory β€” they have defaults. 37. Skipping this sweep = missing fields in the created record = user-visible data loss. 38. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 39. 40. DECISION MATRIX: 41. | isMandatory | Has default | Action | 42. |-------------|-------------|-------------------------------------| 43. | true | YES | Auto-fill with default, no ask | 44. | true | NO | Ask user before creating | 45. | false | YES | Auto-fill with default β€” REQUIRED | 46. | false | NO | Skip if user didn't provide | 47. Only after ALL mandatory fields resolved AND default sweep complete β†’ proceed with create. 48. === END MANDATORY PRE-CREATE CHECK === 49. 50. === ERROR RECOVERY: CO.MANDATORY_FIELDS_MISSING === 51. If create fails with error code 'CO.MANDATORY_FIELDS_MISSING', DO NOT give up. Auto-recover: 52. 1. Parse the 'MISSING_FIELDS' list from the error response (comma-separated field labels). 53. 2. Match each label against 'fields[].label' and 'systemFields[].label' from the Fetch UDF Layout response. 54. 3. For matched UDF fields: check 'listOptions[field.listName]' for valid option IDs. 55. 4. For matched system fields (e.g. 'Status'): check 'customListObjs.testSuiteState' from project info. 56. 5. If the field has a 'defaultValues' entry: auto-fill it silently. 57. 6. If no default exists: ask the user ONLY for the missing fields by label. 58. 7. Retry create with the resolved values added to the payload. 59. NEVER ask user to 'try again' manually β€” resolve and retry automatically. 60. === END ERROR RECOVERY === 61. 62. === DATE FORMAT CHECK (MANDATORY β€” EVERY CREATE REQUEST) === 63. ALWAYS call 'Fetch QMetry Project Info' before every create request β€” not only when the user explicitly mentions a date. 64. Any UDF field could be a DATETIMEPICKER. Wrong format causes QMetry to silently discard the field value (API returns success but value is NOT stored β€” no error). 65. STEP 1: From project info, read dateTimeFormatID (e.g. 3). 66. STEP 2: Find entry in dateTimeFormatNew where id === dateTimeFormatID β†’ read its unique_value (e.g. 'yyyy-MM-dd'). 67. STEP 3: unique_value pattern: yyyy=4-digit year, MM=2-digit month (01-12), dd=2-digit day, MMM=3-letter month (Jan/Feb/...). 68. Example: id=1 β†’ MM-dd-yyyy β†’ '10-25-2000' | id=2 β†’ dd-MM-yyyy β†’ '25-10-2000' | id=3 β†’ yyyy-MM-dd β†’ '2000-10-25' | id=4 β†’ dd-MMM-yyyy β†’ '25-Oct-2000' 69. STEP 4: For EVERY DATETIMEPICKER field in the payload: parse any user-provided date and re-format it using the active unique_value pattern before sending. 70. NEVER assume a date format β€” always derive it from dateTimeFormatID. Wrong format = silent data loss. 71. === END DATE FORMAT CHECK === 72. 73. If parentFolderId is not provided, it will be auto-resolved to the root test suite folder using project info (rootFolders.TS.id). 74. To get valid values for testsuiteOwner, testSuiteState, etc., call the 'Admin/Get info Service' API (FETCH_PROJECT_INFO tool) and use the returned customListObjs IDs. 75. CRITICAL: For testsuiteOwner mapping - Call API 'Admin/Get info Service', from the response get value from customListObjs.owner[].id. Match the user by customListObjs.owner[].name. 76. If the user provides an owner name (testsuiteOwner), fetch project info, find the matching owner in customListObjs.owner[index].name or customListObjs.owner[index].uniqueLabel, and use its ID in the payload as testsuiteOwner. If the name is not found, skip the testsuiteOwner field (it is not required) and show a user-friendly message: 'Test suite created without owner, as given owner is not available in the current project.' 77. CRITICAL: For testSuiteState mapping - Call API 'Admin/Get info Service', from the response get value from customListObjs.testSuiteState[].id. Match the state by customListObjs.testSuiteState[].name. 78. If the user provides a test suite state name(testSuiteState), fetch project info, find the matching state in customListObjs.testSuiteState[index].name, and use its ID in the payload as testSuiteState. If the name is not found, skip the testSuiteState field (it is not required) and show a user-friendly message: 'Test suite created without test suite state, as given state is not available in the current project.' 79. parentFolderId is required; use the root folder ID from project info (rootFolders.TS.id) or a specific folder. 80. Release/cycle mapping is optional but useful for planning. 81. If the user wants to link or associate a release and cycle to the test suite, set associateRelCyc: true in the payload. 82. CRITICAL: For releaseCycleMapping.releaseId - Call API 'Release/List' (or use project info projects[].releases[].releaseID), from the response get value from data[].releaseID or projects[].releases[].releaseID. Match the release by name. 83. CRITICAL: For releaseCycleMapping.buildID - Call API 'Cycle/List' (or use project info projects[].releases[].builds[].buildID), from the response get value from data[].buildID or projects[].releases[].builds[].buildID. Match the build/cycle by name. 84. If the user provides a release name, map it to its ID from projects[].releases[].releaseID in the project info response, and use that ID as releaseId in releaseCycleMapping. 85. If the user provides a build/cycle name, map it to its ID from projects[].releases[].builds[].buildID in the project info response, and use that ID as buildID in releaseCycleMapping. 86. Example payload: releaseCycleMapping: [ { releaseId: , buildID: } ] 87. Example: For 'Air' release and 'Air Q1-19' cycle in MAC project, use releaseId: 10286 and buildID: 18411 88. LLM should ensure that provided release/cycle names or IDs exist in the current project before using them in the payload. If not found, skip and show a user-friendly message: 'Test suite created without release/cycle association, as given release/cycle is not available in the current project.' 89. All IDs (testSuiteState from customListObjs.testSuiteState[index].id, testsuiteOwner from customListObjs.owner[index].id, releaseId from projects.releases[index].releaseID, buildID from projects.releases.builds[index].buildID) must be valid for your QMetry instance. 90. If a custom field is mandatory, include it in the UDF object. 91. 92. UDF (User Defined Fields) WORKFLOW FOR CREATE: 93. 1. Call 'Fetch UDF Layout' with entityType='TS', pageName='ADD' to discover field names, types, and list option IDs. 94. IF listOptions[field.listName] is empty after Fetch UDF Layout, the tool already tried a metadata fallback. If STILL empty, ask the user to provide the option ID from the QMetry UI β€” do NOT guess numeric IDs. 95. 2. For LOOKUPLIST fields: pick one ID from listOptions[field.listName][].id. 96. 3. For MULTILOOKUPLIST fields: pick an array of IDs. 97. 4. For CASCADINGLIST fields: pick parent ID, then call 'Fetch Cascade Child Values' for child ID. Pass { parent: parentId, child: childId }. 98. 5. Pass all UDF values via 'udfFields' param: { fieldName: value }. 99. 6. Mandatory UDF fields (isMandatory=true) MUST be included or create will fail. 100. 101. ╔══════════════════════════════════════════════════════════════════╗ 102. β•‘ POST-CREATE MANDATORY: Link Platform to Test Suite β•‘ 103. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• 104. AFTER every successful test suite creation, ALWAYS call 'Link Platforms to Test Suite'. 105. Reason: QMetry UI enforces platform association β€” test suites without a linked platform 106. are invalid for execution and will show validation errors in the UI. 107. STEP 1: Call 'Fetch Platforms' to get available platforms for the project. 108. STEP 2: If the user specified a platform, match by name and use its platformID. 109. If no platform was specified, use the first available non-archived platform. 110. STEP 3: Call 'Link Platforms to Test Suite' with qmTsId= and qmPlatformId=. 111. NEVER report the test suite as successfully created without completing this platform link step. 112. β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
udfFieldsNoFlat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.
descriptionNo
parentFolderIdNoTest Suite parent folder ID - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific folder ID. System will fetch project info using the projectKey and extract rootFolders.TS.id automatically. Manual folder ID only needed if you want to target a specific sub-folder.
testSuiteStateNo
testsuiteOwnerNo
associateRelCycNo
isAutomatedFlagNo
releaseCycleMappingNoRelease/cycle association for the test suite. CRITICAL SHAPE DIFFERENCE vs Test Case mapping: Test Suite uses: { releaseId: number, buildID: number } Test Case uses: { release: number, cycle: number[], version: number } β€” DO NOT use TC shape here. Set associateRelCyc=true when providing this array. Example: [{ releaseId: 92112, buildID: 130831 }]

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description comprehensively discloses behavioral traits beyond annotations: it mandates a pre-create call to 'Fetch UDF Layout', warns about silent data loss with date formats, specifies the post-create platform linking requirement, and details error recovery for 'CO.MANDATORY_FIELDS_MISSING'. Annotations are minimal (readOnlyHint=false, destructiveHint=false), so the description carries the full burden and does it exceptionally well, with no contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely long (over 100 lines) and includes repetitive instructions (e.g., repeated calls to 'Fetch UDF Layout', multiple mentions of the same ID mapping steps). It uses ASCII art boxes and numbered steps that add structure but become verbose. While every sentence adds some value, the density hinders quick scanning; it could be condensed by removing redundant warnings and consolidating repeated steps.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is richly complete for the tool's complexity: it covers prerequisites (pre-create UDF layout call), mandatory and default field handling, error recovery procedures, date formatting, post-create platform linking, and examples with expected outputs. It accounts for the 9 parameters, nested objects, and the absence of an output schema by describing the output in the output description section. It's thoroughly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 33%, but the description provides extensive parameter semantics beyond the schema: for parentFolderId it explains auto-resolution, for releaseCycleMapping it clarifies the exact key names (releaseId vs release, buildID vs cycle) and the critical shape difference from Test Cases, and for udfFields it elaborates on types and source of values. It also details how to obtain IDs for testsuiteOwner and testSuiteState, filling in all gaps left by the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool creates a new test suite in QMetry with metadata and release/cycle mapping, using a specific verb ('Create') and resource ('Test Suite'). It distinguishes from siblings like qmetry_update_test_suite and qmetry_create_test_case, and provides use cases and examples that reinforce the purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description is explicit about when to use this tool (for creating test suites) and when not to (e.g., do not use Test Case mapping shape), and provides clear alternatives or related steps like calling 'Fetch UDF Layout' before creation and 'Link Platforms to Test Suite' after. It also includes step-by-step prerequisites and decision matrices, making it highly actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_execute_quality_gate_reportQMetry: Execute Quality Gate ReportA
Read-onlyIdempotent
Inspect

Execute a quality gate report by forwarding the request to the backend analytics engine and returning the results.

Toolset: AI Agent

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • reportName (string) required: Report name identifier (e.g. 'RR' for Release Readiness).

  • gateIdentifier (string) required: Gate identifier to execute the report against (e.g. 'GATE1').

  • projectId (number) required: Numeric project ID for the report scope.

  • releaseId (number): Optional release ID to scope the report to a specific release.

  • cycleIds (array): Optional array of cycle IDs to scope the report to specific cycles.

  • page (number): Optional page number for paginated report results.

  • limit (number): Maximum number of records to return per page. Defaults to 100 if not specified. (default: 100)

Use Cases: 1. Generate a release readiness report for a specific project, release, and cycle 2. Execute a quality gate assessment to evaluate project health against gate criteria 3. Run a gate report scoped to specific cycles within a release 4. Produce analytics data for quality gate evaluation and decision-making

Examples:

  1. Execute a release readiness report for project 45851, release 90698, cycle 129140

{
  "reportName": "RR",
  "gateIdentifier": "GATE1",
  "projectId": 45851,
  "releaseId": 90698,
  "cycleIds": [
    129140
  ]
}

Expected Output: { "data": [...], "total": 0, "success": true, "page": {} }

  1. Execute a gate report without release/cycle scoping

{
  "reportName": "RR",
  "gateIdentifier": "GATE1",
  "projectId": 45851
}

Expected Output: { "data": [...], "total": 0, "success": true, "page": {} }

Hints: 1. REQUIRED: 'reportName' identifies the report type (e.g. 'RR' for Release Readiness). 2. REQUIRED: 'gateIdentifier' identifies which gate to evaluate (e.g. 'GATE1'). 3. REQUIRED: 'projectId' must be a valid numeric project ID. 4. OPTIONAL: 'releaseId' scopes the report to a specific release. 5. OPTIONAL: 'cycleIds' scopes the report to specific test cycles within the release. 6. OPTIONAL: 'page' specifies the page number for paginated report results. 7. OPTIONAL: 'limit' sets the maximum number of records per page (defaults to 100). 8. Call 'Fetch Quality Gate Configuration' first to discover available gates and report parameters. 9. The response is returned exactly as received from the backend β€” no transformation is applied.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoOptional page number for paginated report results.
limitNoMaximum number of records to return per page. Defaults to 100 if not specified.
cycleIdsNoOptional array of cycle IDs to scope the report to specific cycles.
projectIdYesNumeric project ID for the report scope.
releaseIdNoOptional release ID to scope the report to a specific release.
projectKeyNoProject key - unique identifier for the projectdefault
reportNameYesReport name identifier (e.g. 'RR' for Release Readiness).
gateIdentifierYesGate identifier to execute the report against (e.g. 'GATE1').

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description notes that the tool forwards the request to the backend analytics engine and returns results unchanged: 'The response is returned exactly as received from the backend β€” no transformation is applied.' This goes beyond the annotations by clarifying the pass-through behavior and expected output. Annotations already cover read-only, idempotent, and non-destructive behavior, so extra depth is not required.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear headings for parameters, use cases, examples, and hints, and opens with a concise summary sentence. However, it is verbose and repeats schema field descriptions almost verbatim, making it longer than necessary. Several hints also restate information already present in the parameter list.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter operation with no output schema, this description is exceptionally complete: it documents every parameter, gives multiple examples, includes expected output, states pagination defaults, and names the prerequisite fetch-config tool. An agent has everything needed to call this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value through concrete examples with actual project/release/cycle IDs, expected output shapes, and hints that clarify required vs optional parameters. It also explains the relationship between releaseId and cycleIds scoping, which goes beyond the schema's individual field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb and resource: 'Execute a quality gate report by forwarding the request to the backend analytics engine and returning the results.' It clearly distinguishes the action from sibling tools like qmetry_fetch_quality_gate_configuration, which discovers gates, and qmetry_export_html_report, which exports a report. The use cases reinforce the purpose by describing release readiness and gate assessment scenarios.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use cases explicitly describe when to use the tool, such as generating a release readiness report or evaluating project health against gate criteria. Hint 8 adds a clear prerequisite workflow: call 'Fetch Quality Gate Configuration' first to discover available gates and report parameters. It does not explicitly exclude sibling tools, but the integration guidance is strong enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_export_html_reportQMetry: Export HTML ReportA
Idempotent
Inspect

Export HTML content as a downloadable report file via the backend.

Toolset: AI Agent

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • htmlContent (string) required: HTML content to be exported as a report.

  • fileName (string) required: Name for the exported report file (without extension). Letters, digits, '.', '_' and '-' only - no path separators or '..' sequences.

Use Cases: 1. Export a generated release readiness report as an HTML file 2. Save AI-generated quality gate analysis as a downloadable report 3. Create a shareable HTML document from report content

Examples:

  1. Export a release readiness report

{
  "htmlContent": "<h1>Release Readiness Report</h1><p>Generated by AI Agent.</p>",
  "fileName": "release-readiness-report"
}

Expected Output: Report file generated and returned by the backend.

Hints: 1. REQUIRED: 'htmlContent' is the full HTML string to export. 2. REQUIRED: 'fileName' is the report file name (without extension). 3. Typically called after generating report content from 'Execute Quality Gate Report' results.

ParametersJSON Schema
NameRequiredDescriptionDefault
fileNameYesName for the exported report file (without extension). Letters, digits, '.', '_' and '-' only - no path separators or '..' sequences.
projectKeyNoProject key - unique identifier for the projectdefault
htmlContentYesHTML content to be exported as a report.

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=true and destructiveHint=false, covering the main safety profile. The description adds 'via the backend', 'downloadable', and an 'Expected Output' line saying a report file is generated and returned, but it does not disclose storage, overwrite behavior, permissions, or response format beyond that. This is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded, and the description is organized into scannable sections (Parameters, Use Cases, Examples, Hints). Some content repeats the schema, but the example and use-case guidance earn their place without making the description unwieldy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 3-parameter tool with no output schema, the description covers what the tool does, when to use it, what to provide, and roughly what to expect ('Report file generated and returned by the backend'). It could specify the exact response format or file delivery mechanism, but nothing needed to invoke the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value with a concrete JSON example, clarifies that htmlContent must be the complete HTML string, and reinforces the fileName constraints (no extension, no path separators), helping an agent construct valid arguments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb ('Export'), a clear resource ('HTML content'), and an outcome ('a downloadable report file via the backend'). It is unambiguous and distinct from the QMetry sibling tools, and the hint that it is typically called after 'Execute Quality Gate Report' further clarifies its role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides three concrete use cases and a hint about when to call it ('Typically called after generating report content from Execute Quality Gate Report results'). It gives clear context but does not explicitly state when not to use this tool or name an alternative export tool, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_automation_statusQMetry: Fetch Automation StatusA
Read-onlyIdempotent
Inspect

Fetches the status of an automation import job by request ID.

Toolset: Automation

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • requestID (number) required: Numeric request ID from import automation response. CRITICAL: parameter name is 'requestID' β€” do NOT use 'requestId', 'jobId', or other variants. Accepts a string or number.

Use Cases: 1. 1. Check if an automation import job is completed or still in progress. 2. 2. Retrieve status, progress, and details for a specific automation import request. 3. 3. Monitor automation result processing for CI/CD integrations.

Examples:

  1. Fetch status for request ID 12345

{
  "requestID": 12345
}

Expected Output: Status, progress, and details of the automation import job for request ID 12345.

ParametersJSON Schema
NameRequiredDescriptionDefault
requestIDYesNumeric request ID from import automation response. CRITICAL: parameter name is 'requestID' β€” do NOT use 'requestId', 'jobId', or other variants. Accepts a string or number.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safe, repeatable nature of the call is established without the description. The description adds some context by stating that the response contains status, progress, and details, but it does not cover potential operational traits such as eventual consistency, polling intervals, or error conditions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a clear one-sentence purpose and uses helpful sections for use cases and examples. However, the 'Parameters' section largely duplicates the input schema, and the use-case list has a formatting artifact ('1. 1.'), so not every section earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter, read-only status lookup with no output schema, the description covers the request ID provenance, the optional project key, a concrete JSON example, and a summary of the expected output. It could be more complete by naming the companion import tool that returns the request ID and by specifying the exact result fields, but these are minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description mostly restates the schema's parameter docs. It does reinforce the critical naming warning ('requestID', not requestId/jobId), but that warning is already present in the schema, so no new meaning is added. There is also a slight tension between schema type 'number' and the text saying it accepts a string or number.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence uses a specific verb ('Fetches'), resource ('status of an automation import job'), and lookup key ('request ID'), so it is immediately distinguishable from the many other qmetry fetch tools. Use cases reinforce that this is a polling/monitoring operation rather than a mutation or test-execution tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Three explicit use cases tell an agent when this tool applies: checking completion, retrieving status/progress/details, and monitoring CI/CD automation result processing. It does not name when-not-to-use or point to a sibling like qmetry_import_automation_test_results, so it stops short of explicit alternative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_buildsQMetry: Fetch BuildsA
Read-onlyIdempotent
Inspect

Fetch QMetry builds from the current project

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with builds list and pagination metadata

Use Cases: 1. Fetch all from the current project 2. Fetch all available builds for test execution planning 3. Get build metadata for test run assignments 4. List builds for reporting and analytics 5. Filter builds by name or archive status 6. Get paginated build results for large projects 7. Retrieve build information for CI/CD integration 8. Search for specific builds using filters 9. Get build details for test execution history

Examples:

  1. Get all builds (default behavior)

{}

Expected Output: List of all builds with default pagination (10 items per page)

  1. Get builds with custom pagination

{
  "page": 1,
  "limit": 10,
  "start": 0
}

Expected Output: List of builds with custom pagination settings

  1. Filter builds by name

{
  "filter": "[{\"value\":\"Build 1.0\",\"type\":\"string\",\"field\":\"name\"}]"
}

Expected Output: Filtered list of builds matching the name criteria

  1. Filter builds by archive status

{
  "filter": "[{\"value\":[1,0],\"type\":\"list\",\"field\":\"isArchived\"}]"
}

Expected Output: List of builds filtered by archive status (archived and non-archived)

Hints: 1. Use 'default' project key when user doesn't specify one 2. Default pagination: start=0, page=1, limit=10 3. Filter parameter should be a JSON string with filter criteria 4. Common filter fields: 'name' (string), 'isArchived' (list of 0,1) 5. Empty payload {} is sent when no parameters are provided 6. Builds are also known as 'drops' in QMetry terminology 7. Use builds for associating test executions with specific software versions

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds useful behavioral context beyond those annotations: it notes that an empty payload {} is sent when no parameters are provided, explains the output is a JSON object with pagination metadata, and clarifies filter behavior with example expected outputs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a clear one-sentence purpose, but it becomes bloated: the parameter section duplicates the schema descriptions, the use cases list nine overlapping items, and the hints repeat defaults already present in the schema and examples. Several sentences do not add unique value, making the overall structure longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only fetch tool with no required parameters and no output schema, the description is largely complete: it explains the output shape, demonstrates all parameter behaviors through examples, and provides filter syntax. Minor gaps remain, such as not detailing the exact structure of the build objects returned, but overall the agent has enough context to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already documented structurally. The description goes beyond the schema by showing concrete filter JSON examples, naming common filter fields ('name', 'isArchived'), and explaining pagination defaults and behavior with expected outputs, which adds practical meaning an agent can use when constructing calls.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Fetch'), a concrete resource ('QMetry builds'), and a scoping qualifier ('from the current project'). It also adds the clarifying note that builds are known as 'drops', which further disambiguates the term and makes the tool identifiable among the many qmetry_ siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a detailed 'Use Cases' section listing when to fetch builds, including for test execution planning, reporting, CI/CD integration, and filtered or paginated retrieval. It does not explicitly name alternatives to use instead, but the use cases give clear contextual guidance for when this tool is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_cascade_child_valuesQMetry: Fetch Cascade Child ValuesA
Read-onlyIdempotent
Inspect

Fetch the child values of a CASCADINGLIST UDF field for a given parent item ID. Use this before bulk-updating a CASCADINGLIST Test Run UDF to discover valid child item IDs.

Toolset: UDF

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • id (number) required: Numeric ID of the parent cascade list item to fetch child values for. Get this from the 'lookupOptions' returned by 'Fetch Test Run UDF Metadata' for a CASCADINGLIST field β€” each option has an 'id' field.

  • isArchReq (boolean): Whether to include archived child items in the response (default: false). (default: false)

Output Description: JSON object with 'parentId' (the input ID), 'parentName' (the parent item's display name), 'children' array (each item has id, name, uniqueLabel, isArchived), and a '_note' explaining how to use the IDs.

Use Cases: 1. Find valid child values for a CASCADINGLIST UDF before bulk-updating test runs 2. List all child options available under a specific parent cascade item 3. Resolve child item ID when user knows the parent but not the child 4. Discover cascade hierarchy for a UDF field before setting it on test executions

Examples:

  1. Fetch child values for parent cascade item with ID 5173534

{
  "id": 5173534
}

Expected Output: { parentId: 5173534, parentName: "India", children: [{ id: 5173535, name: "i1", uniqueLabel: "i1", isArchived: false }, ...], _note: "Use 'id' from 'children' as the 'child' value in the CASCADINGLIST update." }

  1. Fetch child values including archived items for parent ID 5126498

{
  "id": 5126498,
  "isArchReq": true
}

Expected Output: { parentId: 5126498, parentName: "abc", children: [...], _note: "..." }

Hints: 1. MANDATORY WORKFLOW for CASCADINGLIST bulk update:

  1. Call 'Fetch Test Run UDF Metadata' β†’ get the CASCADINGLIST field's 'fieldID' (projectUserFieldID) and parent item options from 'lookupOptions'.

  2. Call this tool ('Fetch Cascade Child Values') with a parent item 'id' from step 1 β†’ get child item IDs.

  3. Call 'Bulk Update Test Run UDFs' with value: { parent: , child: } and the 'fieldID' from step 1. 2. The parent item IDs are in the 'lookupOptions' map returned by 'Fetch Test Run UDF Metadata'. Each entry under the field's listName contains items with 'id' β€” use that 'id' as the 'id' parameter here. IMPORTANT: If 'lookupOptions' from Fetch Test Run UDF Metadata is empty for the CASCADINGLIST field, the API did not return parent options. In this case, ask the user to provide the parent item ID from the QMetry UI β€” do NOT guess. 3. The response 'children' array contains objects with 'id', 'name', 'uniqueLabel', and 'isArchived'. Use 'id' as the 'child' value in the bulk update payload. 4. Set 'isArchReq: true' only if the user explicitly asks to include archived/inactive child options. 5. This endpoint requires 'scope' and 'orgcode' headers β€” these are injected automatically from the session context when 'Set Project Info' has been called. If you see an authorization error, call 'Set Project Info' first. 6. Do NOT call this tool for STRING, NUMBER, DATETIMEPICKER, LOOKUPLIST, or MULTILOOKUPLIST fields β€” only CASCADINGLIST (fieldType: 'CASCADINGLIST') fields have a parent-child hierarchy.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesNumeric ID of the parent cascade list item to fetch child values for. Get this from the 'lookupOptions' returned by 'Fetch Test Run UDF Metadata' for a CASCADINGLIST field β€” each option has an 'id' field.
isArchReqNoWhether to include archived child items in the response (default: false).
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite annotations already marking readOnlyHint and idempotentHint true, the description adds behavior not inferable from annotations: output shape, archived-item flag semantics, the 'scope'/'orgcode' header prerequisite, and the instruction not to guess when lookupOptions is empty. No contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long, but it is well-structured with Parameters, Output Description, Use Cases, Examples, and Hints, and the critical workflow is front-loaded in the first sentence. A little redundancy exists (e.g., 'default: false' appears twice in the isArchReq line and examples restate the output shape), so it is not perfectly concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool without an output schema, the description fully specifies return fields ('parentId', 'parentName', 'children', '_note'), includes auth prerequisites, addresses empty lookupOptions, provides examples, and covers the complete sequence for bulk-update usage. Nothing needed to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds practical meaning beyond the schema: how to obtain 'id' from lookupOptions, when to set 'isArchReq' (only if explicitly requested), and how to use returned IDs as 'child' in the update payload. The parameter list itself largely repeats the schema, which is acceptable given schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence uses a specific verb and resource ('Fetch the child values of a CASCADINGLIST UDF field for a given parent item ID') and immediately connects it to a distinct workflow step (before bulk-updating a CASCADINGLIST Test Run UDF). This differentiates it from siblings like qmetry_fetch_test_run_udf_metadata and qmetry_bulk_update_test_run_udfs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance ('Use this before bulk-updating...'), lists four concrete use cases, and closes with an exclusion ('Do NOT call this tool for STRING, NUMBER...'). It also embeds a mandatory workflow referencing the metadata and bulk-update tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_defects_or_issuesQMetry: Fetch Defects or IssuesA
Read-onlyIdempotent
Inspect

Fetch QMetry defects or issues - automatically handles viewId resolution based on project

Toolset: Issues

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • viewId (number): ViewId for issues - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.IS.viewId automatically.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

  • isJiraIntegrated (boolean): Send true if current project is Integrated with Jira (default: false)

  • sort (string): Sort Records - refer json schema, Possible property - entityKey, name, typeAlias, stateAlias, createdDate, createdByAlias, updatedDate, updatedByAlias, priorityAlias, createdSystem, linkedTcrCount, linkedRqCount, dfOwner, attachmentCount, environmentText (default: "[{"property":"name","direction":"ASC"}]")

Output Description: JSON object with 'data' array containing issues. Each issue has 'id' (numeric defect ID β€” use this as defectId for Fetch Issue Details), 'entityKey', 'name'/'summary', and other fields. There is no 'DefectId' field in this response β€” 'id' is the defect identifier.

Use Cases: 1. List all issues in a project 2. Search for specific issues using filters 3. Get paginated issue results

Examples:

  1. Get all issues from default project - system will auto-fetch viewId

{}

Expected Output: List of issues from default project with auto-resolved viewId

  1. Get all issues from UT project - system will auto-fetch UT project's viewId

{
  "projectKey": "UT"
}

Expected Output: List of issues from UT project using UT's specific IS viewId

  1. Get issues by release/cycle filter

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"},{\"value\":[111577],\"type\":\"list\",\"field\":\"cycle\"}]"
}

Expected Output: Issues associated with Release 8.12 (ID: 55178) and Cycle 8.12.1 (ID: 111577)

  1. Get issues by release only

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"}]"
}

Expected Output: All defects or issues associated with Release 8.12 (ID: 55178)

  1. Get issues by cycle only

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[111577],\"type\":\"list\",\"field\":\"cycle\"}]"
}

Expected Output: All defects or issues associated with Cycle 8.12.1 (ID: 111577)

  1. Search for specific issue by entity key

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-IS-636\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Issues matching the entity key criteria

  1. Search for multiple defects or issues by comma-separated entity keys

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-IS-636,MAC-IS-637,MAC-IS-638\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Issues matching any of the specified entity keys

Hints: 1. CRITICAL WORKFLOW: Always use the SAME projectKey for both project info and issues fetching 2. Step 1: If user specifies projectKey (like 'UT', 'MAC'), use that EXACT projectKey for project info 3. Step 2: Get project info using that projectKey, extract latestViews.IS.viewId 4. Step 3: Use the SAME projectKey and the extracted IS viewId for fetching issues 5. Step 4: If user doesn't specify projectKey, use 'default' for both project info and issues fetching 6. NEVER mix project keys - if user says 'MAC project', use projectKey='MAC' for everything 7. For search by issues key (like MAC-IS-1684), use filter: '[{"type":"string","value":"MAC-IS-1684","field":"entityKeyId"}]' 8. RELEASE/CYCLE FILTERING: Use release and cycle IDs, not names, for filtering 9. For release filter: '[{"value":[releaseId],"type":"list","field":"release"}]' 10. For cycle filter: '[{"value":[cycleId],"type":"list","field":"cycle"}]' 11. For combined release+cycle: '[{"value":[releaseId],"type":"list","field":"release"},{"value":[cycleId],"type":"list","field":"cycle"}]' 12. Get release/cycle IDs from FETCH_RELEASES_AND_CYCLES tool before filtering 13. FILTER FIELDS: name, stateAlias, typeAlias, entityKeyId, createdDate, createdByAlias, updatedDate, updatedByAlias, createdSystem, dfOwner, priorityAlias, linkedTcrCount, linkedRqCount, attachmentCount, componentAlias, environmentText 14. SORT FIELDS: entityKey, name, typeAlias, stateAlias, createdDate, createdByAlias, updatedDate, updatedByAlias, priorityAlias, createdSystem, linkedTcrCount, linkedRqCount, dfOwner, attachmentCount, environmentText 15. For multiple entity keys, use comma-separated values in filter 16. Use pagination for large result sets (start, page, limit parameters) 17. This tool is essential for defect management and issue tracking 18. Critical for quality assurance and defect lifecycle analysis 19. Use for compliance reporting and issue traceability 20. Helps maintain visibility into project defects and issues

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
sortNoSort Records - refer json schema, Possible property - entityKey, name, typeAlias, stateAlias, createdDate, createdByAlias, updatedDate, updatedByAlias, priorityAlias, createdSystem, linkedTcrCount, linkedRqCount, dfOwner, attachmentCount, environmentText[{"property":"name","direction":"ASC"}]
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
viewIdNoViewId for issues - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.IS.viewId automatically.
projectKeyNoProject key - unique identifier for the projectdefault
isJiraIntegratedNoSend true if current project is Integrated with Jira

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, but the description adds valuable behavioral context: viewId is auto-resolved, response contains 'id' rather than a 'DefectId' field, release/cycle filters require numeric IDs not names, and pagination defaults are stated. No contradiction with the readOnly annotation exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized and front-loaded, but it is overlong: the Parameters section duplicates schema text, several Hints repeat parameter lists, and Hints 17-20 are filler. Strong content is present, but it is buried under redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter tool with no output schema, the description covers output shape, viewId resolution, projectKey consistency, filter construction, release/cycle prerequisites, pagination, and concrete examples. An agent has enough information to invoke the tool correctly without external lookup.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description substantially exceeds it by providing exact filter JSON patterns for release, cycle, entityKeyId, comma-separated multiple keys, and sort fields. It also clarifies that viewId should normally be left empty because the system resolves it automatically.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line clearly states the operation and resource: 'Fetch QMetry defects or issues,' and adds the key scoping behavior of automatic viewId resolution. It does not explicitly differentiate itself from siblings like qmetry_fetch_issue_details or qmetry_fetch_issues_linked_to_test_case, though the examples make the list/search intent clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use Cases state concrete triggers: list all issues, search with filters, and get paginated results. Hints give a step-by-step workflow, including using the same projectKey, resolving viewId automatically, and fetching release/cycle IDs from FETCH_RELEASES_AND_CYCLES before filtering. It lacks explicit 'do-not-use-if' routing for fetching a single issue versus this list-oriented tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_executions_by_test_suiteQMetry: Fetch Executions by Test SuiteA
Read-onlyIdempotent
Inspect

Get executions for a given test suite in QMetry

Toolset: Test Suites

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tsID (number) required: Test Suite numeric ID. CRITICAL: the parameter name is 'tsID' β€” do NOT use 'testSuiteId', 'testSuiteID', 'suiteId', or other variants. Accepts a string or number. NOTE: To get the tsID - Call API 'Testsuite/Fetch Testsuite' From the response, get value of following attribute -> data[].id

  • tsFolderID (number): Test Suite folder ID. CRITICAL: the parameter name is 'tsFolderID' β€” do NOT use 'testSuiteFolderId', 'folderId', 'folderID', or other variants. Accepts a string or number. Get from project info response β†’ rootFolders.TS.id. Use FETCH_PROJECT_INFO tool first if not provided by user.

  • gridName (string): Grid Name to be displayed (default 'TESTEXECUTIONLIST')

  • viewId (number): ViewId for test execution - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TE.viewId automatically.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with executions array containing execution details, status, platforms, releases, and execution metadata

Use Cases: 1. Get all executions for a specific test suite for reporting purposes 2. Analyze test execution results and trends within a test suite 3. Filter executions by release, cycle, platform, or automation status 4. Monitor test suite execution performance across different environments 5. Generate execution reports for specific test suites 6. Track execution history and patterns for test suite optimization 7. Validate test suite execution coverage across releases and cycles 8. Audit test execution data for compliance and quality assurance 9. Export execution data for external reporting and analytics

Examples:

  1. Get all executions for test suite ID 194955

{
  "tsID": 194955
}

Expected Output: List of executions for the test suite with execution details, status, and metadata

  1. Get executions with test suite folder and view ID

{
  "tsID": 194955,
  "tsFolderID": 126554,
  "viewId": 41799
}

Expected Output: Executions filtered by test suite folder and specific view configuration

  1. Filter executions by release and cycle

{
  "tsID": 194955,
  "filter": "[{\"type\":\"list\",\"value\":[55178],\"field\":\"releaseID\"},{\"type\":\"list\",\"value\":[111577],\"field\":\"cycleID\"}]"
}

Expected Output: Executions filtered by specific release (55178) and cycle (111577)

  1. Filter executions by platform and automation status

{
  "tsID": 194955,
  "filter": "[{\"type\":\"list\",\"value\":[12345],\"field\":\"platformID\"},{\"type\":\"boolean\",\"value\":true,\"field\":\"isAutomatedFlag\"}]"
}

Expected Output: Automated executions filtered by specific platform (12345)

  1. Get only active (non-archived) executions

{
  "tsID": 194955,
  "filter": "[{\"value\":[0],\"type\":\"list\",\"field\":\"isArchived\"}]"
}

Expected Output: Active executions that are not archived

  1. Get executions with custom pagination and grid name

{
  "tsID": 194955,
  "gridName": "TESTEXECUTIONLIST",
  "page": 1,
  "limit": 25
}

Expected Output: Paginated list of executions with 25 items per page using specific grid configuration

Hints: 1. !MOST IMPORTANT HOW TO GET viewId: 2. CRITICAL: Always resolve and use the correct test execution viewId for the current project when calling this tool. 3. The viewId parameter must be fetched from the active project's info (latestViews.TEL.viewId). 4. Each QMetry project may have a different test execution list viewId, so using a stale or incorrect viewId will result in incomplete or invalid executions list data by test suite id. 5. Usage workflow: 6. 1. Fetch project info for the current project (Admin/Get info Service). 7. 2. Extract latestViews.TEL.viewId from the response. 8. 3. Use this viewId in the Fetch Test Case Runs by Test Suite Run API call. 9. Example: 10. { 11. tsID: 1533730, 12. viewId: 94194, 13. gridName: 'TESTEXECUTIONLIST' 14. } 15. CRITICAL: tsID parameter is REQUIRED - this is the Test Suite numeric ID 16. HOW TO GET tsID: 17. 1. Call API 'Testsuite/Fetch Testsuite' to get available test suites 18. 2. From the response, get value of following attribute -> data[].id 19. 3. Example: Test Suite 'Regression Suite' might have ID 194955 20. HOW TO GET tsFolderID (optional): 21. 1. Call API 'Testsuite/List of folders' to get test suite folders 22. 2. From the response, get value of following attribute -> data[].id 23. 3. Example: Test Suite folder might have ID 126554 24. FILTER CAPABILITIES: Extensive filtering by execution properties 25. FILTER FIELDS: releaseID (list), cycleID (list), platformID (list), isAutomatedFlag (boolean), isArchived (list) 26. RELEASE/CYCLE FILTERING: Use numeric IDs in list format (get from FETCH_RELEASES_AND_CYCLES) 27. PLATFORM FILTERING: Use numeric platform IDs (get from FETCH_PLATFORMS) 28. AUTOMATION STATUS: Use boolean true/false for isAutomatedFlag field 29. ARCHIVE STATUS: 0=Active executions, 1=Archived executions 30. GRID NAME: Default is 'TESTEXECUTIONLIST' - used for execution list display configuration 31. VIEW ID: Optional numeric identifier for specific execution view configurations 32. Multiple filter conditions are combined with AND logic 33. Use pagination for large execution result sets (start, page, limit parameters) 34. This tool is essential for test execution analysis and reporting 35. Critical for monitoring test suite performance and execution trends 36. Use for compliance reporting and execution audit trails 37. Essential for test execution planning and resource optimization

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
tsIDYesTest Suite numeric ID. CRITICAL: the parameter name is 'tsID' β€” do NOT use 'testSuiteId', 'testSuiteID', 'suiteId', or other variants. Accepts a string or number. NOTE: To get the tsID - Call API 'Testsuite/Fetch Testsuite' From the response, get value of following attribute -> data[<index>].id
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
viewIdNoViewId for test execution - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TE.viewId automatically.
gridNameNoGrid Name to be displayed (default 'TESTEXECUTIONLIST')
projectKeyNoProject key - unique identifier for the projectdefault
tsFolderIDNoTest Suite folder ID. CRITICAL: the parameter name is 'tsFolderID' β€” do NOT use 'testSuiteFolderId', 'folderId', 'folderID', or other variants. Accepts a string or number. Get from project info response β†’ rootFolders.TS.id. Use FETCH_PROJECT_INFO tool first if not provided by user.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly/idempotent/destructive safety, and the description adds meaningful behavior: the system auto-resolves viewId from project info, incorrect/stale viewId can yield incomplete results, and filter semantics (AND logic, archive values). Minor internal inconsistency exists between 'system automatically resolves' in the schema and 'always resolve and use' in hints, which prevents a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a wall of text with nine near-duplicate use cases, numbered hints that repeat parameters already in the schema, and contradictory viewId guidance. The useful content is buried under 30+ lines of redundancy; this is not an appropriately concise definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter tool with no output schema, coverage is strong: all parameters are described, examples cover common cases, and the output is summarized as a JSON object with executions and metadata. The internal viewId contradiction (TE vs TEL, auto-resolve vs must-resolve) and lack of a more detailed response format keep it from full completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real value: critical parameter naming caveats (use 'tsID', not variants), how to obtain tsID/tsFolderID, and explicit filter field syntax with examples. This goes beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description opens with a specific action and resource: 'Get executions for a given test suite in QMetry'. It clearly identifies what is returned, but it does not explicitly differentiate this from sibling tools like qmetry_fetch_test_case_runs_by_test_suite_run, so it stops short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section provides clear contexts (reporting, trend analysis, filtering by release/cycle/platform, monitoring, auditing) and the hints give an explicit prerequisite workflow (fetch project info, get viewId). It lacks explicit exclusions or 'use X instead' guidance, so it doesn't earn a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_issue_detailsQMetry: Fetch Issue DetailsA
Read-onlyIdempotent
Inspect

Fetch full detail data for a QMetry issue including UDF field values

Toolset: Issues

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • defectId (number) required: Issue DefectId (numeric ID) β€” use data[].id from Fetch Issues/Defects response. The field in the list API response is named 'id', not 'DefectId'.

Output Description: JSON object with data property containing full issue details including UDFTypeData map and all UDF field values

Use Cases: 1. Get UDF field values for a specific issue 2. Retrieve full issue metadata including custom fields 3. Inspect issue details before updating UDF values

Examples:

  1. Fetch details for issue with DefectId 1430676

{
  "defectId": 1430676
}

Expected Output: Full issue detail object with UDFTypeData map and all UDF field values including MUL1, TCR_STR, etc.

Hints: 1. CRITICAL: Use 'data[].id' from Fetch Issues/Defects response as 'defectId'. The list API response field is named 'id' β€” there is no 'DefectId' field in the list response. Do NOT guess or derive defectId from the entity key suffix. 2. AUTO-RESOLVE: If user provides an issue entity key (e.g. VKMCP2-IS-1, MAC-IS-10), first call Fetch Defects or Issues with filter '[{"type":"string","value":"VKMCP2-IS-1","field":"entityKeyId"}]', then use 'data[].id' as defectId. 3. UDF VALUES: Response includes a 'UDFTypeData' map with all UDF field values for the issue. 4. WORKFLOW: To fetch issue UDF values β€” (1) Fetch Issues with entityKey filter β†’ get data[0].id, (2) in parallel Fetch UDF Layout entityType='IS' pageName='DETAIL' β†’ get field labels/types, (3) call this tool with defectId=data[0].id β†’ read UDFTypeData.

ParametersJSON Schema
NameRequiredDescriptionDefault
defectIdYesIssue DefectId (numeric ID) β€” use data[<index>].id from Fetch Issues/Defects response. The field in the list API response is named 'id', not 'DefectId'.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context: the response shape (JSON object with data property), the presence of a UDFTypeData map, and the important warning that DefectId must be sourced from data[<index>].id rather than guessed from entity keys.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured with clear sections (Toolset, Parameters, Output, Use Cases, Examples, Hints). The critical disambiguation information justifies the length, though some points like UDFTypeData are repeated more than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description's output description fills that gap. It covers the key parameter, workflow, example invocation, expected output shape, and the entity-key resolution path. For a two-parameter read-only tool with strong annotations, this is complete enough for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Even though schema coverage is 100%, the description adds substantial meaning: it explains the critical source of defectId, warns against deriving it from the entity key suffix, provides an AUTO-RESOLVE workflow for entity keys, and gives a concrete example. This goes well beyond the schema's parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Fetch full detail data for a QMetry issue including UDF field values.' This clearly distinguishes the tool from list-oriented siblings like qmetry_fetch_defects_or_issues and mutation tools like qmetry_update_issue, while also signaling it is single-issue detail retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases ('Get UDF field values', 'Retrieve full issue metadata', 'Inspect issue details before updating') and a detailed workflow for resolving entity keys. It does not explicitly name alternatives to avoid, but the context and workflow make the intended usage clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_issue_executionsQMetry: Fetch Issue ExecutionsA
Read-onlyIdempotent
Inspect

Get test case executions linked to a QMetry-native (non-Jira) defect/issue. ALWAYS present results as a unified table: Test Suite Key | Test Suite Name | Release | Cycle | Platform | Executed Version | Execution Status | <UDF Label columns…>. NEVER show a separate type+value UDF breakdown β€” always combine identification fields and UDF values in one table per execution row.

Toolset: Issues

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • linkedAssetId (number) required: Numeric defect ID of the QMetry issue. CRITICAL: parameter name is 'linkedAssetId' β€” do NOT use 'issueId', 'defectId', 'id', or other variants. Accepts a string or number. To get this ID, call the Fetch Defects or Issues tool and use data[].id from the response.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • platformID (string): Platform ID to filter executions by environment/platform

  • filter (string): JSON filter string. Supported fields: tcName (string), linkageLevel (string), executedVersion (string), runStatusName (list of status names), platformID (list of numeric IDs), executionCreatedByLoginAlias (list of usernames), isTestSuiteArchived (list: [1] active, [0] archived, [1,0] both). Example: '[{"type":"string","value":"login","field":"tcName"}]' (default: "[]")

Output Description: JSON object with 'data' array of execution records, 'hasTcRunUdf' boolean flag, and 'total' count. Each execution record ALWAYS contains these mandatory identification fields: 'tsEntityKey' (Test Suite Key, e.g. 'MAC-TS-42'), 'tsName' (Test Suite Name), 'releaseName' (Release), 'cycleName' (Cycle), 'platformName' (Platform/environment), 'executedVersion' (Executed Version of the test case), 'runStatusName' (Execution Status label), 'tcRunID' (numeric Test Run ID), 'tcName' (Test Case Name), 'tcEntityKey' (Test Case Key), and 'testRunUdfs' (array of objects each with name, label, fieldID, fieldType, value β€” use 'label' for display headers, null if not set). For LOOKUPLIST and MULTILOOKUPLIST fields, 'value' contains the resolved human-readable display name (from qmUDFList), not the raw internal uniqueLabel key. ALL project-defined UDF fields are always included, even those with no value. When hasTcRunUdf is false, a 'testRunUdfNote' field provides a professional explanation instead.

Use Cases: 1. Get all test executions linked to a specific defect 2. Audit which test cases were run against a given issue 3. Filter executions by run status (failed, passed, etc.) for an issue 4. Filter executions by platform/environment for an issue 5. Filter executions by tester/executor for an issue 6. Show archived and active test suite executions for an issue 7. View UDF (custom field) values on executions linked to an issue 8. Track test coverage and execution progress for a defect

Examples:

  1. Get all executions linked to issue ID 9598240

{
  "linkedAssetId": 9598240
}

Expected Output: Present as ONE unified table β€” never as a separate type+value UDF breakdown. Example: | Test Suite Key | Test Suite Name | Release | Cycle | Platform | Executed Version | Execution Status | Tested By | Environments UDF | Execution Type | | MAC-TS-42 | Regression Suite | R1 | Sprint1 | Chrome | 1 | Failed | varis | chrome, edge, safari | Functional | | MAC-TS-43 | Login Suite | R1 | Sprint1 | Firefox | 2 | Blocked | john | firefox | Regression | Columns in order: Test Suite Key (tsEntityKey) | Test Suite Name (tsName) | Release (releaseName) | Cycle (cycleName) | Platform (platformName) | Executed Version (executedVersion) | Execution Status (runStatusName) | then one column per UDF label. Use the UDF 'label' as column header. Show null UDF values as '-'.

  1. Get executions with pagination (page 1, 20 records)

{
  "linkedAssetId": 9598240,
  "page": 1,
  "start": 0,
  "limit": 20
}

Expected Output: First 20 executions linked to the issue

  1. Filter executions by run status (failed or passed)

{
  "linkedAssetId": 9509016,
  "filter": "[{\"type\":\"list\",\"field\":\"runStatusName\",\"value\":[\"failed\",\"passed\"]}]",
  "page": 1,
  "start": 0,
  "limit": 20
}

Expected Output: Executions with failed or passed status for the issue

  1. Filter executions by test case name

{
  "linkedAssetId": 9509016,
  "filter": "[{\"type\":\"string\",\"value\":\"login\",\"field\":\"tcName\"}]",
  "page": 1,
  "start": 0,
  "limit": 20
}

Expected Output: Executions where test case name contains 'login'

  1. Filter by platform, status, and tester

{
  "linkedAssetId": 9509016,
  "filter": "[{\"type\":\"list\",\"field\":\"runStatusName\",\"value\":[\"failed\"]},{\"type\":\"list\",\"field\":\"platformID\",\"value\":[100145]},{\"type\":\"list\",\"field\":\"executionCreatedByLoginAlias\",\"value\":[\"Varis Khan\"]}]",
  "page": 1,
  "start": 0,
  "limit": 20
}

Expected Output: Failed executions on platform 100145 created by Varis Khan

  1. Filter by status and include archived test suite executions

{
  "linkedAssetId": 9509016,
  "filter": "[{\"type\":\"list\",\"field\":\"runStatusName\",\"value\":[\"failed\",\"passed\"]},{\"value\":[1,0],\"type\":\"list\",\"field\":\"isTestSuiteArchived\"}]",
  "page": 1,
  "start": 0,
  "limit": 20
}

Expected Output: Executions with failed/passed status including archived test suites

  1. Filter by execution version and linkage level

{
  "linkedAssetId": 9509016,
  "filter": "[{\"type\":\"string\",\"value\":\"1\",\"field\":\"executedVersion\"},{\"type\":\"string\",\"value\":\"Test Case\",\"field\":\"linkageLevel\"}]",
  "page": 1,
  "start": 0,
  "limit": 20
}

Expected Output: Executions at Test Case linkage level for version 1

Hints: 1. === MANDATORY RESPONSE FORMAT β€” READ THIS BEFORE RENDERING ANY OUTPUT === 2. 3. PIVOT RULE β€” CRITICAL: 4. The 'testRunUdfs' field on each execution is an array of { name, label, fieldID, fieldType, value }. 5. You MUST pivot this array into TABLE COLUMNS β€” do NOT render it as rows. 6. β†’ Each testRunUdfs[i].label = a column header in the unified table 7. β†’ Each testRunUdfs[i].value = the cell value for that execution's row 8. β†’ testRunUdfs[i].fieldType = INTERNAL METADATA β€” NEVER show this as a column 9. β†’ testRunUdfs[i].fieldID = INTERNAL METADATA β€” NEVER show this as a column 10. 11. FORBIDDEN PATTERNS β€” NEVER do any of these: 12. ❌ Do NOT render a separate sub-table (UDF Label | Type | Value) per execution 13. ❌ Do NOT show 'Type' or 'fieldType' as a visible column 14. ❌ Do NOT group output by tcRunID with individual breakdowns beneath each 15. ❌ Do NOT show raw UDF field keys (e.g. 'TRString', '8260LUP') as headers β€” use 'label' 16. 17. REQUIRED OUTPUT β€” ONE unified table, all executions as rows: 18. | Test Suite Key | Test Suite Name | Release | Cycle | Platform | Executed Version | Execution Status | <UDF Label 1> | <UDF Label 2> | ... | 19. |----------------|-----------------|---------|-------|----------|------------------|------------------|---------------|---------------|-----| 20. | MAC-TS-42 | Login Suite | R1 | S1 | Chrome | v1 | Failed | varis | chrome, edge | ... | 21. 22. MANDATORY COLUMNS (always first, in this order): 23. 1. Test Suite Key β†’ tsEntityKey (e.g. 'MAC-TS-42') 24. 2. Test Suite Name β†’ tsName (test suite display name) 25. 3. Release β†’ releaseName 26. 4. Cycle β†’ cycleName 27. 5. Platform β†’ platformName 28. 6. Executed Version β†’ executedVersion 29. 7. Execution Status β†’ runStatusName 30. 8. Tested By β†’ executionCreatedByLoginAlias/testedBy when present 31. 9+. One column per UDF field β€” use testRunUdfs[i].label as header, testRunUdfs[i].value as cell. 32. 33. Null UDF values β†’ show as '-'. If hasTcRunUdf is false, show columns 1-8 only. 34. ISSUE EXECUTION UDF SOURCE β€” CRITICAL: 35. Do NOT call 'Fetch Test Run UDF Values' for issue execution UDFs. 36. Do NOT create or use another issue-specific UDF fetch tool. 37. Use this tool's response directly: it calls /rest/execution/getExecutionsForIssue for execution rows, parses each row's udfjson for saved UDF values, and uses Test Run UDF metadata to include all configured UDF labels with null/empty values. 38. === END MANDATORY RESPONSE FORMAT === 39. 40. CRITICAL: linkedAssetId is REQUIRED - this is the numeric defect ID from QMetry (not entity key like VKT-IS-5) 41. HOW TO GET linkedAssetId: Call Fetch Defects or Issues tool β†’ use data[].id from the response 42. AUTO-RESOLVE: If user provides an issue entity key (e.g. VKT-IS-5, MAC-IS-10), first call Fetch Defects or Issues with that entity key as filter, extract data[].id, then use it as linkedAssetId 43. AUTO-RESOLVE FILTER EXAMPLE: to resolve VKT-IS-5 β†’ use filter '[{"type":"string","value":"VKT-IS-5","field":"entityKeyId"}]' in Fetch Defects or Issues tool 44. This tool supports QMetry-native issues only β€” do NOT use for Jira-integrated projects 45. API SOURCE: Execution rows and saved UDF values come from /rest/execution/getExecutionsForIssue. The udfjson field contains saved Test Run UDF values, e.g. Tested_By, execution_type, Country_mcp_udf, environments_udf. 46. METADATA SOURCE: This tool also calls Test Run UDF metadata once to get all available labels, fieldIDs, field types, list options (qmUDFList), and empty fields. Merge metadata fields with udfjson values by UDF name. 47. RESPONSE FIELDS: hasTcRunUdf=true means executions have UDF data; each execution includes a 'testRunUdfs' array with ALL project-defined UDF fields 48. ALL UDF FIELDS: ALL project-defined Test Run UDF fields are returned for every execution β€” including fields not yet set (value: null) 49. Each element in testRunUdfs: { name, label, fieldID, fieldType, value } β€” use fieldID when calling 'Bulk Update Test Run UDFs' 50. VALUE RESOLUTION: For LOOKUPLIST and MULTILOOKUPLIST fields, values are resolved from their internal uniqueLabel key to the human-readable display name using qmUDFList lookup options. Always display the resolved name, not the raw uniqueLabel. 51. EXAMPLE testRunUdfs: [{ "name": "TRString", "label": "TR String", "fieldID": 229241, "fieldType": "STRING", "value": "test" }, { "name": "lookup_browser", "label": "Lookup Browser MCP", "fieldID": 229433, "fieldType": "LOOKUPLIST", "value": "Chrome" }, { "name": "dateField", "label": "Date", "fieldID": 229255, "fieldType": "DATETIMEPICKER", "value": null }] 52. FILTER FIELDS: 53. - tcName (string): filter by test case name substring 54. - linkageLevel (string): 'Test Case' or 'Test Step' 55. - executedVersion (string): version number as string e.g. '1' 56. - runStatusName (list): e.g. ["failed","passed","in progress"] 57. - platformID (list): numeric platform IDs e.g. [100145]. Get from FETCH_PLATFORMS tool 58. - executionCreatedByLoginAlias (list): usernames/login aliases e.g. ["john.doe"] 59. - isTestSuiteArchived (list): [1] active only, [0] archived only, [1,0] both 60. FILTER FORMAT: JSON string array β€” '[{"type":"list","field":"runStatusName","value":["failed"]}]' 61. Multiple filter conditions are combined with AND logic 62. Use pagination (page, start, limit) for large result sets 63. Get platform IDs using the FETCH_PLATFORMS tool before filtering by platformID 64. Execution status names are case-sensitive β€” use lowercase: 'failed', 'passed', 'in progress', 'blocked', 'not run' 65. hasTcRunUdf: false β†’ No Test Run UDFs configured; testRunUdfs will not appear; a 'testRunUdfNote' field explains this.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoJSON filter string. Supported fields: tcName (string), linkageLevel (string), executedVersion (string), runStatusName (list of status names), platformID (list of numeric IDs), executionCreatedByLoginAlias (list of usernames), isTestSuiteArchived (list: [1] active, [0] archived, [1,0] both). Example: '[{"type":"string","value":"login","field":"tcName"}]'[]
platformIDNoPlatform ID to filter executions by environment/platform
projectKeyNoProject key - unique identifier for the projectdefault
linkedAssetIdYesNumeric defect ID of the QMetry issue. CRITICAL: parameter name is 'linkedAssetId' β€” do NOT use 'issueId', 'defectId', 'id', or other variants. Accepts a string or number. To get this ID, call the Fetch Defects or Issues tool and use data[<index>].id from the response.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior, and the description adds substantial behavioral detail without contradicting them. It discloses the underlying API (/rest/execution/getExecutionsForIssue), the udfjson parsing behavior, mandatory response fields, UDF value resolution from qmUDFList, null-value handling, and the hasTcRunUdf false behavior β€” far more than annotations alone provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Although the description is well-sectioned and front-loaded, it is heavily redundant: the mandatory unified-table/pivot instruction appears in the opening paragraph, Output Description, Use Case examples, and again in the numbered Hints block. The Hints section restates columns, UDF handling, filter fields, and examples that were already given, so many sentences do not earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the full burden of explaining return values β€” and it does so thoroughly. It inventories the data array fields, mandatory identification columns, UDF object shape, total count, hasTcRunUdf flag, testRunUdfNote behavior, filter format, pagination, and ID resolution workflow. Nothing essential for calling or interpreting the tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds significant value beyond the schema. It explains precisely how to obtain linkedAssetId from another tool, warns against using incorrect parameter names, documents every filter field with types and examples, specifies case-sensitivity of status names, and clarifies pagination semantics. This materially improves correct invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource statement: 'Get test case executions linked to a QMetry-native (non-Jira) defect/issue.' This precisely distinguishes it from sibling fetch tools like qmetry_fetch_test_case_executions and qmetry_fetch_executions_by_test_suite, and the 'non-Jira' qualifier further narrows scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly explains when to use it (for executions linked to a QMetry-native issue), when NOT to use it (Jira-integrated projects), and explicitly routes to alternatives such as 'Fetch Defects or Issues' for resolving linkedAssetId and 'FETCH_PLATFORMS' for platform IDs. It also warns against calling 'Fetch Test Run UDF Values' for this data, which is strong exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_issues_linked_to_test_caseQMetry: Fetch Issues Linked to Test CaseA
Read-onlyIdempotent
Inspect

Get issues that are linked (or not linked) to a specific test case in QMetry

Toolset: Issues

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tcID (number) required: Test Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.

  • getLinked (boolean): True to get only those issues that are linked with this Test case Run, False to get those issues which are not linked with this Test case Run. Default value true (get linked issues). (default: true)

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with issues array containing issue details, priorities, status, and linkage information

Use Cases: 1. Get all issues linked to a specific test case for defect tracking 2. Find issues that are NOT linked to a test case (gap analysis) 3. Generate traceability reports between test cases and issues 4. Filter issues by type, priority, status, or owner 5. Monitor issue resolution progress for specific test cases 6. Audit issue-test case relationships for compliance 7. Filter issues by summary content or execution version 8. Get issue details for test execution planning 9. Track linkage level (Test Case vs Test Step level) 10. Quality assurance - ensure proper issue tracking

Examples:

  1. Get all issues linked to test case ID 4495658 (default behavior)

{
  "tcID": 4495658
}

Expected Output: List of issues linked to the test case with issue details, status, and metadata

  1. Get all issues linked to test case ID 4495658 (explicit)

{
  "tcID": 4495658,
  "getLinked": true
}

Expected Output: List of issues linked to the test case with issue details, status, and metadata

  1. Get issues NOT linked to test case (gap analysis)

{
  "tcID": 4495658,
  "getLinked": false
}

Expected Output: List of issues that are NOT linked to the test case

  1. Get linked issues with pagination

{
  "tcID": 4495658,
  "getLinked": true,
  "limit": 25,
  "page": 1
}

Expected Output: Paginated list of issues linked to the test case

  1. Filter linked issues by summary content (using default getLinked=true)

{
  "tcID": 4495658,
  "filter": "[{\"value\":\"login\",\"type\":\"string\",\"field\":\"summary\"}]"
}

Expected Output: Issues linked to test case that contain 'login' in their summary

  1. Filter linked issues by status and priority

{
  "tcID": 4495658,
  "getLinked": true,
  "filter": "[{\"value\":[1,2],\"type\":\"list\",\"field\":\"issueState\"},{\"value\":[1],\"type\":\"list\",\"field\":\"issuePriority\"}]"
}

Expected Output: High priority issues in Open or In Progress status

  1. Filter issues by execution version

{
  "tcID": 4495658,
  "getLinked": true,
  "filter": "[{\"value\":\"2\",\"type\":\"string\",\"field\":\"executedVersion\"}]"
}

Expected Output: Issues linked to version 2 of the test case execution

Hints: 1. CRITICAL: tcID parameter is REQUIRED - this is the Test Case numeric ID 2. getLinked parameter is OPTIONAL - defaults to true if not provided 3. HOW TO GET tcID: 4. 1. Call FETCH_TEST_CASES with filter on entityKeyId to resolve test case key to numeric ID 5. 2. From response, use data[index].tcID field 6. 3. Example: MAC-TC-1684 β†’ tcID: 4495658 7. getLinked=true (default): Returns issues that ARE linked to the test case 8. getLinked=false: Returns issues that are NOT linked to the test case (useful for gap analysis) 9. If getLinked is not specified, it defaults to true (linked issues) 10. FILTER CAPABILITIES: Extensive filtering by issue properties 11. FILTER FIELDS: summary (string), executedVersion (string), linkageLevel (string), issueType (list), issuePriority (list), issueState (list), owner (list) 12. LINKAGE LEVEL: 'Test Case' for test case level links, 'Test Step' for step level links 13. ISSUE TYPE IDs: Typically 1=Bug, 2=Enhancement, 3=Task (verify with your QMetry instance) 14. ISSUE PRIORITY IDs: Typically 1=High, 2=Medium, 3=Low (verify with your QMetry instance) 15. ISSUE STATUS IDs: Typically 1=Open, 2=In Progress, 3=Resolved, 4=Closed (verify with your QMetry instance) 16. OWNER IDs: Use numeric user IDs from QMetry user management 17. Multiple filter conditions are combined with AND logic 18. Use pagination for large issue result sets (start, page, limit parameters) 19. This tool is essential for defect tracking and traceability audits 20. Helps establish relationships between test failures and reported issues 21. Critical for impact analysis when test cases change 22. Use for compliance reporting and quality metrics

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
tcIDYesTest Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
getLinkedNoTrue to get only those issues that are linked with this Test case Run, False to get those issues which are not linked with this Test case Run. Default value true (get linked issues).
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds genuine behavioral context beyond that: filter conditions combine with AND logic, linkageLevel distinguishes 'Test Case' vs 'Test Step', and filter ID mappings for issueType/issuePriority/issueState are disclosed. It does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Sections (Parameters, Output, Use Cases, Examples, Hints) make it navigable and the critical tcID/getLinked facts are usefully front-loaded and repeated. But it is over-long and redundant: hints 1-2 and 7-9 restate the parameter section, six of the ten use cases say essentially 'traceability/compliance', and the numbered sub-list for 'HOW TO GET tcID' is malformed. Several closing hints ('essential for defect tracking', 'critical for impact analysis') are filler that does not help an agent invoke the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with no output schema and non-trivial filter JSON syntax, coverage is strong: return shape is described (issues array with priorities, status, linkage info), filter syntax is shown in three examples, ID mappings are given, and pagination is demonstrated. Minor gaps remain: the interaction between the start and page pagination parameters is never clarified, and error behavior (invalid tcID, empty results) is not addressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, giving a baseline of 3, but the description adds real value beyond the schema: the CRITICAL naming warning for tcID, a step-by-step procedure to resolve MAC-TC-1684 β†’ tcID via FETCH_TEST_CASES, the list of valid filter field names (summary, executedVersion, linkageLevel, issueType, issuePriority, issueState, owner), and the numeric ID-to-label mappings used inside filter values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource: 'Get issues that are linked (or not linked) to a specific test case in QMetry'. The linked/not-linked duality is stated up front, and the tcID hints explicitly distinguish this from entity-key lookups and from the sibling qmetry_fetch_linked_issues_of_test_case_run by scoping to a test case, not a test case run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides 10 concrete use cases (defect tracking, gap analysis, traceability audits) and seven worked examples covering the getLinked=true/false distinction, pagination, and filtering. Also gives a resolution procedure referencing sibling FETCH_TEST_CASES to obtain tcID. However, it never explicitly excludes the near-sibling qmetry_fetch_linked_issues_of_test_case_run or states when that alternative should be chosen instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_linked_issues_of_test_case_runQMetry: Fetch Linked Issues of Test Case RunA
Read-onlyIdempotent
Inspect

Get issues that are linked (or not linked) to a specific test case run in QMetry

Toolset: Issues

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • entityId (number) required: Id of Test case run (required for fetching linked issues). CRITICAL: the parameter name is 'entityId' β€” do NOT use 'tcRunId', 'testCaseRunId', 'runId', or other variants. Accepts a string or number. NOTE: To get the entityId - Call API 'Execution/Fetch Testcase Run ID' From the response, get value of following attribute -> data[].tcRunID

  • getLinked (boolean): True to get only those issues that are linked with this Test case Run, False to get those issues which are not linked with this Test case Run. Default value true (get linked issues). (default: true)

  • getColumns (boolean): Whether to get column information in response. (default: true)

  • istcrFlag (boolean): Set True for test case run operations (default: true)

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with issues array containing issue details, priorities, status, owner information, and linkage metadata

Use Cases: 1. Get all issues linked to a specific test case run for defect tracking 2. Find issues that are NOT linked to a test case run (gap analysis) 3. Generate defect reports and traceability matrix for test case runs 4. Monitor issue resolution progress for specific test case executions 5. Analyze test execution quality by examining linked defects 6. Filter issues by type, priority, status, or owner for test case runs 7. Audit issue-test case run relationships for compliance 8. Track defect lifecycle in relation to test execution results 9. Quality assurance - ensure proper issue tracking for failed test runs 10. Impact analysis - see which issues affect specific test executions

Examples:

  1. Get all issues linked to test case run ID 1121218

{
  "entityId": 1121218,
  "getColumns": true,
  "getLinked": true
}

Expected Output: List of issues linked to the test case run with issue details, status, and metadata

  1. Get issues NOT linked to test case run (gap analysis)

{
  "entityId": 1121218,
  "getColumns": true,
  "getLinked": false
}

Expected Output: List of issues that are NOT linked to test case run for gap analysis

  1. Filter linked issues by issue type and status

{
  "entityId": 1121218,
  "getColumns": true,
  "getLinked": true,
  "filter": "[{\"type\":\"list\",\"value\":[1],\"field\":\"typeAlias\"},{\"type\":\"list\",\"value\":[1,2],\"field\":\"stateAlias\"}]"
}

Expected Output: Bug type issues in Open or In Progress status

  1. Search linked issues by name and priority

{
  "entityId": 1121218,
  "getColumns": true,
  "getLinked": true,
  "filter": "[{\"type\":\"string\",\"value\":\"login\",\"field\":\"name\"},{\"type\":\"list\",\"value\":[1],\"field\":\"priorityAlias\"}]"
}

Expected Output: High priority issues containing 'login' in their name

  1. Filter issues by date range and entity key

{
  "entityId": 1121218,
  "getColumns": true,
  "getLinked": true,
  "filter": "[{\"value\":\"2024-01-01\",\"type\":\"date\",\"field\":\"createdDate\",\"comparison\":\"gt\"},{\"value\":\"2024-12-31\",\"type\":\"date\",\"field\":\"createdDate\",\"comparison\":\"lt\"},{\"type\":\"string\",\"value\":\"BUG-001,BUG-002\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Specific issues created within date range

  1. Filter issues by owner and created system

{
  "entityId": 1121218,
  "getColumns": true,
  "getLinked": true,
  "filter": "[{\"type\":\"list\",\"value\":[123],\"field\":\"dfOwner\"},{\"type\":\"list\",\"value\":[\"QMetry\"],\"field\":\"createdSystem\"}]"
}

Expected Output: Issues owned by specific user and created in QMetry

Hints: 1. WORKFLOW CRITICAL: NEVER use user-provided IDs directly as entityId! 2. ALWAYS fetch execution data first to get proper tcRunID values! 3. 4. WHEN USER ASKS: 'fetch linked issues of test suite [ID]' OR 'linked issues of test run [ID]': 5. STEP 1: Identify what type of ID the user provided 6. STEP 2A: If Test Suite ID β†’ fetch executions by test suite β†’ get tsRunID β†’ fetch test runs β†’ get tcRunID 7. STEP 2B: If Test Run ID β†’ fetch test case runs by test suite run β†’ get tcRunID 8. STEP 2C: If Test Case ID β†’ fetch test case executions β†’ get tcRunID 9. STEP 3: Use tcRunID as entityId for this tool 10. 11. ID HIERARCHY: Test Suite β†’ Test Suite Runs β†’ Test Case Runs (tcRunID = entityId) 12. ID HIERARCHY: Test Case β†’ Test Case Executions (tcRunID = entityId) 13. 14. CRITICAL: entityId parameter is REQUIRED - this is the Test Case Run numeric ID (tcRunID) 15. HOW TO GET entityId: 16. 1. Call appropriate execution APIs to get test case runs 17. 2. From the response, extract data[].tcRunID 18. 3. Use tcRunID as entityId for this tool 19. 4. Example: tcRunID 1121218 becomes entityId: 1121218 20. 21. getLinked=true (default): Returns issues that ARE linked to the test case run 22. getLinked=false: Returns issues that are NOT linked to the test case run (useful for gap analysis) 23. istcrFlag=true (default): Set to true for test case run operations 24. getColumns=true (default): Include column metadata in response 25. 26. FILTER CAPABILITIES: Support extensive filtering by issue properties 27. FILTER FIELDS: name (string), typeAlias (list), stateAlias (list), entityKeyId (string), createdDate (date with comparison), createdByAlias (list), updatedDate (date with comparison), createdSystem (list), updatedByAlias (list), dfOwner (list), priorityAlias (list), linkedTcrCount (numeric), linkedRqCount (numeric), attachmentCount (numeric), componentAlias (list), environmentText (string), affectedRelease (list) 28. ISSUE TYPE IDs: Typically 1=Bug, 2=Enhancement, 3=Task (verify with your QMetry instance) 29. ISSUE STATE IDs: Typically 1=Open, 2=In Progress, 3=Resolved, 4=Closed (verify with your QMetry instance) 30. ISSUE PRIORITY IDs: Typically 1=High, 2=Medium, 3=Low (verify with your QMetry instance) 31. DATE FILTERING: Use 'gt' (greater than) and 'lt' (less than) comparisons for date fields 32. ENTITY KEY SEARCH: Use comma-separated values for multiple issue keys 33. CREATED SYSTEM: Use 'QMetry' or 'JIRA' to filter by creation system 34. OWNER IDs: Use numeric user IDs from QMetry user management 35. COMPONENT/LABEL IDs: Use numeric IDs for component/label filtering 36. ENVIRONMENT TEXT: Filter by environment description text 37. AFFECTED RELEASE: Use release IDs for filtering by affected releases 38. LINKED COUNT FILTERS: Use numeric values for linkedTcrCount, linkedRqCount, attachmentCount 39. Multiple filter conditions are combined with AND logic 40. Use pagination for large result sets (start, page, limit parameters) 41. This tool is essential for defect tracking and traceability audits 42. Critical for understanding test execution quality and issue relationships 43. Use for compliance reporting and issue lifecycle management 44. Helps establish relationships between test failures and reported issues 45. Essential for impact analysis when test case runs change or fail

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
entityIdYesId of Test case run (required for fetching linked issues). CRITICAL: the parameter name is 'entityId' β€” do NOT use 'tcRunId', 'testCaseRunId', 'runId', or other variants. Accepts a string or number. NOTE: To get the entityId - Call API 'Execution/Fetch Testcase Run ID' From the response, get value of following attribute -> data[<index>].tcRunID
getLinkedNoTrue to get only those issues that are linked with this Test case Run, False to get those issues which are not linked with this Test case Run. Default value true (get linked issues).
istcrFlagNoSet True for test case run operations
getColumnsNoWhether to get column information in response.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, and the description does not contradict these. It adds valuable behavioral detail: getLinked toggles linked vs. unlinked issues, entityId must be sourced from a prior execution API response, filter conditions are ANDed, and the response is an issues array. It does not discuss error cases or how getColumns affects the response shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is highly repetitive and overlong for a simple read-only fetch tool. The first sentence is clear and front-loaded, but the use-case list is largely filler, and the Hints section repeats the same entityId workflow and filter guidance multiple times.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter tool with no output schema, this description is exceptionally complete: it documents output as a JSON object with issues, all parameters through the schema plus examples, pagination, filter fields, default behaviors, and the critical ID-provenance workflow. An agent has enough context to call this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so a baseline of 3 applies, but the description substantially exceeds that baseline. It explains how to obtain entityId from data[<index>].tcRunID, documents getLinked semantics, enumerates filterable fields and comparison operators, and provides six worked JSON examples with valid filter syntax. The only minor wrinkle is saying entityId accepts a string or number while the schema types it as number.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get issues that are linked (or not linked) to a specific test case run in QMetry.' This clearly distinguishes the tool's scope from nearby QMetry tools that fetch issues for test cases, requirements, or suites.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives strong usage context: getLinked=true versus false for gap analysis, extensive use cases, and critical workflow prerequisites such as 'NEVER use user-provided IDs directly as entityId' and 'ALWAYS fetch execution data first.' It does not explicitly name alternative sibling tools or say when not to use this tool over them, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_platformsQMetry: Fetch PlatformsA
Read-onlyIdempotent
Inspect

Fetch QMetry platforms from the current project

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • sort (string): Sort criteria as JSON string (default '[{"property":"platformID","direction":"DESC"}]') (default: "[{"property":"platformID","direction":"DESC"}]")

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with platforms list and pagination metadata

Use Cases: 1. Fetch all platforms from the current project 2. Get platform metadata for test execution planning 3. List platforms for test environment selection 4. Filter platforms by name or properties 5. Get paginated platform results for large projects 6. Retrieve platform information for cross-platform testing 7. Search for specific platforms using filters 8. Get platform details for test execution assignment

Examples:

  1. Get all platforms (default behavior)

{}

Expected Output: List of all platforms with default pagination (10 items per page)

  1. Get platforms with custom pagination

{
  "page": 1,
  "limit": 10,
  "start": 0
}

Expected Output: List of platforms with custom pagination settings

  1. Filter platforms by name

{
  "filter": "[{\"value\":\"Chrome\",\"type\":\"string\",\"field\":\"name\"}]"
}

Expected Output: Filtered list of platforms matching the name criteria

  1. Filter platforms by archive status

{
  "filter": "[{\"value\":[1,0],\"type\":\"list\",\"field\":\"isArchived\"}]"
}

Expected Output: List of platforms filtered by archive status (archived and non-archived)

  1. Get only archived platforms

{
  "filter": "[{\"value\":[1],\"type\":\"list\",\"field\":\"isArchived\"}]"
}

Expected Output: List of only archived platforms

  1. Get only active/non-archived platforms

{
  "filter": "[{\"value\":[0],\"type\":\"list\",\"field\":\"isArchived\"}]"
}

Expected Output: List of only active/non-archived platforms

  1. Get platforms with custom sorting

{
  "sort": "[{\"property\":\"name\",\"direction\":\"ASC\"}]"
}

Expected Output: List of platforms sorted by name in ascending order

Hints: 1. Use 'default' project key when user doesn't specify one 2. Default pagination: start=0, page=1, limit=10 3. Filter parameter should be a JSON string with filter criteria 4. Sort parameter should be a JSON string with sort criteria 5. Default sort: platformID descending 6. Common filter fields: 'name' (string), 'isArchived' (list of 0,1) for archive status 7. IMPORTANT: Always use 'isArchived' field for filtering by archive status, even though response shows 'isPlatformArchived' 8. Archive status values: 0 = active/non-archived, 1 = archived 9. Empty payload {} is sent when no parameters are provided 10. Use platforms for cross-platform testing and environment selection

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
sortNoSort criteria as JSON string (default '[{"property":"platformID","direction":"DESC"}]')[{"property":"platformID","direction":"DESC"}]
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnly, idempotent, and non-destructive behavior, and the description adds substantial behavioral context: default project key, default pagination, default sort order, JSON-string filter/sort syntax, and the important gotcha that filtering must use 'isArchived' even though responses show 'isPlatformArchived'. It also clarifies archive status value meanings.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is heavily over-long and repetitive: parameter defaults are restated from the schema, eight overlapping use cases are listed, seven examples cover mostly the same scenarios, and ten hints repeat earlier information. It is well-sectioned, but many sentences do not earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite lacking an output schema, the description provides enough through its output description, examples, and hints: expected output shape, pagination behavior, filtering patterns, archive-status handling, and default behavior. An agent can confidently select and call this tool without external lookups.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Though the schema already documents all six parameters with 100% coverage, the description adds significant practical meaning beyond it: concrete JSON examples for filter and sort, common filter fields, archive status semantics, and the empty-payload convention for defaults. This is exactly the kind of extra guidance agents need to invoke it correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Fetch QMetry platforms from the current project.' It clearly distinguishes this tool from sibling fetch tools (releases, builds, test cases) by naming the exact resource and scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section gives clear context for when to call this tool, such as fetching all platforms, filtering by name or archive status, and retrieving paginated results. It does not explicitly name alternatives or exclusions, but the fetch-versus-link sibling tools are distinct enough that the intended usage is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_qmetry_list_projectsQMetry: Fetch QMetry list ProjectsA
Read-onlyIdempotent
Inspect

Fetch QMetry projects list including projectID, name, projectKey, isArchived, viewIds and folderPath needed for other operations

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • params (object) required

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object containing list of projects details, including Version, Build Number(if greater than or equal to 1), Project ID, name, Project Key and Date Format

Use Cases: 1. Get project list to check user how many project access to particular apikey 2. Retrieve available fields of each project list including projectID, name, projectKey, isArchived, viewIds and folderPath needed for other operations 3. Validate project access and permissions

Examples:

  1. Get list of project available to user

{
  "params": {
    "showArchive": false
  }
}

Expected Output: Project active/non archived list including some important fields like projectID, name, projectKey, isArchived, viewIds and folderPath needed for other operations

  1. Get projects with custom pagination

{
  "params": {
    "showArchive": false
  },
  "page": 1,
  "limit": 10,
  "start": 0
}

Expected Output: List of projects with custom pagination settings

  1. Get not active/archived projects

{
  "params": {
    "showArchive": true
  }
}

Expected Output: List of all projects including archived ones (showArchive: true sent in payload)

  1. Filter projects by name

{
  "filter": "[{\"value\":\"MAC\",\"type\":\"string\",\"field\":\"name\"}]"
}

Expected Output: Filtered list of projects matching the name criteria

  1. Filter projects by project key

{
  "filter": "[{\"value\":\"MAC\",\"type\":\"string\",\"field\":\"projectKey\"}]"
}

Expected Output: List of projects filtered by project key (e.g. 'MAC', 'UT', etc.)

Hints: 1. Fetch list of projects available to user 2. Use 'default' project key when user doesn't specify one 3. Use params.showArchive: true/false to get archived/non-archived projects, default is false when not provided 4. Pagination supported for large result sets (start, page, limit parameters) 5. Filter parameter should be a JSON string with filter criteria 6. Common filter fields: 'name' (string), 'projectKey' (string)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
paramsYes
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the read-only safety profile is covered. The description adds meaningful context beyond that: showArchive behavior, pagination semantics, filter-as-JSON-string, projectKey defaulting, and approximate output fields. There is a minor inconsistency between the fields listed in the opening line and the Output Description, but overall the behavioral picture is strong.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than average, but it is well structured with Parameters, Output Description, Use Cases, Examples, and Hints sections. It is front-loaded with the core purpose. There is some redundancy between the parameter list, examples, and hints, but the organization makes the length manageable for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description compensates with an Output Description, expected-output examples, and hints about filtering and pagination. It covers the main things an agent needs to call this tool correctly. The slight mismatch between the opening field list and Output Description prevents a perfect score.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 83%, so the schema already documents most parameters well. The description adds value by giving concrete examples of filter payloads, clarifying showArchive true/false behavior, and explaining how start/page/limit work together. This goes beyond the schema without needing to restate everything.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Fetch QMetry projects list', and enumerates the exact fields returned (projectID, name, projectKey, isArchived, viewIds, folderPath). This clearly separates it from sibling tools like qmetry_fetch_qmetry_project_info and qtm4j_get_projects, which are singular/context-setting operations rather than project-list fetches.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section gives concrete contexts: checking how many projects an API key can access, retrieving project fields needed for other operations, and validating project access/permissions. This is clear usage context, though it does not explicitly name alternatives or state when not to use this tool, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_qmetry_project_infoQMetry: Fetch QMetry Project InfoA
Read-onlyIdempotent
Inspect

Fetch QMetry project information including viewId and folderPath needed for other operations

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

Output Description: JSON object containing project details, viewIds, folderPaths, project configuration, 'dateTimeFormatID' (active date format ID), and 'dateTimeFormatNew' array (each entry: { id, name, unique_value } β€” use unique_value of the matching entry as the date format pattern for all API date fields).

Use Cases: 1. Get project configuration before fetching test cases 2. Retrieve available viewIds for test case listing 3. Get folderPath information for project navigation 4. Validate project access and permissions

Examples:

  1. Get default project info

{}

Expected Output: Project configuration with viewIds, folderPaths, and project details

  1. Get specific project info

{
  "projectKey": "MAC"
}

Expected Output: MAC project configuration with available views and folders

Hints: 1. Always call this first when user doesn't provide viewId or folderPath 2. Use 'default' project key when user doesn't specify one 3. Extract viewId from latestViews.TC.viewId for test case operations 4. Use empty string '' as folderPath for root directory 5. DATE FORMAT β€” IMPORTANT: response contains 'dateTimeFormatID' (number) and 'dateTimeFormatNew' (array). 6. dateTimeFormatID = active format ID for this project. 7. dateTimeFormatNew = [{ id, name, unique_value }] β€” find entry where id === dateTimeFormatID. 8. unique_value is the authoritative date format pattern using Java/QMetry conventions: 9. yyyy = 4-digit year | MM = 2-digit month (01-12) | dd = 2-digit day | MMM = 3-letter month (Jan/Feb/...) 10. Example mappings: 11. unique_value 'MM-dd-yyyy' β†’ format: 12-25-2024 12. unique_value 'dd-MM-yyyy' β†’ format: 25-12-2024 13. unique_value 'yyyy-MM-dd' β†’ format: 2024-12-25 14. unique_value 'dd-MMM-yyyy' β†’ format: 25-Dec-2024 15. Always use this format when sending date values in any create/update payload. 16. Always parse user-provided dates and reformat them to this pattern before sending to API.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnly, idempotent, and non-destructive. The description adds substantial behavioral context by detailing the response structure (viewIds, folderPaths, dateTimeFormatID/New), explaining how to extract viewId from latestViews.TC.viewId, and providing an exhaustive date format mapping that is critical for downstream API calls.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections (Toolset, Parameters, Output, Use Cases, Examples, Hints). It is lengthy, especially the date format hints, but that detail is necessary and each section serves a purpose. Slight redundancy in the numbered hints keeps it from a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description takes on the full burden of explaining return values. It thoroughly covers the response fields, provides two examples with expected outputs, and includes critical date format instructions. It is complete for an agent to know when to call and how to interpret results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and includes a description and default for projectKey. The description adds value with examples, clarification that 'default' is used when unspecified, and context that folderPath for root is an empty string (though that's not a direct parameter). It goes beyond the schema but is not exceptionally rich.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it fetches QMetry project information including viewId and folderPath, and explicitly notes this is needed for other operations. This distinguishes it from sibling tools like qmetry_fetch_qmetry_list_projects and qmetry_set_qmetry_project_info.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit use cases and the hint 'Always call this first when user doesn't provide viewId or folderPath', giving clear when-to-use guidance. However, it doesn't explicitly contrast with alternative tools (e.g., list projects, set project info), so it lacks full when-not-to-use exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_quality_gate_configurationQMetry: Fetch Quality Gate ConfigurationA
Read-onlyIdempotent
Inspect

Fetch the quality gate configuration for a project and AI agent, including assessment scope and gate criteria.

Toolset: AI Agent

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • projectId (number) required: Numeric project ID for which to fetch the quality gate configuration. This is the internal numeric identifier, not the project key.

  • agentIdentifier (string) required: Unique identifier of the AI agent whose gate configuration should be retrieved.

Use Cases: 1. Retrieve gate criteria and thresholds before generating a quality gate report 2. Check which quality gates are configured for a project and agent 3. Inspect assessment scope and gate parameters for release readiness evaluation

Examples:

  1. Fetch gate configuration for project 45851 and agent 'RR'

{
  "projectId": 45851,
  "agentIdentifier": "RR"
}

Expected Output: Gate configuration object with assessment scope, gate criteria, and thresholds.

Hints: 1. REQUIRED: 'projectId' must be a valid numeric project ID (not the project key). 2. REQUIRED: 'agentIdentifier' must match a registered AI agent identifier. 3. Call this tool before 'Execute Quality Gate Report' to understand the gate criteria and available gates. 4. If AI Agent is not enabled for the project, the backend will return an appropriate error.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesNumeric project ID for which to fetch the quality gate configuration. This is the internal numeric identifier, not the project key.
projectKeyNoProject key - unique identifier for the projectdefault
agentIdentifierYesUnique identifier of the AI agent whose gate configuration should be retrieved.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds meaningful behavioral context beyond annotations: it specifies the expected output (assessment scope, gate criteria, thresholds) and discloses that the backend will return an error if AI Agent is not enabled for the project.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections (toolset, parameters, use cases, examples, hints) and the core purpose is front-loaded. It is somewhat long and repeats parameter information already present in the schema, but each section earns its place by providing actionable guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema and the tool has three parameters, the description is thorough: it explains the purpose, details each parameter's role, provides a concrete example, lists use cases, and includes error-behavior hints. An agent has everything it needs to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds emphasis on the distinction between projectId and projectKey ('internal numeric identifier, not the project key'), and includes an example with real values, but these largely restate or reinforce what the schema already documents rather than adding new semantic meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Fetch') and resource ('quality gate configuration') and clearly scopes it to a project and AI agent, including what the configuration contains (assessment scope and gate criteria). It also distinguishes itself from the sibling 'Execute Quality Gate Report' by explicitly positioning this as the pre-step to that tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear when-to-use guidance: 'Call this tool before Execute Quality Gate Report' and enumerates three concrete use cases. It does not explicitly state when not to use it or name other alternatives, but the timing against a specific sibling is a strong contextual cue.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_releases_and_cyclesQMetry: Fetch Releases and CyclesA
Read-onlyIdempotent
Inspect

Fetch QMetry releases and cycles from the current project

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • showArchive (boolean): Whether to include archived records in the results. When true, returns both active and archived items. When false, returns only active (non-archived) items. Applies to any entity type being fetched (test cases, requirements, releases, cycles, builds, platforms, etc.).

Output Description: JSON object with project hierarchy containing releases and their associated cycles

Use Cases: 1. Fetch associated releases and cycles of current project 2. Fetch available releases and cycles of current project 3. Get release and cycle information for test planning 4. List all releases and cycles in a project 5. Search for specific releases using release name or ID 6. Fetch cycle lists based on release ID 7. Search for specific cycles using cycle name or ID 8. Get project structure for test planning and execution 9. Retrieve release hierarchy for reporting purposes

Examples:

  1. Get active releases and cycles (default behavior)

{}

Expected Output: List of active releases and cycles excluding archived ones (showArchive: false sent in payload)

  1. Get active/unarchived releases and cycles explicitly

{
  "showArchive": false
}

Expected Output: List of active releases and cycles excluding archived ones (showArchive: false sent in payload)

  1. Get not active/archived releases and cycles

{
  "showArchive": true
}

Expected Output: List of all releases and cycles including archived ones (showArchive: true sent in payload)

Hints: 1. Use 'default' project key when user doesn't specify one 2. PAYLOAD SCENARIOS: 3. - No showArchive parameter β†’ payload: {showArchive: false} β†’ Returns only active releases/cycles 4. - showArchive: false β†’ payload: {showArchive: false} β†’ Returns only active/non-archived releases/cycles 5. - showArchive: true β†’ payload: {showArchive: true} β†’ Returns all releases/cycles including archived ones 6. Default behavior always excludes archived items unless explicitly requested 7. Releases contain cycles - use this hierarchy for test execution planning 8. Each release can have multiple cycles representing different testing phases

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyNoProject key - unique identifier for the projectdefault
showArchiveNoWhether to include archived records in the results. When true, returns both active and archived items. When false, returns only active (non-archived) items. Applies to any entity type being fetched (test cases, requirements, releases, cycles, builds, platforms, etc.).

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already mark it read-only/idempotent/non-destructive, and the description adds concrete archive-filtering behavior: 'Default behavior always excludes archived items unless explicitly requested' and the three payload examples. It also discloses the output as a JSON project hierarchy and clarifies the default for showArchive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The content is well-structured with headings and bullets, but it is redundant: Examples 1 and 2 describe the same false/default behavior, and Hints 3–6 restate the same payload scenarios. The odd numbering in Hints ('2. PAYLOAD SCENARIOS:' followed by 3-6) further hurts clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter, read-only fetch with no output schema, the description is comprehensive: it defines parameters, provides three examples, and explains the release-cycle hierarchy and default behavior. It does not discuss pagination or error cases, but these are less critical for a simple read tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema covers both parameters fully, so baseline is 3; the description adds value by explaining that omitting showArchive results in showArchive:false, and that projectKey defaults to 'default'. However, most parameter semantics already exist in the schema, so the bonus is modest.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Fetch QMetry releases and cycles from the current project', giving a clear verb and resource. It is unambiguous versus sibling fetch tools because it names releases and cycles, though it does not explicitly contrast itself with related fetch tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use Cases list several reasons ('test planning', 'reporting', 'search for specific releases using release name or ID') and Hints explain the release-cycle hierarchy for planning. It gives clear operational context but never states when not to use it or recommends an alternative sibling tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_requirement_detailsQMetry: Fetch Requirement DetailsA
Read-onlyIdempotent
Inspect

Get detailed information for a specific QMetry requirement by numeric ID

Toolset: Requirements

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • id (number) required: Requirement numeric ID (required for fetching specific requirement details). This is the internal numeric identifier, not the entity key like 'MAC-RQ-730'. You can get this ID from requirement search results or by using filters.

  • version (number) required: Requirement version number (required for fetching specific requirement version details). This is the internal numeric identifier for the version.

Output Description: JSON object with requirement details including ID, key, summary, description, status, and all metadata

Use Cases: 1. Get requirement details by numeric ID 2. Retrieve requirement metadata for reporting 3. Get requirement summary and properties 4. Fetch requirement details before linking or updating 5. Access requirement field values and custom fields 6. Get requirement version-specific information

Examples:

  1. Get requirement details by numeric ID

{
  "id": 4791316,
  "version": 1
}

Expected Output: Detailed requirement information including summary, description, status, and all fields

Hints: 1. This API requires a numeric ID parameter, not entity key 2. If user provides entityKey (e.g., MAC-RQ-730), first call FETCH_REQUIREMENTS with a filter on entityKeyId to resolve the numeric ID 3. After resolving entityKey β†’ numeric ID, call this tool with the resolved numeric ID 4. Version parameter is required - use 1 for the latest version unless user specifies otherwise 5. This tool provides complete requirement information including all custom fields 6. Use this tool to get detailed requirement information that's not available in the list view

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesRequirement numeric ID (required for fetching specific requirement details). This is the internal numeric identifier, not the entity key like 'MAC-RQ-730'. You can get this ID from requirement search results or by using filters.
versionYesRequirement version number (required for fetching specific requirement version details). This is the internal numeric identifier for the version.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is established. The description adds meaningful behavioral context beyond annotations: it requires the internal numeric ID rather than the entity key, explains that version is required with 1 meaning latest, and states that the response includes all custom fields and metadata. There is no contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with bolded sections and front-loads the core purpose, but it is redundant: parameter text duplicates the schema, Output Description and Expected Output repeat each other, and several Use Cases restate the same point. The useful hints are buried after a lot of repetitive material.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Even without an output schema, the description provides the complete calling picture: required parameters, how to resolve entityKey to numeric ID, version defaulting, expected output fields, and a concrete example. For a read-only fetch of a single requirement by ID/version, nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description adds value by giving concrete guidance: version defaults to 1 for latest, entityKey must first be resolved via FETCH_REQUIREMENTS, and an example payload shows realistic values. It repeats the schema's parameter text, but the hints and example make the numeric-ID/version semantics more actionable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get detailed information for a specific QMetry requirement by numeric ID.' It further differentiates itself from the list-oriented sibling by stating this tool returns details 'not available in the list view' and by explicitly warning that the numeric ID is not the entity key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Hints section explicitly tells the agent when to use this tool versus an alternative: if the user provides an entityKey, first call FETCH_REQUIREMENTS to resolve it, then call this tool. It also gives the default version behavior ('use 1 for the latest version unless user specifies otherwise') and identifies this as the tool for complete requirement information beyond the list view.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_requirementsQMetry: Fetch RequirementsA
Read-onlyIdempotent
Inspect

Fetch QMetry requirements - automatically handles viewId resolution based on project

Toolset: Requirements

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • viewId (number): ViewId for requirements - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.RQ.viewId automatically.

  • folderPath (string): Folder path for requirements - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression". (default: "")

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • scope (string): Scope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon. (default: "project")

  • getSubEntities (boolean): Whether to include sub-entities.

  • hideEmptyFolders (boolean): Whether to hide empty folders.

  • folderSortColumn (string): Folder sort column (default 'name')

  • folderSortOrder (string): Folder sort order (ASC or DESC)

  • isJiraFilter (boolean): 'false' if using qmetry filter (default: false)

  • filterType (enum): Pass 'QMETRY' or 'JIRA' (default: "QMETRY")

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

  • udfFilter (string): User-defined field filter as JSON string (default '[]') (default: "[]")

  • sort (string): Sort Records - refer json schema, Possible property - name, entityKey, associatedVersion, priorityAlias, createdDate, createdByAlias, updatedDate, updatedByAlias, requirementStateAlias, linkedTcCount, linkedDfCount, attachmentCount, createdSystem, owner (default: "[{"property":"name","direction":"ASC"}]")

Output Description: JSON object with 'data' array containing requirements and pagination info

Use Cases: 1. List all requirements in a project 2. Search for specific requirements using filters 3. Browse requirements in specific folders 4. Get paginated requirement results 5. Filter requirements by name or properties 6. Get requirement metadata for test planning

Examples:

  1. Get all requirements from default project - system will auto-fetch viewId

{}

Expected Output: List of requirements from default project with auto-resolved viewId

  1. Get all requirements from UT project - system will auto-fetch UT project's viewId

{
  "projectKey": "UT"
}

Expected Output: List of requirements from UT project using UT's specific RQ viewId

  1. Search for specific requirements by entity key

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-RQ-123\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Filtered requirements matching the entity key criteria

  1. Search for multiple requirements by comma-separated entity keys

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-RQ-123,MAC-RQ-456,MAC-RQ-789\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Requirements matching any of the specified entity keys

  1. Filter requirements by state (e.g., Open, Approved)

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"Open\",\"field\":\"requirementStateAlias\"}]"
}

Expected Output: Requirements with 'Open' state

  1. Filter requirements by priority

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"High\",\"field\":\"priorityAlias\"}]"
}

Expected Output: Requirements with 'High' priority

  1. Filter requirements by archive status

{
  "filter": "[{\"value\":[1,0],\"type\":\"list\",\"field\":\"isArchived\"}]"
}

Expected Output: List of requirements filtered by archive status (archived and non-archived)

  1. Get only archived requirements

{
  "filter": "[{\"value\":[1],\"type\":\"list\",\"field\":\"isArchived\"}]"
}

Expected Output: List of only archived requirements

  1. Sort requirements by name in ascending order

{
  "projectKey": "MAC",
  "sort": "[{\"property\":\"name\",\"direction\":\"ASC\"}]"
}

Expected Output: Requirements sorted alphabetically by name

  1. Sort requirements by creation date (newest first)

{
  "projectKey": "MAC",
  "sort": "[{\"property\":\"createdDate\",\"direction\":\"DESC\"}]"
}

Expected Output: Requirements sorted by creation date, newest first

  1. Sort requirements by entity key

{
  "projectKey": "MAC",
  "sort": "[{\"property\":\"entityKey\",\"direction\":\"ASC\"}]"
}

Expected Output: Requirements sorted by entity key (MAC-RQ-1, MAC-RQ-2, etc.)

  1. Sort requirements by linked test case count

{
  "projectKey": "MAC",
  "sort": "[{\"property\":\"linkedTcCount\",\"direction\":\"DESC\"}]"
}

Expected Output: Requirements sorted by number of linked test cases, highest first

  1. Complex filter: Requirements by owner with specific state

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"john.doe\",\"field\":\"owner\"},{\"type\":\"string\",\"value\":\"Approved\",\"field\":\"requirementStateAlias\"}]"
}

Expected Output: Requirements owned by john.doe with 'Approved' state

  1. Multi-field sort: Priority first, then creation date

{
  "projectKey": "MAC",
  "sort": "[{\"property\":\"priorityAlias\",\"direction\":\"DESC\"},{\"property\":\"createdDate\",\"direction\":\"ASC\"}]"
}

Expected Output: Requirements sorted by priority (High to Low), then by creation date (oldest first)

  1. Filter requirements by specific release and cycle

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"},{\"value\":[111577],\"type\":\"list\",\"field\":\"cycle\"}]"
}

Expected Output: Requirements associated with Release 8.12 (ID: 55178) and Cycle 8.12.1 (ID: 111577)

  1. Filter requirements by release only

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"}]"
}

Expected Output: All requirements associated with Release 8.12 (ID: 55178)

Hints: 1. CRITICAL WORKFLOW: Always use the SAME projectKey for both project info and requirement fetching 2. Step 1: If user specifies projectKey (like 'UT', 'MAC'), use that EXACT projectKey for project info 3. Step 2: Get project info using that projectKey, extract latestViews.RQ.viewId 4. Step 3: Use the SAME projectKey and the extracted RQ viewId for fetching requirements 5. Step 4: If user doesn't specify projectKey, use 'default' for both project info and requirement fetching 6. NEVER mix project keys - if user says 'MAC project', use projectKey='MAC' for everything 7. For search by requirement key (like MAC-RQ-123), use filter: '[{"type":"string","value":"MAC-RQ-123","field":"entityKeyId"}]' 8. For multiple entity keys, use comma-separated values: '[{"type":"string","value":"MAC-RQ-123,MAC-RQ-456","field":"entityKeyId"}]' 9. Use empty string '' as folderPath for root directory 10. Filter supports QMETRY and JIRA types - default is QMETRY 11. FILTER FIELDS: entityKeyId, name, requirementStateAlias, priorityAlias, owner, createdByAlias, updatedByAlias, createdSystem 12. SORT FIELDS: name, entityKey, associatedVersion, priorityAlias, createdDate, createdByAlias, updatedDate, updatedByAlias, requirementStateAlias, linkedTcCount, linkedDfCount, attachmentCount, createdSystem, owner 13. SORT DIRECTIONS: ASC (ascending), DESC (descending) 14. Multiple filters: Use array with multiple objects for AND conditions 15. Multiple sort criteria: Use array with multiple objects, first takes priority 16. Filter format: [{'type':'string','value':'filterValue','field':'fieldName'}] 17. Sort format: [{'property':'fieldName','direction':'ASC|DESC'}] 18. RELEASE/CYCLE FILTERING: Use release and cycle IDs from fetch_releases_and_cycles tool 19. For specific release: '[{"value":[releaseId],"type":"list","field":"release"}]' 20. For specific cycle: '[{"value":[cycleId],"type":"list","field":"cycle"}]' 21. For release AND cycle: '[{"value":[releaseId],"type":"list","field":"release"},{"value":[cycleId],"type":"list","field":"cycle"}]' 22. Example: Release 8.12 (ID: 55178) + Cycle 8.12.1 (ID: 111577) = filter with both IDs

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
sortNoSort Records - refer json schema, Possible property - name, entityKey, associatedVersion, priorityAlias, createdDate, createdByAlias, updatedDate, updatedByAlias, requirementStateAlias, linkedTcCount, linkedDfCount, attachmentCount, createdSystem, owner[{"property":"name","direction":"ASC"}]
limitNoNumber of records (default 10).
scopeNoScope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon.project
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
viewIdNoViewId for requirements - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.RQ.viewId automatically.
udfFilterNoUser-defined field filter as JSON string (default '[]')[]
filterTypeNoPass 'QMETRY' or 'JIRA'QMETRY
folderPathNoFolder path for requirements - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression".
projectKeyNoProject key - unique identifier for the projectdefault
isJiraFilterNo'false' if using qmetry filter
getSubEntitiesNoWhether to include sub-entities.
folderSortOrderNoFolder sort order (ASC or DESC)
folderSortColumnNoFolder sort column (default 'name')
hideEmptyFoldersNoWhether to hide empty folders.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only/idempotent behavior, and the description adds valuable context beyond them: automatic viewId resolution from project info, automatic root folderPath, default pagination values, and the output shape. These details materially change how an agent should invoke the tool, and there is no contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-headered and front-loaded, but it is extremely long and repetitive: parameter details are largely duplicated from the schema, and the 16 examples overlap significantly with the 22 hints. It is organized enough to be usable, but it is not concise and every sentence does not earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, 16-parameter fetch tool with no output schema, this definition is unusually complete: it specifies the required workflow, defaults, output shape, filter/sort syntax, and how to get release/cycle IDs. An agent has sufficient information to call the tool correctly in nearly all intended scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Even though schema coverage is 100%, the description adds substantial meaning beyond the schema: exact JSON syntax for filter and sort, the list of valid filter/sort fields, and worked examples for each common use case. It also clarifies that viewId and folderPath are optional because the system auto-resolves them.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names the action and resource ('Fetch QMetry requirements') and the output description clarifies it returns a 'data' array of requirements, so an agent can recognize it as a listing/search operation. However, it does not explicitly contrast this with siblings like qmetry_fetch_requirement_details, so it misses the full differentiator credit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases section clearly lists intended scenarios: list all requirements, search with filters, browse folders, paginate, and filter by metadata. Hints also point to fetch_releases_and_cycles for release/cycle IDs. It stops short of explicitly stating when NOT to use this tool versus sibling requirement-fetch tools, so no exclusions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_requirements_linked_to_test_caseQMetry: Fetch Requirements Linked to Test CaseA
Read-onlyIdempotent
Inspect

Get requirements that are linked (or not linked) to a specific test case in QMetry

Toolset: Requirements

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tcID (number) required: Test Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.

  • getLinked (boolean): True to get only requirements that are linked with this test case, false to get requirements which are not linked with this test case. Defaults to true (get linked requirements). (default: true)

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • rqFolderPath (string): Folder path for requirements - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression". (default: "")

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with requirements array, traceability information, and pagination metadata

Use Cases: 1. Get all requirements linked to a specific test case for traceability analysis 2. Find requirements that are NOT linked to a test case (gap analysis) 3. Verify test case coverage by checking linked requirements 4. Impact analysis - see which requirements are affected when a test case changes 5. Generate traceability matrix between test cases and requirements 6. Filter linked requirements by various criteria 7. Audit test case-requirement relationships for compliance 8. Identify orphaned requirements or test cases without proper links 9. Plan requirement validation based on test case-requirement associations 10. Quality assurance - ensure all test cases have proper requirement coverage

Examples:

  1. Get all requirements linked to test case ID 594294

{
  "tcID": 594294
}

Expected Output: List of requirements that are linked to test case MAC-TC-1684

  1. Get requirements NOT linked to test case (gap analysis)

{
  "tcID": 594294,
  "getLinked": false
}

Expected Output: List of requirements that are NOT linked to test case MAC-TC-1684

  1. Get linked requirements from specific folder

{
  "tcID": 594294,
  "rqFolderPath": "/CodeSnippets"
}

Expected Output: Linked requirements located in the '/CodeSnippets' folder

  1. Search linked requirements by entity key

{
  "tcID": 594294,
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-RQ-730,MAC-RQ-731\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Linked requirements matching specific entity keys

  1. Filter linked requirements by status

{
  "tcID": 594294,
  "filter": "[{\"type\":\"list\",\"value\":[1,2],\"field\":\"requirementStateAlias\"}]"
}

Expected Output: Linked requirements with Open or Approved status

  1. Filter linked requirements by priority

{
  "tcID": 594294,
  "filter": "[{\"type\":\"list\",\"value\":[1],\"field\":\"priorityAlias\"}]"
}

Expected Output: Linked requirements with High priority

  1. Filter linked requirements by archive status

{
  "tcID": 594294,
  "filter": "[{\"type\":\"list\",\"value\":[0],\"field\":\"isArchived\"}]"
}

Expected Output: Active (non-archived) linked requirements

  1. Search linked requirements by name content

{
  "tcID": 594294,
  "filter": "[{\"type\":\"string\",\"value\":\"authentication\",\"field\":\"name\"}]"
}

Expected Output: Linked requirements with 'authentication' in their name

  1. Filter linked requirements by test case version

{
  "tcID": 594294,
  "filter": "[{\"type\":\"string\",\"value\":\"1\",\"field\":\"tcVersion\"}]"
}

Expected Output: Requirements linked to version 1 of the test case

  1. Filter linked requirements by release and cycle

{
  "tcID": 594294,
  "filter": "[{\"type\":\"list\",\"value\":[55178],\"field\":\"release\"},{\"type\":\"list\",\"value\":[111577],\"field\":\"cycle\"}]"
}

Expected Output: Linked requirements in Release 8.12 and Cycle 8.12.1

Hints: 1. This API requires a numeric tcID parameter, not entity key 2. If user provides entityKey (e.g., MAC-TC-1684), first call FETCH_TEST_CASES with filter on entityKeyId to resolve the numeric tcID 3. After resolving entityKey β†’ tcID, call this tool with the resolved numeric tcID 4. TRACEABILITY WORKFLOW: Use this tool to establish test case-requirement traceability matrix 5. getLinked=true (default): Returns requirements that ARE linked to the test case 6. getLinked=false: Returns requirements that are NOT linked to the test case (useful for gap analysis) 7. rqFolderPath: Use empty string '' for root folder or specific path like '/CodeSnippets' 8. FILTER CAPABILITIES: Support same filters as regular requirement listing 9. FILTER FIELDS: name, entityKeyId, requirementStateAlias, priorityAlias, createdByAlias, tcVersion, release, cycle, isArchived, componentAlias 10. Multiple filter conditions are combined with AND logic 11. For entity key search, use comma-separated values: 'MAC-RQ-1,MAC-RQ-2,MAC-RQ-3' 12. This tool is crucial for compliance, traceability audits, and impact analysis 13. Pagination supported for large result sets (start, page, limit parameters) 14. Use this tool to verify that test cases properly cover requirements 15. Essential for requirement validation and test case completeness analysis

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
tcIDYesTest Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
getLinkedNoTrue to get only requirements that are linked with this test case, false to get requirements which are not linked with this test case. Defaults to true (get linked requirements).
projectKeyNoProject key - unique identifier for the projectdefault
rqFolderPathNoFolder path for requirements - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression".

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as read-only, idempotent, and non-destructive. The description adds useful behavioral context beyond annotations: getLinked semantics, AND-combined filters, automatic root folder behavior, pagination support, and output structure. No contradiction with the annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is heavily front-loaded with a strong first sentence, but it is severely bloated: the parameter section largely duplicates the schema, the 10 use cases are mostly rhetorical restatements, and the 15 hints overlap with both examples and parameter details. Many sentences do not earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 8 parameters, no output schema, and moderately complex filter semantics, the description is complete enough. It explains the output as a JSON object with requirements array, traceability information, and pagination metadata, and provides concrete examples for filters by entity key, status, priority, folder, release, and cycle.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers all parameters with 100% description coverage, so baseline is 3. The description pushes above baseline by adding real semantic value: the tcID vs entityKey distinction, the resolution workflow through FETCH_TEST_CASES, composable filter JSON examples, supported filter fields, and the meaning of getLinked=true/false in practical terms.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific action and resource: 'Get requirements that are linked (or not linked) to a specific test case in QMetry.' This precisely distinguishes the tool from related siblings like qmetry_fetch_test_cases_linked_to_requirement, while its name and description align cleanly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context and workflows: traceability analysis, gap analysis, coverage verification, and impact analysis. Hint 2 explicitly tells agents to first call FETCH_TEST_CASES when only an entity key is available, but it does not directly contrast this tool with the reciprocal sibling qmetry_fetch_test_cases_linked_to_requirement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_case_detailsQMetry: Fetch Test Case DetailsA
Read-onlyIdempotent
Inspect

Get detailed information for a specific QMetry test case by numeric ID - USE THIS for single test case lookup

Toolset: Test Cases

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tcID (number) required: Test Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

Output Description: JSON object with test case details including ID, key, summary, description, and metadata

Use Cases: 1. Get test case details by numeric ID (PREFERRED for single test case) 2. Fetch test case when user provides entityKey (e.g., 'VKMCP-TC-5') 3. Retrieve test case metadata for a specific test case 4. Get test case summary and properties for display or editing 5. Fetch test case details before accessing steps or version details 6. Lookup test case by name or ID without affecting UI filters

Examples:

  1. Get test case details by numeric ID

{
  "tcID": 4468020
}

Expected Output: Detailed test case information including summary, description, status

Hints: 1. USE THIS TOOL when user asks to 'fetch test case VKMCP-TC-5' or 'get test case by ID' or 'find test case X' 2. This API requires a numeric tcID parameter 3. CRITICAL: If user provides entityKey (e.g., MAC-TC-1684), you have TWO options: 4. Option 1 (RECOMMENDED): Ask user for the numeric test case ID 5. Option 2: If you must resolve entityKey, use FETCH_TEST_CASES with filter ONLY ONCE, then immediately use this tool 6. After resolving entityKey β†’ tcID, always use THIS tool (FETCH_TEST_CASE_DETAILS) for subsequent lookups 7. This tool provides metadata and properties; use FETCH_TEST_CASE_STEPS for step-level details 8. This tool does NOT persist filters in UI - safe for single test case lookups 9. ALWAYS prefer this tool over FETCH_TEST_CASES with filters for single test case operations

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
tcIDYesTest Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool read-only, idempotent, and non-destructive. The description adds useful behavioral context beyond that: it does not persist UI filters, it returns metadata/properties rather than step details, and it requires the internal numeric ID rather than the entityKey. No statement contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The content is well-structured with clear sections, bolded headers, and a front-loaded core purpose, but it is substantially overlong. 'USE THIS for single test case lookup' is repeated in the opening, Use Cases, and Hints, and the entityKey resolution guidance spans multiple redundant numbered hints that could be condensed significantly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only single-record fetch with no output schema, the description provides an output description, a worked example, use cases, and fallback strategies for entityKey inputs. An agent has everything needed to select and invoke the tool correctly, including the preferred tool among siblings and how to handle non-numeric identifiers.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all five parameters, including the tcID naming warning and the entityKey distinction. The description's parameter section mainly repeats the schema text verbatim, and the added hints about entityKey resolution are more usage guidance than new parameter semantics. There is also a minor internal inconsistency where the description says tcID 'accepts a string or number' while the schema declares type 'number'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names the exact operation ('Get detailed information for a specific QMetry test case by numeric ID') and explicitly labels it 'USE THIS for single test case lookup', which distinguishes it from sibling tools like qmetry_fetch_test_cases and qmetry_fetch_test_case_steps. The purpose is concrete and immediately actionable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Hints and Use Cases sections give explicit when-to-use guidance ('when user asks to fetch test case VKMCP-TC-5'), explicit when-not-to-use guidance ('use FETCH_TEST_CASE_STEPS for step-level details'), and a clear alternative path for resolving entityKey via FETCH_TEST_CASES. It also states a preferred option to ask the user for the numeric ID, leaving little to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_case_executionsQMetry: Fetch Test Case ExecutionsA
Read-onlyIdempotent
Inspect

Get execution records for a specific test case by numeric ID, including Test Run UDF values. ALWAYS present results as a unified table: Test Suite Key | Test Suite Name | Release | Cycle | Platform | Executed Version | Execution Status | <UDF Label columns…>. NEVER show a separate type+value UDF breakdown β€” always combine identification fields and UDF values in one table per execution row.

Toolset: Test Cases

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tcid (number) required: Test Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.

  • tcversion (number): Test Case version number (optional, defaults to 1). This is the internal numeric identifier for the version.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • scope (string): Scope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon. (default: "project")

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with executions array. Each execution record ALWAYS contains these mandatory identification fields: 'tsEntityKey' (Test Suite Key, e.g. 'MAC-TS-42'), 'testsuiteName' (Test Suite Name), 'releaseName' (Release), 'cycleName' (Cycle), 'platform' (Platform/environment), 'executedVersion' (Executed Version of the test case), 'executionStatus' (Execution Status label), 'tcRunID' (numeric Test Run ID), and 'testRunUdfs' (array of objects each with name, label, fieldID, fieldType, value β€” use 'label' for display headers, null if not set). ALL project-defined UDF fields are always included, even those with no value. Top-level 'hasTcRunUdf' flag indicates whether the project has Test Run UDFs configured. When false, a 'testRunUdfNote' field provides a professional explanation instead.

Use Cases: 1. Get execution history for a specific test case 2. Retrieve test case execution results for reporting 3. Filter executions by test suite, platform, or execution status 4. Get execution data for test case analysis 5. Monitor test case execution trends over time 6. Filter executions by release, cycle, or execution date 7. Get execution details for specific test case versions 8. Audit test execution history for compliance 9. Analyze test case execution performance across different environments 10. Track test execution by specific users or teams 11. Fetch Test Run UDF values for a specific test case's execution records 12. Inspect custom metadata captured during test execution via Test Run UDFs 13. Check whether the project has Test Run UDFs configured (hasTcRunUdf flag)

Examples:

  1. Get all executions for test case ID 1223922

{
  "tcid": 1223922
}

Expected Output: Present as ONE unified table β€” never as a separate type+value UDF breakdown. Example: | Test Suite Key | Test Suite Name | Release | Cycle | Platform | Executed Version | Execution Status | Tested By | Environments UDF | Execution Type | | MAC-TS-42 | Regression Suite | R1 | Sprint1| Chrome | v1 | Passed | varis | chrome, edge, safari | Functional | | MAC-TS-42 | Regression Suite | R1 | Sprint1| Firefox | v2 | Failed | john | firefox | Regression | Columns in order: Test Suite Key (tsEntityKey) | Test Suite Name (testsuiteName) | Release (releaseName) | Cycle (cycleName) | Platform (platform) | Executed Version (executedVersion) | Execution Status | then one column per UDF label. Use the UDF 'label' as column header. Show null UDF values as '-'.

  1. Get executions for specific test case version

{
  "tcid": 1223922,
  "tcversion": 2
}

Expected Output: Execution records for version 2 of the test case

  1. Filter executions by test suite and platform

{
  "tcid": 1223922,
  "filter": "[{\"value\":\"Sample Test Suite\",\"type\":\"string\",\"field\":\"testSuiteName\"},{\"value\":[12345],\"type\":\"list\",\"field\":\"platformID\"}]"
}

Expected Output: Filtered execution records matching test suite and platform criteria

  1. Filter executions by execution status

{
  "tcid": 1223922,
  "filter": "[{\"value\":[\"PASS\"],\"type\":\"list\",\"field\":\"executionStatus\"}]"
}

Expected Output: Execution records with PASS status only

  1. Filter executions by release and cycle

{
  "tcid": 1223922,
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"},{\"value\":[111577],\"type\":\"list\",\"field\":\"cycle\"}]"
}

Expected Output: Execution records filtered by specific release and cycle

  1. Filter executions by date range

{
  "tcid": 1223922,
  "filter": "[{\"value\":\"2024-01-01\",\"type\":\"date\",\"field\":\"executedDate\",\"comparison\":\"gt\"},{\"value\":\"2024-12-31\",\"type\":\"date\",\"field\":\"executedDate\",\"comparison\":\"lt\"}]"
}

Expected Output: Execution records within the specified date range

  1. Filter executions by user

{
  "tcid": 1223922,
  "filter": "[{\"value\":[\"john.doe\"],\"type\":\"list\",\"field\":\"executedBy\"}]"
}

Expected Output: Execution records executed by specific user

  1. Fetch Test Run UDF values for all executions of test case ID 41571999

{
  "tcid": 41571999
}

Expected Output: Present as ONE unified table combining identification fields and UDF values β€” never a separate type+value breakdown. Example: | Test Suite Key | Test Suite Name | Release | Cycle | Platform | Executed Version | Execution Status | Tested By | Environments UDF | Execution Type | Country | | MAC-TS-42 | Login Suite | R1 | Sprint1 | Chrome | v1 | Passed | varis | chrome, edge, safari | Functional | India > i3 | UDF column headers use the UDF 'label' (not raw field key). Null values shown as '-'.

  1. Check if project has Test Run UDFs β€” response includes hasTcRunUdf flag

{
  "tcid": 1223922
}

Expected Output: Response contains hasTcRunUdf: true (UDFs present, testRunUdfs populated) or hasTcRunUdf: false (no UDFs configured, testRunUdfNote explains this)

Hints: 1. === MANDATORY RESPONSE FORMAT β€” READ THIS BEFORE RENDERING ANY OUTPUT === 2. 3. PIVOT RULE β€” CRITICAL: 4. The 'testRunUdfs' field on each execution is an array of { name, label, fieldID, fieldType, value }. 5. You MUST pivot this array into TABLE COLUMNS β€” do NOT render it as rows. 6. β†’ Each testRunUdfs[i].label = a column header in the unified table 7. β†’ Each testRunUdfs[i].value = the cell value for that execution's row 8. β†’ testRunUdfs[i].fieldType = INTERNAL METADATA β€” NEVER show this as a column 9. β†’ testRunUdfs[i].fieldID = INTERNAL METADATA β€” NEVER show this as a column 10. 11. FORBIDDEN PATTERNS β€” NEVER do any of these: 12. ❌ Do NOT render a separate sub-table (UDF Label | Type | Value) per execution 13. ❌ Do NOT show 'Type' or 'fieldType' as a visible column 14. ❌ Do NOT group output by tcRunID with individual breakdowns beneath each 15. ❌ Do NOT show raw UDF field keys (e.g. 'TRString', '8260LUP') as headers β€” use 'label' 16. 17. REQUIRED OUTPUT β€” ONE unified table, all executions as rows: 18. | Test Suite Key | Test Suite Name | Release | Cycle | Platform | Executed Version | Execution Status | <UDF Label 1> | <UDF Label 2> | ... | 19. |----------------|-----------------|---------|-------|----------|------------------|------------------|---------------|---------------|-----| 20. | MAC-TS-42 | Login Suite | R1 | S1 | Chrome | v1 | Passed | varis | chrome, edge | ... | 21. 22. MANDATORY COLUMNS (always first, in this order): 23. 1. Test Suite Key β†’ tsEntityKey (e.g. 'MAC-TS-42') 24. 2. Test Suite Name β†’ testsuiteName (test suite display name) 25. 3. Release β†’ releaseName 26. 4. Cycle β†’ cycleName 27. 5. Platform β†’ platform 28. 6. Executed Version β†’ executedVersion 29. 7. Execution Status β†’ executionStatus 30. 8. Tested By β†’ testedBy/executedBy when present 31. 9+. One column per UDF field β€” use testRunUdfs[i].label as header, testRunUdfs[i].value as cell. 32. 33. Null UDF values β†’ show as '-'. If hasTcRunUdf is false, show columns 1-8 only. 34. UDF DATA SOURCE β€” THIS TOOL IS SELF-CONTAINED: 35. This tool automatically calls Test Run UDF metadata once (project-wide) and parses the udfjson field from each execution row. 36. The 'testRunUdfs' array in every execution record already contains ALL configured UDF fields β€” including fields with no value (null). 37. DO NOT call 'Fetch Test Run UDF Values' after this tool for test case executions β€” that tool uses GET_TESTCASE_RUNS_BY_TESTSUITE_RUN which is for test suite runs, not test case executions. 38. NEVER chain 'Fetch Test Run UDF Values' when the user asks for UDF values of test case executions β€” use testRunUdfs from THIS response directly. 39. === END MANDATORY RESPONSE FORMAT === 40. 41. This API requires a numeric tcid parameter, not entity key 42. If user provides entityKey (e.g., MAC-TC-1684), first call FETCH_TEST_CASES with filter on entityKeyId to resolve the tcid 43. After resolving entityKey β†’ tcid, call this tool with the resolved numeric tcid 44. tcversion parameter is optional - omit to get executions for all versions 45. 46. CRITICAL WORKFLOW FOR LINKED ISSUES: When user asks 'fetch linked issues of test case [ID]' or 'linked issues of execution': 47. YOU MUST FIRST get the execution data using this tool to extract tcRunID before fetching issues! 48. 49. COMPLETE WORKFLOW FOR TEST CASE β†’ LINKED ISSUES: 50. STEP 1: Resolve Test Case ID (if needed) - Use FETCH_TEST_CASES if user provides entity key 51. STEP 2: Fetch Test Case Executions (THIS TOOL) - Input: tcid, Extract: data[].tcRunID values 52. STEP 3: Fetch Linked Issues - Tool: FETCH_LINKED_ISSUES_BY_TESTCASE_RUN, Input: entityId = tcRunID 53. 54. ID MAPPING CRITICAL UNDERSTANDING: 55. - tcid/tcID = Test Case ID (for getting execution data with this tool) 56. - tcRunID = Test Case Run/Execution ID (THIS is entityId for linked issues API) 57. - entityId = tcRunID (what the linked issues API actually needs) 58. 59. NEVER USE tcid DIRECTLY as entityId for linked issues! 60. ALWAYS get tcRunID from executions and use THAT as entityId! 61. 62. EXAMPLE RESPONSE STRUCTURE FROM THIS TOOL: 63. { "data": [{ "tcRunID": 58312120, "testSuiteName": "Suite 1", "executionStatus": "PASS", "testRunUdfs": [...] }] } 64. β†’ Use tcRunID (58312120) as entityId for linked issues API 65. 66. FILTER CAPABILITIES: Support extensive filtering by test suite, platform, status, user, release, cycle, dates, and archive status 67. FILTER FIELDS: testSuiteName (string), platformID (list), executionStatus (list), executedBy (list), project (list), release (list), cycle (list), executedDate (date with comparison), isPlatformArchived (list), isTestSuiteArchived (list), executedVersion (numeric) 68. DATE FILTERING: Use 'gt' (greater than) and 'lt' (less than) comparisons for executedDate field 69. EXECUTION STATUS: Common values include 'PASS', 'FAIL', 'BLOCKED', 'NOT_EXECUTED', 'WIP' (verify with your QMetry instance) 70. PLATFORM/SUITE ARCHIVE: Use [1,0] for both archived and non-archived, [1] for archived only, [0] for active only 71. Multiple filter conditions are combined with AND logic 72. Use pagination for large execution result sets (start, page, limit parameters) 73. Get platform IDs from FETCH_PLATFORMS tool and release/cycle IDs from FETCH_RELEASES_AND_CYCLES tool 74. This tool is essential for test execution reporting, trend analysis, and compliance auditing 75. Execution data includes timestamps, user information, environment details, and test results 76. Use scope parameter to define retrieval context (project, folder, release, cycle) 77. 78. TEST RUN UDF SUPPORT: 79. This tool automatically fetches UDF metadata (project-wide, one call for all executions) and enriches each execution record. 80. ALL project-defined Test Run UDF fields are returned for every execution β€” including fields with no value (value: null). 81. HTML is stripped from rich text (LARGETEXT) UDF field values for clean output. 82. Each execution's 'testRunUdfs' is an array of objects: 83. testRunUdfs: [ 84. { "name": "8260LUP", "label": "Lookup Field", "fieldID": 228563, "fieldType": "LOOKUPLIST", "value": "l1" }, 85. { "name": "TRString", "label": "TR String", "fieldID": 229241, "fieldType": "STRING", "value": "dsf" }, 86. { "name": "notes_run", "label": "Notes Run", "fieldID": 229242, "fieldType": "LARGETEXT", "value": null }, 87. { "name": "cascade_vK", "label": "Cascade VK", "fieldID": 229426, "fieldType": "CASCADINGLIST", "value": { "child": "qq", "parent": "vkc" } } 88. ] 89. Use 'fieldID' from testRunUdfs entries when calling 'Bulk Update Test Run UDFs'. 90. 91. hasTcRunUdf FLAG β€” IMPORTANT: 92. The response contains a 'hasTcRunUdf' boolean flag at the top level. 93. hasTcRunUdf: true β†’ Project has Test Run UDFs configured; each execution record includes 'testRunUdfs' array with all fields. 94. hasTcRunUdf: false β†’ Project has NO Test Run UDFs configured. 95. When hasTcRunUdf is false, the response includes a 'testRunUdfNote' field with a professional explanation. 96. Inform the user: 'No Test Run UDFs are configured for this project. Contact a project administrator to set up Test Run UDF fields.' 97. NEVER attempt to read testRunUdfs from records when hasTcRunUdf is false β€” the field will not be present.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
tcidYesTest Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.
limitNoNumber of records (default 10).
scopeNoScope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon.project
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
tcversionNoTest Case version number (optional, defaults to 1). This is the internal numeric identifier for the version.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the operation read-only and idempotent, and the description adds substantial behavioral detail: the unified-table rendering rule, the hasTcRunUdf flag behavior, null UDF handling, HTML stripping, automatic UDF metadata enrichment, and the tcRunID-needed-for-linked-issues rule. It also warns about forbidden output patterns, making side effects/response behavior highly predictable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely long and repetitive: the UDF pivot rule, mandatory columns, and hasTcRunUdf behavior are restated in the main description, Output Description, Use Cases, Examples, Hints, and final TEST RUN UDF SUPPORT section. This redundancy makes parsing harder despite the first sentence being clear and the content being front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates with a detailed output contract: required fields, UDF array shape, top-level flags, example tables, filter fields, pagination, and the linked-issues workflow. For a complex fetch tool with 8 parameters and a rich response, nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters are already documented; the description goes beyond by providing filter field names, JSON filter examples, date comparison syntax, status values, archive-value semantics, and entity-key-to-tcid resolution guidance. A small deduction because the Parameter section largely repeats the schema descriptions rather than condensing them.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific action and resource: 'Get execution records for a specific test case by numeric ID, including Test Run UDF values.' It also distinguishes itself by explicitly calling out what it is not (e.g., 'Fetch Test Run UDF Values' for test suite runs) and has unique sibling names like fetch_executions_by_test_suite that an agent can contrast against. The verb is concrete and the resource is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description is explicit about when to use the tool, when not to use related tools, and how to chain it: it mandates resolving entity key to tcid first, forbids chaining 'Fetch Test Run UDF Values', and gives a complete three-step workflow for linked issues requiring tcRunID. It also lists 13 use cases and details filter semantics, leaving no ambiguity about invocation context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_case_runs_by_test_suite_runQMetry: Fetch Test Case Runs by Test Suite RunA
Read-onlyIdempotent
Inspect

Get test case runs under a specific test suite run execution in QMetry, including Test Run UDF values. ALWAYS present results as a unified table: Test Case Key | Test Case Summary | Executed Version | Execution Status | <UDF Label columns…>. NEVER show a separate type+value breakdown β€” always combine identification fields and UDF values in one table per run.

Toolset: Test Suites

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tsrunID (string) required: Test Suite Run ID. CRITICAL: the parameter name is 'tsrunID' β€” do NOT use 'testSuiteRunId', 'tsRunID', 'testSuiteRunID', or any other variant. Accepts a string or number (e.g. 731600 or '731600' β€” both are valid). To get this value: Call 'Fetch Executions by Test Suite' β†’ use data[].tsRunID from the response.

  • viewId (number): ViewId for test execution - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TE.viewId automatically.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

  • udfFilter (string): User-defined field filter as JSON string (default '[]') (default: "[]")

  • tcrUdfFilter (string): Test Case Run (TCR) UDF filter as JSON string (default '[]'). Used to filter test case runs by Test Run UDF field values. Format: '[{"type":"list","value":[,],"field":""}]'. Example: '[{"type":"list","value":[5108701,5108697],"field":"8260LUP"}]' filters runs where the UDF field '8260LUP' has those list item IDs. (default: "[]")

  • showTcWithDefects (boolean): Show test case runs with linked defects (default: false)

Output Description: JSON object with test case runs array. Each row ALWAYS contains these mandatory identification fields: 'entityKey' (Test Case Key, e.g. 'MAC-TC-123'), 'summary' (Test Case Summary/name), 'latestVersion' (Executed Version, e.g. 'v1', 'v2'), 'runStatus' (Execution Status label, e.g. 'Passed', 'Failed', 'Not Run'), 'runStatusID' (numeric status ID), 'tcRunID' (numeric Test Run ID), and 'testRunUdfs' (object with Test Run UDF field keys mapped to their values, parsed from the raw 'udfjson' field; HTML stripped from rich text). UDF values can also be fetched in enriched form via FETCH_TEST_RUN_UDF_VALUES or field metadata via FETCH_TEST_RUN_UDF_METADATA. The top-level response includes 'hasTcRunUdf' (boolean), 'total' (count), and pagination metadata.

Use Cases: 1. Get all test case runs under a specific test suite run execution 2. Fetch Test Run UDF values for all test case executions in a test suite run β€” always show Test Case Key, Summary, and Execution Status alongside UDFs 3. Fetch Test Run UDF values for a specific test case execution by tcRunID 4. Filter test case runs by Test Run UDF field values (e.g. show only runs where dropdown UDF = specific option) 5. Analyze individual test case execution results, status, and custom UDF metadata 6. Monitor test case run performance and execution trends 7. Generate detailed test execution reports including custom UDF data 8. Track test case run history and execution patterns 9. Validate test case run coverage and execution completeness 10. Audit test case run data with UDF values for compliance and quality assurance 11. Export detailed test case run data including Test Run UDFs for external reporting 12. Retrieve paginated test case run results for large test suite executions

Examples:

  1. Get all test case runs for test suite run ID '107021'

{
  "tsrunID": "107021",
  "viewId": 6887
}

Expected Output: Present as ONE unified table β€” never as a separate type+value breakdown. Example: | Test Case Key | Test Case Summary | Executed Version | Execution Status | Tested By | Environments UDF | Execution Type | Country | | MAC-TC-5 | Login - valid credential | v1 | Passed | varis | chrome, edge, safari | Functional | India > i3 | | MAC-TC-6 | Login - invalid password | v2 | Failed | john | firefox | Regression | - | Columns: Test Case Key (entityKey) | Test Case Summary (summary) | Executed Version (latestVersion) | Execution Status (runStatus) | then one column per UDF label. Use the UDF 'label' as column header. Show null UDF values as '-'.

  1. Fetch Test Run UDF values for all executions in test suite run '728995'

{
  "tsrunID": "728995",
  "viewId": 79451,
  "start": 0,
  "page": 1,
  "limit": 20
}

Expected Output: Present as ONE unified table combining identification fields and UDF values β€” never a separate type+value breakdown. Example: | Test Case Key | Test Case Summary | Executed Version | Execution Status | Tested By | Environments UDF | Planned Execution Date | Execution Type | | MAC-TC-5 | Login test | v1 | Passed | varis | chrome, edge, safari | - | Functional | HTML is stripped from LARGETEXT UDF fields. Null values shown as '-'.

  1. Filter by Test Run UDF list/dropdown field (single-select or multi-select lookup) β€” runs where UDF '8260LUP' has list item IDs 5108701 or 5108697

{
  "tsrunID": "728995",
  "viewId": 79451,
  "tcrUdfFilter": "[{\"type\":\"list\",\"value\":[5108701,5108697],\"field\":\"8260LUP\"}]"
}

Expected Output: Filtered test case runs matching the lookup list UDF values

  1. Filter by Test Run UDF multi-lookup list field β€” runs where multi-select UDF 'PGTE_MULTILOOKUPLIST' contains list item IDs 5126503 or 5126502

{
  "tsrunID": "728995",
  "viewId": 79451,
  "tcrUdfFilter": "[{\"type\":\"list\",\"value\":[5126503,5126502],\"field\":\"PGTE_MULTILOOKUPLIST\"}]"
}

Expected Output: Filtered test case runs where multi-lookup UDF matches any of the given list item IDs

  1. Filter by Test Run UDF cascading dropdown field β€” runs where cascading UDF 'cascade_vK' has list item IDs 5126498 or 5126499 (must set isCascading:true)

{
  "tsrunID": "728995",
  "viewId": 79451,
  "tcrUdfFilter": "[{\"type\":\"list\",\"value\":[5126498,5126499],\"field\":\"cascade_vK\",\"isCascading\":true}]"
}

Expected Output: Filtered test case runs matching the cascading dropdown UDF values

  1. Filter by Test Run UDF short text field β€” runs where text UDF 'TRString' contains the value 'str'

{
  "tsrunID": "728995",
  "viewId": 79451,
  "tcrUdfFilter": "[{\"type\":\"string\",\"field\":\"TRString\",\"value\":\"str\"}]"
}

Expected Output: Filtered test case runs where short text UDF matches the search string

  1. Filter by Test Run UDF large text field β€” runs where large text UDF 'vk_large_text' contains 'this is large text'

{
  "tsrunID": "728995",
  "viewId": 79451,
  "tcrUdfFilter": "[{\"type\":\"string\",\"field\":\"vk_large_text\",\"value\":\"this is large text\"}]"
}

Expected Output: Filtered test case runs where large text UDF matches the search string

  1. Filter by Test Run UDF date field β€” runs where date UDF 'PGTE_DATEPICKER' is after a specific date (comparison: gt) and before another date (comparison: lt)

{
  "tsrunID": "728995",
  "viewId": 79451,
  "tcrUdfFilter": "[{\"type\":\"date\",\"field\":\"PGTE_DATEPICKER\",\"comparison\":\"gt\",\"value\":\"06-01-2026\"},{\"type\":\"date\",\"field\":\"PGTE_DATEPICKER\",\"comparison\":\"lt\",\"value\":\"06-30-2026\"}]"
}

Expected Output: Filtered test case runs where date UDF falls within the specified range

  1. Filter by Test Run UDF numeric field β€” runs where numeric UDF 'NB_number_TR' equals 2

{
  "tsrunID": "728995",
  "viewId": 79451,
  "tcrUdfFilter": "[{\"type\":\"numeric\",\"value\":2,\"field\":\"NB_number_TR\"}]"
}

Expected Output: Filtered test case runs where numeric UDF equals the specified value

  1. Combine multiple Test Run UDF filters β€” filter by list UDF AND string UDF AND numeric UDF simultaneously

{
  "tsrunID": "728995",
  "viewId": 79451,
  "start": 0,
  "page": 1,
  "limit": 20,
  "tcrUdfFilter": "[{\"type\":\"list\",\"value\":[5108701,5108697],\"field\":\"8260LUP\"},{\"type\":\"string\",\"field\":\"TRString\",\"value\":\"str\"},{\"type\":\"numeric\",\"value\":2,\"field\":\"NB_number_TR\"}]"
}

Expected Output: Test case runs matching ALL specified UDF filter conditions (AND logic)

  1. Full filter combination β€” standard field filter + Test Run UDF filter + show only runs with defects

{
  "tsrunID": "728995",
  "viewId": 79451,
  "start": 0,
  "page": 1,
  "limit": 20,
  "filter": "[]",
  "udfFilter": "[]",
  "tcrUdfFilter": "[{\"type\":\"list\",\"value\":[5108701,5108697],\"field\":\"8260LUP\"},{\"type\":\"string\",\"field\":\"TRString\",\"value\":\"str\"},{\"type\":\"string\",\"field\":\"vk_large_text\",\"value\":\"this is large text\"},{\"type\":\"date\",\"field\":\"PGTE_DATEPICKER\",\"comparison\":\"gt\",\"value\":\"\"},{\"type\":\"date\",\"field\":\"PGTE_DATEPICKER\",\"comparison\":\"lt\",\"value\":\"\"},{\"type\":\"list\",\"value\":[5126503,5126502],\"field\":\"PGTE_MULTILOOKUPLIST\"},{\"type\":\"list\",\"value\":[5126498,5126499],\"field\":\"cascade_vK\",\"isCascading\":true},{\"type\":\"numeric\",\"value\":2,\"field\":\"NB_number_TR\"}]",
  "showTcWithDefects": false
}

Expected Output: Test case runs filtered by all 7 UDF field type conditions simultaneously

  1. Get paginated test case runs with standard runStatus filter

{
  "tsrunID": "2362144",
  "viewId": 104123,
  "start": 0,
  "page": 1,
  "limit": 25,
  "filter": "[{\"type\":\"string\",\"value\":\"Failed\",\"field\":\"runStatus\"}]"
}

Expected Output: Paginated list of Failed test case runs with 25 items per page including testRunUdfs

  1. Show only test case runs with linked defects, filtered by Test Run UDF list value

{
  "tsrunID": "107021",
  "viewId": 6887,
  "showTcWithDefects": true,
  "tcrUdfFilter": "[{\"type\":\"list\",\"value\":[5108701],\"field\":\"8260LUP\"}]"
}

Expected Output: Test case runs with defects where the specified Test Run UDF matches

Hints: 1. === MANDATORY RESPONSE FORMAT β€” READ THIS BEFORE RENDERING ANY OUTPUT === 2. 3. PIVOT RULE β€” CRITICAL: 4. The 'testRunUdfs' field on each row is a key-value map (object) of UDF field keys to their values. 5. You MUST pivot this into TABLE COLUMNS β€” do NOT render it as rows. 6. β†’ Each key in testRunUdfs = use the UDF human-readable label as the column header 7. β†’ Each value in testRunUdfs = the cell value for that run's row 8. β†’ 'fieldType' / 'fieldID' = INTERNAL METADATA β€” NEVER show these as columns 9. 10. FORBIDDEN PATTERNS β€” NEVER do any of these: 11. ❌ Do NOT render a separate sub-table (Label | Type | Value) per tcRunID 12. ❌ Do NOT show 'Type' or 'fieldType' as a visible column 13. ❌ Do NOT group output by tcRunID with individual breakdowns beneath each 14. ❌ Do NOT show raw UDF field keys (e.g. 'TRString', '8260LUP') as column headers β€” use human-readable labels 15. 16. REQUIRED OUTPUT β€” ONE unified table, all runs as rows: 17. | Test Case Key | Test Case Summary | Executed Version | Execution Status | <UDF Label 1> | <UDF Label 2> | ... | 18. |---------------|-------------------|------------------|------------------|---------------|---------------|-----| 19. | VKMCP-TC-1 | Login test | v1 | Not Run | varis | chrome, edge | ... | 20. | VKMCP-TC-2 | Invalid password | v2 | Not Run | john | firefox | ... | 21. 22. MANDATORY COLUMNS (always first, in this order): 23. 1. Test Case Key β†’ entityKey (e.g. 'VKMCP-TC-1') 24. 2. Test Case Summary β†’ summary (test case name) 25. 3. Executed Version β†’ latestVersion (e.g. 'v1', 'v2') 26. 4. Execution Status β†’ runStatus (e.g. 'Not Run', 'Passed', 'Failed') 27. 5. Tested By β†’ testedBy/executedBy when present 28. 6+. One column per UDF field β€” use the human-readable label as header, the value as the cell. 29. 30. Null UDF values β†’ show as '-' in the cell. 31. If testRunUdfs is empty or hasTcRunUdf is false, still show columns 1-5. 32. CHAINING WITH FETCH TEST RUN UDF VALUES: 33. If the user explicitly asks for Test Run UDFs with these test case runs, call 'Fetch Test Run UDF Values' after this tool with sourceContext='testSuiteRun' and sourceRows=. Render that tool's unifiedTableRows as the final answer. 34. === END MANDATORY RESPONSE FORMAT === 35. 36. CRITICAL WORKFLOW FOR FETCHING ALL EXECUTIONS OF A TEST SUITE: 37. When user asks to: 38. - 'fetch all executions' 39. - 'get all test runs' 40. - 'fetch all tcRunIDs for test suite X' 41. - 'update status for all executions of test suite X' 42. STEP 1: First call FETCH_EXECUTIONS_BY_TESTSUITE tool with the test suite ID (tsID, not entityKey) 43. - This returns ALL execution records for that test suite (could be 3, 5, 9, or any number) 44. - Extract ALL tsRunID values from the response data array 45. - Example response: data: [{tsRunID: '2739237', ...}, {tsRunID: '2739236', ...}, {tsRunID: '2739235', ...}] 46. STEP 2: For EACH tsRunID from Step 1, call this tool (FETCH_TEST_CASE_RUNS_BY_TESTSUITE_RUN) 47. - This returns all test case runs (tcRunID values) for that specific execution 48. - Repeat for ALL tsRunID values discovered in Step 1 49. STEP 3: Collect all tcRunID values from all executions 50. - Now you have the complete list of test case runs across ALL executions 51. - Use these for bulk status updates or other operations 52. CRITICAL ERROR TO AVOID: 53. - NEVER assume or hard-code only 2-3 execution IDs 54. - NEVER skip Step 1 - always discover ALL executions first 55. - NEVER fetch tcRunIDs for only some executions - get ALL of them 56. - If there are 9 executions, you must fetch tcRunIDs for all 9, not just 2 57. EXAMPLE WORKFLOW: 58. User: 'Fetch all test case runs for test suite VKMCP-TS-21' 59. Step 1: Call FETCH_EXECUTIONS_BY_TESTSUITE with tsID (resolved from VKMCP-TS-21) 60. Result: Found 9 executions with tsRunIDs: 2739237, 2739236, 2739235, 2739234, 2739233, 2739232, 2739231, 2739230, 2739229 61. Step 2: Call this tool 9 times (once for each tsRunID) 62. Call 1: tsrunID='2739237' -> returns 54 tcRunIDs 63. Call 2: tsrunID='2739236' -> returns 54 tcRunIDs 64. ... (repeat for all 9) 65. Step 3: Total collected: 9 executions Γ— 54 test cases = 486 total tcRunIDs 66. 67. PERFORMANCE CONSIDERATIONS FOR LARGE TEST RUNS: 68. When dealing with large numbers of test case runs (500+, 1000+), follow these guidelines: 69. 1. ALWAYS inform the user about the scale BEFORE starting operations: 70. Example: 'Found 9 executions with approximately 486 test case runs. This will require fetching data from all 9 executions and may take a moment.' 71. 2. For bulk status updates on 1000+ test case runs: 72. - NEVER attempt to update all 1000+ in a single operation 73. - Break into smaller batches of 10-20 test case runs per update 74. - Inform user: 'Found 1000 test case runs. Will process in batches of 20 to ensure reliability and performance.' 75. - Show progress: 'Processing batch 1/50 (20 test runs)...', 'Batch 2/50...' 76. 3. Recommended batch sizes: 77. - For status updates: 10-20 test case runs per batch 78. - For fetching data: Can handle larger batches (50-100) 79. - Adjust based on API response times and timeout limits 80. 4. Always provide progress updates for long-running operations: 81. - Before: 'Processing 1000 test runs in 50 batches of 20...' 82. - During: 'Completed 200/1000 test runs (10 batches)...' 83. - After: 'Successfully updated all 1000 test case runs.' 84. 5. Error handling for batch operations: 85. - If a batch fails, report which batch and continue with remaining 86. - Provide summary at the end: 'Completed 48/50 batches. 2 batches failed (batch 23, 45).' 87. - Allow user to retry failed batches specifically 88. EXAMPLE LARGE-SCALE WORKFLOW: 89. User: 'Update status to Failed for all test runs in VKMCP-TS-21' 90. Step 1: Discover all executions (9 found) 91. Step 2: Fetch all tcRunIDs (486 total) 92. Step 3: Inform user: 'Found 486 test case runs across 9 executions. Will update in 25 batches of 20 runs each.' 93. Step 4: Process in batches with progress updates 94. Step 5: Report completion: 'Successfully updated all 486 test case runs to Failed status.' 95. 96. CRITICAL: tsrunID and viewId parameters are REQUIRED 97. tsrunID is a STRING identifier for the test suite run execution 98. viewId is a NUMERIC identifier for the test execution view 99. !MOST IMPORTANT HOW TO GET tsrunID: 100. 1. Call API 'Execution/Fetch Executions' (FETCH_EXECUTIONS_BY_TESTSUITE) to get ALL available executions 101. 2. From the response, get value of following attribute -> data[].tsRunID for EVERY execution 102. 3. Example: Test Suite might have multiple executions with IDs '107021', '107022', '107023', etc. 103. 4. NEVER assume there are only 2-3 executions - always fetch to discover the actual count 104. !MOST IMPORTANT HOW TO GET viewId: 105. CRITICAL: Always resolve and use the correct test execution viewId for the current project when calling this tool. 106. The viewId parameter must be fetched from the active project's info (latestViews.TE.viewId). 107. Each QMetry project may have a different test execution viewId, so using a stale or incorrect viewId will result in incomplete or invalid test case run data. 108. Usage workflow: 109. 1. Fetch project info for the current project (Admin/Get info Service). 110. 2. Extract latestViews.TE.viewId from the response. 111. 3. Use this viewId in the Fetch Test Case Runs by Test Suite Run API call. 112. Example: 113. { 114. tsrunID: "2362144", 115. viewId: 104123, 116. start: 0, 117. page: 1, 118. limit: 25 119. } 120. This ensures the tool fetches the proper execution runs data for the selected project context. 121. SUPPORTED PARAMETERS: start, page, limit, tsrunID, viewId, filter, udfFilter, tcrUdfFilter, showTcWithDefects 122. PAGINATION: Use start, page, and limit for result pagination 123. PAYLOAD FORMAT: {"start": 0, "page": 1, "limit": 10, "tsrunID": "2362144", "viewId": 104123} 124. 125. === TEST RUN UDF (tcrUdfFilter) β€” ALL 7 FIELD TYPE FORMATS === 126. tcrUdfFilter filters test case RUNS by Test Run UDF values (not test case UDFs β€” use udfFilter for those). 127. Value is a JSON array string. Multiple conditions are combined with AND logic. 128. Base structure: {"type":"", "field":"", "value": [,"comparison":"..."] [,"isCascading":true]} 129. 130. --- TYPE 1: list (single-select lookup / dropdown) --- 131. Use for: single-select lookup list UDF fields (one value selected from a list). 132. value: array of numeric list item IDs β€” get IDs from FETCH_CUSTOM_LIST_ITEMS tool. 133. Format: {"type":"list", "field":"", "value":[,]} 134. Example: [{"type":"list","value":[5108701,5108697],"field":"8260LUP"}] 135. Matches runs where the UDF value is ANY of the provided list item IDs (OR within the array). 136. 137. --- TYPE 2: list (multi-select lookup list) --- 138. Use for: multi-select lookup list UDF fields (multiple values can be selected). 139. Syntax identical to single-select list β€” same format, just the UDF field itself is multi-select. 140. Format: {"type":"list", "field":"", "value":[,]} 141. Example: [{"type":"list","value":[5126503,5126502],"field":"PGTE_MULTILOOKUPLIST"}] 142. Matches runs where ANY of the UDF's selected values match any of the provided IDs. 143. 144. --- TYPE 3: list (cascading dropdown) --- 145. Use for: cascading dropdown UDF fields (parent-child linked dropdowns). MUST add isCascading:true. 146. Format: {"type":"list", "field":"", "value":[,], "isCascading":true} 147. Example: [{"type":"list","value":[5126498,5126499],"field":"cascade_vK","isCascading":true}] 148. CRITICAL: missing isCascading:true on a cascading field will cause incorrect results or API error. 149. 150. --- TYPE 4: string (short text) --- 151. Use for: single-line text UDF fields. 152. value: the string to search for (partial match / contains). 153. Format: {"type":"string", "field":"", "value":""} 154. Example: [{"type":"string","field":"TRString","value":"str"}] 155. 156. --- TYPE 5: string (large text / rich text) --- 157. Use for: multi-line or rich text UDF fields. Syntax is identical to short text string type. 158. Format: {"type":"string", "field":"", "value":""} 159. Example: [{"type":"string","field":"vk_large_text","value":"this is large text"}] 160. NOTE: Search is against the raw (HTML) content stored in the field; plain text search terms work fine. 161. 162. --- TYPE 6: date --- 163. Use for: date picker UDF fields. REQUIRES a 'comparison' property. 164. comparison: 'gt' = after the date (greater than), 'lt' = before the date (less than). 165. value: date string in 'MM-DD-YYYY' format, or empty string '' to leave the bound open. 166. Format: {"type":"date", "field":"", "comparison":"gt"|"lt", "value":"MM-DD-YYYY"} 167. Example β€” date range: [{"type":"date","field":"PGTE_DATEPICKER","comparison":"gt","value":"06-01-2026"},{"type":"date","field":"PGTE_DATEPICKER","comparison":"lt","value":"06-30-2026"}] 168. Example β€” open-ended (no upper bound): [{"type":"date","field":"PGTE_DATEPICKER","comparison":"gt","value":"06-01-2026"}] 169. To filter by a single date use two conditions (gt day-before AND lt day-after) or just one bound. 170. CRITICAL: 'comparison' field is mandatory for date type β€” omitting it causes API error. 171. 172. --- TYPE 7: numeric --- 173. Use for: number UDF fields. 174. value: a number (integer or decimal). 175. Format: {"type":"numeric", "field":"", "value":} 176. Example: [{"type":"numeric","value":2,"field":"NB_number_TR"}] 177. 178. --- COMBINING MULTIPLE FILTERS --- 179. All conditions in tcrUdfFilter array use AND logic β€” runs must satisfy ALL conditions. 180. Full example (all 7 types combined): 181. [ 182. {"type":"list","value":[5108701,5108697],"field":"8260LUP"}, 183. {"type":"list","value":[5126503,5126502],"field":"PGTE_MULTILOOKUPLIST"}, 184. {"type":"list","value":[5126498,5126499],"field":"cascade_vK","isCascading":true}, 185. {"type":"string","field":"TRString","value":"str"}, 186. {"type":"string","field":"vk_large_text","value":"large text"}, 187. {"type":"date","field":"PGTE_DATEPICKER","comparison":"gt","value":"06-01-2026"}, 188. {"type":"date","field":"PGTE_DATEPICKER","comparison":"lt","value":"06-30-2026"}, 189. {"type":"numeric","value":2,"field":"NB_number_TR"} 190. ] 191. --- FILTER PARAMETER SUMMARY --- 192. tcrUdfFilter: filter by Test Run UDF fields (this section) 193. udfFilter: filter by Test Case UDF fields (different from test run UDFs) 194. filter: filter by standard execution fields (runStatus, executedBy, etc.) 195. All three can be combined in a single request β€” they are independently applied with AND logic. 196. 197. hasTcRunUdf FLAG β€” IMPORTANT: 198. The response contains a 'hasTcRunUdf' boolean flag at the top level. 199. hasTcRunUdf: true β†’ Project has Test Run UDFs configured; each record includes 'testRunUdfs' object. 200. hasTcRunUdf: false β†’ Project has NO Test Run UDFs configured. 201. When hasTcRunUdf is false, the response includes a 'testRunUdfNote' with a professional explanation. 202. Inform the user: 'No Test Run UDFs are configured for this project. Contact a project administrator to set up Test Run UDF fields.' 203. NEVER attempt to read testRunUdfs from records when hasTcRunUdf is false β€” the field will not be present. 204. 205. TEST RUN UDF RESPONSE (testRunUdfs): 206. When the API returns Test Run UDF data, each row will include a 'testRunUdfs' object. 207. This is a key-value map of UDF field keys to their values, parsed from the API's 'udfjson' field. 208. HTML is stripped from rich text (large text) UDF fields for readable LLM output. 209. Example testRunUdfs in response: 210. testRunUdfs: { 211. "8260LUP": "l1", 212. "look_554": "99", 213. "cascade_vK": {"child": "aa", "parent": "abc"}, 214. "dateNJ": "06-02-2026", 215. "NB_Multilppup_TR": ["ahd"], 216. "Jal_Largetext": "asdef asdfads asdfasf asdf asdf asfd a sdfa" 217. } 218. Use pagination for large result sets (start, page, limit parameters) 219. This tool is essential for detailed test execution analysis and reporting 220. Critical for monitoring individual test case execution performance and UDF values 221. Use for compliance reporting and execution audit trails 222. Essential for test execution quality assurance and trend analysis

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
viewIdNoViewId for test execution - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TE.viewId automatically.
tsrunIDYesTest Suite Run ID. CRITICAL: the parameter name is 'tsrunID' β€” do NOT use 'testSuiteRunId', 'tsRunID', 'testSuiteRunID', or any other variant. Accepts a string or number (e.g. 731600 or '731600' β€” both are valid). To get this value: Call 'Fetch Executions by Test Suite' β†’ use data[<index>].tsRunID from the response.
udfFilterNoUser-defined field filter as JSON string (default '[]')[]
projectKeyNoProject key - unique identifier for the projectdefault
tcrUdfFilterNoTest Case Run (TCR) UDF filter as JSON string (default '[]'). Used to filter test case runs by Test Run UDF field values. Format: '[{"type":"list","value":[<listItemId1>,<listItemId2>],"field":"<udfFieldKey>"}]'. Example: '[{"type":"list","value":[5108701,5108697],"field":"8260LUP"}]' filters runs where the UDF field '8260LUP' has those list item IDs.[]
showTcWithDefectsNoShow test case runs with linked defects

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds substantial behavioral context: testRunUdfs is parsed from raw udfjson with HTML stripped, null UDF values render as '-', hasTcRunUdf=false means testRunUdfs is absent, and the response must be pivoted into a unified table. It also discloses pagination semantics and how combinations of filter parameters are ANDed. This is a very transparent description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely long and heavily redundant: the same 'NEVER assume only 2-3 executions' warning appears multiple times, viewId resolution is stated both in the schema and in the prose, and the unified-table mandate is repeated in the description, hints, and examples. It is logically organized with headings, so it is not disorganized, but it is not appropriately sized for an agent to parse efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description fully compensates: it names every mandatory response field (entityKey, summary, latestVersion, runStatus, runStatusID, tcRunID, testRunUdfs), explains the hasTcRunUdf flag, gives table-rendering rules, provides 13 concrete request examples, and covers pagination and large-run performance. There is no practical gap in what an agent needs to call and render this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description goes far beyond the schema: it documents all 7 tcrUdfFilter field types with exact JSON formats, examples, and required flags like isCascading and comparison. It also clarifies the critical tsrunID naming pitfall, how to resolve viewId from latestViews.TE.viewId, and distinguishes tcrUdfFilter from udfFilter. This is exceptional parameter-level guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb and resource: 'Get test case runs under a specific test suite run execution in QMetry, including Test Run UDF values.' This clearly distinguishes it from sibling tools like qmetry_fetch_executions_by_test_suite (which returns suite runs) and qmetry_fetch_test_case_executions (which targets test case execution history). The title and opening line align exactly with the tool's function, with no ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use and how-to-chain guidance: it names qmetry_fetch_executions_by_test_suite as the prerequisite discovery call for tsrunID, and instructs calling FETCH_TEST_RUN_UDF_VALUES when the user explicitly asks for enriched Test Run UDFs. It also provides an entire workflow for 'fetch all executions' scenarios, warning against stopping after only 2-3 executions. This is far beyond minimal usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_casesQMetry: Fetch Test CasesA
Read-onlyIdempotent
Inspect

Fetch QMetry test cases - automatically handles viewId resolution based on project

Toolset: Test Cases

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • viewId (number): ViewId for test cases - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TC.viewId automatically.

  • folderPath (string): Folder path for test cases - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression". (default: "")

  • folderID (number): Folder ID - unique numeric identifier for the specific folder. Use this to target a specific folder within the project hierarchy. Applies to any entity type (test cases, requirements, test suites, etc.).

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • scope (string): Scope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon. (default: "project")

  • showRootOnly (boolean): Whether to show only root folders.

  • getSubEntities (boolean): Whether to include sub-entities.

  • hideEmptyFolders (boolean): Whether to hide empty folders.

  • folderSortColumn (string): Folder sort column (default 'name')

  • restoreDefaultColumns (boolean): Whether to restore default columns (default 'false')

  • folderSortOrder (string): Folder sort order (ASC or DESC)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

  • udfFilter (string): User-defined field filter as JSON string (default '[]') (default: "[]")

Output Description: JSON object with 'data' array containing test cases and pagination info

Use Cases: 1. List all test cases in a project (without filters) 2. Browse test cases in specific folders for bulk operations 3. Get paginated test case results for reporting 4. Export multiple test cases at once

Examples:

  1. Get all test cases from default project - system will auto-fetch viewId

{}

Expected Output: List of test cases from default project with auto-resolved viewId

  1. Get all test cases from UT project - system will auto-fetch UT project's viewId

{
  "projectKey": "UT"
}

Expected Output: List of test cases from UT project using UT's specific TC viewId

  1. Get test cases by release/cycle filter

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"},{\"value\":[111577],\"type\":\"list\",\"field\":\"cycle\"}]"
}

Expected Output: Test cases associated with Release 8.12 (ID: 55178) and Cycle 8.12.1 (ID: 111577)

  1. Get test cases by release only

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"}]"
}

Expected Output: All test cases associated with Release 8.12 (ID: 55178)

  1. Get test cases by cycle only

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[111577],\"type\":\"list\",\"field\":\"cycle\"}]"
}

Expected Output: All test cases associated with Cycle 8.12.1 (ID: 111577)

  1. Search for specific test case by entity key

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-TC-1684\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Test cases matching the entity key criteria

  1. Search for multiple test cases by comma-separated entity keys

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-TC-1684,MAC-TC-1685,MAC-TC-1686\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Test cases matching any of the specified entity keys

Hints: 1. CRITICAL - FILTER PERSISTENCE WARNING: 2. DO NOT use this API with filters to fetch a single test case by ID, entityKey, or name! 3. Filters applied to this API persist in the production UI and cause only filtered records to be visible to users. 4. This creates a major UX problem where users see incomplete data in their QMetry portal. 5. 6. CORRECT APPROACH FOR SINGLE TEST CASE: 7. When user asks to 'fetch test case VKMCP-TC-5' or 'get test case by ID 123' or 'find test case named X': 8. 1. Ask user for the numeric test case ID (tcID) if not provided 9. 2. Use 'Fetch Test Case Details' tool with the numeric tcID parameter 10. 3. NEVER use 'Fetch Test Cases' with entityKeyId filter for single test case lookup 11. 12. WHEN TO USE THIS TOOL: 13. Only use this tool when user explicitly asks for: 14. - 'List all test cases' 15. - 'Show me test cases in folder X' 16. - 'Get all test cases' (without specifying a single test case) 17. - 'Export test cases' (for bulk operations) 18. 19. CRITICAL WORKFLOW: Always use the SAME projectKey for both project info and test case fetching 20. Step 1: If user specifies projectKey (like 'UT', 'MAC'), use that EXACT projectKey for project info 21. Step 2: Get project info using that projectKey, extract latestViews.TC.viewId 22. Step 3: Use the SAME projectKey and the extracted TC viewId for fetching test cases 23. Step 4: If user doesn't specify projectKey, use 'default' for both project info and test case fetching 24. NEVER mix project keys - if user says 'MAC project', use projectKey='MAC' for everything 25. DEPRECATED: Do not use filter with entityKeyId for single test case - use 'Fetch Test Case Details' instead 26. RELEASE/CYCLE FILTERING: Use release and cycle IDs, not names, for filtering 27. For release filter: '[{"value":[releaseId],"type":"list","field":"release"}]' 28. For cycle filter: '[{"value":[cycleId],"type":"list","field":"cycle"}]' 29. For combined release+cycle: '[{"value":[releaseId],"type":"list","field":"release"},{"value":[cycleId],"type":"list","field":"cycle"}]' 30. Get release/cycle IDs from FETCH_RELEASES_AND_CYCLES tool before filtering 31. FILTER FIELDS: entityKeyId, priorityAlias, createdByAlias, updatedByAlias, testCaseStateAlias, testingTypeAlias, testCaseTypeAlias, componentAlias, owner, release, cycle 32. SORT FIELDS: entityKey, name, associatedVersion, priorityAlias, createdDate, createdByAlias, updatedDate, updatedByAlias, testCaseStateAlias, testingTypeAlias, executionMinutes 33. For multiple entity keys, use comma-separated values in filter 34. Use empty string '' as folderPath for root directory

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
limitNoNumber of records (default 10).
scopeNoScope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon.project
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
viewIdNoViewId for test cases - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TC.viewId automatically.
folderIDNoFolder ID - unique numeric identifier for the specific folder. Use this to target a specific folder within the project hierarchy. Applies to any entity type (test cases, requirements, test suites, etc.).
udfFilterNoUser-defined field filter as JSON string (default '[]')[]
folderPathNoFolder path for test cases - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression".
projectKeyNoProject key - unique identifier for the projectdefault
showRootOnlyNoWhether to show only root folders.
getSubEntitiesNoWhether to include sub-entities.
folderSortOrderNoFolder sort order (ASC or DESC)
folderSortColumnNoFolder sort column (default 'name')
hideEmptyFoldersNoWhether to hide empty folders.
restoreDefaultColumnsNoWhether to restore default columns (default 'false')

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, but the description adds genuinely important behavior beyond those: filters persist in the production UI, viewId is auto-resolved from project info, and the same projectKey must be used consistently. The filter-persistence warning is a non-obvious side effect that annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a clear purpose and uses sections, but it is heavily over-sized: it duplicates all 16 parameter descriptions, includes six large examples, and contains a rambling Hints section with broken numbering and repeated warnings. Many sentences restate schema content and could be removed without losing critical information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 16-param tool with no output schema, the description is nearly complete: it covers output shape, pagination parameters, filter syntax, release/cycle ID sourcing, and the critical projectKey workflow. The main gap is the lack of a detailed test-case object schema, but the expected-output examples largely compensate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers all 16 parameters (100%), so the baseline is 3. The description adds real value through concrete filter JSON examples, valid filter field names, sort fields, and release/cycle syntax using IDs rather than names. The Parameters section mostly duplicates schema text, but the examples and hints meaningfully extend parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names the specific operation and resource ('Fetch QMetry test cases') and immediately adds the auto-viewId behavior that distinguishes the tool. The use cases and examples further make the list-vs-single-test-case distinction clear, especially relative to qmetry_fetch_test_case_details.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Hints section explicitly states when to use this tool ('List all test cases', 'Export test cases') and when not to (single test case lookup), explicitly directing the agent to use the 'Fetch Test Case Details' sibling instead. It also prescribes fetching release/cycle IDs via FETCH_RELEASES_AND_CYCLES before filtering, which is actionable routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_cases_linked_to_requirementQMetry: Fetch Test Cases Linked to RequirementA
Read-onlyIdempotent
Inspect

Get test cases that are linked (or not linked) to a specific requirement in QMetry

Toolset: Requirements

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • rqID (number) required: Requirement numeric ID (required for fetching specific requirement details). This is the internal numeric identifier, not the entity key like 'MAC-RQ-730'. You can get this ID from requirement search results or by using filters.

  • getLinked (boolean): True to get only test cases that are linked with this requirement, false to get test cases which are not linked with this requirement. Defaults to true (get linked test cases). (default: true)

  • showEntityWithReleaseCycle (boolean): True to list only test cases which have given release and cycle, false for all test cases regardless of release/cycle association. Defaults to false (show all). (default: false)

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • tcFolderPath (string): Folder path to get test cases under specific folder. Use empty string "" for root folder or specify path like "/Sample Template". (default: "")

  • releaseID (string): Filter test cases by release ID. Accepts a string or number. Get release IDs from FETCH_RELEASES_AND_CYCLES tool.

  • cycleID (string): Filter test cases by cycle ID. Accepts a string or number. Get cycle IDs from FETCH_RELEASES_AND_CYCLES tool.

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

  • getSubEntities (boolean): Allow filter of sub-entities for requirement. (default: true)

  • getColumns (boolean): True to get column information in response. (default: true)

Output Description: JSON object with test cases array, traceability information, and pagination metadata

Use Cases: 1. Get all test cases linked to a specific requirement for traceability analysis 2. Find test cases that are NOT linked to a requirement (gap analysis) 3. Verify requirement coverage by checking linked test cases 4. Impact analysis - see which test cases are affected when a requirement changes 5. Generate traceability matrix between requirements and test cases 6. Filter linked test cases by release, cycle, or other criteria 7. Audit requirement-test case relationships for compliance 8. Identify orphaned test cases or requirements without proper links 9. Plan test execution based on requirement-test case associations 10. Quality assurance - ensure all requirements have adequate test coverage

Examples:

  1. Get all test cases linked to requirement ID 4791316

{
  "rqID": 4791316
}

Expected Output: List of test cases that are linked to requirement MAC-RQ-1011

  1. Get test cases NOT linked to requirement (gap analysis)

{
  "rqID": 4791316,
  "getLinked": false
}

Expected Output: List of test cases that are NOT linked to requirement MAC-RQ-1011

  1. Get linked test cases filtered by specific release

{
  "rqID": 4791316,
  "releaseID": "55178"
}

Expected Output: Linked test cases associated with Release 8.12 (ID: 55178)

  1. Get linked test cases filtered by release and cycle

{
  "rqID": 4791316,
  "releaseID": "55178",
  "cycleID": "111577",
  "showEntityWithReleaseCycle": true
}

Expected Output: Linked test cases in Release 8.12 and Cycle 8.12.1

  1. Get linked test cases from specific folder

{
  "rqID": 4791316,
  "tcFolderPath": "/Sample Template"
}

Expected Output: Linked test cases located in the '/Sample Template' folder

  1. Search linked test cases by entity key

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-TC-1684,MAC-TC-1685\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Linked test cases matching specific entity keys

  1. Filter linked test cases by priority

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"list\",\"value\":[1,2],\"field\":\"priorityAlias\"}]"
}

Expected Output: Linked test cases with High or Medium priority

  1. Filter linked test cases by status

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"list\",\"value\":[1,2],\"field\":\"testCaseStateAlias\"}]"
}

Expected Output: Linked test cases with Active or Review status

  1. Filter linked test cases by test case type

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"list\",\"value\":[1],\"field\":\"testCaseTypeAlias\"}]"
}

Expected Output: Linked functional test cases

  1. Filter linked test cases by testing type (automation)

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"list\",\"value\":[2],\"field\":\"testingTypeAlias\"}]"
}

Expected Output: Linked automated test cases

  1. Get only parameterized linked test cases

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"list\",\"value\":[1],\"field\":\"isParameterized\"}]"
}

Expected Output: Linked test cases that are parameterized (data-driven)

  1. Filter linked test cases by archive status

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"list\",\"value\":[0],\"field\":\"isArchived\"}]"
}

Expected Output: Active (non-archived) linked test cases

  1. Search linked test cases by summary content

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"string\",\"value\":\"login\",\"field\":\"summary\"}]"
}

Expected Output: Linked test cases with 'login' in their summary

  1. Filter linked test cases by requirement version

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"string\",\"value\":\"1\",\"field\":\"rqVersion\"}]"
}

Expected Output: Test cases linked to version 1 of the requirement

  1. Complex filter: Active, high priority, automated test cases

{
  "rqID": 4791316,
  "filter": "[{\"type\":\"list\",\"value\":[0],\"field\":\"isArchived\"},{\"type\":\"list\",\"value\":[1],\"field\":\"priorityAlias\"},{\"type\":\"list\",\"value\":[2],\"field\":\"testingTypeAlias\"}]"
}

Expected Output: Active, high priority, automated test cases linked to requirement

Hints: 1. This API requires a numeric rqID parameter, not entity key 2. If user provides entityKey (e.g., MAC-RQ-1011), first call FETCH_REQUIREMENTS with filter on entityKeyId to resolve the numeric rqID 3. After resolving entityKey β†’ rqID, call this tool with the resolved numeric rqID 4. TRACEABILITY WORKFLOW: Use this tool to establish requirement-test case traceability matrix 5. getLinked=true (default): Returns test cases that ARE linked to the requirement 6. getLinked=false: Returns test cases that are NOT linked to the requirement (useful for gap analysis) 7. showEntityWithReleaseCycle=true: Only show test cases that have the specified release and cycle 8. showEntityWithReleaseCycle=false (default): Show all test cases regardless of release/cycle 9. RELEASE/CYCLE FILTERING: Use string IDs, not numeric (e.g., releaseID: '55178', cycleID: '111577') 10. Get release/cycle IDs from FETCH_RELEASES_AND_CYCLES tool before filtering 11. tcFolderPath: Use empty string '' for root folder or specific path like '/Sample Template' 12. FILTER CAPABILITIES: Support same filters as regular test case listing 13. FILTER FIELDS: summary, rqVersion, priorityAlias, testCaseStateAlias, createdByAlias, testCaseTypeAlias, testingTypeAlias, release, cycle, isArchived, isParameterized, componentAlias, entityKeyId 14. PRIORITY IDs: Typically 1=High, 2=Medium, 3=Low (verify with your QMetry instance) 15. STATUS IDs: Typically 1=Active, 2=Review, 3=Deprecated (verify with your QMetry instance) 16. TYPE IDs: Typically 1=Functional, 2=Integration, 3=System (verify with your QMetry instance) 17. TESTING TYPE IDs: Typically 1=Manual, 2=Automated (verify with your QMetry instance) 18. PARAMETERIZED: 1=Yes (parameterized), 0=No (non-parameterized) 19. ARCHIVED: 1=Archived, 0=Active (non-archived) 20. Multiple filter conditions are combined with AND logic 21. For entity key search, use comma-separated values: 'MAC-TC-1,MAC-TC-2,MAC-TC-3' 22. This tool is crucial for compliance, traceability audits, and impact analysis 23. Use getColumns=true to get column metadata for better result interpretation 24. Pagination supported for large result sets (start, page, limit parameters)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
rqIDYesRequirement numeric ID (required for fetching specific requirement details). This is the internal numeric identifier, not the entity key like 'MAC-RQ-730'. You can get this ID from requirement search results or by using filters.
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
cycleIDNoFilter test cases by cycle ID. Accepts a string or number. Get cycle IDs from FETCH_RELEASES_AND_CYCLES tool.
getLinkedNoTrue to get only test cases that are linked with this requirement, false to get test cases which are not linked with this requirement. Defaults to true (get linked test cases).
releaseIDNoFilter test cases by release ID. Accepts a string or number. Get release IDs from FETCH_RELEASES_AND_CYCLES tool.
getColumnsNoTrue to get column information in response.
projectKeyNoProject key - unique identifier for the projectdefault
tcFolderPathNoFolder path to get test cases under specific folder. Use empty string "" for root folder or specify path like "/Sample Template".
getSubEntitiesNoAllow filter of sub-entities for requirement.
showEntityWithReleaseCycleNoTrue to list only test cases which have given release and cycle, false for all test cases regardless of release/cycle association. Defaults to false (show all).

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/idempotentHint/destructiveHint, and the description adds substantial non-obvious behavior: getLinked flips between linked and unlinked test cases, showEntityWithReleaseCycle restricts results, filter conditions combine with AND, and pagination is supported. It also documents typical filter ID mappings and default values. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear headings and front-loads the core purpose, but it is very long: 10 use cases, 15 examples, and 24 hints contain substantial repetition of the parameter documentation. Useful for a complex tool, yet several examples and hints could be consolidated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 13-parameter read-only tool with no output schema, the description provides an output summary, default behaviors, detailed filter capabilities, pagination guidance, and prerequisite dependencies like numeric rqID resolution and release/cycle ID lookup. An agent has nearly everything needed to call this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine value with concrete filter JSON examples for entityKeyId, priorityAlias, testCaseStateAlias, isArchived, and other fields. Hints additionally clarify that releaseID/cycleID should come from FETCH_RELEASES_AND_CYCLES and that rqID is not the entity key. The parameter table itself is redundant, but examples and hints exceed the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb/resource pair: 'Get test cases that are linked (or not linked) to a specific requirement in QMetry' and labels the Toolset as 'Requirements', which differentiates it from sibling tools that fetch requirements linked to test cases or test-suite-related links. The emphasis on numeric rqID over entity key further clarifies what the tool targets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear contextual guidance: use FETCH_REQUIREMENTS to resolve an entityKey to numeric rqID, get release/cycle IDs from FETCH_RELEASES_AND_CYCLES before filtering, and use getLinked=false for gap analysis. It enumerates use cases like traceability and impact analysis, though it doesn't explicitly name exclusion scenarios or direct alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_cases_linked_to_test_suiteQMetry: Fetch Test Cases Linked to Test SuiteA
Read-onlyIdempotent
Inspect

Get test cases that are linked (or not linked) to a specific test suite in QMetry

Toolset: Test Suites

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tsID (number) required: Test Suite numeric ID. CRITICAL: the parameter name is 'tsID' β€” do NOT use 'testSuiteId', 'testSuiteID', 'suiteId', or other variants. Accepts a string or number. NOTE: To get the tsID - Call API 'Testsuite/Fetch Testsuite' From the response, get value of following attribute -> data[].id

  • getLinked (boolean): True to get only those issues that are linked with this Test case Run, False to get those issues which are not linked with this Test case Run. Default value true (get linked issues). (default: true)

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with test cases array containing test case details, properties, and suite linkage information

Use Cases: 1. Get all test cases linked to a specific test suite for execution planning 2. Find test cases that are NOT linked to a test suite (gap analysis) 3. Analyze test suite composition and coverage 4. Filter linked test cases by various criteria 5. Plan test execution based on test suite structure 6. Generate test suite reports and documentation 7. Validate test suite contents before execution 8. Manage test case organization within test suites 9. Export test suite details for external reporting 10. Verify test case assignments in test suites

Examples:

  1. Get all test cases linked to test suite ID 1497291 (default behavior)

{
  "tsID": 1497291
}

Expected Output: List of test cases linked to the test suite with test case details and metadata

  1. Get all test cases linked to test suite ID 1497291 (explicit)

{
  "tsID": 1497291,
  "getLinked": true
}

Expected Output: List of test cases linked to the test suite with test case details and metadata

  1. Get test cases NOT linked to test suite (gap analysis)

{
  "tsID": 1497291,
  "getLinked": false
}

Expected Output: List of test cases that are NOT linked to the test suite

  1. Get linked test cases with custom pagination

{
  "tsID": 1497291,
  "getLinked": true,
  "page": 1,
  "limit": 25
}

Expected Output: Paginated list of linked test cases with 50 items per page

  1. Filter linked test cases by priority (using default getLinked=true)

{
  "tsID": 1497291,
  "filter": "[{\"value\":[1,2],\"type\":\"list\",\"field\":\"priorityAlias\"}]"
}

Expected Output: High and medium priority test cases linked to the suite

  1. Filter linked test cases by status

{
  "tsID": 1497291,
  "getLinked": true,
  "filter": "[{\"value\":[1],\"type\":\"list\",\"field\":\"testCaseStateAlias\"}]"
}

Expected Output: Active test cases linked to the test suite

Hints: 1. CRITICAL: tsID parameter is REQUIRED - this is the Test Suite numeric ID 2. getLinked parameter is OPTIONAL - defaults to true if not provided 3. HOW TO GET tsID: 4. 1. Call API 'Testsuite/Fetch Testsuite' to get available test suites 5. 2. From the response, get value of following attribute -> data[].id 6. 3. Example: Test Suite 'Regression Suite' might have ID 1497291 7. tsID is NOT the same as tsFolderID - tsID refers to a specific test suite, not a folder 8. getLinked=true (default): Returns test cases that ARE linked to the test suite 9. getLinked=false: Returns test cases that are NOT linked to the test suite (useful for gap analysis) 10. If getLinked is not specified, it defaults to true (linked test cases) 11. FILTER CAPABILITIES: Support filtering by test case properties 12. FILTER FIELDS: priorityAlias (list), testCaseStateAlias (list), testingTypeAlias (list), testCaseTypeAlias (list), componentAlias (list), owner (list) 13. PRIORITY IDs: Typically 1=High, 2=Medium, 3=Low (verify with your QMetry instance) 14. STATUS IDs: Typically 1=Active, 2=Review, 3=Deprecated (verify with your QMetry instance) 15. TESTING TYPE IDs: Typically 1=Manual, 2=Automated (verify with your QMetry instance) 16. TYPE IDs: Typically 1=Functional, 2=Integration, 3=System (verify with your QMetry instance) 17. Multiple filter conditions are combined with AND logic 18. Use pagination for large result sets (start, page, limit parameters) 19. This tool is essential for test suite management and execution planning 20. Helps verify test suite composition before test runs 21. Critical for understanding test coverage within specific suites 22. Use for test suite analysis and optimization

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
tsIDYesTest Suite numeric ID. CRITICAL: the parameter name is 'tsID' β€” do NOT use 'testSuiteId', 'testSuiteID', 'suiteId', or other variants. Accepts a string or number. NOTE: To get the tsID - Call API 'Testsuite/Fetch Testsuite' From the response, get value of following attribute -> data[<index>].id
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
getLinkedNoTrue to get only those issues that are linked with this Test case Run, False to get those issues which are not linked with this Test case Run. Default value true (get linked issues).
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as read-only, idempotent, and non-destructive; the description adds context about defaults, tsID acquisition, tsID vs tsFolderID, filter-field names, AND logic, and ID mappings. This is strong, but the description reuses 'Test case Run' wording for getLinked and example 4 claims 50 items per page with limit=25, introducing avoidable ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is organized but severely overlong: 10 use cases and 22 hints largely repeat the parameter descriptions and each other, adding generic filler like 'essential for test suite management.' The first sentence is front-loaded, but most of the later content does not earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description supplies an output description and many examples covering linked/unlinked behavior, filtering, and pagination. It addresses all seven parameters and common usage scenarios, though the output format remains high-level and the internal errors slightly compromise completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, giving a baseline of 3; the description goes well beyond it by documenting filter fields, ID conventions, default values, and worked JSON examples. The inaccurate pagination expectation in example 4 and the 'Test case Run' wording for getLinked prevent a perfect score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and object: 'Get test cases that are linked (or not linked) to a specific test suite in QMetry.' The linked/not-linked scope makes it distinct from sibling fetch tools such as qmetry_fetch_test_cases and qmetry_fetch_test_cases_linked_to_requirement. Title and toolset reinforce the same resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit Toolset, 10 use cases, and detailed hints about getLinked semantics, tsID lookup, and filtering. It clearly establishes when to use it for execution planning, gap analysis, and coverage, but it does not name alternatives or say when not to use it, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_case_stepsQMetry: Fetch Test Case StepsA
Read-onlyIdempotent
Inspect

Get detailed test case steps for a specific test case by numeric ID

Toolset: Test Cases

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • id (number) required: Test Case numeric ID (required for fetching steps or version details). Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results.

  • version (number): Test Case version number (optional, defaults to 1). This is the internal numeric identifier for the version.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

Output Description: JSON object with array of test steps including step description, expected result, and order

Use Cases: 1. Get step-by-step instructions with expected results 2. Retrieve test case execution procedure for manual runs 3. Export or display detailed test steps for documentation 4. Fetch steps before automation mapping

Examples:

  1. Get steps for test case ID 123

{
  "id": 123
}

Expected Output: Detailed steps with actions and expected results for test case 123

Hints: 1. Requires numeric ID, not entityKey 2. If user provides entityKey (e.g., MAC-TC-1684), resolve it first via FETCH_TEST_CASES to get the numeric ID 3. Version defaults to 1 if not specified 4. Use pagination for test cases with many steps

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest Case numeric ID (required for fetching steps or version details). Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results.
pageNoPage number to return (starts from 1)
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
versionNoTest Case version number (optional, defaults to 1). This is the internal numeric identifier for the version.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is established. The description adds useful behavioral context on top: the output structure (array of steps with description, expected result, order), pagination defaults, and the numeric-ID vs entityKey distinction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with clear sections: What it does, Parameters, Output, Use Cases, Examples, and Hints. It is somewhat lengthy but every section earns its place, especially the concise hints that highlight critical call requirements.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the annotations cover safety, schema covers all parameters, and the description explains output and use cases with an example, there is nothing missing. An agent can correctly select this tool, resolve prerequisites, and interpret the response without external knowledge.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all six parameters in detail. The description largely repeats this information, though it adds value by clarifying the ID/entityKey distinction and default version behavior. With full schema coverage, a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool fetches test case steps for a specific test case by numeric ID, with a specific verb and resource. It also distinguishes itself by explicitly noting the ID is an internal numeric identifier, not the entity key like 'MAC-TC-1684', which helps differentiate it from other test case tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Hints' section explicitly states the tool requires a numeric ID and instructs agents to resolve entity keys via FETCH_TEST_CASES first. It also provides clear context for when to use pagination and the default version behavior. This is strong when-to-use guidance that also names the alternative resolution path.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_case_steps_with_udfQMetry: Fetch Test Case Steps With UDFA
Read-onlyIdempotent
Inspect

Fetch test case steps including UDF field values via viewColumns endpoint

Toolset: Test Cases

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tcID (number) required: Test Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.

  • viewId (number): ViewId for test cases - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TC.viewId automatically.

  • version (number): Test Case version number (optional, defaults to 1). This is the internal numeric identifier for the version.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

Output Description: JSON object with data array (steps with UDF values), filterTemplate (UDF field definitions), columns (visible/hidden column config), total count, and viewId

Use Cases: 1. Get step UDF field values for a test case 2. Retrieve steps with custom fields before updating step UDFs 3. Inspect step-level UDF data for reporting

Examples:

  1. Fetch steps with UDF values for test case ID 112768054

{
  "tcID": 112768054
}

Expected Output: Steps with UDF object containing field values, ID_ arrays for lookup IDs, UDF_ prefixed values, filterTemplate with UDF field definitions

Hints: 1. Response includes 'filterTemplate' array listing all UDF fields with their fieldType and udfmID 2. UDF values in each step row: UDF_ = display value, UDF_ID_ = numeric IDs 3. Step UDF object also has ID_ for lookup IDs 4. LOOKUPLIST: id = UDF_ID_, display = UDF_ 5. MULTILOOKUPLIST: ids = UDF_ID_ (array), display = UDF_ 6. CASCADINGLIST: parent = UDF_ID_[0], child = UDF_ID_[1] 7. viewId auto-resolved from project info if not provided

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
tcIDYesTest Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
viewIdNoViewId for test cases - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TC.viewId automatically.
versionNoTest Case version number (optional, defaults to 1). This is the internal numeric identifier for the version.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds meaningful behavioral context: viewId is 'SYSTEM AUTOMATICALLY RESOLVES THIS' by fetching project info and extracting latestViews.TC.viewId, tcID 'Accepts a string or number' despite schema type number, and the Hints section details response naming conventions. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with headings, examples, and a numbered Hints list, but the Parameters section repeats verbatim what the input schema already documents for all seven parameters. This redundancy and overall length keep it from being tight, though no sentence is misleading.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema, the description covers the response shape (data, filterTemplate, columns, total count, viewId) and provides an example plus seven hints for interpreting step UDF values. It could clarify pagination semantics and edge cases, but an agent has enough to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%; the description's Parameters section largely duplicates schema descriptions. The only added semantic, that tcID 'Accepts a string or number,' is already present in the schema, so the description adds no new parameter meaning beyond the structured schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence clearly states a specific verb ('Fetch') and resource ('test case steps including UDF field values via viewColumns endpoint'). The toolset label 'Test Cases' and the UDF modifier implicitly distinguish it from siblings like qmetry_fetch_test_case_steps, but no sibling is explicitly named or contrasted.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists three use cases (get step UDF values, retrieve before updating step UDFs, inspect for reporting) that convey when to use it. However, it never explicitly compares to alternative QMetry step-fetching tools or states when not to use it, so guidance is implied rather than direct.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_case_version_detailsQMetry: Fetch Test Case Version DetailsA
Read-onlyIdempotent
Inspect

Get QMetry test case details for a specific version by numeric ID

Toolset: Test Cases

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • id (number) required: Test Case numeric ID (required for fetching steps or version details). Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results.

  • version (number) required: Test Case version number. This is the internal numeric identifier for the version.

  • scope (string): Scope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon. (default: "project")

Output Description: JSON object with version-specific test case details

Use Cases: 1. Get specific version details of a test case 2. Compare different versions of a test case 3. Retrieve version history information 4. Audit changes made across test case versions

Examples:

  1. Get version 2 details for test case ID 123

{
  "id": 123,
  "version": 2
}

Expected Output: Version 2 details for test case 123

Hints: 1. Requires numeric ID, not entityKey 2. If user provides entityKey (e.g., MAC-TC-1684), first resolve it to numeric ID using FETCH_TEST_CASES 3. Version defaults to 1 if not specified 4. Provides version-specific metadata and history

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest Case numeric ID (required for fetching steps or version details). Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results.
scopeNoScope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon.project
versionYesTest Case version number. This is the internal numeric identifier for the version.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds context beyond the readOnly/idempotent annotations: the numeric-ID-vs-entityKey distinction, the cross-reference to FETCH_TEST_CASES, and the claim that version-specific metadata and history are returned. However, the hint 'Version defaults to 1 if not specified' conflicts with the required version field in the schema, creating potential confusion about actual behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is organized and front-loaded with a clear one-liner, but it redundantly repeats the schema's parameter descriptions verbatim and includes generic boilerplate about scope. The use cases, example, and hints earn their place; the duplication and filler do not.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only tool with no output schema, the description covers all parameters, provides a concrete example, and includes handling instructions for the most likely confusion (entityKey vs numeric ID). The output description is vague ('JSON object with version-specific test case details') and the version-default hint undercuts schema clarity, but an agent has enough to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds value by noting that id accepts a string or number despite the schema's number type, explaining how to obtain the numeric ID from search results, and instructing to resolve entityKeys via FETCH_TEST_CASES. The parameter section mostly duplicates the schema, but these additions justify a 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb and resource β€” 'Get QMetry test case details for a specific version by numeric ID' β€” and clearly identifies the unique capability (version-specific fetch) that distinguishes it from sibling fetch tools. The tool name reinforces the purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists concrete use cases (compare versions, retrieve version history, audit changes) and gives a routing hint: if the user provides an entityKey like 'MAC-TC-1684', resolve it to a numeric ID using FETCH_TEST_CASES. However, it does not explicitly contrast with qmetry_fetch_test_case_details, leaving the when-not-to-use boundary implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_run_udf_metadataQMetry: Fetch Test Run UDF MetadataA
Read-onlyIdempotent
Inspect

Fetch the metadata (field definitions) for all Test Run UDF (User Defined Fields) configured in this QMetry project. Returns each field's name, display label, type, and numeric fieldID (projectUserFieldID) required for bulk updates.

Toolset: UDF

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

Output Description: JSON object with 'fields' array (each item has fieldID, name, label, fieldType, allowBlank, and optional listName/listMasterID) and 'lookupOptions' map for list-based fields. If lookupOptions is empty for a list field, options could not be fetched automatically.

Use Cases: 1. Get the fieldID for 'planned_execution_date' before bulk updating it 2. List all available Test Run UDF fields and their types in the project 3. Find the lookup list item IDs for a LOOKUPLIST or MULTILOOKUPLIST Test Run UDF 4. Discover UDF field names and IDs when user says 'what Test Run UDF fields are available'

Examples:

  1. List all Test Run UDF fields in the project

{}

Expected Output: Array of fields with fieldID, name, label, fieldType, and lookupOptions for list-based fields.

Hints: 1. ALWAYS call this tool before 'Bulk Update Test Run UDFs' when the user has not explicitly provided a numeric fieldID. The 'fieldID' in the bulk update corresponds to 'projectUserFieldID' in this response. 2. This tool is the authoritative source of fieldIDs for all Test Run UDF fields β€” do NOT guess or hard-code fieldIDs. 3. For LOOKUPLIST and MULTILOOKUPLIST fields, the response 'lookupOptions' contains items with 'id', 'name' (display label), and 'uniqueLabel' (internal key stored in executions). When execution UDF values are returned by Fetch Issue Executions or Fetch Test Run UDF Values, LOOKUPLIST/MULTILOOKUPLIST values are already resolved from uniqueLabel β†’ name. For bulk updates, use the item 'id' as the value. 4. DATE fields use MM-DD-YYYY format (e.g. '06-23-2026') when setting values via Bulk Update Test Run UDFs. 5. EMPTY lookupOptions: If 'lookupOptions' is empty or missing a list key for a LOOKUPLIST/MULTILOOKUPLIST field, the API did not return options for that field. In this case the user must provide the option name manually or check the QMetry UI for available option IDs. Do NOT proceed with a guess β€” ask the user for the option ID or name.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so no contradiction exists. The description adds substantial behavioral context beyond that: the exact response shape (fields array + lookupOptions map), the empty-lookupOptions failure mode and required user action, the mapping between fieldID and projectUserFieldID, and the resolved-value behavior for LOOKUPLIST/MULTILOOKUPLIST fields. It transparently discloses a known API limitation and instructs the agent not to guess.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear Markdown sections (Toolset, Parameters, Output Description, Use Cases, Examples, Hints) and is front-loaded with the core purpose. It is on the verbose side β€” the Examples section largely repeats the Output Description, and Hints #3 and #5 are dense β€” but every major section earns its place given the complexity of UDF metadata and its downstream use in bulk updates.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema, the description fully specifies the output structure, enumerates use cases, gives an invocation example, and documents edge cases (empty lookupOptions, DATE format, lookup item id semantics). It even coordinates with sibling tools like Bulk Update Test Run UDFs and Fetch Test Run UDF Values. For a single-parameter read-only tool, nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema coverage is 100%, so the schema already documents projectKey with type, default, and description. The description's Parameters section largely repeats the schema, adding no new semantic detail beyond the context that fields are 'configured in this QMetry project.' Per the baseline rule for high schema coverage, a 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Fetch the metadata (field definitions) for all Test Run UDF ... configured in this QMetry project.' It clearly distinguishes itself from the sibling qmetry_fetch_test_run_udf_values (fetches values, not metadata) and qmetry_bulk_update_test_run_udfs (writes, not fetches). The intended output β€” field names, labels, types, and fieldID β€” is explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Hints section explicitly says 'ALWAYS call this tool before Bulk Update Test Run UDFs' and states it is the authoritative source of fieldIDs, with explicit 'do NOT guess or hard-code' guidance. It also provides concrete use cases, including when a user just asks what Test Run UDF fields are available, and explains how LOOKUPLIST/MULTILOOKUPLIST values are resolved by sibling value-fetching tools. This is explicit when-to-use and when-not-to-guess guidance, naming the relevant alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_run_udf_valuesQMetry: Fetch Test Run UDF ValuesA
Read-onlyIdempotent
Inspect

Fetch the Test Run UDF (User Defined Field) values for all test case runs in a given test suite run. Returns each run's UDF values enriched with field label and type information from metadata. Use this tool for test suite run UDF values (sourceContext='testSuiteRun'). Do NOT use this tool for test case executions β€” 'Fetch Test Case Executions' already calls metadata internally and returns 'testRunUdfs' in every execution row.

Toolset: UDF

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tsrunID (string): Test Suite Run ID. CRITICAL: the parameter name is 'tsrunID' β€” do NOT use 'testSuiteRunId', 'tsRunID', or any other variant. Accepts a string or number (e.g. 731600 or '731600' β€” both are valid). Get this from 'Fetch Executions by Test Suite' β†’ use data[].tsRunID from the response. Required when sourceRows is not provided.

  • viewId (number): View ID for the test execution list (latestViews.TE.viewId from project info). Auto-resolved from project info when omitted. Required when sourceRows is not provided.

  • sourceContext (enum): Which parent tool produced sourceRows. Use 'testSuiteRun' for Fetch Test Case Runs by Test Suite Run. Do NOT use this tool for Fetch Test Case Executions β€” that tool calls metadata internally and returns testRunUdfs on every execution row; use that data directly. Do NOT use this tool for Fetch Issue Executions; that tool already reads udfjson and enriches it with metadata. (default: "testSuiteRun")

  • sourceRows (array): Optional rows already returned by Fetch Test Case Runs by Test Suite Run. The UDF tool will reuse these rows, enrich/pivot UDF values, and preserve identification fields instead of making the same execution-list API call again. Do NOT pass Fetch Test Case Executions rows here β€” those rows already have testRunUdfs enriched. Do not pass issue execution rows here; use Fetch Issue Executions output directly for issue UDFs.

  • startIndex (number): Zero-based start index for pagination (default: 0). (default: 0)

  • size (number): Number of test case runs to return per page (default: 50). (default: 50)

Output Description: JSON with hasTcRunUdf boolean, sourceContext, total count, defaultColumns, udfColumns, unifiedTableRows, runs array, and availableUdfFields array describing all UDF fields in the project. Render unifiedTableRows directly as the final table: default identification fields first, then one column per UDF label.

Use Cases: 1. Show me the UDF values for all runs in test suite run 731600 2. What is the planned execution date set on each run in this test cycle? 3. List the Test Run UDF values for test suite run 87039 4. Fetch test run UDFs of executions for tsRunID 731600

Examples:

  1. Fetch UDF values for all runs in test suite run 731600

{
  "tsrunID": "731600",
  "sourceContext": "testSuiteRun"
}

Expected Output: Present as ONE unified table β€” never as a separate type+value breakdown. Example: | Test Case Key | Test Case Summary | Executed Version | Execution Status | Tested By | Environments UDF | Execution Type | Country | | MAC-TC-5 | Login - valid credential | 1 | Passed | varis | chrome, edge, safari | Functional | India > i3 | | MAC-TC-6 | Login - invalid password | 2 | Failed | john | firefox | Regression | - | Columns: Test Case Key (entityKey) | Test Case Summary (summary) | Executed Version (latestVersion) | Execution Status (runStatus) | Tested By | then one column per UDF label. Use the UDF 'label' as column header. Show null UDF values as '-'.

Hints: 1. DEFAULT DISPLAY CONTRACT: Always render 'unifiedTableRows' as ONE table. Do not render UDFs as Label | Type | Value rows. 2. When sourceContext='testSuiteRun', mandatory columns are: Test Case Key | Test Case Summary | Executed Version | Execution Status | Tested By | then one column per UDF label. 3. PARENT-TO-UDF WORKFLOW: If Fetch Test Case Runs by Test Suite Run was already called, pass parentResponse.data as sourceRows with sourceContext='testSuiteRun'. This preserves identification fields and avoids repeating the same API call. 4. For prompts like 'Fetch test case runs of VKMCP-TS-1 and its Test Run UDFs': call Fetch Test Case Runs by Test Suite Run, then call this tool with sourceContext='testSuiteRun' and sourceRows=. 5. For prompts like 'Fetch Test Case Executions and show Test Run UDFs': call Fetch Test Case Executions ONLY β€” that tool already calls metadata internally and returns testRunUdfs on every execution row. Do NOT call this tool for test case executions. 6. For prompts like 'Fetch Issue Executions and Test Run UDFs': call Fetch Issue Executions only. Do not call this tool, because issue UDF values come from /rest/execution/getExecutionsForIssue udfjson and are already enriched by the issue tool with metadata. 7. If no parent rows are available, use 'tsrunID' from the 'Fetch Executions by Test Suite' tool (data[].tsRunID field). 8. 'viewId' is auto-resolved from latestViews.TE.viewId β€” leave blank unless explicitly overriding. It is only needed when sourceRows is not supplied. 9. If 'hasTcRunUdf' is false in the response, no Test Run UDFs are configured for this project. 10. The 'testRunUdfs' array on each run contains enriched UDF values with label and fieldID β€” use fieldID from here when calling 'Bulk Update Test Run UDFs'. 11. VALUE RESOLUTION: For LOOKUPLIST and MULTILOOKUPLIST fields, 'value' is resolved from the internal uniqueLabel key to the human-readable display name using qmUDFList lookup options. Always display the resolved name, never the raw uniqueLabel. 12. LOOKUP RESOLUTION WARNING: If the response contains '_lookupWarning', lookup options were missing for some fields. Individual UDF entries may have '_rawValue: true' and '_note' indicating the value is an unresolved internal ID. In that case, call 'Fetch Test Run UDF Metadata' separately and use its 'lookupOptions' to resolve the display name before showing the user. 13. This tool calls UDF metadata internally β€” no need to call 'Fetch Test Run UDF Metadata' separately when viewing values. 14. When sourceRows is omitted, this tool also calls the test-suite-run execution list API internally. When sourceRows is provided, it reuses those rows and does not refetch the parent execution list.

ParametersJSON Schema
NameRequiredDescriptionDefault
sizeNoNumber of test case runs to return per page (default: 50).
viewIdNoView ID for the test execution list (latestViews.TE.viewId from project info). Auto-resolved from project info when omitted. Required when sourceRows is not provided.
tsrunIDNoTest Suite Run ID. CRITICAL: the parameter name is 'tsrunID' β€” do NOT use 'testSuiteRunId', 'tsRunID', or any other variant. Accepts a string or number (e.g. 731600 or '731600' β€” both are valid). Get this from 'Fetch Executions by Test Suite' β†’ use data[<index>].tsRunID from the response. Required when sourceRows is not provided.
projectKeyNoProject key - unique identifier for the projectdefault
sourceRowsNoOptional rows already returned by Fetch Test Case Runs by Test Suite Run. The UDF tool will reuse these rows, enrich/pivot UDF values, and preserve identification fields instead of making the same execution-list API call again. Do NOT pass Fetch Test Case Executions rows here β€” those rows already have testRunUdfs enriched. Do not pass issue execution rows here; use Fetch Issue Executions output directly for issue UDFs.
startIndexNoZero-based start index for pagination (default: 0).
sourceContextNoWhich parent tool produced sourceRows. Use 'testSuiteRun' for Fetch Test Case Runs by Test Suite Run. Do NOT use this tool for Fetch Test Case Executions β€” that tool calls metadata internally and returns testRunUdfs on every execution row; use that data directly. Do NOT use this tool for Fetch Issue Executions; that tool already reads udfjson and enriches it with metadata.testSuiteRun

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool read-only, idempotent, and non-destructive, so the description does not need to restate that. It adds substantial behavioral context beyond annotations: it calls UDF metadata internally, reuses parent rows when sourceRows is provided, resolves LOOKUPLIST values, handles lookup warnings, and describes the hasTcRunUdf false case. This is exactly the kind of behavior an agent needs to predict outcomes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-organized into Parameters, Output Description, Use Cases, Examples, and Hints. There is some redundancy β€” the same 'do not use for test case executions' warning appears in the description, parameter docs, and hints, and default values are repeated from the schema β€” but the repetition serves critical routing decisions and the structure keeps the content scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema, the description fully compensates by listing response fields, explaining the unified table rendering contract, providing two concrete invocation examples, and covering edge cases such as missing UDF metadata and unresolved lookup values. For a tool with seven parameters and conditional workflows, nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, but the description adds important semantics beyond the schema: the critical exact parameter name 'tsrunID', string-or-number acceptance, the conditional requirement based on sourceRows, the sourceRows reuse contract, and viewId auto-resolution. The warnings about which parent rows are and are not valid materially improve correct invocation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Fetch the Test Run UDF values'), a precise resource ('all test case runs in a given test suite run'), and the enrichment provided (field label and type information). It explicitly distinguishes itself from related QMetry tools by warning against use for 'Fetch Test Case Executions' and 'Fetch Issue Executions', so an agent can select it correctly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage guidance is explicit and detailed: use for sourceContext='testSuiteRun', pass sourceRows from the parent tool to avoid a redundant call, and explicitly do NOT use for test case executions or issue executions. The Use Cases and Hints sections further clarify when to invoke this tool versus sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_suite_detailsQMetry: Fetch Test Suite DetailsA
Read-onlyIdempotent
Inspect

Fetch full detail data for a QMetry test suite including UDF field values

Toolset: Test Suites

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • id (number) required: Test Suite ID (numeric ID)

  • scope (string): Scope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon. (default: "project")

Output Description: JSON object with full test suite details including UDFTypeData map and all UDF field values

Use Cases: 1. Get UDF field values for a specific test suite 2. Retrieve full test suite metadata including custom fields 3. Inspect test suite details before updating UDF values

Examples:

  1. Fetch details for test suite with ID 142013

{
  "id": 142013
}

Expected Output: Full test suite detail object with UDFTypeData, UDF field values, and metadata

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest Suite ID (numeric ID)
scopeNoScope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon.project
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds useful behavioral context by specifying that the response includes UDFTypeData and all UDF field values, and it clarifies this is a fetch-only detail operation. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a clear one-sentence summary and organized into useful sections, but the Parameters section duplicates schema information and the Toolset line adds little. It is structured but somewhat longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple fetch operation with three parameters and no output schema, the description is complete: it explains what the tool returns, lists concrete use cases, and provides a practical example with expected output. It lacks only deeper return-structure detail, which would be useful but is not critical for this simple tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers all three parameters with descriptions, defaults, and required status, so schema coverage is 100%. The description's 'Parameters' section mostly repeats the schema rather than adding new semantic meaning, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Fetch'), a clear resource ('full detail data for a QMetry test suite'), and a distinguishing focus ('including UDF field values'). This distinguishes it from sibling list-style tools like qmetry_fetch_test_suites and detail fetch tools for other entity types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section gives clear context for when to invoke the tool: retrieving UDF values, fetching full metadata, and inspecting details before updating UDFs. It does not explicitly name alternatives or exclusion conditions, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_suitesQMetry: Fetch Test SuitesA
Read-onlyIdempotent
Inspect

Fetch QMetry test suites - automatically handles viewId resolution based on project

Toolset: Test Suites

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • viewId (number): ViewId for test suites - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TS.viewId automatically.

  • folderPath (string): Folder path for test suites - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression". (default: "")

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • scope (string): Scope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon. (default: "project")

  • getSubEntities (boolean): Whether to include sub-entities.

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

  • udfFilter (string): User-defined field filter as JSON string (default '[]') (default: "[]")

  • sort (string): Sort Records - refer json schema, Possible property - entityKey, name, testsuiteStatus, linkedPlatformCount, linkedTcCount, createdDate, createdByAlias, updatedDate, updatedByAlias, attachmentCount, owner, remExecutionTime, totalExecutionTime (default: "[{"property":"name","direction":"ASC"}]")

Output Description: JSON object with 'data' array containing test suites and pagination info

Use Cases: 1. List all test suites in a project 2. Search for specific test suites using filters 3. Browse test suites in specific folders 4. Get paginated test suite results

Examples:

  1. Get all test suites from default project - system will auto-fetch viewId

{}

Expected Output: List of test suites from default project with auto-resolved viewId

  1. Get all test suites from UT project - system will auto-fetch UT project's viewId

{
  "projectKey": "UT"
}

Expected Output: List of test suites from UT project using UT's specific TS viewId

  1. Get test suites by release/cycle filter

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"},{\"value\":[111577],\"type\":\"list\",\"field\":\"cycle\"}]"
}

Expected Output: Test suites associated with Release 8.12 (ID: 55178) and Cycle 8.12.1 (ID: 111577)

  1. Get test suites by release only

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[55178],\"type\":\"list\",\"field\":\"release\"}]"
}

Expected Output: All test suites associated with Release 8.12 (ID: 55178)

  1. Get test suites by cycle only

{
  "projectKey": "MAC",
  "filter": "[{\"value\":[111577],\"type\":\"list\",\"field\":\"cycle\"}]"
}

Expected Output: All test suites associated with Cycle 8.12.1 (ID: 111577)

  1. Search for specific test suite by entity key

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-TS-1684\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Test suites matching the entity key criteria

  1. Search for multiple test suites by comma-separated entity keys

{
  "projectKey": "MAC",
  "filter": "[{\"type\":\"string\",\"value\":\"MAC-TS-1684,MAC-TS-1685,MAC-TS-1686\",\"field\":\"entityKeyId\"}]"
}

Expected Output: Test suites matching any of the specified entity keys

Hints: 1. CRITICAL WORKFLOW: Always use the SAME projectKey for both project info and test suite fetching 2. Step 1: If user specifies projectKey (like 'UT', 'MAC'), use that EXACT projectKey for project info 3. Step 2: Get project info using that projectKey, extract latestViews.TS.viewId 4. Step 3: Use the SAME projectKey and the extracted TS viewId for fetching test suites 5. Step 4: If user doesn't specify projectKey, use 'default' for both project info and test suite fetching 6. NEVER mix project keys - if user says 'MAC project', use projectKey='MAC' for everything 7. For search by test suite key (like MAC-TS-1684), use filter: '[{"type":"string","value":"MAC-TS-1684","field":"entityKeyId"}]' 8. RELEASE/CYCLE FILTERING: Use release and cycle IDs, not names, for filtering 9. For release filter: '[{"value":[releaseId],"type":"list","field":"release"}]' 10. For cycle filter: '[{"value":[cycleId],"type":"list","field":"cycle"}]' 11. For combined release+cycle: '[{"value":[releaseId],"type":"list","field":"release"},{"value":[cycleId],"type":"list","field":"cycle"}]' 12. Get release/cycle IDs from FETCH_RELEASES_AND_CYCLES tool before filtering 13. FILTER FIELDS: name, release, cycle, platform, isArchived, testsuiteStatus, createdByAlias, createdDate, entityKeyId, attachmentCount, linkedPlatformCount, linkedTcCount, updatedByAlias, updatedDate, owner, remExecutionTime, and totalExecutionTime 14. SORT FIELDS: entityKey, name, testsuiteStatus, linkedPlatformCount, linkedTcCount, createdDate, createdByAlias, updatedDate, updatedByAlias, attachmentCount, remExecutionTime, and totalExecutionTime 15. For multiple entity keys, use comma-separated values in filter 16. Use empty string '' as folderPath for root directory

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
sortNoSort Records - refer json schema, Possible property - entityKey, name, testsuiteStatus, linkedPlatformCount, linkedTcCount, createdDate, createdByAlias, updatedDate, updatedByAlias, attachmentCount, owner, remExecutionTime, totalExecutionTime[{"property":"name","direction":"ASC"}]
limitNoNumber of records (default 10).
scopeNoScope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon.project
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
viewIdNoViewId for test suites - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TS.viewId automatically.
udfFilterNoUser-defined field filter as JSON string (default '[]')[]
folderPathNoFolder path for test suites - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression".
projectKeyNoProject key - unique identifier for the projectdefault
getSubEntitiesNoWhether to include sub-entities.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior. The description adds substantial beyond-annotation context: automatic viewId resolution, root folder defaulting, release/cycle ID semantics, and output shape with a data array and pagination info. This gives the agent a clear behavioral model without contradicting the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loads the critical auto-viewId behavior, but it is very long and repetitive. Parameter defaults and descriptions are duplicated from the schema, and the 16 numbered hints contain overlapping information, making it less concise than it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 11 optional parameters and no output schema, the description is remarkably complete. It covers automatic viewId resolution, folder behavior, filtering by release/cycle, pagination, sort fields, filter fields, and multiple worked examples with expected outputs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description greatly enriches parameter understanding: it explains exactly how viewId and folderPath are auto-resolved, provides concrete filter JSON patterns, enumerates sort and filter fields, and shows comma-separated entity key examples. This goes far beyond the baseline for schema-covered parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool fetches QMetry test suites and emphasizes automatic viewId resolution, which is a specific, actionable behavior. However, it does not explicitly distinguish itself from closely related sibling tools like qmetry_fetch_test_suite_details or qmetry_fetch_test_suites_for_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear 'Use Cases' list and strong contextual hints, such as fetching release/cycle IDs from FETCH_RELEASES_AND_CYCLES before filtering and using the same projectKey across calls. It lacks explicit exclusions or direct comparisons to sibling tools, so the agent has to infer when other QMetry fetch tools would be more appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_test_suites_for_test_caseQMetry: Fetch Test Suites for Test CaseA
Read-onlyIdempotent
Inspect

Get test suites that can be linked to test cases in QMetry with automatic viewId resolution

Toolset: Test Suites

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tsFolderID (number): Test Suite folder ID. CRITICAL: the parameter name is 'tsFolderID' β€” do NOT use 'testSuiteFolderId', 'folderId', 'folderID', or other variants. Accepts a string or number. Get from project info response β†’ rootFolders.TS.id. Use FETCH_PROJECT_INFO tool first if not provided by user.

  • viewId (number): ViewId for test suite folders - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TSFS.viewId automatically.

  • start (number): Start index for pagination - defaults to 0 (default: 0)

  • page (number): Page number to return (starts from 1) (default: 1)

  • limit (number): Number of records (default 10). (default: 10)

  • getColumns (boolean): Whether to get column information in response. (default: true)

  • filter (string): Filter criteria as JSON string (default '[]') (default: "[]")

Output Description: JSON object with test suites array and pagination metadata

Use Cases: 1. Get test suites available for linking with test cases 2. Find appropriate test suites for test case organization 3. Browse test suites in specific folders for better management 4. Filter test suites by release, cycle, or archive status 5. Organize test execution by grouping test cases into test suites 6. Plan test suite structure for comprehensive test coverage 7. Manage test case categorization for reporting purposes 8. Search for existing test suites before creating new ones 9. Get root test suite folder contents using project info

Examples:

  1. Get test suites from root folder using auto-resolved viewId

{
  "tsFolderID": 113557
}

Expected Output: List of test suites available in the root test suite folder with auto-resolved viewId

  1. Get test suites with custom pagination and auto-resolved viewId

{
  "tsFolderID": 113557,
  "page": 1,
  "limit": 25
}

Expected Output: Paginated list of test suites with 20 items per page

  1. Filter test suites by release with auto-resolved viewId

{
  "tsFolderID": 113557,
  "filter": "[{\"type\":\"list\",\"value\":[55178],\"field\":\"release\"}]"
}

Expected Output: Test suites associated with Release 8.12 (ID: 55178)

  1. Filter test suites by cycle with auto-resolved viewId

{
  "tsFolderID": 113557,
  "filter": "[{\"type\":\"list\",\"value\":[111577],\"field\":\"cycle\"}]"
}

Expected Output: Test suites associated with Cycle 8.12.1 (ID: 111577)

  1. Get only active (non-archived) test suites

{
  "tsFolderID": 113557,
  "filter": "[{\"value\":[0],\"type\":\"list\",\"field\":\"isArchived\"}]"
}

Expected Output: List of active test suites (not archived)

  1. Filter test suites by release and cycle

{
  "tsFolderID": 113557,
  "filter": "[{\"type\":\"list\",\"value\":[55178],\"field\":\"release\"},{\"type\":\"list\",\"value\":[111577],\"field\":\"cycle\"}]"
}

Expected Output: Test suites associated with both Release 8.12 (ID: 55178) and Cycle 8.12.1 (ID: 111577)

  1. Get test suites with column information

{
  "tsFolderID": 113557,
  "getColumns": true
}

Expected Output: Test suites list with detailed column metadata for better interpretation

  1. Search test suites from specific sub-folder with manual viewId

{
  "tsFolderID": 42,
  "viewId": 104316
}

Expected Output: Test suites available in specific folder ID 42 for test case linking

Hints: 1. CRITICAL: tsFolderID is REQUIRED - Test Suite folder ID will be auto-resolved if not provided 2. viewId will be AUTOMATICALLY RESOLVED from project info if not provided 3. HOW TO GET tsFolderID: 4. 1. Call FETCH_PROJECT_INFO tool first to get project configuration 5. 2. From the response, use rootFolders.TS.id for the root test suite folder 6. 3. Example: rootFolders.TS.id = 113557 (MAC project root TS folder) 7. 4. If user doesn't specify tsFolderID, automatically use rootFolders.TS.id from project info 8. VIEWID AUTO-RESOLUTION: 9. 1. System automatically fetches project info using the projectKey 10. 2. Extracts latestViews.TSFS.viewId automatically 11. 3. Example: latestViews.TSFS.viewId = 104316 (MAC project TSFS view) 12. 4. Manual viewId only needed if you want to override the automatic resolution 13. WORKFLOW: System automatically handles project info if tsFolderID or viewId is not provided 14. PROJECT INFO STRUCTURE: rootFolders.TS.id contains the root test suite folder ID 15. PROJECT INFO STRUCTURE: latestViews.TSFS.viewId contains the test suite folder view ID 16. For sub-folders: Use specific folder IDs if you know them, or call folder listing APIs 17. FILTER CAPABILITIES: Same as other QMetry list operations 18. FILTER FIELDS: release, cycle, isArchived, name, status, priority 19. RELEASE/CYCLE FILTERING: Use numeric IDs in list format (get from FETCH_RELEASES_AND_CYCLES) 20. ARCHIVE FILTERING: 0=Active, 1=Archived 21. getColumns=true provides additional metadata for result interpretation 22. Multiple filter conditions are combined with AND logic 23. Pagination supported for large result sets (start, page, limit parameters) 24. This tool helps organize test cases into logical test suites 25. Essential for test execution planning and test case management 26. Use this before creating new test suites to check existing ones

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number to return (starts from 1)
limitNoNumber of records (default 10).
startNoStart index for pagination - defaults to 0
filterNoFilter criteria as JSON string (default '[]')[]
viewIdNoViewId for test suite folders - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TSFS.viewId automatically.
getColumnsNoWhether to get column information in response.
projectKeyNoProject key - unique identifier for the projectdefault
tsFolderIDNoTest Suite folder ID. CRITICAL: the parameter name is 'tsFolderID' β€” do NOT use 'testSuiteFolderId', 'folderId', 'folderID', or other variants. Accepts a string or number. Get from project info response β†’ rootFolders.TS.id. Use FETCH_PROJECT_INFO tool first if not provided by user.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, and the description adds substantial behavioral context: automatic viewId resolution, auto-fetching project info, filter capabilities, AND logic for multiple filters, archive encoding, and pagination behavior. The only slight blemish is the internally confusing statement that tsFolderID is 'REQUIRED' while also being auto-resolved if not provided, which could muddy an agent's understanding of optionality.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is heavily bloated: 26 numbered hints, repeated parameter explanations, and redundant example output statements. The first sentence is concise and effective, but subsequent sections repeat schema information and include overlapping instructions. While organized into sections, many sentences do not earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 8 parameters and no output schema, the description is largely complete: it provides examples for filtering, pagination, auto-resolution, and project info extraction, and describes the output as 'JSON object with test suites array and pagination metadata'. Minor gaps remain, such as the ambiguous 'REQUIRED' vs auto-resolved tsFolderID and a lack of detailed return field descriptions, but overall it equips an agent to call the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description far exceeds the schema by explaining the tsFolderID naming trap, how to derive folder IDs from project info, viewId auto-resolution, filter JSON syntax with concrete examples, archive encoding, and pagination semantics. This adds substantial meaning that the schema alone does not convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get test suites that can be linked to test cases in QMetry', which clearly identifies the operation. It also includes a Toolset label and use cases. However, it does not explicitly name or differentiate from the sibling qmetry_fetch_test_suites, relying on the 'for test case' phrasing to imply distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides contextual guidance such as 'Use this before creating new test suites to check existing ones' and explains the workflow for auto-resolving tsFolderID and viewId. It also lists use cases, but it does not explicitly state when to prefer this tool over qmetry_fetch_test_suites or other alternatives, nor does it specify exclusions. Guidance is present but not decisive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_fetch_udf_layoutQMetry: Fetch UDF LayoutA
Read-onlyIdempotent
Inspect

Fetch UDF (User Defined Field) definitions for Test Case, Test Suite, Issue, or Requirement entities. Returns field names, types, fieldIDs, and lookup option IDs. Call this BEFORE creating or updating an entity with UDF values.

Toolset: UDF

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • entityType (enum) required: Entity type to fetch UDF field definitions for. 'TC' = Test Case (also returns step UDFs in stepFields), 'TS' = Test Suite, 'IS' = Issue, 'RQ' = Requirement.

  • pageName (enum): 'ADD' returns fields for create operations (no fieldID needed). 'DETAIL' returns fields for update operations and includes fieldID (projectUserFieldID) required by the UDF update wrapper. Call with 'DETAIL' before updating an entity's UDF values. (default: "ADD")

Output Description: JSON with 'entityType', 'pageName'. All data is scoped by entityType key in the QMetry newlayout response: TC uses qmUDF.TC / qmSDF.TC / qmDefaultValue.TC; TS uses qmUDF.TS / qmSDF.TS / qmDefaultValue.TS; IS uses qmUDF.IS / qmSDF.IS / qmDefaultValue.IS. 'fields' array β€” UDF fields (name, label, fieldTypeName, fieldID, isMandatory, optional listName); isMandatory=true when allowBlank=false in QMetry. 'systemFields' array β€” system fields like Summary/Priority/Status (name, label, fieldTypeName, isMandatory); isMandatory=true when allowBlank=false. 'defaultValues' object β€” { fieldName: defaultValueId } pre-configured defaults (auto-fill when user omits the field; no need to ask user). TC-only: 'stepFields' β€” step UDF fields; 'stepSystemFields' β€” step system fields (mandatory via tcSteps[].mandatory=true); 'stepDefaultValues' β€” { fieldName: defaultValueId } from qmTCSDefaultValue.TCS. 'listOptions' map (listName β†’ [{id, name, isArchived}]). '_note' with workflow instructions.

Use Cases: 1. Discover UDF fields before creating a Test Case with custom fields 2. Get fieldIDs before updating a Test Suite's UDF values 3. Find valid dropdown option IDs for a LOOKUPLIST UDF before setting a value 4. Identify mandatory UDF fields before creating an Issue 5. List all step-level UDF fields available for Test Case steps 6. Get CASCADINGLIST parent option IDs before fetching child values 7. Discover UDF fields before creating or updating a Requirement

Examples:

  1. Get UDF field definitions for creating a Test Case

{
  "entityType": "TC",
  "pageName": "ADD"
}

Expected Output: { fields: [{ name: 'custom_text', label: 'Custom Text', fieldTypeName: 'STRING', fieldID: null, isMandatory: false }, ...], stepFields: [{ name: 'step_field', label: 'Step Field', fieldTypeName: 'STRING', ... }], listOptions: { myListKey: [{ id: 101, name: 'Option A' }] } }

  1. Get UDF fieldIDs for updating a Test Suite

{
  "entityType": "TS",
  "pageName": "DETAIL"
}

Expected Output: { fields: [{ name: 'dropdown_field', label: 'Dropdown', fieldTypeName: 'LOOKUPLIST', fieldID: 2002, isMandatory: false, listName: 'myListKey' }, ...], listOptions: { myListKey: [{ id: 101, name: 'Option A' }, { id: 102, name: 'Option B' }] } }

  1. Get UDF field definitions for creating an Issue

{
  "entityType": "IS",
  "pageName": "ADD"
}

Expected Output: { fields: [{ name: 'TCR_STR', label: 'String Field', fieldTypeName: 'STRING', fieldID: null }, ...], listOptions: {} }

  1. Get UDF field definitions for creating a Requirement

{
  "entityType": "RQ",
  "pageName": "ADD"
}

Expected Output: { fields: [{ name: 'rq_custom_field', label: 'Custom Field', fieldTypeName: 'STRING', fieldID: null, isMandatory: false }, ...], listOptions: {} }

  1. Get UDF fieldIDs for updating a Requirement

{
  "entityType": "RQ",
  "pageName": "DETAIL"
}

Expected Output: { fields: [{ name: 'rq_dropdown_field', label: 'Dropdown', fieldTypeName: 'LOOKUPLIST', fieldID: 3002, isMandatory: false, listName: 'myListKey' }, ...], listOptions: { myListKey: [{ id: 101, name: 'Option A' }] } }

Hints: 1. CALL THIS TOOL FIRST: Before creating or updating TC/TS/IS/RQ entities with UDF values, always call this tool to discover field names, types, and valid option IDs. 2. pageName='ADD': Use before CREATE operations β€” returns field names + types + list options. fieldID is null (not needed on create). 3. pageName='DETAIL': Use before UPDATE operations β€” returns fieldID (projectUserFieldID) required in the UDF wrapper. 4. WORKFLOW FOR CREATE with UDFs: 5. 1. Call Fetch UDF Layout with entityType + pageName='ADD' 6. 2. For LOOKUPLIST/MULTILOOKUPLIST: pick IDs from listOptions[field.listName] 7. 3. For CASCADINGLIST: pick parent ID from listOptions[field.listName], then call Fetch Cascade Child Values to get child IDs 8. 4. Pass UDF values via 'udfFields' param on the create tool: { fieldName: value } 9. 5. Example: { udfFields: { custom_text: 'value', lookup_field: 101, multi_field: [101, 102], cascade_field: { parent: 201, child: 202 } } } 10. WORKFLOW FOR UPDATE with UDFs: 11. 1. Call Fetch UDF Layout with entityType + pageName='DETAIL' to get fieldIDs 12. 2. For LOOKUPLIST/MULTILOOKUPLIST: pick IDs from listOptions[field.listName] 13. 3. For CASCADINGLIST: pick parent ID, then call Fetch Cascade Child Values for child IDs 14. 4. Pass both 'udfFields' (flat root keys) AND 'UDF' wrapper (with fieldID) on the update tool: 15. udfFields: { custom_text: 'new value', lookup_field: 102 } 16. UDF: { custom_text: { fieldID: 1001, value: 'new value' }, lookup_field: { fieldID: 1002, value: 102 } } 17. 5. For LOOKUPLIST/MULTILOOKUPLIST in update: also set flat alias key β€” e.g. lookup_fieldAlias: 'Option B' 18. stepFields (TC only): Step-level UDF definitions are separate from entity-level UDF fields. They appear in stepFields (not fields). Use them when setting UDF values on test case steps. 19. listOptions: A map of listName β†’ [{id, name, isArchived}]. Use 'id' as the UDF value for LOOKUPLIST/MULTILOOKUPLIST. Only include non-archived options unless user explicitly wants archived items. 20. EMPTY listOptions: If listOptions[field.listName] is missing or empty for a lookup field, the newlayout endpoint did not return those options. This tool automatically attempts a fallback to the UDF metadata endpoint to populate them. If listOptions is STILL empty after this tool returns, call 'Fetch Test Run UDF Metadata' with the same entityType β€” its 'lookupOptions' map uses the same listName keys and contains the full option list. 21. isMandatory: If true, this field MUST be included. Source: allowBlank=false in QMetry API (for UDF/system fields) or mandatory=true (for step system fields). 22. DATETIMEPICKER fields: date value MUST match the project's active date format. 23. Get format: project info β†’ dateTimeFormatID β†’ find in dateTimeFormatNew where id matches β†’ read unique_value. 24. unique_value pattern: yyyy=4-digit year, MM=2-digit month (01-12), dd=2-digit day, MMM=3-letter month (Jan/Feb/...). 25. Re-format user date to match before sending. NEVER guess the format β€” always check project info first. 26. defaultValues: Pre-configured defaults from QMetry. If a mandatory field has a defaultValues entry, use that value automatically without asking the user. Only ask user for mandatory fields with NO default. 27. stepDefaultValues: Same as defaultValues but for test case step fields. 28. PRE-CREATE MANDATORY CHECK WORKFLOW (CRITICAL β€” do this before every create): 29. 1. Call Fetch UDF Layout with pageName='ADD' for the entity type 30. 2. Check systemFields: for each field where isMandatory=true, check if defaultValues[field.name] exists 31. - Has default β†’ use defaultValues[field.name] as the value, no need to ask user 32. - No default β†’ MUST ask user to provide value before creating 33. 3. Check fields (UDF): same logic β€” isMandatory=true + no defaultValues entry β†’ ask user 34. 4. For TC steps: check stepSystemFields and stepFields isMandatory, use stepDefaultValues for auto-fill 35. 5. Only after all mandatory fields are resolved (via default or user input) β†’ proceed with create 36. This tool is scoped per project β€” list options, fieldIDs, and defaults are all project-specific.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNameNo'ADD' returns fields for create operations (no fieldID needed). 'DETAIL' returns fields for update operations and includes fieldID (projectUserFieldID) required by the UDF update wrapper. Call with 'DETAIL' before updating an entity's UDF values.ADD
entityTypeYesEntity type to fetch UDF field definitions for. 'TC' = Test Case (also returns step UDFs in stepFields), 'TS' = Test Suite, 'IS' = Issue, 'RQ' = Requirement.
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark readOnly/idempotent/non-destructive, and the description adds substantial behavior beyond that: pageName-dependent fieldID presence, defaultValues auto-fill behavior, project-scoped data, fallback attempts to the metadata endpoint, mandatory-field semantics, and date-format requirements for DATETIMEPICKER fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core call-to-action and organized with Parameters, Output Description, Use Cases, Examples, and Hints. It is verbose and repeats some workflow guidance (e.g. ADD vs DETAIL semantics appear in the schema, parameters, output notes, and hints), but given the tool's complexity the structure earns most of its length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the full burden of explaining return values, and it does so thoroughly: fields, systemFields, defaultValues, stepFields, listOptions, _note, plus representative examples for each entityType/pageName combination. It also covers downstream usage of the returned IDs/options, making the tool safely callable by an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the parameter section largely mirrors the schema, so the baseline is already high. The description adds useful meaning beyond the schema with project-scoped caveats ('list options, fieldIDs, and defaults are all project-specific') and direct instructions about when ADD/DETAIL fieldIDs are needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Fetch UDF definitions for Test Case, Test Suite, Issue, or Requirement entities,' and states what is returned (field names, types, fieldIDs, lookup option IDs). It also scopes the tool's role relative to create/update flows and related UDF siblings, so the agent can distinguish it from fetch_cascade_child_values and fetch_test_run_udf_metadata.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says 'Call this BEFORE creating or updating an entity with UDF values' and gives separate ADD vs DETAIL workflows for create vs update. It also names alternatives/fallbacks: Fetch Cascade Child Values for cascading lists and Fetch Test Run UDF Metadata when listOptions remain empty.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_import_automation_test_resultsQMetry: Import Automation Test ResultsA
Read-onlyIdempotent
Inspect

Import/Publish automation test results from TestNG, JUnit, Cucumber, Robot, HPUFT, or QAF frameworks into QMetry

Toolset: Automation

Parameters:

  • file (string) required: Base64 encoded file content or file path. User must upload result file (.json, .xml, .zip up to 30 MB)

  • fileName (string) required: Original filename with extension (.json, .xml, or .zip)

  • entityType (enum) required: Format of result file: TESTNG, CUCUMBER, JUNIT, HPUFT, QAF, or ROBOT

  • automationHierarchy (enum): TestNG/JUnit hierarchy: 1=Test Case-Test Step, 2=Test Case only, 3=Test Suite-Test Case. Default: 1

  • testsuiteName (string): Custom test suite name. Ignored if automationHierarchy=3 for JUnit or =2 for ROBOT

  • testsuiteId (string): Reuse existing Test Suite by ID or Entity Key. Ignored if automationHierarchy=3 for JUnit or =2 for ROBOT

  • tsFolderPath (string): Test suite folder path. Creates folder if doesn't exist. Ignored if reusing test suite

  • tcFolderPath (string): Test case folder path. Creates folder if doesn't exist. Ignored if reusing test case

  • platformID (string): Platform ID or Platform Name. Default: 'No Platform'

  • projectID (string): Project ID, Project Key, or Project name. Overrides project in header

  • releaseID (string): Release ID or Release name. Requires projectID if provided

  • cycleID (string): Cycle ID or Cycle name. Requires releaseID and projectID if provided

  • buildID (string): Build ID or Build name

  • testcase_fields (string): JSON string with test case system fields and UDFs. Ignored if reusing test case. Example: {"component":["com1"], "priority":"High"}

  • testsuite_fields (string): JSON string with test suite system fields and UDFs. Ignored if reusing test suite. Example: {"testSuiteState":"Open", "testsuiteOwner":"user"}

  • skipWarning (enum): 0=Fail if summary >255 chars, 1=Truncate summary to 255 chars. Default: 0

  • is_matching_required (string): True=Create new TC if summary/steps don't match, False=Reuse linked TC. Default: True

Use Cases: 1. 1. Import TestNG XML results after CI/CD pipeline execution 2. 2. Publish JUnit test results to QMetry for tracking and reporting 3. 3. Upload Cucumber JSON results with custom test suite organization 4. 4. Import Robot Framework results with specific release/cycle mapping 5. 5. Link automation results to existing test suites for version tracking 6. 6. Create new test suites with custom names and folder structures 7. 7. Map test results to specific platforms (browsers, OS, devices) 8. 8. Associate imported results with releases, cycles, and builds 9. 9. Bulk import multiple test results from ZIP archive 10. 10. Configure test case/suite fields and user-defined fields during import

Examples:

  1. Basic TestNG result import

{
  "file": "<base64_encoded_testng_xml_content>",
  "fileName": "testng-results.xml",
  "entityType": "TESTNG"
}

Expected Output: Auto-generated test suite created with UTC timestamp, test cases auto-linked, execution results updated, 'No Platform' linked

  1. JUnit results with custom test suite name

{
  "file": "<base64_encoded_junit_xml_content>",
  "fileName": "junit-results.xml",
  "entityType": "JUNIT",
  "testsuiteName": "Regression Suite - Build 123",
  "automationHierarchy": "1"
}

Expected Output: Test suite 'Regression Suite - Build 123' created with Test Case-Test Step hierarchy

  1. Cucumber results with platform and release mapping

{
  "file": "<base64_encoded_cucumber_json_content>",
  "fileName": "cucumber-results.json",
  "entityType": "CUCUMBER",
  "platformID": "Chrome 120",
  "releaseID": "Release 2.0",
  "cycleID": "Sprint 15",
  "testsuiteName": "API Automation Tests"
}

Expected Output: Test suite created, linked to Chrome platform, Release 2.0, and Sprint 15 cycle

  1. Reuse existing test suite

{
  "file": "<base64_encoded_testng_xml_content>",
  "fileName": "testng-regression.xml",
  "entityType": "TESTNG",
  "testsuiteId": "PROJ-TS-42"
}

Expected Output: Test cases auto-linked to existing test suite PROJ-TS-42, execution results updated

  1. Import with folder organization

{
  "file": "<base64_encoded_junit_xml_content>",
  "fileName": "junit-results.xml",
  "entityType": "JUNIT",
  "tsFolderPath": "/Automation/Regression",
  "tcFolderPath": "/Automation/API Tests",
  "testsuiteName": "API Regression Suite"
}

Expected Output: Test suite created in '/Automation/Regression' folder, test cases in '/Automation/API Tests' folder

  1. Import with test case custom fields

{
  "file": "<base64_encoded_testng_xml_content>",
  "fileName": "testng-results.xml",
  "entityType": "TESTNG",
  "testcase_fields": "{\"priority\":\"High\",\"testCaseType\":\"Automated\",\"component\":[\"API\",\"Backend\"],\"testcaseOwner\":\"john.doe\",\"estimatedTime\":\"2h:30m:0s\"}"
}

Expected Output: Test cases created with High priority, Automated type, API and Backend components

  1. Import ZIP file with multiple results

{
  "file": "<base64_encoded_zip_content>",
  "fileName": "test-results.zip",
  "entityType": "JUNIT",
  "testsuiteName": "Full Regression Suite",
  "skipWarning": "1"
}

Expected Output: Multiple test results imported from ZIP, summaries truncated if >255 chars

  1. Import with custom hierarchy for JUnit

{
  "file": "<base64_encoded_junit_xml_content>",
  "fileName": "junit-results.xml",
  "entityType": "JUNIT",
  "automationHierarchy": "3",
  "projectID": "PROJ"
}

Expected Output: Multiple test suites created per tag, test cases per tag

  1. Import with build and platform mapping

{
  "file": "<base64_encoded_testng_xml_content>",
  "fileName": "testng-results.xml",
  "entityType": "TESTNG",
  "buildID": "Build-1.2.3",
  "platformID": "Safari 17",
  "releaseID": "Release 1.2",
  "cycleID": "QA Cycle"
}

Expected Output: Results linked to Build 1.2.3, Safari 17 platform, Release 1.2, QA Cycle

  1. Import with test suite and test case fields

{
  "file": "<base64_encoded_cucumber_json_content>",
  "fileName": "cucumber-results.json",
  "entityType": "CUCUMBER",
  "testsuite_fields": "{\"testSuiteState\":\"In Progress\",\"testsuiteOwner\":\"jane.smith\",\"description\":\"Sprint 15 automation results\"}",
  "testcase_fields": "{\"priority\":\"Medium\",\"component\":[\"UI\",\"Frontend\"],\"userDefinedFields\":{\"reviewedDate\":\"11-20-2024\",\"environment\":\"Staging\"}}"
}

Expected Output: Test suite and test cases created with custom fields and UDFs

Hints: 1. 1. CRITICAL: User MUST upload a valid result file before calling this tool 2. 2. USER FILE UPLOAD REQUIRED: Ask user to provide file in chat - system will convert to base64 3. 3. FILE REQUIREMENTS: 4. - Supported extensions: .json, .xml, .zip 5. - Maximum size: 30 MB 6. - ZIP files must contain files matching the specified entityType format 7. 4. REQUIRED PARAMETERS: 8. - file: Base64 encoded content or file path 9. - fileName: Original filename with extension 10. - entityType: TESTNG, CUCUMBER, JUNIT, HPUFT, QAF, or ROBOT 11. 5. ENTITY TYPES: 12. - TESTNG: TestNG XML format 13. - JUNIT: JUnit XML format 14. - CUCUMBER: Cucumber JSON format 15. - ROBOT: Robot Framework XML format 16. - HPUFT: HP UFT format 17. - QAF: QAF format 18. 6. AUTOMATION HIERARCHY (TestNG/JUnit only): 19. - TestNG: 20. * 1 (default): = Test Case, = Test Step 21. * 2: = Test Case only 22. * 3: under = Test Case, = Test Step 23. - JUnit: 24. * 1 (default): = Test Case, = Test Step 25. * 2: = Test Case only 26. * 3: = Test Suite, = Test Case (creates multiple test suites) 27. 7. TEST SUITE OPTIONS: 28. - testsuiteName: Custom name for new test suite 29. - testsuiteId: Reuse existing test suite by ID or Entity Key (e.g., 'PROJ-TS-42') 30. - tsFolderPath: Create test suite in specific folder (e.g., '/Automation/Regression') 31. - Note: testsuiteName/testsuiteId ignored if automationHierarchy=3 for JUnit or =2 for ROBOT 32. 8. TEST CASE OPTIONS: 33. - tcFolderPath: Create test cases in specific folder (e.g., '/Automation/API Tests') 34. - Folders created automatically if they don't exist 35. 9. LINKING OPTIONS: 36. - platformID: Platform ID or name (e.g., 'Chrome 120', 'Safari 17') 37. - projectID: Project ID, key, or name (overrides header project) 38. - releaseID: Release ID or name (requires projectID) 39. - cycleID: Cycle ID or name (requires releaseID and projectID) 40. - buildID: Build ID or name 41. 10. GET IDs FROM OTHER TOOLS: 42. - Platform IDs: Use 'Platform/List' API (FETCH_PLATFORMS tool) 43. - Project IDs: Use 'Project/List' API (FETCH_PROJECTS tool) 44. - Release IDs: Use 'Release/List' API (FETCH_RELEASES_CYCLES tool) 45. - Cycle IDs: Use 'Cycle/List' API (FETCH_RELEASES_CYCLES tool) 46. - Build IDs: Use 'Build/List' API (FETCH_BUILDS tool) 47. - Test Suite IDs: Use 'Testsuite/Fetch' API (FETCH_TEST_SUITES tool) 48. 11. CUSTOM FIELDS (testcase_fields): 49. - JSON string with system fields and UDFs 50. - System fields: component, priority, testCaseState, testCaseType, testcaseOwner, estimatedTime, description 51. - Example: {"component":["API"], "priority":"High", "testcaseOwner":"user"} 52. - Ignored if reusing existing test case 53. 12. CUSTOM FIELDS (testsuite_fields): 54. - JSON string with system fields and UDFs 55. - System fields: testSuiteState, testsuiteOwner, description 56. - Example: {"testSuiteState":"Open", "testsuiteOwner":"user"} 57. - Ignored if reusing existing test suite 58. 13. USER DEFINED FIELDS (UDFs): 59. - Include in testcase_fields or testsuite_fields under 'userDefinedFields' key 60. - Example: {"userDefinedFields": {"reviewedDate": "11-20-2024", "environment": "Staging"}} 61. - UDF types: STRING, LARGETEXT, LOOKUPLIST, MULTILOOKUPLIST, DATEPICKER, NUMBER 62. - See tool metadata for UDF validation rules and auto-create behavior 63. 14. SKIP WARNING OPTIONS: 64. - skipWarning='0' (default): Fail import if test case summary >255 characters 65. - skipWarning='1': Truncate summary to 255 characters and continue import 66. 15. MATCHING BEHAVIOR: 67. - is_matching_required='true' (default): Create new TC/version if summary/steps don't match 68. - is_matching_required='false': Reuse existing TC version if entity key or summary matches 69. 16. IMPORT BEHAVIOR EXAMPLES: 70. - Only file + entityType β†’ Auto-generated test suite, 'No Platform', test cases auto-linked 71. - file + entityType + platformID β†’ Auto-generated test suite with specified platform 72. - file + entityType + testsuiteId β†’ Results updated in existing test suite 73. - file + entityType + platformID + testsuiteId β†’ Results updated in existing test suite with platform 74. 17. FOLDER CREATION: 75. - If tsFolderPath or tcFolderPath specified and doesn't exist, it will be created automatically 76. - Use forward slashes for folder paths (e.g., '/Parent/Child') 77. 18. ESTIMATED TIME FORMAT: 78. - Format: '2h:30m:15s' or '4h' or '7m' or '0s' 79. - Range: 0 to 99999 minutes 80. 19. OWNER FIELDS: 81. - Use userAlias (username) not display name 82. - testcaseOwner: User must have Test Case module rights 83. - testsuiteOwner: User must have Test Suite module rights 84. - Owner not set if user not found or lacks permissions 85. 20. LOOKUPLIST/MULTILOOKUPLIST BEHAVIOR: 86. - If value doesn't exist and auto-create is ON: Value added to list 87. - If value doesn't exist and auto-create is OFF: Field blank or default value 88. - MULTILOOKUPLIST: New values added, old values persist 89. 21. MANDATORY FIELD VALIDATION: 90. - If mandatory system/UDF field missing: 91. * Auto-create OFF + value doesn't exist = Import FAIL 92. * Auto-create ON + value doesn't exist = Import SUCCESS (value created) 93. * Value exists = Import SUCCESS 94. 22. ERROR HANDLING: 95. - Check file size before upload (must be ≀30 MB) 96. - Validate file extension matches entityType 97. - Ensure required dependencies: cycleID requires releaseID and projectID 98. - If import fails, check QMetry UI for detailed error messages 99. 23. WORKFLOW: 100. Step 1: Ask user to upload result file in chat 101. Step 2: System converts file to base64 102. Step 3: Collect entityType and optional parameters 103. Step 4: Call this tool with file data and configuration 104. Step 5: QMetry processes file and creates/updates test artifacts 105. Step 6: Return import results with test suite and execution details 106. 24. USER INTERACTION REQUIRED: 107. - ALWAYS ask user to upload file before calling this tool 108. - Display supported formats: .json, .xml, .zip (up to 30 MB) 109. - Ask for entityType (framework used) 110. - Ask for optional parameters based on user's needs 111. 25. PERFORMANCE TIPS: 112. - For large imports, consider using ZIP files 113. - Reusing existing test suites is faster than creating new ones 114. - Use automationHierarchy wisely to control test case/suite structure

ParametersJSON Schema
NameRequiredDescriptionDefault
fileYesBase64 encoded file content or file path. User must upload result file (.json, .xml, .zip up to 30 MB)
buildIDNoBuild ID or Build name
cycleIDNoCycle ID or Cycle name. Requires releaseID and projectID if provided
fileNameYesOriginal filename with extension (.json, .xml, or .zip)
projectIDNoProject ID, Project Key, or Project name. Overrides project in header
releaseIDNoRelease ID or Release name. Requires projectID if provided
entityTypeYesFormat of result file: TESTNG, CUCUMBER, JUNIT, HPUFT, QAF, or ROBOT
platformIDNoPlatform ID or Platform Name. Default: 'No Platform'
skipWarningNo0=Fail if summary >255 chars, 1=Truncate summary to 255 chars. Default: 0
testsuiteIdNoReuse existing Test Suite by ID or Entity Key. Ignored if automationHierarchy=3 for JUnit or =2 for ROBOT
tcFolderPathNoTest case folder path. Creates folder if doesn't exist. Ignored if reusing test case
tsFolderPathNoTest suite folder path. Creates folder if doesn't exist. Ignored if reusing test suite
testsuiteNameNoCustom test suite name. Ignored if automationHierarchy=3 for JUnit or =2 for ROBOT
testcase_fieldsNoJSON string with test case system fields and UDFs. Ignored if reusing test case. Example: {"component":["com1"], "priority":"High"}
testsuite_fieldsNoJSON string with test suite system fields and UDFs. Ignored if reusing test suite. Example: {"testSuiteState":"Open", "testsuiteOwner":"user"}
automationHierarchyNoTestNG/JUnit hierarchy: 1=Test Case-Test Step, 2=Test Case only, 3=Test Suite-Test Case. Default: 1
is_matching_requiredNoTrue=Create new TC if summary/steps don't match, False=Reuse linked TC. Default: True

TDQS

A3.9/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=true and idempotentHint=true, but the description describes a mutating operation: importing results creates/updates test suites, test cases, and execution results, and mentions folder creation and field updates. This contradicts the read-only annotation, creating an annotation contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely long but well-structured with clear sections (Parameters, Use Cases, Examples, Hints, Workflow). It contains some redundancy (e.g., double numbering, repeated reminders), but all sections provide useful operational context that earns their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 17-parameter tool with no output schema, the description covers file requirements, entity types, hierarchy mappings, optional params, dependencies, field behavior, error handling, workflow, and expected outputs. It is exhaustive and would allow an agent to use the tool without needing external documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description's 'Parameters' and 'Hints' sections add extensive meaning beyond the schema: dependencies (cycleID requires releaseID and projectID), automationHierarchy behavior per framework, custom field formats, lookup-list behavior, error handling, and multiple examples. Even with full schema coverage, it enriches parameter semantics significantly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Import/Publish automation test results from TestNG, JUnit, Cucumber, Robot, HPUFT, or QAF frameworks into QMetry', clearly naming the verb, resource, and scope. It distinguishes this tool from sibling fetch/update/link tools, which all have different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Workflow' section provides a step-by-step process and the 'Use Cases' list specific scenarios (e.g., 'Import TestNG XML results after CI/CD pipeline execution'). However, it does not explicitly name alternative tools for cases not suited to import (e.g., manual test case creation), relying on the implied distinction from siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_requirements_linked_test_cases_to_test_suiteQMetry: Requirements Linked Test Cases to Test SuiteAInspect

Link test cases (including those linked to requirements) to a test suite in QMetry.

Toolset: Test Suites

Parameters:

  • tsID (number) required: Id of Test Suite (required). CRITICAL: parameter name is 'tsID' β€” do NOT use 'testSuiteId', 'testSuiteID', or other variants.

  • tcvdIDs (array) required: Array of Test Case Version IDs (required if fromReqs is true). CRITICAL: parameter name is 'tcvdIDs' β€” do NOT use 'tcVersionIds', 'testCaseVersionIds', or other variants.

  • fromReqs (boolean): Link TestCases from Requirements (optional, default true)

Output Description: JSON object with linkage status and details.

Use Cases: 1. Link requirements linked test cases to a test suite 2. Bulk link multiple requirements linked test cases to a suite 3. Automate test suite composition from requirements linked test cases

Examples:

  1. VT-RQ-18 Requirements Linked test cases to a test suite

{
  "tsID": 8674,
  "tcvdIDs": [
    5448504,
    5448503
  ],
  "fromReqs": true
}

Expected Output: Test cases QTM-TC-32 and QTM-TC-35 linked to test suite 8674.

  1. VT-RQ-19 Requirements Linked test cases to test suites id 1487397

{
  "tsID": 1487397,
  "tcvdIDs": [
    5448504,
    5448503
  ],
  "fromReqs": true
}

Expected Output: Test cases VT-TC-9 and VT-TC-10 linked to test suite 1487397.

  1. VT-RQ-20 Requirements Linked test case to test suite VT-TS-3

{
  "tsID": 1487397,
  "tcvdIDs": [
    5448504,
    5448503,
    5448505,
    5448506
  ],
  "fromReqs": true
}

Expected Output: Test cases VT-TC-4, VT-TC-1, VT-TC-101, and VT-TC-22 linked to test suite VT-TS-3.

Hints: 1. To get the tsID, call the Fetch Test Suites for Test Case API with rootFolderId otherwise if given folderid so use that and from response get the id. 2. To get the requirement linked tcvdIDs by requirement entityKey, call the Fetch Test Cases Linked to Requirement API by to fetch If user provides entityKey (e.g., MAC-RQ-1011), first call FETCH_REQUIREMENTS with filter on entityKeyId to resolve the numeric rqID and get the linked test cases version ids. 3. Set fromReqs to true to link requirements linked test cases instead of direct test case linkage.

ParametersJSON Schema
NameRequiredDescriptionDefault
tsIDYesId of Test Suite (required). CRITICAL: parameter name is 'tsID' β€” do NOT use 'testSuiteId', 'testSuiteID', or other variants.
tcvdIDsYesArray of Test Case Version IDs (required if fromReqs is true). CRITICAL: parameter name is 'tcvdIDs' β€” do NOT use 'tcVersionIds', 'testCaseVersionIds', or other variants.
fromReqsNoLink TestCases from Requirements (optional, default true)

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a non-read, mutating operation; the description reinforces that with 'Link' and gives an expected output shape. It adds some useful context like the default behavior of fromReqs and the prerequisite API calls, but it does not disclose potential side effects, idempotency behavior, or what happens when understanding selected test cases are already linked.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with headers and front-loaded with the first sentence, but it is long. The 'Parameters' section repeats content already present in the input schema, and the three examples are highly redundant, mostly changing only IDs and expected test-case names.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still provides both a general 'JSON object with linkage status and details' and worked examples that show concrete expected outcomes. It also supplies prerequisite APIs and hints, so the agent can compose a successful invocation. It falls slightly short of fully complete because the output object's exact shape and failure modes are not specified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description goes further by explaining how to obtain tsID, how to discover tcvdIDs from a requirement entityKey, and giving concrete example values with expected linked test cases, which materially helps the agent construct valid arguments.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a precise action: 'Link test cases (including those linked to requirements) to a test suite in QMetry.' This clearly identifies the verb, resource, and domain, and the 'requirements linked' framing plus the fromReqs flag distinguishes it from the sibling qmetry_link_test_cases_to_test_suite.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases, practical hints for resolving prerequisite ids, and guidance on when to set fromReqs to true. It does not explicitly name the direct-link sibling as an alternative or state when not to use this tool, but the workflow hints are strong enough for correct selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_set_qmetry_project_infoQMetry: Set QMetry Project InfoA
Idempotent
Inspect

Set current QMetry project for your account

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

Output Description: JSON object containing project configuration details, confirmation of project switch, and available project metadata,including Version, Build Number(if greater than or equal to 1), Project ID, name, Project Key and Date Format

Use Cases: 1. Switch to a specific project before performing test case operations 2. Set project context for batch operations on test cases 3. Configure the default project for the current session 4. Validate access to a specific project before proceeding with operations

Examples:

  1. Set default project as active

{
  "projectKey": "default"
}

Expected Output: Project context set to 'default' with confirmation of project details

  1. Switch to UT project

{
  "projectKey": "UT"
}

Expected Output: Project context switched to 'UT' project with available configurations

  1. Set MAC project as active for test case operations

{
  "projectKey": "MAC"
}

Expected Output: Project context set to 'MAC' with viewIds and folder structure

Hints: 1. Always set the project context before performing test case operations in multi-project environments 2. Use the same project key that you'll use in subsequent test case operations 3. Common project keys include 'default', 'UT', 'MAC', 'VT' - check with your QMetry admin for available projects 4. This operation must be performed before fetching test cases if working with non-default projects 5. The project context persists for the current session until changed again

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already provide idempotentHint=true and destructiveHint=false. The description adds useful behavioral context beyond this: the project context persists for the current session, and the operation must be performed before fetching test cases in non-default projects. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than strictly necessary but is well-structured with Toolset, Parameters, Output Description, Use Cases, Examples, and Hints. Content is front-loaded and each section serves a purpose, though some redundancy exists between examples and hints.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with a single optional parameter and no output schema, the description is complete: it explains the return payload, gives prerequisite context, documents session persistence, and provides examples. An agent has enough information to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single projectKey parameter, so the baseline is 3. The description adds value with concrete examples, common project keys like 'UT', 'MAC', and 'default', and guidance to use the same key as in subsequent test case operations.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Set current QMetry project for your account.' It clearly distinguishes itself from sibling fetch/project-info tools by emphasizing the 'set' action and project-context switching purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Use Cases and Hints explicitly state when to use the tool: before test case operations, for batch operations, for configuring default project, and for validating project access. It lacks explicit 'when not to use' guidance or named alternatives, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_update_cycleQMetry: Update CycleA
Idempotent
Inspect

Update an existing cycle in QMetry for test execution planning

Toolset: Projects

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • cycle (object) required

Output Description: JSON object containing the updated cycle details and confirmation of update

Use Cases: 1. Update cycle name for better organization 2. Modify cycle dates to reflect schedule changes 3. Adjust testing phase timelines within a release 4. Update cycle metadata for sprint tracking 5. Revise milestone dates for test execution planning 6. Rename cycles to match updated sprint naming conventions

Examples:

  1. Update cycle name

{
  "cycle": {
    "name": "Alpha_v1_Updated",
    "buildID": 1494,
    "releaseID": 3729
  }
}

Expected Output: Cycle updated successfully with new name 'Alpha_v1_Updated'

  1. Update cycle dates

{
  "cycle": {
    "startDate": "10-10-2018",
    "targetDate": "11-11-2018",
    "buildID": 1494,
    "releaseID": 3729
  }
}

Expected Output: Cycle dates updated successfully with new start date 10-10-2018 and target date 11-11-2018

  1. Update cycle name and dates together

{
  "cycle": {
    "name": "Sprint 2 - Updated",
    "startDate": "15-01-2024",
    "targetDate": "31-01-2024",
    "buildID": 1494,
    "releaseID": 3729
  }
}

Expected Output: Cycle updated with new name and dates successfully

Hints: 1. CRITICAL: cycle.buildID is REQUIRED - must provide the build ID to identify the cycle to update 2. CRITICAL: cycle.releaseID is REQUIRED - must provide the release ID to identify the cycle to update 3. HOW TO GET buildID and releaseID: 4. 1. Call FETCH_RELEASES_CYCLES tool (API: 'Cycle/List') to get all cycles 5. 2. From the response, get buildID from projects.releases[].builds[].buildID 6. 3. From the response, get releaseID from projects.releases[].releaseID 7. 4. Use those numeric IDs in cycle.buildID and cycle.releaseID parameters 8. Example: Cycle 'Sprint 2' might have buildID: 1494 and releaseID: 3729 9. CRITICAL WORKFLOW - IF USER PROVIDES CYCLE NAME: 10. 1. User says: 'Update cycle Sprint 2 to change dates' 11. 2. You MUST first call FETCH_RELEASES_CYCLES tool to get all cycles 12. 3. Search the response for cycle with matching name 'Sprint 2' 13. 4. Extract buildID and releaseID from the matching cycle 14. 5. Use those IDs in cycle.buildID and cycle.releaseID parameters 15. 6. If cycle name not found, inform user and list available cycles 16. Example workflow: 17. - User request: 'Update cycle Alpha_v1 name to Alpha_v1_Updated' 18. - Step 1: Call FETCH_RELEASES_CYCLES 19. - Step 2: Find cycle where name = 'Alpha_v1', get its buildID (e.g., 1494) and releaseID (e.g., 3729) 20. - Step 3: Call UPDATE_CYCLE with cycle.buildID = 1494 and cycle.releaseID = 3729 21. CYCLE IDENTIFICATION: 22. - NEVER assume or guess buildID or releaseID - always fetch from API 23. - Cycle names are user-defined strings (e.g., 'Sprint 2', 'Alpha_v1', 'Regression Cycle') 24. - buildID and releaseID are numeric identifiers assigned by QMetry 25. - Match cycle names case-insensitively when searching 26. - If multiple cycles match the name, ask user to clarify or use the most recent one 27. - FETCH_RELEASES_CYCLES returns: projects.releases[].builds[] array with name, buildID, and releaseID 28. Date format depends on QMetry instance configuration: DD-MM-YYYY or MM-DD-YYYY 29. Check your QMetry instance settings to determine the correct date format 30. NOTE: To verify/update the Date Format - Go to QMetry -> User Profile 31. If dates are in wrong format, QMetry will return an error - verify format with admin 32. You can update name, startDate, or targetDate independently or together 33. Only include the fields you want to update - other fields will remain unchanged 34. startDate and targetDate help with sprint planning and milestone tracking 35. Cycle hierarchy: Project β†’ Release β†’ Cycle β†’ Test Execution 36. After updating a cycle, you can verify changes using FETCH_RELEASES_CYCLES tool 37. DIFFERENCE FROM CREATE_CYCLE: This tool updates an EXISTING cycle, while CREATE_CYCLE creates a new one

ParametersJSON Schema
NameRequiredDescriptionDefault
cycleYes
projectKeyNoProject key - unique identifier for the projectdefault

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations, the description discloses partial-update behavior ('Only include the fields you want to update - other fields will remain unchanged'), date format dependency, the need to verify via FETCH_RELEASES_CYCLES, and the expected JSON output. These details add meaningful context beyond the idempotentHint and readOnlyHint annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long and somewhat repetitive, but it is well-structured into Parameters, Output, Use Cases, Examples, and Hints, and front-loads the purpose. Some instructions are restated multiple times, but the overall organization makes it easy for an agent to navigate.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has nested required identifiers and a prerequisite lookup workflow, and the description covers all of that: how to get buildID/releaseID, what to do when a cycle name is not found, which fields are optional, date format handling, expected output, and post-update verification. Nothing essential is missing for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Even though schema coverage is listed at 50%, the description fully compensates by documenting buildID and releaseID as required identifiers, explaining how to obtain them, providing real JSON examples, and clarifying optional updateable fields and date formats. This makes the parameters actionable beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Update an existing cycle in QMetry for test execution planning', which clearly identifies the verb, resource, and scope. It also explicitly distinguishes itself from CREATE_CYCLE, making it easy to separate from the sibling qmetry_create_cycle tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance: it explains that buildID and releaseID must be obtained by calling FETCH_RELEASES_CYCLES first, warns never to guess these IDs, and describes a full workflow for resolving a user-supplied cycle name. It also notes the difference from CREATE_CYCLE, covering alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_update_issueQMetry: Update IssueAInspect

Update an existing QMetry issue by DefectId and/or entityKey.

Toolset: Issues

Parameters:

  • DefectId (number) required: ID of the defect/issue to be updated. CRITICAL: the parameter name is 'DefectId' (capital D) β€” do NOT use 'defectId', 'issueId', 'id', or other variants. Accepts a string or number.

  • entityKey (string): Entity Key of the defect/issue to be updated

  • issueType (number): Issue type ID (e.g. Bug, Enhancement, etc.)

  • issuePriority (number): Issue priority ID (e.g. High, Medium, Low, etc.)

  • summary (string): Summary or title of the defect/issue

  • description (string): Detailed description of the defect/issue

  • issueOwner (number): Owner/user ID for the issue

  • affectedRelease (number): Release IDs affected by this issue

  • affectedCycles (number): Cycle IDs affected by this issue

  • udfFields (record<string, union>): Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.

  • UDF (record<string, object>): UDF wrapper required for update. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value.

Output Description: JSON object with update status and details.

Use Cases: 1. Update issue summary (title) 2. Change issue priority, type, or owner 3. Update affected release or cycles 4. Update description or environment 5. Bulk update using DefectId and/or entityKey

Examples:

  1. Update issue summary

{
  "DefectId": 118150,
  "summary": "Money withdrawal is success even if insufficient amount_updated"
}

Expected Output: Issue summary updated successfully.

  1. Update issue priority

{
  "DefectId": 118150,
  "issuePriority": 189340
}

Expected Output: Issue priority updated successfully.

  1. Update issue type

{
  "DefectId": 118150,
  "issueType": 189337
}

Expected Output: Issue type updated successfully.

  1. Update affected release

{
  "DefectId": 118150,
  "affectedRelease": 3730
}

Expected Output: Affected release updated successfully.

Hints: 1. === DEFAULT VALUES β€” APPLY FOR ANY UNSET FIELD === 2. ALWAYS call 'Fetch UDF Layout' with entityType='IS', pageName='DETAIL' before updating. 3. defaultValues (from Fetch UDF Layout, i.e. qmDefaultValue.IS): { fieldName: defaultValueId } β€” sweep ALL entries. 4. defaultValues can contain BOTH system field defaults AND UDF field defaults β€” handle each differently: 5. β†’ SYSTEM field defaults (fields in qmSDF.IS, e.g. component/Labels, issueType, issuePriority): route to TOP-LEVEL params, NOT to udfFields. 6. - MULTILOOKUPLIST system fields (e.g. component): default is a single ID β€” wrap in array: component: [5232632]. 7. - LOOKUPLIST system fields (e.g. issueType, issuePriority): default is a single ID β€” use directly. 8. β†’ UDF field defaults (fields in qmUDF.IS, e.g. str1, lookup19, age19): route to udfFields param. 9. For EACH key in defaultValues: if user did not explicitly provide that field β†’ include it in payload using default, routed correctly. 10. This applies to non-mandatory fields too (e.g. component/Labels). Omitting them = data loss. 11. IMPORTANT: QMetry's API sometimes returns 'defaultValues: {}' (empty) β€” if so, skip auto-apply and ask user for mandatory fields without defaults. 12. systemFields (from qmSDF.IS): isMandatory=true fields must retain a valid value after update. 13. IMPORTANT: QMetry's API sometimes returns 'systemFields: []' (empty). If so, treat summary, issueType, issuePriority as always mandatory. 14. fields/UDF (from qmUDF.IS): isMandatory=true UDF fields must be included if being changed. 15. === END DEFAULT VALUES === 16. 17. === DATE FORMAT CHECK (MANDATORY β€” EVERY UPDATE REQUEST) === 18. ALWAYS call 'Fetch QMetry Project Info' before every update request β€” not only when the user explicitly mentions a date. 19. Any UDF field could be a DATETIMEPICKER. Wrong format causes QMetry to silently discard the field value (API returns success but value is NOT stored β€” no error). 20. STEP 1: From project info, read dateTimeFormatID (e.g. 3). 21. STEP 2: Find entry in dateTimeFormatNew where id === dateTimeFormatID β†’ read its unique_value (e.g. 'yyyy-MM-dd'). 22. STEP 3: unique_value pattern: yyyy=4-digit year, MM=2-digit month (01-12), dd=2-digit day, MMM=3-letter month (Jan/Feb/...). 23. Example: id=1 β†’ MM-dd-yyyy β†’ '10-25-2000' | id=2 β†’ dd-MM-yyyy β†’ '25-10-2000' | id=3 β†’ yyyy-MM-dd β†’ '2000-10-25' | id=4 β†’ dd-MMM-yyyy β†’ '25-Oct-2000' 24. STEP 4: For EVERY DATETIMEPICKER field in the payload: parse any user-provided date and re-format it using the active unique_value pattern before sending. 25. NEVER assume a date format β€” always derive it from dateTimeFormatID. Wrong format = silent data loss. 26. === END DATE FORMAT CHECK === 27. 28. To get the DefectId, call the Issue/Fetch issue tool and use data[].id from the response. 29. if you have pass issue key (VT-IS-5, MAC-IS-10 etc.) then first fetch issue by issue key to get issue id. 30. Along with DefectId, pass only those fields which are to be updated. 31. Refer to the Create Issue tool for valid field mappings and values. 32. You can update summary, priority, type, affectedRelease, affectedCycles, description, sync_with, issueOwner, component, environment, tcRunID, etc. 33. If you provide entityKey, it will be used for additional validation but DefectId is required. 34. 35. UDF (User Defined Fields) WORKFLOW FOR UPDATE: 36. 1. Call 'Fetch UDF Layout' with entityType='IS', pageName='DETAIL' to get field names, fieldIDs (projectUserFieldID), and list option IDs. 37. IF listOptions[field.listName] is empty after Fetch UDF Layout, the tool already tried a metadata fallback. If STILL empty, ask the user to provide the option ID from the QMetry UI β€” do NOT guess numeric IDs. 38. 2. For LOOKUPLIST fields: pick one ID from listOptions[field.listName][].id. 39. 3. For MULTILOOKUPLIST fields: pick array of IDs; also pass alias flat key (e.g., fieldNameAlias: 'Option Label'). 40. 4. For CASCADINGLIST fields: pick parent ID + fetch child with 'Fetch Cascade Child Values'. Pass { parent: parentId, child: childId }. 41. 5. Pass BOTH 'udfFields' (flat root values) AND 'UDF' wrapper (with fieldID) β€” both required for update. 42. Example: udfFields: { is_field: 'value' }, UDF: { is_field: { fieldID: 3001, value: 'value' } } 43. 6. Mandatory UDF fields (isMandatory=true) MUST be included.

ParametersJSON Schema
NameRequiredDescriptionDefault
UDFNoUDF wrapper required for update. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value.
summaryNoSummary or title of the defect/issue
DefectIdYesID of the defect/issue to be updated. CRITICAL: the parameter name is 'DefectId' (capital D) β€” do NOT use 'defectId', 'issueId', 'id', or other variants. Accepts a string or number.
entityKeyNoEntity Key of the defect/issue to be updated
issueTypeNoIssue type ID (e.g. Bug, Enhancement, etc.)
udfFieldsNoFlat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.
issueOwnerNoOwner/user ID for the issue
descriptionNoDetailed description of the defect/issue
issuePriorityNoIssue priority ID (e.g. High, Medium, Low, etc.)
affectedCyclesNoCycle IDs affected by this issue
affectedReleaseNoRelease IDs affected by this issue

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations by warning about silent data loss from incorrect date formats, the requirement to apply default values to avoid data loss, the API returning success even when values are not stored, and the need to include both udfFields and UDF during update. This is highly valuable behavioral context for an agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very long and duplicates some parameter/schema content, which slightly hurts conciseness. However, it is extremely well structured with separate Parameters, Use Cases, Examples, and Hints sections, numbered instructions, and examples that make the length easier to navigate.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex 11-parameter mutation tool with nested UDF objects and no output schema, the description is comprehensive. It covers mandatory-field fallbacks, default-value handling, date format derivation, UDF workflow, required companion calls, and even the expected output format. This is sufficient context for successful invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description still adds meaningful parameter context: the exact DefectId casing requirement, UDF field type mappings, default routing for system fields versus UDF fields, date format rules, and the required UDF wrapper structure. It far exceeds what the schema alone provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific operation: 'Update an existing QMetry issue by DefectId and/or entityKey.' It clearly identifies the resource and the required identifiers, and it is easily distinguishable from sibling tools like qmetry_create_defect_or_issue or qmetry_fetch_issue_details.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides use cases, prerequisites, and references to supporting tools such as 'Fetch UDF Layout', 'Fetch QMetry Project Info', 'Issue/Fetch issue', and 'Create Issue tool'. It gives clear context for when to use the tool, though it does not explicitly state when not to use it or contrast with the create tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_update_requirementQMetry: Update RequirementAInspect

Update an existing QMetry requirement by rqId and rqVersionId.

Toolset: Requirements

Parameters:

  • rqId (number) required: Requirement numeric ID (required). This is the internal numeric identifier, not the entity key like 'MAC-RQ-730'.

  • rqVersionId (number) required: Requirement version ID (required for update).

  • updateWithVersion (boolean): Pass 'true' to create a new version of the requirement instead of updating the existing version in place.

  • name (string): Name of the requirement.

  • description (string): Description of the requirement.

  • component (array): Component (Label) IDs associated with the requirement.

  • requirementOwner (number): Owner ID of the requirement.

  • requirementState (number): State ID of the requirement.

  • priority (number): Priority ID of the requirement.

  • attachments (object): Attachment changes. ADD and REMOVE are both required arrays (use empty arrays for no change).

  • udfFields (record<string, union>): Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.

  • UDF (record<string, object>): UDF wrapper required for update operations. Keys = UDF field names. Each value must include fieldID and value. Also set matching flat key in udfFields for the LOOKUPLIST Alias display. Example: { custom_text: { fieldID: 1001, value: 'new value' } }

Output Description: JSON object containing the updated requirement ID, summary, and update metadata.

Use Cases: 1. Update the name, description, priority, owner, state, or component of a requirement 2. Add or remove attachments on a requirement 3. Update custom field (UDF) values on a requirement 4. Create a new version of a requirement while updating it (updateWithVersion)

Examples:

  1. Update the priority of a requirement

{
  "rqId": 2073,
  "rqVersionId": 2087,
  "updateWithVersion": false,
  "priority": 688865
}

Expected Output: Requirement priority updated successfully.

  1. Update name, description, owner, and state

{
  "rqId": 2073,
  "rqVersionId": 2087,
  "name": "Updated login requirement",
  "description": "Users must be able to log in with SSO.",
  "requirementOwner": 8,
  "requirementState": 688912,
  "component": [
    689030
  ]
}

Expected Output: Requirement updated with new name, description, owner, state, and component.

Hints: 1. If the user provides an entityKey (e.g., MAC-RQ-730), first call 'Fetch Requirements' with a filter on entityKeyId to resolve rqId and rqVersionId. 2. updateWithVersion: pass 'true' to create a new version of the requirement instead of updating the current version in place. 3. To get valid values for priority, requirementOwner, requirementState, and component, call the 'Admin/Get info Service' API (FETCH_PROJECT_INFO tool) and use the returned customListObjs IDs. 4. If the user provides a name instead of an ID for owner/state/priority, fetch project info, find the matching entry by name in the relevant customListObjs list, and use its ID. If not found, skip that field and tell the user it was omitted because the value wasn't available in the current project. 5. attachments (if used) requires both ADD and REMOVE arrays β€” pass empty arrays when there is nothing to add or remove. 6. Only fields explicitly listed in this tool's parameters are supported β€” releaseCycleMapping and associateRelCyc are create-only and not supported here. 7. HARD GATE: before updating, this tool checks project info for isExtTrackerConfigured, extTrackerType, and isRQConfigured. If the project has an external tracker configured (isExtTrackerConfigured=true) and isRQConfigured=true, and the tracker is Jira (extTrackerType=1) or Azure (extTrackerType=3), the update is refused with an error β€” do NOT retry or work around this. Tell the user requirements for this project must be updated directly in the external system (Jira or Azure), then synced into QMetry. 8. 9. UDF (User Defined Fields) WORKFLOW FOR UPDATE: 10. 1. Call 'Fetch UDF Layout' with entityType='RQ', pageName='DETAIL' to get field names, fieldIDs (projectUserFieldID), and list option IDs. 11. IF listOptions[field.listName] is empty after Fetch UDF Layout, the tool already tried a metadata fallback. If STILL empty, ask the user to provide the option ID from the QMetry UI β€” do NOT guess numeric IDs. 12. 2. For LOOKUPLIST fields: pick one ID from listOptions[field.listName][].id. 13. 3. For MULTILOOKUPLIST fields: pick array of IDs; also pass alias flat key (e.g., fieldNameAlias: 'Option Label'). 14. 4. For CASCADINGLIST fields: pick parent ID + fetch child with 'Fetch Cascade Child Values'. Pass { parent: parentId, child: childId }. 15. 5. Pass BOTH 'udfFields' (flat root values) AND 'UDF' wrapper (with fieldID) β€” both required for update. Example: udfFields: { rq_field: 'value' }, UDF: { rq_field: { fieldID: 2001, value: 'value' } } 16. 6. Mandatory UDF fields (isMandatory=true) MUST be included.

ParametersJSON Schema
NameRequiredDescriptionDefault
UDFNoUDF wrapper required for update operations. Keys = UDF field names. Each value must include fieldID and value. Also set matching flat key in udfFields for the LOOKUPLIST Alias display. Example: { custom_text: { fieldID: 1001, value: 'new value' } }
nameNoName of the requirement.
rqIdYesRequirement numeric ID (required). This is the internal numeric identifier, not the entity key like 'MAC-RQ-730'.
priorityNoPriority ID of the requirement.
componentNoComponent (Label) IDs associated with the requirement.
udfFieldsNoFlat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.
attachmentsNoAttachment changes. ADD and REMOVE are both required arrays (use empty arrays for no change).
descriptionNoDescription of the requirement.
rqVersionIdYesRequirement version ID (required for update).
requirementOwnerNoOwner ID of the requirement.
requirementStateNoState ID of the requirement.
updateWithVersionNoPass 'true' to create a new version of the requirement instead of updating the existing version in place.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations convey only low-level flags, so the description carries the behavioral burden and does so well. It discloses updateWithVersion as in-place vs new-version behavior, explains that UDF updates require BOTH udfFields and the UDF wrapper, requires ADD/REMOVE arrays for attachments, and reveals the HARD GATE that blocks updates for externally tracked projects with a clear do-not-retry instruction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded and organized into sections, which helps. However, the Parameters list largely repeats a 100%-covered schema, and the Hints section has a numbering break: item 8 is blank, item 9 introduces a workflow, and item 10 restarts numbered sub-items. The bloat and malformed numbering make it heavier to parse than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 12 parameters, nested objects, and no output schema, the definition is unusually complete: examples, ID-resolution workflow, valid-value sources, the external-tracker hard gate, and UDF edge cases are all present. Minor gaps remainβ€”the exact shape of attachment ADD/REMOVE entries and the output object are only loosely describedβ€”so it stops short of perfect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds substantial meaning beyond the schema: rqId is disambiguated from entity keys, concrete examples show valid values, and the UDF workflow explains fieldID discovery, list-option lookup, mandatory UDFs, cascade child fetching, and the dual udfFields/UDF requirement. This is far more than a restatement of the JSON Schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names the verb (update), the object (an existing QMetry requirement), and the required identifiers (rqId, rqVersionId), making it immediately distinguishable from qmetry_create_requirement and qmetry_fetch_requirements. It also sharpens the semantics by clarifying that rqId is the internal numeric identifier, not an entity key like 'MAC-RQ-730'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section explicitly states what the tool is for: field updates, attachment changes, UDF updates, and creating a new version. Hints 1, 6, and 7 provide strong when-not-to-use guidance, including resolving entity keys via Fetch Requirements, rejecting create-only fields like releaseCycleMapping, and refusing updates when an external Jira/Azure tracker is configured.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_update_test_caseQMetry: Update Test CaseAInspect

Update an existing QMetry test case OR create a new version by tcID and tcVersionID, with auto-resolution from entityKey.

Toolset: Test Cases

Parameters:

  • projectKey (string): Project key - unique identifier for the project (default: "default")

  • tcID (number) required: Test Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.

  • tcVersionID (number) required: Test Case version number. This is the internal numeric identifier for the version.

  • tcVersion (number): Test Case version number (required when withVersion=true for creating new version). This is the current version number from which a new version will be created.

  • withVersion (boolean): Pass 'true' if you want to create a new version of the test case with incremented version number. When true, a new version is created (e.g., if current version is 2, new version 3 is created). When false or omitted, updates the existing version specified by tcVersionID. IMPORTANT: Always send proper tcVersionID to identify which version the request is for.

  • versionComment (string): Comment or description for the new version (used only when withVersion=true). Helps track what changed in this new version. Example: 'Updated test steps for new requirements'

  • notruncurrent (boolean): Flag to control execution behavior for current version when creating a new version. Used in conjunction with withVersion=true.

  • notrunall (boolean): Flag to control execution behavior for all versions when creating a new version. Used in conjunction with withVersion=true.

  • folderPath (string): Folder path for test suites - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression". (default: "")

  • scope (string): Scope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon. (default: "project")

  • isStepUpdated (boolean): Set to true when steps are being added, updated, or removed. Required when including 'steps' or 'removeSteps' arrays.

  • steps (array)

  • removeSteps (array)

  • name (string)

  • priority (number)

  • component (array)

  • owner (number)

  • testCaseState (number)

  • testCaseType (number)

  • estimatedTime (number): Estimated execution time in seconds. Example: 7200 for 2 hours

  • executionMinutes (number)

  • testingType (number)

  • description (string)

  • updateOnlyMetadata (boolean): Set to true to update only metadata fields without touching test steps. When true, steps and removeSteps are ignored.

  • udfFields (record<string, union>): Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.

  • UDF (record<string, object>): UDF wrapper required for update operations. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value. Also set matching flat key in udfFields for the LOOKUPLIST Alias display. Example: { custom_text: { fieldID: 1001, value: 'new value' } }

Output Description: JSON object containing the test case ID, version ID, summary, update/creation metadata. When withVersion=true (version creation), response includes new version number and version ID. When withVersion=false/omitted (existing version update), response includes updated fields confirmation.

Use Cases: 1. Update test case summary (name) 2. Change priority, owner, or state of a test case 3. Edit, add, or remove test steps 4. Update only metadata (no steps) 5. Create a new version of a test case (withVersion=true) 6. Update a specific version of a test case (without withVersion flag) 7. Bulk update using entityKey auto-resolution 8. Modify test case description or estimated time 9. Change test case type or component 10. Update testing type or custom fields 11. Update, add and remove test case steps 12. Version control for test case evolution tracking

Examples:

  1. Update test case summary (existing version update)

{
  "tcID": 4519260,
  "tcVersionID": 5448492,
  "name": "MAC Test11"
}

Expected Output: Test case summary updated. tcID and tcVersionID auto-resolved from entityKey. Only 'name' field changed. Version remains the same.

  1. Create NEW VERSION with updated summary and description

{
  "tcID": 4572654,
  "tcVersionID": 5514384,
  "tcVersion": 1,
  "name": "Add two numbers 2 v2",
  "description": "Test Description version 2",
  "withVersion": true,
  "versionComment": "version 2 comment add",
  "notruncurrent": true,
  "notrunall": true
}

Expected Output: New version created (version 2). Test case now has incremental version with updated summary and description. Original version 1 remains unchanged.

  1. Create NEW VERSION with all metadata fields (release, cycle, priority, owner, etc.)

{
  "tcID": 4572654,
  "tcVersionID": 5514384,
  "tcVersion": 1,
  "name": "Facebook Login Validation Failed update from MCP V2",
  "description": "Existing description V2",
  "priority": 2355751,
  "testcaseOwner": 6963,
  "testCaseState": 2355753,
  "testCaseType": 2355762,
  "estimatedTime": 7200,
  "withVersion": true,
  "versionComment": "Created version 2 with updated metadata",
  "notruncurrent": true,
  "notrunall": true,
  "folderPath": 602290,
  "scope": "project"
}

Expected Output: New test case version 2 created with updated summary, description, priority (High), owner (umang.savaliya), state, type, and estimated time (2 hours). Version comment added for tracking.

  1. Update EXISTING VERSION 2 (not creating new version)

{
  "tcID": 4572654,
  "tcVersionID": 5514385,
  "name": "Updated version 2 name",
  "priority": 2355752
}

Expected Output: Version 2 updated with new name and priority. No new version created because withVersion flag is not set. This is a normal update of existing version.

  1. Update priority to High and owner to john.doe (existing version)

{
  "tcID": 4519260,
  "tcVersionID": 5448492,
  "priority": 505015,
  "testcaseOwner": 6963
}

Expected Output: Priority and owner updated. Field IDs auto-resolved from project info. tcID/tcVersionID resolved from entityKey. Existing version modified.

  1. Update steps (edit, add, remove) - existing version

{
  "tcID": 4519260,
  "tcVersionID": 5448492,
  "steps": [
    {
      "orderId": 1,
      "description": "Step 22",
      "inputData": "Input 22",
      "expectedOutcome": "Outcome 22",
      "tcStepID": 3014032
    },
    {
      "orderId": 2,
      "description": "Step3",
      "inputData": "Input 3",
      "expectedOutcome": "Outcome 3"
    }
  ],
  "removeSteps": [
    {
      "tcStepID": 3014031,
      "description": "Step 1",
      "orderId": 1
    }
  ],
  "isStepUpdated": true
}

Expected Output: Steps updated: Step 22 edited (tcStepID preserved), Step3 added (no tcStepID), Step 1 removed. tcID/tcVersionID auto-resolved. Existing version modified.

  1. Create NEW VERSION with updated steps

{
  "tcID": 4572654,
  "tcVersionID": 5514384,
  "tcVersion": 1,
  "name": "Add two numbers 2 v2",
  "steps": [
    {
      "orderId": 1,
      "description": "I and u have a calculator",
      "inputData": "",
      "expectedOutcome": "",
      "tcStepID": 38001791
    },
    {
      "orderId": 2,
      "description": "I add 41 and 31",
      "inputData": "",
      "expectedOutcome": "",
      "tcStepID": 38001793
    },
    {
      "orderId": 3,
      "description": "the result should be 72",
      "inputData": "",
      "expectedOutcome": "",
      "tcStepID": 38001792
    }
  ],
  "withVersion": true,
  "versionComment": "version 2 with preserved steps",
  "notruncurrent": true,
  "notrunall": true,
  "isStepUpdated": true
}

Expected Output: New version 2 created with all steps from version 1 preserved. Steps carry forward with their tcStepID values. Version comment added for tracking.

  1. Update only metadata (no steps) - existing version

{
  "tcID": 4519260,
  "tcVersionID": 5448492,
  "updateOnlyMetadata": true,
  "name": "New Name"
}

Expected Output: Metadata updated only. Steps unchanged. tcID/tcVersionID auto-resolved. Existing version modified.

  1. Create NEW VERSION from existing version 2 with updated steps (working payload for linked test cases)

{
  "tcID": 4594145,
  "tcVersionID": 5536706,
  "tcVersion": 2,
  "name": "Mock Test Case - E-commerce Checkout Flow - v3",
  "steps": [
    {
      "orderId": 1,
      "description": "Open browser and navigate to e-commerce website",
      "expectedOutcome": "Homepage loads successfully with product catalog",
      "inputData": "URL: https://example-shop.com",
      "tcStepID": 38129471
    },
    {
      "orderId": 2,
      "description": "Search for product",
      "expectedOutcome": "Search results display relevant products",
      "inputData": "Search term: 'wireless headphones'",
      "tcStepID": 38129475
    },
    {
      "orderId": 3,
      "description": "Select product and add to cart",
      "expectedOutcome": "Product added to cart, cart counter increments",
      "inputData": "Click 'Add to Cart' button",
      "tcStepID": 38129472
    },
    {
      "orderId": 4,
      "description": "Proceed to checkout",
      "expectedOutcome": "Checkout page displays with cart summary",
      "inputData": "Click cart icon and 'Proceed to Checkout'",
      "tcStepID": 38129473
    },
    {
      "orderId": 5,
      "description": "Complete payment",
      "expectedOutcome": "Order confirmation page displayed",
      "inputData": "Fill payment details and submit",
      "tcStepID": 38129474
    },
    {
      "orderId": 6,
      "description": "Verify order confirmation email received",
      "expectedOutcome": "Email with order details received in inbox",
      "inputData": "Check email account for confirmation"
    },
    {
      "orderId": 7,
      "description": "Check order status in account dashboard",
      "expectedOutcome": "Order status shows as 'Processing' with tracking information",
      "inputData": "Navigate to My Orders section"
    }
  ],
  "withVersion": true,
  "versionComment": "Created version 3: Added 2 new verification steps (email and order status check)",
  "notrunall": false,
  "notruncurrent": false,
  "scope": "project"
}

Expected Output: New version 3 created successfully from version 2. Test case now has 7 steps (5 preserved + 2 new). Key: tcVersion=2 was used because version 2 already existed in system. notrunall and notruncurrent both false (not true). Result shows tcVersion: 3 in response with new tcVersionID.

Hints: 1. === DEFAULT VALUES β€” APPLY FOR ANY UNSET FIELD === 2. Call 'Fetch UDF Layout' with entityType='TC', pageName='DETAIL' before updating. 3. defaultValues (from Fetch UDF Layout): { fieldName: defaultValueId } β€” sweep ALL entries. 4. For EACH key in defaultValues: if user did not explicitly provide that field β†’ include it in payload with the default value. 5. This applies to non-mandatory fields too (e.g. priority, estimatedTime). Omitting them = data loss. 6. === END DEFAULT VALUES === 7. 8. === DATE FORMAT CHECK (MANDATORY β€” EVERY UPDATE REQUEST) === 9. ALWAYS call 'Fetch QMetry Project Info' before every update request β€” not only when the user explicitly mentions a date. 10. Any UDF field could be a DATETIMEPICKER. Wrong format causes QMetry to silently discard the field value (API returns success but value is NOT stored β€” no error). 11. STEP 1: From project info, read dateTimeFormatID (e.g. 3). 12. STEP 2: Find entry in dateTimeFormatNew where id === dateTimeFormatID β†’ read its unique_value (e.g. 'yyyy-MM-dd'). 13. STEP 3: unique_value pattern: yyyy=4-digit year, MM=2-digit month (01-12), dd=2-digit day, MMM=3-letter month (Jan/Feb/...). 14. Example: id=1 β†’ MM-dd-yyyy β†’ '10-25-2000' | id=2 β†’ dd-MM-yyyy β†’ '25-10-2000' | id=3 β†’ yyyy-MM-dd β†’ '2000-10-25' | id=4 β†’ dd-MMM-yyyy β†’ '25-Oct-2000' 15. STEP 4: For EVERY DATETIMEPICKER field in the payload: parse any user-provided date and re-format it using the active unique_value pattern before sending. 16. NEVER assume a date format β€” always derive it from dateTimeFormatID. Wrong format = silent data loss. 17. === END DATE FORMAT CHECK === 18. 19. CRITICAL - VERSION CREATION vs UPDATE DISTINCTION: 20. This tool supports TWO MODES using the SAME API endpoint: 21. 22. MODE 1: CREATE NEW VERSION (withVersion=true) 23. - Purpose: Create an incremental version of the test case (e.g., v1 β†’ v2, v2 β†’ v3) 24. - When to use: User explicitly asks to 'create new version', 'create version 2', 'increment version' 25. - Required fields: tcID, tcVersionID (of source version), tcVersion (current version number), withVersion=true 26. - Optional but recommended: versionComment (track what changed), notruncurrent, notrunall 27. - Behavior: Creates a NEW test case version with incremented version number. Source version remains unchanged. 28. - Example: If current version is 1, setting withVersion=true creates version 2 29. - Use cases: Updating test case for new requirements, creating variants for different scenarios, version control 30. 31. MODE 2: UPDATE EXISTING VERSION (withVersion=false or omitted) 32. - Purpose: Modify fields of an EXISTING version without creating a new version 33. - When to use: User asks to 'update test case', 'modify version X', 'change summary' (without mentioning new version) 34. - Required fields: tcID, tcVersionID (of version to update) 35. - Do NOT include: withVersion flag, versionComment, tcVersion 36. - Behavior: Updates the specified version in-place. No new version is created. 37. - Example: Updating version 2's summary - only version 2 is modified, no version 3 is created 38. - Use cases: Fixing typos, updating metadata, modifying steps in existing version 39. 40. CRITICAL FIELD UNDERSTANDING: 41. - tcVersionID: The VERSION ID (numeric identifier) of the version you're working with 42. - tcVersion: The VERSION NUMBER (1, 2, 3, etc.) - only needed when withVersion=true 43. - tcID: The TEST CASE ID (remains same across all versions) 44. - Example: Test case VKMCP-TC-10 (tcID: 4572654) has version 1 (tcVersionID: 5514384, tcVersion: 1) 45. - When creating version 2 from version 1: Send tcVersionID=5514384 (source), tcVersion=1 (current), withVersion=true 46. 47. HOW TO DETERMINE WHICH MODE: 48. - User says 'create new version' β†’ MODE 1 (withVersion=true) 49. - User says 'create version 2' β†’ MODE 1 (withVersion=true) 50. - User says 'update test case with new version' β†’ MODE 1 (withVersion=true) 51. - User says 'update test case VKMCP-TC-10 summary' β†’ MODE 2 (no withVersion, update existing version) 52. - User says 'update version 2 summary' β†’ MODE 2 (no withVersion, update existing version 2) 53. - User says 'change priority of version 1' β†’ MODE 2 (no withVersion, update version 1) 54. - If ambiguous, ask user: 'Do you want to create a new version or update the existing version?' 55. 56. VERSION CREATION WORKFLOW (withVersion=true): 57. Step 1: Fetch test case details to get current tcID, tcVersionID, and tcVersion 58. Step 2: Optionally fetch current steps if they need to be preserved/modified 59. Step 3: Prepare payload with: 60. - tcID (test case ID) 61. - tcVersionID (source version ID to create from) 62. - tcVersion (current version number) 63. - withVersion: true (CRITICAL flag) 64. - versionComment (recommended: describe what changed) 65. - Updated fields (name, description, priority, steps, etc.) 66. - notruncurrent: true (recommended) 67. - notrunall: true (recommended) 68. Step 4: Call update API - a new version will be created with incremented version number 69. Step 5: New version inherits all fields from source version, with your specified updates applied 70. 71. EXISTING VERSION UPDATE WORKFLOW (no withVersion): 72. Step 1: Fetch test case details to get tcID and tcVersionID of the version to update 73. Step 2: Prepare payload with: 74. - tcID (test case ID) 75. - tcVersionID (version ID to update) 76. - DO NOT include withVersion, versionComment, or tcVersion 77. - Only include fields you want to change 78. Step 3: Call update API - specified version is updated in-place 79. Step 4: No new version is created, only specified fields are modified 80. 81. FIELD MAPPING FOR VERSION CREATION: 82. When creating a new version, include ALL fields you want the new version to have: 83. - name: Test case summary (required if different from source) 84. - description: Test case description (required if different from source) 85. - priority: Priority ID (get from project info customListObjs.priority[index].id) 86. - testcaseOwner: Owner ID (get from project info customListObjs.owner[index].id) 87. - testCaseState: State ID (get from project info customListObjs.testCaseState[index].id) 88. - testCaseType: Type ID (get from project info customListObjs.testCaseType[index].id) 89. - testingType: Testing type ID (get from project info customListObjs.testingType[index].id) 90. - component: Array of component IDs (get from project info customListObjs.component[index].id) 91. - estimatedTime: Time in seconds (e.g., 7200 for 2 hours) 92. - steps: Array of step objects (include tcStepID from source version to preserve steps) 93. - folderPath: Folder path or folder ID 94. - scope: Usually 'project' 95. 96. STEPS HANDLING IN VERSION CREATION: 97. When creating a new version WITH steps: 98. - To PRESERVE existing steps: Include them with their original tcStepID values 99. - To ADD new steps: Include them WITHOUT tcStepID 100. - To MODIFY steps: Include them with tcStepID and updated description/data 101. - To REMOVE steps: Include them in removeSteps array 102. - Set isStepUpdated: true if any steps are modified, added, or removed 103. - If no steps are included, new version may inherit steps from source (verify with QMetry docs) 104. 105. If user provides entityKey (e.g., MAC-TC-1684), first call FETCH_TEST_CASES with a filter on entityKeyId to resolve the tcID and tcVersionID. 106. To get valid values for priority, owner, component, etc., call the project info tool and use the returned customListObjs IDs. 107. If the user provides a priority name (e.g. 'Blocker'), fetch project info, find the matching priority in customListObjs.priority[index].name, and use its ID in the payload. If the name is not found, skip the priority field (it is not required) and show a user-friendly message: 'Test case updated without priority, as given priority is not available in the current project.' 108. If the user provides a component name, fetch project info, find the matching component in customListObjs.component[index].name, and use its ID in the payload. If the name is not found, skip the component field (it is not required) and show a user-friendly message: 'Test case updated without component, as given component is not available in the current project.' 109. If the user provides an owner name, fetch project info, find the matching owner in customListObjs.owner[index].name, and use its ID in the payload as testcaseOwner. If the name is not found, skip the testcaseOwner field (it is not required) and show a user-friendly message: 'Test case updated without owner, as given owner is not available in the current project.' 110. If the user provides a test case state name, fetch project info, find the matching state in customListObjs.testCaseState[index].name, and use its ID in the payload as testCaseState. If the name is not found, skip the testCaseState field (it is not required) and show a user-friendly message: 'Test case updated without test case state, as given state is not available in the current project.' 111. If the user provides a test case type name, fetch project info, find the matching type in customListObjs.testCaseType[index].name, and use its ID in the payload as testCaseType. If the name is not found, skip the testCaseType field (it is not required) and show a user-friendly message: 'Test case updated without test case type, as given type is not available in the current project.' 112. If the user provides a testing type name, fetch project info, find the matching type in customListObjs.testingType[index].name, and use its ID in the payload as testingType. If the name is not found, skip the testingType field (it is not required) and show a user-friendly message: 'Test case updated without testing type, as given testing type is not available in the current project.' 113. Example: If user says 'Update test case with title "High priority test case" and set priority to "Blocker"', first call project info, map 'Blocker' to its ID, and use that ID for the priority field in the update payload. If user says 'set priority to "Urgent"' and 'Urgent' is not found, skip the priority field and show: 'Test case updated without priority, as given priority is not available in the current project.' 114. CRITICAL: To update test case steps without Duplication, use the following rules: 115. - ANTI-DUPLICATION RULE: The tcStepID field is THE KEY to prevent duplication: 116. * WITH tcStepID = UPDATE existing step (QMetry modifies the existing step in place) 117. * WITHOUT tcStepID = CREATE new step (QMetry adds a brand new step) 118. - For steps to be UPDATED: ALWAYS fetch existing steps first using FETCH_TEST_CASE_STEPS, then include the tcStepID in the step object. 119. - For steps to be ADDED: omit tcStepID completely in the step object. 120. - For steps to be REMOVED: add a full removeSteps object for each step to be deleted, matching the removeTestCaseStep interface. 121. - CRITICAL WARNING - DO NOT ADD UNSOLICITED STEPS: 122. * ONLY add, edit, or remove steps that the user EXPLICITLY requested 123. * DO NOT invent, create, or add extra steps based on assumptions or best practices 124. * DO NOT add 'helpful' steps that the user did not ask for 125. * When user says 'remove step 1', the result should have (N-1) steps, not N steps with extras 126. * When user says 'add 1 step', ONLY add that 1 step, nothing more 127. * When user says 'update step 2', ONLY update step 2, do not add or modify other steps 128. * If unsure what user wants, ASK first rather than adding steps autonomously 129. - WORKFLOW TO AVOID DUPLICATION: 130. 1. Call FETCH_TEST_CASE_STEPS to get all existing steps with their tcStepID values 131. 2. For steps you want to KEEP/UPDATE: Include them in steps[] WITH their original tcStepID 132. 3. For steps you want to ADD: Include them in steps[] WITHOUT tcStepID (ONLY if user requested) 133. 4. For steps you want to REMOVE: Include them in removeSteps[] with full details 134. 5. Always set isStepUpdated: true if steps are added, updated, or removed 135. 6. VERIFY your steps array matches user's explicit request (count and content) 136. - Example: If user says 'Edit step 1 to say ...', FIRST fetch steps to get tcStepID for step 1, THEN include it in the steps array with updated fields and the ORIGINAL tcStepID. 137. - Example: If user says 'Add a new step after step 2', add EXACTLY ONE new object to steps array with no tcStepID (not multiple steps). 138. - Example: If user says 'Remove step 3', add the full step object to removeSteps array, including tcStepID and all required fields. Do NOT add replacement steps. 139. - Example: If test case has 3 steps and user says 'remove step 1', result should have 2 steps (step 2 and step 3 with updated orderIds), NOT 3 steps with extras. 140. - Example: If user says 'add one mock step', add EXACTLY ONE step (not 2 or 3 steps even if they seem related). 141. - COMPLETE PAYLOAD EXAMPLE: { tcID: 123, tcVersionID: 456, steps: [{tcStepID: 1001, orderId: 1, description: 'Updated'}, {orderId: 2, description: 'New'}], removeSteps: [{tcStepID: 1002, orderId: 3, ...}], isStepUpdated: true } 142. - If only metadata is updated (no steps), set updateOnlyMetadata: true and do not include steps/removeSteps. 143. - Always preserve orderId sequence for proper step ordering. 144. - If user prompt is ambiguous, ask for clarification or show a user-friendly error. 145. - WARNING: Omitting tcStepID for existing steps will cause DUPLICATION - the API will create duplicates instead of updating! 146. - FINAL VERIFICATION BEFORE SENDING REQUEST: 147. * Count steps in your payload vs what user requested 148. * If user said 'add 1 step', steps array should have (existing_count + 1) items total 149. * If user said 'remove 1 step', steps array should have (existing_count - 1) items total, removeSteps should have 1 item 150. * If user said 'update step X', steps array should have same count as before, with step X's tcStepID preserved 151. * NEVER include steps the user did not explicitly mention or request 152. Steps are optional but recommended for manual test cases. 153. If the user provides a prompt like 'update test case with steps as step 1 - Go to login page, step 2 - give credential, step 3 - go to test case page, step 4 - create test case', LLM should parse each step and convert it into the steps payload array, mapping each step to an object with orderId, description, and optionally inputData and expectedOutcome. 154. Example mapping: 'step 1 - Go to login page' β†’ { orderId: 1, description: 'Go to login page' }. 155. LLM should increment orderId for each step, use the step text as description, and optionally infer inputData/expectedOutcome if provided in the prompt. 156. Demo steps payload: steps: [ { orderId: 1, description: 'First Step', inputData: 'First Data', expectedOutcome: 'First Outcome', UDF: { customField1: 'Custom Field Data A', customField2: 'Custom Field Data B' } }, ... ] 157. UDF fields in steps must match your QMetry custom field configuration. 158. All IDs (priority, owner, etc.) must be valid for your QMetry instance. 159. If a custom field is mandatory, include it in the UDF object. 160. 161. ADDITIONAL VERSION CREATION GUIDANCE: 162. - versionComment field: STRONGLY RECOMMENDED when withVersion=true. Helps track why version was created. 163. Example comments: 'Updated for Sprint 5 requirements', 'Fixed test steps based on code review', 'Version 2 for production environment' 164. - notruncurrent and notrunall flags: Control execution behavior when creating versions. Set both to true as best practice. 165. - folderPath: Can be string path or numeric folder ID. Usually inherited from source version if not specified. 166. - attachments: Use ADD/REMOVE arrays to manage attachments when creating new version or updating existing version. 167. - estimatedTime vs executionMinutes: Use estimatedTime (in seconds) for version creation. executionMinutes (in minutes) is legacy field. 168. 169. REAL-WORLD VERSION CREATION EXAMPLES: 170. Example 1: User says 'create a new version of test case VKMCP-TC-10 with summary = "Facebook Login Validation Failed update from MCP V2", description = used existing description by at last add V2 text, release = default, cycle = default' 171. β†’ Workflow: 172. 1. Fetch VKMCP-TC-10 details to get tcID, tcVersionID, tcVersion, current description 173. 2. Fetch project info to get default release ID and cycle ID 174. 3. Append ' V2' to current description 175. 4. Send payload with: tcID, tcVersionID (source), tcVersion (current), withVersion=true, name='...V2', description='...V2', versionComment='Created version 2', release/cycle IDs 176. β†’ Result: New incremental version created (e.g., version 1 β†’ version 2) with updated summary, modified description, associated with default release/cycle 177. 178. Example 2: User says 'update version 2 summary, release, cycle, priority' 179. β†’ Workflow: 180. 1. Fetch test case details to get version 2's tcVersionID 181. 2. Fetch project info to get priority, release, cycle IDs 182. 3. Send payload with: tcID, tcVersionID (of version 2), WITHOUT withVersion flag, with updated summary, release, cycle, priority 183. β†’ Result: Version 2 updated in-place. No new version created. Only specified fields modified. 184. 185. COMMON PITFALLS TO AVOID: 186. - Pitfall 1: Setting withVersion=true when user wants to update existing version β†’ Creates unwanted new version 187. - Pitfall 2: Omitting versionComment when creating new version β†’ Lost tracking of why version was created 188. - Pitfall 3: Not fetching current tcVersionID before update β†’ Updating wrong version or causing error 189. - Pitfall 4: Using tcVersion for normal updates β†’ tcVersion only needed when withVersion=true 190. - Pitfall 5: Not including tcStepID for steps when creating version β†’ Steps may duplicate instead of preserving 191. - Pitfall 6: Confusing tcVersionID (version identifier) with tcVersion (version number) β†’ Wrong API payload 192. 193. CRITICAL ERROR HANDLING: TC.VERSION_NOT_SYNCED FOR LINKED TEST CASES 194. 195. PROBLEM: Test cases linked to test suites or with existing executions cannot be updated directly 196. TRIGGERS: 197. βœ“ Test case is linked to one or more test suites 198. βœ“ Test case has execution history (test runs exist) 199. βœ“ Test case is part of a release/cycle with test runs 200. βœ“ Any step modification when test case has associations 201. 202. ERROR CODE: TC.VERSION_NOT_SYNCED 203. ERROR MESSAGE: 'This Test Case has been associated. Create a new version to save modifications.' 204. 205. AUTOMATIC RETRY STRATEGY (IMPLEMENT THIS PATTERN): 206. When you receive TC.VERSION_NOT_SYNCED error, AUTOMATICALLY retry with version creation: 207. DO NOT ask user for confirmation - this is expected behavior for associated test cases 208. 209. 1. FIRST ATTEMPT (Normal Update): 210. - Try updating with isStepUpdated=true (for step changes) or without withVersion flag 211. - If error code is TC.VERSION_NOT_SYNCED, proceed to step 2 212. - If error is different, report the error to user 213. 214. 2. AUTOMATIC RETRY (Version Creation): 215. - Use the SAME tcID, tcVersionID, and steps array from first attempt 216. - Add these flags to payload: 217. * withVersion: true (CRITICAL - enables version creation) 218. * tcVersion: (get from test case details or executions) 219. * notrunall: false (use false, not true) 220. * notruncurrent: false (use false, not true) 221. * scope: 'project' (always required) 222. * versionComment: 'Auto-created version due to test suite association' (or custom message) 223. - Set isStepUpdated: true whenever you modify steps (including when withVersion=true) 224. - IMPORTANT: Include ALL existing steps with tcStepID + new steps without tcStepID 225. 226. 3. VERIFICATION: 227. - Check response for new tcVersionID (will be different from source) 228. - Verify tcVersion incremented (e.g., 1β†’2, 2β†’3) 229. - Confirm success message: 'Test Case updated successfully' 230. 231. UI BEHAVIOR COMPARISON: 232. QMetry UI shows a popup: 'Save as new version?' with optional comment field 233. API equivalent: Automatic retry with withVersion=true after detecting TC.VERSION_NOT_SYNCED 234. 235. REAL-WORLD EXAMPLE FROM UI PAYLOADS: 236. 237. First Attempt (FAILS with TC.VERSION_NOT_SYNCED): 238. pseudo 239. { 240. "tcID": 4594140, 241. "tcVersionID": 5536696, 242. "withVersion": false, 243. "notrunall": false, 244. "steps": [ 245. // ... 5 existing steps with tcStepID ... 246. // ... 1 new step without tcStepID (orderId: 6) ... 247. ], 248. "removeSteps": [], 249. "isStepUpdated": true 250. } 251. 252. Response: 400 - TC.VERSION_NOT_SYNCED error 253. 254. Second Attempt (SUCCEEDS - Creates Version 2): 255. pseudo 256. { 257. "withVersion": true, // NEW: Version creation flag 258. "notrunall": false, 259. "notruncurrent": false, 260. "steps": [ 261. // SAME steps array as first attempt 262. // ... 5 existing steps with tcStepID ... 263. // ... 1 new step without tcStepID ... 264. ], 265. "removeSteps": [], 266. "scope": "project", 267. "tcID": 4594140, // SAME tcID 268. "tcVersion": 1, // NEW: Current version number 269. "tcVersionID": 5536696, // SAME tcVersionID (source version) 270. "versionComment": "test", // NEW: Version comment (optional) 271. // NOTE: isStepUpdated field is NOT included when withVersion=true 272. } 273. 274. Response: 200 - Success, new tcVersionID created (e.g., 5536697), tcVersion=2 275. 276. IMPLEMENTATION PSEUDO-CODE: 277. typescript 278. try { 279. // First attempt: Normal update 280. const response = await updateTestCase({ 281. tcID, tcVersionID, steps, isStepUpdated: true 282. }); 283. } catch (error) { 284. if (error.code === 'TC.VERSION_NOT_SYNCED') { 285. // Automatic retry with version creation 286. const testCaseDetails = await fetchTestCaseDetails(tcID); 287. // CRITICAL: Use the LATEST version number from system 288. const latestVersion = testCaseDetails.tcVersion; // e.g., 2 if v2 exists 289. const response = await updateTestCase({ 290. tcID, 291. tcVersionID, // Same source version 292. tcVersion: latestVersion, // Use latest version number (not always 1!) 293. steps, // Same steps array 294. // DO NOT include isStepUpdated when withVersion=true 295. withVersion: true, // Enable version creation 296. notrunall: false, // Use false (verified working value) 297. notruncurrent: false, // Use false (verified working value) 298. scope: 'project', // Always required 299. versionComment: 'Auto-created version due to test suite association' 300. }); 301. } else { 302. throw error; // Different error, report to user 303. } 304. } 305. 306. 307. KEY INSIGHTS: 308. - DO NOT ask user for confirmation - auto-retry is expected behavior 309. - Use SAME tcVersionID in both attempts (source version for creation) 310. - Second attempt creates NEW version (tcVersionID changes in response) 311. - Steps array is IDENTICAL in both attempts 312. - tcVersion parameter is ONLY in second attempt (withVersion=true) 313. - This matches QMetry UI behavior where popup auto-triggers version creation 314. 315. CRITICAL: INCREMENTAL tcVersion SELECTION RULE 316. 317. PROBLEM: When multiple versions exist, which tcVersion should you use? 318. SOLUTION: Use the VERSION NUMBER of the version you are creating FROM (the latest existing version) 319. 320. RULE: When creating a new version, tcVersion must equal the CURRENT LATEST VERSION in the system 321. 322. EXAMPLES: 323. - If only version 1 exists: Use tcVersion: 1 (creates version 2 from v1) 324. - If version 1 and 2 exist: Use tcVersion: 2 (creates version 3 from v2) 325. - If version 1, 2, and 3 exist: Use tcVersion: 3 (creates version 4 from v3) 326. 327. WORKFLOW TO DETERMINE CORRECT tcVersion: 328. 1. Call FETCH_TEST_CASE_DETAILS or FETCH_TEST_CASE_EXECUTIONS 329. 2. Check the highest tcVersion number in the system 330. 3. Use that number as your tcVersion parameter in the update payload 331. 4. This ensures you're creating from the latest version, not an old one 332. 333. REAL-WORLD SCENARIO: 334. Scenario: Test case VKMCP-TC-43 has version 2 already created in UI 335. Wrong Approach (will fail): tcVersion: 1, withVersion: true β†’ TC.VERSION_NOT_SYNCED error 336. Correct Approach (will succeed): tcVersion: 2, withVersion: true β†’ Creates version 3 successfully 337. 338. VERIFIED WORKING PAYLOAD (from user's Postman testing): 339. json 340. { 341. "notrunall": false, 342. "notruncurrent": false, 343. "scope": "project", 344. "tcID": 4594145, 345. "tcVersion": 2, // KEY: Use version 2 because v2 already exists 346. "tcVersionID": 5536706, // Source version ID (stays same) 347. "versionComment": "Added new steps", // Describes what changed 348. "withVersion": true, // Enable version creation 349. "steps": [ 350. // 5 existing steps with tcStepID (preserved from source) 351. // 2 new steps without tcStepID (to be added) 352. ] 353. } 354. 355. Result: New version 3 created successfully with 7 total steps 356. 357. DEFAULT VALUES FOR TC.VERSION_NOT_SYNCED RETRIES (THIS PATTERN): 358. - notrunall: false - For this retry pattern, override any usual true default 359. - notruncurrent: false - For this retry pattern, override any usual true default 360. - scope: "project" - Always use this 361. - withVersion: true - Required when creating a new version (both initial and retries) 362. 363. WHEN TO APPLY THIS PATTERN: 364. βœ“ Adding steps to test case linked to test suite 365. βœ“ Editing steps in test case with existing executions 366. βœ“ Removing steps from associated test case 367. βœ“ Any modification to steps when TC.VERSION_NOT_SYNCED occurs 368. βœ“ When creating new versions from existing versions (always check latest version number) 369. 370. WHEN NOT TO APPLY: 371. βœ— Test case is NOT linked to test suite (normal update works) 372. βœ— Only updating metadata (name, priority, etc.) without steps 373. βœ— Different error codes (handle appropriately) 374. 375. BENEFITS OF THIS APPROACH: 376. 1. Seamless UX - LLM handles version creation automatically 377. 2. Matches UI behavior - no manual intervention needed 378. 3. Preserves test history - creates proper version trail 379. 4. Maintains test suite linkage - version creation preserves associations 380. 5. Handles incremental versions correctly - uses latest version as source 381. 382. 🎯 GRACEFUL HANDLING SUMMARY: 383. 384. When adding/editing/removing steps from test cases: 385. 1. Always TRY normal update first (without withVersion flag) 386. 2. If TC.VERSION_NOT_SYNCED error received: 387. - Fetch latest version number from test case details/executions 388. - Automatically retry with withVersion=true + correct tcVersion 389. - Use notrunall=false, notruncurrent=false, scope='project' 390. - Include version comment describing the changes 391. 3. Report success with new version details to user 392. 4. NEVER ask for confirmation - handle it transparently 393. 394. This ensures test cases with executions or suite associations are handled gracefully 395. without user intervention, matching the QMetry UI experience exactly. 396. 397. executionMinutes time is in minutes (legacy field). 398. estimatedTime is in seconds (preferred for version creation). 399. Description and testingType are optional but recommended for clarity. 400. 401. UDF (User Defined Fields) WORKFLOW FOR UPDATE: 402. 1. Call 'Fetch UDF Layout' with entityType='TC', pageName='DETAIL' to get field names, fieldIDs (projectUserFieldID), and list option IDs. 403. IF listOptions[field.listName] is empty after Fetch UDF Layout, the tool already tried a metadata fallback. If STILL empty, ask the user to provide the option ID from the QMetry UI β€” do NOT guess numeric IDs. 404. 2. For LOOKUPLIST fields: pick one ID from listOptions[field.listName][].id. 405. 3. For MULTILOOKUPLIST fields: pick an array of IDs; also pass the alias flat key (e.g., fieldNameAlias: 'Option Label'). 406. 4. For CASCADINGLIST fields: pick parent ID + fetch child with 'Fetch Cascade Child Values'. Pass { parent: parentId, child: childId }. 407. 5. Pass BOTH 'udfFields' (flat root values) AND 'UDF' wrapper (with fieldID) β€” both required for update. 408. Example: udfFields: { custom_text: 'new value' }, UDF: { custom_text: { fieldID: 1001, value: 'new value' } } 409. 6. Mandatory UDF fields (isMandatory=true) MUST be included. 410. STEP UDFs for update: Use same step UDF field names from 'Fetch UDF Layout' stepFields. Pass in each step's 'UDF' object. 411. For MULTILOOKUPLIST step UDFs in update: use { ADD: [id1, id2], REMOVE: [id3] } format.

ParametersJSON Schema
NameRequiredDescriptionDefault
UDFNoUDF wrapper required for update operations. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value. Also set matching flat key in udfFields for the LOOKUPLIST Alias display. Example: { custom_text: { fieldID: 1001, value: 'new value' } }
nameNo
tcIDYesTest Case numeric ID. CRITICAL: the parameter name is 'tcID' β€” do NOT use 'testCaseId', 'testCaseID', 'tcId', or other variants. Accepts a string or number. This is the internal numeric identifier, not the entity key like 'MAC-TC-1684'. You can get this ID from test case search results or by using filters.
ownerNo
scopeNoScope of the operation - defines the context for data retrieval. Common values: 'project' (default), 'folder', 'release', 'cycle'. Applies to any entity type being fetched or operated upon.project
stepsNo
priorityNo
componentNo
notrunallNoFlag to control execution behavior for all versions when creating a new version. Used in conjunction with withVersion=true.
tcVersionNoTest Case version number (required when withVersion=true for creating new version). This is the current version number from which a new version will be created.
udfFieldsNoFlat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.
folderPathNoFolder path for test suites - SYSTEM AUTOMATICALLY SETS TO ROOT. Leave empty unless you want specific folder. System will automatically use empty string "" (root directory). Only specify if user wants specific folder like "Automation/Regression".
projectKeyNoProject key - unique identifier for the projectdefault
descriptionNo
removeStepsNo
tcVersionIDYesTest Case version number. This is the internal numeric identifier for the version.
testingTypeNo
withVersionNoPass 'true' if you want to create a new version of the test case with incremented version number. When true, a new version is created (e.g., if current version is 2, new version 3 is created). When false or omitted, updates the existing version specified by tcVersionID. IMPORTANT: Always send proper tcVersionID to identify which version the request is for.
testCaseTypeNo
estimatedTimeNoEstimated execution time in seconds. Example: 7200 for 2 hours
isStepUpdatedNoSet to true when steps are being added, updated, or removed. Required when including 'steps' or 'removeSteps' arrays.
notruncurrentNoFlag to control execution behavior for current version when creating a new version. Used in conjunction with withVersion=true.
testCaseStateNo
versionCommentNoComment or description for the new version (used only when withVersion=true). Helps track what changed in this new version. Example: 'Updated test steps for new requirements'
executionMinutesNo
updateOnlyMetadataNoSet to true to update only metadata fields without touching test steps. When true, steps and removeSteps are ignored.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only indicate readOnlyHint=false and destructiveHint=false, so the description carries the behavioral burden and exceeds it. It discloses the version-creation behavior, automatic retry on TC.VERSION_NOT_SYNCED, silent data-loss risk for date-format mismatches, default-value sweeping, and anti-duplication semantics for tcStepID. There is no contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely long and repetitive: version-creation workflows, anti-duplication rules, and TC.VERSION_NOT_SYNCED guidance are repeated multiple times with overlapping examples. Headings help organization, but the length and redundancy undermine scannability for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, the description includes an output description, twelve use cases, nine fully worked examples, field-resolution workflows, error handling, retry logic, and UDF layout prerequisites. For a 26-parameter tool with nested objects, this is functionally complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds substantial meaning beyond the schema: it warns about the exact 'tcID' parameter name, clarifies tcVersionID versus tcVersion, explains withVersion/isStepUpdated/updateOnlyMetadata, and gives detailed UDF field rules. However, several examples use 'testcaseOwner' while the schema defines 'owner', and one example passes folderPath as a number while the schema types it as a string, which could mislead an agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear verb and resource: 'Update an existing QMetry test case OR create a new version by tcID and tcVersionID, with auto-resolution from entityKey.' It states exactly what the tool does and differentiates it from sibling create/fetch tools in the QMetry toolset.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description contains explicit mode guidance: MODE 1 (withVersion=true) versus MODE 2 (update existing version), with concrete user-phrase mappings, required fields for each mode, and instructions to ask the user when ambiguous. It also specifies when to apply and when not to apply the TC.VERSION_NOT_SYNCED retry pattern.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qmetry_update_test_suiteQMetry: Update Test SuiteAInspect

Update an existing QMetry test suite by id(testsuite numeric id), with auto-resolution from entityKey.

Toolset: Test Suites

Parameters:

  • id (number) required: Id of Test Suite to be updated (required)

  • TsFolderID (number): Folder ID where Test Suite resides - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific folder ID. System will fetch project info using the projectKey and extract rootFolders.TS.id automatically. Manual folder ID only needed if you want to override the automatic resolution.

  • entityKey (string) required: Entity Key of Test Suite to be updated (required)

  • name (string): Name of the Test Suite

  • description (string): Description of the Test Suite

  • testsuiteOwner (number): Owner ID of the Test Suite

  • testSuiteState (number): State of the Test Suite

  • udfFields (record<string, union>): Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.

  • UDF (record<string, object>): UDF wrapper required for update. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value.

Output Description: JSON object containing the new test suite ID, summary, and creation metadata.

Use Cases: 1. Update test suite summary (name) 2. Change owner, or state of a test suite 3. Bulk update using entityKey auto-resolution 4. Modify test suite description

Examples:

  1. Update test suite summary (updated name)

{
  "id": 1505898,
  "entityKey": "VT-TS-7",
  "TsFolderID": 1644087,
  "name": "MAC Test11"
}

Expected Output: Test suite summary updated. Only 'name' field changed. Field IDs auto-resolved from project info. id(test suite numeric id) resolved from entityKey. TsFolderID auto-resolved. from the project info. info on rootFolders.TS.id.

  1. Update state to Open and owner of the test suite

{
  "id": 1505898,
  "entityKey": "VT-TS-7",
  "TsFolderID": 1644087,
  "testSuiteState": 505036,
  "testsuiteOwner": 6963
}

Expected Output: State and owner updated. Example uses: testSuiteState=505036 (Open from customListObjs.testSuiteState[index].id), testsuiteOwner=6963 (umang.savaliya from customListObjs.owner[index].id). Field IDs auto-resolved from project info. id(test suite numeric id) resolved from entityKey. TsFolderID auto-resolved from the project info rootFolders.TS.id.

  1. Update only description of the test suite

{
  "id": 1505898,
  "entityKey": "VT-TS-7",
  "TsFolderID": 1644087,
  "description": "Updated description for the test suite."
}

Expected Output: description updated only. Field IDs auto-resolved from project info. id(test suite numeric id) resolved from entityKey. TsFolderID auto-resolved. from the project info. info on rootFolders.TS.id.

Hints: 1. === DEFAULT VALUES β€” APPLY FOR ANY UNSET FIELD === 2. ALWAYS call 'Fetch UDF Layout' with entityType='TS', pageName='DETAIL' before updating. 3. defaultValues (from Fetch UDF Layout): { fieldName: defaultValueId } β€” sweep ALL entries. 4. For EACH key in defaultValues: if user did not explicitly provide that field β†’ include it in payload with the default value. 5. This applies to non-mandatory fields too (e.g. testSuiteState). Omitting them = data loss. 6. systemFields (from qmSDF.TS): isMandatory=true fields must retain a valid value after update. 7. fields/UDF (from qmUDF.TS): isMandatory=true UDF fields must be included if being changed. 8. === END DEFAULT VALUES === 9. 10. === DATE FORMAT CHECK (MANDATORY β€” EVERY UPDATE REQUEST) === 11. ALWAYS call 'Fetch QMetry Project Info' before every update request β€” not only when the user explicitly mentions a date. 12. Any UDF field could be a DATETIMEPICKER. Wrong format causes QMetry to silently discard the field value (API returns success but value is NOT stored β€” no error). 13. STEP 1: From project info, read dateTimeFormatID (e.g. 3). 14. STEP 2: Find entry in dateTimeFormatNew where id === dateTimeFormatID β†’ read its unique_value (e.g. 'yyyy-MM-dd'). 15. STEP 3: unique_value pattern: yyyy=4-digit year, MM=2-digit month (01-12), dd=2-digit day, MMM=3-letter month (Jan/Feb/...). 16. Example: id=1 β†’ MM-dd-yyyy β†’ '10-25-2000' | id=2 β†’ dd-MM-yyyy β†’ '25-10-2000' | id=3 β†’ yyyy-MM-dd β†’ '2000-10-25' | id=4 β†’ dd-MMM-yyyy β†’ '25-Oct-2000' 17. STEP 4: For EVERY DATETIMEPICKER field in the payload: parse any user-provided date and re-format it using the active unique_value pattern before sending. 18. NEVER assume a date format β€” always derive it from dateTimeFormatID. Wrong format = silent data loss. 19. === END DATE FORMAT CHECK === 20. 21. If user provides entityKey (e.g., MAC-TS-7), first call Fetch Test Suites with a filter on entityKeyId to resolve the id (test suite numeric id) and TsFolderID from rootFolders.TS.id. 22. To get valid values for owner, state, etc., call the 'Admin/Get info Service' API (FETCH_PROJECT_INFO tool) and use the returned customListObjs IDs. 23. CRITICAL: For testsuiteOwner mapping - Call API 'Admin/Get info Service', from the response get value from customListObjs.owner[].id. Match the user by customListObjs.owner[].name. 24. If the user provides an owner name, fetch project info, find the matching user in customListObjs.owner[index].name, and use its ID in the payload as testsuiteOwner. If the name is not found, skip the testsuiteOwner field (it is not required) and show a user-friendly message: 'Test suite updated without owner, as given owner is not available in the current project.' 25. CRITICAL: For testSuiteState mapping - Call API 'Admin/Get info Service', from the response get value from customListObjs.testSuiteState[].id. Match the state by customListObjs.testSuiteState[].name. 26. If the user provides a test suite state name, fetch project info, find the matching state in customListObjs.testSuiteState[index].name, and use its ID in the payload as testSuiteState. If the name is not found, skip the testSuiteState field (it is not required) and show a user-friendly message: 'Test suite updated without test suite state, as given state is not available in the current project.' 27. If either owner or state is not found in project info, the update for that field will be skipped and a user-friendly message will be shown to the user. 28. UDF fields in steps must match your QMetry custom field configuration. 29. All IDs (testSuiteState from customListObjs.testSuiteState[index].id, testsuiteOwner from customListObjs.owner[index].id) must be valid for your QMetry instance. 30. If a custom field is mandatory, include it in the UDF object. 31. 32. UDF (User Defined Fields) WORKFLOW FOR UPDATE: 33. 1. Call 'Fetch UDF Layout' with entityType='TS', pageName='DETAIL' to get field names, fieldIDs (projectUserFieldID), and list option IDs. 34. IF listOptions[field.listName] is empty after Fetch UDF Layout, the tool already tried a metadata fallback. If STILL empty, ask the user to provide the option ID from the QMetry UI β€” do NOT guess numeric IDs. 35. 2. For LOOKUPLIST fields: pick one ID from listOptions[field.listName][].id. 36. 3. For MULTILOOKUPLIST fields: pick array of IDs; also pass alias flat key (e.g., fieldNameAlias: 'Option Label'). 37. 4. For CASCADINGLIST fields: pick parent ID + fetch child with 'Fetch Cascade Child Values'. Pass { parent: parentId, child: childId }. 38. 5. Pass BOTH 'udfFields' (flat root values) AND 'UDF' wrapper (with fieldID) β€” both required for update. 39. Example: udfFields: { ts_field: 'value' }, UDF: { ts_field: { fieldID: 2001, value: 'value' } } 40. 6. Mandatory UDF fields (isMandatory=true) MUST be included.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesId of Test Suite to be updated (required)
UDFNoUDF wrapper required for update. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value.
nameNoName of the Test Suite
entityKeyYesEntity Key of Test Suite to be updated (required)
udfFieldsNoFlat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.
TsFolderIDNoFolder ID where Test Suite resides - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific folder ID. System will fetch project info using the projectKey and extract rootFolders.TS.id automatically. Manual folder ID only needed if you want to override the automatic resolution.
descriptionNoDescription of the Test Suite
testSuiteStateNoState of the Test Suite
testsuiteOwnerNoOwner ID of the Test Suite

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=false, destructiveHint=false, but the description goes far beyond by disclosing critical behaviors: silent data loss on wrong date formats, mandatory default value sweeping, auto-resolution of TsFolderID, and the need to call supporting tools. It also warns about mandatory UDF fields and the consequences of omitting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely long and repetitive, with many numbered hints that repeat the same information (e.g., auto-resolution of TsFolderID mentioned multiple times). While it is well-structured with sections, the verbosity and redundancy reduce its conciseness. It could be trimmed significantly without losing value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 parameters, nested objects, no output schema), the description is exceptionally complete. It covers all necessary preconditions, workflows, error handling, and fallback messages. The examples and hints provide comprehensive guidance for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds substantial meaning: it explains the auto-resolution of TsFolderID, the dual requirement of udfFields and UDF wrapper, the exact structure for CASCADINGLIST, and the need to fetch UDF layout for field IDs. It also clarifies the date format handling and the mapping of owner/state names to IDs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates an existing QMetry test suite by numeric id with auto-resolution from entityKey. It distinguishes from siblings like qmetry_create_test_suite and qmetry_fetch_test_suites by specifying the update action and the auto-resolution mechanism.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides extensive usage guidance: when to use (update operations), when not to (e.g., creation via qmetry_create_test_suite), and explicit alternatives like 'Fetch UDF Layout' and 'Fetch QMetry Project Info'. It also details prerequisites and fallback behaviors for owner/state resolution.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_create_test_caseQTM4J: Create Test CaseAInspect

Create a new test case in a QTM4J project. Supports auto-resolving human-readable names for priority, status, labels, and components.

Toolset: Test Cases

Parameters:

  • summary (string) required: Test case summary/title

  • description (string): Test case description

  • folderId (number): Folder ID to place the test case in

  • priority (string): Priority name (e.g., 'High', 'Medium', 'Low'). Auto-resolved to ID.

  • status (string): Status name (e.g., 'To Do', 'In Progress', 'Done'). Auto-resolved to ID.

  • assignee (string): Assignee account ID

  • reporter (string): Reporter account ID

  • components (array): List of component names (e.g., ['UI', 'Cloud']). Auto-resolved to IDs.

  • labels (array): List of label names (e.g., ['Release_1', 'Sprint 1']). Auto-resolved to IDs.

  • steps (array): List of test steps

Output Description: JSON object with test case ID, key, version number, and summary. Warnings included if any fields were skipped.

Use Cases: 1. Create a basic test case with just a summary 2. Create a test case with priority and status using names from set_project_context response 3. Create a test case with labels and components by exact name 4. Add detailed test steps with step descriptions, test data, and expected results 5. Create a test case in a specific folder using folderId 6. Set assignee and reporter using Jira account IDs 7. Create test cases for manual testing with step-by-step instructions 8. Create test cases with all metadata fields for comprehensive test management

Examples:

  1. Create a simple test case (project must be set via set_project_context first)

{
  "summary": "Search Functionality"
}

Expected Output: Test case created with key 'SCRUM-TC-xxx'

  1. Create a test case with priority and status

{
  "summary": "Search Functionality",
  "description": "Verify search functionality works correctly",
  "priority": "High",
  "status": "To Do"
}

Expected Output: Test case created with resolved priority and status IDs

  1. Create a test case with labels, components, and steps

{
  "summary": "Search Functionality",
  "description": "Search Functionality Test",
  "priority": "High",
  "status": "To Do",
  "labels": [
    "Release_1",
    "Sprint 1"
  ],
  "components": [
    "UI",
    "Cloud"
  ],
  "steps": [
    {
      "stepDetails": "Enter a keyword in the search box",
      "testData": "Keyword = \"Test\"",
      "expectedResult": "The keyword should be visible in the search box"
    },
    {
      "stepDetails": "Click on the Search button",
      "testData": "Click on Search Button",
      "expectedResult": "Search results matching the keyword should be displayed"
    },
    {
      "stepDetails": "Verify the search results",
      "testData": "Expected results list",
      "expectedResult": "Results should be relevant to the entered keyword"
    }
  ]
}

Expected Output: Test case created with resolved labels/components/priority/status and 3 steps

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. Priority and status values were returned by set_project_context. Use NLP to map user input (e.g., 'Major' β†’ 'High', 'Critical' β†’ 'Blocker'). 3. If priority or status name is not found, the operation proceeds without that field and a warning is returned. 4. Labels and components are resolved on demand. If a name is not found, it is skipped with a warning. 5. Steps: ALWAYS include all three fields β€” stepDetails, testData, and expectedResult. Generate reasonable values if not provided. 6. folderId is optional. assignee and reporter accept Jira account IDs.

ParametersJSON Schema
NameRequiredDescriptionDefault
stepsNoList of test steps
labelsNoList of label names (e.g., ['Release_1', 'Sprint 1']). Auto-resolved to IDs.
statusNoStatus name (e.g., 'To Do', 'In Progress', 'Done'). Auto-resolved to ID.
summaryYesTest case summary/title
assigneeNoAssignee account ID
folderIdNoFolder ID to place the test case in
priorityNoPriority name (e.g., 'High', 'Medium', 'Low'). Auto-resolved to ID.
reporterNoReporter account ID
componentsNoList of component names (e.g., ['UI', 'Cloud']). Auto-resolved to IDs.
descriptionNoTest case description

Output Schema

ParametersJSON Schema
NameRequiredDescription
idYesUnique test case ID
keyYesTest case key (e.g., 'SCRUM-TC-190')
summaryYesTest case summary
versionNoYesVersion number

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description explains key runtime behavior: name-to-ID auto-resolution, graceful handling of unknown priority/status/label/component values with warnings, and the requirement that steps include stepDetails, testData, and expectedResult. It also states what the response contains, including warnings for skipped fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well organized and front-loaded with the core purpose and key behavior. The parameter list duplicates the schema somewhat, and the eight use cases are somewhat repetitive, but given the tool's 10 parameters and complex auto-resolution behavior, the structure and detail are earned.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers prerequisites, parameter semantics, edge cases, step structure, output shape, and examples. Nothing an agent needs to invoke this 10-parameter tool correctly is missing, and the presence of an output schema plus a detailed description makes it fully self-contained.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description still adds substantial meaning: examples for priority and status, exact label/component conventions, step field requirements, NLP mapping guidance ('Major' β†’ 'High'), and details about auto-resolution and skipping. This goes well beyond the schema's field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Create a new test case in a QTM4J project,' and immediately adds the distinctive auto-resolution behavior. This clearly differentiates it from sibling tools like zephyr_create_test_case, qmetry_create_test_case, and the related qtm4j_update_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Hints section is explicit about the prerequisite: set_project_context must be called first and NEVER auto-select a project. Eight concrete use cases also explain what scenarios the tool supports. It does not explicitly state when-not-to-use relative to alternatives, but the prerequisite and use-case list give clear operating conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_create_test_cycleQTM4J: Create Test CycleAInspect

Create a new test cycle in a QTM4J project. Supports auto-resolving human-readable names for priority and status. Always creates in the 'MCP Generated' folder. projectId is injected automatically from the active project context.

Toolset: Test Cycles

Parameters:

  • summary (string) required: Short title of the test cycle. Must not be blank. Max 255 chars.

  • description (string): Detailed description of the test cycle. Max 65 535 characters.

  • priority (string): Priority name (e.g., 'High', 'Medium', 'Low'). Auto-resolved to ID.

  • status (string): Status name (e.g., 'To Do', 'In Progress', 'Done'). Auto-resolved to ID.

  • assignee (string): Assignee account ID

  • reporter (string): Reporter account ID

  • labels (array): List of label names (e.g., ['Release_1', 'Sprint 1']). Auto-resolved to IDs.

  • components (array): List of component names (e.g., ['UI', 'Cloud']). Auto-resolved to IDs.

  • plannedStartDate (string): Planned start date. Format: 'dd/MMM/yyyy HH:mm' e.g. '10/May/2026 00:00'. Must be ≀ plannedEndDate when both are provided.

  • plannedEndDate (string): Planned end date. Format: 'dd/MMM/yyyy HH:mm' e.g. '15/May/2026 00:00'. Must be β‰₯ plannedStartDate when both are provided.

Output Description: JSON object with the new test cycle's id and key (e.g. 'TRWT-TR-218'). Warnings included if any fields were skipped.

Use Cases: 1. Create a test cycle with summary, priority, status, labels, or components 2. Set planned start and end dates on a new test cycle

Examples:

  1. Create a simple test cycle (project must be set via set_project_context first)

{
  "summary": "Smoke Test Cycle"
}

Expected Output: Test cycle created with key 'SCRUM-TR-xxx'

  1. Create a test cycle with priority, status, labels, and components

{
  "summary": "Regression Suite – Sprint 42",
  "description": "End-to-end regression covering payment and checkout modules.",
  "priority": "High",
  "status": "To Do",
  "labels": [
    "Release_1",
    "Sprint 1"
  ],
  "components": [
    "UI",
    "Cloud"
  ],
  "plannedStartDate": "10/May/2026 00:00",
  "plannedEndDate": "15/May/2026 00:00"
}

Expected Output: Test cycle created with resolved priority, status, labels, and components

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. If any priority, status, label, or component name cannot be resolved, the cycle is still created but a warning is returned. Suggest the closest available value from the set_project_context response and ask the user to confirm before retrying. 3. All cycles are placed in the 'MCP Generated' folder β€” do not pass folderId. 4. Date format: 'dd/MMM/yyyy HH:mm' e.g. '10/May/2026 00:00'. Month must be capitalised. plannedStartDate must be ≀ plannedEndDate.

ParametersJSON Schema
NameRequiredDescriptionDefault
labelsNoList of label names (e.g., ['Release_1', 'Sprint 1']). Auto-resolved to IDs.
statusNoStatus name (e.g., 'To Do', 'In Progress', 'Done'). Auto-resolved to ID.
summaryYesShort title of the test cycle. Must not be blank. Max 255 chars.
assigneeNoAssignee account ID
priorityNoPriority name (e.g., 'High', 'Medium', 'Low'). Auto-resolved to ID.
reporterNoReporter account ID
componentsNoList of component names (e.g., ['UI', 'Cloud']). Auto-resolved to IDs.
descriptionNoDetailed description of the test cycle. Max 65 535 characters.
plannedEndDateNoPlanned end date. Format: 'dd/MMM/yyyy HH:mm' e.g. '15/May/2026 00:00'. Must be β‰₯ plannedStartDate when both are provided.
plannedStartDateNoPlanned start date. Format: 'dd/MMM/yyyy HH:mm' e.g. '10/May/2026 00:00'. Must be ≀ plannedEndDate when both are provided.

Output Schema

ParametersJSON Schema
NameRequiredDescription
idYesOpaque permanent identifier of the created test cycle. Use this in all subsequent API calls.
keyYesHuman-readable project-scoped key in format '<PROJECT_KEY>-TR-<number>'. e.g. 'TRWT-TR-218'.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavior beyond annotations: it discloses automatic injection of projectId, auto-resolution of names to IDs, fallback warnings when resolution fails, the fixed 'MCP Generated' folder placement, date format capitalization requirements, and the plannedStartDate <= plannedEndDate constraint. No stated behavior contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well organized with Parameters, Output Description, Use Cases, Examples, and Hints. The parameter section somewhat duplicates the input schema, but the examples and targeted hints earn their place and make the tool easier to invoke correctly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 10 parameters, one required parameter, no enums, and an output schema, the description is unusually complete. It covers prerequisites, project context, default folder behavior, failure/warning behavior, date relationships, output shape, and realistic examples. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds value through examples, human-readable resolution semantics, warning behavior for unresolved values, and date-format hints such as capitalizing the month. It mostly echoes the schema but enriches it with operational guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Create a new test cycle in a QTM4J project.' It also adds distinguishing context such as auto-resolving priority/status names, creating in the 'MCP Generated' folder, and injected projectId, which separates it from sibling create tools for Qmetry and Zephyr.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear when-to-use guidance: the prerequisite set_project_context must be called first, project auto-selection is forbidden, and folderId must not be passed because cycles always go to 'MCP Generated'. It does not explicitly name an alternative such as qtm4j_update_test_cycle for edits, but the create-versus-update boundary is strongly implied by the name and content.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_get_automation_historyQTM4J: Get Automation HistoryA
Read-onlyIdempotent
Inspect

Retrieve a paginated history of past automation result uploads for a QTM4J project.

Toolset: Test Automation

Parameters:

  • startAt (number): Zero-indexed starting position for pagination (default: 0). (default: 0)

  • maxResults (number): Maximum number of records to return per page (default: 20, max: 100). (default: 20)

Output Description: Paginated list of automation import history. Each record includes: format, processStatus, importStatus, startTime, endTime, trackingId, detailedMessage, and a summary array. summary[0] contains: testCycleIssueKey, testCycleSummary, testCasesCreated, testCaseVersionsCreated, testCaseVersionsReused, testStepsCreated. Render as individual cards separated by dividers, NOT a table. Show '1–N of total' count above. Never show raw fileName.

Use Cases: 1. Review past automation result uploads for a project 2. Check the status of recent automation imports 3. Audit CI/CD automation upload history 4. Paginate through all historical automation uploads

Examples:

  1. Get the first page of automation upload history (default page size 20)

{}

Expected Output: Paginated list of automation history records with upload status and metadata

  1. Get the second page of automation upload history

{
  "startAt": 20,
  "maxResults": 20
}

Expected Output: Next 20 automation history records

  1. Get up to 50 records starting from the beginning

{
  "startAt": 0,
  "maxResults": 50
}

Expected Output: Up to 50 automation history records

Hints: 1. NO PROJECT CONTEXT REQUIRED: Do NOT call set_project_context and do NOT ask the user for a project key, project ID, or any other project details. This tool works independently. 2. PAGINATION: startAt is zero-indexed (default: 0), maxResults controls page size (default: 20, max: 100). Increment startAt by maxResults to fetch the next page. 3. Returns an empty data array (not an error) when no history records exist. 4. DISPLAY FORMAT: Show '1–N of ' above all cards. Render each record as a card separated by --- dividers. Each card has two sections:

PRIMARY SECTION (always first): heading with status emoji (βœ… SUCCESS / ❌ FAILED) + test cycle key and name; then format, startβ†’end time, message, summary stats (test cases/versions created/reused/test steps), tracking ID.

EXTRA DETAILS SECTION (at the bottom of the card, under a 'Details' sub-label): any remaining non-null fields from the record such as fileSize, extraAttributes values, etc.

Skip any field that is null, missing, or false. NEVER show the raw fileName. NEVER use a table.

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position for pagination (default: 0).
maxResultsNoMaximum number of records to return per page (default: 20, max: 100).

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataNo
totalNo
startAtNo
maxResultsNo

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false. The description goes well beyond these by disclosing that the tool returns an empty data array rather than an error when no history exists, that pagination is zero-indexed with a max of 100, and that it works without any project context. These behaviors are meaningful and not inferable from the annotations alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded with the core purpose, then organized into Parameters, Output Description, Use Cases, Examples, and Hints. However, it is somewhat verbose: defaults are repeated ('(default: 0)' twice, '(default: 20)' twice), and display-format instructions appear both in the Output Description and again in Hints #4. Nearly all content is useful, but the redundancy prevents a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool of this complexityβ€”paginated, read-only, with specific display rendering and a project-context pitfallβ€”the description is complete. It covers the output record fields, the summary sub-structure, edge behavior (empty array), formatting rules, and explicitly warns against unnecessary context calls. An agent has everything needed to call this tool correctly and render the result as intended.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although the schema already documents both parameters at 100% coverage, the description adds substantial practical semantics: it explains that startAt is zero-indexed, that maxResults controls page size, and explicitly states 'Increment startAt by maxResults to fetch the next page.' The three JSON examples demonstrate first-page, second-page, and larger-page usage, which is especially helpful for an agent selecting pagination values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a precise verb-resource pairing: 'Retrieve a paginated history of past automation result uploads for a QTM4J project.' This clearly identifies the tool as a read-only history/audit operation and implicitly distinguishes it from the sibling upload tool qtm4j_upload_automation_result. The title and toolset further reinforce the domain without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section explicitly enumerates when the tool should be used: reviewing history, checking import status, auditing CI/CD uploads, and paginating through records. The hints add an important exclusion: no project context is required and set_project_context must not be called. However, it does not explicitly name alternatives or contrast itself with the corresponding upload tool, so it falls just short of full 5-level guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_get_linked_bugs_of_test_case_executionQTM4J: Get Linked Bugs of Test Case ExecutionA
Read-onlyIdempotent
Inspect

Retrieve Jira bugs linked to a test case execution with optional priority and status filtering. Looks up testCaseExecutionId from testCycleKey and testCaseKey; resolves filter names to numeric IDs.

Toolset: Test Executions

Parameters:

  • testCycleKey (string) required: Test cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.

  • testCaseKey (string) required: Test case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.

  • filter (object): Filter criteria for linked bugs. Omit to return all linked bugs. priority and status names are resolved to numeric IDs; unresolvable names are skipped with a warning.

  • level (string): Execution level filter. Default: 'testcase_execution, teststep_execution' (both levels). Use 'testcase_execution' for test-case-level links only, 'teststep_execution' for step-level links only. (default: "testcase_execution, teststep_execution")

  • startAt (number): Zero-based index of the first result to return (pagination offset). (default: 0)

  • maxResults (number): Maximum number of bugs to return per page. Range 1–100; defaults to 20. (default: 20)

Output Description: JSON object with startAt, maxResults, total, and data array of linked bug objects. Each bug has: id, key, summary, status, priority, issueType, level, stepSeqNo, parameterGroup.

Use Cases: 1. List all Jira bugs linked to a test case execution 2. Filter linked bugs by priority or status

Examples:

  1. Get all linked bugs for a test case execution

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42"
}

Expected Output: Paginated list of all bugs linked to the test case execution

  1. Filter by priority

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "filter": {
    "priority": [
      "High"
    ]
  }
}

Expected Output: Linked bugs with High priority

  1. Filter by status

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "filter": {
    "status": [
      "To Do",
      "In Progress"
    ]
  }
}

Expected Output: Linked bugs in To Do or In Progress status

Hints: 1. Call set_project_context before this tool. 2. An empty result (total = 0) means no bugs are linked β€” it is not an error. 3. priority and status names are resolved to numeric IDs; unresolved names are skipped with a warning. 4. Use get_linked_bugs_of_test_step_execution to retrieve bugs linked at a specific step level.

ParametersJSON Schema
NameRequiredDescriptionDefault
levelNoExecution level filter. Default: 'testcase_execution, teststep_execution' (both levels). Use 'testcase_execution' for test-case-level links only, 'teststep_execution' for step-level links only.testcase_execution, teststep_execution
filterNoFilter criteria for linked bugs. Omit to return all linked bugs. priority and status names are resolved to numeric IDs; unresolvable names are skipped with a warning.
startAtNoZero-based index of the first result to return (pagination offset).
maxResultsNoMaximum number of bugs to return per page. Range 1–100; defaults to 20.
testCaseKeyYesTest case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.
testCycleKeyYesTest cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesLinked bugs on the current page; empty when total is 0.
totalYesTotal number of linked bugs matching the filter across all pages. 0 means none are linked.
startAtYesZero-based index of the first item in this page.
maxResultsYesPage size applied to this response.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, and the description goes further by explaining filter-name resolution, warning on unresolvable names, pagination defaults, and that an empty result is not an error. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured with clear sections (Toolset, Parameters, Output, Use Cases, Examples, Hints). It front-loads the core purpose and each section serves a purpose, though some parameter details are redundantly restated from the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers prerequisites, parameter formats, output shape, filtering semantics, pagination behavior, edge cases (empty results), and sibling differentiation. For a relatively complex read tool with nested filter objects, nothing essential for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value by providing practical examples, clarifying the API path behavior for testCycleKey, and explaining resolution semantics for priority/status filters beyond the raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Retrieve'), a specific resource ('Jira bugs linked to a test case execution'), and optional filtering behavior. It also names a sibling tool in the Hints section, `get_linked_bugs_of_test_step_execution`, clarifying the boundary between the two tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent when to use the tool, what to omit to get all bugs, how to filter, and which prerequisite to call (`set_project_context`). It also directs the agent to a sibling tool for step-level retrieval, leaving no ambiguity about alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_get_linked_bugs_of_test_step_executionQTM4J: Get Linked Bugs of Test Step ExecutionA
Read-onlyIdempotent
Inspect

Retrieve Jira bugs linked to a test step execution with optional priority and status filtering. Looks up testStepExecutionId from testCycleKey, testCaseKey, and step sequence number; resolves filter names to numeric IDs.

Toolset: Test Executions

Parameters:

  • testCycleKey (string) required: Test cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.

  • testCaseKey (string) required: Test case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.

  • testStepSeqNo (number) required: sequence number of the test step within the test case (e.g. 2 = the second step).

  • filter (object): Filter criteria for linked bugs. Omit to return all linked bugs. priority and status names are resolved to numeric IDs; unresolvable names are skipped with a warning.

  • startAt (number): Zero-based index of the first result to return (pagination offset). (default: 0)

  • maxResults (number): Maximum number of bugs to return per page. Range 1–100; defaults to 20. (default: 20)

Output Description: JSON object with startAt, maxResults, total, and data array of linked bug objects. Each bug has: id, key, summary, status, priority, issueType, level, stepSeqNo, parameterGroup.

Use Cases: 1. List all Jira bugs linked to a specific test step execution 2. Filter step-level linked bugs by priority or status

Examples:

  1. Get linked bugs for step 2

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "testStepSeqNo": 2
}

Expected Output: Bugs linked to step 2 of the execution

  1. Filter by priority

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "testStepSeqNo": 1,
  "filter": {
    "priority": [
      "High"
    ]
  }
}

Expected Output: High-priority bugs linked to step 1

  1. Filter by status

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "testStepSeqNo": 1,
  "filter": {
    "status": [
      "In Progress"
    ]
  }
}

Expected Output: In Progress bugs linked to step 1

Hints: 1. Call set_project_context before this tool. 2. An empty result (total = 0) means no bugs are linked β€” it is not an error. 3. priority and status names are resolved to numeric IDs; unresolved names are skipped with a warning. 4. Use get_linked_bugs_of_test_case_execution to retrieve all bugs on the test case execution across all steps.

ParametersJSON Schema
NameRequiredDescriptionDefault
filterNoFilter criteria for linked bugs. Omit to return all linked bugs. priority and status names are resolved to numeric IDs; unresolvable names are skipped with a warning.
startAtNoZero-based index of the first result to return (pagination offset).
maxResultsNoMaximum number of bugs to return per page. Range 1–100; defaults to 20.
testCaseKeyYesTest case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.
testCycleKeyYesTest cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.
testStepSeqNoYessequence number of the test step within the test case (e.g. 2 = the second step).

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesLinked bugs on the current page; empty when total is 0.
totalYesTotal number of linked bugs matching the filter across all pages. 0 means none are linked.
startAtYesZero-based index of the first item in this page.
maxResultsYesPage size applied to this response.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool read-only, idempotent, and non-destructive, and the description adds crucial behavioral context: it performs a testStepExecutionId lookup from three keys, resolves priority/status names to numeric IDs, skips unresolvable names with a warning, and clarifies that an empty result is not an error. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured into Parameters, Output Description, Use Cases, Examples, and Hints, with the core action front-loaded. Some parameter details duplicate the schema, but each section serves a clear purpose and the redundancy is acceptable for a complex tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with an output schema, the description covers the output shape, pagination, filtering semantics, prerequisites, and alternative tool routing. Nothing an agent needs to call this tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by explaining the compound relationship among testCycleKey/testCaseKey/testStepSeqNo, the auto-resolution of filter names, and by providing concrete JSON examples that illustrate parameter usage. This extra context lifts it to a 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a precise action: 'Retrieve Jira bugs linked to a test step execution with optional priority and status filtering.' It also explains how the target execution is resolved (from testCycleKey, testCaseKey, and step sequence number), and Hint 4 distinguishes it from the sibling qtm4j_get_linked_bugs_of_test_case_execution. No ambiguity about what this tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

A 'Use Cases' section spells out when to use the tool, and Hint 1 says to call set_project_context first. Hint 4 explicitly names the alternative for retrieving bugs across all steps, defining when not to use this tool. The behavior when omitting the filter is also stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_get_linked_requirementsQTM4J: Get Linked RequirementsA
Read-onlyIdempotent
Inspect

Retrieve the Jira requirements linked to a specific test case in QTM4J. Test case key is resolved to internal ID automatically.

Toolset: Test Cases

Parameters:

  • key (string) required: Test case key in '{PROJECT_KEY}-TC-{number}' format (e.g., 'SCRUM-TC-145'). Required.

  • versionNo (number): Test case version number to retrieve linked requirements for. Defaults to the latest version.

  • maxResults (number): Maximum results per page (1-100). Default: 50.

  • startAt (number): Zero-indexed offset for pagination. Default: 0.

  • sort (string): Sort pattern in 'field:asc|desc' format. Default: 'key:desc'.

Output Description: Paginated list with total, startAt, maxResults, and data array of linked requirement objects (id, key, summary, status, priority, issueType).

Use Cases: 1. Check which Jira stories or bugs a test case covers 2. Audit requirement traceability for a test case 3. Retrieve requirement keys to use in other operations 4. Verify that the correct requirements are linked to a test case before a release

Examples:

  1. Get all requirements linked to a test case

{
  "key": "SCRUM-TC-145"
}

Expected Output: Paginated list of linked requirements with Jira metadata

  1. Get requirements for a specific version

{
  "key": "SCRUM-TC-85",
  "versionNo": 2,
  "maxResults": 20
}

Expected Output: Requirements linked to version 2 of the test case

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. KEY FORMAT: '{PROJECT_KEY}-TC-{number}' β€” e.g. 'SCRUM-TC-145'. 3. versionNo defaults to the latest version. Use search_test_cases to find available versions. 4. Paginate using startAt β€” increment by maxResults until startAt >= total.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyYesTest case key in '{PROJECT_KEY}-TC-{number}' format (e.g., 'SCRUM-TC-145'). Required.
sortNoSort pattern in 'field:asc|desc' format. Default: 'key:desc'.
startAtNoZero-indexed offset for pagination. Default: 0.
versionNoNoTest case version number to retrieve linked requirements for. Defaults to the latest version.
maxResultsNoMaximum results per page (1-100). Default: 50.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesLinked requirements on this page.
totalYesTotal linked requirements (across all pages).
startAtYes
maxResultsYes

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior. The description adds valuable behavioral context beyond those annotations: test case keys are auto-resolved to internal IDs, versionNo defaults to latest, pagination follows a startAt/maxResults pattern, and a project context must be set beforehand. No hidden mutation or side effects are suggested.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than average but well-organized with clear sections: parameters, output, use cases, examples, and hints. Some redundancy exists, such as key format being repeated in the intro, parameter list, and hints, but the structure makes the information easy for an agent to consume and the key details are front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only retrieval tool with five parameters and an output schema, the description is complete: it covers prerequisites, key formatting, pagination, version selection, examples, and output shape. An agent has everything needed to select and invoke the tool correctly, including how to handle multi-page results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the parameter section largely duplicates the schema. However, the description adds operational semantics through examples and hints: how versionNo interacts with search_test_cases, how to paginate by incrementing startAt by maxResults until startAt >= total, and concrete request/expected-output pairs that illustrate parameter usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific verb ('Retrieve'), a specific resource ('Jira requirements linked to a specific test case in QTM4J'), and notes automatic key-to-ID resolution. This clearly differentiates the direction from sibling tools like qtm4j_get_linked_test_cases_for_requirement, which operates in the reverse direction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section gives concrete application contexts: checking requirement coverage, auditing traceability, retrieving keys for downstream operations, and release verification. It also documents the set_project_context prerequisite with a strong 'NEVER auto-select a project' warning, though it does not explicitly name alternatives or state when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_get_linked_requirements_for_test_cycleQTM4J: Get Linked Requirements for Test CycleA
Read-onlyIdempotent
Inspect

Retrieve Jira requirements linked to a QTM4J test cycle. Test cycle key is resolved to internal UID automatically.

Toolset: Test Cycles

Parameters:

  • cycleKey (string) required: Test cycle key in '{PROJECT_KEY}-TR-{id}' format (e.g., 'SCRUM-TR-1'). Resolved to the internal cycle UID automatically.

  • maxResults (number): Maximum results per page (1-100). Default: 50. (default: 50)

  • startAt (number): Zero-indexed offset for pagination. Default: 0. (default: 0)

  • sort (string): Sort pattern in 'field:asc|desc' format. Allowed fields: key, status, priority. Default: 'key:desc'.

Output Description: Paginated result with total, startAt, maxResults, and data array of linked requirement objects (id, key, summary, status, priority, issueType).

Use Cases: 1. Check which Jira stories or bugs are covered by a test cycle 2. Audit requirement traceability for a test cycle 3. Retrieve requirement keys linked to a cycle before a release 4. Verify correct requirements are linked to a test cycle

Examples:

  1. Get all requirements linked to a test cycle

{
  "cycleKey": "SCRUM-TR-1"
}

Expected Output: Paginated list of linked requirements with Jira metadata

  1. Get requirements sorted by priority with custom page size

{
  "cycleKey": "SCRUM-TR-5",
  "sort": "priority:asc",
  "maxResults": 20
}

Expected Output: Requirements linked to cycle sorted by priority ascending

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. CYCLE KEY FORMAT: '{PROJECT_KEY}-TR-{id}' β€” e.g. 'SCRUM-TR-1'. Resolved to internal UID automatically. 3. Allowed sort fields: key, status, priority. Default sort: 'key:desc'. 4. Paginate using startAt β€” increment by maxResults until startAt >= total.

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoSort pattern in 'field:asc|desc' format. Allowed fields: key, status, priority. Default: 'key:desc'.
startAtNoZero-indexed offset for pagination. Default: 0.
cycleKeyYesTest cycle key in '{PROJECT_KEY}-TR-{id}' format (e.g., 'SCRUM-TR-1'). Resolved to the internal cycle UID automatically.
maxResultsNoMaximum results per page (1-100). Default: 50.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesLinked requirements on this page.
totalYesTotal linked requirements (across all pages).
startAtYes
maxResultsYes

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as read-only, idempotent, and non-destructive. The description adds valuable behavior beyond that: cycleKey is automatically resolved to an internal UID, pagination should use startAt incremented by maxResults until startAt >= total, and project context is required. None of this contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured with clear sections for parameters, output, use cases, examples, and hints. The opening sentence is front-loaded and immediately informative. Some content duplicates the input schema and hints repeat parameter details, so it is not perfectly concise, but the operational guidance and examples justify the length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, prerequisites, parameter semantics, pagination behavior, output shape, and realistic use cases. It also provides multiple examples. With an output schema already present, the description does not need to explain return values in depth. Nothing an agent needs to call this tool correctly appears to be missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all four parameters, including defaults and ranges, so baseline is 3. The description adds value with examples showing exact payloads, the auto-resolution of cycleKey, and clarification of how startAt and maxResults drive pagination. It does not meaningfully explain parameters not already in the schema, but the examples and hints push it above baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Retrieve Jira requirements linked to a QTM4J test cycle.' This clearly states the direction of the relationship and distinguishes it from siblings like qtm4j_get_linked_test_cases_for_requirement or qtm4j_get_linked_requirements. The 'Toolset: Test Cycles' label reinforces the scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides concrete use cases such as checking requirement coverage, auditing traceability, and retrieving keys before release. It also states an explicit prerequisite: set_project_context must be called first and a project must never be auto-selected. It lacks explicit 'when not to use this tool' routing to alternatives, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_get_linked_test_cases_for_requirementQTM4J: Get Linked Test Cases for RequirementA
Read-onlyIdempotent
Inspect

Retrieve the test cases linked to a Jira requirement in QTM4J. Requirement key is resolved to internal ID automatically.

Toolset: Requirements

Parameters:

  • requirementKey (string) required: Jira requirement key (e.g., 'SCRUM-1'). Resolved to the internal Jira issue ID automatically.

  • filter (object): Optional filter to narrow down the linked test cases. projectId is auto-filled from the active project context.

  • fields (string): Comma-separated field names to include in each result. Allowed: summary, priority, status, estimatedTime, executed, description, assignee, reporter, labels, components, fixVersions, sprint, isAutomated, folder, updated, created, seqNo, flakyScore, passRateScore.

  • maxResults (number): Maximum results per page (1-100). Default: 50.

  • startAt (number): Zero-indexed offset for pagination. Default: 0.

  • sort (string): Sort in 'field:asc|desc' format. Allowed fields: key, summary, created, updated, estimatedTime, status, priority, latestVersionNo, seqNo, flakyScore, passRateScore.

Output Description: Paginated response with total, startAt, maxResults, and data array of linked test case objects.

Use Cases: 1. Check which test cases cover a Jira story or bug 2. Audit requirement traceability β€” find all test cases for a given requirement 3. Filter linked test cases by status or priority before a release 4. Retrieve test case keys to use in update or link operations

Examples:

  1. Get all test cases linked to a requirement

{
  "requirementKey": "SCRUM-1"
}

Expected Output: Paginated list of linked test cases

  1. Get high priority linked test cases

{
  "requirementKey": "SCRUM-5",
  "filter": {
    "priority": [
      "High"
    ]
  },
  "fields": "key,summary,status,priority",
  "sort": "key:asc"
}

Expected Output: Filtered high-priority test cases linked to requirement

  1. Paginate through linked test cases

{
  "requirementKey": "SCRUM-1",
  "maxResults": 20,
  "startAt": 20
}

Expected Output: Second page of linked test cases

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. REQUIREMENT KEY FORMAT: '{PROJECT_KEY}-{number}' β€” e.g. 'SCRUM-1'. 3. projectId in filter is auto-filled from the active project context β€” do not set it manually. 4. Use the fields param to limit response size β€” only request fields you need. 5. Paginate using startAt β€” increment by maxResults until startAt >= total. 6. filter.testCaseStatus can be 'active', 'archived', or 'deleted' to filter by archive state.

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoSort in 'field:asc|desc' format. Allowed fields: key, summary, created, updated, estimatedTime, status, priority, latestVersionNo, seqNo, flakyScore, passRateScore.
fieldsNoComma-separated field names to include in each result. Allowed: summary, priority, status, estimatedTime, executed, description, assignee, reporter, labels, components, fixVersions, sprint, isAutomated, folder, updated, created, seqNo, flakyScore, passRateScore.
filterNoOptional filter to narrow down the linked test cases. projectId is auto-filled from the active project context.
startAtNoZero-indexed offset for pagination. Default: 0.
maxResultsNoMaximum results per page (1-100). Default: 50.
requirementKeyYesJira requirement key (e.g., 'SCRUM-1'). Resolved to the internal Jira issue ID automatically.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesTest cases on this page
totalYesTotal test cases matching the filter (across all pages)
startAtYesOffset of this page
maxResultsYesPage size used for this response

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

While annotations already mark the operation as readOnly, idempotent, and non-destructive, the description adds meaningful behavioral context: the requirement key is automatically resolved to an internal ID, projectId is auto-filled from active project context, pagination follows a specific startAt/maxResults pattern, and the tool depends on a prior project context setup. This goes well beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but exceptionally well structured: a one-sentence summary, Toolset label, parameter breakdown, output description, use cases, examples, and hints. Each section earns its place, and the core purpose is front-loaded in the first sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with six parameters, nested filters, an output schema, and notable prerequisites, the description is complete. It covers when to use it, how to satisfy the project context prerequisite, key format, filtering, pagination, field selection, and example invocations. Nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds extra value by reinforcing auto-resolution of requirementKey, warning not to set projectId manually, documenting pagination defaults, and giving worked examples that show how filter and sort compose. It is not a full semantic manual, but it meaningfully supplements the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource pair: 'Retrieve the test cases linked to a Jira requirement in QTM4J.' This clearly identifies the operation and differentiates it from sibling tools like qtm4j_link_test_cases_to_requirement and qtm4j_get_linked_requirements, which are conceptually different operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear use cases, a stated prerequisite ('set_project_context must be called before this tool'), and a warning ('NEVER auto-select a project'). It does not explicitly name when not to use this tool versus a sibling, but the context is strong enough for an agent to select it correctly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_get_projectsQTM4J: Get ProjectsA
Read-onlyIdempotent
Inspect

Get all projects from QTM4J with optional filtering

Toolset: Projects

Parameters:

  • projectId (number): Filter by specific project ID

  • search (string): Search text for project key or project name

  • qmetryEnabled (boolean): Filter by QMetry enabled status

  • startAt (number): Zero-indexed starting position for pagination (default: 0)

  • maxResults (number): Maximum number of results per page (1-100) (default: 100)

Output Description: JSON object containing paginated list of projects with IDs, keys, names, and QMetry status, along with pagination metadata

Use Cases: 1. Discover all projects available in QTM4J instance 2. Get project IDs and keys for reference in other operations 3. Find specific projects by ID 4. Search projects by text in project key or name 5. Filter projects by QMetry integration status 6. List projects with pagination for large QTM4J instances 7. Retrieve complete project details (ID, key, name, avatarUrl, projectTypeKey, qmetryEnabled, favorite) 8. Validate project access and permissions 9. Browse available projects before performing other operations

Examples:

  1. Get all projects (default pagination - first 100)

{}

Expected Output: List of all projects with IDs, keys, and names (first 100 projects)

  1. Get the first 10 projects

{
  "maxResults": 10
}

Expected Output: List of first 10 projects with their details

  1. Get a specific project by ID

{
  "projectId": 10000
}

Expected Output: Single project with ID 10000 including key, name, and QMetry status

  1. Search projects by text in project key or name

{
  "search": "SCRUM"
}

Expected Output: Projects matching 'SCRUM' search text in their project keys or names

  1. Get only QMetry-enabled projects

{
  "qmetryEnabled": true
}

Expected Output: List of projects that have QMetry integration enabled

  1. Get projects with custom pagination (page 2)

{
  "startAt": 50,
  "maxResults": 50
}

Expected Output: Second page of projects (items 51-100) with their details

  1. Search QMetry-enabled projects by text

{
  "search": "TEST",
  "qmetryEnabled": true
}

Expected Output: QMetry-enabled projects containing 'TEST' in their project keys

Hints: 1. Project IDs are numeric (e.g., 10000), project keys are strings (e.g., 'SCRUM') 2. Use 'projectId' parameter to filter by a specific project ID 3. Use 'search' parameter to search by text in project key or project name 4. Use 'qmetryEnabled' parameter to filter projects by QMetry integration status 5. Response contains complete project details: id, key, name, favorite, avatarUrl, projectTypeKey, qmetryEnabled 6. Pagination: startAt is zero-indexed, maxResults max is 100, default is 100 7. Default (no parameters) returns first 100 projects 8. Use 'isLast' in response to check if more pages are available 9. To get next page: increment startAt by maxResults (0 β†’ 100 β†’ 200) 10. Use 'total' in response for total count of matching projects

ParametersJSON Schema
NameRequiredDescriptionDefault
searchNoSearch text for project key or project name
startAtNoZero-indexed starting position for pagination
projectIdNoFilter by specific project ID
maxResultsNoMaximum number of results per page (1-100)
qmetryEnabledNoFilter by QMetry enabled status

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesList of projects
totalYesTotal number of projects

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description is fully consistent with these. The description adds valuable behavioral context beyond annotations: pagination defaults, use of isLast and total, parameter interactions, and the complete project fields returned. There is no contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured with clear sections: summary, parameters, output, use cases, examples, and hints. It is front-loaded with the core purpose. Some repetition exists between use cases, examples, and hints, but every section contributes actionable guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is fully complete for a read-only list tool: all five optional parameters are explained, pagination behavior is specified, examples cover typical and combined usages, and expected output contents are described. With output schema available and annotations covering safety, nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds meaningful value through examples showing how parameters combine, clarification that startAt is zero-indexed, maxResults cap/default, and hints about project key versus ID. This goes beyond the raw schema definitions without being redundant.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool gets all projects from QTM4J with optional filtering, which identifies the specific resource and operation. It does not explicitly differentiate from similar sibling tools like qmetry_fetch_qmetry_list_projects or zephyr_get_projects, but the QTM4J branding and detailed scope make the purpose clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides extensive use cases, examples, and hints explaining when to use this tool, such as discovering projects, finding project IDs, filtering by QMetry status, and paginating results. It does not explicitly state when to prefer alternatives, but the context is clear and complete for this tool's own usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_get_test_stepsQTM4J: Get Test StepsA
Read-onlyIdempotent
Inspect

Get test steps for a test case by its key and version. Accepts the human-readable key (e.g. 'SCRUM-TC-145') and resolves it to the internal ID automatically.

Toolset: Test Cases

Parameters:

  • key (string) required: Test case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'. PROJECT_KEY is the Jira project key (e.g. 'SCRUM'). The number is the test case counter within that project (auto-incremented, not related to seqNo). Obtain keys from the search_test_cases tool or directly from QTM4J.

  • versionNo (number): Test case version number. Defaults to the latest version if omitted. Obtain from search_test_cases response field: version.versionNo

  • filter (object): Text filters for test steps β€” each field performs a substring match. Multiple fields are combined with AND.

  • startAt (number): Zero-indexed offset for pagination (URL query param). Default: 0. (default: 0)

  • maxResults (number): Number of steps per page (URL query param). Default: 50. Maximum: 100. (default: 50)

  • sort (string): Sort pattern (URL query param). Format: 'fieldName:order'. Sortable fields: stepDetails, testData, seqNo, expectedResult. Order values: 'asc' or 'desc'. Example: 'seqNo:asc'

Output Description: JSON object with total (total matching steps), startAt, maxResults, and data (array of step objects). Each step has: id, seqNo, stepDetails, testData, expectedResult, attachmentCount. Shared steps also have a 'shareable' object containing shareableTestcaseUID and shareableTestSteps array.

Use Cases: 1. View all steps of a test case before executing it 2. Review steps for a specific test case version 3. Filter steps by action text, test data, or expected result 4. Get steps for a test case found via search_test_cases 5. Inspect shared (reusable) steps embedded in a test case 6. Sort steps by sequence number to view them in execution order 7. Paginate through test cases that have a large number of steps

Examples:

  1. Get all steps for a test case (latest version)

{
  "key": "SCRUM-TC-145"
}

Expected Output: All steps for SCRUM-TC-145 with stepDetails, testData, expectedResult, and any shared step blocks

  1. Get steps for a specific version

{
  "key": "SCRUM-TC-145",
  "versionNo": 2
}

Expected Output: Steps for version 2 of SCRUM-TC-145

  1. Get steps in execution order

{
  "key": "SCRUM-TC-85",
  "sort": "seqNo:asc"
}

Expected Output: All steps sorted by sequence number ascending

  1. Filter steps by action text

{
  "key": "SCRUM-TC-32",
  "filter": {
    "stepDetails": "Open the application"
  }
}

Expected Output: Steps whose stepDetails contain 'Open the application'

  1. Filter steps by expected result

{
  "key": "SCRUM-TC-65",
  "filter": {
    "expectedResult": "logged in successfully"
  }
}

Expected Output: Steps whose expectedResult contains 'logged in successfully'

  1. Filter steps by test data

{
  "key": "SCRUM-TC-125",
  "filter": {
    "testData": "Username: user1"
  }
}

Expected Output: Steps with testData containing 'Username: user1'

  1. Paginate through many steps

{
  "key": "SCRUM-TC-105",
  "startAt": 0,
  "maxResults": 10,
  "sort": "seqNo:asc"
}

Expected Output: First 10 steps in sequence order

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. KEY FORMAT: '{PROJECT_KEY}-TC-{number}' β€” e.g. 'SCRUM-TC-145'. PROJECT_KEY is the Jira project key; the number is the test case counter within that project (auto-incremented, not the same as seqNo). 3. VERSION: versionNo defaults to the latest version. Get the version number from search_test_cases response: version.versionNo. 4. Use search_test_cases to discover test case keys before calling this tool. 5. SHAREABLE STEPS: Steps with a non-null 'shareable' field are references to shared/reusable test cases. The 'shareable.shareableTestSteps' array contains the embedded sub-steps with decimal seqNo values (e.g. '1.1', '1.2'). 6. FILTER: Each filter field is a substring match (case-insensitive). Multiple fields combine with AND. 7. SORT: 'seqNo:asc' shows steps in their natural execution order. Allowed sort fields: stepDetails, testData, seqNo, expectedResult. 8. PAGINATION: startAt and maxResults are URL query params. Default page size is 50, maximum is 100.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyYesTest case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'. PROJECT_KEY is the Jira project key (e.g. 'SCRUM'). The number is the test case counter within that project (auto-incremented, not related to seqNo). Obtain keys from the search_test_cases tool or directly from QTM4J.
sortNoSort pattern (URL query param). Format: 'fieldName:order'. Sortable fields: stepDetails, testData, seqNo, expectedResult. Order values: 'asc' or 'desc'. Example: 'seqNo:asc'
filterNoText filters for test steps β€” each field performs a substring match. Multiple fields are combined with AND.
startAtNoZero-indexed offset for pagination (URL query param). Default: 0.
versionNoNoTest case version number. Defaults to the latest version if omitted. Obtain from search_test_cases response field: version.versionNo
maxResultsNoNumber of steps per page (URL query param). Default: 50. Maximum: 100.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesTest steps on this page
totalYesTotal steps matching the filter (across all pages)
startAtYesOffset of this page
maxResultsYesPage size used for this response

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnlyHint/idempotentHint/destructiveHint annotations, the description discloses meaningful behavior: key auto-resolution to internal ID, default-to-latest version, substring case-insensitive filters combined with AND, pagination defaults (50/100), and shared-step representation with decimal seqNo values. This gives the agent accurate expectations without contradicting the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and organized into Parameters, Output Description, Use Cases, Examples, and Hints. It is verbose and repeats key-format, filter, and sort details across sections, but the structured format keeps it navigable for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex (nested filter object, versioning, pagination, sorting, shared steps), and the description covers all of it, including a required project-context prerequisite, key discovery via search_test_cases, and output semantics. An agent can invoke this tool correctly with the information provided, and the presence of an output schema further reduces missing context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description goes well beyond the schema: it explains the '{PROJECT_KEY}-TC-{number}' key format with examples, clarifies filter substring and AND behavior, details the 'fieldName:order' sort syntax with allowed fields, and documents the output shape (total, startAt, maxResults, data, shareable). This substantially increases parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get test steps for a test case by its key and version.' It clearly describes the function and key format, but it does not explicitly differentiate itself from sibling tools that also retrieve test steps from other providers (e.g., zephyr_get_test_case_steps, qmetry_fetch_test_case_steps), so it stops short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear 'Use Cases' list and workflow hints, including the prerequisite 'set_project_context must be called before this tool' and 'Use search_test_cases to discover test case keys before calling this tool.' It does not explicitly mention when not to use the tool or name alternative step-retrieval tools, but the context is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_search_linked_test_cases_in_test_cycleQTM4J: Search Linked Test Cases in Test CycleA
Read-onlyIdempotent
Inspect

Search and filter test case executions linked to a QTM4J test cycle. Supports pagination, field selection, sorting, and rich filter criteria.

Toolset: Test Cycles

Parameters:

  • cycleKey (string) required: Test Cycle key in '{PROJECT_KEY}-TR-{id}' format (e.g., 'SCRUM-TR-1'). Resolved to the internal cycle UID automatically.

  • fields (array): Fields to include in each result object. Allowed: id, key, summary, description, executionResult, status, priority, environment, tcWithDefects, estimatedTime, actualTime, createdOn, updatedOn, sprint, seqNo, latestTcExecutionId, customFields, flakyScore, passRateScore. Omit to return all fields.

  • maxResults (number): Maximum results per page (1-100). Default: 50. (default: 50)

  • sort (string): Sort pattern in 'field:asc|desc' format (e.g., 'key:desc'). Allowed sort fields: id, key, summary, description, executionResult, status, priority, environment, tcWithDefects, estimatedTime, actualTime, createdOn, updatedOn, sprint, flakyScore, passRateScore.

  • startAt (number): Zero-indexed offset for pagination. Default: 0. (default: 0)

  • filter (object): Optional filter criteria to narrow down results. projectId is auto-filled from the active project context.

Output Description: JSON object with total (total matching executions), startAt, maxResults, and data (array of test case execution objects for this page).

Use Cases: 1. List all test cases linked to a test cycle 2. Find failed or blocked test case executions in a cycle 3. Search for test cases in a cycle by execution result (Pass, Fail, Blocked) 4. Filter test cases in a cycle by priority or status 5. Retrieve test cases assigned to a specific executor in a cycle 6. Get test cases with defects in a cycle 7. Paginate through large test cycle execution lists 8. Request only specific fields to reduce response size

Examples:

  1. List all test cases in a cycle

{
  "cycleKey": "SCRUM-TR-1"
}

Expected Output: Paginated list of test case executions in the cycle (first 50 results)

  1. Find failed test cases in a cycle

{
  "cycleKey": "SCRUM-TR-1",
  "filter": {
    "executionResult": [
      "Fail"
    ]
  },
  "fields": [
    "key",
    "summary",
    "executionResult",
    "priority"
  ]
}

Expected Output: Test case executions with Fail result

  1. Search with pagination and sort

{
  "cycleKey": "SCRUM-TR-5",
  "maxResults": 25,
  "startAt": 0,
  "sort": "key:asc",
  "filter": {
    "status": [
      "To Do"
    ]
  }
}

Expected Output: First 25 To Do test cases in the cycle sorted by key ascending

  1. Filter by execution assignee and environment

{
  "cycleKey": "SCRUM-TR-2",
  "filter": {
    "executionAssignee": [
      "5b10a2844c20165700ede21f"
    ],
    "environment": [
      "Staging"
    ]
  },
  "fields": [
    "key",
    "summary",
    "executionResult",
    "environment",
    "actualTime"
  ]
}

Expected Output: Test cases assigned to the specified user in the Staging environment

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. CYCLE KEY FORMAT: '{PROJECT_KEY}-TR-{id}' β€” e.g. 'SCRUM-TR-1'. Resolved to internal UID automatically. 3. projectId in filter is auto-filled from the active project context β€” do not set it manually. 4. fields is sent as a query parameter; filter is sent in the request body. 5. Allowed fields: id, key, summary, description, executionResult, status, priority, environment, tcWithDefects, estimatedTime, actualTime, createdOn, updatedOn, sprint, seqNo, latestTcExecutionId, customFields, flakyScore, passRateScore. 6. Allowed sort fields: id, key, summary, description, executionResult, status, priority, environment, tcWithDefects, estimatedTime, actualTime, createdOn, updatedOn, sprint, flakyScore, passRateScore. 7. Date range format for filter fields: 'dd/mmm/yyyy,dd/mmm/yyyy' (e.g., '01/Jan/2024,31/Mar/2024'). 8. maxResults defaults to 50, maximum is 100. Use startAt to paginate. 9. executionResult filter accepts values like 'Pass', 'Fail', 'Blocked', 'Unexecuted'.

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoSort pattern in 'field:asc|desc' format (e.g., 'key:desc'). Allowed sort fields: id, key, summary, description, executionResult, status, priority, environment, tcWithDefects, estimatedTime, actualTime, createdOn, updatedOn, sprint, flakyScore, passRateScore.
fieldsNoFields to include in each result object. Allowed: id, key, summary, description, executionResult, status, priority, environment, tcWithDefects, estimatedTime, actualTime, createdOn, updatedOn, sprint, seqNo, latestTcExecutionId, customFields, flakyScore, passRateScore. Omit to return all fields.
filterNoOptional filter criteria to narrow down results. projectId is auto-filled from the active project context.
startAtNoZero-indexed offset for pagination. Default: 0.
cycleKeyYesTest Cycle key in '{PROJECT_KEY}-TR-{id}' format (e.g., 'SCRUM-TR-1'). Resolved to the internal cycle UID automatically.
maxResultsNoMaximum results per page (1-100). Default: 50.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesTest case execution entries on this page.
totalYesTotal test case executions matching the filter (across all pages).
startAtYesOffset of this page.
maxResultsYesPage size used for this response.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as read-only and idempotent, and the description adds substantial behavioral context: the cycleKey is resolved to an internal UID, projectId is auto-filled from project context, fields is sent as a query parameter while filter goes in the body, and date ranges use a specific format. This is exactly the kind of operational detail an agent needs beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well structured with clear headings: summary, parameters, output, use cases, examples, and hints. It is front-loaded with the core purpose, but it is somewhat long and repeats allowed fields, sort fields, and default values across the parameter section and hints, which keeps it from being maximally concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a complex nested filter object, output schema, and six parameters, the description is highly complete. It covers prerequisites, key formats, pagination, sorting, filtering, output shape, and concrete examples, leaving little ambiguity for an agent deciding how to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds practical meaning beyond the schema: examples show realistic filter combinations, the projectId auto-fill behavior is emphasized, and hints clarify request-body vs query-parameter placement. Most parameter semantics are already in the schema, so this is solid but not exceptional.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Search and filter test case executions linked to a QTM4J test cycle.' It also names the toolset ('Test Cycles') and explicitly mentions pagination, field selection, sorting, and filters, which clearly distinguishes it from sibling tools like qtm4j_search_test_cases or qtm4j_get_linked_test_cases_for_requirement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use cases list concrete scenarios such as listing test cases in a cycle, finding failed or blocked executions, and filtering by assignee or environment. The description also gives an explicit prerequisite: set_project_context must be called first and project selection must never be automated. It does not explicitly name alternatives or say when not to use the tool, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_search_test_casesQTM4J: Search Test CasesA
Read-onlyIdempotent
Inspect

Search and filter test cases in a QTM4J project with support for pagination, field selection, and sorting.

Toolset: Test Cases

Parameters:

  • filter (object): Filter criteria β€” multiple fields are combined with AND; multiple values within one field use OR.

  • fields (array): Fields to include in each result object. If omitted, all fields are returned. Available fields: key, summary, description, priority, status, assignee, isAutomated, reporter, estimatedTime, labels, components, fixVersions, sprint, folders, updated, created, executed, flakyScore, passRateScore, aiGenerated, precondition, orderNo, seqNo, version. Example: ['key', 'summary', 'status', 'priority', 'assignee']

  • startAt (number): Zero-indexed offset for pagination (URL query param). First page: 0. Second page: 50 (when maxResults=50). Default: 0. (default: 0)

  • maxResults (number): Number of results per page (URL query param). Default: 50. Maximum: 50 (backend enforced). To page through results, increment startAt by 50 until startAt >= total. (default: 50)

  • sort (string): Sort pattern sent as a URL query param. Format: 'fieldName:order'. For multiple fields, comma-separate: 'priority:asc,created:desc'. Order values: 'asc' (oldest/lowest first) or 'desc' (newest/highest first). Sortable fields: key, summary, created, updated, status, priority, executed. Examples: 'created:desc', 'key:asc', 'priority:desc,created:asc'

Output Description: JSON object with total (total matching test cases), startAt, maxResults, and data (array of test case objects for this page).

Use Cases: 1. Search all test cases in a project 2. Filter test cases by status (e.g., 'Done', 'To Do', 'In Progress') 3. Filter test cases by priority (e.g., 'High', 'Medium', 'Low') 4. Filter test cases by labels and components 5. Search test cases by text in summary and description 6. Filter test cases by assignee or reporter 7. Filter test cases by creation/update date ranges 8. Filter test cases by automation status 9. Request only specific fields to reduce response size 10. Sort results using the sort query param (e.g., 'created:desc', 'priority:asc') 11. Paginate through large result sets using startAt and maxResults 12. Combine multiple filters for complex queries 13. Find all failed test cases that need attention (by status and execution date) 14. Get test cases for sprint planning (filter by sprint and status) 15. Audit test coverage by searching for untested areas (filter by executed date) 16. Find all manual test cases assigned to a specific tester 17. Generate test reports by filtering and sorting test cases 18. Track test case changes over time (filter by update date range) 19. Identify high-priority test cases pending review 20. Search for test cases related to specific features (using searchText) 21. Find duplicate or similar test cases (using searchText) 22. Get automated vs manual test distribution (filter by isAutomated) 23. Monitor test execution trends (filter by executed date ranges) 24. Prepare test execution schedules (filter by assignee and priority)

Examples:

  1. Search all test cases in the project

{}

Expected Output: Paginated list of all test cases with all fields (first 50 results)

  1. Filter test cases by status

{
  "filter": {
    "status": [
      "Done"
    ]
  }
}

Expected Output: List of test cases with 'Done' status

  1. Filter by multiple statuses and priorities with specific fields

{
  "filter": {
    "status": [
      "Done",
      "To Do"
    ],
    "priority": [
      "High",
      "Medium"
    ]
  },
  "fields": [
    "key",
    "summary",
    "status",
    "priority",
    "assignee"
  ]
}

Expected Output: Test cases matching the filters with only the selected fields returned

  1. Search test cases by text in summary/description

{
  "filter": {
    "searchText": "login functionality"
  }
}

Expected Output: Test cases containing 'login functionality' in summary or description

  1. Filter by labels and components

{
  "filter": {
    "labels": [
      "Release_1",
      "Sprint 1"
    ],
    "components": [
      "UI",
      "Cloud"
    ]
  }
}

Expected Output: Test cases tagged with the specified labels and components

  1. Filter by assignee and automation status

{
  "filter": {
    "assignee": [
      "712020:ddc8e24b-2de7-404b-b9ed-3d7b241e2ced"
    ],
    "isAutomated": false
  }
}

Expected Output: Manual test cases assigned to the specified user

  1. Filter by creation date range

{
  "filter": {
    "createdOnFrom": "01/Jan/2026",
    "createdOnTo": "31/Dec/2026"
  }
}

Expected Output: Test cases created during 2026

  1. Paginate and sort by creation date (newest first)

{
  "filter": {
    "status": [
      "Done"
    ]
  },
  "startAt": 0,
  "maxResults": 50,
  "sort": "created:desc"
}

Expected Output: First 50 'Done' test cases sorted by creation date, newest first

  1. Get all available fields for test cases

{
  "filter": {},
  "fields": [
    "key",
    "summary",
    "description",
    "priority",
    "status",
    "assignee",
    "isAutomated",
    "reporter",
    "estimatedTime",
    "labels",
    "components",
    "fixVersions",
    "sprint",
    "folders",
    "updated",
    "created",
    "executed",
    "flakyScore",
    "passRateScore",
    "aiGenerated",
    "precondition",
    "orderNo",
    "seqNo",
    "version"
  ]
}

Expected Output: Test cases with all available fields explicitly requested

  1. Filter by folder and fix version

{
  "filter": {
    "folders": [
      123,
      456
    ],
    "fixVersions": [
      789
    ]
  }
}

Expected Output: Test cases in the specified folders and fix versions

  1. Complex filter: multiple criteria combined with multi-field sort

{
  "filter": {
    "status": [
      "Done",
      "In Progress"
    ],
    "priority": [
      "High"
    ],
    "labels": [
      "Release_1"
    ],
    "isAutomated": false,
    "createdOnFrom": "01/Apr/2026",
    "createdOnTo": "30/Apr/2026"
  },
  "fields": [
    "key",
    "summary",
    "status",
    "priority",
    "created"
  ],
  "sort": "priority:asc,created:desc"
}

Expected Output: High-priority manual test cases created in April 2026 with 'Done' or 'In Progress' status, sorted by priority ascending then creation date descending

  1. Find all automated test cases for CI/CD pipeline

{
  "filter": {
    "isAutomated": true,
    "status": [
      "Done",
      "In Progress"
    ]
  },
  "fields": [
    "key",
    "summary",
    "status",
    "labels",
    "components"
  ]
}

Expected Output: Up to 50 automated test cases ready for execution in CI/CD

  1. Sprint planning: get pending test cases for a team, sorted by priority

{
  "filter": {
    "status": [
      "To Do",
      "In Progress"
    ],
    "assignee": [
      "712020:ddc8e24b-2de7-404b-b9ed-3d7b241e2ced",
      "712020:b8479b55-6d23-478c-a2ad-4c8ce176e1fc"
    ],
    "priority": [
      "High",
      "Medium"
    ]
  },
  "fields": [
    "key",
    "summary",
    "status",
    "priority",
    "assignee",
    "estimatedTime"
  ],
  "sort": "priority:desc"
}

Expected Output: Pending high and medium priority test cases assigned to the team, sorted by priority descending

  1. Test coverage report: find completed cases sorted oldest first

{
  "filter": {
    "status": [
      "Done"
    ]
  },
  "fields": [
    "key",
    "summary",
    "priority",
    "created",
    "assignee"
  ],
  "sort": "created:asc"
}

Expected Output: Completed test cases sorted by creation date, oldest first

  1. Find test cases by keyword for regression testing

{
  "filter": {
    "searchText": "authentication login",
    "status": [
      "Done"
    ]
  },
  "fields": [
    "key",
    "summary",
    "description",
    "labels",
    "components"
  ]
}

Expected Output: All completed test cases related to authentication/login functionality

  1. Weekly test execution summary

{
  "filter": {
    "executedOnFrom": "27/Apr/2026",
    "executedOnTo": "03/May/2026"
  },
  "fields": [
    "key",
    "summary",
    "status",
    "executed",
    "passRateScore",
    "flakyScore"
  ],
  "sort": "executed:desc"
}

Expected Output: Test cases executed in the past week with their pass rates and flaky scores, sorted most-recent first

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. REQUEST STRUCTURE: filter goes in the request body; fields, sort, startAt, and maxResults are URL query parameters. 3. The 'projectId' inside filter is auto-populated from the active project context if not provided. 4. All filter values accept string names directly β€” no ID resolution needed (e.g., status: ['Done'], priority: ['High']). 5. FIELDS: Pass as an array (sent as comma-separated URL param). Example: { fields: ['key', 'summary', 'status'] }. Omit to return all fields. 6. Available fields: key, summary, description, priority, status, assignee, isAutomated, reporter, estimatedTime, labels, components, fixVersions, sprint, folders, updated, created, executed, flakyScore, passRateScore, aiGenerated, precondition, orderNo, seqNo, version 7. SORTING: Use 'sort' with format 'fieldName:order' (asc/desc). Multiple fields: 'priority:asc,created:desc'. Sortable fields: key, summary, created, updated, status, priority, executed. 8. PAGINATION: startAt (default: 0) and maxResults (default: 50, max: 50) are sent as URL query params. Increment startAt by 50 to get the next page. Stop when startAt >= total. 9. Date format for all filter date fields: 'dd/MMM/yyyy' (e.g., '17/Apr/2026', '01/Jan/2026'). Case-sensitive. 10. FILTER LOGIC: Multiple values within one filter field use OR (status: ['Done', 'To Do'] = Done OR To Do). 11. FILTER LOGIC: Different filter fields are combined with AND (status + priority = both must match). 12. The 'searchText' filter searches both summary and description fields (case-insensitive). To get details of a specific test case by its key (e.g., 'SCRUM-TC-145'), pass the key as filter.searchText β€” there is no separate key filter field. 13. For assignee/reporter filters, use Jira account IDs (format: '712020:uuid'). Multiple IDs = OR logic. 14. Omitting filter entirely returns all test cases in the active project (paginated).

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoSort pattern sent as a URL query param. Format: 'fieldName:order'. For multiple fields, comma-separate: 'priority:asc,created:desc'. Order values: 'asc' (oldest/lowest first) or 'desc' (newest/highest first). Sortable fields: key, summary, created, updated, status, priority, executed. Examples: 'created:desc', 'key:asc', 'priority:desc,created:asc'
fieldsNoFields to include in each result object. If omitted, all fields are returned. Available fields: key, summary, description, priority, status, assignee, isAutomated, reporter, estimatedTime, labels, components, fixVersions, sprint, folders, updated, created, executed, flakyScore, passRateScore, aiGenerated, precondition, orderNo, seqNo, version. Example: ['key', 'summary', 'status', 'priority', 'assignee']
filterNoFilter criteria β€” multiple fields are combined with AND; multiple values within one field use OR.
startAtNoZero-indexed offset for pagination (URL query param). First page: 0. Second page: 50 (when maxResults=50). Default: 0.
maxResultsNoNumber of results per page (URL query param). Default: 50. Maximum: 50 (backend enforced). To page through results, increment startAt by 50 until startAt >= total.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesTest cases on this page
totalYesTotal test cases matching the filter (across all pages)
startAtYesOffset of this page
maxResultsYesPage size used for this response

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the read-only/idempotent annotations, the description discloses backend-enforced maxResults=50, auto-populated projectId, AND/OR filter combination, case-sensitive date format, and searchText semantics. These are behavioral details an agent needs and are not stated by the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with clear headings and front-loaded purpose, but it is heavily redundant: 24 use cases largely overlap with 16 examples and the hints. Many lines could be condensed without losing information, so it does not meet the 'every sentence earns its place' bar.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with a nested filter object and no required parameters, the description covers the full calling context: prerequisite, parameter semantics, pagination loop, sort format, filter logic, response shape, and representative examples. Nothing needed to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all parameters in detail (100% coverage), so the baseline is 3. The description adds meaningful operational semantics: projectId auto-population, URL-query versus body placement, no ID resolution for filter values, and pagination increment guidance. This exceeds the schema but shares much of its content.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific verb ('Search and filter'), resource ('test cases in a QTM4J project'), and supporting behaviors ('pagination, field selection, and sorting'). It is immediately distinguishable from siblings like qtm4j_search_test_cycles or qtm4j_get_test_steps, and the Toolset label reinforces scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives extensive usage context through 24 use cases and explicit prerequisites (set_project_context, never auto-select a project). It does not explicitly name alternative tools or say when not to use this tool, but the search/filter scope and required context are clear enough for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_search_test_cyclesQTM4J: Search Test CyclesA
Read-onlyIdempotent
Inspect

Search for test cycles in a QTM4J project by status, owner, folder, date range, or keyword. projectId is injected automatically from the active project context.

Toolset: Test Cycles

Parameters:

  • filter (object): Filter criteria β€” multiple fields are combined with AND; multiple values within one field use OR.

  • fields (array): Fields to include in each result object. If omitted, server returns its default set (NOTE: plannedStartDate and plannedEndDate are NOT in the default response β€” include them explicitly when needed). Available fields: key, summary, description, status, priority, assignee, reporter, isAutomated, plannedStartDate, plannedEndDate, labels, components, fixVersions, sprint, defectCount, estimatedTime, actualTime, created, updated. Example: ['key', 'summary', 'status', 'assignee', 'plannedStartDate', 'plannedEndDate']

  • startAt (number): Zero-indexed offset for pagination (URL query param). Default: 0. (default: 0)

  • maxResults (number): Number of results per page (URL query param). Default: 20. Maximum: 100. To page through results, increment startAt by 20 until startAt >= total. (default: 20)

  • sort (string): Sort pattern sent as a URL query param. Format: 'fieldName:order'. Default: 'key:asc'. Order values: 'asc' (lowest/oldest first) or 'desc' (highest/newest first). Sortable fields: key, summary, status, plannedStartDate, plannedEndDate, defectCount. Examples: 'key:asc', 'plannedStartDate:desc' (default: "key:asc")

Output Description: JSON object with total (matching cycles across all pages), startAt, maxResults, and data (array of test cycle objects for this page). Each item always has id and key. Other fields depend on what was requested via the fields parameter.

Use Cases: 1. Find test cycles by status, priority, assignee, reporter, or folder 2. Find test cycles by planned execution date range (plannedStartDate / plannedEndDate) 3. Find test cycles created or updated within a date range (createdOn / updatedOn) 4. Search test cycles by keyword across key, summary, and description 5. Paginate, sort, and select specific response fields

Examples:

  1. Find all in-progress and to-do cycles

{
  "filter": {
    "status": [
      "In Progress",
      "To Do"
    ]
  }
}

Expected Output: Paginated list of matching test cycles

  1. Find cycles owned by a specific user

{
  "filter": {
    "assignee": [
      "5b10a2844c20165700ede21f"
    ]
  }
}

Expected Output: Test cycles assigned to that user

  1. Find cycles with planned start date in a range, requesting date fields explicitly

{
  "filter": {
    "plannedStartDate": "01/Apr/2026,30/Apr/2026"
  },
  "fields": [
    "key",
    "summary",
    "status",
    "assignee",
    "plannedStartDate",
    "plannedEndDate"
  ]
}

Expected Output: Cycles with planned start date in April 2026 including date fields

  1. Keyword search with sort, pagination, and selected fields

{
  "filter": {
    "searchText": "regression"
  },
  "fields": [
    "key",
    "summary",
    "status",
    "assignee"
  ],
  "sort": "plannedStartDate:asc",
  "startAt": 0,
  "maxResults": 25
}

Expected Output: Cycles matching 'regression', sorted by planned start date

  1. Find cycles created last week

{
  "filter": {
    "createdOn": "01/May/2026,07/May/2026"
  },
  "fields": [
    "key",
    "summary",
    "status",
    "assignee"
  ],
  "sort": "key:asc"
}

Expected Output: Test cycles created between 01 May and 07 May 2026

  1. Find high-priority cycles updated recently by reporter

{
  "filter": {
    "priority": [
      "High"
    ],
    "reporter": [
      "5b10a2844c20165700ede21f"
    ],
    "updatedOn": "01/May/2026,21/May/2026"
  },
  "fields": [
    "key",
    "summary",
    "status",
    "priority",
    "assignee"
  ]
}

Expected Output: High-priority cycles updated in May 2026 reported by that user

  1. All filters combined with explicit field selection

{
  "filter": {
    "status": [
      "In Progress"
    ],
    "priority": [
      "High",
      "Medium"
    ],
    "assignee": [
      "5b10a2844c20165700ede21f"
    ],
    "folderId": 109987,
    "plannedStartDate": "02/Apr/2026,15/May/2026",
    "searchText": "regression"
  },
  "fields": [
    "key",
    "summary",
    "status",
    "priority",
    "assignee",
    "plannedStartDate"
  ],
  "sort": "plannedStartDate:asc",
  "maxResults": 25
}

Expected Output: Test cycles matching all specified filters with selected fields

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. SUPPORTED FILTER FIELDS: status, priority, assignee, reporter, folderId, labels, components, plannedStartDate, plannedEndDate, searchText, createdOn, updatedOn, isAutomated, aiGenerated. Do NOT use any other filter field names. 3. DATE FILTERS: createdOn = creation date; updatedOn = last-updated date; plannedStartDate / plannedEndDate = planned execution window. Format: 'dd/MMM/yyyy,dd/MMM/yyyy' e.g. '01/May/2026,21/May/2026'. Month is case-sensitive. 'Created last week' β†’ createdOn, NOT plannedStartDate. 4. FIELDS: Pass as an array to select what to return. plannedStartDate and plannedEndDate are NOT in the default response β€” include them explicitly. Available: key, summary, description, status, priority, assignee, reporter, isAutomated, plannedStartDate, plannedEndDate, labels, components, fixVersions, sprint, defectCount, estimatedTime, actualTime, created, updated. 5. REQUEST STRUCTURE: filter β†’ request body; fields, sort, startAt, maxResults β†’ URL query params. 6. SORT: Allowed fields: key, summary, status, plannedStartDate, plannedEndDate, defectCount. Format: 'fieldName:asc' or 'fieldName:desc' e.g. 'plannedStartDate:asc'. 7. FOLDER ID: folderId in fields.testCycle and fields.testCase is a numeric ID. Tell the user they can get it by right-clicking the target folder in QTM4J and selecting 'Copy Folder Id'. Always ask the user for the numeric ID directly β€” never try to look it up.

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoSort pattern sent as a URL query param. Format: 'fieldName:order'. Default: 'key:asc'. Order values: 'asc' (lowest/oldest first) or 'desc' (highest/newest first). Sortable fields: key, summary, status, plannedStartDate, plannedEndDate, defectCount. Examples: 'key:asc', 'plannedStartDate:desc'key:asc
fieldsNoFields to include in each result object. If omitted, server returns its default set (NOTE: plannedStartDate and plannedEndDate are NOT in the default response β€” include them explicitly when needed). Available fields: key, summary, description, status, priority, assignee, reporter, isAutomated, plannedStartDate, plannedEndDate, labels, components, fixVersions, sprint, defectCount, estimatedTime, actualTime, created, updated. Example: ['key', 'summary', 'status', 'assignee', 'plannedStartDate', 'plannedEndDate']
filterNoFilter criteria β€” multiple fields are combined with AND; multiple values within one field use OR.
startAtNoZero-indexed offset for pagination (URL query param). Default: 0.
maxResultsNoNumber of results per page (URL query param). Default: 20. Maximum: 100. To page through results, increment startAt by 20 until startAt >= total.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataYesTest cycles on this page
totalYesTotal matching test cycles across all pages
startAtYesOffset of this page
maxResultsYesPage size used for this response

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/idempotent/non-destructive annotations, the description discloses important behaviors: projectId is auto-injected, plannedStartDate/plannedEndDate are not returned by default, date formats are case-sensitive, filter fields combine with AND/OR semantics, and the response shape is described. This gives an agent accurate expectations about what the tool does and what it returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded, opening with a concise summary and using clear headers for output, use cases, examples, and hints. However, it is noticeably long and contains some redundancy: parameter details and hints repeat the field lists, sort rules, and date formats already present in the schema. This is minor given the complexity of the tool, but it keeps the score from being a perfect 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's filter complexity, nested schema, and many available fields, the description is essentially complete. It covers supported filter fields, output shape, pagination behavior, default values, date semantics, example requests, and the critical prerequisite that project context must already be selected. An agent has everything needed to construct valid calls and interpret responses.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Even though schema coverage is 100%, the description adds substantial semantic value: it explains that filter fields combine with AND while values within a field use OR, that date filters use a specific 'dd/MMM/yyyy,dd/MMM/yyyy' format, that planned dates must be explicitly requested, and that sort uses 'fieldName:order.' The examples demonstrate realistic parameter combinations and expected outputs, going well beyond the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Search for test cycles in a QTM4J project by status, owner, folder, date range, or keyword.' It also identifies the toolset (Test Cycles) and clearly separates this read/search operation from create/update cycle tools and from test-case search tools in the sibling list. This is more than a restatement of the title; it names the primary dimensions of search.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for use: the prerequisite 'set_project_context must be called before this tool,' a list of use cases, and explicit guardrails such as 'NEVER auto-select a project' and 'Do NOT use any other filter field names.' It does not explicitly name alternatives like qtm4j_create_test_cycle or qtm4j_search_linked_test_cases_in_test_cycle, so it stops short of true when-to-use vs. alternative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_set_project_contextQTM4J: Set Project ContextA
Idempotent
Inspect

Set the active QTM4J project for the current session. Must be called before any project-specific operation. Pre-loads priority and status values so you can map user-provided names to valid options via NLP.

Toolset: Projects

Parameters:

  • projectKey (string) required: Project key (e.g., 'SCRUM'). Use the get_projects tool to discover available project keys.

Output Description: JSON object with projectId, projectKey, projectName, confirmation message, and availableFields. availableFields contains priority and status options for NLP mapping in subsequent tool calls.

Use Cases: 1. Set the active project at the start of a new conversation 2. Switch to a different project mid-conversation 3. Validate that a project key exists before performing operations 4. Establish project context required by all project-specific tools

Examples:

  1. Set SCRUM project as active

{
  "projectKey": "SCRUM"
}

Expected Output: Project context set to SCRUM (ID: 10000)

  1. Switch to AD project

{
  "projectKey": "AD"
}

Expected Output: Project context switched to AD

Hints: 1. CRITICAL: This tool MUST be called before ANY project-specific tool. 2. NEVER auto-select a project. If the user does not specify a project key, call get_projects first, present the list to the user and ask them to choose. Do NOT pick one on their behalf. 3. The project key must be an exact match (e.g., 'SCRUM', not 'scrum project'). 4. After calling this tool, use the availableFields in the response to map user input via NLP (e.g. user says 'Major' β†’ send 'High', user says 'Critical' β†’ send 'Blocker'). 5. Switching projects clears the cached field metadata of the previous project only. 6. If this tool is called again in the same session, it resets the context to the new project.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectKeyYesProject key (e.g., 'SCRUM'). Use the get_projects tool to discover available project keys.

Output Schema

ParametersJSON Schema
NameRequiredDescription
messageYesConfirmation message
projectIdYesNumeric project ID
projectKeyYesProject key
projectNameYesProject name
availableFieldsNoAvailable field values keyed by field name (e.g. 'priority', 'testcase_status'). Use these to map user input via NLP (e.g. user says 'Major' β†’ send 'High').

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavior beyond annotations: it pre-loads priority/status values for NLP mapping, resets context on repeated calls, and clears only the previous project's cached field metadata when switching. These details align with idempotentHint and neither contradict readOnlyHint nor destructiveHint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but tightly organized with clear sections: purpose, parameters, output, use cases, examples, and hints. Every sentence earns its place, and the critical prerequisite is stated in the first sentence. The structure makes the detailed content easy for an agent to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (one parameter) but contextually critical, and the description covers everything needed to call it correctly: when to call it, how to discover project keys, what the response contains, and what happens on repeated or switching calls. Examples further remove ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the parameter is already documented. The description adds extra value by explaining exact-match requirements, directing the agent to get_projects for valid keys, providing concrete examples, and describing the response's availableFields for subsequent NLP mapping.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Set the active QTM4J project for the current session.' It also states the tool's role as a prerequisite for project-specific operations, making its purpose unmistakable and distinguishable from related tools like qtm4j_get_projects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage guidance is explicit and actionable: it must be called before any project-specific operation, is used for session start or mid-conversation switching, and can validate project keys. It also names get_projects as the discovery alternative and explicitly instructs the agent to never auto-select a project without user confirmation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_start_new_executionQTM4J: Start New ExecutionAInspect

Start a new test case execution within a test cycle. Looks up the internal map ID from testCycleKey and testCaseKey; resolves environmentId and buildId names to numeric IDs.

Toolset: Test Executions

Parameters:

  • testCycleKey (string) required: Test cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.

  • testCaseKey (string) required: Test case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.

  • assignee (string): Jira account ID of the user to assign this execution to (e.g. '5e4a642c1c9d440008f2a2b4'). This is the account ID, not a display name.

  • executionPlannedDate (string): Planned execution date. Format: 'dd/MMM/yyyy' e.g. '15/Oct/2025'. Month must be capitalised (Oct, not oct or OCT).

  • environmentId (string): Environment name (e.g. 'Production', 'Staging', 'Google Chrome', 'Firefox'). Auto-resolved to its numeric ID; an unresolved name is dropped and a warning is returned.

  • buildId (string): Build name (e.g. 'Build 2.0', '1.0.0'). Auto-resolved to its numeric ID; an unresolved name is dropped and a warning is returned.

  • actualTime (string): Time already spent on the execution. Format: 'HH:mm:ss' e.g. '02:30:00' (2 hours 30 minutes 0 seconds).

  • cloneFrom (number): Source testCaseExecutionId to clone. When set, the new execution copies the source's execution and all other body fields are ignored by the server.

  • cloneExecutionCustomFields (boolean): When true, custom field values from the previous execution of this test case are copied into the new execution.

Output Description: Confirmation object with testCycleKey, testCaseKey, and created: true (set when server returns 204).

Use Cases: 1. Start a fresh execution for a test case in a test cycle 2. Clone an existing execution

Examples:

  1. Start an execution for a test case (minimal)

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42"
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', created: true }

  1. Start execution with environment, build and planned date

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "environmentId": "Production",
  "buildId": "Build 2.0",
  "executionPlannedDate": "15/Oct/2025",
  "assignee": "5e4a642c1c9d440008f2a2b4"
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', created: true }

  1. Clone an existing execution

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "cloneFrom": 725981
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', created: true }

  1. Start a fresh execution and carry over custom field values from the previous execution

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "cloneExecutionCustomFields": true
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', created: true }

Hints: 1. Call set_project_context before this tool. 2. executionPlannedDate must be in 'dd/MMM/yyyy' format (e.g. '15/Oct/2025', month 3-letter capitalised). Normalize from any user-provided format before calling. 3. When cloneFrom is non-zero, the server ignores all body fields. 4. assignee must be a Jira account ID (e.g. '5e4a642c1c9d440008f2a2b4'), not a display name. 5. environmentId and buildId accept name as strings that are resolved to numeric IDs; unresolved names are dropped and a warning is returned. 6. actualTime must be in 'HH:mm:ss' format (e.g. '02:30:00'). Always include seconds.

ParametersJSON Schema
NameRequiredDescriptionDefault
buildIdNoBuild name (e.g. 'Build 2.0', '1.0.0'). Auto-resolved to its numeric ID; an unresolved name is dropped and a warning is returned.
assigneeNoJira account ID of the user to assign this execution to (e.g. '5e4a642c1c9d440008f2a2b4'). This is the account ID, not a display name.
cloneFromNoSource testCaseExecutionId to clone. When set, the new execution copies the source's execution and all other body fields are ignored by the server.
actualTimeNoTime already spent on the execution. Format: 'HH:mm:ss' e.g. '02:30:00' (2 hours 30 minutes 0 seconds).
testCaseKeyYesTest case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.
testCycleKeyYesTest cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.
environmentIdNoEnvironment name (e.g. 'Production', 'Staging', 'Google Chrome', 'Firefox'). Auto-resolved to its numeric ID; an unresolved name is dropped and a warning is returned.
executionPlannedDateNoPlanned execution date. Format: 'dd/MMM/yyyy' e.g. '15/Oct/2025'. Month must be capitalised (Oct, not oct or OCT).
cloneExecutionCustomFieldsNoWhen true, custom field values from the previous execution of this test case are copied into the new execution.

Output Schema

ParametersJSON Schema
NameRequiredDescription
createdYesTrue when the server returned 204 No Content, confirming the execution was created successfully.
testCaseKeyYesTest case key of the created execution (e.g. 'SCRUM-TC-145').
testCycleKeyYesKey of the test cycle in which the execution was created (e.g. 'SCRUM-TR-101').

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only state readOnlyHint=false, idempotentHint=false, destructiveHint=false. The description adds substantial behavioral detail: internal map-ID lookup, name-to-ID resolution with drop-and-warn semantics for unresolved names, and cloneFrom causing the server to ignore all other body fields. It also discloses the exact success indicator ('created: true (set when server returns 204)').

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with summary, toolset, parameters, output description, use cases, examples, and hints, and the high-level behavior is front-loaded. It is somewhat long and repeats schema-provided parameter details, but the organization keeps it navigable and each section contributes to correct usage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter write operation, the description covers the full invocation contract: key formats, date/time formats, clone semantics, name-resolution warnings, prerequisite setup, and expected output. Four examples span minimal, fully specified, clone, and custom-field scenarios, leaving no significant gap for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, and the description's parameter list largely mirrors the schema. However, the Examples and Hints add operational meaning beyond the schema: normalize user-provided dates, always include seconds for actualTime, and account ID vs display name emphasis. These extras justify a stronger score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb-resource pair ('Start a new test case execution within a test cycle') and clarifies internal behavior ('Looks up the internal map ID... resolves environmentId and buildId names to numeric IDs'). The Use Cases section enumerates the two intended scenarios, clearly distinguishing this from sibling tools like qtm4j_update_test_case_execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit use cases ('Start a fresh execution... Clone an existing execution') and a prerequisite hint ('Call set_project_context before this tool'). It doesn't explicitly name alternative tools to use for other operations or state when not to use this tool, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_update_test_caseQTM4J: Update Test CaseA
Idempotent
Inspect

Update an existing test case in QTM4J. Supports auto-resolving human-readable names for priority, status, labels, and components. Labels and components support add/delete operations.

Toolset: Test Cases

Parameters:

  • key (string) required: Test case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'. Automatically resolved to the internal ID and latest version.

  • versionNo (number): Test case version number to update. Defaults to the latest version if omitted.

  • summary (string): Updated test case summary/title.

  • description (string): Updated test case description.

  • precondition (string): Updated precondition β€” conditions that must be true before the test is executed.

  • priority (string): Priority name (e.g., 'High', 'Medium', 'Low'). Auto-resolved to ID. Use values from set_project_context response.

  • status (string): Status name (e.g., 'To Do', 'In Progress', 'Done'). Auto-resolved to ID. Use values from set_project_context response.

  • assignee (string): Assignee Jira account ID (e.g., '5b10a2844c20165700ede21f').

  • estimatedTime (string): Estimated time in HH:MM:SS format (e.g., '02:30:00').

  • labels (object): Labels to add or remove by name. Each name is auto-resolved to its ID.

  • components (object): Components to add or remove by name. Each name is auto-resolved to its ID.

Output Description: Confirmation object with the test case key, versionNo updated, and updated: true. Warnings are included if any field names could not be resolved.

Use Cases: 1. Change the priority of a test case (e.g., escalate to 'High') 2. Update the status of a test case after review 3. Add new labels or remove outdated ones without affecting other labels 4. Add or remove components from a test case 5. Update summary, description, or precondition text 6. Reassign a test case to a different team member 7. Set or update the estimated time for a test case 8. Batch-update metadata as part of sprint planning

Examples:

  1. Change the priority of a test case

{
  "key": "SCRUM-TC-145",
  "priority": "High"
}

Expected Output: Test case updated with new priority

  1. Add a label and remove an old one

{
  "key": "SCRUM-TC-145",
  "labels": {
    "add": [
      "Release_2"
    ],
    "delete": [
      "Release_1"
    ]
  }
}

Expected Output: Test case updated β€” Release_2 added, Release_1 removed

  1. Update summary, status, and add a component

{
  "key": "SCRUM-TC-32",
  "summary": "Verify login with MFA enabled",
  "status": "In Progress",
  "components": {
    "add": [
      "Auth"
    ]
  }
}

Expected Output: Test case summary and status updated, Auth component added

  1. Update a specific version

{
  "key": "SCRUM-TC-85",
  "versionNo": 2,
  "assignee": "5b10a2844c20165700ede21f",
  "estimatedTime": "01:30:00"
}

Expected Output: Version 2 of test case updated with new assignee and estimated time

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. KEY FORMAT: '{PROJECT_KEY}-TC-{number}' β€” e.g. 'SCRUM-TC-145'. 3. Priority and status values come from set_project_context. Use NLP to map user intent to available names. 4. If priority or status name is not found, the field is skipped with a warning and other fields are still updated. 5. Labels and components use add/delete β€” you can add and delete in a single call. Names are auto-resolved. 6. To delete ALL current entries of any add/delete field: first call search_test_cases with filter.searchText set to the test case key and include the relevant field in the fields list, extract all current names from the response, then pass them in the delete array of this tool. 7. Only provide the fields you want to change. Omitted fields remain unchanged on the server. 8. estimatedTime must be in HH:MM:SS format (e.g., '02:30:00'). 9. versionNo defaults to the latest version. Use search_test_cases to find available versions if needed.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyYesTest case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'. Automatically resolved to the internal ID and latest version.
labelsNoLabels to add or remove by name. Each name is auto-resolved to its ID.
statusNoStatus name (e.g., 'To Do', 'In Progress', 'Done'). Auto-resolved to ID. Use values from set_project_context response.
summaryNoUpdated test case summary/title.
assigneeNoAssignee Jira account ID (e.g., '5b10a2844c20165700ede21f').
priorityNoPriority name (e.g., 'High', 'Medium', 'Low'). Auto-resolved to ID. Use values from set_project_context response.
versionNoNoTest case version number to update. Defaults to the latest version if omitted.
componentsNoComponents to add or remove by name. Each name is auto-resolved to its ID.
descriptionNoUpdated test case description.
preconditionNoUpdated precondition β€” conditions that must be true before the test is executed.
estimatedTimeNoEstimated time in HH:MM:SS format (e.g., '02:30:00').

Output Schema

ParametersJSON Schema
NameRequiredDescription
keyYesTest case key that was updated
updatedYesConfirms the update was applied
versionNoYesVersion number that was updated

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as non-read-only and non-destructive and idempotent; the description goes beyond them by explaining key auto-resolution, warning-and-skip behavior for unresolvable names, add/delete semantics, and that omitted fields remain unchanged on the server. This gives the agent a faithful model of the side effects and failure behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Despite its length, the description is organized into scannable sections (Parameters, Output Description, Use Cases, Examples, Hints) and the core purpose is front-loaded in the first sentence. The length is justified by the tool's 11 parameters, nested objects, and edge-case behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex mutation tool, the description covers prerequisites, output shape, warning behavior, version handling, and deletion workflow via search_test_cases, plus realistic examples. Nothing essential for selecting and invoking it correctly appears to be missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Even though schema coverage is 100%, the description adds meaning through concrete examples (e.g., 'SCRUM-TC-145', add/delete label JSON, version-specific updates) and operational hints (only changed fields should be passed, unknown names are skipped with a warning, format constraints). These are meaningful beyond the schema's property descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb-resource statement ('Update an existing test case in QTM4J') and further defines its scope with auto-resolution and add/delete support. This clearly distinguishes it from siblings like qtm4j_create_test_case and qtm4j_search_test_cases by focusing on updating an existing entity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides explicit use cases (e.g., change priority, update status, add labels) and a prerequisite (set_project_context must be called first; never auto-select a project). It also gives alternative flow guidance, such as calling search_test_cases before deleting all current entries, making when-to-use and preconditions explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_update_test_case_executionQTM4J: Update Test Case ExecutionA
Idempotent
Inspect

Update a test case execution (execution result, comment, environment, build, assignee, planned date, actual time). Looks up testCaseExecutionId from testCycleKey and testCaseKey; resolves executionResultId, environmentId, and buildId names to numeric IDs.

Toolset: Test Executions

Parameters:

  • testCycleKey (string) required: Test cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.

  • testCaseKey (string) required: Test case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.

  • executionResultId (string): Execution result name, e.g. 'Pass', 'Fail', 'Blocked', 'In Progress', 'Not Executed'. Project-specific (custom results allowed); resolved case-insensitively to a numeric executionResultId.

  • comment (string): Execution comment. Pass null to clear the existing comment.

  • actualTime (string): Time already spent on the execution. Format: 'HH:mm:ss' e.g. '02:30:00' (2 hours 30 minutes 0 seconds). Pass null to clear.

  • executionAssignee (string): Jira account ID of the assignee (not a display name). Pass null to unassign.

  • environmentId (string): Environment name (e.g. 'Production', 'Staging', 'Google Chrome', 'Firefox'). Auto-resolved to its numeric ID; an unresolved name is dropped and a warning is returned.

  • executionPlannedDate (string): Planned execution date. Format: 'dd/MMM/yyyy' e.g. '15/Oct/2025'. Month must be capitalised (Oct, not oct or OCT). Pass null to clear.

  • buildId (string): Build name (e.g. 'Build 2.0', '1.0.0'). Auto-resolved to its numeric ID; an unresolved name is dropped and a warning is returned. Pass null to clear.

Output Description: Confirmation object with testCycleKey, testCaseKey, and updated: true. Warnings are returned in content if any resolvable field (executionResultId, environmentId, buildId) could not be resolved and was dropped.

Use Cases: 1. Set a test case execution result (e.g. Pass, Fail, Blocked) 2. Update comment, environment, build, assignee, planned date, or actual time on an execution

Examples:

  1. Mark test case as Pass

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "executionResultId": "Pass"
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', updated: true }

  1. Update with comment, environment, and actual time

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "executionResultId": "Pass",
  "comment": "All smoke tests passed.",
  "environmentId": "Production",
  "actualTime": "01:30:00"
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', updated: true }

  1. Clear comment

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "comment": null
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', updated: true }

Hints: 1. Call set_project_context before this tool. 2. At least one updatable field must be provided. 3. environmentId and buildId accept environment/build name as strings that are resolved to numeric IDs; unresolved names are dropped and a warning is returned. 4. actualTime must be in 'HH:mm:ss' format (e.g. '01:30:00'). Always include seconds

ParametersJSON Schema
NameRequiredDescriptionDefault
buildIdNoBuild name (e.g. 'Build 2.0', '1.0.0'). Auto-resolved to its numeric ID; an unresolved name is dropped and a warning is returned. Pass null to clear.
commentNoExecution comment. Pass null to clear the existing comment.
actualTimeNoTime already spent on the execution. Format: 'HH:mm:ss' e.g. '02:30:00' (2 hours 30 minutes 0 seconds). Pass null to clear.
testCaseKeyYesTest case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.
testCycleKeyYesTest cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.
environmentIdNoEnvironment name (e.g. 'Production', 'Staging', 'Google Chrome', 'Firefox'). Auto-resolved to its numeric ID; an unresolved name is dropped and a warning is returned.
executionAssigneeNoJira account ID of the assignee (not a display name). Pass null to unassign.
executionResultIdNoExecution result name, e.g. 'Pass', 'Fail', 'Blocked', 'In Progress', 'Not Executed'. Project-specific (custom results allowed); resolved case-insensitively to a numeric executionResultId.
executionPlannedDateNoPlanned execution date. Format: 'dd/MMM/yyyy' e.g. '15/Oct/2025'. Month must be capitalised (Oct, not oct or OCT). Pass null to clear.

Output Schema

ParametersJSON Schema
NameRequiredDescription
updatedYesTrue when the server returned 204 No Content, confirming the execution was updated successfully.
testCaseKeyYesTest case key of the updated execution.
testCycleKeyYesTest cycle key of the updated execution.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses non-obvious behavior: names are resolved to numeric IDs, resolution is case-insensitive, unresolved names are dropped with warnings, null clears existing values, and date/time formats have strict capitalization rules. These details go well beyond what the annotations already communicate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a one-sentence summary, then organized into Parameters, Output, Use Cases, Examples, and Hints. Despite length, every section earns its place and the structure makes the 9-parameter surface area scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter mutating tool requiring project context, the description covers prerequisites, parameter semantics, output shape, and warning behavior. An output schema is also present, so nothing needed to call the tool correctly appears missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds practical meaning with examples, the 'Month must be capitalised' warning, the 'always include seconds' reminder, and null-clearing semantics. It reinforces likely usage mistakes rather than merely restating the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names the exact operation and resource ('Update a test case execution') and enumerates the updatable fields. It also explains the lookup behavior from testCycleKey and testCaseKey plus the ID resolution logic, making it unambiguous against siblings like qtm4j_update_test_step_execution. The Toolset label reinforces its scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Hints explicitly require calling set_project_context first and providing at least one updatable field, and the Use Cases section clarifies typical invocation scenarios. It does not name alternative tools or state when not to use it, so it stops short of an explicit routing guide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_update_test_cycleQTM4J: Update Test CycleA
Idempotent
Inspect

Update an existing test cycle in QTM4J by its human-readable key (e.g. 'SCRUM-TR-101'). Supports auto-resolving human-readable names for status and priority. Labels and components support add/delete operations. Only the fields you provide are changed β€” omitted fields are left as-is. projectId is injected automatically from the active project context.

Toolset: Test Cycles

Parameters:

  • key (string) required: Test cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.

  • summary (string): Updated test cycle name / title. Max 255 characters.

  • description (string): Updated description. Pass null to clear the existing value. Max 65 535 characters.

  • status (string): Status name (e.g., 'To Do', 'In Progress', 'Done'). Auto-resolved to ID. Use values from set_project_context response. Pass null to clear.

  • priority (string): Priority name (e.g., 'High', 'Medium', 'Low'). Auto-resolved to ID. Use values from set_project_context response. Pass null to clear.

  • plannedStartDate (string): Format: 'dd/MMM/yyyy HH:mm' e.g. '15/May/2026 09:00'. Month must be capitalised (May not may). Pass null to clear the existing value.

  • plannedEndDate (string): Format: 'dd/MMM/yyyy HH:mm' e.g. '15/May/2026 09:00'. Month must be capitalised (May not may). Pass null to clear the existing value.

  • assignee (string): Assignee Jira account ID (e.g., '5b10a2844c20165700ede21f'). Pass null to unassign.

  • reporter (string): Reporter Jira account ID (e.g., '5b10a2844c20165700ede21f'). Pass null to clear.

  • labels (object): Labels to add or remove by name. Each name is auto-resolved to its ID.

  • components (object): Components to add or remove by name. Each name is auto-resolved to its ID.

Output Description: Confirmation object with the test cycle key and updated: true. Warnings are included if any field names could not be resolved.

Use Cases: 1. Update summary, status, priority, planned dates, assignee, or reporter 2. Clear a nullable field by passing null (e.g. description: null removes text, assignee: null unassigns owner) 3. Add or remove labels and components atomically without affecting other entries 4. Apply multiple field updates in a single call

Examples:

  1. Rename a test cycle

{
  "key": "SCRUM-TR-101",
  "summary": "Regression Cycle - Sprint 12 Updated"
}

Expected Output: Test cycle updated with new summary

  1. Change status and update planned dates

{
  "key": "SCRUM-TR-101",
  "status": "In Progress",
  "plannedStartDate": "01/May/2026 09:00",
  "plannedEndDate": "31/May/2026 18:00"
}

Expected Output: Test cycle status and planned dates updated

  1. Add a label and remove an old one

{
  "key": "SCRUM-TR-101",
  "labels": {
    "add": [
      "Regression",
      "Smoke"
    ],
    "delete": [
      "Sprint1"
    ]
  }
}

Expected Output: Test cycle updated β€” Regression and Smoke labels added, Sprint1 removed

  1. Clear the description text

{
  "key": "SCRUM-TR-101",
  "description": null
}

Expected Output: Test cycle description cleared

  1. Unassign the owner and clear planned dates

{
  "key": "SCRUM-TR-101",
  "assignee": null,
  "plannedStartDate": null,
  "plannedEndDate": null
}

Expected Output: Test cycle owner unassigned and planned dates cleared

  1. Full update with all fields

{
  "key": "SCRUM-TR-101",
  "summary": "Final Regression Cycle",
  "description": "Updated for sprint 12.",
  "status": "In Progress",
  "priority": "High",
  "plannedStartDate": "15/May/2026 09:00",
  "plannedEndDate": "30/May/2026 18:00",
  "assignee": "5b10a2844c20165700ede21f",
  "labels": {
    "add": [
      "Regression"
    ],
    "delete": [
      "Sprint1"
    ]
  },
  "components": {
    "add": [
      "Backend"
    ],
    "delete": [
      "Frontend"
    ]
  }
}

Expected Output: Test cycle updated with all specified fields

Hints: 1. PREREQUISITE: set_project_context must be called before this tool. NEVER auto-select a project. 2. KEY FORMAT: '{PROJECT_KEY}-TR-{number}' β€” e.g. 'SCRUM-TR-101'. 3. Pass explicit null to CLEAR a nullable field β€” e.g. description: null removes the description text, assignee: null unassigns the owner, plannedStartDate: null removes the date. Omitting a field leaves it unchanged. 4. Status and priority are auto-resolved from human-readable names loaded by set_project_context. If a name cannot be resolved, the cycle is still updated and a warning is returned. 5. Labels and components use add/delete β€” names are auto-resolved to IDs. Both operations can be combined in a single call. 6. Date format: 'dd/MMM/yyyy HH:mm' e.g. '15/May/2026 09:00'. Month must be capitalised (May not may or MAY). 7. Archived test cycles cannot be updated β€” the server returns 400. Unarchive first if needed.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyYesTest cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.
labelsNoLabels to add or remove by name. Each name is auto-resolved to its ID.
statusNoStatus name (e.g., 'To Do', 'In Progress', 'Done'). Auto-resolved to ID. Use values from set_project_context response. Pass null to clear.
summaryNoUpdated test cycle name / title. Max 255 characters.
assigneeNoAssignee Jira account ID (e.g., '5b10a2844c20165700ede21f'). Pass null to unassign.
priorityNoPriority name (e.g., 'High', 'Medium', 'Low'). Auto-resolved to ID. Use values from set_project_context response. Pass null to clear.
reporterNoReporter Jira account ID (e.g., '5b10a2844c20165700ede21f'). Pass null to clear.
componentsNoComponents to add or remove by name. Each name is auto-resolved to its ID.
descriptionNoUpdated description. Pass null to clear the existing value. Max 65 535 characters.
plannedEndDateNoFormat: 'dd/MMM/yyyy HH:mm' e.g. '15/May/2026 09:00'. Month must be capitalised (May not may). Pass null to clear the existing value.
plannedStartDateNoFormat: 'dd/MMM/yyyy HH:mm' e.g. '15/May/2026 09:00'. Month must be capitalised (May not may). Pass null to clear the existing value.

Output Schema

ParametersJSON Schema
NameRequiredDescription
keyYesHuman-readable key of the updated test cycle
updatedYesConfirms the update was applied

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Even though annotations already declare idempotentHint=true and destructiveHint=false, the description adds substantial behavioral detail: projectId is injected automatically, null clears fields, names are auto-resolved to IDs with warnings on failure, labels/components use add/delete semantics, and archived cycles return 400. This goes far beyond what annotations convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured with clear sections: overview, parameters, output, use cases, examples, and hints. It is front-loaded with the core behavior and then provides progressively detailed guidance. Some redundancy exists between the parameter list, use cases, and hints, but the complexity of the tool justifies the thoroughness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 11 parameters, nested add/delete objects, date formatting constraints, prerequisite project context, null-clearing semantics, and conditional server behavior, the description is remarkably complete. It covers prerequisites, failure modes, warnings, output shape, and provides six realistic examples that demonstrate both simple and full updates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description still enriches the parameters meaningfully: key must match '{PROJECT_KEY}-TR-{number}' and is used as the API path parameter, date format requires capitalized months, null has explicit clearing semantics, and status/priority values come from set_project_context. This is well beyond the schema baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Update an existing test cycle in QTM4J by its human-readable key.' It clearly indicates partial-update behavior, names the key format, and differentiates from creation-oriented siblings like qtm4j_create_test_cycle by emphasizing 'existing' test cycles.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides strong contextual guidance: set_project_context is a prerequisite, project selection must never be auto-assumed, omitted fields are left unchanged, and archived test cycles cannot be updated. It does not explicitly name an alternative tool for creating cycles, but the word 'existing' and the Toolset heading make the intended usage clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_update_test_step_executionQTM4J: Update Test Step ExecutionA
Idempotent
Inspect

Update a test step execution (execution result, actual result, comment). Looks up testStepExecutionId from testCycleKey, testCaseKey, and step sequence number; resolves executionResultId name to a numeric ID.

Toolset: Test Executions

Parameters:

  • testCycleKey (string) required: Test cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.

  • testCaseKey (string) required: Test case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.

  • testStepSeqNo (number) required: sequence number of the step to update (e.g. 2 = the second step).

  • executionResultId (string): Execution result name, e.g. 'Pass', 'Fail', 'Blocked', 'In Progress', 'Not Executed'. Project-specific (custom results allowed); resolved case-insensitively to a numeric executionResultId.

  • comment (string): Step execution comment. Pass null to clear the existing comment.

  • actualResult (string): Actual result for this test step. Set to null to clear the existing value.

Output Description: Confirmation object with testCycleKey, testCaseKey, testStepSeqNo, and updated: true. executionResultName is included when the server returns it. Warnings are returned in content if executionResultId name could not be resolved.

Use Cases: 1. Set a test step execution result (e.g. Pass, Fail, Blocked) 2. Update actualResult or comment on a test step execution

Examples:

  1. Mark step 2 as Pass

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "testStepSeqNo": 2,
  "executionResultId": "Pass"
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', testStepSeqNo: 2, executionResultName: 'Pass', updated: true }

  1. Update step 3 with actualResult and comment

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "testStepSeqNo": 3,
  "executionResultId": "Fail",
  "actualResult": "Button not found on page.",
  "comment": "Reproduced consistently in Chrome."
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', testStepSeqNo: 3, executionResultName: 'Fail', updated: true }

  1. Clear actualResult on step 1

{
  "testCycleKey": "PROJ-TR-101",
  "testCaseKey": "PROJ-TC-42",
  "testStepSeqNo": 1,
  "actualResult": null
}

Expected Output: { testCycleKey: 'PROJ-TR-101', testCaseKey: 'PROJ-TC-42', testStepSeqNo: 1, updated: true }

Hints: 1. Call set_project_context before this tool. 2. At least one updatable field must be provided.

ParametersJSON Schema
NameRequiredDescriptionDefault
commentNoStep execution comment. Pass null to clear the existing comment.
testCaseKeyYesTest case key in the format '{PROJECT_KEY}-TC-{number}', e.g. 'SCRUM-TC-145'.
actualResultNoActual result for this test step. Set to null to clear the existing value.
testCycleKeyYesTest cycle key in the format '{PROJECT_KEY}-TR-{number}', e.g. 'SCRUM-TR-101'. Used directly as the API path parameter.
testStepSeqNoYessequence number of the step to update (e.g. 2 = the second step).
executionResultIdNoExecution result name, e.g. 'Pass', 'Fail', 'Blocked', 'In Progress', 'Not Executed'. Project-specific (custom results allowed); resolved case-insensitively to a numeric executionResultId.

Output Schema

ParametersJSON Schema
NameRequiredDescription
updatedYesTrue when the server confirms the update (HTTP 200 OK) and the test step execution is successfully updated.
testCaseKeyYesKey of the test case whose step execution was updated (e.g. 'SCRUM-TC-145').
testCycleKeyYesKey of the test cycle containing the updated step execution (e.g. 'SCRUM-TR-101').
testStepSeqNoYessequence number of the test step whose execution was updated.
executionResultNameNoName of the execution result applied to the step (e.g. 'Pass', 'Fail'). Present only when the server returns it.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses meaningful behavior: it 'looks up testStepExecutionId' from keys and sequence number, resolves executionResultId names case-insensitively to numeric IDs including project-specific custom results, and warns that unresolved names are surfaced in content. It also specifies null-clearing semantics for comment and actualResult, and includes output examples showing the confirmation shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than average but well-structured with sections for Toolset, Parameters, Output, Use Cases, Examples, and Hints. The main verb and resource are front-loaded. While the parameter list duplicates schema content somewhat, the examples and behavioral notes justify the length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is complete for a moderate-complexity update tool: it specifies required context (set_project_context), minimum input requirements (at least one updatable field), lookup logic, output confirmation fields, warning behavior, and concrete examples. With an output schema present and no nested objects, an agent has everything needed to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all parameters. The description adds valuable semantics such as 'Used directly as the API path parameter', the project-specific and case-insensitive resolution of executionResultId, and null-to-clear behavior. Examples illustrate realistic parameter combinations, which pushes this above the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Update a test step execution (execution result, actual result, comment)', which immediately distinguishes it from test case execution updates. It also explains the lookup path through testCycleKey, testCaseKey, and step sequence number, removing ambiguity about which entity is modified.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section clearly states the two intended scenarios: setting an execution result and updating actualResult/comment. The hint 'Call set_project_context before this tool' provides a necessary precondition, and 'At least one updatable field must be provided' gives a practical guardrail. It does not explicitly name alternatives or when not to use this tool, but the context is sufficiently clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qtm4j_upload_automation_resultQTM4J: Upload Automation ResultAInspect

Upload an automation result file to QTM4J and map the results to a test cycle. Supports JUnit XML, TestNG XML, Cucumber JSON, QAF, HP UFT, and SpecFlow formats.

Toolset: Test Automation

Parameters:

  • filePath (string) required: Path to the automation result file on disk. Filesystem contents can change between turns β€” always resolve this from a fresh scan, never from a previously seen path. Supported extensions: .xml, .json, .zip

  • format (enum) required: Format of the result file. Supported values: cucumber, testng, junit, qaf, hpuft, specflow

  • testCycleToReuse (string): Work key of an existing test cycle to reuse (e.g. 'TR-PRJ-1'). If omitted, a new test cycle is created.

  • environment (string): Name of the environment on which the test cycle was executed (e.g. 'Chrome', 'Staging'). Defaults to 'No Environment'.

  • build (string): Build name or version for the test cycle execution (e.g. '1.0.0-beta'). Defaults to blank.

  • isZip (boolean): Set to true when uploading a ZIP archive containing result files. Required for QAF format. (default: false)

  • attachFile (boolean): Set to true to upload attachments referenced in execution results. (default: false)

  • matchTestSteps (boolean): true β€” match test cases by summary AND test steps. false β€” match by summary or key only. (default: true)

  • appendTestName (boolean): Applicable to JUnit/TestNG only. Appends suite/test name to method name in test case summary.

  • fields (object): Additional fields to set on the test cycle, test case, and/or test case execution created during import.

Output Description: trackingId to poll import status, a message from the API, the filePath uploaded, and the format used.

Use Cases: 1. Upload automation results to QTM4J 2. Import test results from a CI/CD pipeline run 3. Link test results to an existing test cycle 4. Create a new test cycle from automation results 5. Upload JUnit, TestNG, Cucumber, QAF, HP UFT, or SpecFlow result files

Examples:

  1. User says 'upload my test results to QTM4J' β€” scan workspace, find single result file, confirm and upload

{
  "filePath": "./target/surefire-reports/TEST-results.xml",
  "format": "junit"
}

Expected Output: trackingId returned; import processing started in QTM4J

  1. User wants results linked to an existing test cycle

{
  "filePath": "./reports/cucumber.json",
  "format": "cucumber",
  "testCycleToReuse": "TR-PRJ-5",
  "environment": "Chrome",
  "build": "2.1.0"
}

Expected Output: Results mapped to test cycle TR-PRJ-5

  1. Upload QAF ZIP and set test cycle metadata

{
  "filePath": "./results/qaf-results.zip",
  "format": "qaf",
  "isZip": true,
  "fields": {
    "testCycle": {
      "summary": "Regression Run 2024-Q1",
      "labels": [
        "regression"
      ],
      "priority": "High"
    }
  }
}

Expected Output: ZIP uploaded; test cycle created with summary, labels, and priority

  1. User provides an unrecognised priority value β€” do NOT silently map to a similar word; ask the user first

{
  "filePath": "./reports/cucumber.json",
  "format": "cucumber",
  "fields": {
    "testCycle": {
      "priority": "critical"
    }
  }
}

Expected Output: Tool is NOT called yet. Inform the user that 'critical' was not recognised as a valid priority and ask them to confirm the correct value (e.g. from the available options). Do not map 'critical' to 'Blocker' or any other value without explicit user confirmation.

Hints: 1. NO PROJECT CONTEXT REQUIRED: Do NOT call set_project_context and do NOT ask the user for a project key, project ID, or any other project details. This tool works independently β€” never prompt the user for project information. 2. FILE DISCOVERY: Always do a fresh scan β€” never reuse a path from a previous turn. If no path is provided, search in order: target/surefire-reports, target/failsafe-reports, build/reports/tests, build/test-results, test-results, reports, cucumber-reports. If exactly one file is found, show the path and inferred format to the user and confirm before uploading. If multiple files are found, list them all and wait for the user to pick one. If nothing is found, ask for the path. Never pick or upload silently. 3. FORMAT INFERENCE: .json β†’ cucumber (unambiguous). For .xml, infer from the file name β€” 'junit'/'surefire' β†’ junit, 'testng' β†’ testng, 'specflow' β†’ specflow, 'hpuft'/'uft' β†’ hpuft. For .zip, ALWAYS set isZip: true, but do NOT assume qaf β€” the zip could contain junit, testng, or cucumber results; if the format cannot be determined from the file name, ask the user. If the file name gives no clear signal for .xml either, ask the user to confirm the format. 4. TEST CYCLE: Only ask for testCycleToReuse if the user explicitly wants to link to an existing cycle. If not mentioned, omit it β€” QTM4J creates a new test cycle automatically. 5. DATE FORMAT: plannedStartDate and plannedEndDate in fields.testCycle MUST be formatted as 'dd/MMM/yyyy HH:mm' (e.g. '14/May/2026 10:30'). Convert any user-provided date (ISO, natural language, relative) to this exact format before sending. 6. FOLDER ID: folderId is a numeric ID. Apply it ONLY to the level the user specifies; if unspecified, default to fields.testCycle only β€” never copy it to both levels. Get the ID from the user directly (right-click folder in QTM4J β†’ 'Copy Folder Id'). 7. ASSIGNEE / REPORTER: assignee and reporter in fields.testCycle and fields.testCase require a Jira Account ID (not a display name or email). Ask the user to provide their Account ID directly. 8. FIELD MAPPING CONFIRMATION: Apply formatting transformations (case correction, date/time conversion) automatically. Only ask for user confirmation when you cannot find a recognised match and need to substitute an unrecognised value with a guessed alternative β€” never silently substitute in that case. 9. TRACKING: Import processing is asynchronous. To check status, call get_automation_history and find the record whose trackingId matches the one returned from this tool.

ParametersJSON Schema
NameRequiredDescriptionDefault
buildNoBuild name or version for the test cycle execution (e.g. '1.0.0-beta'). Defaults to blank.
isZipNoSet to true when uploading a ZIP archive containing result files. Required for QAF format.
fieldsNoAdditional fields to set on the test cycle, test case, and/or test case execution created during import.
formatYesFormat of the result file. Supported values: cucumber, testng, junit, qaf, hpuft, specflow
filePathYesPath to the automation result file on disk. Filesystem contents can change between turns β€” always resolve this from a fresh scan, never from a previously seen path. Supported extensions: .xml, .json, .zip
attachFileNoSet to true to upload attachments referenced in execution results.
environmentNoName of the environment on which the test cycle was executed (e.g. 'Chrome', 'Staging'). Defaults to 'No Environment'.
appendTestNameNoApplicable to JUnit/TestNG only. Appends suite/test name to method name in test case summary.
matchTestStepsNotrue β€” match test cases by summary AND test steps. false β€” match by summary or key only.
testCycleToReuseNoWork key of an existing test cycle to reuse (e.g. 'TR-PRJ-1'). If omitted, a new test cycle is created.

Output Schema

ParametersJSON Schema
NameRequiredDescription
formatYes
messageYes
filePathYes
trackingIdYes

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations by disclosing that import processing is asynchronous and should be tracked via get_automation_history, that a new test cycle is created when testCycleToReuse is omitted, that QAF ZIP uploads require isZip=true, and that unrecognized priority values must not be silently substituted (Example 4). It also documents side-effect-relevant details like date formatting, folderId scoping, and Jira Account ID requirements for assignee/reporter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a front-loaded summary, bold section headers, clear examples, and numbered hints, making it easy to scan. However, the 'Parameters' section largely duplicates the schema descriptions, and the 'Use Cases' list adds limited information beyond the opening paragraph, making the description longer than strictly necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 10 parameters and nested fields, the description covers virtually everything an agent needs: file discovery order, format inference rules, confirmation requirements, test cycle reuse behavior, field conversion rules, and async tracking via get_automation_history. The worked examples illustrate realistic call patterns and even include a case where the tool should not be called, leaving no major operational gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description adds substantial meaning beyond the schema: filePath must be resolved from a fresh scan, testCycleToReuse expects a work key like 'TR-PRJ-1', isZip is required for QAF, matchTestSteps defines matching behavior, and appendTestName only applies to JUnit/TestNG. Hints 5-8 add critical subfield semantics for fields, such as date format, folderId scope, and Account ID requirements, which the schema alone does not fully convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific action ('Upload an automation result file'), the target system (QTM4J), and the outcome (map results to a test cycle), then lists the supported formats. This makes the tool's role immediately distinguishable from siblings like qtm4j_get_automation_history or qtm4j_create_test_cycle. The name itself is also descriptive, and the description reinforces it without repeating it tautologically.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The Hints section provides clear context for when the tool should or should not be called: fresh file scan required, confirm before uploading, ask when multiple/no files found, and explicitly 'Do NOT call set_project_context'. It does not, however, compare this tool against similar-looking siblings such as qmetry_import_automation_test_results, so the alternative-selection guidance is not fully explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_add_prompt_stepReflect: Add Prompt StepAInspect

Add a natural language prompt step to an active Reflect recording session

Toolset: Recording

Parameters:

  • sessionId (string) required: The ID of the Reflect recording session

  • prompt (string) required: The natural language prompt describing the test step. The prompt should describe a single action, assertion, or query. The prompt can only contain literal text; it cannot contain template variables, secrets, or other dynamic syntax. If we are in a Web recording, the prompt can perform browser navigation (e.g. 'Click on the back button', 'Navigate to https://www.example.com') and use the tab and enter keys to navigate (e.g. 'Press the tab key', 'Press the enter key').

ParametersJSON Schema
NameRequiredDescriptionDefault
promptYesThe natural language prompt describing the test step. The prompt should describe a single action, assertion, or query. The prompt can only contain literal text; it cannot contain template variables, secrets, or other dynamic syntax. If we are in a Web recording, the prompt can perform browser navigation (e.g. 'Click on the back button', 'Navigate to https://www.example.com') and use the tab and enter keys to navigate (e.g. 'Press the tab key', 'Press the enter key').
sessionIdYesThe ID of the Reflect recording session

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint=false, etc.), the description discloses that the prompt can only be literal text, cannot contain dynamic syntax or secrets, and can perform browser navigation and tab/enter in Web recordings. It also implies the session must be active. These are behavioral constraints not visible in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and the toolset label helps orientation. However, the 'Parameters' section duplicates the input schema descriptions almost exactly, creating redundancy. The structure is organized but not optimally concise for an agent that already has schema access.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has only two required parameters, both fully documented, and the description adds prompt constraints and web-recording behavior. It does not explicitly state the prerequisite of an open session from reflect_connect_to_session or describe success/error responses, but for a simple append-step mutation tool, the coverage is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%: both sessionId and prompt are described in the input schema, and the description's 'Parameters' block repeats those descriptions nearly verbatim without adding new meaning. Examples like 'Click on the back button' are already present in the schema description, so the description adds no semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Add a natural language prompt step to an active Reflect recording session' – a specific verb ('Add'), a precise resource ('prompt step'), and a target context ('active ... recording session'). This clearly differentiates it from sibling tools like reflect_add_segment or reflect_delete_previous_step without needing to inspect schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for adding prompt steps during an active recording session and gives constraints on prompt content (literal text only, no template variables/secrets) and capabilities in Web recordings. However, it does not explicitly state when to prefer this over alternatives like reflect_add_segment, nor provide exclusions or prerequisites other than 'active'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_add_segmentReflect: Add SegmentAInspect

Insert a reusable test segment into an active Reflect recording session

Toolset: Recording

Parameters:

  • sessionId (string) required: The ID of the Reflect recording session

  • segmentId (number) required: The ID of the segment to add

ParametersJSON Schema
NameRequiredDescriptionDefault
segmentIdYesThe ID of the segment to add
sessionIdYesThe ID of the Reflect recording session

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this operation is not read-only, not idempotent, and not destructive. The description adds a meaningful prerequisite by requiring an 'active' recording session, but it does not disclose expected outcomes, failure modes, or side effects beyond the schema and annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is compact, front-loaded, and immediately conveys the core action. The parameter list is a bit redundant with the input schema, but it is short and organized, so the overall description remains efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter mutation, the description covers the key prerequisite of an active recording session and both parameters are well-documented. It is adequate, but it omits useful context such as the need for the segment to already exist, related lookups for obtaining IDs, or what a successful call returns.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both sessionId and segmentId are already fully documented. The description repeats those same parameter descriptions without adding extra semantics such as how IDs are obtained, valid segment types, or expected data formats.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Insert') and resource ('a reusable test segment') with a precise target ('an active Reflect recording session'). This clearly distinguishes it from related reflect tools: it is not creating a segment (reflect_create_segment) and not adding a prompt step (reflect_add_prompt_step).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'into an active Reflect recording session' implies the intended use case: adding an existing segment while a recording session is active. However, it provides no explicit guidance about when to choose this over reflect_create_segment, reflect_list_segments, or reflect_add_prompt_step, nor any 'when not to use' conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_cancel_suite_executionReflect: Cancel Suite ExecutionC
Read-onlyIdempotent
Inspect

Cancel a reflect suite execution

Toolset: Suites

Parameters:

  • suiteId (string) required: ID of the reflect suite to cancel execution for

  • executionId (string) required: ID of the reflect suite execution to cancel

ParametersJSON Schema
NameRequiredDescriptionDefault
suiteIdYesID of the reflect suite to cancel execution for
executionIdYesID of the reflect suite execution to cancel

TDQS

C2.6/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations declare readOnlyHint=true, but the description says 'Cancel a reflect suite execution,' which is a state-changing operation. This is a direct contradiction. The description also provides no additional behavioral context such as reversibility, effects on ongoing execution, or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and the main action is front-loaded, but the parameter list duplicates the input schema without adding value. It is not bloated, yet it does not use its limited space to convey anything beyond the schema already provides.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a cancellation operation, important context is missing: whether completed executions can be cancelled, what happens to the underlying test run, and whether any confirmation or side effects should be expected. The tool is simple and has rich sibling context, but the description alone leaves these gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents both parameters. The description only repeats the same parameter names and descriptions without adding extra meaning, which matches the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Cancel') and the resource ('a reflect suite execution'), making the tool's purpose unambiguous. However, it is essentially a restatement of the tool title and does not differentiate from sibling tools like reflect_execute_suite or reflect_get_suite_execution_status beyond the verb itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool, what conditions must be met before cancellation, or when an alternative tool would be more appropriate. The agent is left to infer that this is used to stop a running suite execution, but there is no explicit usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_connect_to_sessionReflect: Connect To SessionA
Idempotent
Inspect

Connect to an active Reflect recording session via WebSocket to enable interactive control. When creating or editing a Reflect test using a connected recording session, follow these guidelines:

  1. After connecting to a session, get the list of segments for the session's platform type so you know what actions could be added via segments vs needing to create new steps. Do not list tests, only list segments.

  2. Before performing an action, take a screenshot to understand the current state of the application.

  3. Each add_prompt_step request should perform a single action or assertion. Do not combine multiple actions or assertions into a single step.

  4. Only perform one action at a time unless you're sure the action won't move the application to a different screen. For example, you can send multiple add_prompt_step requests to fill out individual form fields if those fields are visible on the current screen.

  5. Check the list of existing Segments to see if a Segment exists that achieves a similar goal to what you're trying to do next. If so, add the segment instead of creating new steps.

  6. If a step fails, use delete_previous_step to remove it and try a different approach.

  7. After completing a task, if the task required multiple prompt steps, add a final prompt step that validates the current state of the page based on what you see on the screen. In your validation, do not reference information that can change from run to run.

Toolset: Recording

Parameters:

  • sessionId (string) required: The ID of the Reflect recording session to connect to

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe ID of the Reflect recording session to connect to

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already supply readOnlyHint=false, idempotentHint=true, destructiveHint=false, and the description adds context beyond them by revealing the WebSocket mechanism and the interactive control model. It also discloses behavioral expectations for the connected session, such as taking screenshots before actions and performing one add_prompt_step action at a time. It does not cover connection failure or lifecycle behavior, but the annotations lower the bar.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded and the numbered workflow is easy to scan. There is redundancy between items 1 and 5 (both about checking segments) and items 3 and 4 (both about single actions), plus a duplicate of the schema's parameter description, so it is not perfectly concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With only one parameter, no output schema, and annotations present, the description gives enough context to call and use the connected session: it names the prerequisite, identifies how to inspect available segments, and prescribes recovery and validation behavior. The main gap is the absence of any statement about connection result, errors, or how the connection is released, but this is not blocking for invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema already documents sessionId as 'The ID of the Reflect recording session to connect to.' The description's Parameters section repeats this without adding format, provenance, or failure semantics, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence uses a specific verb ('Connect'), identifies the exact resource ('active Reflect recording session') and transport ('WebSocket'), and states the outcome ('enable interactive control'). This makes the tool clearly distinguishable from sibling reflect_* operations such as reflect_add_segment or reflect_get_screenshot.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states an explicit context: when creating or editing a Reflect test with a connected recording session. It also gives concrete operational rules (list segments, take screenshots, use delete_previous_step on failure, add a validation step), and even gives a 'do not list tests, only list segments' exclusion. It does not explicitly frame a when-not-to-use or alternative connection tool, so it falls just short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_create_segmentReflect: Create SegmentAInspect

Create a new Reflect segment which contains an ordered list of steps. Segments are reusable groups of steps that can be referenced from tests, and cannot reference other segments.

Toolset: Tests

Parameters:

  • name (string) required: Name of the segment to create.

  • type (enum) required: Platform of the segment.

  • description (string): Optional description of the segment.

  • deviceProfile (string): Device profile id. Required for a 'web' segment; ignored for 'api' and 'native-mobile'. One of: 'desktop', 'tablet', 'mobile'.

  • steps (array) required: Ordered list of steps that make up the segment. Segments cannot reference other segments. Prefer deterministic, selector-based steps ('click', 'input', 'submit', 'text-validation', 'hover', etc.) over AI-driven 'prompt' steps whenever a stable selector is available. Reserve 'prompt' steps for behavior that can't be expressed with a selector. Any text field in a step (e.g. 'inputText', 'url', 'expectedText', 'requestBody', header values, prompt text) may embed Reflect variable and function references using '${...}' syntax, which are resolved at run time: '${var(name)}' inserts the value of a parameter/variable named 'name' (declare parameters via the top-level 'parameters' field, or assign them mid-run with an 'update-parameters' step); '${sec(name)}' inserts the value of the account secret 'name'. Functions generate dynamic values: '${alphanum(n)}', '${alpha(n)}', '${num(n)}' (random alphanumeric / alphabetic / numeric string of length n), '${range(min, max)}' (random integer, inclusive), '${time(offsetMs)}' and '${datetime(offsetMs)}' (current epoch-millis / date-time, with an optional millisecond offset), and '${date(format, offsetDays)}' (current date formatted with tokens like 'MM/dd/yyyy', with an optional day offset).

  • parameters (array): Optional named parameters (variables) for the segment, each with a 'name' and optional default 'value'. Reference a parameter's value inside any step text field with '${var(name)}'.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the segment to create.
typeYesPlatform of the segment.
stepsYesOrdered list of steps that make up the segment. Segments cannot reference other segments. Prefer deterministic, selector-based steps ('click', 'input', 'submit', 'text-validation', 'hover', etc.) over AI-driven 'prompt' steps whenever a stable selector is available. Reserve 'prompt' steps for behavior that can't be expressed with a selector. Any text field in a step (e.g. 'inputText', 'url', 'expectedText', 'requestBody', header values, prompt text) may embed Reflect variable and function references using '${...}' syntax, which are resolved at run time: '${var(name)}' inserts the value of a parameter/variable named 'name' (declare parameters via the top-level 'parameters' field, or assign them mid-run with an 'update-parameters' step); '${sec(name)}' inserts the value of the account secret 'name'. Functions generate dynamic values: '${alphanum(n)}', '${alpha(n)}', '${num(n)}' (random alphanumeric / alphabetic / numeric string of length n), '${range(min, max)}' (random integer, inclusive), '${time(offsetMs)}' and '${datetime(offsetMs)}' (current epoch-millis / date-time, with an optional millisecond offset), and '${date(format, offsetDays)}' (current date formatted with tokens like 'MM/dd/yyyy', with an optional day offset).
parametersNoOptional named parameters (variables) for the segment, each with a 'name' and optional default 'value'. Reference a parameter's value inside any step text field with '${var(name)}'.
descriptionNoOptional description of the segment.
deviceProfileNoDevice profile id. Required for a 'web' segment; ignored for 'api' and 'native-mobile'. One of: 'desktop', 'tablet', 'mobile'.

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses meaningful behaviors: segments cannot reference other segments, deviceProfile is required for 'web' and ignored for other platform types, and the ${...} variable/function resolution syntax is fully explained. It also provides guidance on preferring selector-based steps over AI prompt steps. No contradiction with annotations was found.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentences are concise and useful, but the description then duplicates nearly all of the input schema's parameter documentation, including the large 'steps' sub-schema. Since the schema already carries this information, the description is significantly longer than it needs to be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Together with the very rich input schema, the description covers all six parameters, required fields, platform-specific behavior, step type guidance, and template syntax. There is no output schema, and while explicit success/error behavior is not described, the information needed to invoke the tool correctly is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter thoroughly. The description largely repeats the schema's parameter text rather than adding new semantic value, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action and resource: 'Create a new Reflect segment' with an ordered list of steps, and explains what segments are for. However, it does not explicitly distinguish itself from the sibling reflect_add_segment, so a 5 for sibling differentiation is not warranted.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives useful context β€” segments are reusable groups of steps referenced from tests and cannot reference other segments β€” which implies when this tool is relevant. It does not state explicit when/when-not conditions or name alternatives, so the usage guidance remains implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_create_testReflect: Create TestAInspect

Create a new Reflect test which contains an ordered list of steps.

Toolset: Tests

Parameters:

  • name (string) required: Name of the test to create.

  • type (enum) required: Platform of the test.

  • description (string): Optional description of the test.

  • deviceProfile (string): Device profile id. Required for a 'web' test; ignored for 'api' and 'native-mobile'. One of: 'desktop', 'tablet', 'mobile'.

  • steps (array) required: Ordered list of steps that make up the test. Web tests must begin with a 'browser-navigate' step. Prefer deterministic, selector-based steps ('click', 'input', 'submit', 'text-validation', 'hover', etc.) over AI-driven 'prompt' steps whenever a stable selector is available. Reserve 'prompt' steps for behavior that can't be expressed with a selector. Any text field in a step (e.g. 'inputText', 'url', 'expectedText', 'requestBody', header values, prompt text) may embed Reflect variable and function references using '${...}' syntax, which are resolved at run time: '${var(name)}' inserts the value of a parameter/variable named 'name' (declare parameters via the top-level 'parameters' field, or assign them mid-run with an 'update-parameters' step); '${sec(name)}' inserts the value of the account secret 'name'. Functions generate dynamic values: '${alphanum(n)}', '${alpha(n)}', '${num(n)}' (random alphanumeric / alphabetic / numeric string of length n), '${range(min, max)}' (random integer, inclusive), '${time(offsetMs)}' and '${datetime(offsetMs)}' (current epoch-millis / date-time, with an optional millisecond offset), and '${date(format, offsetDays)}' (current date formatted with tokens like 'MM/dd/yyyy', with an optional day offset).

  • parameters (array): Optional named parameters (variables) for the test, each with a 'name' and optional default 'value'. Reference a parameter's value inside any step text field with '${var(name)}'.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the test to create.
typeYesPlatform of the test.
stepsYesOrdered list of steps that make up the test. Web tests must begin with a 'browser-navigate' step. Prefer deterministic, selector-based steps ('click', 'input', 'submit', 'text-validation', 'hover', etc.) over AI-driven 'prompt' steps whenever a stable selector is available. Reserve 'prompt' steps for behavior that can't be expressed with a selector. Any text field in a step (e.g. 'inputText', 'url', 'expectedText', 'requestBody', header values, prompt text) may embed Reflect variable and function references using '${...}' syntax, which are resolved at run time: '${var(name)}' inserts the value of a parameter/variable named 'name' (declare parameters via the top-level 'parameters' field, or assign them mid-run with an 'update-parameters' step); '${sec(name)}' inserts the value of the account secret 'name'. Functions generate dynamic values: '${alphanum(n)}', '${alpha(n)}', '${num(n)}' (random alphanumeric / alphabetic / numeric string of length n), '${range(min, max)}' (random integer, inclusive), '${time(offsetMs)}' and '${datetime(offsetMs)}' (current epoch-millis / date-time, with an optional millisecond offset), and '${date(format, offsetDays)}' (current date formatted with tokens like 'MM/dd/yyyy', with an optional day offset).
parametersNoOptional named parameters (variables) for the test, each with a 'name' and optional default 'value'. Reference a parameter's value inside any step text field with '${var(name)}'.
descriptionNoOptional description of the test.
deviceProfileNoDevice profile id. Required for a 'web' test; ignored for 'api' and 'native-mobile'. One of: 'desktop', 'tablet', 'mobile'.

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are all false, giving no safety profile, so the description carries the burden. It states the create operation plainly and adds substantial behavioral context: deviceProfile requirements per type, the ordering constraint for web tests, step-type selection guidance, exact-match semantics for text-validation, and detailed variable/function resolution behavior. It does not mention return values or side effects, but nothing contradicts annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded, but the description is bloated: it re-lists every parameter with text copied from the input schema, including a massive multi-line block on '${...}' variable syntax that already lives in the schema's steps description. Much of the content is redundant and could be replaced with a pointer to the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Input-side guidance is comprehensive: required parameters, step ordering, per-type deviceProfile rules, step-type best practices, and variable syntax are all covered. However, there is no output schema and the description never states what the tool returns (e.g., the created test's ID or confirmation), and it does not mention failure modes or idempotency behavior. This is a notable gap for a creation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description's 'Parameters' section largely duplicates the schema descriptions verbatim, including the long variable-syntax text already present in the steps schema. It adds no new parameter meaning beyond the structured schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Create a new Reflect test which contains an ordered list of steps,' a specific verb+resource statement. It clearly distinguishes this creation tool from sibling Reflect tools like reflect_list_tests, reflect_get_test_detail, and reflect_run_test, which operate on existing tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use vs alternatives or exclusion criteria. The creation purpose is implied by the verb, and the description offers rich guidance on how to construct steps (prefer selector-based steps over 'prompt' steps, web tests must start with 'browser-navigate'), but it never names sibling tools or conditions for choosing this tool over them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_delete_previous_stepReflect: Delete Previous StepA
Destructive
Inspect

Delete the last step added to an active Reflect recording session

Toolset: Recording

Parameters:

  • sessionId (string) required: The ID of the Reflect recording session

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe ID of the Reflect recording session

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=false, so the destructive nature is structurally available. The description adds useful constraints: the session must be active and only the last step is removed. It does not state whether deletion is reversible or what state the session is left in, but this is modest additional context on top of existing annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core action is stated in one clear sentence, with no marketing or filler. The Toolset label is brief, though the Parameters section duplicates schema content without adding information. Overall this is efficient and front-loaded, with minor redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter destructive tool with annotations covering the safety profile, this is nearly sufficient. It identifies the required session ID, the 'active' precondition, and the exact scope ('last step'). It does not describe the response or failure modes, but given the low complexity that is a minor omission.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents sessionId at 100% coverage, and the description's parameter block simply repeats that information verbatim. No additional meaning, such as where to find the session ID or required session state, is added. A baseline of 3 is appropriate because the schema fully covers the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Delete'), resource ('last step'), and context ('active Reflect recording session'). The one-sentence description unambiguously differentiates from sibling tools like reflect_add_prompt_step by targeting the most recently added step, so an agent knows exactly what this action performs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives, such as how to remove an earlier step or whether a different Reflect tool should be used to modify the session. The description only implies the obvious use case (undo the last added step) and omits exclusions or caveats. This is a meaningful gap for a destructive action.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_execute_suiteReflect: Execute SuiteC
Read-onlyIdempotent
Inspect

Execute a reflect suite

Toolset: Suites

Parameters:

  • suiteId (string) required: ID of the reflect suite to execute

ParametersJSON Schema
NameRequiredDescriptionDefault
suiteIdYesID of the reflect suite to execute

TDQS

C2.6/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description says 'execute' a suite, which implies starting a test run and creating an execution β€” a side-effecting operation. This contradicts the readOnlyHint=true annotation, which claims the tool does not modify state. This is an annotation contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the verb, but it spends space repeating the parameter list that already exists in the schema. It is concise but not especially informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an execution tool, the description omits whether execution is asynchronous, what the return value contains, or how to track the resulting run. It also does not mention using reflect_list_suites to discover suite IDs. The single required parameter keeps complexity low, but the missing operational context is significant.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already fully documents suiteId with 100% coverage. The description duplicates the schema text without adding extra meaning such as where to find suiteId, acceptable formats, or default behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

'Execute a reflect suite' names a specific verb and resource, making the core action clear. It does not explicitly differentiate from siblings like reflect_run_test, but the 'suite' resource is enough to identify the primary function.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus reflect_run_test, how to monitor execution, or how to obtain a suiteId. The 'Toolset: Suites' label adds category context but no decision criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_get_screenshotReflect: Get ScreenshotA
Read-onlyIdempotent
Inspect

Capture a screenshot from the current state of an active Reflect recording session

Toolset: Recording

Parameters:

  • sessionId (string) required: The ID of the Reflect recording session

  • format (enum): The image format for the screenshot (png or jpeg)

ParametersJSON Schema
NameRequiredDescriptionDefault
formatNoThe image format for the screenshot (png or jpeg)
sessionIdYesThe ID of the Reflect recording session

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds the active-session precondition and the 'current state' nature of the capture, which is useful, but it does not disclose behavior such as what happens when no active session exists or how the screenshot is returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a clear one-sentence purpose, followed by a compact toolset label and parameter list. It is appropriately short, though the parameter list mostly duplicates the input schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only tool with only two parameters, the description is largely adequate. However, there is no output schema, and the description does not clarify the return format of the screenshot (e.g., binary, base64, URL), nor does it explicitly state the connection prerequisite. This is a meaningful but not severe gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, giving a baseline of 3. The description adds value by qualifying the session as active and placing the screenshot in the current state context, which is not fully captured by the schema's simple 'The ID of the Reflect recording session' description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and object: 'Capture a screenshot from the current state of an active Reflect recording session.' It clearly identifies the resource (Reflect recording session) and distinguishes this operation from sibling Reflect tools that list segments, execute suites, or manage tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool by mentioning an 'active Reflect recording session,' but it does not explicitly state prerequisites, such as connecting to a session first, nor does it provide when-not-to-use guidance or alternatives. It is adequate context but relies on inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_get_suite_execution_statusReflect: Get Suite Execution StatusA
Read-onlyIdempotent
Inspect

Get the status of a reflect suite execution

Toolset: Suites

Parameters:

  • suiteId (string) required: ID of the reflect suite to get execution status for

  • executionId (string) required: ID of the reflect suite execution to get status for

ParametersJSON Schema
NameRequiredDescriptionDefault
suiteIdYesID of the reflect suite to get execution status for
executionIdYesID of the reflect suite execution to get status for

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds no behavioral context beyond 'get status' and does not disclose output shape, possible status values, or error behavior, but it does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short, front-loaded with the purpose, and free of fluff. The duplicated parameter list is somewhat redundant with the schema, but it is not wordy or distracting.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter, read-only status lookup with annotations covering safety, the description is mostly sufficient. However, with no output schema, it leaves return-value shape and possible status values unspecified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents both parameters. The description only repeats the schema's parameter definitions without adding format, constraints, or relationship context between suiteId and executionId.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Get the status') and resource ('reflect suite execution'), with the two identifying IDs. However, it does not explicitly differentiate itself from sibling status-related tools like reflect_get_test_status or reflect_list_suite_executions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: call this tool to retrieve the status of a specific Reflect suite execution. There is no explicit guidance on when to prefer this over related tools, nor any note about how to obtain valid suiteId/executionId values or that executionId may come from a prior execution.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_get_test_detailReflect: Get Test DetailB
Read-onlyIdempotent
Inspect

Get the full detail of a reflect test, including its name, description, and all recorded steps

Toolset: Tests

Parameters:

  • testId (string) required: ID of the reflect test to retrieve details for

ParametersJSON Schema
NameRequiredDescriptionDefault
testIdYesID of the reflect test to retrieve details for

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safe read-only nature is covered. The description adds that the result includes name, description, and all recorded steps, which is useful given there is no output schema, but it does not describe response format, errors, or other behavioral edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is one concise, front-loaded sentence that communicates the essential purpose and return content. The toolset label and parameter block are mildly redundant with the schema, but the overall length is appropriate and not padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter, read-only retrieval tool with strong annotations, the description is mostly complete: it states the required ID and what the response contains. It could be more complete by noting alternatives or exact response shape, but no output schema exists and the description covers the essential invocation context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single testId parameter, and the description's parameter text duplicates the schema exactly. No additional format, example, constraints, or contextual meaning is added beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb-resource pair: getting the full detail of a reflect test. It adds helpful specifics by listing name, description, and all recorded steps, which helps distinguish it from list/status siblings, though it does not explicitly name any alternative sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool instead of siblings such as reflect_list_tests or reflect_get_test_status. The description simply describes the action without exclusions, prerequisites, or alternative routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_get_test_statusReflect: Get Test StatusC
Read-onlyIdempotent
Inspect

Get the status of a reflect test execution

Toolset: Tests

Parameters:

  • executionId (string) required: ID of the reflect test execution to get status for

ParametersJSON Schema
NameRequiredDescriptionDefault
executionIdYesID of the reflect test execution to get status for

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds no behavioral context beyond 'get status'β€”for example, whether this is suitable for polling, how errors are represented, or what status values may be returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core description is a single clear sentence, and the parameter listing is short. However, the parameter block duplicates the input schema and does not add unique value, which keeps it from being a perfect 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does not explain what a status result contains, what statuses are possible, or how the agent should use the response. It also does not connect to the broader reflect workflow (e.g., obtaining an executionId from reflect_run_test). The tool is simple, but key context is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single executionId parameter, and the description repeats the schema's wording almost verbatim. It adds no additional semantics, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('get') and resource ('status of a reflect test execution'), and the tool name and title reinforce this. However, it does not explicitly distinguish itself from closely related sibling tools like reflect_get_suite_execution_status or bearq_get_task_status, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, such as reflect_get_suite_execution_status for suite-level status or other status-checking tools. There is no mention of typical workflows like polling after reflect_run_test, nor any exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_list_segmentsReflect: List SegmentsA
Read-onlyIdempotent
Inspect

Retrieve available reusable test segments for the given platform type. Segments are reusable test steps with an optional set of parameters that can used across multiple tests.

Toolset: Tests

Parameters:

  • platform (enum) required: The platform type to retrieve segments for

  • offset (number): Offset for pagination

  • limit (number): Maximum number of segments to return

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of segments to return
offsetNoOffset for pagination
platformYesThe platform type to retrieve segments for

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the notion of 'available' segments and the reusable-segment definition, but does not disclose much beyond the annotations, such as output format or edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core purpose. The parameter list is slightly redundant with the schema, but it is clearly formatted and does not add unnecessary length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list operation with one required parameter and two optional pagination parameters, the description is sufficient. It explains the resource type, the key platform filter, and pagination controls, though it does not specify the return shape since no output schema is provided.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents all three parameters. The description repeats the parameter names and descriptions but adds no new semantic detail beyond defining what segments are, which is the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence 'Retrieve available reusable test segments for the given platform type' is a specific verb+resource statement that clearly states what the tool does. It also defines what segments are, distinguishing this from sibling list tools like reflect_list_tests and reflect_list_suites.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context that this tool is for retrieving reusable test segments filtered by platform type, and explains the segment concept well enough for an agent to infer when it is appropriate. It does not explicitly name alternatives or exclusions, but the usage context is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_list_suite_executionsReflect: List Suite ExecutionsB
Read-onlyIdempotent
Inspect

List all executions for a given suite

Toolset: Suites

Parameters:

  • suiteId (string) required: ID of the reflect suite to list executions for

ParametersJSON Schema
NameRequiredDescriptionDefault
suiteIdYesID of the reflect suite to list executions for

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds little beyond saying 'all executions', with no mention of pagination, sorting, limits, or response format, but it does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The primary sentence is direct and front-loaded. The 'Toolset' and 'Parameters' sections add some redundancy with the schema, but the overall description is compact and easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read-only list operation, the description is adequate to select and invoke the tool. However, there is no output schema and no description of what an execution entry contains, which leaves the agent guessing about the return shape.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's 'Parameters' section merely repeats the schema text for suiteId. No additional meaning or format details are provided, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'List all executions for a given suite.' This clearly distinguishes the tool from the sibling reflect_get_suite_execution_status (single status) and reflect_list_suites (list suites), even without explicitly naming alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool instead of related siblings such as reflect_get_suite_execution_status, reflect_execute_suite, or reflect_cancel_suite_execution, and no mention of exclusions or prerequisites. The intended use is only implied by the description and title.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_list_suitesReflect: List SuitesB
Read-onlyIdempotent
Inspect

Retrieve a list of all reflect suites available

Toolset: Suites

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare this as read-only, idempotent, and non-destructive, so the description does not need to repeat those facts. It adds minimal new behavioral context beyond stating that a list is retrieved, but for a zero-parameter read-only tool this is not a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main action is front-loaded in a single clear sentence, and the description is appropriately short. The 'Toolset' and 'Parameters' lines are somewhat redundant with the schema, but they do not add meaningful clutter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only listing tool, this description is largely sufficient. It states what the tool returns ('a list of all reflect suites available'), and the annotations cover safety. There is no output schema, but the simple nature of the operation keeps the gap small.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has no parameters, and the description explicitly notes 'Parameters: None,' matching the empty input schema. With zero parameters, the baseline is 4, and the description does not need to explain parameter semantics further.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Retrieve') and resource ('reflect suites'), making the tool's purpose clear. It is implicitly differentiated from sibling tools that list segments, tests, or suite executions, though it does not explicitly name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternative listing tools, and no exclusions or prerequisites are mentioned. The phrase 'all reflect suites available' implies broad use, but the description does not help an agent decide between this and similar reflect listing tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_list_testsReflect: List TestsB
Read-onlyIdempotent
Inspect

List all reflect tests

Toolset: Tests

Parameters: None

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds only the scope word 'all' and does not describe output shape, pagination, or side effects, which is acceptable here because annotations carry most of the behavioral burden.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the main instruction, followed by two compact labels. It is concise with no real filler, though 'List all reflect tests' is somewhat redundant with the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only listing operation, the description plus annotations provide enough information to invoke the tool correctly. The lack of an output schema makes return-value details non-essential, though the description could still mention what a returned test entry contains.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters with 100% schema description coverage, so there are no parameter semantics to clarify. The 0-parameter baseline of 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('List') and a specific resource ('all reflect tests'), making the tool's basic purpose clear. It does not explicitly differentiate itself from sibling listing tools like reflect_list_suites or reflect_list_segments, but it is unambiguous about what it operates on.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus alternatives, and no exclusions are mentioned. The sibling list includes several related listing and execution tools, but the description merely says 'List all reflect tests' and provides no comparative context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reflect_run_testReflect: Run TestC
Read-onlyIdempotent
Inspect

Run a reflect test

Toolset: Tests

Parameters:

  • testId (string) required: ID of the reflect test to run

ParametersJSON Schema
NameRequiredDescriptionDefault
testIdYesID of the reflect test to run

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. But the description adds no behavioral context: it does not say whether the run is asynchronous, whether it returns a test run ID, or how results are retrieved.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded, but much of it duplicates the title and input schema. The 'Toolset' and 'Parameters' sections add structure without contributing meaningful new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a run action with no output schema, the description leaves the agent without information about expected return values, follow-up status checks, or potential failure modes. The tool is simple, but an agent still needs more context to know what happens after invoking it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the single testId parameter is already fully documented. The description repeats the parameter description verbatim and adds no additional semantic guidance about where to find the ID or how it is used.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the specific action 'Run a reflect test' with a clear resource, and the reflect_ prefix distinguishes it from other domains. However, it does not explicitly differentiate it from sibling tools like reflect_execute_suite or reflect_get_test_status, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention when to prefer reflect_execute_suite for suites, nor how reflect_get_test_status or reflect_get_test_detail should be used afterward.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_create_api_from_promptSwagger: Create API from PromptA
Idempotent
Inspect

Generate and save a new API or a new version of an existing API based on a prompt using SmartBear AI. This tool automatically applies organization governance and standardization rules during API generation. The specType parameter determines the format of the generated definition. Use: 'openapi20' for OpenAPI 2.0, 'openapi30x' for OpenAPI 3.0.x, 'openapi31x' for OpenAPI 3.1.x, 'asyncapi2xx' for AsyncAPI 2.x, 'asyncapi30x' for AsyncAPI 3.0.x. Use this tool when creating APIs that comply with governance policies or when generating APIs from natural language descriptions. Use this tool when users ask to create, generate, or design APIs with governance or standardization requirements. Fails with a conflict error if the API version already exists. Returns API details and SwaggerHub URL.

Toolset: Registry API

Parameters:

  • owner (string) required: API owner (organization or user, case-sensitive)

  • apiName (string) required: API name

  • prompt (string) required: The prompt describing the desired API functionality (e.g., 'Create a RESTful API for managing a pet store with endpoints for pets, orders, and inventory')

  • specType (enum): Specification type for the generated API definition. Use: 'openapi20' for OpenAPI 2.0, 'openapi30x' for OpenAPI 3.0.x (default), 'openapi31x' for OpenAPI 3.1.x, 'asyncapi2xx' for AsyncAPI 2.x, 'asyncapi30x' for AsyncAPI 3.0.x (default: "openapi30x")

ParametersJSON Schema
NameRequiredDescriptionDefault
ownerYesAPI owner (organization or user, case-sensitive)
promptYesThe prompt describing the desired API functionality (e.g., 'Create a RESTful API for managing a pet store with endpoints for pets, orders, and inventory')
apiNameYesAPI name
specTypeNoSpecification type for the generated API definition. Use: 'openapi20' for OpenAPI 2.0, 'openapi30x' for OpenAPI 3.0.x (default), 'openapi31x' for OpenAPI 3.1.x, 'asyncapi2xx' for AsyncAPI 2.x, 'asyncapi30x' for AsyncAPI 3.0.xopenapi30x

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
ownerNo
apiNameNo
versionNo
specTypeNo
operationNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate a mutating, non-read-only operation. The description adds meaningful behavioral detail beyond them: automatic governance/standardization application, a conflict error if the version exists, and the fact that API details and a SwaggerHub URL are returned. No contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded and readable, but it contains two overlapping usage sentences and reproduces the parameter descriptions/specType values that are already in the schema. It is adequately sized but not sharply concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a create operation with output schema and full parameter coverage, the description is complete: it states behavior, governance side effects, conflict failure mode, and return value. It does not spell out alternative tools or when not to use it, but that is a minor gap rather than a missing blocker.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The prose repeats the specType enum values and documents owner/API name/prompt, but it does not add semantic meaning beyond what the input schema already provides, such as version handling or format constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource combination: 'Generate and save a new API or a new version of an existing API based on a prompt using SmartBear AI.' This clearly distinguishes it from generic SwaggerHub update/portal/contract tools and scopes the operation to AI-prompt-based API generation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives direct when-to-use guidance: 'Use this tool when creating APIs that comply with governance policies or when generating APIs from natural language descriptions,' plus a second user-intent phrasing. It does not name sibling alternatives or explicitly say when not to use it, so it stops short of 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_create_documentation_pageSwagger: Create Documentation PageA
Idempotent
Inspect

Create a documentation page in a portal product in a single tool call. Supports markdown and html content types. Returns the page location details (productId, sectionId, slug) and a draftUrl to edit it in the portal.

Toolset: Documents

Parameters:

  • portalId (string) required: Portal UUID or subdomain - unique identifier for the portal

  • productId (string) required: Product UUID - unique identifier for the product

  • pageTitle (string) required: Title of the documentation page - will be displayed in navigation (3-255 characters)

  • pageSlug (string): URL slug for the documentation page. 3-255 characters, lowercase, alphanumeric with hyphens, underscores, or dots (e.g. 'my-page'). If not provided, the slug is generated from the page title.

  • pageContent (string): Content of the documentation page. Provide HTML when contentType is 'html', Markdown when contentType is 'markdown'.

  • contentType (enum): Content type of the documentation page. 'markdown' works with both 'internal' and 'external' source. 'html' only works with 'external' source β€” html + internal is not supported by the API and will return an error. (default: "markdown")

  • source (enum): Where the document content is managed. 'internal': editable in both the portal UI and via API. 'external': editable via API only, not in the portal UI. Constraint: 'html' content type only supports 'external' source. (default: "internal")

  • order (number): Order position of the documentation page within its parent section or item (default: 0)

  • parentId (string): Parent table of contents item ID - null for top-level pages, or ID of parent item for nested structure

ParametersJSON Schema
NameRequiredDescriptionDefault
orderNoOrder position of the documentation page within its parent section or item
sourceNoWhere the document content is managed. 'internal': editable in both the portal UI and via API. 'external': editable via API only, not in the portal UI. Constraint: 'html' content type only supports 'external' source.internal
pageSlugNoURL slug for the documentation page. 3-255 characters, lowercase, alphanumeric with hyphens, underscores, or dots (e.g. 'my-page'). If not provided, the slug is generated from the page title.
parentIdNoParent table of contents item ID - null for top-level pages, or ID of parent item for nested structure
portalIdYesPortal UUID or subdomain - unique identifier for the portal
pageTitleYesTitle of the documentation page - will be displayed in navigation (3-255 characters)
productIdYesProduct UUID - unique identifier for the product
contentTypeNoContent type of the documentation page. 'markdown' works with both 'internal' and 'external' source. 'html' only works with 'external' source β€” html + internal is not supported by the API and will return an error.markdown
pageContentNoContent of the documentation page. Provide HTML when contentType is 'html', Markdown when contentType is 'markdown'.

Output Schema

ParametersJSON Schema
NameRequiredDescription
draftUrlNo
productIdNo
sectionIdNo
sectionSlugNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly=false, destructive=false, idempotent=true). The description adds meaningful behavioral context: supported content types, the source/contentType compatibility constraint, and the fact that the response includes page location details and a draftUrl. This goes beyond what the annotations state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The lead sentence is excellent and front-loaded, but the description then repeats the entire schema as a parameter list, which is redundant given the high schema coverage. It is well-structured but not concise; the parameter bullets could be trimmed or omitted without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter create tool, the description is complete: it identifies required and optional parameters, explains the contentType/source interaction, and summarizes the return values (productId, sectionId, slug, draftUrl). The output details are especially useful since they tell the agent what to expect without needing to infer from an output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter list essentially restates the schema descriptions rather than adding new meaning. The baseline of 3 applies because the schema carries the parameter documentation; the description provides no additional semantic value for parameters beyond what is already structured.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Create a documentation page in a portal product in a single tool call.' It also names the supported content types and the return payload, which clearly differentiates this creation action from sibling tools like swagger_get_document, swagger_update_document, and swagger_create_portal_product.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly establishes when to use the tool: when creating a new documentation page in a portal product. It does not explicitly contrast with update or get sibling tools, but the creation intent is unambiguous and the constraint that 'html + internal is not supported' helps avoid a known failure mode.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_create_or_update_apiSwagger: Create or Update APIB
DestructiveIdempotent
Inspect

Create a new API or update an existing API in SwaggerHub Registry for Swagger Studio. The API specification type (OpenAPI, AsyncAPI) is automatically detected from the definition content. APIs are always created with fixed values: version 1.0.0, private visibility, and automock disabled (these values cannot be changed). Returns HTTP 201 for creation, HTTP 200 for update. Response includes 'operation' field indicating whether it was a 'create' or 'update' operation along with API details and SwaggerHub URL.

Toolset: Registry API

Parameters:

  • owner (string) required: Organization name (owner of the API)

  • apiName (string) required: API name

  • definition (string) required: API definition content (OpenAPI/AsyncAPI specification in JSON or YAML format). Format is automatically detected. On create, fixed values are used: version 1.0.0, private visibility, automock disabled, and no project assignment. On update, the API's existing visibility is preserved.

ParametersJSON Schema
NameRequiredDescriptionDefault
ownerYesOrganization name (owner of the API)
apiNameYesAPI name
definitionYesAPI definition content (OpenAPI/AsyncAPI specification in JSON or YAML format). Format is automatically detected. On create, fixed values are used: version 1.0.0, private visibility, automock disabled, and no project assignment. On update, the API's existing visibility is preserved.

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
ownerNo
apiNameNo
versionNo
operationNo

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral context beyond the annotations: automatic spec-type detection, unchangeable fixed values on create, HTTP 201/200 responses, the 'operation' response field, and preservation of existing visibility on update. These details give the agent a clear picture of side effects and outcomes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well organized and front-loaded with the core purpose, but it repeats the fixed-value information in both the main paragraph and the 'definition' parameter description. The 'Toolset: Registry API' line adds little, and the redundancy could be trimmed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 100% parameter coverage, output schema, and annotations, the description covers the essential call semantics, fixed-value constraints, auto-detection, and response behavior. It is complete enough for correct invocation, though it would benefit from explicit update-overwrite semantics and alternative-tool guidance.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents the parameters. The description's parameter list largely duplicates the schema rather than adding new semantic meaning, though it does restate the important create/update behavior for the 'definition' parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the actionβ€”'Create a new API or update an existing API in SwaggerHub Registry'β€”and identifies the resource. It doesn't explicitly differentiate itself from siblings like swagger_patch_api or swagger_create_api_from_prompt, but the primary purpose is unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives such as swagger_patch_api or swagger_create_api_from_prompt. The description explains behavior but not selection criteria, prerequisites, or situations where another tool would be preferred.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_create_portalSwagger: Create PortalA
Idempotent
Inspect

Create a new portal within Swagger.

Toolset: Portals

Parameters:

  • name (string): The display name for the portal - shown to users and in branding (3-40 characters)

  • subdomain (string) required: The portal subdomain - used in the portal URL (e.g., 'myportal' for myportal.example.com). Must be unique, lowercase, 3-20 characters, alphanumeric with hyphens. Recommended: slugified organization name plus a random 3-character suffix (e.g., 'acmecorp-k7p') to avoid collisions

  • offline (boolean): If true, the portal will not be visible to customers - useful for development/staging environments. Defaults to false

  • routing (string): Routing strategy for the portal - either 'browser' (client-side routing) or 'proxy' (server-side routing). Defaults to 'browser'

  • credentialsEnabled (boolean): Whether authentication credentials are enabled for accessing the portal. When true, users can authenticate to access private content. Defaults to true

  • swaggerHubOrganizationId (string) required: The corresponding Swagger organization UUID - required for portal creation. This links the portal to your Swagger organization. Only one Portal per Swagger organization is allowed.

  • openapiRenderer (string): OpenAPI renderer type: 'SWAGGER_UI' (Swagger UI), 'ELEMENTS' (Stoplight Elements), or 'TOGGLE' (allows switching between both with Elements as default). Defaults to 'TOGGLE'

  • pageContentFormat (string): Format for page content rendering - determines how documentation pages are processed: 'HTML', 'MARKDOWN', or 'BOTH'. Defaults to 'HTML'

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoThe display name for the portal - shown to users and in branding (3-40 characters)
offlineNoIf true, the portal will not be visible to customers - useful for development/staging environments. Defaults to false
routingNoRouting strategy for the portal - either 'browser' (client-side routing) or 'proxy' (server-side routing). Defaults to 'browser'
subdomainYesThe portal subdomain - used in the portal URL (e.g., 'myportal' for myportal.example.com). Must be unique, lowercase, 3-20 characters, alphanumeric with hyphens. Recommended: slugified organization name plus a random 3-character suffix (e.g., 'acmecorp-k7p') to avoid collisions
openapiRendererNoOpenAPI renderer type: 'SWAGGER_UI' (Swagger UI), 'ELEMENTS' (Stoplight Elements), or 'TOGGLE' (allows switching between both with Elements as default). Defaults to 'TOGGLE'
pageContentFormatNoFormat for page content rendering - determines how documentation pages are processed: 'HTML', 'MARKDOWN', or 'BOTH'. Defaults to 'HTML'
credentialsEnabledNoWhether authentication credentials are enabled for accessing the portal. When true, users can authenticate to access private content. Defaults to true
swaggerHubOrganizationIdYesThe corresponding Swagger organization UUID - required for portal creation. This links the portal to your Swagger organization. Only one Portal per Swagger organization is allowed.

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
nameNo
offlineNo
routingNo
subdomainNo
customDomainNo
openapiRendererNo
credentialsEnabledNo
swaggerHubOrganizationIdNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds useful behavioral context beyond the annotations: one-portal-per-organization, offline portals not being customer-visible, and credentials controlling access to private content. It does not contradict the readOnlyHint=false or destructiveHint=false annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence and bold section headers make the description easy to scan. However, the full parameter list duplicates the input schema, making the description longer and more redundant than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter creation tool, the description covers required parameters, defaults, allowed values, and the uniqueness constraint, and an output schema exists so return details need not be explained. A brief pointer to update/get siblings would have made it fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents every parameter. The description's parameter section essentially repeats those same descriptions, adding no meaningful new meaning beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Create a new portal within Swagger,' which names a specific verb and resource. The 'Toolset: Portals' label and the create-focused wording clearly distinguish it from siblings such as swagger_update_portal and swagger_get_portal.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The create verb plus the explicit constraint 'Only one Portal per Swagger organization is allowed' gives clear context for when this tool applies. It does not explicitly compare against swagger_update_portal or state exclusions, so it stops short of full when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_create_portal_productSwagger: Create Portal ProductA
Idempotent
Inspect

Create a new product for a specific portal.

Toolset: Products

Parameters:

  • portalId (string) required: Portal UUID or subdomain - unique identifier for the portal instance

  • type (string) required: Product creation type - 'new' to create from scratch or 'copy' to duplicate an existing product

  • productId (string): Source product UUID to copy from - required when type is 'copy', specifies which existing product to duplicate. Omit when type is 'new'

  • name (string) required: Product display name - will be shown to users in the portal navigation and product listings (3-40 characters)

  • slug (string) required: URL-friendly identifier for the product - must be unique within the portal, used in URLs (e.g., 'my-api' becomes /my-api). 3-22 characters, lowercase, alphanumeric with hyphens, underscores, or dots

  • description (string): Product description - explains what the API/product does, shown in product listings and cards (max 110 characters)

  • public (boolean): Whether the product is publicly visible to all portal visitors - false means only authenticated users with appropriate roles can access it

  • hidden (boolean): Whether the product is hidden from the portal landing page navigation menus - useful for internal or draft products

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesProduct display name - will be shown to users in the portal navigation and product listings (3-40 characters)
slugYesURL-friendly identifier for the product - must be unique within the portal, used in URLs (e.g., 'my-api' becomes /my-api). 3-22 characters, lowercase, alphanumeric with hyphens, underscores, or dots
typeYesProduct creation type - 'new' to create from scratch or 'copy' to duplicate an existing product
hiddenNoWhether the product is hidden from the portal landing page navigation menus - useful for internal or draft products
publicNoWhether the product is publicly visible to all portal visitors - false means only authenticated users with appropriate roles can access it
portalIdYesPortal UUID or subdomain - unique identifier for the portal instance
productIdNoSource product UUID to copy from - required when type is 'copy', specifies which existing product to duplicate. Omit when type is 'new'
descriptionNoProduct description - explains what the API/product does, shown in product listings and cards (max 110 characters)

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
urlNo
nameNo
slugNo
hiddenNo
publicNo
portalIdNo
descriptionNo

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate invasive=false, readOnlyHint=false, and idempotentHint=true, so the safety profile is covered. The description adds useful contextual behavior around uniqueness, visibility, and the copy-vs-new flow. However, it mostly mirrors the schema descriptions and does not clarify idempotency behavior or what happens on conflicts such as duplicate slugs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with a one-line purpose followed by a bulleted parameter list, making it easy to scan. It is somewhat long because it duplicates schema descriptions, but every bullet carries relevant details and the opening sentence is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a create operation with 8 parameters, the description covers all parameters, required fields, the copy/new conditional, and field-level behavior such as uniqueness and visibility. An output schema exists and annotations cover idempotency and safety. Minor gaps remain around default values and what a copy operation duplicates beyond the product record.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description restates the same parameter documentation, including the conditional requirement for productId when type is 'copy'. It adds no meaning beyond what the schema already provides, though the brief examples like 'my-api' aids comprehension.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Create'), a specific resource ('product'), and a scope ('for a specific portal'), which clearly distinguishes it from sibling tools like swagger_get_portal_product, swagger_update_portal_product, and swagger_delete_portal_product. The toolset label 'Products' further anchors its purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies it is for creating a new portal product, which is a useful context signal. However, it does not explicitly mention when not to use it or point to alternatives such as swagger_update_portal_product for existing products. Usage guidance is implied by the verb and name rather than explicitly stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_create_table_of_contentsSwagger: Create Table Of ContentsA
Idempotent
Inspect

Create a new table of contents item in a portal product section. Supports API references, HTML content, and Markdown content types.

Toolset: Table Of Contents

Parameters:

  • sectionId (string) required: Section ID - unique identifier for the section within the product

  • type (enum) required: Type of table of contents creation - 'new' to create from scratch or 'copy' to duplicate an existing one

  • title (string) required: Title of the table of contents item - will be displayed in navigation (3-40 characters)

  • slug (string) required: URL-friendly identifier for the table of contents item - must be unique within the section (3-22 characters, lowercase, alphanumeric with hyphens/underscores/dots)

  • order (number) required: Order position of the table of contents item within its parent section or item

  • parentId (string): Parent table of contents item ID - null for top-level items, or ID of parent item for nested structure

  • content (object): Content configuration for the table of contents item

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesURL-friendly identifier for the table of contents item - must be unique within the section (3-22 characters, lowercase, alphanumeric with hyphens/underscores/dots)
typeYesType of table of contents creation - 'new' to create from scratch or 'copy' to duplicate an existing one
orderYesOrder position of the table of contents item within its parent section or item
titleYesTitle of the table of contents item - will be displayed in navigation (3-40 characters)
contentNoContent configuration for the table of contents item
parentIdNoParent table of contents item ID - null for top-level items, or ID of parent item for nested structure
sectionIdYesSection ID - unique identifier for the section within the product

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
documentIdNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false and destructiveHint=false, and the description's 'Create' is consistent. It adds concrete effects: title 'will be displayed in navigation,' slug 'must be unique within the section,' and type 'copy' duplicates an existing item, which goes beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main purpose is front-loaded in the first sentence, which is good. However, the entire parameter list is repeated, duplicating the input schema and adding bulk without new information. It is structured but not as lean as it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complex nested content object and existing output schema, the description supplies essential context: what a TOC item is, where it lives, what content types are allowed, and the roles of each parameter. It does not explain the 'copy' source mechanics or nested conditional rules, but the input schema covers those constraints.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description's parameter list closely mirrors the schema descriptionsβ€”e.g., slug uniqueness and title lengthβ€”without introducing new meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb-object pair: 'Create a new table of contents item in a portal product section,' which clearly distinguishes it from siblings like swagger_list_table_of_contents and swagger_delete_table_of_contents. It also names the supported content types (API references, HTML, Markdown), making the resource scope explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'in a portal product section' and the Toolset label convey the intended context. Content-type support clarifies capability, though the description does not explicitly state when to prefer this over swagger_create_documentation_page or mention exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_delete_portal_productSwagger: Delete Portal ProductA
DestructiveIdempotent
Inspect

Delete a product from a specific portal

Toolset: Products

Parameters:

  • productId (string) required: Product UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product

ParametersJSON Schema
NameRequiredDescriptionDefault
productIdYesProduct UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product

Output Schema

ParametersJSON Schema
NameRequiredDescription
messageYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true, so the description only needs to add context rather than repeat the danger. It adds the useful nuance that deletion is scoped to a specific portal, but it doesn't state whether deletion is permanent or has cascading effects; the annotations cover the basic destructive profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The lead sentence is short, specific, and front-loaded. The subsequent 'Toolset' and 'Parameters' sections are mostly redundant with the schema, but the overall size is still small and not distracting.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter delete operation with a rich annotation set and an output schema, the description is nearly complete. It identifies the operand (portal product), the required identifier format, and the portal-scoped behavior; only irreversible side effects are left implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents productId fully. The description repeats the same text and adds no new meaning beyond what is structured, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific action and resource: 'Delete a product from a specific portal.' This clearly distinguishes it from sibling tools like swagger_create_portal_product or swagger_update_portal_product, and the title reinforces the same meaning.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: whenever a product needs to be removed from a particular portal. It does not explicitly name alternatives or exclusions, but the delete verb plus the portal-scoping phrase provides enough context for an agent to choose it over create/update/list siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_delete_table_of_contentsSwagger: Delete Table Of ContentsA
DestructiveIdempotent
Inspect

Delete table of contents entry. Performs a soft-delete of an entry from the table of contents. Supports recursive deletion of nested items.

Toolset: Table Of Contents

Parameters:

  • tableOfContentsId (string) required: The table of contents UUID, or identifier in the format 'portal-subdomain:product-slug:section-slug:table-of-contents-slug'

  • recursive (boolean): Flag to include all the nested tables of contents (default: false)

ParametersJSON Schema
NameRequiredDescriptionDefault
recursiveNoFlag to include all the nested tables of contents (default: false)
tableOfContentsIdYesThe table of contents UUID, or identifier in the format 'portal-subdomain:product-slug:section-slug:table-of-contents-slug'

Output Schema

ParametersJSON Schema
NameRequiredDescription
successNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true, and the description adds meaningful behavioral context by stating this is a soft-delete and supports recursive deletion of nested items. It does not contradict annotations, and the added soft-delete detail is valuable beyond the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, front-loaded with the core action, and includes useful details like soft-delete and recursion. The parameter list duplicates schema content but is brief and aids quick scanning, so it is not wasteful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and full parameter coverage, the description covers the essential behavior: soft-delete, recursive support, and identifier format. It lacks deeper caveats like recoverability or permission requirements, but for a two-parameter tool this is reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents both parameters. The description repeats the parameter meanings without adding extra nuance, which meets the baseline but does not elevate beyond it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description opens with the direct verb 'Delete' and the precise resource 'table of contents entry', then clarifies the soft-delete nature. This is specific enough to distinguish from sibling tools like swagger_list_table_of_contents and swagger_create_table_of_contents.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies this tool is for deleting TOC entries and mentions recursive deletion as an option, but it does not state when to choose this over alternatives or provide exclusions. The soft-delete and recursive behavior give context, but no explicit when/when-not guidance is present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_get_api_definitionSwagger: Get API DefinitionA
Read-onlyIdempotent
Inspect

Fetch resolved API definition from SwaggerHub Registry based on owner, API name, and version. By default returns the definition as JSON (converting YAML-stored definitions to JSON). Set format:'text' to get the YAML definition β€” required as the source for swagger_patch_api edits.

Toolset: Registry API

Parameters:

  • owner (string) required: API owner (organization or user, case-sensitive)

  • api (string) required: API name (case-sensitive)

  • version (string) required: Version identifier

  • resolved (boolean): Set to true to get the resolved version with all external $refs included (default false)

  • flatten (boolean): Set to true to create models from inline schemas in OpenAPI definition (default false)

  • format (enum): Response format: 'json' (default) may convert YAML to JSON; 'text' returns the definition as YAML β€” required for swagger_patch_api edits.

ParametersJSON Schema
NameRequiredDescriptionDefault
apiYesAPI name (case-sensitive)
ownerYesAPI owner (organization or user, case-sensitive)
formatNoResponse format: 'json' (default) may convert YAML to JSON; 'text' returns the definition as YAML β€” required for swagger_patch_api edits.
flattenNoSet to true to create models from inline schemas in OpenAPI definition (default false)
versionYesVersion identifier
resolvedNoSet to true to get the resolved version with all external $refs included (default false)

Output Schema

ParametersJSON Schema
NameRequiredDescription
definitionYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds useful behavior beyond the annotations: it discloses YAML-to-JSON conversion by default, explains the resolved and flatten options, and specifies that format:'text' returns YAML for patching. These details complement the readOnlyHint and idempotentHint annotations and do not contradict them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded with the purpose and key caveats before the parameter list. It is reasonably concise, but the parameter bullet list duplicates the input schema, which keeps it from being maximally economical.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only lookup with a fully documented schema and an available output schema, the description covers the core selection criteria, default format behavior, optional transformation flags, and the patch-workflow use case. Nothing essential for invoking the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter list in the description largely repeats the input schema verbatim. It adds no substantially new meaning beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a clear verb and resource: 'Fetch resolved API definition from SwaggerHub Registry based on owner, API name, and version.' This identifies the exact lookup operation and differentiates it from sibling tools like swagger_patch_api and swagger_search_apis_and_domains, even without naming them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear operational context: default JSON conversion behavior is explained, and format:'text' is explicitly described as required for swagger_patch_api edits. It does not explicitly mention alternatives for searching or listing definitions, but the usage context is strong and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_get_documentSwagger: Get DocumentA
Read-onlyIdempotent
Inspect

Get document content and metadata by document ID. Useful for retrieving HTML or Markdown content from table of contents items.

Toolset: Documents

Parameters:

  • documentId (string) required: Document UUID - unique identifier for the document

ParametersJSON Schema
NameRequiredDescriptionDefault
documentIdYesDocument UUID - unique identifier for the document

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
typeNo
sourceNo
contentNo
createdAtNo
updatedAtNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, and non-destructive behavior. The description adds useful behavioral context by specifying the content types (HTML or Markdown) and source context (table of contents items), which goes beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose and use case are front-loaded in two concise sentences. The parameter listing is redundant with the schema but not overly bloated, keeping the description easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read tool with rich annotations and an output schema, the description covers what the tool does, when to use it, and what content it returns. Nothing necessary for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description only repeats the schema's parameter documentation without adding extra meaning. This matches the baseline for a single fully documented parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Get document content and metadata by document ID.' The additional note about retrieving HTML or Markdown from table of contents items clearly distinguishes it from sibling tools like swagger_update_document or swagger_get_api_definition.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context by saying it is useful for retrieving HTML or Markdown content from table of contents items. It does not explicitly name alternatives or when-not-to-use, but the use case is specific enough to guide selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_get_portalSwagger: Get PortalB
Read-onlyIdempotent
Inspect

Retrieve information about a specific portal.

Toolset: Portals

Parameters:

  • portalId (string) required: Portal UUID or subdomain - unique identifier for the portal instance

ParametersJSON Schema
NameRequiredDescriptionDefault
portalIdYesPortal UUID or subdomain - unique identifier for the portal instance

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
nameNo
offlineNo
routingNo
subdomainNo
customDomainNo
openapiRendererNo
credentialsEnabledNo
swaggerHubOrganizationIdNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description's read-only language aligns with the annotations but adds no further behavioral context such as error cases, authentication requirements, or response characteristics beyond what the output schema already covers.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded. However, the 'Toolset: Portals' line and the parameter bullet add little value because the input schema already documents the parameter and the toolset is inferable from the tool name and siblings.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple one-parameter, read-only getter with rich annotations and an output schema, so the description is largely complete. The main gap is not telling callers to use swagger_list_portals when they need to discover the portalId first.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the parameter listed in the description ('Portal UUID or subdomain - unique identifier for the portal instance') repeats the schema's own description verbatim. It adds no extra semantic detail like examples, format expectations, or guidance on choosing between UUID vs subdomain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: 'Retrieve information about a specific portal.' The word 'specific' distinguishes this from swagger_list_portals, but it does not explicitly name sibling alternatives or explain how it differs from swagger_resolve_organization_portal.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: call this when you have a portal UUID or subdomain and need that portal's information. However, there is no explicit when-to-use guidance, no mention of alternatives like swagger_list_portals for discovering portal IDs, and no exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_get_portal_productSwagger: Get Portal ProductC
Read-onlyIdempotent
Inspect

Retrieve information about a specific product resource.

Toolset: Products

Parameters:

  • productId (string) required: Product UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product

ParametersJSON Schema
NameRequiredDescriptionDefault
productIdYesProduct UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
urlNo
nameNo
slugNo
hiddenNo
publicNo
portalIdNo
descriptionNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, idempotent, and non-destructive behavior. The description adds no further behavioral context such as auth requirements, response characteristics, or side effects; it mostly restates that this is a retrieval operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is clear and concise, but the Parameters section duplicates the input schema exactly. Not every sentence earns its place, so the description is acceptable but not tightly optimized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, read-only, idempotent get operation with an output schema and safety annotations, the description is largely sufficient. The main gap is lack of routing guidance versus sibling product tools, which is already accounted for in Usage Guidelines.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description's parameter block repeats the schema's productId description verbatim. Since the schema already documents the UUID or 'portal-subdomain:product-slug' format, the description adds no new semantic meaning beyond the structured input.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Retrieve information about a specific product resource.' The word 'specific' distinguishes it from list-type operations, though it does not explicitly name sibling tools like swagger_list_portal_products.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit guidance on when to use this tool versus alternatives such as swagger_list_portal_products or swagger_get_portal. The phrase 'specific product resource' only implies a get-by-id use case; no exclusions, prerequisites, or alternative routing are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_list_organizationsSwagger: List OrganizationsA
Read-onlyIdempotent
Inspect

Get organizations for a user. Returns a list of organizations that the authenticating user is a member of. On-Premise admin gets a list of all organizations in the system.

Toolset: Registry API

Parameters:

  • q (string): Search organizations by partial or full name (case-insensitive)

  • sortBy (enum): The property to sort the results by

  • order (enum): Sort order

  • page (number): 0-based index of the page to return

  • pageSize (number): Number of results per page to return

ParametersJSON Schema
NameRequiredDescriptionDefault
qNoSearch organizations by partial or full name (case-insensitive)
pageNo0-based index of the page to return
orderNoSort order
sortByNoThe property to sort the results by
pageSizeNoNumber of results per page to return

Output Schema

ParametersJSON Schema
NameRequiredDescription
pageNo
itemsNo
pageSizeNo
totalCountNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior. The description adds valuable behavioral context beyond those annotations by explaining access scope: regular users see their own organizations, while On-Premise admins see all organizations. No hidden side effects are implied.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is front-loaded and clear. However, the 'Parameters' section duplicates the input schema, and the 'Toolset: Registry API' line adds limited value, so not every part of the description earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity, read-only list operation with zero required parameters, full schema coverage, an output schema, and safety annotations, the description provides the necessary missing context: who the results are scoped to and how admins differ. Nothing needed for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter block largely repeats the schema's descriptions verbatim. It adds no new detail such as defaults, valid enum values, or boundary semantics beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource ('Get organizations') and clarifies scope by saying it returns the authenticating user's memberships, with a special case for On-Premise admins. It is clear, though it does not explicitly differentiate itself from sibling list tools such as swagger_list_portals.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied by the name and the 'returns a list of organizations' phrasing, and the Toolset label gives context. However, there is no explicit statement of when to use this endpoint versus alternatives (e.g., portal lists or search endpoints), nor any when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_list_portal_productsSwagger: List Portal ProductsB
Read-onlyIdempotent
Inspect

Get products for a specific portal that match your criteria.

Toolset: Products

Parameters:

  • portalId (string) required: Portal UUID or subdomain - unique identifier for the portal instance

ParametersJSON Schema
NameRequiredDescriptionDefault
portalIdYesPortal UUID or subdomain - unique identifier for the portal instance

Output Schema

ParametersJSON Schema
NameRequiredDescription
pageNo
itemsNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish that this is read-only, idempotent, and non-destructive, so the description does not need to restate safety. It adds no meaningful behavioral context beyond the annotated traits; 'match your criteria' is vague and not backed by additional parameters. There is no contradiction with annotations, so a mid-range score is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main sentence is concise and front-loaded with the action. However, the description includes a redundant 'Toolset: Products' label and a parameters section that duplicates the input schema, which means not every element earns its place. It is compact but not zero-waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with a single required parameter, strong read-only annotations, and an output schema, the description plus structured metadata is largely sufficient to invoke the tool correctly. The main gap is the lack of routing among sibling tools, but that is partly covered by the tool's name and the clear portal scope.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage for the only parameter, portalId, and the description repeats the same semantic content rather than adding new meaning. Per the baseline rule, a 3 is appropriate when the schema already documents parameters fully. The vague 'match your criteria' hint is not actionable enough to raise the score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a clear verb and resource: 'Get products for a specific portal'. The plural 'products' and the portal scope distinguish it from sibling tools such as swagger_get_portal_product and swagger_list_portals. However, it doesn't explicitly differentiate from those alternatives or clarify what 'match your criteria' means, so it falls just short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus sibling tools like swagger_get_portal_product, swagger_list_portals, or swagger_list_portal_product_sections. The phrase 'for a specific portal' implies a scoping, but no exclusions, alternatives, or selection criteria are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_list_portal_product_sectionsSwagger: List Portal Product SectionsA
Read-onlyIdempotent
Inspect

Get sections for a specific product within a portal.

Toolset: Sections

Parameters:

  • productId (string) required: Product UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product

  • embed (array): List of related entities to embed in the response - e.g., ['tableOfContents', 'tableOfContents.swaggerhubApi'] to include table of contents and SwaggerHub API details

  • page (number): Page number for paginated results - specifies which page of results to retrieve (default is 1)

  • size (number): Number of items per page for pagination - controls how many results are returned per page (default is 10)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number for paginated results - specifies which page of results to retrieve (default is 1)
sizeNoNumber of items per page for pagination - controls how many results are returned per page (default is 10)
embedNoList of related entities to embed in the response - e.g., ['tableOfContents', 'tableOfContents.swaggerhubApi'] to include table of contents and SwaggerHub API details
productIdYesProduct UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product

Output Schema

ParametersJSON Schema
NameRequiredDescription
pageNo
itemsNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds little behavioral context beyond what the schema provides, such as embed behavior and pagination defaults, but no additional side-effect or rate-limit information. This is acceptable given the strong annotations, but not particularly rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core action, then uses a clean bulleted parameter list. However, the parameter section largely duplicates the schema descriptions, so not every line adds unique value. Still, it is well-organized and easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a straightforward read-only listing tool with one required parameter, an output schema, and clear annotations, the description is largely sufficient. It covers the resource scope and all parameters. It lacks only explicit guidance about when to use it relative to sibling tools, but that is not essential for invoking it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema fully documents all four parameters. The description repeats those parameter explanations without adding substantive new meaning beyond the schema, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear, specific action: 'Get sections for a specific product within a portal.' It identifies the exact resource (sections), the scope (product within portal), and is easily distinguished from sibling tools like swagger_list_portal_products or swagger_get_portal_product. The Toolset: Sections label reinforces the resource type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to choose this tool over alternatives, nor does it mention any exclusions or related tools. It only restates the tool's purpose and parameters. An agent must infer usage solely from the action and resource description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_list_portalsSwagger: List PortalsA
Read-onlyIdempotent
Inspect

Search for available portals within Swagger. Only portals where you have at least a designer role, either at the product level or organization level, are returned.

Toolset: Portals

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
pageNo
itemsNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool as read-only, idempotent, and non-destructive. The description adds valuable behavioral context beyond that: results are filtered by the user's role (at least designer at product or organization level). This gives the agent a clear expectation of visibility without needing to call the tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short, front-loaded, and directly conveys the core purpose and an important filtering rule. The 'Toolset: Portals' line is minor but helps contextualize the domain without adding meaningful overhead.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only list operation with rich annotations and an output schema, the description is complete. It tells the agent what the tool returns and under what conditions, which is sufficient for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description adds no parameter-specific information, but none is needed since the input schema is empty and schema description coverage is effectively complete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the operation ('Search for available portals within Swagger') and the specific resource (portals). It also adds a meaningful scope: only portals where the user has at least a designer role are returned. However, it does not explicitly distinguish itself from sibling tools like swagger_get_portal, though the plural 'portals' and the role filter make the listing intent fairly clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this tool is used to find portals the user can access and is appropriate for listing rather than retrieving a single portal. It does not explicitly state when to use this tool over alternatives like swagger_get_portal or swagger_list_portal_products, so usage guidance is mostly inferential.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_list_table_of_contentsSwagger: List Table Of ContentsA
Read-onlyIdempotent
Inspect

Get table of contents for a section of a product within a portal.

Toolset: Table Of Contents

Parameters:

  • sectionId (string) required: Section ID - unique identifier for the section within the product

  • embed (array): List of related entities to embed in the response - e.g., ['swaggerhubApi'] to include SwaggerHub API details

  • page (number): Page number for paginated results - specifies which page of results to retrieve (default is 1)

  • size (number): Number of items per page for pagination - controls how many results are returned per page (default is 20)

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number for paginated results - specifies which page of results to retrieve (default is 1)
sizeNoNumber of items per page for pagination - controls how many results are returned per page (default is 20)
embedNoList of related entities to embed in the response - e.g., ['swaggerhubApi'] to include SwaggerHub API details
sectionIdYesSection ID - unique identifier for the section within the product

Output Schema

ParametersJSON Schema
NameRequiredDescription
itemsYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds the section/product/portal scoping and pagination defaults, but it does not disclose any additional behavioral context such as response shape, embed limitations, or error conditions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded, and the toolset heading is useful. However, the parameter bullets are redundant with the input schema and add unnecessary length without contributing new information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and annotations covering the read-only, idempotent, non-destructive behavior, the description is sufficient for calling the tool: it states the hierarchy, required sectionId, and pagination defaults. The only notable gap is the lack of explicit guidance on when to use this versus sibling TOC tools, but that is not blocking for invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter section restates the schema's descriptions almost verbatim. It therefore adds no new semantic meaning beyond what the agent can already read from the input schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Get') and a precise resource: 'table of contents for a section of a product within a portal.' This clearly differentiates it from sibling tools operating on portals, products, sections, or on creating/deleting a table of contents.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The resource hierarchy makes the intended use fairly clear, and the 'Toolset: Table Of Contents' heading adds context. However, it never explicitly says when to prefer this tool over sibling create/delete TOC tools or how to obtain a sectionId, so guidance is mostly implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_patch_apiSwagger: Patch APIA
DestructiveIdempotent
Inspect

Apply targeted search/replace edits to a YAML API definition in SwaggerHub Registry. Only OpenAPI and AsyncAPI definitions are supported. Each edit's 'oldString' must be copied exactly from swagger_get_api_definition with format:'text' and without 'resolved' or 'flatten' β€” those options transform the definition, so edits based on them will not match the stored source. JSON definitions are not supported. Nothing is saved unless every edit applies β€” failed edits are returned with 'no_match' or 'ambiguous' status. When 'newVersion' is given the patched definition is saved as a new private version; patching in place keeps the visibility of the existing version. Use this tool to fix specific issues in an existing API without regenerating the whole definition.

Toolset: Registry API

Parameters:

  • owner (string) required: API owner (organization or user, case-sensitive)

  • apiName (string) required: API name (case-sensitive)

  • version (string) required: Version of the definition to patch (base version, e.g. '1.0.0')

  • newVersion (string): Version to save the patched definition as (e.g. '1.0.1'). Must not already exist and is always created as private. Omit to overwrite the base version, which keeps its current visibility. The definition's info.version is updated automatically β€” do not add an edit for it.

  • edits (array) required: Search/replace edits applied sequentially to the YAML definition. Nothing is saved unless every edit applies (atomic).

ParametersJSON Schema
NameRequiredDescriptionDefault
editsYesSearch/replace edits applied sequentially to the YAML definition. Nothing is saved unless every edit applies (atomic).
ownerYesAPI owner (organization or user, case-sensitive)
apiNameYesAPI name (case-sensitive)
versionYesVersion of the definition to patch (base version, e.g. '1.0.0')
newVersionNoVersion to save the patched definition as (e.g. '1.0.1'). Must not already exist and is always created as private. Omit to overwrite the base version, which keeps its current visibility. The definition's info.version is updated automatically β€” do not add an edit for it.

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
savedNo
failedNoOnly present when at least one edit failed
versionNo
operationNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint, idempotentHint), the description reveals atomicity ('Nothing is saved unless every edit applies'), failed-edit statuses ('no_match' or 'ambiguous'), version semantics (newVersion saved as private, in-place keeps visibility, info.version auto-updated), and the resolved/flatten matching trap. All of this is context an agent needs to predict side effects, and nothing contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The narrative is front-loaded and every sentence in it earns its place, but the embedded Parameters section duplicates the input schema nearly verbatim, adding length without new information. Still, the key behavioral details are compactly ordered before the parameter list.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-param tool with a nested edits array, the description plus schema covers everything needed: supported formats, exact-match sourcing, atomicity, version behavior, visibility, and the intended use case. The presence of an output schema covers return value details, so nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema's per-parameter descriptions are rich (case-sensitivity, base version, private version, atomic edits, oldString sourcing). The description's Parameters section mostly duplicates the schema rather than adding new meaning, so it stays at the baseline for full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb+resource: 'Apply targeted search/replace edits to a YAML API definition in SwaggerHub Registry.' It names supported formats (OpenAPI/AsyncAPI) and excludes JSON, distinguishing it from broader tools like swagger_create_or_update_api or swagger_standardize_api. The final sentence reinforces its niche: fixing specific issues without regenerating the whole definition.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit use case: 'Use this tool to fix specific issues in an existing API without regenerating the whole definition.' It also states hard constraints (only OpenAPI/AsyncAPI, JSON not supported) and references swagger_get_api_definition for the source text. However, it doesn't name an explicit alternative for full regeneration, leaving that to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_publish_portal_productSwagger: Publish Portal ProductA
DestructiveIdempotent
Inspect

Publish a product's content to make it live or as preview. This endpoint publishes the current content of a product, making it visible to portal visitors. Use preview mode to test before going live. Optionally provide tableOfContentsId to get a page-specific URL. Returns publication status, a live or preview URL (null if URL building fails), product and portal metadata, and an optional warning when metadata/URL building failed β€” a warning does NOT mean the publish failed.

Toolset: Products

Parameters:

  • productId (string) required: Product UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product

  • tableOfContentsId (string): Optional table of contents UUID, or identifier in the format 'portal-subdomain:product-slug:section-slug:table-of-contents-slug'. When provided, publishPortalProduct uses it to resolve the published URL path for the returned preview/live link.

  • preview (boolean): Whether to publish as preview (true) or live (false). Preview allows testing before going live. Defaults to false (live publication) (default: false)

ParametersJSON Schema
NameRequiredDescriptionDefault
previewNoWhether to publish as preview (true) or live (false). Preview allows testing before going live. Defaults to false (live publication)
productIdYesProduct UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product
tableOfContentsIdNoOptional table of contents UUID, or identifier in the format 'portal-subdomain:product-slug:section-slug:table-of-contents-slug'. When provided, publishPortalProduct uses it to resolve the published URL path for the returned preview/live link.

Output Schema

ParametersJSON Schema
NameRequiredDescription
liveUrlNo
previewNo
successNo
previewUrlNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the operation as destructive, idempotent, and not read-only. The description adds valuable behavior beyond annotations: return values include publication status, a live/preview URL that can be null if URL building fails, product/portal metadata, and an optional warning whose semantics are explicitly clarified β€” 'a warning does NOT mean the publish failed.' This goes beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured: purpose first, then return behavior, then toolset and parameters. There is some redundancy β€” the first two sentences both communicate the publish effect β€” and the parameter section duplicates the schema, but the length is justified by the added warning semantics and identifier format details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a rich output schema present and only three simple parameters, the description provides a complete summary of return values (status, URLs, metadata, warning), explains the optional tableOfContentsId purpose, and clarifies preview versus live behavior. It doesn't cover permissions or idempotency beyond annotations, but the information an agent needs to call it correctly is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters, including defaults and identifier formats. The description restates the parameters and adds the accepted identifier formats, but these are also present in the schema, so the description adds marginal value over the structured fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('publish') and resource ('a product's content') and clearly states the effect: making content live or available as preview, visible to portal visitors. This clearly distinguishes it from sibling CRUD tools like swagger_create_portal_product, swagger_update_portal_product, and swagger_delete_portal_product.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when to invoke the tool (to publish current content to live or preview) and explicitly advises using preview mode to test before going live. It does not name alternative tools or exclusions, but the guidance is sufficient for an agent to choose this operation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_resolve_organization_portalSwagger: Resolve Organization PortalA
Idempotent
Inspect

Resolve portal details for a Swagger organization in a single step. Given an organization UUID, returns the portal ID, subdomain, customDomain (when configured), and the list of products (with productId, productSlug, and productName) for the organization's portal. If the organization has no portal yet, a new portal is created automatically. Use this tool to obtain all portal context needed for subsequent portal and product operations.

Toolset: Portals

Parameters:

  • organizationId (string) required: Swagger organization UUID - the organization to resolve portal details for

ParametersJSON Schema
NameRequiredDescriptionDefault
organizationIdYesSwagger organization UUID - the organization to resolve portal details for

Output Schema

ParametersJSON Schema
NameRequiredDescription
portalIdNo
productsNo
subdomainNo
customDomainNo
portalCreatedNo
organizationIdNo

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses a non-obvious side effect: 'If the organization has no portal yet, a new portal is created automatically.' This adds meaningful behavior beyond the annotations, which only indicate idempotency and non-read-only semantics, and clarifies the operational impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The key purpose and return fields are front-loaded, and the auto-creation behavior and usage guidance follow logically. The 'Parameters' section repeats schema information, but the overall structure is clear and the description remains focused.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with a single required parameter and an output schema, the description covers the purpose, returned data, side effect, and recommended usage context. It does not mention error cases or permissions, but those are not essential for this straightforward resolution operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already fully documents organizationId with type, format, pattern, and description, so the description adds little beyond restating the parameter. At 100% schema description coverage, a baseline of 3 is appropriate; the description does not enrich the parameter semantics further.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Resolve portal details') on a clear resource ('a Swagger organization') and enumerates the returned fields, so an agent can understand the scope. It does not explicitly contrast with siblings like swagger_get_portal or swagger_list_portals, but 'in a single step' and 'all portal context' convey differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent when to use this tool: 'to obtain all portal context needed for subsequent portal and product operations.' However, it does not state when not to use it or name alternative tools such as swagger_get_portal or swagger_list_portals.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_scan_api_standardizationSwagger: Scan API StandardizationA
Read-onlyIdempotent
Inspect

Run a standardization scan against an API definition using the organization's governance and standardization rules. Accepts a raw YAML or JSON OpenAPI/AsyncAPI definition and returns a list of validation errors, the total issue count, and counts grouped by severity. Use this tool when the user provides the API definition content directly (as raw YAML or JSON) and asks to validate, scan, or check the governance or standardization of the API.

Toolset: Registry API

Parameters:

  • orgName (string) required: The organization name to use for standardization rules

  • definition (string) required: API definition content (OpenAPI/AsyncAPI specification in JSON or YAML format) to scan for standardization errors

ParametersJSON Schema
NameRequiredDescriptionDefault
orgNameYesThe organization name to use for standardization rules
definitionYesAPI definition content (OpenAPI/AsyncAPI specification in JSON or YAML format) to scan for standardization errors

Output Schema

ParametersJSON Schema
NameRequiredDescription
countNo
validationNo
countsBySeverityNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Over and above the readOnly/idempotent/non-destructive annotations, it discloses accepted input formats (raw YAML/JSON) and the return shape (validation errors, total issue count, severity counts). No contradiction; it adds useful operational detail without claiming side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Information is front-loaded: purpose, use-case, toolset, parameters. It is slightly redundant by duplicating the schema's parameter descriptions, but remains compact and scannable for an agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 2-param, idempotent, read-only tool with an output schema, the description covers what the tool does, what it accepts, what it returns, and when to use it. Nothing necessary for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for both params, so baseline 3 applies. The description repeats the schema's parameter meanings but adds little new semantic value beyond emphasizing 'raw' content for definition and orgName as rule selection context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a concrete verb/resource pair: running a standardization scan against an API definition using governance rules. It distinguishes itself from registry-based scanning by explicitly limiting to raw YAML/JSON definitions, and from sibling swagger_scan_api_standardization_from_registry by saying 'when user provides content directly'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description clearly states the intended trigger ('when the user provides the API definition content directly... asks to validate, scan, or check'), which routes agents appropriately. It does not explicitly name alternatives or when-not-to-use cases, but the context is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_scan_api_standardization_from_registrySwagger: Scan API Standardization from RegistryA
Read-onlyIdempotent
Inspect

Run a standardization scan on an API that already exists in SwaggerHub Registry, identified by organization name, API name, and version. Fetches the API definition from the registry internally and scans it against the organization's governance and standardization rules. Returns a list of validation errors, total issue count, counts grouped by severity, and a SwaggerHub UI URL for the scanned API. Use this tool when the user identifies the API by org name, API name, and version and asks to validate, scan, or check the governance or standardization of an existing API.

Toolset: Registry API

Parameters:

  • orgName (string) required: The organization name that owns the API and provides the standardization rules (case-sensitive)

  • apiName (string) required: API name (case-sensitive)

  • version (string) required: Version identifier

ParametersJSON Schema
NameRequiredDescriptionDefault
apiNameYesAPI name (case-sensitive)
orgNameYesThe organization name that owns the API and provides the standardization rules (case-sensitive)
versionYesVersion identifier

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
countNo
validationNo
countsBySeverityNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey read-only, idempotent, non-destructive behavior. The description adds useful process detail: it fetches the API definition from the registry internally, scans it against governance/standardization rules, and returns validation errors, issue counts by severity, and a UI URL. This is meaningful context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core behavior and usage are front-loaded, with useful output and process details following. However, the explicit parameter list duplicates the schema, and 'existing API' appears multiple times, introducing minor redundancy. Overall, it remains well-structured and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter read-only scan tool with an output schema, the description is largely complete: it states inputs, internal behavior, return contents, and when to use it. It could be stronger by naming the distinction from swagger_scan_api_standardization explicitly, but the registry-based identification is already clear from the text and name.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters. The description repeats the parameter list and adds slight context (e.g., org provides the standardization rules, case-sensitivity), but does not add substantial meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Run a standardization scan'), the exact resource (an API existing in SwaggerHub Registry), and the identifying triple of org, API name, and version. It clearly differentiates from the sibling swagger_scan_api_standardization by emphasizing the API already exists in the registry and is fetched internally.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit context: 'Use this tool when the user identifies the API by org name, API name, and version and asks to validate, scan, or check the governance or standardization of an existing API.' It does not name alternatives or state exclusions, so it stops short of a full when-to-use vs. alternatives explanation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_search_apis_and_domainsSwagger: Search APIs and DomainsA
Read-onlyIdempotent
Inspect

Search for APIs and Domains in SwaggerHub Registry using the comprehensive /specs endpoint and retrieve metadata including owner, name, description, summary, version, and specification.

Toolset: Registry API

Parameters:

  • query (string): Search query to filter APIs by name, description, or content

  • state (enum): Filter APIs by publication state - ALL (default), PUBLISHED, or UNPUBLISHED

  • tag (string): Filter APIs by tag

  • offset (number): Offset for pagination (0-based, default 0)

  • limit (number): Number of results per page (1-100, default 20)

  • sort (enum): Sort field - NAME, UPDATED, or CREATED (default NAME)

  • order (enum): Sort order - ASC or DESC (default ASC)

  • owner (string): Filter APIs by owner (organization or user)

  • specType (enum): Filter by specification type - API or DOMAIN (default all types)

ParametersJSON Schema
NameRequiredDescriptionDefault
tagNoFilter APIs by tag
sortNoSort field - NAME, UPDATED, or CREATED (default NAME)
limitNoNumber of results per page (1-100, default 20)
orderNoSort order - ASC or DESC (default ASC)
ownerNoFilter APIs by owner (organization or user)
queryNoSearch query to filter APIs by name, description, or content
stateNoFilter APIs by publication state - ALL (default), PUBLISHED, or UNPUBLISHED
offsetNoOffset for pagination (0-based, default 0)
specTypeNoFilter by specification type - API or DOMAIN (default all types)

Output Schema

ParametersJSON Schema
NameRequiredDescription
itemsYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds context by naming the endpoint and the metadata retrieved, but does not disclose rate limits, authentication needs, pagination quirks, or return-format behavior beyond the parameter list.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is strong and front-loaded, but the large parameter section duplicates the schema descriptions exactly. Since the structured schema already contains all 9 parameter definitions, most of the description body adds no new information and could be significantly shortened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 100% schema coverage, output schema present, and annotations covering read-only/idempotent behavior, the description is complete for selecting and invoking the tool. It adds the useful context of the /specs endpoint and the metadata fields returned, leaving no critical selection or calling information missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents every parameter thoroughly. The description repeats those parameter descriptions verbatim rather than adding new meaning about formats, interdependencies, or edge cases, so it stays at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific action verb ('Search'), names the exact resource ('APIs and Domains in SwaggerHub Registry'), and cites the underlying '/specs endpoint'. It also enumerates returned metadata such as owner, name, description, summary, version, and specification, making the tool's purpose unmistakable and distinct from sibling get/list tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: this is the registry search tool for APIs and domains, backed by the /specs endpoint, with many filters. It does not explicitly state when not to use it or name alternatives, but its scope is clear enough that an agent can select it confidently.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_standardize_apiSwagger: Standardize APIA
DestructiveIdempotent
Inspect

Standardize and fix an API definition using AI to ensure compliance with governance policies. Scans the API definition for standardization errors and automatically fixes them using SmartBear AI. Optionally provide 'newVersion' (e.g. patch bump '1.0.0' β†’ '1.0.1') to save the fixed definition as a new version β€” omitting it will overwrite the current version. Returns the number of errors found, the fixed definition, and a URL to view the standardized API. Use this tool when users ask to standardize, fix, govern, or ensure governance compliance of APIs.

Toolset: Registry API

Parameters:

  • owner (string) required: API owner (organization or user, case-sensitive)

  • api (string) required: API name (case-sensitive)

  • version (string) required: Version identifier

  • newVersion (string): The version to save the fixed definition as (e.g. '1.0.1'). Omitting this will overwrite the current version β€” prefer providing a patch bump (e.g. '1.0.0' β†’ '1.0.1') unless the user specifies otherwise.

ParametersJSON Schema
NameRequiredDescriptionDefault
apiYesAPI name (case-sensitive)
ownerYesAPI owner (organization or user, case-sensitive)
versionYesVersion identifier
newVersionNoThe version to save the fixed definition as (e.g. '1.0.1'). Omitting this will overwrite the current version β€” prefer providing a patch bump (e.g. '1.0.0' β†’ '1.0.1') unless the user specifies otherwise.

Output Schema

ParametersJSON Schema
NameRequiredDescription
urlNo
messageNo
errorsFoundNo
savedVersionNo
fixedDefinitionNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses key behavioral consequences beyond annotations: omitting newVersion overwrites the current version, while providing it saves as a new version, and it states the return payload includes error count, fixed definition, and a URL. This aligns with destructiveHint=true and adds useful detail without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with the core behavior first, followed by newVersion behavior, return values, and usage cue. The parameter list duplicates the schema somewhat, but the layout remains scannable and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description gives enough context to invoke correctly: what the tool does, when to use it, how to avoid overwriting by supplying newVersion, and what the response contains. With an output schema present, no critical operational details are missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, so the baseline is 3. The description repeats the parameter explanations but adds little beyond what the schema already provides; no new parameter semantics are introduced.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete action: 'Standardize and fix an API definition using AI to ensure compliance with governance policies' and 'automatically fixes them using SmartBear AI.' It clearly names the resource and the fix/overwrite behavior, and the 'fix' verb distinguishes it from scan-only siblings in the same toolset.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Use this tool when users ask to standardize, fix, govern, or ensure governance compliance of APIs.' It does not name specific alternatives or exclusions, but the when-to-use instruction is clear and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_update_documentSwagger: Update DocumentA
DestructiveIdempotent
Inspect

Update the content or source of an existing document. Supports both HTML and Markdown content types.

Toolset: Documents

Parameters:

  • documentId (string) required: Document UUID - unique identifier for the document

  • content (string): The document content to update (HTML or Markdown based on document type)

  • type (enum): Content type of the document. Note: documents with type 'html' and source 'internal' cannot be edited via API β€” only 'html' + 'external' and all 'markdown' combinations are supported.

  • source (enum): Where the document content is managed. 'internal': editable in both portal UI and API. 'external': editable via API only. Note: 'html' + 'internal' documents cannot be updated via API.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoContent type of the document. Note: documents with type 'html' and source 'internal' cannot be edited via API β€” only 'html' + 'external' and all 'markdown' combinations are supported.
sourceNoWhere the document content is managed. 'internal': editable in both portal UI and API. 'external': editable via API only. Note: 'html' + 'internal' documents cannot be updated via API.
contentNoThe document content to update (HTML or Markdown based on document type)
documentIdYesDocument UUID - unique identifier for the document

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
typeNo
sourceNo
contentNo
createdAtNo
updatedAtNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds a non-obvious behavioral constraint: the combination of type='html' and source='internal' is not allowed, while external source documents are editable via API only. The annotations already convey mutation and destructiveness (readOnlyHint=false, destructiveHint=true, idempotentHint=true), so the description's additional restrictions justify a solid but not maximal score.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence is crisp and the toolset label adds orienting context, but the parameter block replicates the schema's descriptions, adding redundancy rather than new value. It is still structured and readable, so it earns a middle score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and all four parameters documented at 100% coverage, the description covers the key constraints: supported type/source combinations and internal vs external editing. The only minor omission is guidance on how to obtain a documentId, but sibling get_document tools make that inferable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description's parameter section repeats the schema text almost verbatim rather than adding new meaning. It accurately restates the content/type/source semantics and enum options, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific operation ('Update'), a specific resource ('existing document'), and the exact scope ('content or source'), and lists the supported HTML/Markdown types. This clearly marks it as different from create/get siblings like swagger_create_documentation_page and swagger_get_document, even without naming them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The type/source parameter notes explicitly state that documents with type 'html' and source 'internal' cannot be edited via API, and that only 'html' + 'external' and all 'markdown' combinations are supported. The source parameter also names the alternative ('portal UI') for internal editing, so an agent knows when to use this tool and when not to.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_update_portalSwagger: Update PortalB
DestructiveIdempotent
Inspect

Update a specific portal's configuration.

Toolset: Portals

Parameters:

  • portalId (string) required: Portal UUID or subdomain - unique identifier for the portal instance

  • name (string): Update the portal display name - shown to users and in branding (3-40 characters)

  • subdomain (string): Update the portal subdomain - changes the portal URL. Must remain unique across all portals (3-20 characters, lowercase, alphanumeric with hyphens)

  • customDomain (boolean): Enable/disable custom domain for the portal - allows using your own domain instead of the default subdomain

  • gtmKey (string): Google Tag Manager key for analytics tracking - format: GTM-XXXXXX (max 25 characters)

  • offline (boolean): Set portal visibility - true hides portal from customers (useful for maintenance or development)

  • routing (string): Update routing strategy - 'browser' for client-side routing or 'proxy' for server-side routing

  • credentialsEnabled (boolean): Enable/disable authentication credentials for portal access - controls whether users can authenticate to view private content

  • openapiRenderer (string): Change OpenAPI renderer: 'SWAGGER_UI' (Swagger UI), 'ELEMENTS' (Stoplight Elements), or 'TOGGLE' (switch between both)

  • pageContentFormat (string): Update page content format for documentation rendering: 'HTML', 'MARKDOWN', or 'BOTH'

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoUpdate the portal display name - shown to users and in branding (3-40 characters)
gtmKeyNoGoogle Tag Manager key for analytics tracking - format: GTM-XXXXXX (max 25 characters)
offlineNoSet portal visibility - true hides portal from customers (useful for maintenance or development)
routingNoUpdate routing strategy - 'browser' for client-side routing or 'proxy' for server-side routing
portalIdYesPortal UUID or subdomain - unique identifier for the portal instance
subdomainNoUpdate the portal subdomain - changes the portal URL. Must remain unique across all portals (3-20 characters, lowercase, alphanumeric with hyphens)
customDomainNoEnable/disable custom domain for the portal - allows using your own domain instead of the default subdomain
openapiRendererNoChange OpenAPI renderer: 'SWAGGER_UI' (Swagger UI), 'ELEMENTS' (Stoplight Elements), or 'TOGGLE' (switch between both)
pageContentFormatNoUpdate page content format for documentation rendering: 'HTML', 'MARKDOWN', or 'BOTH'
credentialsEnabledNoEnable/disable authentication credentials for portal access - controls whether users can authenticate to view private content

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
nameNo
offlineNo
routingNo
subdomainNo
customDomainNo
openapiRendererNo
credentialsEnabledNo
swaggerHubOrganizationIdNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal destructiveHint=true and idempotentHint=true, so the mutation profile is partially covered. The description's parameter notes add useful side-effect context, such as subdomain 'changes the portal URL' and offline 'hides portal from customers', though it does not discuss permissions, reversibility, or broader destructive impact. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose statement is front-loaded and clear, but the description then duplicates the entire 10-parameter input schema nearly verbatim. Most of the parameter entries do not earn their place because the agent already has that information in the structured schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

All parameters are documented, the required portalId is obvious, and an output schema exists, so an agent can construct a valid call. However, the description lacks guidance on prerequisites, when updating is appropriate, and what post-update side effects to verify, so it is adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description's parameter list essentially mirrors the schema without adding new semantic meaning, so it stays at the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: 'Update a specific portal's configuration.' This distinguishes it from create/get/delete portal tools, though it does not explicitly name sibling tools to differentiate from update_portal_product.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The verb 'Update' and resource 'portal' imply this is for modifying an existing portal, but there is no explicit when/when-not guidance or mention of alternatives like swagger_create_portal or swagger_get_portal. The 'Toolset: Portals' line provides only weak context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

swagger_update_portal_productSwagger: Update Portal ProductA
DestructiveIdempotent
Inspect

Update a product's settings within a specific portal.

Toolset: Products

Parameters:

  • productId (string) required: Product UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product

  • name (string): Update product display name - changes how it appears to users in navigation and listings (3-40 characters)

  • slug (string): Update URL-friendly identifier - must remain unique within the portal, affects product URLs (3-22 characters, lowercase, alphanumeric with hyphens/underscores/dots)

  • description (string): Update product description - explains the API/product functionality, shown in listings (max 110 characters)

  • public (boolean): Change product visibility - true makes it publicly accessible to all visitors, false restricts to authenticated users with roles

  • hidden (boolean): Change navigation visibility - true hides from portal landing page menus while keeping the product accessible via direct links

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoUpdate product display name - changes how it appears to users in navigation and listings (3-40 characters)
slugNoUpdate URL-friendly identifier - must remain unique within the portal, affects product URLs (3-22 characters, lowercase, alphanumeric with hyphens/underscores/dots)
hiddenNoChange navigation visibility - true hides from portal landing page menus while keeping the product accessible via direct links
publicNoChange product visibility - true makes it publicly accessible to all visitors, false restricts to authenticated users with roles
productIdYesProduct UUID or identifier in the format 'portal-subdomain:product-slug' - unique identifier for the product
descriptionNoUpdate product description - explains the API/product functionality, shown in listings (max 110 characters)

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNo
urlNo
nameNo
slugNo
hiddenNo
publicNo
portalIdNo
descriptionNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Although annotations already signal idempotent and destructive behavior, the description adds meaningful side-effect context: changing the slug affects product URLs, public/hidden control visibility versus navigation display, and name changes affect user-facing listings. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded, and the 'Toolset' heading groups context. However, the parameter section duplicates schema descriptions nearly verbatim, adding unnecessary length beyond what the structured schema already provides.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an update tool with one required parameter and six well-documented fields, the description plus schema covers the necessary inputs and key effects. The output schema is present so return-value details need not be in the description; only permissions/preconditions are not addressed, which is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description repeats the schema's parameter descriptions rather than adding new meaning. The bullet list is readable, but it does not compensate for anything missing in the schema, so it earns the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Update a product's settings within a specific portal.' The scope ('within a specific portal') and the title distinguish it from related siblings such as swagger_update_portal, swagger_create_portal_product, and swagger_delete_portal_product.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly signals its use case: updating an existing product's configuration in a portal. It does not explicitly name alternatives or exclusion conditions, so it falls short of full guidance, but the context is clear enough for an agent to select it over create/delete/get siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_create_folderZephyr: Create FolderBInspect

Create a folder called 'Axial Pump Tests' in the project SA for organizing test cases

Toolset: Folders

Parameters:

  • parentId (number): Folder ID of the parent folder. Must be null for root folders.

  • name (string) required: Folder name. Folder name must not contain / and \ characters.

  • projectKey (string) required: Jira project key.

  • folderType (string) required: Valid values: "TEST_CASE", "TEST_PLAN", "TEST_CYCLE"

Examples:

  1. Create a root Folder in project SA for organizing test cases

{
  "parentId": null,
  "name": "Axial Pump Tests",
  "projectKey": "SA",
  "folderType": "TEST_CASE"
}

Expected Output: The newly created Folder with its ID and self link

  1. Create a sub-folder under folder ID 5 in the project MM2 for test plans related to pumps

{
  "parentId": 5,
  "name": "Pump-related Test Plans",
  "projectKey": "MM2",
  "folderType": "TEST_PLAN"
}

Expected Output: The newly created Folder with its ID and self link

  1. Create a Folder called 'Regression Cycles' in project TIS for organizing test cycles

{
  "parentId": null,
  "name": "Regression Cycles",
  "projectKey": "TIS",
  "folderType": "TEST_CYCLE"
}

Expected Output: The newly created Folder with its ID and self link

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesFolder name. Folder name must not contain `/` and `\` characters.
parentIdNoFolder ID of the parent folder. Must be `null` for root folders.
folderTypeYesValid values: `"TEST_CASE"`, `"TEST_PLAN"`, `"TEST_CYCLE"`
projectKeyYesJira project key.

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNoThe ID of the entity
selfNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey that this is not read-only, not destructive, and not idempotent, so the description does not need to restate those. It adds useful constraints like parentId being null for root folders, naming restrictions, and folderType values. It does not mention behavior on duplicate names, required permissions, or side effects beyond creating the folder.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is organized clearly with a Toolset label, parameter list, and examples. However, the opening example sentence is redundant with the first example, and 'Expected Output' is repeated identically three times. The content is useful but could be tightened by removing duplicated or overly specific phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With four parameters, full schema coverage, and an output schema indicated, the description is sufficiently complete for an agent to construct valid calls. It covers required fields, the optional parentId, null handling for root folders, naming constraints, and provides three realistic examples spanning all folderType values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter. The description repeats those descriptions and adds concrete JSON examples, but it does not add material semantic information beyond the schema. The examples do help illustrate valid combinations, but this is baseline-level value given full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear action ('Create a folder') and resource, with folderType values that clarify what kind of folder. However, the opening sentence is oddly specific ('Axial Pump Tests' in project SA) rather than a general statement, and it does not explicitly differentiate from sibling tools like zephyr_create_test_cycle or zephyr_create_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The examples imply when the tool is useful: creating root folders or subfolders for organizing test cases, test plans, and test cycles. But there is no explicit guidance on when to use this tool instead of other Zephyr creation tools, nor any exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_create_test_caseZephyr: Create Test CaseAInspect

Create a new Test Case in Zephyr specified project

Toolset: Test Cases

Parameters:

  • projectKey (string) required: Jira project key.

  • name (string) required

  • objective (string): A description of the objective.

  • precondition (string): Any conditions that need to be met.

  • estimatedTime (number): Estimated duration in milliseconds.

  • componentId (number): ID of a component from Jira.

  • priorityName (string): The priority name.

  • statusName (string): The status name.

  • folderId (number): ID of a folder to place the entity within.

  • ownerId (string): Atlassian Account ID of the Jira user.

  • labels (array): Array of labels associated to this entity.

  • customFields (record<string, any>): Multi-line text fields support HTML and should denote new lines with the <br> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.

Examples:

  1. Create a Test Case in project SA to ensure that the axial pump can be enabled

{
  "projectKey": "SA",
  "name": "Check axial pump",
  "objective": "Ensure the axial pump can be enabled"
}

Expected Output: The newly created Test Case with its details and key

  1. Create a Test Case to ensure that the axial pump can be enabled. The test should be in project MM2, have labels 'automated' and 'mcp', and priority 'High'

{
  "projectKey": "MM2",
  "name": "Check axial pump",
  "objective": "Ensure the axial pump can be enabled",
  "labels": [
    "automated",
    "mcp"
  ],
  "priorityName": "High"
}

Expected Output: The newly created Test Case with its details and key

  1. Create a Test Case for verifying strength of the axial pump with custom field 'Axial pump strength' having value '5' in project SA

{
  "projectKey": "SA",
  "name": "Check axial pump strength",
  "objective": "Make sure the axial pump operates at the required strength",
  "customFields": {
    "Axial pump strength": 5
  }
}

Expected Output: The newly created Test Case with its details and key

  1. Create a Test Case in project MM2 to verify the performance of the axial pump with Jira component having ID 10001, Jira owner having ID 10057 in folder 'Pumps'

{
  "projectKey": "MM2",
  "name": "Check axial pump performance",
  "objective": "Ensure the axial pump performs within acceptable limits",
  "componentId": 10001,
  "ownerJiraUserId": 10057,
  "folderId": 18
}

Expected Output: The newly created Test Case with its details and key

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
labelsNoArray of labels associated to this entity.
ownerIdNoAtlassian Account ID of the Jira user.
folderIdNoID of a folder to place the entity within.
objectiveNoA description of the objective.
projectKeyYesJira project key.
statusNameNoThe status name.
componentIdNoID of a component from Jira.
customFieldsNoMulti-line text fields support HTML and should denote new lines with the \<br\> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.
preconditionNoAny conditions that need to be met.
priorityNameNoThe priority name.
estimatedTimeNoEstimated duration in milliseconds.

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNoThe ID of the entity
keyNo
selfNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already communicate that this is a mutating, non-idempotent action. The description adds some behavioral context, such as the expected output (newly created Test Case with details and key) and customFields formatting rules, but does not go further into auth needs, side effects, or error conditions. This is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence is front-loaded and the examples are genuinely useful. However, the description repeats the full parameter list that the schema already documents, making it longer than necessary. The structure is organized, but the redundancy prevents it from being concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 12 parameters and nested customFields, the examples cover several important usage patterns and the output schema covers return values. Still, the invalid ownerJiraUserId example and the estimatedTime type mismatch leave enough ambiguity that an agent could build a failing request. The description is mostly complete but not fully reliable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is high at 92%, so the parameter list in the description adds little beyond the schema; the examples do add useful mapping from natural-language requests to parameters. However, Example 4 uses 'ownerJiraUserId', which is not in the schema (and the schema sets additionalProperties to false), and gives a numeric value where ownerId expects a string. Also, estimatedTime is described as a number while the schema declares integer. These inconsistencies actively risk misleading an agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a clear verb and resource: 'Create a new Test Case in Zephyr specified project.' This distinguishes it from the many sibling create/update test case tools by naming both the toolset and the product, and from zephyr_create_test_case_steps which is for adding steps rather than creating the test case itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly states what this tool is for: creating a test case in a Zephyr project, with projectKey and name as the required inputs. It does not explicitly say when not to use it or name alternatives, but the purpose is explicit enough that an agent can infer it should be chosen over update/delete/step-creation siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_create_test_case_stepsZephyr: Create Test Case StepsA
Destructive
Inspect

Create steps for a Test Case in Zephyr. Supports inline step definitions or delegating execution to another test case (also known as 'call to test' via UI). Requires a mode: APPEND adds steps to the end of the existing list, OVERWRITE deletes all existing steps and replaces them with the provided ones. Always ask the user to choose between OVERWRITE or APPEND before calling this tool.

Toolset: Test Cases

Parameters:

  • testCaseKey (string) required: The key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

  • mode (string) required: Valid values: "APPEND", "OVERWRITE". OVERWRITE deletes and recreates the test steps and associated custom field values using the provided input. Attachments for existing steps are kept, but those for missing steps are deleted permanently APPEND only adds extra steps to your test steps.

  • items (array) required: The list of test steps. Each step should be an object containing inline or testCase. They should only include one of these fields at a time.

Examples:

  1. To the Test Case SA-T1, add steps that will test a login page.

{
  "testCaseKey": "SA-T1",
  "mode": "APPEND",
  "items": [
    {
      "inline": {
        "description": "Navigate to the login page",
        "expectedResult": "Login page is displayed"
      }
    },
    {
      "inline": {
        "description": "Enter valid credentials and click Submit",
        "expectedResult": "User is redirected to the dashboard"
      }
    }
  ]
}

Expected Output: The ID of the Test Steps resource and the API self URL to fetch it

  1. To the Test Case MM2-T15, replace all existing steps with new ones that test the settings page for an Admin user.

{
  "testCaseKey": "MM2-T15",
  "mode": "OVERWRITE",
  "items": [
    {
      "inline": {
        "description": "Open the settings page",
        "testData": "User role: Admin",
        "expectedResult": "Settings page is accessible"
      }
    },
    {
      "inline": {
        "description": "Change the notification preference",
        "testData": "Preference: Email only",
        "expectedResult": "Notification preference is updated successfully"
      }
    }
  ]
}

Expected Output: The ID of the Test Steps resource and the API self URL to fetch it

  1. To the Test Case SA-T1, add a step that reuses the steps from the Test Case PRJ-T42

{
  "testCaseKey": "SA-T1",
  "mode": "APPEND",
  "items": [
    {
      "testCase": {
        "testCaseKey": "PRJ-T42"
      }
    }
  ]
}

Expected Output: The ID of the Test Steps resource and the API self URL to fetch it

ParametersJSON Schema
NameRequiredDescriptionDefault
modeYesValid values: `"APPEND"`, `"OVERWRITE"`. <br> `OVERWRITE` deletes and recreates the test steps and associated custom field values using the provided input. Attachments for existing steps are kept, but those for missing steps are deleted permanently <br> `APPEND` only adds extra steps to your test steps.
itemsYesThe list of test steps. Each step should be an object containing `inline` or `testCase`. **They should only include one of these fields at a time**.
testCaseKeyYesThe key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNoThe ID of the entity
selfNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as destructive, and the description adds critical specifics: OVERWRITE deletes existing steps and associated custom field values, and attachments for missing steps are deleted permanently. It also warns that APPEND only adds steps. This goes well beyond the annotation and gives the agent the risk context needed for safe invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than average but well structured with bold parameter headers, clear mode definitions, and three practical examples for the non-trivial inline/testCase item structure. The critical destructive mode warning and user-confirmation instruction are front-loaded. It is appropriately sized for the complexity of the items parameter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers required parameters, both modes, destructive consequences, the two item variants, the user-confirmation requirement, and the expected output (step resource ID and API self URL). Combined with the rich input schema, no critical information appears to be missing for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description largely repeats the schema's parameter documentation: key format, mode semantics, and the inline/testCase mutual exclusivity. The examples are helpful but do not add new parameter-level meaning beyond what the schema already provides, so no higher score is warranted.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Create steps for a Test Case in Zephyr.' It also clarifies the two supported step forms (inline definitions and delegation to another test case), which distinguishes this from read-only or execution-step tools. The naming and description together leave no ambiguity about what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly requires the user to choose between OVERWRITE and APPEND and instructs the agent to always ask before calling. It explains exactly what each mode does, which serves as a clear selection guideline. It does not name sibling alternatives for other step operations, so it stops short of full tool-level routing, but the mode guidance is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_create_test_cycleZephyr: Create Test CycleCInspect

Create a new Test Cycle in Zephyr specified project

Toolset: Test Cycles

Parameters:

  • projectKey (string) required: Jira project key.

  • name (string) required

  • description (string): Description outlining the scope.

  • plannedStartDate (string): Planned start date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

  • plannedEndDate (string): The planned end date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

  • jiraProjectVersion (number): Jira Project Version ID. Relates to 'Version' or 'Releases' in Jira projects.

  • statusName (string): The status name.

  • folderId (number): ID of a folder to place the entity within.

  • ownerId (string): Atlassian Account ID of the Jira user.

  • customFields (record<string, any>): Multi-line text fields support HTML and should denote new lines with the <br> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.

Examples:

  1. Create a Test Cycle in project SA to ensure that the axial pump can be enabled

{
  "projectKey": "SA",
  "name": "Check axial pump",
  "description": "Ensure the axial pump can be enabled"
}

Expected Output: The newly created Test Cycle with its details and key

  1. Create a Test Cycle to ensure that the axial pump can be enabled. The test cycle should be in project MM2, have status 'In Progress', and be planned from 2026-03-01 to 2026-03-10

{
  "projectKey": "MM2",
  "name": "Check axial pump",
  "description": "Ensure the axial pump can be enabled",
  "statusName": "In Progress",
  "plannedStartDate": "2026-03-01T00:00:00Z",
  "plannedEndDate": "2026-03-10T00:00:00Z"
}

Expected Output: The newly created Test Cycle with its details and key

  1. Create a Test Cycle for verifying strength of the axial pump with custom field 'Axial pump strength' having value '5' in project SA

{
  "projectKey": "SA",
  "name": "Check axial pump strength",
  "description": "Make sure the axial pump operates at the required strength",
  "customFields": {
    "Axial pump strength": 5
  }
}

Expected Output: The newly created Test Cycle with its details and key

  1. Create a Test Cycle in project MM2 to verify the performance of the axial pump with Jira Project Version ID 10001, owner Atlassian Account ID '12', in folder with ID 18

{
  "projectKey": "MM2",
  "name": "Check axial pump performance",
  "description": "Ensure the axial pump performs within acceptable limits",
  "jiraProjectVersion": 10001,
  "ownerId": "12",
  "folderId": 18
}

Expected Output: The newly created Test Cycle with its details and key

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
ownerIdNoAtlassian Account ID of the Jira user.
folderIdNoID of a folder to place the entity within.
projectKeyYesJira project key.
statusNameNoThe status name.
descriptionNoDescription outlining the scope.
customFieldsNoMulti-line text fields support HTML and should denote new lines with the \<br\> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.
plannedEndDateNoThe planned end date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'
plannedStartDateNoPlanned start date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'
jiraProjectVersionNoJira Project Version ID. Relates to 'Version' or 'Releases' in Jira projects.

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNoThe ID of the entity
keyNo
selfNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal that this is not a read-only operation (readOnlyHint=false, idempotentHint=false), and the description adds no behavioral context beyond 'create'. It does not mention side effects, whether duplicate cycles are possible, or whether referenced statuses, folders, or versions must pre-exist. This does not contradict the annotations, but it also does not add meaningful transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The structure is clear with headings for Toolset, Parameters, and Examples, and the first sentence is front-loaded. However, the parameter section largely repeats the schema descriptions, making the description noticeably longer than necessary. The four examples are useful, but the overall text would benefit from trimming duplicate parameter explanations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 10 parameters, nested objects, and an output schema present, the description is reasonably complete for making a call: all parameters are listed and examples show realistic payloads. It does not, however, explain how to obtain valid values for IDs like folderId, jiraProjectVersion, statusName, or ownerId, nor does it point to lookup tools such as zephyr_get_statuses or zephyr_get_folders.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is high (90%), so the description need not compensate for missing schema meaning. The parameter list mostly duplicates the input schema, while the examples add some practical illustration of valid values and groupings. There is no substantial semantic information beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb ('Create'), a clear resource ('Test Cycle'), and a scope ('in Zephyr specified project'). The 'Toolset: Test Cycles' heading reinforces the resource. It does not explicitly differentiate itself from sibling tools like zephyr_update_test_cycle or qtm4j_create_test_cycle, so it misses the top score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to choose this tool over alternatives such as zephyr_update_test_cycle, zephyr_create_test_execution, or qtm4j_create_test_cycle. The examples illustrate valid calls, but they do not explain when this tool should be preferred or when a sibling tool would be more appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_create_test_executionZephyr: Create Test ExecutionAInspect

Create a new Test Execution for a Test Case within a specific Test Cycle

Toolset: Test Executions

Parameters:

  • projectKey (string) required: Jira project key.

  • testCaseKey (string) required: Key of test case the execution applies to. NOTE: Test cases with call to test, parameters and test data are not supported.

  • testCycleKey (string) required: Key of test cycle the execution applies to.

  • statusName (string) required: The status name.

  • testScriptResults (array)

  • environmentName (string): Environment assigned to the test case.

  • actualEndDate (string): The actual end date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

  • executionTime (number): Actual test execution time in milliseconds.

  • executedById (string): Atlassian Account ID of the Jira user.

  • assignedToId (string): Atlassian Account ID of the Jira user.

  • comment (string): Comment added against overall test case execution.

  • customFields (record<string, any>): Multi-line text fields support HTML and should denote new lines with the <br> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.

Examples:

  1. Create a Passed execution for test case SA-T1 in cycle SA-R1

{
  "projectKey": "SA",
  "testCaseKey": "SA-T1",
  "testCycleKey": "SA-R1",
  "statusName": "Pass"
}

Expected Output: The newly created Test Execution with execution details

  1. Create a Failed execution with execution time, environment and comment

{
  "projectKey": "MM2",
  "testCaseKey": "MM2-T15",
  "testCycleKey": "MM2-R3",
  "statusName": "Fail",
  "environmentName": "Staging",
  "executionTime": 125000,
  "comment": "Step 3 failed due to timeout<br>Logs attached."
}

Expected Output: The newly created Test Execution including environment and timing information

  1. Create execution with custom fields and assignment

{
  "projectKey": "SA",
  "testCaseKey": "SA-T5",
  "testCycleKey": "SA-R2",
  "statusName": "Pass",
  "executedById": "5b10ac8d82e05b22cc7d4ef5",
  "assignedToId": "5b10ac8d82e05b22cc7d4ef6",
  "actualEndDate": "2026-02-17T10:15:30Z",
  "customFields": {
    "Execution Build": "1.0.3",
    "Tested Browser": "Chrome",
    "Execution Date": "2026-02-17"
  }
}

Expected Output: The newly created Test Execution including custom field values

ParametersJSON Schema
NameRequiredDescriptionDefault
commentNoComment added against overall test case execution.
projectKeyYesJira project key.
statusNameYesThe status name.
testCaseKeyYesKey of test case the execution applies to. NOTE: Test cases with call to test, parameters and test data are not supported.
assignedToIdNoAtlassian Account ID of the Jira user.
customFieldsNoMulti-line text fields support HTML and should denote new lines with the \<br\> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.
executedByIdNoAtlassian Account ID of the Jira user.
testCycleKeyYesKey of test cycle the execution applies to.
actualEndDateNoThe actual end date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'
executionTimeNoActual test execution time in milliseconds.
environmentNameNoEnvironment assigned to the test case.
testScriptResultsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNoThe ID of the entity
selfNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal readOnlyHint=false and idempotentHint=false, so the mutating behavior is expected. The description adds a useful unsupported-test-case caveat and shows expected outputs in examples, but does not go deeper into side effects, duplicate executions, or prerequisites.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with headings, notes, and examples, and it front-loads the purpose. However, it redundantly lists all parameters with descriptions that already exist in the input schema, making it longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 12-parameter surface area, the examples cover required fields as well as optional ones like executionTime, environmentName, executedById, customFields, and actualEndDate. An output schema is present, so the description does not need to detail return values; no major invocation-critical gap is evident.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 92%, so the baseline is 3. The parameter list largely repeats the schema descriptions; the main additions are concrete JSON examples and a repeated note about HTML/custom-field formatting, which are helpful but not a major semantic boost over the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and noun ('Create a new Test Execution') and clearly scopes the request to 'a Test Case within a specific Test Cycle.' It differentiates from sibling get/update/delete operations by naming the creation action and resource type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is explicit: create a test execution. It also provides a clear exclusion by warning that test cases with call to test, parameters, and test data are not supported. It does not explicitly name alternative tools for those cases, so it stops short of a perfect 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_create_test_scriptZephyr: Create Test ScriptAInspect

Create a new Test Script of the types Plain Text or BDD in a Zephyr Test Case.

Toolset: Test Cases

Parameters:

  • testCaseKey (string) required: The key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

  • type (enum): Test scripts can be written in plain text or BDD format. The BDD type supports remote execution on a build system via API plugin.

Supported Keywords for BDD: Given, When, Then, And, But.

For more information about BDD and Gherkin syntax, see: https://support.smartbear.com/zephyr/docs/en/test-cases/gherkin-behavior-driven-development--bdd-.html

For Plain Text scripts, we support HTML fragments. To create a step-by-step test script, you should use the POST /testcases/{testCaseKey}/teststeps endpoint.

  • text (string)

Examples:

  1. Create a plain text test script for test case SA-T1 to verify that the axial pump can be enabled

{
  "testCaseKey": "SA-T1",
  "type": "plain",
  "text": "1. Navigate to Pump Settings</br>2. Enable Axial Pump</br>3. Verify pump status is 'Active'"
}

Expected Output: The created test script metadata including its id and self link

  1. Create a BDD test script for test case MM2-T15 to validate axial pump activation

{
  "testCaseKey": "MM2-T15",
  "type": "bdd",
  "text": "Given the axial pump is installed\nWhen the user enables the axial pump\nThen the pump status should be Active"
}

Expected Output: The created test script metadata including its id and self link

  1. Create a BDD test script for test case QA-T100 for axial pump performance validation

{
  "testCaseKey": "QA-T100",
  "type": "bdd",
  "text": "Given the system is running\nWhen the axial pump operates under load\nThen performance metrics should remain within thresholds"
}

Expected Output: The created test script metadata including its id and self link

ParametersJSON Schema
NameRequiredDescriptionDefault
textNo
typeNoTest scripts can be written in plain text or BDD format. The BDD type supports remote execution on a build system via API plugin. Supported Keywords for BDD: Given, When, Then, And, But. For more information about BDD and Gherkin syntax, see: https://support.smartbear.com/zephyr/docs/en/test-cases/gherkin-behavior-driven-development--bdd-.html For Plain Text scripts, we support HTML fragments. To create a step-by-step test script, you should use the POST /testcases/{testCaseKey}/teststeps endpoint.
testCaseKeyYesThe key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNoThe ID of the entity
selfNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal that this is not a read-only, idempotent, or destructive operation. The description adds useful behavioral context by explaining that BDD type supports remote execution via an API plugin, that Plain Text scripts support HTML fragments, and that the response includes metadata with id and self link. This goes beyond just repeating the annotation flags.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with parameters, type guidance, examples, and expected output. It is longer than strictly necessary, and the three examples are somewhat redundant, but each section earns its place by clarifying how to construct valid calls. The front-loaded purpose sentence helps an agent quickly identify the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description does not need to detail return values, but it still mentions the response metadata in the examples. It covers the main parameter semantics, type nuances, and alternatives for step creation. A minor gap is that text and type are not marked as required in the parameter list even though the examples always include them, which leaves slight ambiguity about whether they can be omitted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67%, so the schema already documents testCaseKey and type thoroughly. The description adds value through concrete examples illustrating the text format for both plain and BDD scripts, plus the HTML fragment note for plain text. The text parameter itself remains somewhat underexplained, but the examples compensate reasonably well.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: create a new Test Script in a Zephyr Test Case, and explicitly names the two supported types (Plain Text and BDD). It also distinguishes itself from a related operation by pointing to the POST /testcases/{testCaseKey}/teststeps endpoint for step-by-step test steps, which differentiates it from sibling tools like zephyr_create_test_case_steps.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when to use this tool and, importantly, tells the agent to use the teststeps endpoint for step-by-step scripts instead. It does not explicitly name the sibling tool zephyr_create_test_case_steps, but the endpoint reference serves the same purpose. It could be more explicit about when to choose this over other Zephyr tools, but the guidance is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_environmentsZephyr: Get EnvironmentsA
Read-onlyIdempotent
Inspect

Get environments in Zephyr

Toolset: Environment

Parameters:

  • projectKey (string): Jira project key filter

  • maxResults (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned. (default: 10)

  • startAt (number): Zero-indexed starting position. Should be a multiple of maxResults. (default: 0)

Examples:

  1. Get the first 20 Environments

{
  "maxResults": 20,
  "startAt": 0
}

Expected Output: The first 20 Environments with their details from different projects

  1. Get the first 10 Environments from the project with projectKey TEST

{
  "projectKey": "TEST",
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 Environments with their details from project with projectKey TEST

  1. Get second 10 Environments from the project with projectKey TEST

{
  "projectKey": "TEST",
  "maxResults": 10,
  "startAt": 10
}

Expected Output: The second 10 Environments with their details from project with projectKey TEST

  1. Get Environments starting from the 5th Environment from different projects

{
  "startAt": 5,
  "maxResults": 10
}

Expected Output: Environments starting from the 5th one with their details from different projects

  1. Get 5 Environments starting from the 10th Environment from the project with projectKey PROJ

{
  "startAt": 10,
  "maxResults": 5,
  "projectKey": "PROJ"
}

Expected Output: The five environments starting from the 10th Environment from the project PROJ with their details

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position. Should be a multiple of maxResults.
maxResultsNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned.
projectKeyNoJira project key filter

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNoURL to the next page of results, or null if there are no more results.
totalNoIndicates the total number of items available across all pages.
isLastNoIndicates if this is the last page of results.
valuesNo
startAtYesIndicates the index of the first item returned in the page of results.
maxResultsYesIndicates the maximum number of results in this response. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds useful behavior beyond that: the server may cap maxResults lower than requested, results may be truncated, and callers should check the response's maxResults value. This is relevant operational context for a paginated read-only tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded and the markdown structure is readable, but the Parameters section largely duplicates the input schema verbatim. The five examples are instructive, but several are repetitive, and the internal contradiction in Example 4 makes the extra length less valuable than it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list tool with an output schema, the description covers the key agent-facing needs: optional project filtering, pagination semantics, defaults, and truncation behavior. It is complete enough to call correctly, though the pagination inconsistency and lack of alternative-tool routing keep it from being fully polished.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the description does not need to add much. It restates the schema and provides examples, but it introduces an inconsistency: the description says startAt should be a multiple of maxResults, yet Example 4 uses startAt=5 with maxResults=10. It also states a maxResults request ceiling of 1000 while the schema maximum is far larger. These contradictions reduce the value of the added param guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Get environments in Zephyr', clearly specifying a verb, a resource, and a product domain. The 'Toolset: Environment' label and the tool name itself distinguish it from the many other zephyr_get_* sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The examples make the intended usage fairly clear: list environments, optionally filter by projectKey, and page with startAt/maxResults. However, there is no explicit guidance about when to prefer this tool over an alternative, and no 'when not to use' statement. One example even contradicts the stated pagination rule, which weakens the usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_foldersZephyr: Get FoldersA
Read-onlyIdempotent
Inspect

Get folders, optionally filtered by project and folder type

Toolset: Folders

Parameters:

  • maxResults (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned. (default: 10)

  • startAt (number): Zero-indexed starting position. Should be a multiple of maxResults. (default: 0)

  • projectKey (string): Jira project key filter

  • folderType (string): Folder type filter. Either "TEST_CASE", "TEST_PLAN" or "TEST_CYCLE"

Use Cases: 1. List folders in a Zephyr project 2. Get Test Case folders for a project 3. Get Test Plan folders for a project 4. Get Test Cycle folders for a project 5. Browse the folder structure in Zephyr

Examples:

  1. Get the first 10 Folders

{
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 Folders with their details

  1. Get Folders from the project SA

{
  "projectKey": "SA",
  "maxResults": 10,
  "startAt": 0
}

Expected Output: Folders belonging to project SA with their details

  1. Get Test Case Folders from the project MM2

{
  "projectKey": "MM2",
  "folderType": "TEST_CASE",
  "maxResults": 10,
  "startAt": 0
}

Expected Output: Test Case Folders belonging to project MM2 with their details

  1. Get Test Cycle Folders across all projects

{
  "folderType": "TEST_CYCLE",
  "maxResults": 10,
  "startAt": 0
}

Expected Output: Test Cycle Folders from all projects with their details

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position. Should be a multiple of maxResults.
folderTypeNoFolder type filter. Either `"TEST_CASE"`, `"TEST_PLAN"` or `"TEST_CYCLE"`
maxResultsNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned.
projectKeyNoJira project key filter

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNoURL to the next page of results, or null if there are no more results.
totalNoIndicates the total number of items available across all pages.
isLastNoIndicates if this is the last page of results.
valuesNo
startAtYesIndicates the index of the first item returned in the page of results.
maxResultsYesIndicates the maximum number of results in this response. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds behavioral context beyond the annotations by explaining that maxResults may be lowered by the server, results may be truncated, and the response's maxResults value should be checked. This is valuable for an agent handling pagination. Annotations already cover read-only/idempotent/non-destructive behavior, so the description supplements rather than replaces that information.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than minimal but well-structured with a one-sentence summary, parameter list, use cases, and examples. Each section earns its place, and the most important scoping information is front-loaded. It could be trimmed by removing duplication with the schema, but overall it is organized and readable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that an output schema exists, annotations cover safety, and the description covers pagination, filtering behavior, and examples, the definition is complete for an agent to invoke this tool correctly. All four optional parameters are explained, and the examples cover both unfiltered and filtered calls.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3, but the description goes further by showing parameter combinations in concrete examples with expected outputs, such as filtering by projectKey and folderType. It also repeats and clarifies the maxResults lower-bound behavior, which helps an agent apply the parameters correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Get folders', and immediately clarifies optional filters by project and folder type. This distinguishes it from sibling tools like zephyr_get_projects or zephyr_get_test_cycles without requiring schema inspection. The Toolset heading reinforces its scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Use Cases' section provides clear scenarios such as listing folders in a project and filtering by folder type, giving an agent concrete context for when to call this tool. It does not explicitly mention alternatives or when not to use it, so it misses the top score, but the guidance is clear and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_prioritiesZephyr: Get prioritiesC
Read-onlyIdempotent
Inspect

Get Zephyr Test Case priorities with optional filters

Toolset: Priorities

Parameters:

  • maxResults (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned. (default: 10)

  • startAt (number): Zero-indexed starting position. Should be a multiple of maxResults. (default: 0)

  • projectKey (string): Jira project key filter

Examples:

  1. Get the first 10 priorities

{
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 priorities with their details

  1. Get priorities for a specific project

{
  "projectKey": "PROJ"
}

Expected Output: The priorities for project PROJ

  1. Get all priorities

{}

Expected Output: All priorities

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position. Should be a multiple of maxResults.
maxResultsNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned.
projectKeyNoJira project key filter

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNoURL to the next page of results, or null if there are no more results.
totalNoIndicates the total number of items available across all pages.
isLastNoIndicates if this is the last page of results.
valuesNo
startAtYesIndicates the index of the first item returned in the page of results.
maxResultsYesIndicates the maximum number of results in this response. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints.

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/idempotentHint and destructiveHint false, so the safety profile is covered. The description repeats the schema's truncation note but then contradicts the default-pagination behavior by promising 'All priorities' for an empty parameter set.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is organized with toolset, parameters, and examples, but the parameters section repeats the input schema nearly verbatim and the maxResults note is verbose. It is structured but not particularly concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list operation with an output schema and optional filters, most invocation details are present: defaults, projectKey format, and paging examples. However, the incorrect 'all priorities' example leaves a gap in how to page through or truly retrieve all results, so an agent could invoke it with the wrong expectation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, and the description's parameter section largely duplicates the schema. The examples add combination scenarios, but the third example assigns incorrect semantics to the default maxResults value, making the guidance unreliable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get Zephyr Test Case priorities with optional filters' and reinforces it with a 'Toolset: Priorities' label. This is enough to distinguish it from sibling zephyr_get_projects/zephyr_get_statuses, though it doesn't explicitly contrast with them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied through examples and optional-filter descriptions rather than stated explicitly. There is no direct statement of when to choose this over alternatives, and one example is misleading: {} is labeled 'Get all priorities' even though maxResults defaults to 10.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_projectZephyr: Get ProjectA
Read-onlyIdempotent
Inspect

Get details of project specified by id or key in Zephyr

Toolset: Projects

Parameters:

  • projectIdOrKey (string) required: The Zephyr project ID or Jira project key

Examples:

  1. Get the project with id 1

{
  "projectIdOrKey": "1"
}

Expected Output: The project with its details

  1. Get the project with key 'PROJ'

{
  "projectIdOrKey": "PROJ"
}

Expected Output: The project with its details

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdOrKeyYesThe Zephyr project ID or Jira project key

Output Schema

ParametersJSON Schema
NameRequiredDescription
idYesThe ID of the project in Zephyr.
keyYes
enabledYesIndicates whether the project has Zephyr enabled on it.
jiraProjectIdYesThe ID of the project in Jira.

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds no additional behavioral context such as error behavior, identifier precedence, or access requirements; the examples and 'Expected Output' lines mostly restate the obvious.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The main sentence is front-loaded and the structure separates toolset, parameter, and examples clearly. It is slightly padded by the repeated 'Expected Output: The project with its details' lines, but remains compact for a one-parameter lookup.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With only one required parameter, a full input schema, an output schema, and read-only annotations, the definition provides enough to invoke the tool correctly. It could be more complete by pointing to zephyr_get_projects for when no single identifier is known, but that gap is minor and covered under usage guidance.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes projectIdOrKey and provides a validation pattern, so description-level parameter explanation is not necessary. The two examples ('1' and 'PROJ') illustrate the accepted formats but add little meaning beyond the schema's pattern and property description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear verb and resource: 'Get details of project specified by id or key in Zephyr.' It unambiguously targets a single project and names the identifying input, but it does not explicitly contrast with the plural sibling zephyr_get_projects, so it stops short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'specified by id or key' implies the tool is for looking up one project when an identifier is already known. It does not state exclusions or name alternatives such as zephyr_get_projects for listing projects, so guidance remains implicit rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_projectsZephyr: Get ProjectsA
Read-onlyIdempotent
Inspect

Get details of projects in Zephyr

Toolset: Projects

Parameters:

  • maxResults (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned. (default: 10)

  • startAt (number): Zero-indexed starting position. Should be a multiple of maxResults. (default: 0)

Examples:

  1. Get the first 10 projects

{
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 projects with their details

  1. Get any project

{
  "maxResults": 1
}

Expected Output: One project with its details

  1. Get five projects starting from the 7th project of the list

{
  "maxResults": 5,
  "startAt": 6
}

Expected Output: The 7th to the 11th projects with their details

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position. Should be a multiple of maxResults.
maxResultsNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned.

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNoURL to the next page of results, or null if there are no more results.
totalNoIndicates the total number of items available across all pages.
isLastNoIndicates if this is the last page of results.
valuesNo
startAtYesIndicates the index of the first item returned in the page of results.
maxResultsYesIndicates the maximum number of results in this response. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints.

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the read-only nature is already disclosed. The description adds valuable behavioral context beyond annotations: it explains server-side truncation, that maxResults may be lowered by the server, and that the response's maxResults field must be checked to confirm actual returned count. This is genuinely useful operational detail not present in the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear title, parameter breakdown, and examples. The examples are helpful but somewhat verbose for an API description; the 'Expected Output' lines add marginal value. Still, the information is organized and front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a simple paginated list operation with an output schema present, so return values are documented elsewhere. The description covers pagination behavior, defaults, and server-side truncation caveats. It does not mention sorting or filtering, but for a simple 'get all projects' listing, the provided guidance is sufficient for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both parameters fully documented in the schema. The description repeats the parameter details and adds three concrete examples, which reinforce usage but do not add meaning beyond the schema. Baseline 3 is appropriate since the schema carries the semantic load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: "Get details of projects in Zephyr." This is clear and aligns with the tool name. It does not explicitly distinguish itself from the sibling tool 'zephyr_get_project' (singular), which could fetch a single project, but the plural form and description imply a list operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes pagination parameters and examples showing how to get the first projects or a range, which implies usage for listing projects. However, it does not explicitly state when to use this tool versus alternatives like 'zephyr_get_project' or 'qtm4j_get_projects', leaving the agent to infer the distinction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_statusesZephyr: Get StatusesA
Read-onlyIdempotent
Inspect

Get statuses of different types of test artifacts in Zephyr

Toolset: Statuses

Parameters:

  • maxResults (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned. (default: 10)

  • startAt (number): Zero-indexed starting position. Should be a multiple of maxResults. (default: 0)

  • projectKey (string): Jira project key filter

  • statusType (enum): Determines which type of entity the status belongs to.

Examples:

  1. Get the first 10 statuses

{
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 statuses with their details from different projects and test artifact types

  1. Get 10 test case statuses

{
  "maxResults": 10,
  "statusType": "TEST_CASE"
}

Expected Output: A list of statuses related to test cases with their details

  1. Get five statuses from the project PROJ

{
  "maxResults": 5,
  "projectKey": "PROJ"
}

Expected Output: The first five statuses from the project PROJ with their details

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position. Should be a multiple of maxResults.
maxResultsNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned.
projectKeyNoJira project key filter
statusTypeNoDetermines which type of entity the status belongs to.

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNoURL to the next page of results, or null if there are no more results.
totalNoIndicates the total number of items available across all pages.
isLastNoIndicates if this is the last page of results.
valuesNo
startAtYesIndicates the index of the first item returned in the page of results.
maxResultsYesIndicates the maximum number of results in this response. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints.

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is established. The description adds valuable runtime behavior: the server may enforce a lower maxResults limit, the result set may be truncated, and callers should verify the maxResults value in the response. It also documents the startAt/maxResults relationship, which is useful beyond the annotation metadata.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized and front-loaded with its purpose, but it duplicates much of the input schema's parameter documentation, including the full maxResults caveat verbatim. The three examples are useful, but the expected-output lines are vague ('with their details'), making the text longer than necessary without proportionally increasing clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With zero required parameters, 100% schema coverage, an output schema, and annotations declaring the tool read-only and idempotent, the description fills the remaining gaps effectively. It explains defaults, filtering options, pagination/truncation behavior, and provides representative examples, so nothing critical is missing for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all four parameters, including defaults, the enum values, and the server-limit caveat for maxResults. The description largely restates those schema definitions, and while the examples show useful filter combinations, they do not add substantial meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource: 'Get statuses of different types of test artifacts in Zephyr,' which clearly identifies a distinct resource among the Zephyr getter tools. The statusType enum values (TEST_CASE, TEST_PLAN, TEST_CYCLE, TEST_EXECUTION) and the 'Toolset: Statuses' label further differentiate it from sibling tools like zephyr_get_priorities or zephyr_get_test_cycles.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly name alternatives or exclusions, but its examples create clear usage context: a default listing, filtering by statusType, and filtering by projectKey. This is sufficient for an agent to recognize when this read-only status lookup is the appropriate call, though no explicit 'use this instead of X' guidance is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_caseZephyr: Get Test CaseA
Read-onlyIdempotent
Inspect

Get details of test case specified by key in Zephyr

Toolset: Test Cases

Parameters:

  • testCaseKey (string) required: The key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

Examples:

  1. Get the test case with key 'SA-T10'

{
  "testCaseKey": "SA-T10"
}

Expected Output: The test case with its details

  1. Get the test case with key 'MM2-T1'

{
  "testCaseKey": "MM2-T1"
}

Expected Output: The test case with its details

ParametersJSON Schema
NameRequiredDescriptionDefault
testCaseKeyYesThe key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

Output Schema

ParametersJSON Schema
NameRequiredDescription
idYesThe ID of the entity
keyYesThe test case key
nameYes
linksNoA list of links for this test case.
ownerNo
folderNoID and link to the folder resource.
labelsNoArray of labels associated to this entity.
statusYesID and link to the status resource.
projectYesID and link relative to Zephyr project.
priorityYesID and link to the priority resource.
componentNoID and link to the Jira component resource.
createdOnNoData and time test case was created. Format: yyyy-MM-dd'T'HH:mm:ss'Z'. This field is read-only, cannot be updated.
objectiveNoA description of the objective.
testScriptNo
customFieldsNoMulti-line text fields support HTML and should denote new lines with the \<br\> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.
preconditionNoAny conditions that need to be met.
estimatedTimeNoEstimated duration in milliseconds.

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description confirms it is a read operation and mentions expected output, but it does not add behavioral context such as not-found behavior, permissions, or pagination, which would have added value beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear opening sentence, a parameters section, and examples. The two examples are somewhat redundant since they only differ in the key value, but the overall length is appropriate and the important information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a simple single-parameter read operation, and the description covers the parameter, its format, and examples. An output schema exists, so return-value details are not required in the description. Nothing essential is missing for an agent to successfully call this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description adds value with the key format '[A-Z]+-T[0-9]+' and concrete examples like 'SA-T10'. These examples clarify what valid test case keys look like beyond the schema's looser pattern.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get details of test case') and the identifying input ('specified by key in Zephyr'), making the tool's purpose unambiguous. It does not explicitly name sibling tools, so it stops short of a perfect 5, but the singular focus on a key distinguishes it from list-oriented siblings like zephyr_get_test_cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to prefer this tool over alternatives. The description provides examples but does not mention related tools such as zephyr_get_test_cases or zephyr_get_test_case_steps, nor does it state when a list versus a single test case is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_casesZephyr: Get Test CasesA
Read-onlyIdempotent
Inspect

Get details of test cases in Zephyr

Toolset: Test Cases

Parameters:

  • projectKey (string): Jira project key filter

  • folderId (number): Folder ID filter

  • limit (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the limit value in the response to confirm how many results were actually returned. (default: 10)

  • startAtId (number): Zero-indexed starting position for ID-based pagination. (default: 0)

  • updatedAfter (string): Filter only entities updated after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

Examples:

  1. Get the first 10 Test Cases

{
  "limit": 10,
  "startAtId": 1
}

Expected Output: The first 10 Test Cases with their details

  1. Get any Test Case

{
  "limit": 1
}

Expected Output: One Test Case with its details

  1. Get five Test Cases starting from the ID 123

{
  "limit": 5,
  "startAtId": 123
}

Expected Output: Five Test Cases starting from the ID 123 with their details

  1. Get one Test Case from the project PROJ

{
  "projectKey": "PROJ",
  "limit": 1
}

Expected Output: One Test Case from project PROJ with its details

  1. Get one Test Case from the folder with ID 123

{
  "folderId": 123,
  "limit": 1
}

Expected Output: One Test Case from folder with ID 123 with its details

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the limit value in the response to confirm how many results were actually returned.
folderIdNoFolder ID filter
startAtIdNoZero-indexed starting position for ID-based pagination.
projectKeyNoJira project key filter
updatedAfterNoFilter only entities updated after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNo
limitYes
valuesNo
nextStartAtIdYes

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint=true, idempotentHint=true, destructiveHint=false), the description discloses an important behavioral trait: the server may enforce a lower limit than requested, causing the result set to be truncated, and instructs the agent to 'Always check the limit value in the response.' It also explains that startAtId is zero-indexed for ID-based pagination. This adds meaningful behavioral context beyond what the annotations already communicate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with headings for toolset, parameters, and examples, and the truncation warning is useful. However, the parameter section largely duplicates the schema descriptions, and the five examples are somewhat repetitiveβ€”each shows a similar pattern with one varying filter. This could be more concise, though the organization makes it easily scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, filtered list tool with an output schema present, the description covers all necessary invocation details: all optional filters, pagination parameters, defaults, truncation behavior, and illustrative examples. It does not describe the response structure, but the output schema exists and would carry that information. The description is sufficiently complete for an agent to call the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already fully documents every parameter, including defaults, maximums, formats, and descriptions. The tool description repeats this information nearly verbatim rather than adding new meaning. The examples illustrate parameter combinations but do not provide additional semantic detail about the parameters themselves beyond what the schema already states. A baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Get details of test cases in Zephyr.' This clearly identifies the operation and the domain, and the plural 'test cases' helps distinguish it from the singular 'zephyr_get_test_case.' However, it does not explicitly state that it returns a list or paginated results, nor does it explicitly contrast itself with sibling tools like 'zephyr_get_test_case' or 'zephyr_get_test_case_steps.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides multiple examples showing how to filter by project, folder, pagination, and limit, which implies when an agent might want to use this tool. However, there is no explicit guidance about when to use this tool versus alternatives (e.g., 'for a single test case, use zephyr_get_test_case'), nor any mention of when this tool should not be used. The usage context is implied through examples rather than stated directly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_case_stepsZephyr: Get Test Case StepsA
Read-onlyIdempotent
Inspect

Get details of test case steps in Zephyr

Toolset: Test Cases

Parameters:

  • testCaseKey (string) required: The key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

  • maxResults (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned. (default: 10)

  • startAt (number): Zero-indexed starting position. Should be a multiple of maxResults. (default: 0)

Examples:

  1. Get the first 10 test case steps for test case with key 'SA-T1'

{
  "testCaseKey": "SA-T1",
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 test case steps with their details

  1. Get any test case step for test case with key 'SA-T1'

{
  "testCaseKey": "SA-T1",
  "maxResults": 1
}

Expected Output: One test case step with its details

  1. Get five test case steps starting from the 7th test case step of the list for test case with key 'SA-T1'

{
  "testCaseKey": "SA-T1",
  "maxResults": 5,
  "startAt": 6
}

Expected Output: The 7th to the 11th test case steps with their details

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position. Should be a multiple of maxResults.
maxResultsNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned.
testCaseKeyYesThe key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNoURL to the next page of results, or null if there are no more results.
totalNoIndicates the total number of items available across all pages.
isLastNoIndicates if this is the last page of results.
valuesNoThe list of test steps
startAtYesIndicates the index of the first item returned in the page of results.
maxResultsYesIndicates the maximum number of results in this response. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints.

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey read-only, idempotent, non-destructive behavior. The description adds a genuinely useful caveat that the server may enforce a lower maxResults than requested and truncate results, and it instructs the caller to check the returned maxResults. This adds value beyond the annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded and the content is organized into clear sections for parameters and examples. It is somewhat longer than strictly necessary because the parameter documentation duplicates the schema, but the examples are meaningful and well structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and read-only annotations, the description covers the required key format, optional parameters, defaults, pagination, truncation behavior, and example calls. It does not explicitly explain ordering or how to choose between similar Zephyr get tools, but the essential context for calling this tool is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds concrete examples and clarifies pagination semantics, including startAt as zero-indexed and the 7th-to-11th-step example. The 'get any test case step' example is slightly misleading because it actually returns the first result by default, preventing a perfect score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get details of test case steps in Zephyr,' which clearly identifies the operation and distinguishes it from Zephyr tools that create, link, or retrieve execution steps. It does not explicitly name sibling alternatives, but the test-case-step resource is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to prefer this tool over alternatives such as zephyr_get_test_execution_steps or zephyr_get_test_case. The intended use is only implied by the name and first sentence; no exclusions, prerequisites, or alternative-selection criteria are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_cycleZephyr: Get Test CycleA
Read-onlyIdempotent
Inspect

Get details of test cycle specified by id or key in Zephyr

Toolset: Test Cycles

Parameters:

  • testCycleIdOrKey (string) required: The ID or key of the test cycle.

Examples:

  1. Get the test cycle with id 1

{
  "testCycleIdOrKey": "1"
}

Expected Output: The test cycle with its details

  1. Get the test cycle with key 'SA-R40'

{
  "testCycleIdOrKey": "SA-R40"
}

Expected Output: The test cycle with its details

ParametersJSON Schema
NameRequiredDescriptionDefault
testCycleIdOrKeyYesThe ID or key of the test cycle.

Output Schema

ParametersJSON Schema
NameRequiredDescription
idYesThe ID of the entity
keyYesUnique key of the test cycle
nameYesName of the Test Cycle
linksNoRepresents all links that a Test Cycle has.
ownerNo
folderNoID and link to the folder resource.
statusYesID and link to the status resource.
projectYesID and link relative to Zephyr project.
descriptionNoDescription outlining the scope.
customFieldsNoMulti-line text fields support HTML and should denote new lines with the \<br\> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.
plannedEndDateNoThe planned end date of the test cycle. This field cannot be blank. Setting it as null or excluding it from the request will leave the field values unchanged. ISO 8601 Format (i.e., yyyy-MM-dd'T'HH:mm:ss'Z')
plannedStartDateNoPlanned start date of the test cycle. This field cannot be blank. Setting it as null or excluding it from the request will leave the field values unchanged. ISO 8601 Format (i.e., yyyy-MM-dd'T'HH:mm:ss'Z')
jiraProjectVersionNoID and Link to fetch information about Jira Project version. Relates to 'Version' or 'Releases' in Jira projects.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool read-only, idempotent, and non-destructive, so the description is not burdened with safety disclosure. It adds expected output wording ('The test cycle with its details') and concrete examples, but no deeper behavioral context such as not-found behavior or response shape. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded, and the parameter list and examples are logically organized. The repeated 'Expected Output: The test cycle with its details' after both examples is slightly redundant, but overall the structure is easy to scan and every section earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, read-only tool with a full input schema, an output schema, and two worked examples, the description provides everything needed to invoke it correctly. Nothing critical is missing given the simple complexity and the presence of structured metadata.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value by pairing the schema pattern with concrete examples ('1' and 'SA-R40') that make it clear how to supply either a numeric ID or a key, which is more immediately useful than the raw regex alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get details') and a specific resource ('test cycle specified by id or key'), which unambiguously identifies what the tool does. It is clearly differentiated from the sibling zephyr_get_test_cycles by its singular scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by showing examples for both ID and key inputs, but it never explicitly states when to choose this tool over the plural zephyr_get_test_cycles or other related tools. No alternative is named and no exclusion criteria are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_cyclesZephyr: Get Test CyclesA
Read-onlyIdempotent
Inspect

Get details of Test Cycles in Zephyr

Toolset: Test Cycles

Parameters:

  • projectKey (string): Jira project key filter

  • folderId (number): Folder ID filter

  • jiraProjectVersionId (number): Jira Project Version ID. Relates to 'Version' or 'Releases' in Jira projects.

  • maxResults (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned. (default: 10)

  • startAt (number): Zero-indexed starting position. Should be a multiple of maxResults. (default: 0)

Examples:

  1. Get the first 10 Test Cycles

{
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 Test Cycles with their details

  1. Get any Test Cycle

{
  "maxResults": 1
}

Expected Output: One Test Cycle with its details

  1. Get five Test Cycles starting from the 7th Test Cycles of the list

{
  "maxResults": 5,
  "startAt": 6
}

Expected Output: The 7th to the 11th Test Cycles with their details

  1. Get one Test Cycle from the project PROJ

{
  "projectKey": "PROJ",
  "maxResults": 1
}

Expected Output: One Test Cycle from project PROJ with its details

  1. Get one Test Cycle from the folder with ID 123

{
  "folderId": 123,
  "maxResults": 1
}

Expected Output: One Test Cycle from folder with ID 123 with its details

  1. Get one Test Cycle from the version 456

{
  "jiraProjectVersionId": 456,
  "maxResults": 1
}

Expected Output: One Test Cycle from version 456 with its details

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position. Should be a multiple of maxResults.
folderIdNoFolder ID filter
maxResultsNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned.
projectKeyNoJira project key filter
jiraProjectVersionIdNoJira Project Version ID. Relates to 'Version' or 'Releases' in Jira projects.

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNoURL to the next page of results, or null if there are no more results.
totalNoIndicates the total number of items available across all pages.
isLastNoIndicates if this is the last page of results.
valuesNo
startAtYesIndicates the index of the first item returned in the page of results.
maxResultsYesIndicates the maximum number of results in this response. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints.

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable behavioral context beyond that: default/max pagination limits, server-side limit enforcement, potential result truncation, and the instruction to check maxResults in the response. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections and helpful examples, but it duplicates the schema's parameter descriptions almost verbatim and includes several repetitive 'Expected Output' lines. It earns some value from examples, but is longer than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and annotations covering safety, the description covers the remaining practical needs: all filters, pagination behavior, truncation warning, and six usage examples. It is complete enough for an agent to invoke the tool correctly, though it does not explicitly discuss combining multiple filters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all five parameters thoroughly. The description mostly repeats that text and adds examples, but it does not provide much new semantic meaning beyond what the parameter descriptions already cover. Baseline 3 applies here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb, resource, and scope: 'Get details of Test Cycles in Zephyr.' The plural 'Test Cycles' signals this is a listing operation and distinguishes it from the singular sibling zephyr_get_test_cycle, though it does not explicitly name or contrast the sibling relationship.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The examples show practical scenarios for using the tool with different filters, which implies when it is appropriate to call it. However, there is no explicit guidance on when not to use it or when to prefer a sibling tool such as zephyr_get_test_cycle for a single cycle or zephyr_get_test_cycle_links for links.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_cycles_linked_to_a_jira_issueZephyr: Get Test Cycles linked to a Jira issueA
Read-onlyIdempotent
Inspect

Get test cycles linked to a Jira issue in Zephyr

Toolset: Issue Links

Parameters:

  • issueKey (string) required: The key of the Jira issue

Examples:

  1. Check which test cycles are linked to Jira issue PROJ-123

{
  "issueKey": "PROJ-123"
}

Expected Output: The List of test cycles linked to Jira issue PROJ-123 with their IDs

ParametersJSON Schema
NameRequiredDescriptionDefault
issueKeyYesThe key of the Jira issue

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already convey readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the tool's safety profile is clear. The description adds useful scoping context ('linked to a Jira issue') and notes that output is a list of cycles with IDs, but it does not disclose any additional behavioral caveats such as pagination or link-type filtering. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded: purpose, toolset, parameter list, then example. The example JSON is useful and the expected output is summarized. It is concise without being under-specified.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, read-only tool with an output schema and clear annotations, the description is largely sufficient. It gives a concrete invocation and expected output. The main gap is lack of sibling differentiation, but that is not crucial for invoking this simple tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: the schema already documents issueKey with type, requiredness, and a pattern. The description repeats this and adds a concrete example value ('PROJ-123'), but does not meaningfully extend the parameter semantics beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Get test cycles linked to a Jira issue in Zephyr', and the toolset label 'Issue Links' adds context. It is clear in what it does, but it does not explicitly differentiate itself from similar siblings like zephyr_get_test_cycle_links or zephyr_get_test_executions_linked_to_a_jira_issue.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The example ('Check which test cycles are linked to Jira issue PROJ-123') implies when the tool should be used. However, there is no explicit guidance about when to choose this tool over alternatives, nor any exclusions or when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_executionZephyr: Get Test ExecutionB
Read-onlyIdempotent
Inspect

Get details of test execution specified by id or key in Zephyr

Toolset: Test Executions

Parameters:

  • testExecutionIdOrKey (string) required: The ID or key of the test execution. Test execution keys are of the format [A-Z]+-E[0-9]+

Examples:

  1. Get the test execution with id 1

{
  "testExecutionIdOrKey": "1"
}

Expected Output: The test execution with its details

  1. Get the test execution with key 'PROJ-E123'

{
  "testExecutionIdOrKey": "PROJ-E123"
}

Expected Output: The test execution with its details

ParametersJSON Schema
NameRequiredDescriptionDefault
testExecutionIdOrKeyYesThe ID or key of the test execution. Test execution keys are of the format [A-Z]+-E[0-9]+

Output Schema

ParametersJSON Schema
NameRequiredDescription
idYesThe ID of the entity
keyNoTest execution key
linksNo
commentNoComment added against overall test case execution.
projectYesID and link relative to Zephyr project.
testCaseYes
automatedNoIndicates if the test execution was done manually or not.
testCycleNo
environmentNoID and link to the environment resource.
assignedToIdNoAtlassian Account ID of the Jira user.
customFieldsNoMulti-line text fields support HTML and should denote new lines with the \<br\> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.
executedByIdNoAtlassian Account ID of the Jira user.
actualEndDateNoThe actual end date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'
estimatedTimeNoEstimated duration in milliseconds.
executionTimeNoActual test execution time in milliseconds.
jiraProjectVersionNoID and Link to fetch information about Jira Project version. Relates to 'Version' or 'Releases' in Jira projects.
testExecutionStatusYesID and link to the status resource.

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is clear. The description adds no behavioral context beyond the parameter format and examples; it does not mention response behavior, error cases, or any side effects, though none are expected for a read-only get operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections for parameters and examples, and it is appropriately compact for a one-parameter tool. The repeated 'Expected Output: The test execution with its details' lines are somewhat redundant and add little value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, read-only, single-parameter fetch tool, the definition is largely complete: the schema covers the parameter, annotations cover the behavioral safety profile, and an output schema exists. The examples clarify both supported identifier formats, leaving no critical gap for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema includes the same parameter description plus a regex pattern. The description adds two concrete examples using an id ('1') and a key ('PROJ-E123'), which is helpful, but it does not provide meaning beyond what the schema already conveys.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Get details'), the resource ('test execution'), and the lookup mechanism ('specified by id or key'), making the tool's purpose clear. It does not explicitly name sibling tools like zephyr_get_test_executions, but the singular resource and the id/key qualifier distinguish it well enough.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternatives such as zephyr_get_test_executions (the plural listing tool). The intended use is only implied by the parameter name and examples, and there is no mention of exclusions or conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_executionsZephyr: Get Test ExecutionsA
Read-onlyIdempotent
Inspect

Get test executions with optional filters

Toolset: Test Executions

Parameters:

  • projectKey (string): Jira project key filter

  • testCycle (string): Test cycle key filter.

  • testCase (string): Test case key filter.

  • actualEndDateAfter (string): Filter for 'Actual End Date' after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

  • actualEndDateBefore (string): Filter for 'Actual End Date' before the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

  • includeStepLinks (boolean): If true, execution step issue links will be included in the response (default: false)

  • jiraProjectVersionId (number): Jira Project Version ID. Relates to 'Version' or 'Releases' in Jira projects.

  • onlyLastExecutions (boolean): If true, includes only the last execution of each test cycle item (test case), and all ad-hoc test executions. (default: false)

  • limit (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the limit value in the response to confirm how many results were actually returned. (default: 10)

  • startAtId (number): Zero-indexed starting position for ID-based pagination. (default: 0)

  • updatedAfter (string): Filter only entities updated after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

Examples:

  1. Get the first 10 test executions

{
  "limit": 10,
  "startAtId": 0
}

Expected Output: The first 10 test executions with their details

  1. Get 5 test executions for the project PROJ

{
  "projectKey": "PROJ",
  "limit": 5
}

Expected Output: Up to 5 test executions for project PROJ

  1. Get some test executions that finished after 01/Jan/2024

{
  "actualEndDateAfter": "2024-01-01T00:00:00Z"
}

Expected Output: Test executions that ended after 2024-01-01

  1. Get test executions with step links included

{
  "includeStepLinks": true
}

Expected Output: Test executions with step links included

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the limit value in the response to confirm how many results were actually returned.
testCaseNoTest case key filter.
startAtIdNoZero-indexed starting position for ID-based pagination.
testCycleNoTest cycle key filter.
projectKeyNoJira project key filter
updatedAfterNoFilter only entities updated after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'
includeStepLinksNoIf true, execution step issue links will be included in the response
actualEndDateAfterNoFilter for 'Actual End Date' after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'
onlyLastExecutionsNoIf true, includes only the last execution of each test cycle item (test case), and all ad-hoc test executions.
actualEndDateBeforeNoFilter for 'Actual End Date' before the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'
jiraProjectVersionIdNoJira Project Version ID. Relates to 'Version' or 'Releases' in Jira projects.

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNo
limitYes
valuesNo
nextStartAtIdYes

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds useful extra behavior beyond those annotations, including the server's ability to enforce a lower limit, truncation risk, and the instruction to check the returned limit value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is concise and front-loaded, but the body largely duplicates the schema's parameter descriptions verbatim and appends several examples with trivial 'Expected Output' lines. The description is longer than it needs to be for the additional value it provides.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema exists and annotations cover the safety profile, the description is largely complete: it covers all filters, pagination behavior, defaults, and limit truncation. It does not explicitly position itself against sibling tools, but nothing critical seems missing for calling the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter. The description mostly repeats those same descriptions and adds examples that illustrate valid combinations, but it does not materially deepen the meaning of any parameter beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource ('Get test executions') and signals that filters are optional, which implies a list/query operation. It distinguishes from the singular sibling 'zephyr_get_test_execution' by pluralizing the resource and emphasizing filtering, though it does not explicitly name any sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this is the tool for retrieving filtered test executions through examples and filter parameters, but it never explicitly says when to use this tool versus alternatives like the singular 'zephyr_get_test_execution' or the various linked-execution tools. The intended context is inferable, but not stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_executions_linked_to_a_jira_issueZephyr: Get test executions linked to a Jira issueA
Read-onlyIdempotent
Inspect

Get test executions linked to a Jira issue in Zephyr

Toolset: Issue Links

Parameters:

  • issueKey (string) required: The key of the Jira issue

Examples:

  1. Check which test executions are linked to Jira issue PROJ-123

{
  "issueKey": "PROJ-123"
}

Expected Output: The List of test executions linked to Jira issue PROJ-123 with their keys and versions

ParametersJSON Schema
NameRequiredDescriptionDefault
issueKeyYesThe key of the Jira issue

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds the linked-to-issue scoping and an expected output note about keys and versions, but does not disclose behavior such as handling of empty results or pagination; this is adequate given the output schema and annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a clear opening sentence, followed by a useful Toolset label and a concrete JSON example. The Parameters section duplicates schema content, adding minor redundancy, but overall the structure remains compact and scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a single-required-parameter read-only lookup with an output schema and a read/idempotent annotation profile. The description, example, and expected-output note together give an agent everything needed to invoke it correctly; no additional operational context is required.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with issueKey already documented by type, required status, pattern, and a description. The description repeats the same parameter and provides an example ('PROJ-123'), which is mildly helpful but does not substantially extend the meaning already present in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb ('Get') and resource ('test executions linked to a Jira issue in Zephyr'). The qualifier 'linked to a Jira issue' differentiates it from the more generic zephyr_get_test_executions and related link-oriented siblings, so an agent can determine the tool's scope without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The use case is implied through the issueKey parameter and the example ('Check which test executions are linked to Jira issue PROJ-123'). However, the description never explicitly tells the agent when to choose this tool over alternatives such as zephyr_get_test_executions or zephyr_get_test_execution_links, and no when-not-to-use guidance is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_execution_stepsZephyr: Get Test Execution StepsA
Read-onlyIdempotent
Inspect

Get details of test execution steps in Zephyr

Toolset: Test Executions

Parameters:

  • testExecutionIdOrKey (string) required: The ID or key of the test execution. Test execution keys are of the format [A-Z]+-E[0-9]+

  • maxResults (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned. (default: 10)

  • startAt (number): Zero-indexed starting position. Should be a multiple of maxResults. (default: 0)

  • testDataRowNumber (number): The id of the test data row to retrieve.

Examples:

  1. Get the first 10 test execution steps for test execution with ID 1

{
  "testExecutionIdOrKey": "1",
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 test execution steps with their details

  1. Get the first 10 test execution steps for test execution with key 'SA-E1'

{
  "testExecutionIdOrKey": "SA-E1",
  "maxResults": 10,
  "startAt": 0
}

Expected Output: The first 10 test execution steps with their details

  1. Get any test execution step for test execution with key 'SA-E1'

{
  "testExecutionIdOrKey": "SA-E1",
  "maxResults": 1
}

Expected Output: One test execution step with its details

  1. Get five test execution steps starting from the 7th test execution step for test execution with key 'SA-E1'

{
  "testExecutionIdOrKey": "SA-E1",
  "maxResults": 5,
  "startAt": 6
}

Expected Output: The 7th to the 11th test execution steps with their details

  1. Get test execution steps from the test data row 1 from test execution with key 'SA-E1'

{
  "testExecutionIdOrKey": "SA-E1",
  "testDataRowNumber": 1,
  "maxResults": 10,
  "startAt": 0
}

Expected Output: Test execution steps for the specified test data row

ParametersJSON Schema
NameRequiredDescriptionDefault
startAtNoZero-indexed starting position. Should be a multiple of maxResults.
maxResultsNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the maxResults value in the response to confirm how many results were actually returned.
testDataRowNumberNoThe id of the test data row to retrieve.
testExecutionIdOrKeyYesThe ID or key of the test execution. Test execution keys are of the format [A-Z]+-E[0-9]+

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNoURL to the next page of results, or null if there are no more results.
totalNoIndicates the total number of items available across all pages.
isLastNoIndicates if this is the last page of results.
valuesNoThe list of test steps
startAtYesIndicates the index of the first item returned in the page of results.
maxResultsYesIndicates the maximum number of results in this response. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints.

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal read-only, idempotent, and non-destructive behavior. The description adds meaningful behavioral context beyond that: the server may enforce a lower maxResults limit than requested, results may be truncated, and the caller should check the maxResults value in the response. This is useful, non-contradictory disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a clear one-line purpose and organized into Parameters and Examples sections. It is somewhat redundant with the schema's parameter descriptions, but the examples earn their place by illustrating pagination, startAt behavior, and testDataRowNumber usage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only paginated list operation with an output schema and full annotations, the description covers the key format, pagination caveat, startAt semantics, and test data row use cases well. The main completeness issue is the inconsistent maxResults limit between the description and the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All four parameters are fully described in the input schema, so the baseline is 3. The description restates those details and adds five worked examples showing valid combinations and expected outputs, which is helpful. However, it conflicts with the schema by claiming maxResults can be requested up to 1000 while the schema states a maximum of 9007199254740991, creating ambiguity for an agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Get details of test execution steps in Zephyr') and the Toolset label helps orient it within Test Executions. It does not explicitly distinguish itself from sibling tools like zephyr_get_test_case_steps or zephyr_get_test_execution, so an agent must infer those boundaries from the resource names.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The examples show how to call the tool, and the purpose implies when it is appropriate, but there is no explicit 'use this instead of X' guidance or exclusionary context versus related Zephyr tools. Usage is implied rather than stated as clear decision criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_plansZephyr: Get Test PlansA
Read-onlyIdempotent
Inspect

Get details of Test Plans in Zephyr

Toolset: Test Plans

Parameters:

  • projectKey (string): Jira project key filter

  • limit (number): Specifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000.

Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the limit value in the response to confirm how many results were actually returned. (default: 10)

  • startAtId (number): Zero-indexed starting position for ID-based pagination. (default: 0)

  • updatedAfter (string): Filter only entities updated after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

Examples:

  1. Get the first 10 Test Plans

{
  "limit": 10,
  "startAtId": 0
}

Expected Output: The first 10 Test Plans with their details

  1. Get any Test Plan

{
  "limit": 1
}

Expected Output: One Test Plan with its details

  1. Get five Test Plans starting from the ID 123

{
  "limit": 5,
  "startAtId": 123
}

Expected Output: Five Test Plans starting from the ID 123 with their details

  1. Get one Test Plan from the project PROJ

{
  "projectKey": "PROJ",
  "limit": 1
}

Expected Output: One Test Plan from project PROJ with its details

  1. Get Test Plans updated after a given time

{
  "updatedAfter": "2024-01-01T00:00:00Z",
  "limit": 10
}

Expected Output: Up to 10 Test Plans updated after 2024-01-01 with their details

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoSpecifies the maximum number of results to return in a single call. The default value is 10, and the maximum value that can be requested is 1000. Note that the server may enforce a lower limit than requested, depending on resource availability or other internal constraints. If this happens, the result set may be truncated. Always check the limit value in the response to confirm how many results were actually returned.
startAtIdNoZero-indexed starting position for ID-based pagination.
projectKeyNoJira project key filter
updatedAfterNoFilter only entities updated after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

Output Schema

ParametersJSON Schema
NameRequiredDescription
nextNo
limitYes
valuesNo
nextStartAtIdYes

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, and the description does not contradict them. It adds valuable behavioral context by warning that 'the server may enforce a lower limit than requested' and advising the agent to 'Always check the limit value in the response', which clarifies possible result truncation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with sections for toolset, parameters, and examples, and it front-loads the purpose. However, it is longer than necessary: the parameter definitions duplicate the schema, and the five examples with 'Expected Output' lines are repetitive and could be trimmed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, idempotent list operation with an output schema and rich annotations, the description covers all essential call details: filters, defaults, pagination, date format, and the truncation caveat. The examples further clarify usage, and the output schema removes the need to describe return values in prose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description repeats the schema's parameter docs but adds value through five concrete examples showing valid parameter combinations and expected outcomes, such as using startAtId for ID-based pagination and projectKey for project filtering.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific action and resource: 'Get details of Test Plans in Zephyr', reinforced by the 'Toolset: Test Plans' label. It is distinct from sibling tools like zephyr_get_test_cycles and zephyr_get_test_cases, though it does not explicitly state the comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides numerous examples showing parameter combinations, such as 'Get five Test Plans starting from the ID 123' and 'Get Test Plans updated after a given time', which implies usage. However, it never explicitly states when to choose this tool over alternative sibling tools for test cycles or test cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_get_test_scriptZephyr: Get Test ScriptA
Read-onlyIdempotent
Inspect

Get the Test Script (Plain Text or BDD) for a given Test Case in Zephyr

Toolset: Test Cases

Parameters:

  • testCaseKey (string) required: The key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

Examples:

  1. Get the test script for test case with key 'SA-T1'

{
  "testCaseKey": "SA-T1"
}

Expected Output: The test script with its type (plain or bdd), text content, and id

  1. Retrieve the BDD test script content for test case with key 'MM2-T15'

{
  "testCaseKey": "MM2-T15"
}

Expected Output: The test script with its type (plain or bdd), text content, and id

  1. Get the test script for test case with key 'QA-T100' to review the test instructions

{
  "testCaseKey": "QA-T100"
}

Expected Output: The test script with its type (plain or bdd), text content, and id

ParametersJSON Schema
NameRequiredDescriptionDefault
testCaseKeyYesThe key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

Output Schema

ParametersJSON Schema
NameRequiredDescription
idNoThe ID of the entity
textYes
typeYesTest scripts can be written in plain text or BDD format. The BDD type supports remote execution on a build system via API plugin. Supported Keywords for BDD: Given, When, Then, And, But. For more information about BDD and Gherkin syntax, see: https://support.smartbear.com/zephyr/docs/en/test-cases/gherkin-behavior-driven-development--bdd-.html For Plain Text scripts, we support HTML fragments. To create a step-by-step test script, you should use the POST /testcases/{testCaseKey}/teststeps endpoint.

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the read-only safety profile is covered. The description adds useful output context by specifying the script type, text content, and id, but it does not go beyond what a simple read operation would imply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is reasonably organized but contains redundancy: the parameter section duplicates the schema, and the same expected output is repeated in all three examples. The first sentence is clear, but several sentences do not earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read-only tool with an output schema and full annotations, the description is complete enough for an agent to call it correctly. It identifies the required parameter, describes the resource being fetched, and gives concrete examples.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully documents testCaseKey, including format guidance, so the baseline is 3. The description repeats this same information rather than adding new semantic detail, and the examples only illustrate valid key formats already captured in the pattern.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action: getting the Test Script for a given Test Case, and distinguishes it from related sibling tools such as zephyr_get_test_case and zephyr_create_test_script by focusing on the script artifact in Plain Text or BDD form. The resource and operation are unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description establishes a clear context for use: retrieve the Test Script for a specific Test Case, with examples showing realistic applications such as reviewing test instructions. It does not explicitly mention alternatives or when not to use this tool, so it misses the full exclusionary guidance that would justify a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_update_test_caseZephyr: Update Test CaseA
Idempotent
Inspect

Update an existing Test Case in Zephyr. This operation fetches the current test case and merges your updates with it to prevent accidental property deletion. Properties which are not included in the tool call will be left unchanged. To remove a property, set it to null explicitly. For fields that accept multiple values, such as labels, if the field is provided, it will override the previous values. For example, if labels is provided with the values ["label1", "label2"], the Test Case will now only have those two labels, and any previous labels will be removed. If you want to add a label, you would need to specify in the prompt the intention to add a label.

Toolset: Test Cases

Parameters:

  • testCaseKey (string) required: The key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+

  • id (number): The ID of the entity

  • key (string): The test case key

  • name (string)

  • project (object): ID and link relative to Zephyr project.

  • objective (string): A description of the objective.

  • precondition (string): Any conditions that need to be met.

  • estimatedTime (number): Estimated duration in milliseconds.

  • labels (array): Array of labels associated to this entity.

  • component (number): The ID of the Jira component to associate.

  • priority (object): ID and link to the priority resource.

  • status (object): ID and link to the status resource.

  • folder (number): The ID of the folder to move the test case into.

  • owner (string): Atlassian Account ID of the Jira user to set as owner.

  • customFields (record<string, any>): Multi-line text fields support HTML and should denote new lines with the <br> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.

Examples:

  1. Update the name of the test case 'SA-T10' to 'Check axial pump' and objective to 'To ensure the axial pump can be enabled'

{
  "testCaseKey": "SA-T10",
  "name": "Check axial pump",
  "objective": "To ensure the axial pump can be enabled"
}

Expected Output: The test case should be updated, but no output is expected.

  1. Update the test case 'MM2-T1' by setting labels 'Regression','Performance' and 'Automated' and changing the priority to the one with id 2.

{
  "testCaseKey": "MM2-T1",
  "priority": {
    "id": 2
  },
  "labels": [
    "Regression",
    "Performance",
    "Automated"
  ]
}

Expected Output: The test case should be updated, but no output is expected.

  1. Update test case 'SA-T5', by setting the custom field 'Build Number' to 20, 'Release Date' to '2020-01-01' and setting the Test Cases's estimated time to 3600000 milliseconds.

{
  "testCaseKey": "SA-T5",
  "estimatedTime": 3600000,
  "customFields": {
    "Build Number": 20,
    "Release Date": "2020-01-01"
  }
}

Expected Output: The test case should be updated, but no output is expected.

  1. Remove the component from test case 'SA-T20'.

{
  "testCaseKey": "SA-T20",
  "component": null
}

Expected Output: The test case should be updated, but no output is expected.

  1. Remove a specific custom field 'Pre-Condition(s)' from test case 'SA-T15' while keeping other custom fields intact

{
  "testCaseKey": "SA-T15",
  "customFields": {
    "Pre-Condition(s)": null,
    "Implemented": false
  }
}

Expected Output: The test case should be updated, but no output is expected.

  1. Remove test case from folder

{
  "testCaseKey": "SA-T15",
  "folder": null
}

Expected Output: The test case should be updated, but no output is expected.

ParametersJSON Schema
NameRequiredDescriptionDefault
idNoThe ID of the entity
keyNoThe test case key
nameNo
ownerNoAtlassian Account ID of the Jira user to set as owner.
folderNoThe ID of the folder to move the test case into.
labelsNoArray of labels associated to this entity.
statusNoID and link to the status resource.
projectNoID and link relative to Zephyr project.
priorityNoID and link to the priority resource.
componentNoThe ID of the Jira component to associate.
objectiveNoA description of the objective.
testCaseKeyYesThe key of the test case. Test case keys are of the format [A-Z]+-T[0-9]+
customFieldsNoMulti-line text fields support HTML and should denote new lines with the \<br\> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.
preconditionNoAny conditions that need to be met.
estimatedTimeNoEstimated duration in milliseconds.

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already include idempotentHint=true, but the description adds critical behavior beyond that: it fetches the current entity and merges updates, requires explicit null to delete, and states that multi-value fields like labels replace previous values. No contradiction with annotations, and this behavior is essential for correct use.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the most important merge semantics, which is good. However, it includes a lengthy parameter list that duplicates the input schema, making the description longer than necessary. The six examples are useful, but the redundant parameter section hurts conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 15 parameters, nested objects, and no output schema, the description covers everything needed for correct invocation: merge behavior, null-to-remove, label override semantics, customFields format rules, and six illustrative examples. Expected outputs are even stated in the examples, so missing output schema is not a gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 93%, so the schema already documents most parameters well. The description's parameter list largely duplicates schema descriptions, though the main narrative adds helpful context like testCaseKey format and customFields value formats. The examples add practical meaning, but the description doesn't substantially augment parameter semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Update an existing Test Case in Zephyr,' a specific verb and resource that clearly distinguishes it from sibling create/get tools. It further clarifies the update model (fetch-and-merge) so an agent knows exactly what this tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when to use the tool: to update an already-existing test case, with explicit semantics for partial updates and property removal. It does not name alternatives or state when not to use it, but the context is strong enough without them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_update_test_cycleZephyr: Update Test CycleA
Idempotent
Inspect

Update an existing Test Cycle in Zephyr. This operation fetches the current test cycle and merges your updates with it to prevent accidental property deletion. To remove a property, set it to null explicitly. The plannedStartDate and plannedEndDate fields cannot be cleared

Toolset: Test Cycles

Parameters:

  • testCycleIdOrKey (string) required: The ID or key of the test cycle.

  • id (number): The ID of the entity

  • key (string): Unique key of the test cycle

  • name (string)

  • project (object): ID and link relative to Zephyr project.

  • jiraProjectVersion (number): The ID of the Jira project version (release) to associate.

  • status (object): ID and link to the status resource.

  • folder (number): The ID of the folder to move the test cycle into.

  • description (string): Description outlining the scope.

  • plannedStartDate (string): Planned start date of the test cycle. This field cannot be blank. Setting it as null or excluding it from the request will leave the field values unchanged. ISO 8601 Format (i.e., yyyy-MM-dd'T'HH:mm:ss'Z')

  • plannedEndDate (string): The planned end date of the test cycle. This field cannot be blank. Setting it as null or excluding it from the request will leave the field values unchanged. ISO 8601 Format (i.e., yyyy-MM-dd'T'HH:mm:ss'Z')

  • owner (string): Atlassian Account ID of the Jira user to set as owner.

  • customFields (record<string, any>): Multi-line text fields support HTML and should denote new lines with the <br> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.

Examples:

  1. Update the name of the test cycle 'SA-R40' to 'Sprint 1 Regression - Updated' and set description.

{
  "testCycleIdOrKey": "SA-R40",
  "name": "Sprint 1 Regression - Updated",
  "description": "Updated regression scope for Sprint 1"
}

Expected Output: The test cycle should be updated, but no output is expected.

  1. Update planned dates for test cycle id '1' (keep everything else unchanged).

{
  "testCycleIdOrKey": "1",
  "plannedStartDate": "2018-05-19T13:15:13Z",
  "plannedEndDate": "2018-05-20T13:15:13Z"
}

Expected Output: The test cycle should be updated, but no output is expected.

  1. Change folder and status for test cycle 'SA-R40' by setting folder id and status id.

{
  "testCycleIdOrKey": "SA-R40",
  "folder": 100006,
  "status": {
    "id": 10000
  }
}

Expected Output: The test cycle should be updated, but no output is expected.

  1. Update custom fields on test cycle 'SA-R40' while keeping other custom fields intact.

{
  "testCycleIdOrKey": "SA-R40",
  "customFields": {
    "Build Number": 20,
    "Release Date": "2020-01-01"
  }
}

Expected Output: The test cycle should be updated, but no output is expected.

  1. Remove the owner from test cycle 'SA-R40'.

{
  "testCycleIdOrKey": "SA-R40",
  "owner": null
}

Expected Output: The test cycle should be updated, but no output is expected.

  1. Remove a specific custom field 'Pre-Condition(s)' from test cycle 'SA-R40' while keeping other custom fields intact.

{
  "testCycleIdOrKey": "SA-R40",
  "customFields": {
    "Pre-Condition(s)": null,
    "Implemented": false
  }
}

Expected Output: The test cycle should be updated, but no output is expected.

ParametersJSON Schema
NameRequiredDescriptionDefault
idNoThe ID of the entity
keyNoUnique key of the test cycle
nameNo
ownerNoAtlassian Account ID of the Jira user to set as owner.
folderNoThe ID of the folder to move the test cycle into.
statusNoID and link to the status resource.
projectNoID and link relative to Zephyr project.
descriptionNoDescription outlining the scope.
customFieldsNoMulti-line text fields support HTML and should denote new lines with the \<br\> tag. Dates should be in the format 'yyyy-MM-dd'. Users should have values of Jira User Account IDs.
plannedEndDateNoThe planned end date of the test cycle. This field cannot be blank. Setting it as null or excluding it from the request will leave the field values unchanged. ISO 8601 Format (i.e., yyyy-MM-dd'T'HH:mm:ss'Z')
plannedStartDateNoPlanned start date of the test cycle. This field cannot be blank. Setting it as null or excluding it from the request will leave the field values unchanged. ISO 8601 Format (i.e., yyyy-MM-dd'T'HH:mm:ss'Z')
testCycleIdOrKeyYesThe ID or key of the test cycle.
jiraProjectVersionNoThe ID of the Jira project version (release) to associate.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the provided annotations by disclosing that the operation fetches the current cycle and merges updates, that null explicitly removes properties, and that plannedStartDate and plannedEndDate cannot be cleared. It also states through examples that no output is expected, adding practical behavioral clarity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The critical behavioral guidance is front-loaded in the opening paragraph, and the examples are well-structured and purposeful. The description is somewhat long because it repeats many parameter details already present in the schema, but the organization keeps it usable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 13-parameter update tool with no output schema, the description covers the key invocation behaviors: merge semantics, null deletion, date restrictions, and common update workflows via examples. Minor gaps remain around the relationship between testCycleIdOrKey and the separate id/key fields, but the schema and annotations cover the essential contract.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 92%, so the schema already documents most parameters. The description adds value through six examples that clarify how to use nested status/folder objects, customFields, date formats, and null-based property removal, which the schema alone does not fully convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence, "Update an existing Test Cycle in Zephyr," pairs a specific verb with a concrete resource and a platform scope. This clearly distinguishes it from siblings like zephyr_create_test_cycle, zephyr_get_test_cycle, and the other zephyr update/execute operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description establishes clear context by emphasizing "existing" Test Cycle and explains merge semantics rather than replacement. It does not explicitly name alternatives or state when not to use the tool, but the wording is sufficient to route an agent away from create/get variants.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_update_test_executionZephyr: Update Test ExecutionB
Idempotent
Inspect

Update an existing Test Execution in Zephyr. This operation only updates specified fields in the payload and ignores null or undefined values.

Toolset: Test Executions

Parameters:

  • testExecutionIdOrKey (string) required: The ID or key of the test execution. Test execution keys are of the format [A-Z]+-E[0-9]+

  • statusName (string): The status name.

  • environmentName (string): Environment assigned to the test case.

  • actualEndDate (string): The actual end date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'

  • executionTime (number): Actual test execution time in milliseconds.

  • executedById (string): Atlassian Account ID of the Jira user.

  • assignedToId (string): Atlassian Account ID of the Jira user.

  • comment (string): Comment added against overall test case execution.

Examples:

  1. Update the status name to 'PASS' and the environment name to 'ENV-1' in the test execution 'SA-E40'.

{
  "testExecutionIdOrKey": "SA-E40",
  "statusName": "PASS",
  "environmentName": "ENV-1"
}

Expected Output: The test execution should be updated, but no output is expected.

  1. Update execution time and actual end date for test execution id '1' (keep everything else unchanged).

{
  "testExecutionIdOrKey": "1",
  "executionTime": "2018-05-19T13:15:13Z",
  "actualEndDate": "2018-05-20T13:15:13Z"
}

Expected Output: The test execution should be updated, but no output is expected.

  1. For test execution 'SA-E40', update the test executor and assignee to be the user with ID 10000.

{
  "testExecutionIdOrKey": "SA-E40",
  "executedById": "10000",
  "assignedToId": "10000"
}

Expected Output: The test execution should be updated, but no output is expected.

  1. In test execution 'SA-E40', add a comment saying that this execution was updated via API.

{
  "testExecutionIdOrKey": "SA-E40",
  "comment": "execution updated via API"
}

Expected Output: The test execution should be updated, but no output is expected.

  1. Remove the assigned user from test execution 'SA-E40'.

{
  "testExecutionIdOrKey": "SA-E40",
  "assignedToId": null
}

Expected Output: The test execution should be updated, but no output is expected.

ParametersJSON Schema
NameRequiredDescriptionDefault
commentNoComment added against overall test case execution.
statusNameNoThe status name.
assignedToIdNoAtlassian Account ID of the Jira user.
executedByIdNoAtlassian Account ID of the Jira user.
actualEndDateNoThe actual end date of the test cycle. Format: yyyy-MM-dd'T'HH:mm:ss'Z'
executionTimeNoActual test execution time in milliseconds.
environmentNameNoEnvironment assigned to the test case.
testExecutionIdOrKeyYesThe ID or key of the test execution. Test execution keys are of the format [A-Z]+-E[0-9]+

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description usefully discloses partial-update semantics and states that no output is expected, but its behavior is contradictory: it says null values are ignored, yet example 5 uses assignedToId: null to remove an assignee. Additionally, example 2 sends an ISO date string for executionTime even though the description and schema define it as an integer in milliseconds, muddying the expected behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well organized: a terse purpose sentence, parameter list, and labeled JSON examples. It is somewhat longer than necessary because the same 'no output expected' line is repeated across all five examples, but the structure is clear and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

All eight parameters are documented and the examples cover common operations, plus the description explicitly notes that no output is returned. The definition is not fully reliable, though, because the null-clearing behavior and executionTime example conflict with the stated semantics, so an agent cannot confidently handle removal or time updates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the description did not need to carry the parameter-documentation burden, and the parameter list mostly mirrors the schema. The examples add context, but one example contradicts the schema by placing a date-time string in executionTime, which can mislead an agent into sending an invalid payload.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific action and resource: 'Update an existing Test Execution in Zephyr,' and clarifies that it is a partial update that targets only specified fields. It is clearly distinguishable from get/create executions by the word 'existing,' though it does not explicitly name sibling tools or contrast with zephyr_update_test_execution_steps.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The examples show practical scenarios (changing status, updating dates, assigning users, commenting, clearing a field), which imply when to use the tool. However, there is no explicit 'use this instead of X' guidance or mention of when zephyr_create_test_execution or zephyr_update_test_execution_steps would be more appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zephyr_update_test_execution_stepsZephyr: Update Test Execution StepsA
Idempotent
Inspect

Update test steps for a given Test Execution in Zephyr. This operation updates the provided steps with their execution status and actual results. Only the fields included in the request will be modified.

Toolset: Test Executions

Parameters:

  • testExecutionIdOrKey (string) required: The ID or key of the test execution. Test execution keys are of the format [A-Z]+-E[0-9]+

  • steps (array)

Examples:

  1. Mark the status of all steps in the test execution 'SA-E1' as 'Pass'. Set the actual result of step 1 to 'Dashboard widgets loaded correctly' and step 2 to 'Navigation menu responded correctly to user interactions'.

{
  "testExecutionIdOrKey": "SA-E1",
  "steps": [
    {
      "statusName": "Pass",
      "actualResult": "Dashboard widgets loaded correctly"
    },
    {
      "statusName": "Pass",
      "actualResult": "Navigation menu responded correctly to user interactions"
    }
  ]
}

Expected Output: Test steps are updated successfully, but no output is expected.

  1. Update only the status of step 2 in test execution 'SA-E5' to 'Fail'. Do not modify any other fields.

{
  "testExecutionIdOrKey": "SA-E5",
  "steps": [
    {},
    {
      "statusName": "Fail"
    }
  ]
}

Expected Output: The test execution steps are updated, but no output is expected.

  1. Update only the actual results of the steps in test execution '10'. Set the actual result of step 1 to 'API returned 500 error' and step 2 actual result to 'API returned 200 success'

{
  "testExecutionIdOrKey": "10",
  "steps": [
    {
      "actualResult": "API returned 500 error"
    },
    {
      "actualResult": "API returned 200 success"
    }
  ]
}

Expected Output: Test steps are updated successfully, but no output is expected.

ParametersJSON Schema
NameRequiredDescriptionDefault
stepsNo
testExecutionIdOrKeyYesThe ID or key of the test execution. Test execution keys are of the format [A-Z]+-E[0-9]+

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as a non-read-only, idempotent, non-destructive operation. The description adds useful context: only fields included in the request are modified, and the examples demonstrate that empty step objects leave steps untouched and that no output is returned. It does not explicitly spell out positional indexing, though examples imply it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is organized with examples and expected outputs, but it is longer than needed: the no-output note is repeated three times and the Parameters list largely duplicates schema info that is already present. Each example still earns its place by showing partial update behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The examples cover common invocation patterns and the description notes there is no expected output, which is helpful given there is no output schema. However, it never states how the steps array maps to test execution steps, whether statusName values are restricted, or what happens if the array length mismatches the number of steps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With schema coverage at 50%, the description compensates by showing complete JSON examples for status-only, actual-result-only, and combined updates, including an empty object to leave a step unchanged. The Parameters section itself is thin for steps, but the examples give the agent enough structural understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the exact verb (update), resource (test steps for a given Test Execution), and the fields affected (execution status and actual results). It also clarifies partial-update semantics, which distinguishes it from the sibling update_test_execution tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description and examples make clear the tool is for updating step-level status/results, but it never explicitly contrasts it with siblings such as zephyr_update_test_execution or says when not to use it. The intended usage is inferred from the title and examples rather than stated as guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 298 tool updatesv0.40.0
    • Addedbearq_chat_with_qa_lead
    • Addedbearq_delete_test_cases
    • Addedbearq_expand_application_model
    • Addedbearq_get_task
    • Addedbearq_get_task_status
    • Addedbearq_list_environments
    • Addedbearq_run_regression_tests
    • Addedbearq_run_test_cases
    • Addedbearq_run_tests_in_functional_areas
    • Addedbearq_stop_task
    • Addedbearq_wait_for_task
    • Addedbugsnag_get_build
    • Addedbugsnag_get_current_project
    • Addedbugsnag_get_error
    • Addedbugsnag_get_event
    • Addedbugsnag_get_event_details_from_dashboard_url
    • Addedbugsnag_get_events_on_an_error
    • Addedbugsnag_get_network_endpoint_groupings
    • Addedbugsnag_get_release
    • Addedbugsnag_get_span_group
    • Addedbugsnag_get_trace
    • Addedbugsnag_list_project_errors
    • Addedbugsnag_list_project_event_filters
    • Addedbugsnag_list_projects
    • Addedbugsnag_list_releases
    • Addedbugsnag_list_span_groups
    • Addedbugsnag_list_spans
    • Addedbugsnag_list_trace_fields
    • Addedbugsnag_set_network_endpoint_groupings
    • Addedbugsnag_update_error
    • Addedcollaborator_create_remote_system_configuration
    • Addedcollaborator_create_review
    • Addedcollaborator_delete_remote_system_configuration
    • Addedcollaborator_edit_remote_system_configuration
    • Addedcollaborator_find_review_by_id
    • Addedcollaborator_get_reviews
    • Addedcollaborator_reject_review
    • Addedcollaborator_reviewservice_action
    • Addedcollaborator_test_remote_system_configuration_connection
    • Addedcollaborator_update_remote_system_configuration_webhook
    • Addedcontract-testing_add_label_to_pacticipant
    • Addedcontract-testing_admin_add_role_to_user
    • Addedcontract-testing_admin_create_role
    • Addedcontract-testing_admin_create_system_account
    • Addedcontract-testing_admin_create_team
    • Addedcontract-testing_admin_create_user
    • Addedcontract-testing_admin_delete_role
    • Addedcontract-testing_admin_delete_team
    • Addedcontract-testing_admin_delete_user
    • Addedcontract-testing_admin_get_role
    • Addedcontract-testing_admin_get_system_account_tokens
    • Addedcontract-testing_admin_get_team
    • Addedcontract-testing_admin_get_team_user
    • Addedcontract-testing_admin_get_user
    • Addedcontract-testing_admin_invite_users
    • Addedcontract-testing_admin_list_permissions
    • Addedcontract-testing_admin_list_roles
    • Addedcontract-testing_admin_list_team_users
    • Addedcontract-testing_admin_list_teams
    • Addedcontract-testing_admin_list_users
    • Addedcontract-testing_admin_patch_team_users
    • Addedcontract-testing_admin_remove_role_from_user
    • Addedcontract-testing_admin_remove_user_from_team
    • Addedcontract-testing_admin_reset_roles
    • Addedcontract-testing_admin_set_team_users
    • Addedcontract-testing_admin_set_user_roles
    • Addedcontract-testing_admin_update_role
    • Addedcontract-testing_admin_update_team
    • Addedcontract-testing_admin_update_user
    • Addedcontract-testing_can_i_deploy
    • Addedcontract-testing_check_pactflow_ai_entitlements
    • Addedcontract-testing_create_environment
    • Addedcontract-testing_create_pacticipant
    • Addedcontract-testing_create_secret
    • Addedcontract-testing_create_webhook
    • Addedcontract-testing_delete_all_integrations
    • Addedcontract-testing_delete_branch
    • Addedcontract-testing_delete_environment
    • Addedcontract-testing_delete_integration
    • Addedcontract-testing_delete_pacticipant
    • Addedcontract-testing_delete_secret
    • Addedcontract-testing_delete_webhook
    • Addedcontract-testing_execute_webhook
    • Addedcontract-testing_generate_pact_tests
    • Addedcontract-testing_get_audit_log
    • Addedcontract-testing_get_bdct_consumer_by_consumer_version
    • Addedcontract-testing_get_bdct_consumer_contract_verification_results
    • Addedcontract-testing_get_bdct_consumer_contracts
    • Addedcontract-testing_get_bdct_consumer_pact_test_results_by_consumer
    • Addedcontract-testing_get_bdct_cross-contract_verification_results
    • Addedcontract-testing_get_bdct_provider_by_consumer_version
    • Addedcontract-testing_get_bdct_provider_check_results_by_consumer
    • Addedcontract-testing_get_bdct_provider_contract
    • Addedcontract-testing_get_bdct_provider_contract_verification_results
    • Addedcontract-testing_get_bdct_x-contract_test_results_by_consumer
    • Addedcontract-testing_get_branch
    • Addedcontract-testing_get_branch_versions
    • Addedcontract-testing_get_current_user
    • Addedcontract-testing_get_currently_deployed_versions
    • Addedcontract-testing_get_currently_supported_versions
    • Addedcontract-testing_get_deployed_versions_for_version
    • Addedcontract-testing_get_environment
    • Addedcontract-testing_get_integrations_by_team
    • Addedcontract-testing_get_latest_pacticipant_version
    • Addedcontract-testing_get_metrics
    • Addedcontract-testing_get_pacticipant
    • Addedcontract-testing_get_pacticipant_label
    • Addedcontract-testing_get_pacticipant_network
    • Addedcontract-testing_get_pacticipant_version
    • Addedcontract-testing_get_pacts_for_verification
    • Addedcontract-testing_get_provider_states
    • Addedcontract-testing_get_released_versions_for_version
    • Addedcontract-testing_get_secret
    • Addedcontract-testing_get_system_preferences
    • Addedcontract-testing_get_team_metrics
    • Addedcontract-testing_get_user_preferences
    • Addedcontract-testing_get_webhook
    • Addedcontract-testing_list_api_tokens
    • Addedcontract-testing_list_branches
    • Addedcontract-testing_list_environments
    • Addedcontract-testing_list_integrations
    • Addedcontract-testing_list_labels
    • Addedcontract-testing_list_pacticipant_versions
    • Addedcontract-testing_list_pacticipants
    • Addedcontract-testing_list_pacticipants_by_label
    • Addedcontract-testing_list_secrets
    • Addedcontract-testing_list_webhooks
    • Addedcontract-testing_matrix
    • Addedcontract-testing_patch_pacticipant
    • Addedcontract-testing_publish_consumer_contracts
    • Addedcontract-testing_publish_provider_contract
    • Addedcontract-testing_record_deployment
    • Addedcontract-testing_record_release
    • Addedcontract-testing_regenerate_api_token
    • Addedcontract-testing_remove_label_from_pacticipant
    • Addedcontract-testing_review_pact_tests
    • Addedcontract-testing_test_execute_webhooks
    • Addedcontract-testing_update_environment
    • Addedcontract-testing_update_pacticipant
    • Addedcontract-testing_update_pacticipant_version
    • Addedcontract-testing_update_secret
    • Addedcontract-testing_update_webhook
    • Changedqmetry_bulk_update_test_case_execution_status1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_bulk_update_test_run_udfs1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_create_cycle1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_create_defect_or_issue1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_create_release1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Addedqmetry_create_requirement
    • Changedqmetry_create_test_case3 fields changed
      • addedInput schema / properties / skipSteps
        Added value: +{
        +  "description": "Set to true ONLY when the user explicitly says they do NOT want steps created (e.g. 'create test case without steps', 'no steps', 'skip steps'). When true, the 'steps' field must be omitted entirely. When false or absent (the default), steps MUST always be included β€” auto-generate them from context if the user did not provide them.",
        +  "type": "boolean"
        +}
      • changedInput schema / properties / steps / description
        Previous value: -"STEPS INCLUSION RULE β€” read before deciding whether to include this field:\n\nDEFAULT BEHAVIOR: OMIT 'steps' entirely from the payload. Do NOT include steps: [] (empty array).\nA test case without steps is valid and is the normal case when the user did not mention steps.\n\nINCLUDE 'steps' ONLY in these two scenarios:\n  SCENARIO 1 β€” User explicitly mentions steps in their prompt.\n    Examples: 'create test case with steps', 'step 1: open browser, step 2: click login',\n    'add these steps: ...', 'include steps', 'create with following steps'.\n    When user provides step text, parse each step into { orderId, description, inputData?, expectedOutcome? }.\n\n  SCENARIO 2 β€” Any field in 'stepSystemFields' OR 'stepFields' from Fetch UDF Layout has isMandatory=true.\n    stepSystemFields = built-in step fields (description, expectedOutcome, etc.).\n    stepFields = step-level UDF fields (custom fields configured per project).\n    If EITHER array has isMandatory=true on any entry, the backend REQUIRES at least 1 step.\n    In this case you MUST include at least 1 step even if the user did not mention steps.\n    Also fill mandatory step UDF fields from stepFields in step.UDF β€” use stepDefaultValues if defaults exist, else placeholder.\n    Ask the user for step content OR create a placeholder step with description='Step 1'.\n\nNEVER include 'steps' in any other scenario β€” omitting it keeps the payload clean and avoids BE errors.\nNEVER send steps: [] (empty array) β€” either omit the field or send at least 1 valid step object.\n\nStep object fields:\n  orderId (required): sequential integer starting at 1\n  description (required): step action text\n  inputData (optional): test data for this step\n  expectedOutcome (optional): what should happen after this step\n  UDF (optional): step-level custom fields\n  tcStepID (omit on create β€” only used when updating existing steps)"New value: +"STEPS RULE β€” include this field unless the user explicitly says NOT to create steps.\n\nNEVER send steps: [] (empty array) β€” always send at least 1 valid step object.\nOmit this field entirely (and set skipSteps: true) ONLY when the user explicitly asks to skip steps.\n\nHOW TO POPULATE:\n  - If user explicitly provides steps: parse each step into { orderId, description, inputData?, expectedOutcome? }.\n  - If user does NOT provide steps (and did not say to skip them): auto-generate meaningful steps based on the test case name, description, and context.\n    Use your knowledge to infer 2-5 logical, realistic steps for the feature or flow being tested.\n    Example: name='Login Test Case' β†’ [{orderId:1, description:'Navigate to login page'}, {orderId:2, description:'Enter credentials'}, {orderId:3, description:'Submit and verify success'}]\n  - If user explicitly said NOT to create steps: omit this field and set skipSteps: true.\n\nSTEP DEFAULT VALUES:\n  After building the steps array, check 'stepDefaultValues' from Fetch UDF Layout.\n  stepDefaultValues shape: { fieldName: defaultValue }\n  For each step: for each key in stepDefaultValues, if the user did NOT explicitly provide a value for that field β†’ add it to step.UDF with the default value.\n  Auto-apply silently β€” do NOT ask the user.\n\nStep object fields:\n  orderId (required): sequential integer starting at 1\n  description (required): step action text\n  inputData (optional): test data for this step\n  expectedOutcome (optional): what should happen after this step\n  UDF (optional): step-level custom fields β€” auto-fill defaults from stepDefaultValues\n  tcStepID (omit on create β€” only used when updating existing steps)"
      • addedInput schema / properties / steps / minItems
        Added value: +1
    • Changedqmetry_execute_quality_gate_report1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_export_html_report4 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / fileName / description
        Previous value: -"Name for the exported report file (without extension)."New value: +"Name for the exported report file (without extension). Letters, digits, '.', '_' and '-' only - no path separators or '..' sequences."
      • addedInput schema / properties / fileName / maxLength
        Added value: +255
      • addedInput schema / properties / fileName / pattern
        Added value: +"^[a-zA-Z0-9._-]+$"
    • Changedqmetry_fetch_automation_status1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_builds1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_cascade_child_values1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_defects_or_issues2 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / viewId / description
        Previous value: -"ViewId for issues - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.IS.viewId automatically. Manual viewId only needed if you want to override the automatic resolution."New value: +"ViewId for issues - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.IS.viewId automatically."
    • Changedqmetry_fetch_executions_by_test_suite2 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / viewId / description
        Previous value: -"ViewId for test execution - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TE.viewId automatically. Manual viewId only needed if you want to override the automatic resolution."New value: +"ViewId for test execution - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TE.viewId automatically."
    • Changedqmetry_fetch_issue_details1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_issue_executions1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_issues_linked_to_test_case1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_linked_issues_of_test_case_run1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_platforms1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_qmetry_list_projects1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_quality_gate_configuration1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_requirement_details1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_requirements2 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / viewId / description
        Previous value: -"ViewId for requirements - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.RQ.viewId automatically. Manual viewId only needed if you want to override the automatic resolution."New value: +"ViewId for requirements - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.RQ.viewId automatically."
    • Changedqmetry_fetch_requirements_linked_to_test_case1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_case_details1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_case_executions1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_case_runs_by_test_suite_run2 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / viewId / description
        Previous value: -"ViewId for test execution - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TE.viewId automatically. Manual viewId only needed if you want to override the automatic resolution."New value: +"ViewId for test execution - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TE.viewId automatically."
    • Changedqmetry_fetch_test_case_steps1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_case_steps_with_udf2 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / viewId / description
        Previous value: -"ViewId for test cases - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TC.viewId automatically. Manual viewId only needed if you want to override the automatic resolution."New value: +"ViewId for test cases - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TC.viewId automatically."
    • Changedqmetry_fetch_test_case_version_details1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_cases2 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / viewId / description
        Previous value: -"ViewId for test cases - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TC.viewId automatically. Manual viewId only needed if you want to override the automatic resolution."New value: +"ViewId for test cases - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TC.viewId automatically."
    • Changedqmetry_fetch_test_cases_linked_to_requirement1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_cases_linked_to_test_suite1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_run_udf_metadata1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_run_udf_values1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_suite_details1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_fetch_test_suites2 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / viewId / description
        Previous value: -"ViewId for test suites - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TS.viewId automatically. Manual viewId only needed if you want to override the automatic resolution."New value: +"ViewId for test suites - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TS.viewId automatically."
    • Changedqmetry_fetch_test_suites_for_test_case2 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / viewId / description
        Previous value: -"ViewId for test suite folders - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TSFS.viewId automatically. Manual viewId only needed if you want to override the automatic resolution."New value: +"ViewId for test suite folders - SYSTEM AUTOMATICALLY RESOLVES THIS. Leave empty unless you have a specific viewId. System will fetch project info using the projectKey and extract latestViews.TSFS.viewId automatically."
    • Changedqmetry_fetch_udf_layout3 fields changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
      • changedInput schema / properties / entityType / description
        Previous value: -"Entity type to fetch UDF field definitions for. 'TC' = Test Case (also returns step UDFs in stepFields), 'TS' = Test Suite, 'IS' = Issue."New value: +"Entity type to fetch UDF field definitions for. 'TC' = Test Case (also returns step UDFs in stepFields), 'TS' = Test Suite, 'IS' = Issue, 'RQ' = Requirement."
      • changedInput schema / properties / entityType / enum
        Previous value: -[
        -  "TC",
        -  "TS",
        -  "IS"
        -]New value: +[
        +  "TC",
        +  "TS",
        +  "IS",
        +  "RQ"
        +]
    • Changedqmetry_link_issues_to_testcase_run1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Changedqmetry_link_platforms_to_test_suite1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Addedqmetry_link_test_case_to_issues
    • Changedqmetry_update_cycle1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Addedqmetry_update_requirement
    • Changedqmetry_update_test_case1 field changed
      • removedInput schema / properties / baseUrl
        Removed value: -{
        -  "description": "The base URL for the QMetry instance (must be a valid URL)",
        -  "format": "uri",
        -  "type": "string"
        -}
    • Addedqtm4j_create_test_case
    • Addedqtm4j_create_test_cycle
    • Addedqtm4j_get_automation_history
    • Addedqtm4j_get_linked_bugs_of_test_case_execution
    • Addedqtm4j_get_linked_bugs_of_test_step_execution
    • Addedqtm4j_get_linked_requirements
    • Addedqtm4j_get_linked_requirements_for_test_cycle
    • Addedqtm4j_get_linked_test_cases_for_requirement
    • Addedqtm4j_get_projects
    • Addedqtm4j_get_test_steps
    • Addedqtm4j_link_bugs_to_test_case_execution
    • Addedqtm4j_link_bugs_to_test_step_execution
    • Addedqtm4j_link_requirements_to_test_case
    • Addedqtm4j_link_requirements_to_test_cycle
    • Addedqtm4j_link_test_cases_to_requirement
    • Addedqtm4j_link_test_cases_to_test_cycle
    • Addedqtm4j_search_linked_test_cases_in_test_cycle
    • Addedqtm4j_search_test_cases
    • Addedqtm4j_search_test_cycles
    • Addedqtm4j_set_project_context
    • Addedqtm4j_start_new_execution
    • Addedqtm4j_unlink_requirements_from_test_case
    • Addedqtm4j_unlink_requirements_from_test_cycle
    • Addedqtm4j_unlink_test_cases_from_requirement
    • Addedqtm4j_unlink_test_cases_from_test_cycle
    • Addedqtm4j_update_test_case
    • Addedqtm4j_update_test_case_execution
    • Addedqtm4j_update_test_cycle
    • Addedqtm4j_update_test_step_execution
    • Addedqtm4j_upload_automation_result
    • Addedreflect_add_prompt_step
    • Addedreflect_add_segment
    • Addedreflect_cancel_suite_execution
    • Addedreflect_connect_to_session
    • Addedreflect_create_segment
    • Addedreflect_create_test
    • Addedreflect_delete_previous_step
    • Addedreflect_execute_suite
    • Addedreflect_get_screenshot
    • Addedreflect_get_suite_execution_status
    • Addedreflect_get_test_detail
    • Addedreflect_get_test_status
    • Addedreflect_list_segments
    • Addedreflect_list_suite_executions
    • Addedreflect_list_suites
    • Addedreflect_list_tests
    • Addedreflect_run_test
    • Addedswagger_create_api_from_prompt
    • Addedswagger_create_documentation_page
    • Addedswagger_create_or_update_api
    • Addedswagger_create_portal
    • Addedswagger_create_portal_product
    • Addedswagger_create_table_of_contents
    • Addedswagger_delete_portal_product
    • Addedswagger_delete_table_of_contents
    • Addedswagger_get_api_definition
    • Addedswagger_get_document
    • Addedswagger_get_portal
    • Addedswagger_get_portal_product
    • Addedswagger_list_organizations
    • Addedswagger_list_portal_product_sections
    • Addedswagger_list_portal_products
    • Addedswagger_list_portals
    • Addedswagger_list_table_of_contents
    • Addedswagger_patch_api
    • Addedswagger_publish_portal_product
    • Addedswagger_resolve_organization_portal
    • Addedswagger_scan_api_standardization
    • Addedswagger_scan_api_standardization_from_registry
    • Addedswagger_search_apis_and_domains
    • Addedswagger_standardize_api
    • Addedswagger_update_document
    • Addedswagger_update_portal
    • Addedswagger_update_portal_product
    • Addedzephyr_create_folder
    • Addedzephyr_create_test_case
    • Addedzephyr_create_test_case_issue_link
    • Addedzephyr_create_test_case_steps
    • Addedzephyr_create_test_case_web_link
    • Addedzephyr_create_test_cycle
    • Addedzephyr_create_test_cycle_issue_link
    • Addedzephyr_create_test_cycle_web_link
    • Addedzephyr_create_test_execution
    • Addedzephyr_create_test_execution_issue_link
    • Addedzephyr_create_test_script
    • Addedzephyr_get_environments
    • Addedzephyr_get_folders
    • Addedzephyr_get_issue_link_test_cases
    • Addedzephyr_get_priorities
    • Addedzephyr_get_project
    • Addedzephyr_get_projects
    • Addedzephyr_get_statuses
    • Addedzephyr_get_test_case
    • Addedzephyr_get_test_case_links
    • Addedzephyr_get_test_case_steps
    • Addedzephyr_get_test_cases
    • Addedzephyr_get_test_cycle
    • Addedzephyr_get_test_cycle_links
    • Addedzephyr_get_test_cycles
    • Addedzephyr_get_test_cycles_linked_to_a_jira_issue
    • Addedzephyr_get_test_execution
    • Addedzephyr_get_test_execution_links
    • Addedzephyr_get_test_execution_steps
    • Addedzephyr_get_test_executions
    • Addedzephyr_get_test_executions_linked_to_a_jira_issue
    • Addedzephyr_get_test_plans
    • Addedzephyr_get_test_script
    • Addedzephyr_update_test_case
    • Addedzephyr_update_test_cycle
    • Addedzephyr_update_test_execution
    • Addedzephyr_update_test_execution_steps
  2. 2 tool updatesv0.39.0
    • Changedqmetry_bulk_update_test_case_execution_status2 fields changed
      • removedInput schema / properties / dropID / anyOf
        Removed value: -[
        -  {
        -    "type": "number"
        -  },
        -  {
        -    "type": "string"
        -  }
        -]
      • addedInput schema / properties / dropID / type
        Added value: +[
        +  "number",
        +  "string"
        +]
    • Changedqmetry_link_issues_to_testcase_run2 fields changed
      • removedInput schema / properties / issueIds / items / anyOf
        Removed value: -[
        -  {
        -    "type": "string"
        -  },
        -  {
        -    "type": "number"
        -  }
        -]
      • addedInput schema / properties / issueIds / items / type
        Added value: +[
        +  "string",
        +  "number"
        +]
  3. 52 tool updatesv0.37.0
    • Changedqmetry_bulk_update_test_case_execution_status1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_bulk_update_test_run_udfs1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_create_cycle1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_create_defect_or_issue4 fields changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
      • addedInput schema / properties / environment
        Added value: +{
        +  "description": "Environment where the issue was found (e.g. 'Chrome', 'Firefox', 'Production'). Free-text string β€” no ID lookup needed.",
        +  "type": "string"
        +}
      • addedInput schema / properties / issueState
        Added value: +{
        +  "description": "Issue status ID. Optional by default β€” QMetry allows admins to make this mandatory at the project level. Get valid IDs from project info β†’ customListObjs.issueState[index].id. Common values: Open, Reopened, Resolved, Closed.",
        +  "type": "number"
        +}
      • addedInput schema / properties / udfFields
        Added value: +{
        +  "additionalProperties": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "items": {
        +          "type": "number"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "properties": {
        +          "child": {
        +            "type": "number"
        +          },
        +          "parent": {
        +            "type": "number"
        +          }
        +        },
        +        "required": [
        +          "parent"
        +        ],
        +        "type": "object"
        +      },
        +      {
        +        "type": "null"
        +      }
        +    ]
        +  },
        +  "description": "Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
    • Changedqmetry_create_release1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_create_test_case9 fields changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
      • addedInput schema / properties / estimatedTime / description
        Added value: +"Estimated execution time in SECONDS (e.g. 3600 = 1 hour, 36000 = 10 hours). NOT minutes."
      • addedInput schema / properties / releaseCycleMapping / description
        Added value: +"Release/cycle mapping. Set associateRelCyc=true when providing this. version field defaults to 1 if not specified."
      • addedInput schema / properties / releaseCycleMapping / items / properties / version / default
        Added value: +1
      • addedInput schema / properties / steps / description
        Added value: +"STEPS INCLUSION RULE β€” read before deciding whether to include this field:\n\nDEFAULT BEHAVIOR: OMIT 'steps' entirely from the payload. Do NOT include steps: [] (empty array).\nA test case without steps is valid and is the normal case when the user did not mention steps.\n\nINCLUDE 'steps' ONLY in these two scenarios:\n  SCENARIO 1 β€” User explicitly mentions steps in their prompt.\n    Examples: 'create test case with steps', 'step 1: open browser, step 2: click login',\n    'add these steps: ...', 'include steps', 'create with following steps'.\n    When user provides step text, parse each step into { orderId, description, inputData?, expectedOutcome? }.\n\n  SCENARIO 2 β€” Any field in 'stepSystemFields' OR 'stepFields' from Fetch UDF Layout has isMandatory=true.\n    stepSystemFields = built-in step fields (description, expectedOutcome, etc.).\n    stepFields = step-level UDF fields (custom fields configured per project).\n    If EITHER array has isMandatory=true on any entry, the backend REQUIRES at least 1 step.\n    In this case you MUST include at least 1 step even if the user did not mention steps.\n    Also fill mandatory step UDF fields from stepFields in step.UDF β€” use stepDefaultValues if defaults exist, else placeholder.\n    Ask the user for step content OR create a placeholder step with description='Step 1'.\n\nNEVER include 'steps' in any other scenario β€” omitting it keeps the payload clean and avoids BE errors.\nNEVER send steps: [] (empty array) β€” either omit the field or send at least 1 valid step object.\n\nStep object fields:\n  orderId (required): sequential integer starting at 1\n  description (required): step action text\n  inputData (optional): test data for this step\n  expectedOutcome (optional): what should happen after this step\n  UDF (optional): step-level custom fields\n  tcStepID (omit on create β€” only used when updating existing steps)"
      • addedInput schema / properties / steps / items / properties / UDF / additionalProperties / anyOf
        Added value: +[
        +  {
        +    "type": "string"
        +  },
        +  {
        +    "type": "number"
        +  },
        +  {
        +    "items": {
        +      "type": "number"
        +    },
        +    "type": "array"
        +  },
        +  {
        +    "properties": {
        +      "child": {
        +        "type": "number"
        +      },
        +      "parent": {
        +        "type": "number"
        +      }
        +    },
        +    "required": [
        +      "parent"
        +    ],
        +    "type": "object"
        +  },
        +  {
        +    "properties": {
        +      "ADD": {
        +        "items": {},
        +        "type": "array"
        +      },
        +      "REMOVE": {
        +        "items": {},
        +        "type": "array"
        +      }
        +    },
        +    "required": [
        +      "ADD",
        +      "REMOVE"
        +    ],
        +    "type": "object"
        +  },
        +  {
        +    "items": {
        +      "additionalProperties": {},
        +      "propertyNames": {
        +        "type": "string"
        +      },
        +      "type": "object"
        +    },
        +    "type": "array"
        +  },
        +  {
        +    "additionalProperties": {},
        +    "propertyNames": {
        +      "type": "string"
        +    },
        +    "type": "object"
        +  }
        +]
      • removedInput schema / properties / steps / items / properties / UDF / additionalProperties / type
        Removed value: -"string"
      • addedInput schema / properties / steps / items / properties / UDF / description
        Added value: +"Step-level UDF values. Keys = UDF field names. Values depend on field type: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string (DD-MM-YYYY), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs. CASCADINGLIST requires THREE sibling keys in this UDF object β€” e.g. for field 'proj': proj: {parent: parentId, child: childId}, proj_value: [{FieldID:'proj', FieldValue:[{id:parentId, value:'parentLabel', child:{id:childId, value:'childLabel'}}], type:'CASCADINGLIST'}], proj_selectedList: {id: udfmID, name:'proj', type:'CASCADINGLIST'}. Get parentLabel/childLabel from Fetch Cascade Child Values. Get udfmID from Fetch UDF Layout stepFields[].projectUserFieldID."
      • addedInput schema / properties / udfFields
        Added value: +{
        +  "additionalProperties": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "items": {
        +          "type": "number"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "properties": {
        +          "child": {
        +            "type": "number"
        +          },
        +          "parent": {
        +            "type": "number"
        +          }
        +        },
        +        "required": [
        +          "parent"
        +        ],
        +        "type": "object"
        +      },
        +      {
        +        "type": "null"
        +      }
        +    ]
        +  },
        +  "description": "Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
    • Changedqmetry_create_test_suite6 fields changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
      • addedInput schema / properties / releaseCycleMapping / description
        Added value: +"Release/cycle association for the test suite. CRITICAL SHAPE DIFFERENCE vs Test Case mapping:   Test Suite uses: { releaseId: number, buildID: number }   Test Case uses:  { release: number, cycle: number[], version: number } β€” DO NOT use TC shape here. Set associateRelCyc=true when providing this array. Example: [{ releaseId: 92112, buildID: 130831 }]"
      • addedInput schema / properties / releaseCycleMapping / items / properties / buildID / description
        Added value: +"Cycle/Build numeric ID. CRITICAL: key is 'buildID' (capital D) β€” do NOT use 'cycleId', 'cycle', 'buildId', or 'cycleID'. Get from project info β†’ projects[<index>].releases[<index>].builds[<index>].buildID. NEVER use the TC mapping key 'cycle' here β€” test suite mapping uses 'buildID'."
      • addedInput schema / properties / releaseCycleMapping / items / properties / releaseId / description
        Added value: +"Release numeric ID. CRITICAL: key is 'releaseId' (lowercase d) β€” do NOT use 'releaseID', 'release', or 'releaseKey'. Get from project info β†’ projects[<index>].releases[<index>].releaseID. NEVER use the TC mapping key 'release' here β€” test suite mapping uses 'releaseId'."
      • changedInput schema / properties / releaseCycleMapping / items / required
        Previous value: -[
        -  "buildID",
        -  "releaseId"
        -]New value: +[
        +  "releaseId",
        +  "buildID"
        +]
      • addedInput schema / properties / udfFields
        Added value: +{
        +  "additionalProperties": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "items": {
        +          "type": "number"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "properties": {
        +          "child": {
        +            "type": "number"
        +          },
        +          "parent": {
        +            "type": "number"
        +          }
        +        },
        +        "required": [
        +          "parent"
        +        ],
        +        "type": "object"
        +      },
        +      {
        +        "type": "null"
        +      }
        +    ]
        +  },
        +  "description": "Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
    • Changedqmetry_execute_quality_gate_report1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_export_html_report1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_automation_status1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_builds1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_cascade_child_values1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_defects_or_issues1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_executions_by_test_suite1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Addedqmetry_fetch_issue_details
    • Changedqmetry_fetch_issue_executions1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_issues_linked_to_test_case1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_linked_issues_of_test_case_run1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_platforms1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_qmetry_list_projects1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_qmetry_project_info1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_quality_gate_configuration1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_releases_and_cycles1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_requirement_details1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_requirements1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_requirements_linked_to_test_case1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_case_details1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_case_executions1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_case_runs_by_test_suite_run1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_case_steps1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Addedqmetry_fetch_test_case_steps_with_udf
    • Changedqmetry_fetch_test_case_version_details1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_cases1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_cases_linked_to_requirement1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_cases_linked_to_test_suite1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_run_udf_metadata1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_run_udf_values1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Addedqmetry_fetch_test_suite_details
    • Changedqmetry_fetch_test_suites1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_fetch_test_suites_for_test_case1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Addedqmetry_fetch_udf_layout
    • Changedqmetry_import_automation_test_results1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_link_issues_to_testcase_run1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_link_platforms_to_test_suite1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_link_requirements_to_testcase1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_link_test_cases_to_test_suite1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_requirements_linked_test_cases_to_test_suite1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_set_qmetry_project_info1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_update_cycle1 field changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
    • Changedqmetry_update_issue3 fields changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
      • addedInput schema / properties / UDF
        Added value: +{
        +  "additionalProperties": {
        +    "properties": {
        +      "fieldID": {
        +        "description": "Numeric field ID (projectUserFieldID) from 'Fetch UDF Layout' with pageName='DETAIL'. Required for update UDF wrapper.",
        +        "exclusiveMinimum": 0,
        +        "maximum": 9007199254740991,
        +        "type": "integer"
        +      },
        +      "value": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "number"
        +          },
        +          {
        +            "items": {
        +              "maximum": 9007199254740991,
        +              "minimum": -9007199254740991,
        +              "type": "integer"
        +            },
        +            "type": "array"
        +          },
        +          {
        +            "properties": {
        +              "child": {
        +                "maximum": 9007199254740991,
        +                "minimum": -9007199254740991,
        +                "type": "integer"
        +              },
        +              "parent": {
        +                "maximum": 9007199254740991,
        +                "minimum": -9007199254740991,
        +                "type": "integer"
        +              }
        +            },
        +            "required": [
        +              "parent"
        +            ],
        +            "type": "object"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "UDF value. Type depends on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: 'MM-DD-YYYY' or 'DD-MM-YYYY', LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs, CASCADINGLIST: { parent, child }."
        +      }
        +    },
        +    "required": [
        +      "fieldID",
        +      "value"
        +    ],
        +    "type": "object"
        +  },
        +  "description": "UDF wrapper required for update. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
      • addedInput schema / properties / udfFields
        Added value: +{
        +  "additionalProperties": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "items": {
        +          "type": "number"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "properties": {
        +          "child": {
        +            "type": "number"
        +          },
        +          "parent": {
        +            "type": "number"
        +          }
        +        },
        +        "required": [
        +          "parent"
        +        ],
        +        "type": "object"
        +      },
        +      {
        +        "type": "null"
        +      }
        +    ]
        +  },
        +  "description": "Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
    • Changedqmetry_update_test_case8 fields changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
      • addedInput schema / properties / UDF
        Added value: +{
        +  "additionalProperties": {
        +    "properties": {
        +      "fieldID": {
        +        "description": "Numeric field ID (projectUserFieldID) from 'Fetch UDF Layout' with pageName='DETAIL'. Required for update UDF wrapper.",
        +        "exclusiveMinimum": 0,
        +        "maximum": 9007199254740991,
        +        "type": "integer"
        +      },
        +      "value": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "number"
        +          },
        +          {
        +            "items": {
        +              "maximum": 9007199254740991,
        +              "minimum": -9007199254740991,
        +              "type": "integer"
        +            },
        +            "type": "array"
        +          },
        +          {
        +            "properties": {
        +              "child": {
        +                "maximum": 9007199254740991,
        +                "minimum": -9007199254740991,
        +                "type": "integer"
        +              },
        +              "parent": {
        +                "maximum": 9007199254740991,
        +                "minimum": -9007199254740991,
        +                "type": "integer"
        +              }
        +            },
        +            "required": [
        +              "parent"
        +            ],
        +            "type": "object"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "UDF value. Type depends on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: 'MM-DD-YYYY' or 'DD-MM-YYYY', LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs, CASCADINGLIST: { parent, child }."
        +      }
        +    },
        +    "required": [
        +      "fieldID",
        +      "value"
        +    ],
        +    "type": "object"
        +  },
        +  "description": "UDF wrapper required for update operations. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value. Also set matching flat key in udfFields for the LOOKUPLIST Alias display. Example: { custom_text: { fieldID: 1001, value: 'new value' } }",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
      • addedInput schema / properties / removeSteps / items / properties / UDF / additionalProperties / anyOf
        Added value: +[
        +  {
        +    "type": "string"
        +  },
        +  {
        +    "type": "number"
        +  },
        +  {
        +    "items": {
        +      "type": "number"
        +    },
        +    "type": "array"
        +  },
        +  {
        +    "properties": {
        +      "child": {
        +        "type": "number"
        +      },
        +      "parent": {
        +        "type": "number"
        +      }
        +    },
        +    "required": [
        +      "parent"
        +    ],
        +    "type": "object"
        +  },
        +  {
        +    "properties": {
        +      "ADD": {
        +        "items": {},
        +        "type": "array"
        +      },
        +      "REMOVE": {
        +        "items": {},
        +        "type": "array"
        +      }
        +    },
        +    "required": [
        +      "ADD",
        +      "REMOVE"
        +    ],
        +    "type": "object"
        +  },
        +  {
        +    "items": {
        +      "additionalProperties": {},
        +      "propertyNames": {
        +        "type": "string"
        +      },
        +      "type": "object"
        +    },
        +    "type": "array"
        +  },
        +  {
        +    "additionalProperties": {},
        +    "propertyNames": {
        +      "type": "string"
        +    },
        +    "type": "object"
        +  }
        +]
      • removedInput schema / properties / removeSteps / items / properties / UDF / additionalProperties / type
        Removed value: -"string"
      • addedInput schema / properties / steps / items / properties / UDF / additionalProperties / anyOf
        Added value: +[
        +  {
        +    "type": "string"
        +  },
        +  {
        +    "type": "number"
        +  },
        +  {
        +    "items": {
        +      "type": "number"
        +    },
        +    "type": "array"
        +  },
        +  {
        +    "properties": {
        +      "child": {
        +        "type": "number"
        +      },
        +      "parent": {
        +        "type": "number"
        +      }
        +    },
        +    "required": [
        +      "parent"
        +    ],
        +    "type": "object"
        +  },
        +  {
        +    "properties": {
        +      "ADD": {
        +        "items": {},
        +        "type": "array"
        +      },
        +      "REMOVE": {
        +        "items": {},
        +        "type": "array"
        +      }
        +    },
        +    "required": [
        +      "ADD",
        +      "REMOVE"
        +    ],
        +    "type": "object"
        +  },
        +  {
        +    "items": {
        +      "additionalProperties": {},
        +      "propertyNames": {
        +        "type": "string"
        +      },
        +      "type": "object"
        +    },
        +    "type": "array"
        +  },
        +  {
        +    "additionalProperties": {},
        +    "propertyNames": {
        +      "type": "string"
        +    },
        +    "type": "object"
        +  }
        +]
      • removedInput schema / properties / steps / items / properties / UDF / additionalProperties / type
        Removed value: -"string"
      • addedInput schema / properties / steps / items / properties / UDF / description
        Added value: +"Step-level UDF values. Keys = UDF field names. Values depend on field type: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string (DD-MM-YYYY), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs. CASCADINGLIST requires THREE sibling keys in this UDF object β€” e.g. for field 'proj': proj: {parent: parentId, child: childId}, proj_value: [{FieldID:'proj', FieldValue:[{id:parentId, value:'parentLabel', child:{id:childId, value:'childLabel'}}], type:'CASCADINGLIST'}], proj_selectedList: {id: udfmID, name:'proj', type:'CASCADINGLIST'}. Get parentLabel/childLabel from Fetch Cascade Child Values. Get udfmID from Fetch UDF Layout stepFields[].projectUserFieldID."
      • addedInput schema / properties / udfFields
        Added value: +{
        +  "additionalProperties": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "items": {
        +          "type": "number"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "properties": {
        +          "child": {
        +            "type": "number"
        +          },
        +          "parent": {
        +            "type": "number"
        +          }
        +        },
        +        "required": [
        +          "parent"
        +        ],
        +        "type": "object"
        +      },
        +      {
        +        "type": "null"
        +      }
        +    ]
        +  },
        +  "description": "Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
    • Changedqmetry_update_test_suite3 fields changed
      • changedInput schema / $schema
        Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
      • addedInput schema / properties / UDF
        Added value: +{
        +  "additionalProperties": {
        +    "properties": {
        +      "fieldID": {
        +        "description": "Numeric field ID (projectUserFieldID) from 'Fetch UDF Layout' with pageName='DETAIL'. Required for update UDF wrapper.",
        +        "exclusiveMinimum": 0,
        +        "maximum": 9007199254740991,
        +        "type": "integer"
        +      },
        +      "value": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "number"
        +          },
        +          {
        +            "items": {
        +              "maximum": 9007199254740991,
        +              "minimum": -9007199254740991,
        +              "type": "integer"
        +            },
        +            "type": "array"
        +          },
        +          {
        +            "properties": {
        +              "child": {
        +                "maximum": 9007199254740991,
        +                "minimum": -9007199254740991,
        +                "type": "integer"
        +              },
        +              "parent": {
        +                "maximum": 9007199254740991,
        +                "minimum": -9007199254740991,
        +                "type": "integer"
        +              }
        +            },
        +            "required": [
        +              "parent"
        +            ],
        +            "type": "object"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "UDF value. Type depends on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: 'MM-DD-YYYY' or 'DD-MM-YYYY', LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs, CASCADINGLIST: { parent, child }."
        +      }
        +    },
        +    "required": [
        +      "fieldID",
        +      "value"
        +    ],
        +    "type": "object"
        +  },
        +  "description": "UDF wrapper required for update. Keys = UDF field names. Each value must include fieldID (from 'Fetch UDF Layout' with pageName='DETAIL') and value.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
      • addedInput schema / properties / udfFields
        Added value: +{
        +  "additionalProperties": {
        +    "anyOf": [
        +      {
        +        "type": "string"
        +      },
        +      {
        +        "type": "number"
        +      },
        +      {
        +        "items": {
        +          "type": "number"
        +        },
        +        "type": "array"
        +      },
        +      {
        +        "properties": {
        +          "child": {
        +            "type": "number"
        +          },
        +          "parent": {
        +            "type": "number"
        +          }
        +        },
        +        "required": [
        +          "parent"
        +        ],
        +        "type": "object"
        +      },
        +      {
        +        "type": "null"
        +      }
        +    ]
        +  },
        +  "description": "Flat UDF field values. Keys = UDF field names from 'Fetch UDF Layout'. Values depend on fieldTypeName: STRING/LARGETEXT: string, NUMBER: number, DATETIMEPICKER: date string ('14-08-2026'), LOOKUPLIST: numeric ID, MULTILOOKUPLIST: array of IDs [101, 102], CASCADINGLIST: { parent: 101, child: 102 }. Call 'Fetch UDF Layout' with entityType + pageName='ADD' to discover available fields and list option IDs.",
        +  "propertyNames": {
        +    "type": "string"
        +  },
        +  "type": "object"
        +}
  4. 254 tool updatesv0.34.0
    • Removedbearq_chat_with_qa_lead
    • Removedbearq_delete_test_cases
    • Removedbearq_expand_application_model
    • Removedbearq_get_task
    • Removedbearq_get_task_status
    • Removedbearq_list_environments
    • Removedbearq_run_regression_tests
    • Removedbearq_run_test_cases
    • Removedbearq_run_tests_in_functional_areas
    • Removedbearq_stop_task
    • Removedbearq_wait_for_task
    • Removedbugsnag_get_build
    • Removedbugsnag_get_current_project
    • Removedbugsnag_get_error
    • Removedbugsnag_get_event
    • Removedbugsnag_get_event_details_from_dashboard_url
    • Removedbugsnag_get_events_on_an_error
    • Removedbugsnag_get_network_endpoint_groupings
    • Removedbugsnag_get_release
    • Removedbugsnag_get_span_group
    • Removedbugsnag_get_trace
    • Removedbugsnag_list_project_errors
    • Removedbugsnag_list_project_event_filters
    • Removedbugsnag_list_projects
    • Removedbugsnag_list_releases
    • Removedbugsnag_list_span_groups
    • Removedbugsnag_list_spans
    • Removedbugsnag_list_trace_fields
    • Removedbugsnag_set_network_endpoint_groupings
    • Removedbugsnag_update_error
    • Removedcollaborator_create_remote_system_configuration
    • Removedcollaborator_create_review
    • Removedcollaborator_delete_remote_system_configuration
    • Removedcollaborator_edit_remote_system_configuration
    • Removedcollaborator_find_review_by_id
    • Removedcollaborator_get_reviews
    • Removedcollaborator_reject_review
    • Removedcollaborator_reviewservice_action
    • Removedcollaborator_test_remote_system_configuration_connection
    • Removedcollaborator_update_remote_system_configuration_webhook
    • Removedcontract-testing_add_label_to_pacticipant
    • Removedcontract-testing_admin_add_role_to_user
    • Removedcontract-testing_admin_create_role
    • Removedcontract-testing_admin_create_system_account
    • Removedcontract-testing_admin_create_team
    • Removedcontract-testing_admin_create_user
    • Removedcontract-testing_admin_delete_role
    • Removedcontract-testing_admin_delete_team
    • Removedcontract-testing_admin_delete_user
    • Removedcontract-testing_admin_get_role
    • Removedcontract-testing_admin_get_system_account_tokens
    • Removedcontract-testing_admin_get_team
    • Removedcontract-testing_admin_get_team_user
    • Removedcontract-testing_admin_get_user
    • Removedcontract-testing_admin_invite_users
    • Removedcontract-testing_admin_list_permissions
    • Removedcontract-testing_admin_list_roles
    • Removedcontract-testing_admin_list_team_users
    • Removedcontract-testing_admin_list_teams
    • Removedcontract-testing_admin_list_users
    • Removedcontract-testing_admin_patch_team_users
    • Removedcontract-testing_admin_remove_role_from_user
    • Removedcontract-testing_admin_remove_user_from_team
    • Removedcontract-testing_admin_reset_roles
    • Removedcontract-testing_admin_set_team_users
    • Removedcontract-testing_admin_set_user_roles
    • Removedcontract-testing_admin_update_role
    • Removedcontract-testing_admin_update_team
    • Removedcontract-testing_admin_update_user
    • Removedcontract-testing_can_i_deploy
    • Removedcontract-testing_check_pactflow_ai_entitlements
    • Removedcontract-testing_create_environment
    • Removedcontract-testing_create_pacticipant
    • Removedcontract-testing_create_secret
    • Removedcontract-testing_create_webhook
    • Removedcontract-testing_delete_all_integrations
    • Removedcontract-testing_delete_branch
    • Removedcontract-testing_delete_environment
    • Removedcontract-testing_delete_integration
    • Removedcontract-testing_delete_pacticipant
    • Removedcontract-testing_delete_secret
    • Removedcontract-testing_delete_webhook
    • Removedcontract-testing_execute_webhook
    • Removedcontract-testing_generate_pact_tests
    • Removedcontract-testing_get_audit_log
    • Removedcontract-testing_get_bdct_consumer_by_consumer_version
    • Removedcontract-testing_get_bdct_consumer_contract_verification_results
    • Removedcontract-testing_get_bdct_consumer_contracts
    • Removedcontract-testing_get_bdct_consumer_pact_test_results_by_consumer
    • Removedcontract-testing_get_bdct_cross-contract_verification_results
    • Removedcontract-testing_get_bdct_provider_by_consumer_version
    • Removedcontract-testing_get_bdct_provider_check_results_by_consumer
    • Removedcontract-testing_get_bdct_provider_contract
    • Removedcontract-testing_get_bdct_provider_contract_verification_results
    • Removedcontract-testing_get_bdct_x-contract_test_results_by_consumer
    • Removedcontract-testing_get_branch
    • Removedcontract-testing_get_branch_versions
    • Removedcontract-testing_get_current_user
    • Removedcontract-testing_get_currently_deployed_versions
    • Removedcontract-testing_get_currently_supported_versions
    • Removedcontract-testing_get_deployed_versions_for_version
    • Removedcontract-testing_get_environment
    • Removedcontract-testing_get_integrations_by_team
    • Removedcontract-testing_get_latest_pacticipant_version
    • Removedcontract-testing_get_metrics
    • Removedcontract-testing_get_pacticipant
    • Removedcontract-testing_get_pacticipant_label
    • Removedcontract-testing_get_pacticipant_network
    • Removedcontract-testing_get_pacticipant_version
    • Removedcontract-testing_get_pacts_for_verification
    • Removedcontract-testing_get_provider_states
    • Removedcontract-testing_get_released_versions_for_version
    • Removedcontract-testing_get_secret
    • Removedcontract-testing_get_system_preferences
    • Removedcontract-testing_get_team_metrics
    • Removedcontract-testing_get_user_preferences
    • Removedcontract-testing_get_webhook
    • Removedcontract-testing_list_api_tokens
    • Removedcontract-testing_list_branches
    • Removedcontract-testing_list_environments
    • Removedcontract-testing_list_integrations
    • Removedcontract-testing_list_labels
    • Removedcontract-testing_list_pacticipant_versions
    • Removedcontract-testing_list_pacticipants
    • Removedcontract-testing_list_pacticipants_by_label
    • Removedcontract-testing_list_secrets
    • Removedcontract-testing_list_webhooks
    • Removedcontract-testing_matrix
    • Removedcontract-testing_patch_pacticipant
    • Removedcontract-testing_publish_consumer_contracts
    • Removedcontract-testing_publish_provider_contract
    • Removedcontract-testing_record_deployment
    • Removedcontract-testing_record_release
    • Removedcontract-testing_regenerate_api_token
    • Removedcontract-testing_remove_label_from_pacticipant
    • Removedcontract-testing_review_pact_tests
    • Removedcontract-testing_test_execute_webhooks
    • Removedcontract-testing_update_environment
    • Removedcontract-testing_update_pacticipant
    • Removedcontract-testing_update_pacticipant_version
    • Removedcontract-testing_update_secret
    • Removedcontract-testing_update_webhook
    • Addedqmetry_execute_quality_gate_report
    • Addedqmetry_export_html_report
    • Addedqmetry_fetch_quality_gate_configuration
    • Removedqtm4j_create_test_case
    • Removedqtm4j_create_test_cycle
    • Removedqtm4j_get_automation_history
    • Removedqtm4j_get_linked_bugs_of_test_case_execution
    • Removedqtm4j_get_linked_bugs_of_test_step_execution
    • Removedqtm4j_get_linked_requirements
    • Removedqtm4j_get_linked_requirements_for_test_cycle
    • Removedqtm4j_get_linked_test_cases_for_requirement
    • Removedqtm4j_get_projects
    • Removedqtm4j_get_test_steps
    • Removedqtm4j_link_bugs_to_test_case_execution
    • Removedqtm4j_link_bugs_to_test_step_execution
    • Removedqtm4j_link_requirements_to_test_case
    • Removedqtm4j_link_requirements_to_test_cycle
    • Removedqtm4j_link_test_cases_to_requirement
    • Removedqtm4j_link_test_cases_to_test_cycle
    • Removedqtm4j_search_linked_test_cases_in_test_cycle
    • Removedqtm4j_search_test_cases
    • Removedqtm4j_search_test_cycles
    • Removedqtm4j_set_project_context
    • Removedqtm4j_start_new_execution
    • Removedqtm4j_unlink_requirements_from_test_case
    • Removedqtm4j_unlink_requirements_from_test_cycle
    • Removedqtm4j_unlink_test_cases_from_requirement
    • Removedqtm4j_unlink_test_cases_from_test_cycle
    • Removedqtm4j_update_test_case
    • Removedqtm4j_update_test_case_execution
    • Removedqtm4j_update_test_cycle
    • Removedqtm4j_update_test_step_execution
    • Removedqtm4j_upload_automation_result
    • Removedreflect_add_prompt_step
    • Removedreflect_add_segment
    • Removedreflect_cancel_suite_execution
    • Removedreflect_connect_to_session
    • Removedreflect_create_segment
    • Removedreflect_create_test
    • Removedreflect_delete_previous_step
    • Removedreflect_execute_suite
    • Removedreflect_get_screenshot
    • Removedreflect_get_suite_execution_status
    • Removedreflect_get_test_detail
    • Removedreflect_get_test_status
    • Removedreflect_list_segments
    • Removedreflect_list_suite_executions
    • Removedreflect_list_suites
    • Removedreflect_list_tests
    • Removedreflect_run_test
    • Removedswagger_create_api_from_prompt
    • Removedswagger_create_documentation_page
    • Removedswagger_create_or_update_api
    • Removedswagger_create_portal
    • Removedswagger_create_portal_product
    • Removedswagger_create_table_of_contents
    • Removedswagger_delete_portal_product
    • Removedswagger_delete_table_of_contents
    • Removedswagger_get_api_definition
    • Removedswagger_get_document
    • Removedswagger_get_portal
    • Removedswagger_get_portal_product
    • Removedswagger_list_organizations
    • Removedswagger_list_portal_product_sections
    • Removedswagger_list_portal_products
    • Removedswagger_list_portals
    • Removedswagger_list_table_of_contents
    • Removedswagger_publish_portal_product
    • Removedswagger_resolve_organization_portal
    • Removedswagger_scan_api_standardization
    • Removedswagger_scan_api_standardization_from_registry
    • Removedswagger_search_apis_and_domains
    • Removedswagger_standardize_api
    • Removedswagger_update_document
    • Removedswagger_update_portal
    • Removedswagger_update_portal_product
    • Removedzephyr_create_folder
    • Removedzephyr_create_test_case
    • Removedzephyr_create_test_case_issue_link
    • Removedzephyr_create_test_case_steps
    • Removedzephyr_create_test_case_web_link
    • Removedzephyr_create_test_cycle
    • Removedzephyr_create_test_cycle_issue_link
    • Removedzephyr_create_test_cycle_web_link
    • Removedzephyr_create_test_execution
    • Removedzephyr_create_test_execution_issue_link
    • Removedzephyr_create_test_script
    • Removedzephyr_get_environments
    • Removedzephyr_get_issue_link_test_cases
    • Removedzephyr_get_priorities
    • Removedzephyr_get_project
    • Removedzephyr_get_projects
    • Removedzephyr_get_statuses
    • Removedzephyr_get_test_case
    • Removedzephyr_get_test_case_links
    • Removedzephyr_get_test_case_steps
    • Removedzephyr_get_test_cases
    • Removedzephyr_get_test_cycle
    • Removedzephyr_get_test_cycle_links
    • Removedzephyr_get_test_cycles
    • Removedzephyr_get_test_cycles_linked_to_a_jira_issue
    • Removedzephyr_get_test_execution
    • Removedzephyr_get_test_execution_links
    • Removedzephyr_get_test_execution_steps
    • Removedzephyr_get_test_executions
    • Removedzephyr_get_test_executions_linked_to_a_jira_issue
    • Removedzephyr_get_test_plans
    • Removedzephyr_get_test_script
    • Removedzephyr_update_test_case
    • Removedzephyr_update_test_cycle
    • Removedzephyr_update_test_execution
    • Removedzephyr_update_test_execution_steps
  5. 1 tool updatev0.30.1
    • Changedswagger_create_portal1 field changed
      • changedInput schema / properties / subdomain / description
        Previous value: -"The portal subdomain - used in the portal URL (e.g., 'myportal' for myportal.example.com). Must be unique, lowercase, 3-20 characters, alphanumeric with hyphens"New value: +"The portal subdomain - used in the portal URL (e.g., 'myportal' for myportal.example.com). Must be unique, lowercase, 3-20 characters, alphanumeric with hyphens. Recommended: slugified organization name plus a random 3-character suffix (e.g., 'acmecorp-k7p') to avoid collisions"
  6. 18 tool updates
    • Addedbearq_delete_test_cases
    • Removedbearq_refine_all_draft_tests
    • Removedbearq_refine_test_cases
    • Removedbearq_refine_tests_in_functional_areas
    • Addedqtm4j_get_linked_bugs_of_test_case_execution
    • Addedqtm4j_get_linked_bugs_of_test_step_execution
    • Changedqtm4j_get_linked_test_cases_for_requirement3 fields changed
      • addedOutput schema / properties / data / items / properties / summary / anyOf
        Added value: +[
        +  {
        +    "type": "string"
        +  },
        +  {
        +    "type": "null"
        +  }
        +]
      • removedOutput schema / properties / data / items / properties / summary / type
        Removed value: -"string"
      • removedOutput schema / properties / data / items / required
        Removed value: -[
        -  "id",
        -  "key",
        -  "summary"
        -]
    • Addedqtm4j_link_bugs_to_test_case_execution
    • Addedqtm4j_link_bugs_to_test_step_execution
    • Changedqtm4j_search_linked_test_cases_in_test_cycle2 fields changed
      • changedOutput schema / properties / data / items / properties / executionResult / anyOf
        Previous value: -[
        -  {
        -    "type": "string"
        -  },
        -  {
        -    "type": "null"
        -  }
        -]New value: +[
        +  {
        +    "anyOf": [
        +      {
        +        "additionalProperties": false,
        +        "properties": {
        +          "color": {
        +            "type": "string"
        +          },
        +          "id": {
        +            "type": "number"
        +          },
        +          "name": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      {
        +        "type": "string"
        +      }
        +    ]
        +  },
        +  {
        +    "type": "null"
        +  }
        +]
      • changedOutput schema / properties / data / items / properties / executionResult / description
        Previous value: -"Execution result (e.g., 'Pass', 'Fail', 'Blocked')."New value: +"Execution result object (e.g., { id, name, color }) or string label."
    • Changedqtm4j_search_test_cases3 fields changed
      • addedOutput schema / properties / data / items / properties / summary / anyOf
        Added value: +[
        +  {
        +    "type": "string"
        +  },
        +  {
        +    "type": "null"
        +  }
        +]
      • removedOutput schema / properties / data / items / properties / summary / type
        Removed value: -"string"
      • removedOutput schema / properties / data / items / required
        Removed value: -[
        -  "id",
        -  "key",
        -  "summary"
        -]
    • Addedqtm4j_start_new_execution
    • Addedqtm4j_update_test_case_execution
    • Addedqtm4j_update_test_step_execution
    • Changedswagger_list_organizations3 fields changed
      • addedOutput schema / properties / items / items / properties / description
        Added value: +{
        +  "type": "string"
        +}
      • removedOutput schema / properties / items / items / properties / email
        Removed value: -{
        -  "type": "string"
        -}
      • addedOutput schema / properties / items / items / properties / url
        Added value: +{
        +  "type": "string"
        +}
    • Addedzephyr_get_test_plans
    • Changedzephyr_update_test_case3 fields changed
      • changedInput schema / properties / component / description
        Previous value: -"ID and link to the Jira component resource."New value: +"The ID of the Jira component to associate."
      • changedInput schema / properties / folder / description
        Previous value: -"The ID of the folder, to remove folder set it's value to null"New value: +"The ID of the folder to move the test case into."
      • changedInput schema / properties / owner / description
        Previous value: -"Atlassian Account ID of the Jira user."New value: +"Atlassian Account ID of the Jira user to set as owner."
    • Changedzephyr_update_test_cycle3 fields changed
      • changedInput schema / properties / folder / description
        Previous value: -"ID and link to the folder resource."New value: +"The ID of the folder to move the test cycle into."
      • changedInput schema / properties / jiraProjectVersion / description
        Previous value: -"ID and Link to fetch information about Jira Project version. Relates to 'Version' or 'Releases' in Jira projects."New value: +"The ID of the Jira project version (release) to associate."
      • changedInput schema / properties / owner / description
        Previous value: -"Atlassian Account ID of the Jira user."New value: +"Atlassian Account ID of the Jira user to set as owner."
  7. 39 tool updatesv0.30.0
    • Addedbearq_list_environments
    • Changedbearq_run_regression_tests1 field changed
      • addedInput schema / properties / environment
        Added value: +{
        +  "description": "Target environment name to run tests against. Omit to use the workspace default.",
        +  "minLength": 1,
        +  "type": "string"
        +}
    • Changedbearq_run_test_cases1 field changed
      • addedInput schema / properties / environment
        Added value: +{
        +  "description": "Target environment name to run tests against. Omit to use the workspace default.",
        +  "minLength": 1,
        +  "type": "string"
        +}
    • Changedbearq_run_tests_in_functional_areas1 field changed
      • addedInput schema / properties / environment
        Added value: +{
        +  "description": "Target environment name to run tests against. Omit to use the workspace default.",
        +  "minLength": 1,
        +  "type": "string"
        +}
    • Addedreflect_create_segment
    • Addedreflect_create_test
    • Changedswagger_create_api_from_prompt1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "apiName": {
        +      "type": "string"
        +    },
        +    "operation": {
        +      "enum": [
        +        "create",
        +        "update"
        +      ],
        +      "type": "string"
        +    },
        +    "owner": {
        +      "type": "string"
        +    },
        +    "specType": {
        +      "type": "string"
        +    },
        +    "url": {
        +      "type": "string"
        +    },
        +    "version": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_create_documentation_page1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "draftUrl": {
        +      "type": "string"
        +    },
        +    "productId": {
        +      "type": "string"
        +    },
        +    "sectionId": {
        +      "type": "string"
        +    },
        +    "sectionSlug": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_create_or_update_api1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "apiName": {
        +      "type": "string"
        +    },
        +    "operation": {
        +      "enum": [
        +        "create",
        +        "update"
        +      ],
        +      "type": "string"
        +    },
        +    "owner": {
        +      "type": "string"
        +    },
        +    "url": {
        +      "type": "string"
        +    },
        +    "version": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_create_portal1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "credentialsEnabled": {
        +      "type": "boolean"
        +    },
        +    "customDomain": {
        +      "type": "string"
        +    },
        +    "id": {
        +      "type": "string"
        +    },
        +    "name": {
        +      "type": "string"
        +    },
        +    "offline": {
        +      "type": "boolean"
        +    },
        +    "openapiRenderer": {
        +      "type": "string"
        +    },
        +    "routing": {
        +      "type": "string"
        +    },
        +    "subdomain": {
        +      "type": "string"
        +    },
        +    "swaggerHubOrganizationId": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_create_portal_product1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "description": {
        +      "type": "string"
        +    },
        +    "hidden": {
        +      "type": "boolean"
        +    },
        +    "id": {
        +      "type": "string"
        +    },
        +    "name": {
        +      "type": "string"
        +    },
        +    "portalId": {
        +      "type": "string"
        +    },
        +    "public": {
        +      "type": "boolean"
        +    },
        +    "slug": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_create_table_of_contents6 fields changed
      • addedInput schema / properties / content / allOf
        Added value: +[
        +  {
        +    "if": {
        +      "properties": {
        +        "type": {
        +          "const": "apiUrl"
        +        }
        +      }
        +    },
        +    "message": "URL is required when content type is 'apiUrl'",
        +    "then": {
        +      "properties": {
        +        "documentId": false
        +      },
        +      "required": [
        +        "url"
        +      ]
        +    }
        +  },
        +  {
        +    "if": {
        +      "properties": {
        +        "type": {
        +          "enum": [
        +            "html",
        +            "markdown"
        +          ]
        +        }
        +      }
        +    },
        +    "message": "url must not be set when content type is 'html' or 'markdown'",
        +    "then": {
        +      "properties": {
        +        "apiSpec": false,
        +        "url": false
        +      }
        +    }
        +  }
        +]
      • changedInput schema / properties / content / properties / apiSpec / description
        Previous value: -"API specification format for API URL content"New value: +"API specification format for API URL content - only used when type is 'apiUrl'"
      • changedInput schema / properties / content / properties / documentId / description
        Previous value: -"Document ID for HTML or Markdown content"New value: +"Document ID for HTML or Markdown content - only used when type is 'html' or 'markdown'"
      • changedInput schema / properties / content / properties / source / description
        Previous value: -"Source of the document content - 'internal' allows to edit content in both UI and API, 'external' enables editing only via API."New value: +"Source of the document content - 'internal' allows to edit content in both UI and API, 'external' enables editing only via API. Not used when type is 'apiUrl'."
      • changedInput schema / properties / content / properties / url / description
        Previous value: -"URL for API reference content (required when type is 'apiUrl')"New value: +"URL for API reference content - required when type is 'apiUrl', must end with '/swagger.json' or '/swagger.yaml'. Not used for 'html'/'markdown'."
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "documentId": {
        +      "type": "string"
        +    },
        +    "id": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_delete_portal_product1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "message": {
        +      "type": "string"
        +    }
        +  },
        +  "required": [
        +    "message"
        +  ],
        +  "type": "object"
        +}
    • Changedswagger_delete_table_of_contents1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "success": {
        +      "type": "boolean"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_get_api_definition1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "definition": {
        +      "type": "string"
        +    }
        +  },
        +  "required": [
        +    "definition"
        +  ],
        +  "type": "object"
        +}
    • Changedswagger_get_document1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "content": {
        +      "type": "string"
        +    },
        +    "createdAt": {
        +      "type": "string"
        +    },
        +    "id": {
        +      "type": "string"
        +    },
        +    "source": {
        +      "enum": [
        +        "internal",
        +        "external"
        +      ],
        +      "type": "string"
        +    },
        +    "type": {
        +      "enum": [
        +        "html",
        +        "markdown"
        +      ],
        +      "type": "string"
        +    },
        +    "updatedAt": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_get_portal1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "credentialsEnabled": {
        +      "type": "boolean"
        +    },
        +    "customDomain": {
        +      "type": "string"
        +    },
        +    "id": {
        +      "type": "string"
        +    },
        +    "name": {
        +      "type": "string"
        +    },
        +    "offline": {
        +      "type": "boolean"
        +    },
        +    "openapiRenderer": {
        +      "type": "string"
        +    },
        +    "routing": {
        +      "type": "string"
        +    },
        +    "subdomain": {
        +      "type": "string"
        +    },
        +    "swaggerHubOrganizationId": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_get_portal_product1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "description": {
        +      "type": "string"
        +    },
        +    "hidden": {
        +      "type": "boolean"
        +    },
        +    "id": {
        +      "type": "string"
        +    },
        +    "name": {
        +      "type": "string"
        +    },
        +    "portalId": {
        +      "type": "string"
        +    },
        +    "public": {
        +      "type": "boolean"
        +    },
        +    "slug": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_list_organizations1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "items": {
        +      "items": {
        +        "additionalProperties": {},
        +        "properties": {
        +          "email": {
        +            "type": "string"
        +          },
        +          "id": {
        +            "type": "string"
        +          },
        +          "memberCount": {
        +            "type": "number"
        +          },
        +          "name": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    },
        +    "page": {
        +      "type": "number"
        +    },
        +    "pageSize": {
        +      "type": "number"
        +    },
        +    "totalCount": {
        +      "type": "number"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_list_portal_product_sections1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "items": {
        +      "items": {
        +        "additionalProperties": {},
        +        "properties": {
        +          "id": {
        +            "type": "string"
        +          },
        +          "order": {
        +            "type": "number"
        +          },
        +          "productId": {
        +            "type": "string"
        +          },
        +          "slug": {
        +            "type": "string"
        +          },
        +          "title": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    },
        +    "page": {
        +      "additionalProperties": false,
        +      "properties": {
        +        "number": {
        +          "type": "number"
        +        },
        +        "size": {
        +          "type": "number"
        +        },
        +        "totalElements": {
        +          "type": "number"
        +        },
        +        "totalPages": {
        +          "type": "number"
        +        }
        +      },
        +      "type": "object"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_list_portal_products1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "items": {
        +      "items": {
        +        "additionalProperties": {},
        +        "properties": {
        +          "description": {
        +            "type": "string"
        +          },
        +          "hidden": {
        +            "type": "boolean"
        +          },
        +          "id": {
        +            "type": "string"
        +          },
        +          "name": {
        +            "type": "string"
        +          },
        +          "portalId": {
        +            "type": "string"
        +          },
        +          "public": {
        +            "type": "boolean"
        +          },
        +          "slug": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    },
        +    "page": {
        +      "additionalProperties": false,
        +      "properties": {
        +        "number": {
        +          "type": "number"
        +        },
        +        "size": {
        +          "type": "number"
        +        },
        +        "totalElements": {
        +          "type": "number"
        +        },
        +        "totalPages": {
        +          "type": "number"
        +        }
        +      },
        +      "type": "object"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_list_portals1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "items": {
        +      "items": {
        +        "additionalProperties": {},
        +        "properties": {
        +          "credentialsEnabled": {
        +            "type": "boolean"
        +          },
        +          "customDomain": {
        +            "type": "string"
        +          },
        +          "id": {
        +            "type": "string"
        +          },
        +          "name": {
        +            "type": "string"
        +          },
        +          "offline": {
        +            "type": "boolean"
        +          },
        +          "openapiRenderer": {
        +            "type": "string"
        +          },
        +          "routing": {
        +            "type": "string"
        +          },
        +          "subdomain": {
        +            "type": "string"
        +          },
        +          "swaggerHubOrganizationId": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    },
        +    "page": {
        +      "additionalProperties": false,
        +      "properties": {
        +        "number": {
        +          "type": "number"
        +        },
        +        "size": {
        +          "type": "number"
        +        },
        +        "totalElements": {
        +          "type": "number"
        +        },
        +        "totalPages": {
        +          "type": "number"
        +        }
        +      },
        +      "type": "object"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_list_table_of_contents1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "items": {
        +      "items": {
        +        "additionalProperties": {},
        +        "properties": {
        +          "id": {
        +            "type": "string"
        +          },
        +          "order": {
        +            "type": "number"
        +          },
        +          "parentId": {
        +            "anyOf": [
        +              {
        +                "type": "string"
        +              },
        +              {
        +                "type": "null"
        +              }
        +            ]
        +          },
        +          "slug": {
        +            "type": "string"
        +          },
        +          "title": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    }
        +  },
        +  "required": [
        +    "items"
        +  ],
        +  "type": "object"
        +}
    • Changedswagger_publish_portal_product1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "liveUrl": {
        +      "anyOf": [
        +        {
        +          "type": "string"
        +        },
        +        {
        +          "type": "null"
        +        }
        +      ]
        +    },
        +    "preview": {
        +      "type": "boolean"
        +    },
        +    "previewUrl": {
        +      "anyOf": [
        +        {
        +          "type": "string"
        +        },
        +        {
        +          "type": "null"
        +        }
        +      ]
        +    },
        +    "success": {
        +      "type": "boolean"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_resolve_organization_portal1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "customDomain": {
        +      "type": "string"
        +    },
        +    "organizationId": {
        +      "type": "string"
        +    },
        +    "portalCreated": {
        +      "type": "boolean"
        +    },
        +    "portalId": {
        +      "type": "string"
        +    },
        +    "products": {
        +      "items": {
        +        "additionalProperties": false,
        +        "properties": {
        +          "productId": {
        +            "type": "string"
        +          },
        +          "productName": {
        +            "type": "string"
        +          },
        +          "productSlug": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    },
        +    "subdomain": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_scan_api_standardization1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "count": {
        +      "type": "number"
        +    },
        +    "countsBySeverity": {
        +      "additionalProperties": {
        +        "type": "number"
        +      },
        +      "propertyNames": {
        +        "type": "string"
        +      },
        +      "type": "object"
        +    },
        +    "validation": {
        +      "items": {
        +        "additionalProperties": false,
        +        "properties": {
        +          "description": {
        +            "type": "string"
        +          },
        +          "line": {
        +            "type": "number"
        +          },
        +          "severity": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_scan_api_standardization_from_registry1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "count": {
        +      "type": "number"
        +    },
        +    "countsBySeverity": {
        +      "additionalProperties": {
        +        "type": "number"
        +      },
        +      "propertyNames": {
        +        "type": "string"
        +      },
        +      "type": "object"
        +    },
        +    "url": {
        +      "type": "string"
        +    },
        +    "validation": {
        +      "items": {
        +        "additionalProperties": false,
        +        "properties": {
        +          "description": {
        +            "type": "string"
        +          },
        +          "line": {
        +            "type": "number"
        +          },
        +          "severity": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_search_apis_and_domains1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "items": {
        +      "items": {
        +        "additionalProperties": {},
        +        "properties": {
        +          "description": {
        +            "type": "string"
        +          },
        +          "name": {
        +            "type": "string"
        +          },
        +          "owner": {
        +            "type": "string"
        +          },
        +          "specification": {
        +            "type": "string"
        +          },
        +          "url": {
        +            "type": "string"
        +          },
        +          "version": {
        +            "type": "string"
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "type": "array"
        +    }
        +  },
        +  "required": [
        +    "items"
        +  ],
        +  "type": "object"
        +}
    • Changedswagger_standardize_api1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "errorsFound": {
        +      "type": "number"
        +    },
        +    "fixedDefinition": {
        +      "type": "string"
        +    },
        +    "message": {
        +      "type": "string"
        +    },
        +    "savedVersion": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_update_document1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": false,
        +  "properties": {
        +    "content": {
        +      "type": "string"
        +    },
        +    "createdAt": {
        +      "type": "string"
        +    },
        +    "id": {
        +      "type": "string"
        +    },
        +    "source": {
        +      "enum": [
        +        "internal",
        +        "external"
        +      ],
        +      "type": "string"
        +    },
        +    "type": {
        +      "enum": [
        +        "html",
        +        "markdown"
        +      ],
        +      "type": "string"
        +    },
        +    "updatedAt": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_update_portal1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "credentialsEnabled": {
        +      "type": "boolean"
        +    },
        +    "customDomain": {
        +      "type": "string"
        +    },
        +    "id": {
        +      "type": "string"
        +    },
        +    "name": {
        +      "type": "string"
        +    },
        +    "offline": {
        +      "type": "boolean"
        +    },
        +    "openapiRenderer": {
        +      "type": "string"
        +    },
        +    "routing": {
        +      "type": "string"
        +    },
        +    "subdomain": {
        +      "type": "string"
        +    },
        +    "swaggerHubOrganizationId": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedswagger_update_portal_product1 field changed
      • changedOutput schema / (root)
        Previous value: -nullNew value: +{
        +  "$schema": "http://json-schema.org/draft-07/schema#",
        +  "additionalProperties": {},
        +  "properties": {
        +    "description": {
        +      "type": "string"
        +    },
        +    "hidden": {
        +      "type": "boolean"
        +    },
        +    "id": {
        +      "type": "string"
        +    },
        +    "name": {
        +      "type": "string"
        +    },
        +    "portalId": {
        +      "type": "string"
        +    },
        +    "public": {
        +      "type": "boolean"
        +    },
        +    "slug": {
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
    • Changedzephyr_create_test_case1 field changed
      • changedInput schema / properties / componentId / minimum
        Previous value: -0New value: +1
    • Changedzephyr_get_test_case_links1 field changed
      • addedOutput schema / description
        Added value: +"A list of links for this test case."
    • Changedzephyr_get_test_case_steps1 field changed
      • addedOutput schema / description
        Added value: +"Response body when retrieving test steps"
    • Changedzephyr_get_test_cycle1 field changed
      • addedOutput schema / description
        Added value: +"Details of a test cycle"
    • Changedzephyr_get_test_cycle_links1 field changed
      • addedOutput schema / description
        Added value: +"Represents all links that a Test Cycle has."
    • Changedzephyr_get_test_execution_steps1 field changed
      • addedOutput schema / description
        Added value: +"Response body when retrieving test steps for a test execution"
    • Changedzephyr_get_test_script1 field changed
      • addedOutput schema / description
        Added value: +"Response body when retrieving test scripts"
  8. 2 tool updatesv0.28.0
    • Changedswagger_create_documentation_page2 fields changed
      • addedInput schema / properties / pageSlug
        Added value: +{
        +  "description": "URL slug for the documentation page. 3-255 characters, lowercase, alphanumeric with hyphens, underscores, or dots (e.g. 'my-page'). If not provided, the slug is generated from the page title.",
        +  "type": "string"
        +}
      • changedInput schema / properties / pageTitle / description
        Previous value: -"Title of the documentation page - will be displayed in navigation (3-255 characters, used to generate the page slug)"New value: +"Title of the documentation page - will be displayed in navigation (3-255 characters)"
    • Changedzephyr_get_test_executions1 field changed
      • addedInput schema / properties / updatedAfter
        Added value: +{
        +  "description": "Filter only entities updated after the given time. Format: yyyy-MM-dd'T'HH:mm:ss'Z'",
        +  "format": "date-time",
        +  "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$",
        +  "type": "string"
        +}
  9. 2 tool updatesv0.27.2
    • Changedzephyr_get_test_case_steps1 field changed
      • addedOutput schema / properties / values / items / properties / id
        Added value: +{
        +  "anyOf": [
        +    {
        +      "minimum": 1,
        +      "type": "number"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "The ID of the test step."
        +}
    • Changedzephyr_get_test_execution_steps1 field changed
      • addedOutput schema / properties / values / items / properties / id
        Added value: +{
        +  "anyOf": [
        +    {
        +      "type": "number"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "The id of the test step (test script result)"
        +}
  10. 287 tool updatesv0.27.1
    • First observedbearq_chat_with_qa_lead
    • First observedbearq_expand_application_model
    • First observedbearq_get_task
    • First observedbearq_get_task_status
    • First observedbearq_refine_all_draft_tests
    • First observedbearq_refine_test_cases
    • First observedbearq_refine_tests_in_functional_areas
    • First observedbearq_run_regression_tests
    • First observedbearq_run_test_cases
    • First observedbearq_run_tests_in_functional_areas
    • First observedbearq_stop_task
    • First observedbearq_wait_for_task
    • First observedbugsnag_get_build
    • First observedbugsnag_get_current_project
    • First observedbugsnag_get_error
    • First observedbugsnag_get_event
    • First observedbugsnag_get_event_details_from_dashboard_url
    • First observedbugsnag_get_events_on_an_error
    • First observedbugsnag_get_network_endpoint_groupings
    • First observedbugsnag_get_release
    • First observedbugsnag_get_span_group
    • First observedbugsnag_get_trace
    • First observedbugsnag_list_project_errors
    • First observedbugsnag_list_project_event_filters
    • First observedbugsnag_list_projects
    • First observedbugsnag_list_releases
    • First observedbugsnag_list_span_groups
    • First observedbugsnag_list_spans
    • First observedbugsnag_list_trace_fields
    • First observedbugsnag_set_network_endpoint_groupings
    • First observedbugsnag_update_error
    • First observedcollaborator_create_remote_system_configuration
    • First observedcollaborator_create_review
    • First observedcollaborator_delete_remote_system_configuration
    • First observedcollaborator_edit_remote_system_configuration
    • First observedcollaborator_find_review_by_id
    • First observedcollaborator_get_reviews
    • First observedcollaborator_reject_review
    • First observedcollaborator_reviewservice_action
    • First observedcollaborator_test_remote_system_configuration_connection
    • First observedcollaborator_update_remote_system_configuration_webhook
    • First observedcontract-testing_add_label_to_pacticipant
    • First observedcontract-testing_admin_add_role_to_user
    • First observedcontract-testing_admin_create_role
    • First observedcontract-testing_admin_create_system_account
    • First observedcontract-testing_admin_create_team
    • First observedcontract-testing_admin_create_user
    • First observedcontract-testing_admin_delete_role
    • First observedcontract-testing_admin_delete_team
    • First observedcontract-testing_admin_delete_user
    • First observedcontract-testing_admin_get_role
    • First observedcontract-testing_admin_get_system_account_tokens
    • First observedcontract-testing_admin_get_team
    • First observedcontract-testing_admin_get_team_user
    • First observedcontract-testing_admin_get_user
    • First observedcontract-testing_admin_invite_users
    • First observedcontract-testing_admin_list_permissions
    • First observedcontract-testing_admin_list_roles
    • First observedcontract-testing_admin_list_team_users
    • First observedcontract-testing_admin_list_teams
    • First observedcontract-testing_admin_list_users
    • First observedcontract-testing_admin_patch_team_users
    • First observedcontract-testing_admin_remove_role_from_user
    • First observedcontract-testing_admin_remove_user_from_team
    • First observedcontract-testing_admin_reset_roles
    • First observedcontract-testing_admin_set_team_users
    • First observedcontract-testing_admin_set_user_roles
    • First observedcontract-testing_admin_update_role
    • First observedcontract-testing_admin_update_team
    • First observedcontract-testing_admin_update_user
    • First observedcontract-testing_can_i_deploy
    • First observedcontract-testing_check_pactflow_ai_entitlements
    • First observedcontract-testing_create_environment
    • First observedcontract-testing_create_pacticipant
    • First observedcontract-testing_create_secret
    • First observedcontract-testing_create_webhook
    • First observedcontract-testing_delete_all_integrations
    • First observedcontract-testing_delete_branch
    • First observedcontract-testing_delete_environment
    • First observedcontract-testing_delete_integration
    • First observedcontract-testing_delete_pacticipant
    • First observedcontract-testing_delete_secret
    • First observedcontract-testing_delete_webhook
    • First observedcontract-testing_execute_webhook
    • First observedcontract-testing_generate_pact_tests
    • First observedcontract-testing_get_audit_log
    • First observedcontract-testing_get_bdct_consumer_by_consumer_version
    • First observedcontract-testing_get_bdct_consumer_contract_verification_results
    • First observedcontract-testing_get_bdct_consumer_contracts
    • First observedcontract-testing_get_bdct_consumer_pact_test_results_by_consumer
    • First observedcontract-testing_get_bdct_cross-contract_verification_results
    • First observedcontract-testing_get_bdct_provider_by_consumer_version
    • First observedcontract-testing_get_bdct_provider_check_results_by_consumer
    • First observedcontract-testing_get_bdct_provider_contract
    • First observedcontract-testing_get_bdct_provider_contract_verification_results
    • First observedcontract-testing_get_bdct_x-contract_test_results_by_consumer
    • First observedcontract-testing_get_branch
    • First observedcontract-testing_get_branch_versions
    • First observedcontract-testing_get_current_user
    • First observedcontract-testing_get_currently_deployed_versions
    • First observedcontract-testing_get_currently_supported_versions
    • First observedcontract-testing_get_deployed_versions_for_version
    • First observedcontract-testing_get_environment
    • First observedcontract-testing_get_integrations_by_team
    • First observedcontract-testing_get_latest_pacticipant_version
    • First observedcontract-testing_get_metrics
    • First observedcontract-testing_get_pacticipant
    • First observedcontract-testing_get_pacticipant_label
    • First observedcontract-testing_get_pacticipant_network
    • First observedcontract-testing_get_pacticipant_version
    • First observedcontract-testing_get_pacts_for_verification
    • First observedcontract-testing_get_provider_states
    • First observedcontract-testing_get_released_versions_for_version
    • First observedcontract-testing_get_secret
    • First observedcontract-testing_get_system_preferences
    • First observedcontract-testing_get_team_metrics
    • First observedcontract-testing_get_user_preferences
    • First observedcontract-testing_get_webhook
    • First observedcontract-testing_list_api_tokens
    • First observedcontract-testing_list_branches
    • First observedcontract-testing_list_environments
    • First observedcontract-testing_list_integrations
    • First observedcontract-testing_list_labels
    • First observedcontract-testing_list_pacticipant_versions
    • First observedcontract-testing_list_pacticipants
    • First observedcontract-testing_list_pacticipants_by_label
    • First observedcontract-testing_list_secrets
    • First observedcontract-testing_list_webhooks
    • First observedcontract-testing_matrix
    • First observedcontract-testing_patch_pacticipant
    • First observedcontract-testing_publish_consumer_contracts
    • First observedcontract-testing_publish_provider_contract
    • First observedcontract-testing_record_deployment
    • First observedcontract-testing_record_release
    • First observedcontract-testing_regenerate_api_token
    • First observedcontract-testing_remove_label_from_pacticipant
    • First observedcontract-testing_review_pact_tests
    • First observedcontract-testing_test_execute_webhooks
    • First observedcontract-testing_update_environment
    • First observedcontract-testing_update_pacticipant
    • First observedcontract-testing_update_pacticipant_version
    • First observedcontract-testing_update_secret
    • First observedcontract-testing_update_webhook
    • First observedqmetry_bulk_update_test_case_execution_status
    • First observedqmetry_bulk_update_test_run_udfs
    • First observedqmetry_create_cycle
    • First observedqmetry_create_defect_or_issue
    • First observedqmetry_create_release
    • First observedqmetry_create_test_case
    • First observedqmetry_create_test_suite
    • First observedqmetry_fetch_automation_status
    • First observedqmetry_fetch_builds
    • First observedqmetry_fetch_cascade_child_values
    • First observedqmetry_fetch_defects_or_issues
    • First observedqmetry_fetch_executions_by_test_suite
    • First observedqmetry_fetch_issue_executions
    • First observedqmetry_fetch_issues_linked_to_test_case
    • First observedqmetry_fetch_linked_issues_of_test_case_run
    • First observedqmetry_fetch_platforms
    • First observedqmetry_fetch_qmetry_list_projects
    • First observedqmetry_fetch_qmetry_project_info
    • First observedqmetry_fetch_releases_and_cycles
    • First observedqmetry_fetch_requirement_details
    • First observedqmetry_fetch_requirements
    • First observedqmetry_fetch_requirements_linked_to_test_case
    • First observedqmetry_fetch_test_case_details
    • First observedqmetry_fetch_test_case_executions
    • First observedqmetry_fetch_test_case_runs_by_test_suite_run
    • First observedqmetry_fetch_test_case_steps
    • First observedqmetry_fetch_test_case_version_details
    • First observedqmetry_fetch_test_cases
    • First observedqmetry_fetch_test_cases_linked_to_requirement
    • First observedqmetry_fetch_test_cases_linked_to_test_suite
    • First observedqmetry_fetch_test_run_udf_metadata
    • First observedqmetry_fetch_test_run_udf_values
    • First observedqmetry_fetch_test_suites
    • First observedqmetry_fetch_test_suites_for_test_case
    • First observedqmetry_import_automation_test_results
    • First observedqmetry_link_issues_to_testcase_run
    • First observedqmetry_link_platforms_to_test_suite
    • First observedqmetry_link_requirements_to_testcase
    • First observedqmetry_link_test_cases_to_test_suite
    • First observedqmetry_requirements_linked_test_cases_to_test_suite
    • First observedqmetry_set_qmetry_project_info
    • First observedqmetry_update_cycle
    • First observedqmetry_update_issue
    • First observedqmetry_update_test_case
    • First observedqmetry_update_test_suite
    • First observedqtm4j_create_test_case
    • First observedqtm4j_create_test_cycle
    • First observedqtm4j_get_automation_history
    • First observedqtm4j_get_linked_requirements
    • First observedqtm4j_get_linked_requirements_for_test_cycle
    • First observedqtm4j_get_linked_test_cases_for_requirement
    • First observedqtm4j_get_projects
    • First observedqtm4j_get_test_steps
    • First observedqtm4j_link_requirements_to_test_case
    • First observedqtm4j_link_requirements_to_test_cycle
    • First observedqtm4j_link_test_cases_to_requirement
    • First observedqtm4j_link_test_cases_to_test_cycle
    • First observedqtm4j_search_linked_test_cases_in_test_cycle
    • First observedqtm4j_search_test_cases
    • First observedqtm4j_search_test_cycles
    • First observedqtm4j_set_project_context
    • First observedqtm4j_unlink_requirements_from_test_case
    • First observedqtm4j_unlink_requirements_from_test_cycle
    • First observedqtm4j_unlink_test_cases_from_requirement
    • First observedqtm4j_unlink_test_cases_from_test_cycle
    • First observedqtm4j_update_test_case
    • First observedqtm4j_update_test_cycle
    • First observedqtm4j_upload_automation_result
    • First observedreflect_add_prompt_step
    • First observedreflect_add_segment
    • First observedreflect_cancel_suite_execution
    • First observedreflect_connect_to_session
    • First observedreflect_delete_previous_step
    • First observedreflect_execute_suite
    • First observedreflect_get_screenshot
    • First observedreflect_get_suite_execution_status
    • First observedreflect_get_test_detail
    • First observedreflect_get_test_status
    • First observedreflect_list_segments
    • First observedreflect_list_suite_executions
    • First observedreflect_list_suites
    • First observedreflect_list_tests
    • First observedreflect_run_test
    • First observedswagger_create_api_from_prompt
    • First observedswagger_create_documentation_page
    • First observedswagger_create_or_update_api
    • First observedswagger_create_portal
    • First observedswagger_create_portal_product
    • First observedswagger_create_table_of_contents
    • First observedswagger_delete_portal_product
    • First observedswagger_delete_table_of_contents
    • First observedswagger_get_api_definition
    • First observedswagger_get_document
    • First observedswagger_get_portal
    • First observedswagger_get_portal_product
    • First observedswagger_list_organizations
    • First observedswagger_list_portal_product_sections
    • First observedswagger_list_portal_products
    • First observedswagger_list_portals
    • First observedswagger_list_table_of_contents
    • First observedswagger_publish_portal_product
    • First observedswagger_resolve_organization_portal
    • First observedswagger_scan_api_standardization
    • First observedswagger_scan_api_standardization_from_registry
    • First observedswagger_search_apis_and_domains
    • First observedswagger_standardize_api
    • First observedswagger_update_document
    • First observedswagger_update_portal
    • First observedswagger_update_portal_product
    • First observedzephyr_create_folder
    • First observedzephyr_create_test_case
    • First observedzephyr_create_test_case_issue_link
    • First observedzephyr_create_test_case_steps
    • First observedzephyr_create_test_case_web_link
    • First observedzephyr_create_test_cycle
    • First observedzephyr_create_test_cycle_issue_link
    • First observedzephyr_create_test_cycle_web_link
    • First observedzephyr_create_test_execution
    • First observedzephyr_create_test_execution_issue_link
    • First observedzephyr_create_test_script
    • First observedzephyr_get_environments
    • First observedzephyr_get_issue_link_test_cases
    • First observedzephyr_get_priorities
    • First observedzephyr_get_project
    • First observedzephyr_get_projects
    • First observedzephyr_get_statuses
    • First observedzephyr_get_test_case
    • First observedzephyr_get_test_case_links
    • First observedzephyr_get_test_case_steps
    • First observedzephyr_get_test_cases
    • First observedzephyr_get_test_cycle
    • First observedzephyr_get_test_cycle_links
    • First observedzephyr_get_test_cycles
    • First observedzephyr_get_test_cycles_linked_to_a_jira_issue
    • First observedzephyr_get_test_execution
    • First observedzephyr_get_test_execution_links
    • First observedzephyr_get_test_execution_steps
    • First observedzephyr_get_test_executions
    • First observedzephyr_get_test_executions_linked_to_a_jira_issue
    • First observedzephyr_get_test_script
    • First observedzephyr_update_test_case
    • First observedzephyr_update_test_cycle
    • First observedzephyr_update_test_execution
    • First observedzephyr_update_test_execution_steps

TDQS

C2.6/5.0
Disambiguation2/5

Product prefixes provide some orientation, but multiple toolsets overlap heavily on the same domain: qtm4j_, qmetry_, zephyr_, reflect_, and bearq_ all manage test cases, cycles, and executions, so an agent cannot tell which product to use from the name alone. Within contract-testing, the ten get_bdct_* variants differ only subtly, and tools like contract-testing_test_execute_webhooks vs contract-testing_execute_webhook are near-duplicates.

Naming Consistency2/5

The same operations are named differently across the server (list_ vs fetch_ vs get_ vs search_, create vs add, update vs edit/patch), and prefixes mix hyphens (contract-testing_) with underscores (qmetry_). Some names are awkward or inconsistent, such as contract-testing_get_bdct_x-contract_test_results_by_consumer and bearq_run_tests_in_functional_areas, though nearly everything is at least snake_case.

Tool Count1/5

308 tools is an extreme count that will overwhelm any agent's selection process and context. Even for a multi-product QA suite, bundling this many operations into one MCP server is far beyond a coherent, manageable scope (typical well-scoped servers are 3-15 tools).

Completeness2/5

The aggregate covers many workflows, but individual products have notable gaps: Zephyr has no delete for test cases/cycles, Reflect has no update/delete for tests or segments, BearQ has no create/update except via a chat escape hatch, and QTM4J/QMetry lack delete operations for core entities. The overlapping duplicate surfaces also make gaps harder to detect, so agents will hit dead ends in several sub-domains.

Maintenance

ActivityActive
ResponsivenessWithin a week

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    B
    maintenance
    Enables AI assistants to interact with New Relic monitoring and observability data through programmatic access to New Relic APIs. Supports APM management, NRQL queries, alert policies, synthetic monitoring, dashboards, infrastructure monitoring, and deployment tracking.
    26
    6
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables AI assistants to interact with Sauce Labs testing platform through natural language, providing access to device cloud management, test job analysis, build monitoring, and testing infrastructure insights. Supports both Virtual Device Cloud (VDC) and Real Device Cloud (RDC) with comprehensive test analytics and team collaboration features.
    34
    10
    Apache 2.0
  • A
    license
    B
    quality
    D
    maintenance
    Enables AI assistants to interact with TestRail test management systems through comprehensive API integration. Supports retrieving and updating test cases, projects, suites, runs, and results, plus adding attachments and managing test data through natural language commands.
    18
    36
    MIT
  • F
    license
    B
    quality
    F
    maintenance
    Enables AI assistants to interact with Xray Test Management for both Cloud and Server deployments. Supports test execution, importing results from multiple formats (JUnit, Cucumber, Robot Framework, TestNG), and managing test plans and executions.
    8
    3
    -

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/SmartBear/smartbear-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server