Skip to main content
Glama

@m00nsolutions/mcp-server

npm CI node license

Агентное управление тестированием через MCP. Этот сервер позволяет ИИ-ассистенту выполнять работу, а не читать о ней: создавать тест-кейсы, планировать и наполнять ручные тестовые прогоны, привязывать автоматизацию к покрываемым ею кейсам, выпускать релизы и следить за состоянием проектов в M00N Report — AI-нативной платформе управления тестированием.

m00nreport/mcp-server MCP server

Какой вариант установки мне нужен?

Этот npm-пакет (stdio)

Удалённый коннектор (размещённый)

Как запускается

npx @m00nsolutions/mcp-server как локальный процесс

Без локального процесса; клиент общается с API по HTTP

Авторизация

MCP-ключ в M00N_API_KEY

OAuth 2.1 с PKCE, экран согласия в браузере

Клиенты

Любой MCP-клиент: Claude Code, Claude Desktop, Cursor и другие

только claude.ai и claude.com

Если вы подключаетесь не с claude.ai или claude.com, используйте этот пакет с MCP-ключом. Именно этому посвящена остальная часть этого README. О коннекторе читайте в руководстве по коннектору.

Related MCP server: TestRail MCP Server

Требования

Node 20 или новее и любой клиент с поддержкой MCP.

Установка

Шаг установки не нужен. Клиент запускает сервер через npx, как настроено ниже.

Чтобы зафиксировать конкретную версию в общей среде или в CI, добавьте её к имени пакета в args, например ["-y", "@m00nsolutions/mcp-server@<version>"]. Команда npm view @m00nsolutions/mcp-server versions покажет список опубликованных версий. Плейсхолдер, а не число, здесь намеренно: конкретный пример устареет после следующего релиза, а устаревший пример будет рекомендовать фиксировать версию, в которой уже нет текущих исправлений.

Быстрый старт

1. Получите MCP-ключ

В M00N Report откройте Settings -> MCP Keys и создайте ключ. Он начинается с m00n_mcp_, за ним следуют 48 шестнадцатеричных символов. Это не те учётные данные, которые используют репортёры тестов; их учётные данные начинаются с m00n_ и здесь отклоняются.

MCP-ключ несёт ровно те разрешения, что и учётная запись, которой он принадлежит, не больше, и его можно дополнительно ограничить. Ограничение убирает инструменты из списка, который видит клиент, так что более короткий список инструментов, чем вы ожидали, — это решение об ограничении, а не сбой.

Ключ должен храниться в конфигурации вашего MCP-клиента на вашем компьютере. Не помещайте его в файл рабочей области, который находится в репозитории.

2. Подключите клиента

Claude Code, одной строкой:

claude mcp add m00n --env M00N_API_URL=https://m00nreport.com --env M00N_API_KEY=m00n_mcp_... -- npx -y @m00nsolutions/mcp-server

Все остальные клиенты принимают тот же JSON-блок:

{
  "mcpServers": {
    "m00n": {
      "command": "npx",
      "args": ["-y", "@m00nsolutions/mcp-server"],
      "env": {
        "M00N_API_URL": "https://m00nreport.com",
        "M00N_API_KEY": "m00n_mcp_your_key_here"
      }
    }
  }
}

Куда помещается этот блок:

Клиент

Файл конфигурации

Cursor

~/.cursor/mcp.json

Claude Desktop, macOS

~/Library/Application Support/Claude/claude_desktop_config.json

Claude Desktop, Windows

%APPDATA%\Claude\claude_desktop_config.json

Для собственного инстанса укажите в M00N_API_URL свой корневой origin без суффикса /api. Завершающий слэш будет удалён автоматически.

3. Проверьте подключение

Перезапустите клиент, а затем попросите его:

Перечисли мои проекты M00N Report

Список ваших проектов означает, что сервер подключён и ключ действителен. Ошибка или сообщение ассистента о том, что нет такого инструмента, означает обратное; см. Устранение неполадок.

Инструменты

Восемь областей. Полный список с аргументами находится на странице /documentation/mcp/tools в актуальном виде рядом с кодом. Сервер загружает свой список из вашего инстанса при запуске, поэтому клиент видит то, что ваш инстанс поддерживал на момент подключения клиента.

Область

Что охватывается

Проекты и аналитика

Проекты, тенденции запусков и статистика, история по каждому тесту, поиск по автоматизированным тестам

Тест-кейсы

Создание, чтение, обновление, перемещение, удаление, массовое редактирование и привязка автоматизированного теста к кейсу, который он покрывает

Папки и наборы

Иерархия папок и наборов, а также статистика по каждой папке

Тестовые коллекции

Многократно используемые именованные наборы кейсов, которые помещаются в любой прогон

Ручные прогоны

Сборка прогона из наборов, коллекций или отдельных кейсов с последующей фиксацией результатов по каждому кейсу

Релизы

Создание и управление релизами, привязка и отвязка запусков, которые к ним относятся

Здоровье и покрытие

Композитная проверка состояния, пробелы покрытия в обеих направлениях, предложения по кейсам, сканирование возможностей, экспорт отчётов

Внешние ссылки

Прикрепление и удаление тикетов Jira или Linear в кейсе

Прогон — это ручной тестовый запуск: исполнители, окружение, даты и пошаговые результаты, зафиксированные на снимке каждого кейса в момент его добавления.

Сколько инструментов может видеть ваш ключ, зависит от вашего инстанса:

curl -s -H "X-MCP-Key: $M00N_API_KEY" "$M00N_API_URL/api/mcp/tools" | jq .count

Ограниченный ключ видит меньше. Число здесь намеренно не указано: состав инструментов меняется с каждым релизом, а цифра в README — это то место, которое никто никогда не вспоминает обновлять.

Промпты

Направляемые многошаговые сценарии, которые клиент может предлагать по имени. Каждый принимает один необязательный аргумент project — имя или UUID; если его опустить, сервер выберет проект сам, когда в области видимости только один.

Prompt

Что делает

analyze_flaky_tests

Находит тесты, которые проходят и падают без изменения кода, и ранжирует их

debug_test_failure

Разбирает один сбой до причины по его истории и трейсам.

release_readiness

Оценивает, безопасно ли выпускать релиз.

generate_test_cases

Черновик ручных кейсов для области, где их нет.

weekly_health_report

Подводит итоги недели по всем проектам.

investigate_regression

Найдите, что изменилось между неуспешным и падающим прогоном.

run_manual_execution

Проводит ручной тестовый цикл от сборки до результатов по каждому кейсу.

Ресурсы

Представления только для чтения, которые клиент может получил _вызывая инструмента: все проекты, сводка состояния проектов, отчёт о нестабильных тестах, текущие падающие тесты, структура папок, отчёта о запуске, содержимое тест-кейса, сводка по релизаам.

Конфигурация

Переменная окружения

Обязательность

По умолчанию

Описание

M00N_API_URL

для подключения

URL вашего M00N Report. Корневой origin, без суффикса /api. Завершённый слеш удаляется автоматически.

M00N_API_KEY

для подключения

MCP-ключ (m00n_mcp_...). Всё без этого префикса отклоняется до первого запроса.

M00N_DEBUG

нет

false

Добавляет в stderr детали по каждому запросу, включая аргументы каждого вызова инструмента. Шаги запуска и ошибки логируются в любом случае.

M00N_TIMEOUT_MS

нет

30000

Таймаут запроса, число миллисекунд. 60s, 1e4, 30,000 и 1.5 отклоняются с предупреждением в stderr, и используется значение по умолчанию.

M00N_INSECURE_SSL

нет

false

Принимать самоподписанный сертификат. Только для собственных инстансов.

Оба булевых значения включаются при true, 1 или yes в любом регистре. Любое другое значение, включая false и 0, оставляет их выключенными.

Задайте либо обе первые две переменный, либо ни одной. Если задана одна, это считается ошибкой и отклоняется, потому что полуконфигурированный сервер, который всё равно запускается, более сложен для диагностики, чем тот, который не запускается.

Базовый URL здесь — M00N_API_URL; репортёры тестов называют этот же URL M00N_SERVER_URL. Все шесть пакетов читают M00N_API_KEY, но у них это проектный API-ключ, а не MCP-ключ, поэтому одна экспортированная переменная оболочки не может служить обоим.

Запуск без ключа

При старте без M00N_SERVER_URL и без M00N_API_KEY сервер работает в режиме предпросмотра: он перечисляет свои инструменты, промпты и ресурсы из встроенного микроснимка и отклоняет каждый вызов сообщением о том, что нужно задать. Никуда ничего не отправляется, потому что отправлять некуда.

Это то, что позволяет инспектору, клиенту или каталогу показать инструменты до того, как кто‑то зарегистрировался. Это описание сервера, а не работающий сервер, поэтому список берётся из снапшота, а не от вашего инстанса: ограниченный ключ обычно видит меньше инструментов, чем показывает снапшот.

Поддерживающие обновляют снапшот командой npm run snapshot:refresh с использованием неограниченного ключа.

Собственный инстанс

Укажите M00N_API_URL на ваш собственный инстанс и выдайте MCP-ключ в этом инстансе. Всё остальное идентично, и это важно, когда причина собственного развёртывания — то, что тестовые данные не могут покинуть вашу сеть.

Если используется самоподписанный сертификат, добавьте "M00N_INSECURE_SSL": "true" в тот же блок env. Полная информация: /documentation/mcp/self-hosted.

Что отправляется

Через HTTPS по адресу M00N_API_URL передаются вызовы инструментов, которые делает ваш ассистент, и их аргументы. Сервер — это прокси к вашему собственному инстансу и не хранит ничего сам.

Каждый вызов инструмента записывается в журнал аудита MCP с интерфейсом в приложении, а ограничение частоты запросов на ключ сдерживают выбитого из-под контроля агента. Ключ никогда не записывается ни в этот журнал, ни в stderr. При установке M00N_DEBUG аргументы каждого вызова инструмента выводятся в stderr, поэтому относитесь к этому выводу как к самим данным.

Устранение неполадок

Симптом

Причина

Клиент не показывает инструменты M00N Report

Конфигурация не была перезагружена. Полностью перезапустите клиент, а не только диалог.

Инструменты перечислены, но каждый вызов отвечает running unconfigured

Ни M00N_API_URL, ни M00N_API_KEY не дошли до процесса, поэтому он запустился в режиме предпросмотра. Проверьте, что блок env действительно применяется к этому серверу.

M00N_API_KEY should start with "m00n_mcp_"

Это проектный API-ключ — такие используют тестовые репортёры. MCP-ключи выдаются отдельно: Settings -> MCP Keys.

Authentication failed: Invalid or expired MCP key

Ключ неверный, отозван или относится к другому экземпляру. Или M00N_API_URL содержит суффикс /api, из-за чего каждый запрос возвращает 401, каким бы правильным ни был ключ. Сначала проверьте URL, затем выпустите новый ключ.

WARNING: M00N_API_URL is http://...

Это не ошибка, и сервер продолжает работать. Это значит, что ключ передаётся без шифрования в каждом запросе. Используйте https, если только экземпляр не находится в доверенной частной сети. Loopback-адреса предупреждение не вызывают.

... redirected to ...

URL отвечает перенаправлением, а перенаправления не выполняются, потому что MCP-ключ уехал бы туда, куда они указывают. Задайте M00N_API_URL как адрес, который отвечает напрямую, обычно в виде https://.

Cannot connect to M00N Report API at ...

По этому адресу никто не прослушивает. Экземпляр остановлен либо порт закрыт для вас.

Cannot resolve M00N Report API host

Имя хоста не разрешается. Опечатка или частное имя, которое вызывается из-за пределов его сети.

Request to M00N Report API timed out

Увеличьте M00N_TIMEOUT_MS или проверьте сетевой путь.

SSL certificate error connecting to ...

Самоподписанный сертификат на самостоятельном экземпляре. Установите M00N_INSECURE_SSL в true.

Not found during ...

Остальная часть строки — это собственный ответ сервера, и в нём обычно указывается решение, например какие проекты существуют. Прочитайте его перед изменением настроек.

Rate limit exceeded. Retry after Ns.

1000 вызовов в минуту на ключ в скользящем окне. В сообщении указано, сколько ждать. Агент, выполняющий задачу, редко приближается к лимиту; его достижение обычно означает зацикливание.

Инструментов меньше, чем ожидалось

Ключ ограничен по правам доступа, либо экземпляр старше пакета. Список инструментов формируется вашим экземпляром.

Сервер пишет журнал в stderr независимо от переменной M00N_DEBUG, и большинство клиентов показывают его в своей MCP-панели. Установите M00N_DEBUG в true, чтобы добавить в журнал каждый запрос и его аргументы.

Известные ограничения

  • Список инструментов считывается один раз, при запуске. Переопределите права ключа или обновите экземпляр — клиент сохранит старый список до тех пор, пока вы не перезапустите клиент.

  • Если настроен, GET /api/health должен быть доступен. Он проверяется раньше всего остального, и при сбое сервер останавливается; поэтому прокси, передающий только /api/mcp/*, не показывает ни одного инструмента, хотя каждый из них работал бы.

  • Ключ с правами на ноль инструментов не запускается. Сервер завершается, а не подключается с пустым списком. Это происходит при заданной конфигурации; если учётных данных нет вовсе, он запускается в режиме предпросмотра.

  • Промпты и ресурсы необязательны. Если их список загрузить не удалось, сервер всё равно запускается, причина выводится в stderr, и клиент не будет видеть ни одного из них в течение всей сессии.

  • Перенаправления отклоняются, а не выполняются. Ключ находится в пользовательском заголовке, а обработчики перенаправлений удаляют только распознанные заголовки авторизации; поэтому при переходе по перенаправлению ключ попал бы к следующему отвечающему хосту. О перенаправлении сообщается вместе с адресом, на который оно указывало.

  • Только stdio. Для HTTP-транспорта используйте размещённый соединитель (hosted connector).

Поддержка

Лицензия

Лицензия MIT. См. LICENSE.

Available Tools

55 tools
add_cases_to_collectionA
Idempotent

Add test cases to an existing test collection. Cases must exist and belong to the same project. Duplicate cases are skipped.

ParametersJSON Schema
NameRequiredDescriptionDefault
caseIdsYesNumeric test case IDs to add (e.g., [1, 2, 3])
collectionIdYesTest collection UUID

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses behavioral traits beyond the annotations: duplicate cases are skipped, and the cases must exist and belong to the same project. These are valuable operational details that align with, and enrich, the idempotentHint=true annotation. It does not specify failure behavior for invalid cases or collection, but the core behavior is clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler. The core operation is front-loaded, and the caveats are stated briefly and precisely. Every sentence provides useful action-oriented information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter mutating operation, the description is largely sufficient: it gives the target, the precondition, and the duplicate-skipping behavior. With idempotentHint=true and destructiveHint=false already supplied, the description covers most operational concerns. It does not describe what happens if a case belongs to another project or if the collection is missing, but that is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema coverage is 100%, so the schema already explains parameter formats and meanings. The description adds operational meaning such as duplicate handling and project membership, but it does not need to restate parameter semantics. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly specifies the action (add), the object (test cases), and the target (an existing test collection). This differentiates it from related tools like add_cases_to_execution and remove_cases_from_collection without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states when to use this tool: when adding test cases to an existing collection. The preconditions that cases must exist in the same project help clarify correct use. It does not explicitly exclude alternatives or name sibling tools, but the target resource is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_cases_to_executionA

Add individual test cases to a Manual Test Execution as a new named group. Unlike suites and Collections, loose cases have no identity of their own, so name is required and the group it creates is what appears in the execution; the call returns its groupId. caseIds are NUMERIC ids, never TC-numbers. Use add_suites_to_execution or add_collections_to_execution when the cases are already gathered into one of those. Cases from another project are skipped, and an empty result creates no group. Returns {added, skipped, groupId}.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesLabel for this case group
caseIdsYesNumeric case ids
executionIdYesExecution UUID

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations only provide basic readOnly/destructive hints, so the description adds meaningful behavioral context: it creates a named group, returns groupId, skips cases from other projects, and does not create a group when the result is empty. This goes well beyond the schema and gives the agent a clear model of side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and every sentence adds value: sibling differentiation, numeric ID clarification, alternative routing, and edge-case behavior. No filler or repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema present, the description properly states the return shape as {added, skipped, groupId}. All three required parameters are semantically covered, sibling alternatives are addressed, and important non-obvious behaviors are disclosed. This is a complete description for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers all three parameters, but the description adds important nuance beyond the schema: caseIds are numeric IDs and 'never TC-numbers', and name is required because loose cases have no identity of their own. It does not add detail about executionId beyond what the schema says, but the added semantic guidance is substantive.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Add individual test cases to a Manual Test Execution as a new named group') and clearly differentiates this tool from add_suites_to_execution and add_collections_to_execution by explaining the distinction between loose cases and gathered suites/collections. An agent can immediately tell what this tool is for relative to its siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names alternatives and the conditions under which they should be used: 'Use add_suites_to_execution or add_collections_to_execution when the cases are already gathered into one of those.' It also clarifies an edge case behavior ('empty result creates no group'), helping the agent decide whether this is the right tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_collections_to_executionA

Add whole Test Collections to a Manual Test Execution. Each is snapshotted in and keeps its own identity inside the execution. Pick this when the work is already gathered into a Collection. Use add_suites_to_execution to take the current cases of a suite instead, and add_cases_to_execution for loose cases, which need a new named group. Collections from another project are skipped, as are shadow collections, and one already in the execution is skipped rather than snapshotted twice. Returns {added, skipped}.

ParametersJSON Schema
NameRequiredDescriptionDefault
executionIdYesExecution UUID
collectionIdsYesTest Collection UUIDs

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations carry no behavioral signal beyond non-read-only, but the description discloses key behaviors: snapshots collections and keeps identity, skips collections from another project, skips shadow collections, skips duplicates instead of double-snapshotting, and returns {added, skipped}. This fully covers likely agent concerns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise yet information-dense; every sentence adds value. The main purpose and selection guidance are front-loaded, and the edge-case behaviors are summarized compactly at the end.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite lacking an output schema, the description states the return shape {added, skipped}, covers selection guidance, explains snapshot identity, and enumerates all skip cases. It gives an agent everything needed to invoke this tool successfully and predict results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents both parameters with descriptions and 100% coverage, so the baseline is 3. The description adds contextual meaning about collections being snapshotted but does not add new per-parameter syntax or format details beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Add whole Test Collections to a Manual Test Execution') and clearly distinguishes it from sibling tools by contrasting snapshotting whole collections with adding suite cases or loose cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly provides the selection cue: 'Pick this when the work is already gathered into a Collection.' It also names alternatives add_suites_to_execution and add_cases_to_execution with their conditions, so an agent can choose correctly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_suites_to_executionA

Add whole Suites to a Manual Test Execution. The cases each suite holds are snapshotted in as they stand now, and the suite keeps its own identity inside the execution. Pick this when the work is organised by suite. Use add_collections_to_execution when it is already gathered into a Test Collection, and add_cases_to_execution for loose cases, which need a new named group instead. Suites from another project are skipped, and a suite already in the execution is skipped rather than snapshotted twice. Returns {added, skipped}.

ParametersJSON Schema
NameRequiredDescriptionDefault
suiteIdsYesSuite (folder) UUIDs
executionIdYesExecution UUID

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses key behavioral details beyond the annotations: cases are snapshotted at their current state, suites keep their own identity, cross-project suites are skipped, and duplicate suites are skipped rather than snapshotted twice. It also states the return shape, {added, skipped}, which compensates for the lack of an output schema. No contradiction with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence earns its place. The description front-loads the core behavior, then covers alternatives, edge cases, and return shape without unnecessary filler. It is concise yet information-dense.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given only two schema-covered parameters, no output schema, and clear annotations, this description is complete enough for an agent to select and invoke the tool correctly. It explains behavior, selection criteria, exceptions, and expected output in a compact form.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and both executionId and suiteIds already have clear schema descriptions. The tool description reinforces the meaning of suiteIds with 'whole Suites' and snapshotting context, but it does not add additional parameter-level detail beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Add whole Suites to a Manual Test Execution.' It clearly explains what happens to the cases inside the suites and how suite identity is preserved, making the tool's function obvious. The description also differentiates it from related collection-based and case-based tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says when to use this tool: 'Pick this when the work is organised by suite.' It also names the alternatives, add_collections_to_execution and add_cases_to_execution, and states the conditions for choosing them instead. This is strong routing guidance with no ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bulk_update_casesA
Idempotent

Update many test cases in one call - status, priority and tags - up to 100 at a time, all belonging to the same project. Returns updated, skipped, notFound and an errors array. Two different failure modes, and only one of them is all-or-nothing. Unknown or archived ids do NOT stop the batch: they land in notFound and in errors while every other case is still written, so read updated rather than treating the call as a whole. An unrecognised status or priority value, or ids spanning two projects, is rejected outright with nothing written. Each field is replace-all across the batch and there is no undo, so snapshot the current values with list_test_cases first if you may need to put them back.

ParametersJSON Schema
NameRequiredDescriptionDefault
caseIdsYesArray of numeric case IDs to update (e.g., [1, 2, 3])
updatesYesUpdates to apply. At least one field must be provided.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description thoroughly explains partial failure behavior, all-or-nothing failure modes, and the fact that invalid IDs do not abort the batch. It also warns about no undo and recommends snapshotting with list_test_cases first. This adds substantial behavioral detail well beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and limits, then follows a logical sequence: output, failure modes, batch behavior, and undo safety. Every sentence adds information; there is little redundancy for a tool with this much edge behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

It covers edge cases, return shape, project constraint, id limits, and practical guidance. The main missing detail is the exact meaning of the `skipped` return member, which is listed but never defined.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema by explaining that each field is replace-all across the batch, that project span matters, and that the response should be read via `updated` rather than treated as a single success/failure result.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a clear verb and resource: 'Update many test cases in one call,' and distinguishes it from single-case updates by focusing on bulk, batch, and same-project constraints. It is specific about what fields can be updated: status, priority, and tags.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives strong context for when to use it: bulk operations up to 100 cases, all belonging to the same project. It also suggests preconditions by referencing list_test_cases for snapshotting values and for reading available statuses. It does not explicitly say 'for a single case, use update_test_case,' but the bulk framing makes the intended scope clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_executionA

Create a Manual Test Execution, the top-level container on the Executions page, which aggregates Test Collections, suites and named case groups. NOT the same as a Test Collection: use create_test_collection for a single grouping of cases, and this to assemble an execution out of several. Returns the whole execution under an execution key: id, name, description, status, priority, environment, assignedTo, startDate, endDate, projectId and createdAt. The id you need for everything else is execution.id, not a top-level id. It starts empty, so follow with add_suites_to_execution, add_collections_to_execution or add_cases_to_execution to put something in it.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesExecution name
statusNoOptional. One of: not_executed, in_testing, passed, failed, blocked (plus any project-custom values). Defaults to 'not_executed'.
endDateNoOptional ISO date (>= startDate)
priorityNoOptional. One of: low, medium, high, critical. Defaults to 'medium'.
projectIdYesProject UUID
startDateNoOptional ISO date
assignedToNoOptional assignee user id (must be a project member)
releaseIdsNoOptional release UUIDs to link
descriptionNoOptional description
environmentNoOptional environment label (e.g. QA, Staging)

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a write operation, and the description adds valuable behavioral detail: the created execution starts empty, and the response embeds the entire execution under an 'execution' key. It also flags the critical quirk that execution.id is the useful identifier, not a top-level id, which agents need to know for subsequent calls.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence earns its place: it states the purpose, clarifies the boundary vs. create_test_collection, explains the response shape and key id, and closes with actionable next steps. No filler or redundant restating of the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a creation tool with 10 parameters and no output schema, this description is unusually complete. It covers what the tool creates, what the response contains, how to identify the created resource, and which subsequent tools to invoke. Nothing critical is missing for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already well documented in the input schema. The description does not explain parameter specifics beyond noting the required projectId and name, but it compensates slightly by explaining what happens after creation. This meets the baseline for full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action and resource ('Create a Manual Test Execution') and defines it as the top-level container on the Executions page. It explicitly distinguishes this tool from create_test_collection and notes the intended composite scope, so an agent can clearly tell it apart from sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives direct usage direction: use this tool for assembling an execution from several collections/suites, use create_test_collection for a single grouping of cases. It also tells the agent what to do next after creation, listing add_suites_to_execution, add_collections_to_execution, and add_cases_to_execution, leaving no ambiguity about when and how to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_folderA

Create a folder or a suite in a project. Use type "suite" for a leaf that holds test cases, and "folder" for a container that holds other folders and suites. Omitting type gives "folder" at the root and "suite" when nested, which is usually what you want. Suites cannot be parents, so a suite always needs parentId. Returns the new folder id, its type, and its path. To read an existing one use get_folder; to list them use list_folders.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesFolder/suite name
typeNoOPTIONAL - auto-detected. "folder" can contain children, "suite" is a leaf for test cases. Defaults to "folder" at root, "suite" when nested.
parentIdNoParent folder UUID. Omit for root-level folder. Note: suites cannot be parents.
projectIdYesProject UUID to create folder in

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (which already indicate this is not read-only), the description explains important behavioral nuances: omitted type defaults to 'folder' at root but 'suite' when nested, suites cannot be parents, and suites always require parentId. It also discloses the return value shape. This adds meaningful context without contradicting the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and information-dense. Every sentence adds actionable detail: creation action, type semantics, default behavior, parent constraint, return value, and reading/list alternatives. No filler or repetition of the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that there is no output schema, this description sufficiently covers the return values and key constraints. It explains core creation rules, the parentId requirement for suites, the role of 'folder' vs 'suite', and directs users to get_folder/list_folders for reads. This gives an agent what it needs to invoke the tool correctly in different scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all four parameters with high coverage, so the baseline is 3. However, the description adds useful semantic clarity on top, particularly around the 'type' default behavior and the requirement that a suite needs parentId. This helps an agent reason about valid parameter combinations beyond simple field definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a concrete verb and resource: 'Create a folder or a suite in a project.' It distinguishes the purpose of suites vs folders and differentiates itself from sibling tools by name ('get_folder', 'list_folders'). An agent can tell exactly what this tool does and how it differs from related tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance on when to use 'suite' vs 'folder', explains the default type behavior at root vs nested, and states when parentId is required. It also names sibling tools for reading and listing existing folders, making the appropriate context for this tool unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_releaseA

Create a new release in a project. Releases track versions and aggregate manual and automated test results. projectId comes from list_projects. Omit status to default to "planned". Returns the created release with id, name, version, description, status, startDate, endDate, projectId, projectName, createdAt and updatedAt; keep the id, it is what the other release tools take. Not idempotent - calling twice with the same name creates two separate releases, so check list_releases first. When the release already exists and only its name, version, dates or status should change, use update_release instead. The new release starts empty and nothing else is modified: attach automated launches with link_launches_to_release, and attach manual executions from the execution side via the releaseIds argument of create_execution or update_execution.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesRelease name (e.g., "Sprint 42", "v2.1.0 Release")
statusNoOptional. Release status from the project vocabulary, typically planned, in_progress, completed or cancelled, plus any the project added; call get_project for the list. Omit to use whichever value the project marks as its default.
endDateNoTarget end/release date (YYYY-MM-DD)
versionNoVersion string (e.g., "2.1.0", "2026-Q1")
projectIdYesProject UUID to create release in
startDateNoRelease start date (YYYY-MM-DD)
descriptionNoDetailed description of the release scope

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal a read/write safety profile, and the description meaningfully extends this by declaring the call is not idempotent, that duplicate names create separate releases, and that the new release starts empty without modifying anything else. It also states what is returned, including the fields and why keeping the id matters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: the main purpose comes first, then defaults, duplicate behavior, alternatives, and downstream workflow. It gives a lot of needed context without repeating schema content or adding filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 7 parameters, no output schema, and no enums, the description closes the important gaps itself: return shape, id retention for sibling tools, duplicate pitfalls, property updates route, and the relationship to related execution and launch tools. The only omissions, such as error conditions, are not critical enough to hurt an agent correctly invoking this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the structured field descriptions already explain each parameter. The description adds useful extra meaning, such as sourcing projectId from list_projects, the status default behavior, and where to resolve project-specific release statuses via get_project. This is a strong complement to the schema, though some parameters still rely mostly on their own schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Create a new release in a project') on a specific resource and explains what releases are for. It clearly differentiates this tool from update_release and delete_release, which is enough for an agent to select it correctly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete before-you-call guidance: get projectId from list_projects, check list_releases first because the call is not idempotent, and use update_release when only modifying an existing release. It also explains how manual executions should be linked on the execution side, so the tool is placed in the correct workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_caseA

Create a new test case in a test suite. Test cases MUST live inside a suite (not a bare folder). You can provide either: (A) a folderId pointing to an existing suite, or (B) a projectId with optional folderName/suiteName to auto-create the folder→suite hierarchy. If folderId points to a folder (not a suite), a suite is auto-created inside it. Returns the full created case object including shareUrl. Tags are auto-created if they do not exist. IMPORTANT: omit status and priority and the case takes whichever values THIS PROJECT marks as its defaults, which are Draft and Medium only until a project edits them. Call get_project to see what a given project uses.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoTags to assign to the case. New tags are auto-created.
stepsNoArray of test steps with action and expected result
titleYesTest case title (required)
statusNoOptional. Omit it and the case takes whichever value the project marks as its default, which is what the UI does. The vocabulary is per-project and a project may have renamed it or added its own, so call get_project for this project list and send a value from it rather than guessing.
folderIdNoSuite (or folder) UUID where the case will be created. If this points to a folder, a suite is auto-created inside it. Alternative: use projectId + folderName + suiteName instead.
priorityNoOptional. Omit it and the case takes whichever value the project marks as its default. The vocabulary is per-project, so call get_project for this project list and send a value from it rather than guessing.
dependsOnNoNumeric case ID this case depends on
projectIdNoProject UUID. Required when folderId is not provided. Used to find or auto-create the folder→suite hierarchy.
suiteNameNoName of the suite to find or create inside the folder (default: "General"). Used with projectId when folderId is not provided.
folderNameNoName of the folder to find or create (default: "General"). Used with projectId when folderId is not provided.
descriptionNoDetailed description of the test case. Supports Markdown: **bold**, *italic*, ~~strikethrough~~, inline `code`, # / ## / ### headings, "- " bullet and "1. " numbered lists, [text](url) links, and "> " blockquotes. Plain text works too. Images and fenced code blocks are not supported.
customFieldsNoCustom field values as key-value pairs
externalLinksNoOptional list of external ticket links to attach to the created case. Each link is inserted atomically with the case - any failure rolls back the entire creation. For Jira, externalUrl is fetched from the Jira API (just pass the issue key or full /browse/ URL as externalId); for other providers, externalUrl is required.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations by disclosing side effects: tags are auto-created, folders/suites are auto-created when needed, and status/priority silently fall back to project defaults if omitted. It also states that the full created case object including shareUrl is returned. This gives the agent a strong model of what will happen at runtime.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long, but the tool is complex and almost every sentence earns its place: the suite constraint, the two creation paths, auto-creation behaviors, return value, tag behavior, and project defaults. It is organized with parenthetical labels and an 'IMPORTANT' callout, making the density digestible.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 13 parameters, no output schema, and no behavioral annotations, the description is remarkably complete: it explains return shape, required placement, alternate creation modes, auto-creation side effects, default-value behavior, and tells the agent where to look for project-specific vocabulary. The external link rollback behavior is also covered in the schema's nested parameter description, so nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all parameters at 100% coverage, so the baseline is 3. The description adds decision-level meaning on top: which parameter branch to choose, what gets auto-created, and the important behavior of omitting status/priority. This is meaningful guidance beyond the schema, though the schema itself is already quite descriptive.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Create a new test case in a test suite.' It clearly distinguishes creation from related utilities like update_test_case, delete_test_case, and move_test_case. The nuance that test cases must live inside a suite is front and center.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly explains the two mutually exclusive ways to call it: via folderId or via projectId with folderName/suiteName. It also tells the agent when to consult get_project for project-specific status/priority defaults. It does not explicitly contrast with update_test_case, but the create vs. update semantics are clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_collectionA

Create a Test Collection and optionally add test cases to it. This is one Test Collection, NOT the top-level Executions-page container - use create_execution for that (an execution can then include this collection via add_collections_to_execution). Returns the created collection with count of added tests. Omit priority to default to "medium". A new Collection always starts at status not_started and there is no status argument here: use update_test_collection if it needs to start anywhere else, which is a second call.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesCollection name (e.g., "Sprint 42 Regression")
caseIdsNoNumeric test case IDs to add to collection (e.g., [1, 2, 3])
folderIdNoFolder UUID to place collection in (omit for root level)
priorityNoOptional, defaults to medium. Unlike a case priority, a collection priority is NOT project-configurable: the storage layer accepts only these four, in lowercase, so a project-custom case priority cannot be used here. Matching ignores case, and the value is stored lowercase.
projectIdYesProject UUID to create collection in
releaseIdNoLink to a release UUID
assignedToNoEmail address of the assigned tester
descriptionNoDetailed description of the test collection
environmentNoTarget environment (e.g., "staging", "production")

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses important non-obvious behaviors: newly created collections always start at status not_started, no status argument is accepted, priority defaults to 'medium', and priority matching is case-insensitive but stored lowercase. It also states return behavior (created collection with count of added tests) and clarifies the distinction from an execution container. These details go well beyond the minimal annotation hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is information-dense but efficiently organized, front-loading the core purpose and the critical disambiguation from executions. Every sentence adds value, and there is no filler or repetition of schema basics.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter creation tool, the description covers the essential call context: what it returns, how defaults work, what it cannot do, and how it relates to adjacent sibling operations. The absence of an output schema is adequately mitigated by explicitly stating the created collection and added test count.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Though the schema already documents all nine parameters, the description adds meaningful semantic nuance such as the default for priority, the lowercase-only constraint, the fact that collection priority cannot be project-customized, and that caseIds are optional. It does not restate every parameter, but it enriches the most behaviorally important one.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action ('Create a Test Collection') and the resource, and immediately distinguishes it from the top-level Executions-page container ('NOT the top-level Executions-page container - use create_execution'). This makes the tool's purpose unambiguous and separates it from sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly explains when to use this tool versus create_execution and update_test_collection, including routing to add_collections_to_execution for attaching collections to executions. This gives an agent clear decision guidance with named alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_executionA
DestructiveIdempotent

Soft-delete an entire Manual Test Execution, and unlink any Test Collections that pointed to it. The Collections themselves, and the suites and cases behind every source, are left intact: only the execution and its result rows go. Reversible only via support or the database, so use with care. IMPORTANT: do any per-source cleanup FIRST. Once the execution is deleted, remove_source_from_execution answers Execution not found, so there is no route back to its sources. To remove just one source group instead of the whole execution, use remove_source_from_execution. Returns deleted: true.

ParametersJSON Schema
NameRequiredDescriptionDefault
executionIdYesExecution UUID

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already carry destructiveHint=true and idempotentHint=true, but the description adds the soft-delete semantics (only result rows are removed; collections/suites/cases survive), the reversibility path (support or database only), the 'Execution not found' trap after deletion, and the return contract ('returns deleted: true'). Nothing in the description contradicts the annotations — the idempotentHint is consistent with a re-call best trouble behavior — so no annotation contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The key verb, scope, and warning are front-loaded; every sentence in the long description earns its keep, particularly the ordering warning and the explicit alternative. It loses only a point to mild filler ('use with care') and a slight redundancy between the two sentences about the remove_source consequence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive tool with no output schema, this is complete: it states the return value, what is destroyed, what survives, the ordering precondition, and the post-condition failure mode. The idempotence trait is already covered by annotations. An agent needs nothing else to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With a single parameter at 100% schema-description coverage ('Execution UUID'), the schema already fully documents executionId, so the description carries no obligation to re-explain it. The description does not add syntax or formatting detail beyond the schema, correctly leaving parameter semantics to the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ('soft-delete an entire Manual Test Execution') and a concrete side effect ('unlink any Test Collections that pointed to it'). It scopes what is destroyed versus preserved and explicitly distinguishes itself from the sibling remove_source_from_execution, so an agent can tell them apart without perusing the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly names the alternative that selects it away ('To remove just one source group instead of the whole execution, use remove_source_from_execution') and hard-codes a sequencing prerequisite ('do any per-source cleanup FIRST'). It also warns about the consequence of ignoring that ordering, so when-to-use and when-not-to-use are both given from inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_folderA
DestructiveIdempotent

Soft delete (archive) a folder and all its contents. Archives all nested folders and test cases recursively. Can be restored via the UI.

ParametersJSON Schema
NameRequiredDescriptionDefault
folderIdYesFolder UUID to archive

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already flag destructiveHint and idempotentHint, but the description adds essential behavior beyond these: the deletion is soft/archival, recursive across nested folders and test cases, and reversible through the UI. That is concrete, useful information that the annotations alone do not provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences front-load the essential behavior ('soft delete', 'all its contents', 'recursive', 'restorable'). Every sentence carries necessary information and there is no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple destructive tool with one fully-documented parameter and no output schema, the description fully covers the operation's scope, side effects, and reversibility. There is nothing critical missing for an agent to safely decide whether to invoke this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Since schema description coverage is 100%, the baseline is 3. The description adds meaning to folderId by explaining that identifying that folder also archives all nested folders and test cases, not just the folder itself. This reinforces the subtree-wide implication of the single parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource ('Soft delete (archive) a folder') and immediately clarifies scope ('all its contents'). It distinguishes itself from sibling tools such as delete_test_case, delete_test_collection, and update_folder by emphasizing recursive archiving of nested folders and test cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when to use this tool: when a folder and all nested content should be archived, and notes that restoration is only via the UI. It does not explicitly say 'don't use this if you want permanent deletion' or point to a non-recursive alternative, but the recursive wording implies this.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_releaseA
DestructiveIdempotent

Soft delete (archive) a release. The release can be restored via the UI. All linked Test Collections and launches are preserved.

ParametersJSON Schema
NameRequiredDescriptionDefault
releaseIdYesRelease UUID to archive

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral context beyond the annotations: the deletion is soft, restoration is possible through the UI, and linked Test Collections and launches are preserved. This is exactly the kind of non-obvious behavior an agent needs to know and is not captured in the structured annotation fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences deliver all essential information with no redundancy or filler. The primary action is front-loaded, and subsequent sentences provide valuable non-obvious details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, no-output-schema tool, the description is complete. It states the action, the restoration path, and the preservation of related resources, giving an agent enough to invoke it correctly without missing important consequences.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents releaseId as 'Release UUID to archive' with 100% coverage. The description adds little beyond the schema, but it is consistent with the parameter's meaning. Given full schema coverage, the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Soft delete (archive) a release.' It immediately distinguishes this tool from sibling delete tools by naming the target resource and clarifying the action is a soft delete/archive rather than a permanent purge.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear context for when to use the tool: when you need to archive a release. It also indirectly sets expectations by noting restoration is done via the UI, implying this is not a permanent delete. It does not explicitly name alternatives, but the resource-specific purpose makes the usage context sufficiently clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_test_caseA
DestructiveIdempotent

Soft delete (archive) a test case. The case can be restored via the UI. Also unlinks any connected autotests.

ParametersJSON Schema
NameRequiredDescriptionDefault
caseIdYesNumeric test case ID to archive

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description adds valuable behavioral details: the operation is a soft delete, the case can be restored via the UI, and connected autotests are unlinked. These side effects are not visible from annotations alone and help the agent predict the full impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two tight sentences with no wasted words. The core operation is front-loaded, followed by the restore path and the important side effect of unlinking autotests.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter mutation with annotations, this description is sufficient. It explains what happens, how it can be undone, and what additional effects occur, so an agent can safely predict the behavior without an output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the only parameter, caseId, is already documented as 'Numeric test case ID to archive'. The tool description adds no additional parameter meaning beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource ('a test case') and the specific behavior ('Soft delete (archive)'). It also separates it from sibling deletion tools by emphasizing archiving and restoration.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use case is implied: call this when you want to archive a test case, and the restoration note implies the operation is reversible. However, it does not explicitly mention alternatives or when not to use it, such as removing a case from a collection instead of deleting it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_test_collectionA
DestructiveIdempotent

Archive (soft delete) a test collection. The collection can be restored via the UI. Test executions are preserved.

ParametersJSON Schema
NameRequiredDescriptionDefault
collectionIdYesTest collection UUID to archive

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds value beyond the annotations. Although destructiveHint and idempotentHint already mark the operation as destructive and repeatable, the description clarifies the exact nature: it is an archive/soft delete, the collection is recoverable via UI, and test executions remain intact. This materially refines the meaning of the destructive hint and helps the agent avoid assuming irreversible deletion.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences deliver high-value information with no redundant words. The most important fact, 'soft delete', is front-loaded, and the following sentences add critical behavioral nuance about restoration and preserved executions.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is complete for a simple one-parameter mutation. It explains the operation type, reversibility, and impact on test executions. There is no output schema, so the absence of return-format details is acceptable. The core behavior an agent needs to invoke this tool safely and correctly is fully covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is only one parameter and the schema description coverage is 100%. The schema already explains that collectionId is the test collection UUID to archive. The tool description does not add further parameter-specific detail, which is acceptable because the schema carries that burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific operation: 'Archive (soft delete) a test collection'. It identifies both the verb and the resource, and explicitly clarifies it is a soft delete. This distinguishes it from the sibling delete_* tools, especially delete_test_case and delete_execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly name alternative tools or state when not to use it, but it gives clear behavioral context: restoration via the UI and preservation of test executions. For an agent deciding whether to use this tool, the 'soft delete' framing and reversibility strongly imply this is the safer deletion pathway.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

export_reportA
Read-onlyIdempotent

Generate a formatted report for a project. Supports summary, failures, trends, and release reports in markdown or JSON. Returns the report content as a string. The summary report carries a runs block of total, passed and failed, which counts LAUNCHES and does not add up: a launch stored as finished, running or interrupted is in total and in neither bucket, exactly as get_launches_statistics explains. It also carries a tests block of total, passed, failed, skipped, flaky and passRate, where passRate is (passed + flaky) / total: flaky counts as a pass because the test passed on a retry, and total excludes skipped, so passed + failed + flaky adds back to total and the figure can be checked by hand. It is floored to two decimals rather than rounded, so a period with a single failure never prints as a flat 100 - which also means it can sit a hundredth below the same rate from get_launches_statistics, which rounds. NOTE: reportType "release" requires the releaseId argument (get it from list_releases) and ignores days, since it covers everything linked to the release however old. It reports manualTesting with runCount, totalTests, passed, failed, blocked, inTesting and notExecuted, and automation with launchCount, totalTests, passedTests, failedTests, skippedTests, flakyTests and passRate.

ParametersJSON Schema
NameRequiredDescriptionDefault
daysNoLookback period for report data
formatNoOutput formatmarkdown
projectIdYesProject UUID to generate report for
releaseIdNoRelease UUID (required for release report type)
reportTypeNoType of report: summary (overview), failures (failed tests), trends (pass rate over time), release (release status)summary

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavior beyond the annotations: runs counts launches and does not add up correctly, passRate counts flaky as a pass, passRate is floored rather than rounded, and release reports ignore days. These are non-obvious behavioral details that the schema and annotations do not provide. There is no contradiction with the readOnly, idempotent, non-destructive annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded, and almost every detail contributes important behavioral context. It is dense and long, but not padded. A little restructuring with clearer separation between report-type notes would improve scannability, so it is strong but not a perfect 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a report tool with no output schema, the description explains that the return value is a string and details both summary report fields and release report fields. The failures and trends report contents are left to inferrence from the reportType enum, which is acceptable but slightly incomplete. Overall, the description is robust enough for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All five parameters are already described in the schema, so the baseline is 3. The description adds meaningful semantics around reportType, the release/releaseId relationship, and the difference between summary and release report contents. That goes beyond the schema's short field descriptions, earning a 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific action and resource: 'Generate a formatted report for a project.' It then enumerates the supported report types and output formats, making it easy to understand both what the tool does and how it differs from sibling tools that return raw metrics such as get_launches_statistics or get_launches_trends.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete usage conditions: reportType 'release' requires releaseId from list_releases, and days is ignored in that mode because release reports cover the entire release. It also references get_launches_statistics to clarify shared semantics. It does not explicitly say when to choose this tool over sibling reporting/data tools, but the context is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_case_execution_historyA
Read-onlyIdempotent

Get how one manual test case has fared over time, across every Manual Test Execution and release it appeared in, newest first. Returns executions and summary. The headline figures are NESTED under summary, not at the top level: summary carries totalExecutions, executedCount, passed, failed, blocked, inTesting, notExecuted and passRate. totalExecutions counts memberships, including times the case was added and never run. executedCount is every row that has actually been touched, which INCLUDES rows still in_testing, so passed + failed + blocked + notExecuted can come to less than totalExecutions and the shortfall is inTesting. passRate is passed divided by executedCount, so an unfinished in_testing row sits in the denominator and drags the rate down. That denominator is not the one list_executions uses: its progress.passRate divides by the total case count, counting never-run cases against the rate, so the two figures share a name and are not comparable. Each row carries id, status, runId, runName, executedBy, executedAt, notes and source. source is a STRING, not an object: it reads either "standalone", and then the row also carries executionId and executionName as siblings, or "release", and then it carries releaseId, releaseName and releaseVersion instead. Compare it as a string - row.source === "release" - because row.source.type is undefined on every row and filters silently to nothing. A row whose source is release carries releaseId, releaseName and releaseVersion but no executionId, even though it does sit inside a Manual Test Execution. It can still be traced there, by either of two joins: the row id is the same value get_execution_cases returns as executionCaseId, and the row runId is the same value get_execution returns as sources[].id. Both joins run FROM an execution, though, and nothing takes a runId or an executionCaseId and hands back the execution holding it. Use get_release on the releaseId the row does carry and match against its linkedExecutions: list_executions has NO releaseId filter, and an unrecognised argument is ignored rather than rejected, so passing one there silently returns every execution in the project. IMPORTANT: limit bounds the summary as well as the list, so those figures describe the returned page rather than the whole window, and days defaults to 30, so a case last tested longer ago returns almost nothing and reads as never tested. Pass days explicitly; 365 is the ceiling, so for a case older than a year there is no way to ask for all of its history at once. This covers MANUAL results only: use get_test_history for an automated test across launches, and get_execution_cases for the rows inside one execution.

ParametersJSON Schema
NameRequiredDescriptionDefault
daysNoLookback period in days
limitNoMaximum number of executions to return
caseIdYesInternal numeric test case id (the "id" from list_test_cases / get_test_case / create_test_case). This is NOT the user-facing "TC-N" number (caseNumber/displayId) - if the user gave you a "TC-N", resolve it first via list_test_cases(search: "TC-N") and use the returned id. Project is derived from the case.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description's existing annotations declare readonly and non-destructive, and the description goes far beyond them by revealing the exact scenarios: summary figures are nested rather than top-level, passRate divides by executedCount while list_executions's progress.passRate divides by total case count, source is a STRING whose .type is undefined and filter silently to nothing, and totalExecutions counts memberships including never-run entries. It also discloses that limit bounds the summary aggregates, not just the list, and that no inverse join exists from runId/executionCaseId back to the owning execution — all non-obvious behavior that an agent would otherwise be unaware of.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The content is dense and earns its place, but the description is a single wall of text of roughly 400 words with no paragraph breaks, no bulleted structure, and several compressed explanations repeated (the source-is-a-string caveat is described twice, the passRate denominator difference appears both in the middle and near the end). The opening sentence is front-loaded and strong, but the mid-portion is very hard to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the full burden of explaining the return shape — and it does so thoroughly for both the row fields (id, status, runId, runName, executedBy, executedAt, notes, source with its standalone/release variants) and the summary fields — totalExecutions, executedCount, passed, failed, blocked, inTesting, notExecuted and passRate. It also documents edge cases (never-run memberships, inTesting shortfall, 365-day ceiling, both join paths and the lack of a reverse lookup) and identifies which sibling covers what, so nothing that an agent needs to invoke and interpret the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for all three parameters, so the baseline is 3. The description contributes additional semantics beyond schema: days at its default of 30 can produce a near-empty result that falsely reads as 'never tested,' 365 is a hard ceiling with no way to query all history at once, and limit bounds the summary aggregates rather than just page size — none of which the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific verb ('get'), a precise resource ('how one manual test case has fared over time'), and an explicit scope ('across every Manual Test Execution and release it appeared in, newest first'). It further differentiates itself from siblings by stating that only MANUAL results are covered, so an agent can distinguish it from get_test_history and get_execution_cases without opening their schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly routes to alternatives: 'use get_test_history for an automated test across launches, and get_execution_cases for the rows inside one execution.' It also warns against the tempting workaround of passing a releaseId to list_executions, since that tool has no releaseId filter and silently ignores unrecognized arguments, and instructs the caller to pass days explicitly because the 30-day default makes older cases read as never tested.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_coverage_gapsA
Read-onlyIdempotent

Identify test coverage gaps across a whole project. Every count here is of LIVE cases: archived ones are excluded throughout, and a project can hold several times its reported total once includeDeleted is passed to list_test_cases. Returns summary (totalCases, linkedCases, unlinkedCases, automationRate), foldersWithLowCoverage (folderId, name, caseCount, linkedCount, automationRate) for folders under automationThreshold - counted over the cases held DIRECTLY by each folder, so leaf suites appear and the containers above them do not, whatever sits nested beneath - unlinkedCases (caseId, displayId, title, status, priority, folderName) for cases with no linked autotest, and the reverse gap unlinkedAutotests, meaning autotests seen in the most recent launches that are linked to no case. IMPORTANT: unlinkedCases is an array but unlinkedAutotests is an OBJECT, shaped launchesScanned, totalUnlinked and items. items is hard-capped at 50 rows however large totalUnlinked is and whatever limit you pass, since limit sizes unlinkedCases and does not reach it, and it is drawn only from the last recentLaunches launches, 10 by default. So a title missing from items does NOT mean it is linked: it may be beyond the cap or older than the window. Two different definitions of linked are at work in one response, so read them apart. unlinkedAutotests ALONE uses the broad one: a test counts as linked there if it was linked with link_autotest_to_case OR if its title carries a [TC-N] marker that ingest resolved onto the test row. summary.linkedCases, summary.automationRate, foldersWithLowCoverage and unlinkedCases all use the narrow one and see only explicit links, which is also what get_test_case and the automationStatus filter of list_test_cases see. A case can therefore be counted unlinked in the same payload whose unlinkedAutotests treats its marker-bound test as covered. To settle one title, read get_test_case with includeRecentExecutions: a marker-bound test shows up there against its case and nowhere else, and search_tests will not help because it returns no linkage fields at all. totalUnlinked counts DISTINCT test titles, not rows or attempts, and is scoped to the same recentLaunches window as items rather than to the whole project. Read-only, nothing is written; close a gap afterwards with link_autotest_to_case. Pick this when the question is which cases or folders lack automation project-wide. Prefer get_feature_scan when the scope is a single Jira issue and Jira is connected, run_health_check when you want one scored assessment in which coverage is only one dimension, search_tests when you already know the autotest and want its results, and suggest_test_cases when the gap is a case that does not exist yet rather than a missing link. projectId comes from list_projects.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of unlinked cases to return
projectIdYesProject UUID to analyze
recentLaunchesNoHow many of the most recent launches to scan for autotests that are not linked to any test case (reverse coverage gap). Default and max: 10.
automationThresholdNoFolders below this automation % are flagged as low coverage (default: 50)
includeUnlinkedCasesNoInclude list of test cases without linked autotests

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, and the description reinforces 'Read-only, nothing is written.' Even with that coverage, the description adds substantial non-obvious context: LIVE-only counting with archived/stubs inference, two mutually exclusive definitions of 'linked' active in the same response, a hard 50-row cap on items, a recentLaunches window, and a precise disambiguation path via get_test_case. This is far beyond what annotations convey, and nothing contradicts them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded and almost every sentence carries unique information, but the middle section is one dense paragraph mixing output shapes, caps, windows, and linkage definitions, with contorted phrasing like 'since limit sizes unlinkedCases and does not reach it, and it is drawn only from the last recentLaunches launches.' Bulleted structure would meaningfully reduce parse effort for an agent. It is bloated rather than merely long.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the full burden of describing return values — and it fully carries it: it names all four output blocks (summary, foldersWithLowCoverage, unlinkedCases, unlinkedAutotests), lists the fields of each, explains the two linkage conventions, the 50-row cap, the window scoping, and the exact procedure for settling a single title with get_test_case. For a tool this intricate, even the edge cases (missing from items ≠ linked) are documented.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description genuinely adds meaning on top by linking parameters to output behavior: limit sizes unlinkedCases but does not reach the hard-capped unlinkedAutotests items, recentLaunches (default 10) scopes both totalUnlinked and items, and automationThreshold drives foldersWithLowCoverage aggregation. These semantics make the parameters' effects explicitly discoverable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb + object + scope: 'Identify test coverage gaps across a whole project.' It also differentiates from siblings later by contrasting the project-wide scope with the single-Jira-issue scope of get_feature_scan, so an agent can tell it apart without opening other schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Contains an explicit routing block: 'Pick this when the question is which cases or folders lack automation project-wide,' followed by named alternatives with their conditions (get_feature_scan for a single Jira issue, run_health_check for a scored assessment, suggest_test_cases for gaps that are nonexistent cases, search_tests for known autotests). Nothing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_executionA
Read-onlyIdempotent

Get one Manual Test Execution with everything attached to it. Returns execution (the container itself), sources (one entry per suite, Collection or case group added), releases, externalLinks, and progress. Each source carries id, name, sourceType, status and the case counts total, passed, failed, blocked, inTesting and notExecuted. Two traps in there. First, sources[].id is the id of the GROUP inside this execution, not the suite or Collection id you added: it is what remove_source_from_execution and set_execution_source_status take as runId. Second, execution.status and sources[].status come from two different vocabularies, execution_status and test_run_status, so not_executed and not_started are both correct in their own place; call get_project for the values a project actually uses. And both are SET BY A PERSON, not derived from the counts beside them: an execution can read passed with all four of its cases still not_executed, and a source can read completed with seven of seven unrun or blocked with nothing blocked. Never infer one from the other - read progress and the per-source counts for what actually happened. list_executions and get_release repeat these labels unchanged, so the same caution applies wherever they appear. progress carries the same counts across the whole execution PLUS passRate and completionRate, both whole percentages from 0 to 100. Their bases are not the same and the surprising one is passRate: it is passed divided by TOTAL, so every case not yet run counts against it and an execution with one pass, one failure and one unrun case reports 33 rather than 50. completionRate is (total - notExecuted) / total. The entries in sources carry the counts but not these two, so apply the same two formulas to a source if you need them per source. Use get_execution_cases for the individual case rows, which is also where report_case_result gets its executionCaseId, and list_executions to find an id.

ParametersJSON Schema
NameRequiredDescriptionDefault
executionIdYesExecution UUID

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Even though annotations already declare readOnlyHint and idempotentHint, the description adds substantial non-obvious behavior: statuses are manually set and not inferable from counts, execution.status and sources[].status use different vocabularies, and passRate is divided by total rather than by completed cases. It also warns that list_executions and get_release repeat these labels unchanged, extending the caution beyond this single tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: it fronts the return structure, then packages the traps, formulas, and sibling routing in clearly labeled units. The length is justified by the actual semantic complexity of the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, one-parameter tool with no output schema, this description is exceptionally complete. It covers return shape, field semantics, status vocabulary, count-to-status relationship, percentage formulas, and the exact way to obtain the required id. Nothing needed for safe invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers the single parameter fully with 'Execution UUID', so the description does not need to restate it. It does add a useful hint to use list_executions to find an id, but that is more usage guidance than expanded parameter semantics. Baseline 3 is appropriate because the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb-resource pair, 'get one Manual Test Execution', and itemizes exactly what the tool returns: execution, sources, releases, externalLinks, and progress. It also differentiates itself from siblings by explicitly pointing to get_execution_cases for individual case rows and list_executions for finding an id.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit routing guidance: use get_execution_cases for individual case rows, use list_executions to find an id, and warns that sources[].id is what remove_source_from_execution and set_execution_source_status consume. This is exactly the when-to-use-this-vs-alternatives information an agent needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_execution_casesA
Read-onlyIdempotent

List the individual case-run rows of a Manual Test Execution so you can report results against them. Returns each row with executionCaseId (a UUID, and the value report_case_result takes), caseId, caseNumber, title, group and groupId, sourceType, current status, the case-level result fields (notes, causedBy, durationMinutes, executedBy, executedAt), parentCases, the externalLinks of the case (Jira or Linear tickets, present only if you also hold integrations:read), and steps (0-based index, action, expected, current step status, issueDescription). groupId is the one to keep: it is what report_case_result wants when the same case sits in more than one source group, while group is only that group name. sourceType says whether the group came from a suite, a Test Collection or a bare set of cases. Note that the title field is called title here while get_case_execution_history calls the same thing caseTitle. caseId is null on rows whose underlying case has been deleted, so target those by executionCaseId, which is always present. executedBy here is the internal user UUID of whoever last recorded a result, not a name, and nothing on this surface resolves a user UUID - get_case_execution_history returns that same field for the same rows as an EMAIL ADDRESS, so call that one when you need to say who. It can be null even where executedAt is set, on seeded or imported results, and it records who last WROTE a result rather than who owns the case: a case row carries no assignee at all. durationMinutes is written only by report_case_result and has no control in the UI, so it is null on most rows. Everything report_case_result writes reads back here. Use after add_suites_to_execution, add_collections_to_execution or add_cases_to_execution, and before report_case_result. Supports paging and filtering by group, status or title.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
offsetNo
searchNoOptional case-title search
statusNoOptional case-status filter (e.g. not_executed, passed, failed)
groupIdNoOptional: only cases in this source group (execution_test_run UUID)
executionIdYesExecution UUID

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnlyHint and idempotentHint are true, and the description goes well beyond that: it explains nullability of caseId and durationMinutes, the UUID-vs-email executionBy discrepancy, conditional externalLinks based on integrations:read scope, and that report_case_result writes are visible here. This gives an agent a strong behavioral model of the tool without contradicting the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense, but nearly every sentence carries actionable information about return fields, nullability, or caveats. It is not a model of brevity, but because there is no output schema, the field-level detail is justified and the core workflow guidance is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only listing endpoint with no output schema, the description covers the returned fields, their types/nullability, permissions, workflow placement, and how this tool differs from get_case_execution_history. No important usage dimension appears to be missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents search, status, groupId, and executionId, while limit and offset rely on defaults and bounds rather than descriptive text. The description compensates by explaining that pagination and filtering by group/status/title are supported and by establishing groupId as the significant identifier for report_case_result, which is exactly the kind of semantic weight an agent needs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'List the individual case-run rows of a Manual Test Execution' and adds a clear purpose ('so you can report results against them'). It also distinguishes itself from get_case_execution_history by pointing out the title field naming difference and the user UUID vs email difference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states where this tool fits in a workflow: use it after add_suites_to_execution, add_collections_to_execution, or add_cases_to_execution, and before report_case_result. It also directs the agent to get_case_execution_history when a user UUID needs to be resolved to an email address, giving a clear alternative-selection rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feature_scanA
Read-onlyIdempotent

Get a comprehensive feature health scan by Jira issue key. Returns Jira context (summary, status, type, assignee), test coverage stats (case count, automation ratio, pass rate, risk score), linked test cases, and autotest results with recent run trends. UNITS: automationRatio, executionCoverage, autotestPassRate and manualPassRate are FRACTIONS from 0 to 1 here, so 0.2 means 20 percent. Every sibling answers in percent from 0 to 100 - get_coverage_gaps automationRate, run_health_check - so multiply before quoting one beside the other. IMPORTANT: each case row carries BOTH definitions of linked at once. automationStatus and linkedAutotestCount, and the automationRatio built from them, count only explicit links made with link_autotest_to_case, while the same row launches and lastAutotestResult also reflect a test bound by a [TC-N] title marker. So a row reading automationStatus manual with linkedAutotestCount 0 can sit beside passing automated launches: that case IS covered and was simply never linked explicitly. Read the row launches before calling a case uncovered, and treat automationRatio as a floor. Automatically detects issue hierarchy and aggregates child issues (2 levels deep: epic→story→subtask). Requires Jira to be connected to the organization.

ParametersJSON Schema
NameRequiredDescriptionDefault
jiraKeyYesJira issue key (e.g., PROJ-111). Can be any issue type - epics, stories, tasks, bugs, sub-tasks. Epics automatically aggregate all child issue data.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Even with annotations already marking this as read-only, idempotent, and non-destructive, the description adds substantial behavioral detail: it documents unit differences versus siblings, explains that automationRatio is a conservative floor, reveals the dual meaning of 'linked' between explicit links and title-marker links, and states the aggregation depth. Nothing in the description contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but every section earns its place: the purpose is front-loaded, and the UNITS and IMPORTANT sections are clearly labeled warnings that prevent real misuse. It reads like a structured briefing rather than a rambling explanation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite missing an output schema, the description is unusually complete: it lists the main return categories, explains units, describes the nuanced interpretation of linked test cases, notes hierarchy aggregation, and states the Jira connection prerequisite. An agent has enough context to invoke this tool and interpret its results accurately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already describes jiraKey fully at 100% coverage, so the baseline is 3. The description adds value beyond the schema by specifying the aggregation depth ('2 levels deep: epic→story→subtask') and by making clear that the single key drives the whole feature health result, including linked cases and autotest trends.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get a comprehensive feature health scan by Jira issue key.' It then enumerates concrete return contents such as Jira context, coverage stats, linked test cases, and autotest trends. It also clarifies how this tool differs from other health-related tools by highlighting its unique unit convention and its aggregation of child issues.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when this tool is appropriate, especially for 'comprehensive' feature health and multi-level issue aggregation. However, it does not explicitly say when to prefer siblings like get_coverage_gaps or run_health_check, aside from noting they return percentages rather than fractions. There are no explicitly stated exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_folderA
Read-onlyIdempotent

Get one folder or suite by id. Returns id, name, type, parentId, projectId, projectName, path, depth, casesCount, childFolderCount, displayOrder, createdAt and updatedAt. casesCount counts cases DIRECTLY in this folder, so a container folder reports 0 however much sits beneath it; childFolderCount tells you whether that is the case. Set includeStats for a statistics object over the folder AND everything nested under it, which is where totalCases, automatedCases, automationRate, status and priority breakdowns, tag usage and time-based metrics live: expect statistics.totalCases to exceed casesCount. Cases whose status or priority is null are left out of those two breakdowns, so statusCounts and priorityCounts need not sum to totalCases, and the shortfall is the count of unset ones rather than an error. Set includeCases for the most recent cases directly in it, capped by casesLimit; they come back under recentCases, not cases. Every count here is of LIVE rows: archived cases and archived subfolders are excluded from casesCount, childFolderCount and the whole statistics object, so a suite reporting 45 can still answer with several times that through list_test_cases with includeDeleted. Use list_folders to see the tree or find an id; use this when you have one id and want detail.

ParametersJSON Schema
NameRequiredDescriptionDefault
folderIdYesFolder UUID to get. Obtain from list_folders.
casesLimitNoMaximum number of cases to include (when includeCases=true)
includeCasesNoInclude list of recent test cases in folder
includeStatsNoInclude detailed statistics (automation rate, status/priority breakdown)

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations establish read-only, idempotent, and non-destructive behavior. The description adds a wealth of nuanced behavior: casesCount only counts direct children, statistics cover nested items, null status/priority are excluded, and archived rows are excluded from all counts. This goes far beyond the annotations and is essential for interpreting results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is lengthy but each sentence and clause carries a distinct fact about scope, lifetime, or counting semantics. It front-loads the core summary of returned fields and then systematically explains the flags and edge conditions. No filler or repetition exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This tool has no output schema, so the description fully compensates by enumerating returned fields and the caveats of archived data, direct versus nested case counts, and shortcut behavior. It also gives alternatives for cases requiring archived data or the full tree. An agent has enough information to call and interpret this tool reliably.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all four parameters, so the baseline is 3. The description provides meaningful extra semantics: includeStats yields a statistics object over the whole subtree, includeCases returns recent cases under recentCases rather than cases, and casesLimit caps that list. It even clarifies that folderId comes from list_folders.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Get') and resource ('folder or suite by id') and explicitly lists the returned fields. It distinguishes get_folder from list_folders, and the name and description clearly frame it as a detail-fetch tool rather than a tree/navigation tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use list_folders to see the tree or find an id; use this when you have one id and want detail.' It also notes when archived data is needed, pointing to list_test_cases with includeDeleted, giving the agent clear routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_launches_statisticsA
Read-onlyIdempotent

List the launches of a project, newest first, each with its own pass, fail, skip and duration figures. Every launch row carries id, launch, status, startedAt, endedAt, durationSeconds, tags and stats - the NAME of the launch is in a field called launch, not title or name. The status is REPORTED by the CI job rather than derived from the numbers beside it, and it runs BOTH ways. A launch can be stored failed with stats.failed 0, because the job failed for a reason no test recorded. And a launch that reported nothing at all can be stored passed with stats.total 0 - one here sits 75 minutes after a real run that failed 145 tests, so reading the newest launch as green clears nothing and the earlier failures still stand. Check stats.total before treating a green newest launch as evidence. Never infer one from the other; to answer what failed, read the tests with get_launch_tests. Every launch row also carries a stats object of total, passed, failed, skipped, flaky, retries and running. total EXCLUDES skipped even though skipped is printed beside it, so the identity that holds is passed + failed + flaky = total; retries counts retry ATTEMPTS, not tests that were retried, which is why it can exceed flaky. Returns launches, a summary over the whole window (totalRuns, passedRuns, failedRuns, runningRuns, runPassRate, totalTests, totalPassed, totalFailed, totalSkipped, totalFlaky, testPassRate, avgDurationSeconds), the period and filters applied, and pagination with the real total; there is no top-level total. Read the two rates carefully, because they answer different questions and neither denominator is obvious. runPassRate is the share of LAUNCHES that ended green, out of totalRuns - and totalRuns is EVERY launch in the window. Only three of the end statuses have a bucket - passedRuns, failedRuns and runningRuns - so a launch stored as finished, interrupted, timedOut or completed is counted in totalRuns and in none of them, and the three need not add up to totalRuns. That shortfall is why get_launches_trends and run_health_check answer a slightly higher run pass rate over the same window: they count only launches that passed or failed, so their denominator is smaller. testPassRate is the share of TESTS, computed as (totalPassed + totalFlaky) / totalTests: flaky counts as a pass, because a flaky test passed on a retry and the launch is green on it, and totalTests already EXCLUDES skipped, so totalPassed + totalFailed + totalFlaky is the whole of it while totalSkipped sits outside. Both rates are null, not 0, when the window held nothing at all - testPassRate with no tests, runPassRate with no launches - since nought percent would read as a verdict on work that never ran. IMPORTANT: days defaults to 30, so a project idle for longer returns an empty list, which means nothing in the window rather than nothing at all. Use limit, offset and search to page, statuses or tags to narrow. Reach for get_launches_trends when the question is about the project over time rather than about particular launches, and get_launch_tests when you have one launch and want the tests in it.

ParametersJSON Schema
NameRequiredDescriptionDefault
daysNoLookback period in days
tagsNoFilter by tags
limitNoNumber of launches to return
offsetNoPagination offset
searchNoSearch the launch NAME (the launch field on each row), its id, or its tags. Launches carry no trigger metadata at all - nothing records that a run was nightly, on merge or manual, and there is no branch or CI-job field - so a request naming a run by its schedule can only be resolved through a name or tag the reporter happened to set, or by startedAt. If a name search comes back empty, say the run cannot be identified rather than assuming the newest launch is the one meant: three launches here carry byte-identical stats.
statusesNoFilter by launch status. Set by the reporters through the ingest service, so this list is fixed and not project-configurable. A launch that ended without an explicit status, and whose totals did not reconcile, is stored as finished: that is usually an empty or half-reported launch, and it counts toward totalRuns while sitting in none of the passed, failed or running buckets.
projectIdYesProject UUID to query

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as readOnly, idempotent, and safe. The description goes well beyond these by exposing crucial statistical behaviors: statuses are reported by the CI job, not derived from the numbers; total excludes skipped; retries counts attempts; runPassRate and testPassRate have different denominators; rates are null when there is no data; and pagination has a real total with no top-level total. None of this contradicts the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is unusually long, but it is packed with distinct near-miss pitfalls that are directly relevant to correct invocation and interpretation. It front-loads the purpose and row schema, then expands into statistical pitfalls that an agent would otherwise never guess. A small amount of trimming and structural formatting could improve scannability, but no sentence felt purely filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Even though there is no output schema, the description reveals the entire launch object field set, the aggregated summary object's exact fields, the pagination behavior, the meaning of null rates, and the empty-list interpretation. It also covers how to distinguish this tool from related alternatives. For a statistical listing tool with this level of edge-case nuance, nothing needed for a careful call is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Though the schema already describes every parameter with 100% coverage, the description adds meaningful extra semantics: days means a lookback window and an empty list can mislead; search looks at the launch field, id, or tags and carries no trigger metadata; statuses have a fixed enum set; pagination through limit/offset is explained in context; statuses alone do not sum to totalRuns. This far exceeds the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'List the launches of a project, newest first', then enumerates the fields returned (id, launch, status, startedAt, endedAt, durationSeconds, tags, stats). It also explicitly differentiates from siblings by telling the agent when to use get_launches_trends and get_launch_tests, so the purpose is unmistakable even among many similar tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance: get_launches_trends for questions about the project over time, get_launch_tests for a single launch's tests. It also explains under what conditions an empty list means no data in the window (days defaults to 30) and warns that a name search matching nothing should be reported as 'cannot be identified' rather than assuming the newest launch. This is far beyond a generic read tool description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_launch_testsA
Read-onlyIdempotent

Get every test inside one launch, with results, error messages and stack traces for the failures. IMPORTANT: rows are one per retry ATTEMPT, not one per test, so a retried test appears more than once and the row count can exceed the launch stats counts, while the stats on the launch itself count tests rather than attempts. Two traps follow, and both have produced a wrong count. First, testId is minted per ROW, so every attempt has a different one and deduplicating on testId still counts attempts: the identity of a test across its attempts is titlePath, optionally with filePath. Second, retry is 0-based but a retry:0 row is NOT guaranteed to exist - a launch can hold a retry:1 row whose first attempt was never stored - so counting rows where retry is 0 undercounts. Count distinct titlePath. Each row also carries status, previousStatus, durationMs, tags and errors. previousStatus reads superseded on an attempt a later one replaced, which looks like the clean way to filter, and is not: on a measured launch of 4472 rows, dropping the superseded ones left 4460 against 4457 real tests, because a retried attempt can carry its own earlier result there instead. The launch stats embedded in this response follow the same convention as everywhere else: total EXCLUDES skipped, so a launch with total 4417 and skipped 40 really does hold 4457 distinct tests, and passed counts first-time passes with flaky held separately. Use this once you have a launch id, which get_launches_statistics gives you; use get_test_history to follow one test across launches instead. For each failed test that has artifacts the response carries an attachments array of id, name, contentType, size and url. The url is fully qualified: if it is an API URL, fetch it with the same X-MCP-Key header you use for this server, while presigned storage URLs need no header. That is how you reach screenshots, traces, videos and source files when investigating a failure.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoNumber of tests to return
offsetNoPagination offset
searchNoSearch in test title path or file path
launchIdYesRun/launch UUID to get tests from. Obtain from get_launches_statistics or get_launches_trends.
statusesNoFilter by ATTEMPT-ROW status, not by how a test ended (default: all). A test that failed once and passed on retry still returns its failed row here, so counting the filtered rows, or even counting distinct titlePath over them, answers how many tests had this status at SOME attempt - never how many ended that way. A launch whose own stats.failed is 0 can return 15 failed rows across 13 titles. For tests that ENDED failed, fetch the rows unfiltered and keep the highest-retry row per titlePath, or read stats.failed on the launch itself.
includeErrorsNoInclude error details for failed tests

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes far beyond annotations by exposing non-obvious behavior: rows are per retry attempt, retry is 0-based but a retry:0 row may not exist, testId is per-row rather than per-test, previousStatus 'superseded' is explicitly unreliable, and launch stats exclude skipped tests. It also documents authentication requirements for artifact URLs, which is valuable operational behavior not visible in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long, but it is front-loaded with a clear one-sentence summary followed by essential caveats. Nearly every sentence earns its place because the retry-attempt semantics are genuinely non-obvious and easy to get wrong. It is slightly verbose in explaining examples and traps, but the structure stays readable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, yet the description fully compensates by listing row fields (status, previousStatus, durationMs, tags, errors) and explaining the attachments array with url, name, id, contentType, size. It also covers pagination semantics indirectly through limit/offset in the schema and covers retry counting pitfalls, test-identity semantics, launch-stats conventions, and artifact authentication. An agent has enough context to invoke the tool and interpret results correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is already strong. The description adds real semantic value beyond the schema by explaining that statuses filters by ATTEMPT-ROW status rather than final test outcome, and by noting where launchId comes from (get_launches_statistics or get_launches_trends). It does not re-explain limit, offset, or search, which is appropriate since the schema covers them.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get every test inside one launch, with results, error messages and stack traces for the failures.' It also distinguishes this tool from related siblings by noting get_test_history is for following a test across launches and that get_launches_statistics is the source of the launch id. The caveat about one row per retry attempt further clarifies exactly what the tool returns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool: 'Use this once you have a launch id, which get_launches_statistics gives you' and provides the alternative for a different need: 'use get_test_history to follow one test across launches instead.' It also warns against using row-level filters for test-level counts, giving clear guidance on what this tool should not be used for.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_projectA
Read-onlyIdempotent

Get a project's configuration: its meta, the system-field vocabularies it actually uses (statuses/priorities/environments), and its case custom-field definitions. Call this to discover the VALID status and priority values before setting any of them: case_status and case_priority (test cases), execution_status (a Manual Test Execution overall), test_case_execution_status (a case inside one), test_run_status (a source group or a Test Collection), release_status (releases) and test_environment. Every one of them is project-customizable, so never hardcode a value or assume the stock list. Each entry is { value, label, isDefault }: send the VALUE to other tools, never the label, and isDefault marks what a create tool will use when you omit the field. Returns { project, systemFields{ case_status, case_priority, execution_status, test_case_execution_status, test_run_status, release_status, test_environment }, customFields[] }.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesProject UUID (from list_projects)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as read-only, and the description adds the useful behavioral detail that the returned vocabularies are authoritative, project-specific, and that callers must use the VALUE rather than the label elsewhere. It also explains isDefault semantics, which helps other tools behave correctly after this call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is somewhat verbose but every sentence contributes meaningful guidance. It is appropriately front-loaded with a clear summary, then details, usage warning, and return shape. The only minor downside is that it lists many field names inline, making the text denser than necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 1-parameter read-only configuration tool, the description is complete: it states the tool's purpose, when to use it, what the valid-looking fields mean, how defaults behave, and the full return structure. No additional details such as auth, side effects, or prerequisites are needed for this operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is only one required parameter (the project ID), and schema description coverage is 100%, so the schema mostly says what is needed. The description focuses on the output rather than how to provide the project ID, so it adds limited parameter meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Get a project's configuration') and its full scope: metadata, system-field vocabularies, and custom-field definitions. It clearly differentiates this discovery tool from the many sibling get_* and list_* tools by focusing on valid settings rather than a single entity's contents.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says when to use it: before setting any status or priority value. It warns not to hardcode values and explains that these vocabularies are project-customizable, which prevents misuse. This gives an agent clear selection and invocation guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_releaseA
Read-onlyIdempotent

Get one release by releaseId with full detail: name, version, status, projectId, projectName, dates, and a stats rollup for linked manual executions (count, total, passed, failed, blocked, inTesting, notExecuted) and linked automated launches (count, total, passed, failed, skipped). The status on a release is a label a person applied, not a verdict derived from these numbers, so read the rollup rather than the word. The manual rollup counts EXECUTIONS only: a Test Collection whose releaseId points at this release is not included and adds nothing to the counts, so a release can read total 0 while collections filed under it hold results. It also counts result ROWS rather than surviving cases: a row whose underlying case was deleted keeps its recorded result and stays in every figure here, showing up in get_execution_cases with caseId null - on one release 13 of the 16 reported passes are against cases that no longer exist. And rows are SUMMED across the linked executions with no deduplication, so a case in two executions counts twice and can read passed in one and not_executed in the other. Page get_execution_cases and count the null-caseId rows and the repeated caseIds before quoting these numbers anywhere that matters. Check list_test_collections with releaseId before reporting a release as untested. By default it also returns the linkedExecutions and launches arrays, each entry with its own name, status, dates and per-item stats; set includeLinkedExecutions or includeLaunches to false to drop an array and keep the response small, since the stats rollup comes back either way. Read the launches rollup carefully, because it does not follow the same convention as get_launches_statistics: passed here ALREADY INCLUDES flaky, there is no separate flaky field, and total excludes skipped - so passed + failed equals total exactly while skipped sits outside it, and adding skipped back appears to overshoot. Use list_releases instead to browse, search or filter the releases in a project and to obtain releaseId; it returns the same rollup per release but never those two arrays, so call get_release once the release of interest is known. Read only, nothing is modified. An archived release still resolves here, unlike list_releases which hides it unless includeDeleted is set; the deleted flag in the response says which it is.

ParametersJSON Schema
NameRequiredDescriptionDefault
releaseIdYesRelease UUID. Obtain from list_releases.
includeLaunchesNoInclude linked automated launches with stats
includeLinkedExecutionsNoInclude the linked manual executions with per-execution stats. Set false to keep the response small - the stats.linkedExecutions totals are returned either way.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes substantially beyond the readOnly/idempotent/destructiveHint annotations, revealing critical behavior: status is a person-applied label and not a derived verdict; manual rollups count executions only, result rows rather than surviving cases, and no deduplication; deleted-case artifacts show as null-caseId; and the launches rollup has special flaky/skipped semantics inconsistent with get_launches_statistics. This is rich, non-obvious, and fully consistent with the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but remarkably dense—every sentence adds a distinct caveat, envelope alternative, or semantic clarification that an agent needs before trusting the output. It fronts the core detail, then progressively deepens into data-quality warnings and finally routing guidance, with no filler to trim.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema, the description names the return payload, explains rollup semantics, warns about misleading counts, provides the exact lookup method to troubleshoot archived releases, and routes agents to list_releases for browsing. It is difficult to think of a required input-output or edge-case detail missing for successful invocation and interpretation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, and the description adds meaningful context on top: releaseId should be obtained from list_releases, includeLinkedExecutions/includeLaunches control only the payload arrays while the stats rollup still comes back either way. This is more than simple schema restatement, though less than exhaustive since nested response shape is covered only narratively.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and target resource—'Get one release by releaseId with full detail'—and immediately enumerates the returned fields. It also distinguishes itself from list_releases by noting that list_releases browses, searches, and filters while get_release fetches a single already-identified release.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly instructs when to use list_releases instead (browse, search, filter, obtain releaseId) and when to call get_release once the release is known. It also explains when to pass includeLinkedExecutions and includeLaunches false to keep the response small, and warns about archived releases versus list_releases requiring includeDeleted.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_caseA
Read-onlyIdempotent

Get one manual test case in full: id, caseNumber, displayId, title, description, status, priority, tags, ordered steps, folderId/folderName, projectId/projectName, dependsOn, customFields, createdAt/updatedAt and a shareUrl for pasting into Slack or Jira. Optional flags add linkedAutotests (includeLinkedAutotests, default true), recentAutotestExecutions from CI runs (includeRecentExecutions) and an edit-history audit trail (includeHistory). Read-only; nothing is modified. caseId is the internal numeric id from list_test_cases / create_test_case, not a user-facing TC-N; the project is derived from the case. Choose list_test_cases instead to browse, filter or count many cases, or to turn a TC-N into an id - it returns summaries without steps, so use it to pick the case and this tool to read it. Choose get_case_execution_history instead for how the case has fared in Manual Test Executions over time (pass rate plus an execution timeline); includeRecentExecutions here reports automated CI results only. search_tests covers automated tests, which are separate records from manual cases. IMPORTANT if you plan to write back: description and steps are RICH TEXT and do not round-trip as they arrive. Both come back as arrays of formatting nodes rather than strings, and a step reads as title and expected while create_test_case and update_test_case take action and expected. So sending a step straight back sets the wrong field to the wrong type. Send plain strings or Markdown on write and the server converts them; to edit one step, rebuild the whole steps array from your own text, since steps is replace-all.

ParametersJSON Schema
NameRequiredDescriptionDefault
caseIdYesInternal numeric test case id (the "id" from list_test_cases / get_test_case / create_test_case). This is NOT the user-facing "TC-N" number (caseNumber/displayId) - if the user gave you a "TC-N", resolve it first via list_test_cases(search: "TC-N") and use the returned id. Project is derived from the case.
includeHistoryNoInclude the edit audit trail: field, oldValue, newValue, changeType, userName, userEmail and changedAt per entry. It is NOT a complete account of the case, for two separate reasons. COVERAGE: the trail holds only the changes it audits, and updatedAt can be newer than every entry in it - one case here carries updatedAt two months after its latest recorded change - while the case itself has no updatedBy field. VOLUME: the array is capped at the 50 most recent entries, with no truncation flag, no total and no way to page further back, so a trail of exactly 50 IS a truncated one and its oldest entry is not the oldest change. Two traps inside the rows as well: an entry records that a field was SUBMITTED rather than that its value moved, so oldValue and newValue are often identical - 28 of 50 on one case here - and userName is frequently just the email address again. So this answers what changed and by whom AMONG AUDITED, RECENT changes, and cannot always answer who last touched the case.
includeLinkedAutotestsNoInclude autotest links made with link_autotest_to_case. These are the only links reported here, and they are not the only links that exist: a reporter can also bind a test to a case by putting a [TC-N] marker in the test title, which ingest resolves to a case id on the test row itself. get_coverage_gaps counts that second kind as covered in its unlinkedAutotests section ONLY: its summary, automationRate, foldersWithLowCoverage and unlinkedCases all use the same narrow, explicit-link definition this list does. So a test can be absent here and still be linked, and a case can sit in unlinkedCases while a marker-bound test covers it, which makes automationRate a floor rather than the whole picture.
includeRecentExecutionsNoInclude autotest results for this case from CI. There is NO time window: the latest result per distinct test title is returned, up to 20, with no date cutoff at all - so an empty array is DEFINITIVE and means no automated test has ever reported against this case, rather than none lately. This is also the only place a test bound to the case by a [TC-N] title marker appears, which makes it the arbiter when linkedAutotests comes back empty.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark readOnlyHint/idempotentHint/destructiveHint, and the description is fully consistent — yet it adds a lot of disclosure beyond the annotations: the exact return field set, that 'nothing is changed', the history caveats (50-entry cap with no truncation flag, oldValue and newValue are often identical), the no-time-window execution semantics ('empty array is DEFINITIVE'), and the rich-text round-trip trap where sending a read step back corrupts the write. No contraction found.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Extremely information-dense; every sentence carries real operational value — the sibling routing, the definitive empty-array semantics, the marker-bound-line caveat, the replace-all steps warning. The most critical usage note ('caseId is the internal numeric id… not a TC-N') is appropriately prominent at the end of the first part. It loses one point for being a single long monolithic paragraph — splitting the sibling routing, the link nuances, and the write-back warning would make it easier to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a high-complexity tool — 4 params, 3 optional flags with surprising semantics (no time win dow, 50-entry cap, explicit-versus-marker links), one required non-obvious id, read-only with a write-back footgun, and no output schema. The description covers all of these: the shared return list, the suffix-share-difference between includeHistory and includeRecentExecutions, the exotic-autotest binding discrepancy, the rich text versus markdown asymmetry, and the page ambiguity of the id. Nothing of consequence is left to inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline of 3 applies; the schema already picks up each parameter in detail (caseId as internal id vs TC-N, defaults, semantics of includeHistory/includeRecentExecutions/includeLinkedAutotests). The main description only repeats this framing ('Optional flags add…') and points the id semantics, without adding anything beyond what the schema provides. It does not compensate further nor does it need to.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Get one manual test case in full') and an explicit field list (id, caseNumber, title, steps, folderId/folderName, shareUrl, etc.) that precisely bounds the operation. It also differentiates itself from the most confusable siblings — list_test_cases, get_case_execution_history and search_tests — within the same description, so there is no ambiguity about which tool is which.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit decision rules with alternatives: 'Choose list_test_cases instead to browse, filter or count many cases, or to turn a TC-N into an id', 'Choose get_case_execution_history instead for how the case has fared in Manual Test Executions over time', and flags that search_tests covers a separate record type. The 'IMPORTANT if you plan to write back' warning further conditions the behavior expected of the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_collectionA
Read-onlyIdempotent

Get one Test Collection by UUID. Returns metadata (name, status, priority, environment, assignedTo, project, folder, release, dates), stats (totalTests, passed, failed, blocked, notExecuted, skipped, inTesting), and unless includeTestDetails is false a tests array of caseId, caseNumber, caseTitle, casePriority, status, executedBy, executedAt, notes. Each of those rows also carries an executionId, which is the id of the row itself inside this Collection and NOT a Manual Test Execution id: passing it to get_execution finds nothing. Read-only, changes nothing. collectionId comes from list_test_collections, which is the tool for finding or comparing collections across a project rather than inspecting one. For a Manual Test Execution (the Executions-page container) use get_execution and get_execution_cases instead - a different entity, and only the rows returned by get_execution_cases can be reported against with report_case_result.

ParametersJSON Schema
NameRequiredDescriptionDefault
collectionIdYesTest collection UUID. Obtain from list_test_collections.
includeTestDetailsNoInclude list of test cases with execution status

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/idempotent annotations, the description adds crucial behavioral context: the executionId is the row's own ID and not a Manual Test Execution ID, so passing it to get_execution would fail. It also discloses the conditional nature of the tests array based on includeTestDetails. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but dense with useful distinctions. It is front-loaded with the core action and resource, and every sentence conveys a worthwhile fact, though a few phrases are slightly redundant with the annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description fully compensates by listing metadata, stats, and the tests array structure, including the crucial executionId caveat. The agent has everything needed to invoke the tool and interpret the result correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description enriches parameter understanding by stating where collectionId comes from and clarifying that includeTestDetails controls the tests array. This adds value beyond the schema's property descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Get one Test Collection by UUID') and enumerates exactly what is returned. It explicitly distinguishes this tool from get_execution and list_test_collections, so an agent can reliably tell them apart.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit routing guidance: use list_test_collections to find collections, use get_execution/get_execution_cases for Manual Test Executions, and clarifies that executionId rows are not execution IDs. This is exemplary when-to-use versus alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_historyA
Read-onlyIdempotent

Get the history of one AUTOMATED test across launches, newest first. Returns executions and summary - the attempt rows are in executions, matching get_case_execution_history, and there is NO history key. The headline figures are NESTED under summary rather than at the top level: totalExecutions, passRate, flipCount and reliabilityStatus. titlePath takes ONE segment of the path - the argument is titlePath, not title - and matching is a case-insensitive substring, so it can hit many tests. The response says how many: matchedTests is the number of DISTINCT tests the substring found, matchedTitlePath is the one these figures describe (the most recently seen), and otherMatches lists up to ten of the rest. A common leaf title such as should reject unauthenticated request matches 25 tests in one project here. When matchedTests is above 1, narrow the substring before quoting anything: the summary describes matchedTitlePath alone, never the group. IMPORTANT: there is one row per retry ATTEMPT, not per launch, so a test retried three times inside one launch contributes three rows; the retry field is 0-based and tells them apart. Group by runId before quoting anything as a per-launch figure, because totalExecutions and passRate are attempt-level. Each row carries testId, titlePath, filePath, runId, runName, runStartedAt, runTags, status and durationMs. reliabilityStatus is derived from the pass ratio over the window and does NOT separate a test that fails every retry from one that passes on a later retry; read retry and flipCount together to tell broken from flaky. limit bounds the summary as well as the list, and days defaults to 30. search_tests will find a segment for you. Use get_case_execution_history for how a MANUAL case has fared, and get_launch_tests for every test inside one launch rather than one test across many.

ParametersJSON Schema
NameRequiredDescriptionDefault
daysNoLookback period in days
limitNoMaximum number of executions to return
projectIdYesProject UUID
titlePathYesONE segment of the title path, normally the test name itself (e.g. "should validate credentials"). Matching is a case-insensitive substring of the stored title-path array, so a " > "-joined path matches nothing and returns an empty history rather than an error. search_tests and get_launch_tests return titlePath as an array - pick one segment out of it.

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite readOnlyHint=true, the description adds substantial behavioral detail not inferable from annotations: one row per retry attempt, 0-based retry field, no history key, summary nesting, matchedTests count, matchedTitlePath selection, and reliabilityStatus limitations. It fully discloses edge cases and response semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence conveys a distinct, necessary caveat or fact. It opens with the core purpose, then systematically addresses response structure, common pitfalls, retry semantics, and alternative tools. No redundant phrases, and the warnings are grouped logically.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema, the description thoroughly explains the return shape, nested summary fields, edge cases, and interpretation guidance. It covers all four parameters, identifies behavioral pitfalls, and routes to sibling tools where appropriate. Despite length, it leaves little room for an agent to misinvoke.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description goes far beyond schema text by explaining titlePath is a single segment, matching is case-insensitive substring, how substring ambiguity works, and how limit and days affect the response. It clarifies the relationship between matchedTests and matchedTitlePath, which is essential for correct parameter use.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Get the history of one AUTOMATED test across launches, newest first.' It clearly distinguishes from related tools by defining the exact scope (one test across launches) versus get_case_execution_history for manual cases and get_launch_tests for all tests in one launch.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use and when-not-to-use guidance, naming alternatives: 'Use get_case_execution_history for how a MANUAL case has fared, and get_launch_tests for every test inside one launch rather than one test across many.' It also warns when multiple tests match and advises narrowing the substring.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_executionsA
Read-onlyIdempotent

List Manual Test Executions for a project. Each row returns id, name, status, priority, environment, assignedTo, startDate, endDate, createdAt and progress. There is NO updatedAt on an execution, here or in get_execution, and startDate and endDate are planning dates a person types and are usually null - so createdAt cannot tell you when an execution was last worked on. For that, read get_execution_cases and take the newest executedAt across its rows. assignedTo is a bare user UUID that nothing on this surface resolves to a person. Use this to find an existing execution id before adding sources. progress carries every case bucket (passed/failed/blocked/inTesting/notExecuted, which sum to total) plus passRate and completionRate, both whole percentages from 0 to 100. passRate here is passed divided by TOTAL, so every case that has not been run yet counts against it and a young execution reads low through no fault of its results. That is a different denominator from get_case_execution_history, whose passRate divides by the rows actually executed, so do not compare the two numbers.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
offsetNo
searchNoOptional name search
statusNoOptional status filter, matched EXACTLY and case-sensitively against the execution_status vocabulary of the project; call get_project for the values it uses, since a project can rename them or add its own. An unrecognised value is rejected rather than answering with an empty list, which used to be indistinguishable from a genuine nothing-found. Note this matches the label a person set on the EXECUTION, not what is inside it: an execution holding blocked cases usually does not itself read blocked, so to find blocked WORK list without a filter and read progress.blocked.
projectIdYesProject UUID

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description adds behavioral context beyond them: there is NO updatedAt on executions, startDate/endDate are human-typed planning dates usually null, assignedTo is a bare unresolved user UUID, and the progress.passRate denominator divides by total cases rather than executed ones. It openly discloses these semantic traps and other gotchas an agent must know before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence earns its place: no redundant filler, and the opening sentence identifies what the tool returns. However, it is a ~200-word wall of prose; a bulleted breakdown per field would be much more scannable, and the density requires careful parsing by the agent. Efficient but not optimized for scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates fully: it enumerates the returned fields and the shape of progress (five buckets that sum to total, pass`Completeness note: passRate/completenessRate). It also enumerates the confusing fields (updatedAt absence, planning dates, unresolved UUID). It omits straightforward context such as default ordering and whether the list is newest-first or how paging responds, but those are secondary for the tool(to-find-id) core use case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 60%: projectId and status are richly documented, with status getting substantial guidance (case-sensitive vocabulary, rejection, label-vs-content distinction), while limit and offset have no descriptions in the schema nor any in the tool description. The description itself adds little about parameters directly — most of its detail concerns output values and the status caveat is in the schema. Paging params remain underspecified for an unannotated numeric name.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and object: 'List Manual Test Executions for a project.' It names the exact return columns, scopes the tool to manual executions, and differentiates itself from get_execution and get_execution_case by explaining what it is not (no updatedAt, no per-execution extended details). An agent can distinguish this from all 50+ siblings without opening other schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit routing is given: 'Use this to find an existing execution id before adding anything sources.' It also gives an explicit when-not and alternative: when you need to know the last time an execution was worked, 'read get_execution_cases and take the newest executedAt'; and it warns the passRate here must not be compared with get_case_execution_history's. This is unambiguous selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_foldersA
Read-onlyIdempotent

List the folder and suite tree of a project as a flat array under folders, with a total count. Every row carries id, name, type (folder or suite), parentId, path, depth, casesCount, childFolderCount, displayOrder, createdAt and updatedAt, which is enough to find the suites without a second call. It is NOT enough to rank folders by size: casesCount counts only the cases directly in a folder, so it reads 0 on most containers and ranking by it surfaces the biggest leaf rather than the biggest branch. For a size ranking, rebuild the tree from parentId and sum the descendants, or call get_folder with includeStats for one subtree at a time. casesCount and childFolderCount count LIVE rows only, so archived cases and archived subfolders are invisible here unless you pass includeDeleted. Use this to browse or to find an id, then get_folder for one folder in detail. Suites are the leaf nodes that hold test cases; use list_test_cases for the cases themselves.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesProject UUID to list folders from
includeDeletedNoInclude archived/deleted folders

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as read-only and idempotent, so no safety contradiction. The description adds valuable behavioral context: casesCount only counts direct cases, childFolderCount and casesCount ignore archived data unless includeDeleted is passed, and the output contains enough fields to rebuild the tree. These are non-obvious behaviors an agent needs to know.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but every sentence adds useful information: output shape, field list, use cases, caveats about counting, and routing to siblings. It is front-loaded with the core purpose and then covers the two most important behavioral caveats. A slightly shorter version would be possible, but the density of useful guidance justifies the length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must stand in for return-value documentation. It does this thoroughly by enumerating all key fields, indicating the response structure as flat array under folders, and covering edge cases around archived content and hierarchical counting. It also integrates sibling routing, so detail can be retrieved with get_folder and test cases with list_test_cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already describes the two parameters, so the baseline is 3. The description earns an extra point by explaining the practical effect of includeDeleted: 'archived cases and archived subfolders are invisible here unless you pass includeDeleted.' It clarifies that the tree representation includes enough fields to derive paths and descendants from parentId, going beyond the raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'List the folder and suite tree of a project as a flat array under folders, with a total count.' It clearly identifies what the tool does, the kind of data returned, and how it differs from get_folder and list_test_cases by explicitly naming those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance on when to use this tool versus alternatives: 'Use this to browse or to find an id, then get_folder for one folder in detail' and 'use list_test_cases for the cases themselves.' It also explains when this tool is NOT appropriate, such as for size ranking, and names the better alternative with get_folder includeStats.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_projectsA
Read-onlyIdempotent

List the projects this key can reach. START HERE: almost every other tool needs a project UUID and this is where you get one. Admins see every project in the organisation, managers and users only the ones they are assigned to, so an empty list means no access rather than no projects. Ordered by the display order set in the UI first, then oldest created first, so do not read anything into the first entry beyond that. Returns each project with its id, name and metadata. Use get_project for one project in detail, including the status and priority vocabularies it actually uses, which you need before setting any status.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, and the description adds meaningful behavioral context beyond those: role-based visibility, the meaning of an empty list, and the display-order-then-created order. It also states the return fields (id, name, metadata), which helps the agent know what to expect before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence carries intent: the START HERE hook, access scope, empty-list interpretation, ordering caveat, return fields, and pointer to get_project. It is dense but not wasteful, and the most actionable guidance is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless listing tool, the description covers purpose, access behavior, ordering, return shape, and the correct next-step tool. There is no missing information that would prevent the agent from invoking it correctly or interpreting its results properly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is no parameter ambiguity for the description to resolve. The description instead clarifies the output fields and access semantics, which is useful given that no granular schema or output schema exists.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('List'), the resource ('projects'), and scopes it to what the current key can reach. It clearly distinguishes itself from get_project, which provides detailed single-project information, and its START HERE framing establishes it as the entry point for obtaining project UUIDs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent to start here because almost every other tool needs a project UUID. It also gives concrete interpretation guidance: an empty list means no access rather than no projects, and the ordering should not be over-interpreted. It routes to get_project for detail and for the actual status/priority vocabularies before setting status.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_releasesA
Read-onlyIdempotent

List releases for a project with aggregated stats for manual tests and automated launches. Returns project, a releases array, and pagination holding the real total; there is no top-level count or data field. Supports filtering by status and search. Reach for get_release when you have one id and want its linked executions and launches.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoNumber of releases to return
offsetNoPagination offset
searchNoSearch in release name, version, or description
statusNoFilter by release status. Values come from the project vocabulary, typically planned, in_progress, completed or cancelled, plus any the project added; call get_project for the list it uses. An unrecognised value is rejected rather than quietly ignored. Like an execution status and a launch status, a release status is SET BY A PERSON and never derived from the stats beside it: a release can read completed with half its cases unexecuted and its only linked launch failed, and a linked launch can postdate completedAt. Read the rollup for what actually happened.
projectIdYesProject UUID to list releases from
includeDeletedNoInclude archived releases

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly flag, the description discloses return shape ('eras array', 'pagination holding real total', no top-level count or data flag) and flags an important domain nuance about release status being SET BY A PERSON and potentially inconsistent with aggregate stats. This directly prevents an agent from misinterpreting the aggregated data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded; the first sentence states the core purpose, the second sets expectations about the return shape, and the third adds routing and filtering context. Every sentence supplies meaningful information with no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema, the description explains the top-level return structure and the pagination format, so the agent knows how to parse results. Combined with the detailed input schema, it is sufficient to call the tool correctly and understand how its results differ from related tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% description coverage for every parameter, including status vocabulary and rejection behavior. The description adds little about the parameters themselves beyond saying filtering by status and search is supported, so it stays at the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states a clear verb ('List') and target ('releases for a project'), and specifies it returns aggregated stats for manual tests and automated rseats. It also explicitly differentiates itself from get_release by noting get_release is for when a single flag id exists and linked executions or launches are needed.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The final sentence gives an explicit rule: use get_flag when you have one id and need its linked executions and launches, implying list_releases is for the broader project-level list. It also names possible returns/filtering ('by status and search') so an flag can decide this is the right entry point.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_test_casesA
Read-onlyIdempotent

List test cases in a project with filtering and pagination. Can filter by folder, status, priority, tags, or a search query. The cases come back in an array called cases, the available status and priority values under meta, and the row count under pagination.total; there is no top-level total. Every row also carries createdAt and updatedAt, and updatedAt tracks edits to the case DEFINITION only: recording a manual result, or an automated test running against the case, does not move it, so updatedAt equal to createdAt means never edited rather than never used. There is no lastExecutedAt on a row - for when a case was last actually run, use get_case_execution_history for manual results or get_test_case with includeRecentExecutions for automated ones. Every case row also carries hasLinkedAutotests and linkedAutotestCount, which is the cheapest way to answer whether a case is automated - they count links made with link_autotest_to_case, the same narrow definition the automationStatus filter uses and the same one behind get_coverage_gaps summary and automationRate. Only the unlinkedAutotests section of that tool additionally treats a [TC-N] title marker as coverage. meta.availablePriorities can contain the entry custom, which is a marker rather than a priority a case can hold. Each case includes a shareUrl for sharing in Slack, Jira and the like. To resolve a user-facing TC-30 (a displayId) to its internal numeric id, pass search: "TC-30", since search matches the case number and accepts the TC- prefix, then read the matching case id.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoFilter by tags (any matching tag)
limitNoNumber of cases to return
offsetNoPagination offset
searchNoSearch in case title, description, or case number
statusNoFilter by case status (e.g., Draft, Ready, Approved). Values are project-specific.
folderIdNoFilter to cases held DIRECTLY by this folder UUID. It does NOT recurse, so a container folder answers with 0 however many cases sit nested beneath it, and that zero looks exactly like an empty folder. For a subtree count call get_folder with includeStats and read statistics.totalCases; to list the cases, walk the tree with list_folders and ask for each suite. Omit to list every case in the project.
priorityNoFilter by priority (e.g., Low, Medium, High, Critical). Values are project-specific.
projectIdYesProject UUID to list cases from
includeDeletedNoInclude archived/deleted cases
automationStatusNoFilter by automation status: "automated" (has linked autotests) or "not_automated" (no linked autotests)

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Even though annotations already declare the operation read-only and idempotent, the description adds extensive behavioral detail: return shape, updatedAt semantics, absence of lastExecutedAt, the meaning of linkedAutotestCount, the custom priority marker, and the no-top-level-total quirk. This goes well beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but every sentence earns its place; it front-loads the primary purpose and then packs edge cases, return-shape details, and exclusions without padding. The structure moves from what the tool does, to return shape, to caveats, to alternatives — a clear logical flow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list endpoint with no output schema, the description is extraordinarily complete: it covers return fields, pagination shape, field semantics, filter behavior, deletion handling, automation interpretation, and alternatives for missing data. There is no meaningful gap left for an agent to discover by trial and error.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description adds crucial semantics beyond the schema: search accepts TC- prefix, folderId does not recurse and a zero result can look like an empty folder, automationStatus uses the same narrow definition as linkedAutotests, and availablePriorities.custom is a marker rather than a real priority. These details materially improve correct tool usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource — 'List test cases in a project with filtering and pagination' — and immediately communicates the tool's scope. It clearly distinguishes listing cases from related sibling tools like get_execution_cases or list_test_collections.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly directs agents to alternatives when appropriate, such as using get_folder for subtree counts, get_case_execution_history for manual execution history, and get_test_case with includeRecentExecutions for automated runs. It also explains how to resolve display IDs with search, giving concrete when-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_test_collectionsA
Read-onlyIdempotent

List Test Collections in a project - the groupings used for sprint, release or ad-hoc manual testing. Each entry returns id, name, status, priority, environment, assignedTo, folderId/folderName, releaseId/releaseName, start and end dates, and stats (totalTests, passed, failed, blocked, notExecuted, skipped, inTesting), plus a pagination block. Use it to find a collection UUID, filtered by folder, release or status; then call get_test_collection for one collection with its individual test cases and their execution status. A Test Collection is one grouping of cases, not the top-level Executions-page container, so use list_executions to find those. For the case library itself rather than runs of it, use list_test_cases. Shadow collections created inside an execution are never listed, and archived ones only with includeDeleted.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoNumber of collections to return
offsetNoPagination offset
statusNoFilter by collection status. Unlike case and release statuses, a collection status is NOT project-configurable: the storage layer accepts only these four, so ignore whatever test_run_status get_project reports for this purpose. An unrecognised value is rejected rather than quietly ignored.
folderIdNoFilter by folder UUID. Omit to list all collections.
rootOnlyNoIf true, only return collections not in any folder (root level)
projectIdYesProject UUID to list collections from
releaseIdNoFilter by release UUID
includeDeletedNoInclude archived collections

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already mark the tool as read-only, idempotent, and non-destructive, and the description adds meaningful behavioral context: shadow collections are excluded, archived collections require includeDeleted, and the response includes a pagination block. These are details an agent cannot infer from annotations alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but not padded; it front-loads the purpose and return contents, then handles sibling distinctions and behavioral caveats. Every sentence adds information an agent needs before calling the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with no output schema, the description adequately enumerates the returned fields, mentions pagination, names the filter dimensions, and differentiates it from nearby siblings. Nothing essential to selecting and invoking the tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description reinforces that folderId, releaseId, and status are filters and that the tool is meant for finding a collection UUID, but it does not need to repeat what the schema already documents in full.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: "List Test Collections in a project." It also distinguishes this tool from list_executions, list_test_cases, and get_test_collection, making its scope unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent when to use this tool: to find a collection UUID filtered by folder, release, or status, then call get_test_collection. It also gives exclusion rules: shadow collections are never listed, archived ones need includeDeleted, and executions/case libraries belong to other tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

move_test_caseA
Idempotent

Move a test case to a different folder within the same project. Cross-project moves are not supported.

ParametersJSON Schema
NameRequiredDescriptionDefault
caseIdYesNumeric test case ID to move
targetFolderIdYesTarget folder UUID (must be in the same project)

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already provide the safety profile (read-only false, destructive false, idempotent true), and the description mostly reinforces that moves stay within the same project. The 'different folder' and 'same project' scope adds a small amount of behavioral context, but doesn't disclose anything beyond the schema constraints or annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one concise, front-loaded sentence that states the action, the scope, and the exception. It does not repeat the schema or add filler; every words earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter move operation with no output schema, the description effectively captures what the tool does and its main boundary condition. A bit more detail about expected result or failure behavior would be nice, but the annotations already carry the idempotent and destructive-profile information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents both parameters completely (caseId and targetFolderId, with their types and the same-project requirement). The description just restates the folder constraint and adds no extra detail about parameter values, formats, or derived behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Move'), a clear resource ('test case'), and an explicit target ('a different folder within the same project'). It also names the key limitation ('Cross-project moves are not supported'), which distinguishes this tool from any generic update or move operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states exactly when to use the tool: moving a test case to another folder within the same project. It also explicitly says when not to use it: cross-project moves are not supported. No alternative sibling exists for moving a test case, so no further routing guidance is needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

remove_cases_from_collectionA
DestructiveIdempotent

Remove test cases from a test collection. Returns collectionId, collectionName and removed (count of case rows actually deleted); ids not present in the collection are ignored rather than raising an error. Destructive and not reversible: the recorded status and step results for those cases in this collection are deleted with them, while the test cases themselves stay in the project library. add_cases_to_collection is the inverse. To retire the whole collection instead use delete_test_collection, which archives it and preserves executions; to drop a source group from an Executions-page container use remove_source_from_execution; to archive the case itself project-wide use delete_test_case. collectionId comes from list_test_collections; caseIds are the numeric internal ids from list_test_cases, not TC-numbers.

ParametersJSON Schema
NameRequiredDescriptionDefault
caseIdsYesNumeric test case IDs to remove (e.g., [1, 2, 3])
collectionIdYesTest collection UUID

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations, it discloses that action is destructive and not reversible, deletes status/step results for these cases in this collection, ignores ids not present instead of raising an error, and preserves the test cases themselves in the project library. This aligns with destructiveHint=true and idempotentHint=true and adds concrete consequences, so no contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition opens with a crisp one-summary action, then uses each following sentence for a high-value detail: return value, edge-case behavior, destructive consequence set, and sibling routing. No filler or restatement of schema-aware items; the text is dense but fully purposeful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by naming the return values (collectionId, collectionName, removed) and the count semantics. It covers error behavior, idempotency, and the scope of damage, and it routes all nearby sibling cases. An agent has enough to decide when to call it and to establish expectation about the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Even though the schema covers both parameters, the description adds meaningful provenance: collectionId comes from list_test_collections, and caseIds are the numeric internal ids from list_test_cases, explicitly warning against TC-numbers. That disambiguates a common source error for this API family and goes well beyond the field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description leads with a specific verb+resource pair: 'Remove test cases from a test collection.' It also disambiguates from nearby tools by explaining that test cases themselves stay in the project while the collection membership is removed, and it positions delete_test_case as the tool for project-wide case archival.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly names the inverse operation (add_cases_to_collection) and gives when-to-use alternatives for related but distinct scenarios: delete_test_collection for retiring the whole collection, remove_source_from_execution for source groups on an Executions page, and delete_test_case for archiving the case project-wide. It also tells the agent where IDs come from, which is a definitive selection and parameter-prefill hint.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

remove_source_from_executionA
DestructiveIdempotent

Remove a source group (a suite, collection, or named case-group - one execution_test_run) from an execution. This deletes that group and ALL its case-run rows and step results (cascade). Get the runId (group id) from get_execution (sources[].id) or get_execution_cases (groupId). Does not delete the execution itself. Returns { removed: true }.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesThe source group to remove, which is the id get_execution returns as sources[].id - NOT the id of the suite or Collection the group was created from, and there is no sourceId argument. Confusingly it is spelled runId here because the underlying row is an execution_test_run.
executionIdYesExecution UUID

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate a destructive, non-read-only operation, and the description adds valuable behavioral context beyond them: the cascade deletion of case-run rows and step results, the fact that the execution remains intact, and the exact return value { removed: true }. This meaningfully informs the agent about side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each carrying essential information: what is removed, the destructive cascade effect, how to find the required id, and the return value. There is no filler or redundant restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive two-parameter operation with no output schema, the description covers the removal target, how to obtain the id, the cascade impact, the boundary (execution not deleted), and the response shape. Combined with a thorough parameter schema and annotations, nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100% with detailed runId explanation; the description further reinforces how to get runId and emphasizes that runId is the source group id, not the original suite/collection id. This adds practical lookup guidance beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (Remove), the resource (a source group from an execution), and precisely defines what a source group is: a suite, collection, or named case-group represented by one execution_test_run. It also distinguishes itself from deleting the whole execution, so an agent can correctly separate it from sibling tools like delete_execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives actionable guidance on how to obtain the runId: from get_execution (sources[].id) or get_execution_cases (groupId). It states that the execution itself is not deleted, which hints at the difference from delete_execution, but it does not explicitly name the alternative or specify when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

report_case_resultA
Idempotent

Record the result of running one case inside a Manual Test Execution: set per-step pass/fail and/or the case status, atomically and idempotently. Target the case by executionCaseId (from get_execution_cases) OR by executionId + caseId (numeric; if the case is in several groups, pass groupId or use executionCaseId). A Test Collection is NOT an Execution, and the trap has a name: the per-case rows get_test_collection returns carry a field called executionId, which is the id of the row inside that Collection and is rejected here with Execution case not found. To record results against a Collection, add it to an Execution with add_collections_to_execution and use the rows get_execution_cases gives you. status: an explicit value (passed/failed/blocked/in_testing/not_executed), or "auto" to derive it from the steps (failed then blocked then in_testing then passed), or omit it to leave the case status unchanged (e.g. when only writing one step). steps[].status is one of not_executed/passed/failed/blocked/skipped; index is the 0-based position in the case steps. The containing source group's status auto-rolls from its cases' results, where EXECUTED means anything other than not_executed - so a case parked in_testing counts as executed, and the same is true of the completionRate get_execution reports. A group whose every case is in_testing therefore reads completed, and the execution reads 100 percent complete, while the work is still going on. The rule: all not_executed -> not_started, some executed -> in_progress, all executed -> blocked if the group contains a blocked case, else completed (test_run_status has no failed; failures show via the group's counters). An explicit set_execution_source_status override lasts until the next report on that group recomputes it. The status of the EXECUTION itself does not roll at all: it stays wherever it was until you move it with update_execution, so an execution whose cases have all been run can still read not_executed. Returns the case status, the group status, the steps written, and fresh execution progress.

ParametersJSON Schema
NameRequiredDescriptionDefault
notesNoOptional free-text notes for the case run
stepsNoPer-step results to upsert.
caseIdNoAlternative to executionCaseId: the numeric case id (with executionId).
statusNoCase status from the project vocabulary, typically not_executed, in_testing, passed, failed or blocked, plus any the project added; call get_project for the list it uses. Pass "auto" to derive it from the step results, or omit to leave the status unchanged. The per-step status inside steps[] is a different and fixed vocabulary.
groupIdNoOptional: disambiguate when the case is in several source groups.
causedByNoOptional: which step or thing caused failure
executionIdNoAlternative to executionCaseId: the execution UUID (with caseId).
durationMinutesNoOptional: time spent, in minutes
executionCaseIdNoThe case-run row UUID (from get_execution_cases). Preferred.

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the idempotentHint and non-destructive annotations, the description discloses atomicity, the auto-derivation rules, how group status rolls up, that execution status does not roll, and that set_execution_source_status overrides last only until the next report recomputes. This gives the agent substantial behavioral transparency beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every section earns its place: targeting, the Collection trap, status semantics, roll-up rules, overrides, and return value each address a failure mode an agent would hit. It is front-loaded with the main action and then builds outward into edge cases.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter tool with no output schema, the description covers targeting, required/preferred parameters, status vocabularies, idempotency, atomicity, roll-up behavior, side effects, and even what is returned. It is complete enough for an agent to select and invoke the tool correctly without further research.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds critical semantic meaning beyond it: executionCaseId is preferred, executionId+caseId is the alternative, executionId from get_test_collection is rejected, and status 'auto' has a precise derivation order. It also clarifies the step-status vocabulary is separate from the case-status vocabulary.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific action and resource: 'Record the result of running one case inside a Manual Test Execution' and clarifies the exact scope by setting per-step pass/fail and/or case status. It also explicitly warns that 'A Test Collection is NOT an Execution', which separates it from collection-related siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent how to target the case, including the preferred executionCaseId and the alternative executionId + caseId, and when to use groupId. It also gives a concrete migration path for Collections, 'add it to an Execution with add_collections_to_execution', and points to update_execution for moving execution-level status.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_health_checkA
Read-onlyIdempotent

Get a holistic health assessment for a project: a composite score from 0 to 100 blending automation stability, coverage and manual execution. IMPORTANT: healthScore is NOT comparable BETWEEN projects. The composite weights automation 50 percent, coverage 30 and manual execution 20, then RENORMALISES over whichever of the three actually have data - so a project with no manual executions is scored out of the other two rather than penalised for the gap, and a project holding one test case can score 100 on coverage. A project can therefore outrank another while scoring worse on every dimension the two share. Compare subScores one dimension at a time and say which dimensions each project has. The status label, healthy or warning or critical, is banded straight off that same number and is exactly as incomparable, so do not rank by the word either. Quote healthScore only for one project against itself, and only across calls made with the SAME days and the same include[] - a narrower include[] drops dimensions out of the divisor and moves the score on its own - a wider window can catch work that adds a whole dimension to the renormalisation and move the score with no change in health. One project here scores 20 at 30 days and 21 at 90 purely because two manual executions came into view. The default is 14, so three calls with three windows give three rankings. It returns per-dimension subScores and sections for automation, whose three failure counts mean three different things: failedTests is tests that ENDED failed, consistentlyFailingCount is tests that failed at least three times and NEVER passed in the window, and flakyTestCount is distinct tests that both passed and failed. A test that recovers on retry belongs to the last of those and to none of the others. Its testPassRate counts flaky as a pass, so it is (passedTests + flakyResults) / totalTests, the same formula get_launches_statistics uses, and like that tool it answers null rather than 0 when the window held nothing. This section counts ONLY launches that ended passed or failed, while get_launches_statistics counts every launch in the window, so a launch stored as finished, running, interrupted, timedOut or completed contributes its tests there and not here. The difference between the two totalTests is exactly the tests inside those launches, and it reconciles to the launch: subtract them and the counts match. The rates still agree to about a decimal place because the excluded launches are usually small; the section reports BOTH flaky numbers because they are different units, flakyResults being flaky results and flakyTestCount being DISTINCT tests that both passed and failed over the window, which is what drives the flakiness penalty), testAssets (case and suite counts, status and priority breakdown - both count only cases that HAVE a value, with no residual bucket for the unset ones, so neither is guaranteed to sum to totalCases: in one project here byStatus accounts for 65 of 87 and byPriority for 70), coverage (automation rate and pass-rate trend), manualExecution (pass, fail, blocked, completion) and releases (status counts and active releases). IMPORTANT: days defaults to 14 here while get_launches_trends defaults to 30, so pass the same days to both before comparing their answers. Use include[] to limit which sections are computed. Reach for get_coverage_gaps when you want the specific cases and folders that lack automation rather than one score, and get_feature_scan when the scope is a single Jira issue.

ParametersJSON Schema
NameRequiredDescriptionDefault
daysNoLookback period for analysis (default: 14 days)
includeNoWhich sections to compute. Defaults to all. The composite score uses whichever of automation/coverage/manualExecution are present and have data.
projectIdYesProject UUID to assess

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint and idempotentHint, so the description does not need to repeat that. It goes well beyond annotations by explaining the non-comparability of healthScore across projects, the renormalisation over available dimensions, and the exact meaning of the three different failure counts. These behavioral traits are not deducible from annotations or schema, and they materially affect how an agent interprets results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately front-loaded with a crisp summary before diving into warnings. It is, however, very dense and somewhat sprawling, with several examples and parentheticals that could be condensed. Given the genuinely subtle behavior being described, the length earns its place, but a clearer structure or shorter phrasing would improve it.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no output schema, the description carries full responsibility for the return values. It does this well by enumerating the return sections (automation, testAssets, coverage, manualExecution, releases) and explaining the key subScores and edge cases. It also covers parameter interactions and cross-tool context. No critical behavioral corner required for correct invocation is left unexplained.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% coverage for projectId, days, and include, including defaults and enum values. The description adds a crucial semantic layer beyond the schema: it explains that narrow include[] changes the renormalisation denominator and therefore shifts the composite score, and that days defaults to 14 here but to 30 in get_launches_trends. This is useful but not needed for every parameter, so a 4 is fair rather than a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a clear, specific action: 'Get a holistic health assessment for a project: a composite score from 0 to 100.' It names the resource (project) and the narrow output focus. It also explicitly separates itself from siblings: get_coverage_gaps and get_feature_scan are described as alternatives for different use cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent when to prefer alternatives: 'Reach for get_coverage_gaps when you want the specific cases and folders that lack automation rather than one score, and get_feature_scan when the scope is a single Jira issue.' It also warns about comparing days with get_launches_trends, and explains how the include parameter changes what is computed. This is strong, scenario-based guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_testsA
Read-onlyIdempotent

Search AUTOMATED tests by title, file path or tag, and get how each has been doing. Each row carries titlePath (an array; get_test_history wants ONE segment of it), filePath, lastStatus, reliability, totalExecutions, passedCount, failedCount, passRate, avgDurationMs and lastSeen. IMPORTANT: days defaults to 7 here, the shortest window of any tool - a test that has not run in the last week is simply absent, which reads as does not exist rather than has not run lately. Pass days explicitly whenever the question is not about this week. The counts are of ATTEMPTS, so a retried test contributes more than one, and passRate is passedCount divided by totalExecutions on that basis; reliability is derived from the same rows, reading flaky when both a pass and a failure appear in the window. It returns NO linkage information at all, so it cannot answer whether a test is tied to a manual case: use get_test_case with includeRecentExecutions for that. Use get_test_history for one test in full detail over a longer window - it takes the segment as titlePath, where this tool takes query - and get_launch_tests for every test in one launch.

ParametersJSON Schema
NameRequiredDescriptionDefault
daysNoLookback period for recent executions
limitNoMaximum number of tests to return
queryYesSearch query (case-insensitive substring of the title-path array or the file path). Pass ONE title segment, e.g. "should validate credentials"; a " > "-joined path matches nothing.
statusNoFilter by what happened in the WINDOW, not by how the test stands now, and the two ends are not symmetrical: failed selects tests with AT LEAST ONE failed attempt, while passed selects only tests where EVERY attempt passed. So a test that failed once a month ago and has been green since is returned by failed and withheld by passed. It cannot answer what is failing now. lastStatus on each row comes closest: it is the FINAL attempt of the most recent launch in the window, so a test that failed and then passed on retry reads passed. But the window is still the window - a test that has not run since it was fixed carries whatever it last did - so for the present state take the newest launch from get_launches_statistics and read it with get_launch_tests. flaky selects tests with both a pass and a failure in the window.all
projectIdYesProject UUID

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite strong annotations (readOnly, idempotent, non-destructive), the description adds substantial behavioral context: it reveals that counts are attempts (retries inflate counts), explains passRate and reliability computation, highlights the asymmetry between failed/passed filters, and clarifies what lastStatus means. This goes far beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense and front-loaded with the core purpose, then covers edge cases and alternatives. Every sentence serves a purpose; there is no fluff. It is long because the tool's behavior is complex. A slight structural improvement could be grouping related concepts, but the existing organization is functional and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema and 5 parameters plus complex semantics, the description fully compensates: enumerates every row field (titlePath, lastStatus, reliability, etc.), explains absence semantics, distinguishes attempts from test cases, and explicitly states what the tool cannot answer (linkage, current failing status) while providing the correct tools to route those cases. Nothing needed for correct use is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although the schema already covers all params at 100%, the description adds deep semantics: days is the shortest window and must be extended unless the question is about the current week; query should be a single titlePath segment rather than a joined path; status filtering is window-based and asymmetric, with failed selected by at least one failure while passed requires all attempts passed. This is meaningful value over the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise action ('Search AUTOMATED tests'), the search dimensions (title, file path, tag), and what results include (pass rate, reliability, etc.). It explicitly names sibling tools it is not (get_test_case, get_test_history, get_launch_tests), so an agent can disambiguate immediately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use and when-not-to-use guidance: it notes the default 7-day window, warns that absence means 'not run' rather than 'nonexistent', and routes to get_test_case for linkage, get_test_history for one test in detail, and get_launch_tests for a full launch. This leaves no ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_execution_source_statusA
Idempotent

Set the status of one source group (a collection, suite, or named case-group - the per-group status dropdown on the execution page) inside an execution. Get the runId (group id) from get_execution (sources[].id) or get_execution_cases (groupId). status is validated against the project's test_run_status values (default: not_started, in_progress, completed, blocked) - call get_project to confirm. This is the GROUP status; use report_case_result for an individual case's status.

ParametersJSON Schema
NameRequiredDescriptionDefault
runIdYesSource group UUID (execution_test_run id)
statusYesGroup status: not_started/in_progress/completed/blocked (+ project-custom)
executionIdYesExecution UUID

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=true and destructiveHint=false, and the description does not contradict them. It adds behavioral context by specifying that status is validated against project test_run_status values, listing defaults, and clarifying the write operates on the group level rather than the case level.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core purpose, followed by lookup guidance, status validation details, and a sibling distinction. Every sentence carries useful information with little redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter write operation with full schema coverage and helpful annotations, the description provides all needed context: what the resource is, where to get the ID, how status is validated, and the critical boundary against the sibling case-level tool. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema by explaining where to obtain runId (from get_execution sources[].id or get_execution_cases groupId) and what the status values typically are, including project-custom possibilities.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Set the status of one source group' inside an execution. It clarifies what a source group is and explicitly distinguishes this from setting an individual case's status by naming report_case_result. The agent can tell this apart from the broad sibling set.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear when-to-use guidance and deliberately differentiates from the per-case tool: 'This is the GROUP status; use report_case_result for an individual case's status.' It also tells the agent where to fetch the runId and how to confirm valid status values, making the correct use unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

suggest_test_casesA
Read-only

Return template-based test case scaffolding for a feature description: structured ideas with titles, steps and priorities. This is pattern matching, not model output. Use it as a project-aware starting point, refine it yourself, then persist the result with create_test_case.

ParametersJSON Schema
NameRequiredDescriptionDefault
testTypeNoType of test cases to suggestall
projectIdYesProject UUID for context (uses project settings for status/priority values)
maxSuggestionsNoMaximum number of test case suggestions
featureDescriptionYesDescription of the feature to generate test cases for. Be specific about functionality, user flows, and edge cases.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description reveals important behavioral characteristics not present in the annotations: this is pattern matching, not model output, and the results are scaffolding that require human/AI refinement before persistence. The readOnlyHint already signals safety, but the description adds the expectation that results are provisional and not final artifacts. This meaningfully helps the agent decide how to use the output.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a compact two-sentence definition that front-loads the tool's purpose and then adds just enough behavioral and next-step context. Every clause serves a clear function: output expectation, pattern-matching caveat, and handoff to its sibling. There is no padding or reiteration of schema fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the fully documented input schema, read-only annotation, and simple output shape stated in the description, the tool is fully scoped. The description even covers what to do with the returned scaffolding, which is essential for a suggestion-only tool. Nothing critical is missing for an agent to correctly invoke it and handle the output.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all four parameters with 100% coverage, including defaults and constraints. The description adds minimal extra parameter semantics, mostly reinforcing that featureDescription is the source input and projectId makes suggestions project-aware. That is helpful but only tangentially, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as returning template-based test case scaffolding for a feature description, and specifies the output format (titles, steps, priorities). It explicitly contrasts itself with model output and create_test_case, which helps an agent understand its distinct role among siblings. The verb 'Return' plus the resource 'test case scaffolding' makes the core purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit usage context: use this tool to generate project-aware starting suggestions, refine them yourself, then persist them with create_test_case. It implicitly communicates that create_test_case is the follow-on tool for actual creation, but it does not enumerate situations where this tool should be avoided or other alternatives preferred. This is strong but not fully exhaustive guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_executionA
Idempotent

Update a Manual Test Execution's own fields, including its overall status. Set status to in_testing while running, and passed/failed/blocked when done. status is validated against the project's execution_status values (default: not_executed, in_testing, passed, failed, blocked) - call get_project to read the exact values a project uses; do not hardcode. Does not change case results - use report_case_result for those, or set_execution_source_status for a source group's status.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNo
statusNoExecution status: not_executed/in_testing/passed/failed/blocked (+ project-custom). Use get_project to confirm valid values.
endDateNoISO date (>= startDate)
priorityNolow/medium/high/critical
startDateNoISO date
assignedToNoAssignee user id (must be a project member)
releaseIdsNoOptional. The COMPLETE set of releases this execution should be linked to (replace-all): existing links not in the list are removed, new ones added. Pass [] to unlink all. Omit to leave links untouched. Release ids must belong to the execution's project.
descriptionNo
environmentNo
executionIdYesExecution UUID

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description adds meaningful behavioral context: status values are project-configurable, setting status follows a during/after pattern, and releaseIds is a replace-all operation. It also warns that this tool does not modify case results, which is important behavioral information not captured by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and well-structured: it opens with the tool's core purpose, immediately gives the key status workflow, and then clarifies alternative tools. Every sentence carries operational value, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 10 parameters, no output schema, and meaningful behavioral traps like replace-all release links and project-custom statuses, the description covers all critical operational needs. It also references the exact sibling tools an agent might confuse it with and provides a concrete fallback for discovering valid status values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers 70% of parameters with descriptions, and the description adds practical semantics on top: status should be in_testing while running and passed/failed/blocked when done, and releaseIds replaces the entire release-link set. It does not elaborate on every param interpretation, but the added guidance is valuable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Update'), a specific resource ('a Manual Test Execution's own fields'), and distinguishes the tool from sibling operations by explicitly saying it does not change case results. It is instantly clear what this tool is for and what it is not for.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: use this for execution-level fields and status, use report_case_result for case results, and use set_execution_source_status for source group status. It also instructs calling get_project to confirm project-specific status values rather than hardcoding.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_folderA
Idempotent

Rename a folder or suite and/or re-parent it. Set parentId to null to move it to root level, or omit it to keep the current parent. Returns the updated id, name, type, parentId, path, depth, casesCount and childFolderCount. A move carries everything nested inside along with it, so child paths change too; the new parent must be a folder in the same project, since suites cannot be parents and a folder cannot move into itself or into one of its own descendants. It never changes type and never creates or removes anything: use create_folder for a new folder or suite, delete_folder to archive one with all its contents, and move_test_case to relocate a single test case rather than its whole suite. Get folderId from list_folders.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew folder name
folderIdYesFolder UUID to update
parentIdNoNew parent folder UUID. Set to null to move to root level. Omit to keep current parent.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations, the description explains meaningful behavioral consequences: moving carries all nested children and changes their paths, it never changes type, and it never creates or destroys anything. It also clarifies that re-parenting to root changes depth/path. This is exactly the context an agent needs to understand what happens to the rest of the tree.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: core action, parentId behavior, return payload, move side effects, exclusions, and sibling routing. It is front-loaded with the primary purpose and avoids any redundant restatement of the tool name or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that there is no output schema, the description properly documents the return value fields (id, name, type, parentId, path, depth, casesCount, childFolderCount). It also covers key edge cases and sibling boundaries. For a mutation tool with three simple parameters, nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema already documents all three parameters. The description adds value by explaining the practical meaning of setting parentId to null versus omitting it, and by noting the same-project requirement for the new parent. The name parameter has no additional semantics beyond the schema, so it doesn't get a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a precise verb and resource: 'Rename a folder or suite and/or re-parent it.' It clearly distinguishes itself from sibling tools by explicitly naming create_folder, delete_folder, and move_test_case as alternatives for different operations. An agent can tell exactly what this tool does without opening other definitions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use and when-not-to-use guidance: use create_folder for new folders/suites, delete_folder for archiving, and move_test_case for moving a single case. It also adds operational constraints like 'the new parent must be a folder in the same project' and 'a folder cannot move into itself or into one of its own descendants.' This leaves no ambiguity about when to invoke it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_releaseA
Idempotent

Update release metadata. Only provided fields are updated; omitted fields remain unchanged. Use get_release first to see current values.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew release name
statusNoNew release status, from the project vocabulary, typically planned, in_progress, completed or cancelled, plus any the project added; call get_project for the list it uses. Setting it to completed also stamps completed_at.
endDateNoNew end date (YYYY-MM-DD)
versionNoNew version string
releaseIdYesRelease UUID to update
startDateNoNew start date (YYYY-MM-DD)
descriptionNoNew description

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description adds important partial-update semantics: only provided fields are updated and omitted fields remain unchanged. This helps prevent destructive accidental field resets. It does not describe return values, but annotations already cover idempotency and non-destructiveness.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two compact sentences with no filler. It front-loads the purpose, follows with the most important behavioral rule, and ends with a directly actionable prerequisite.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the full 100% schema coverage, the idempotentHint annotation, and the simple update semantics, this description is complete enough. It covers what the tool does, the partial-update behavior, and the recommended sequencing with get_release.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so each parameter is already documented. The description adds critical cross-parameter semantics: omitted fields are left untouched, which is essential for correctly forming an update request. This goes beyond what individual parameter descriptions convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Update' with the resource 'release metadata', making the operation unambiguous. It distinguishes itself from sibling tools like create_release, delete_release, and get_release by stating exactly what it modifies.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: it is for updating release metadata and explicitly instructs agents to call get_release first to fetch current values. It explains the partial-update behavior, but it does not explicitly list exclusions or name alternative tools for similar actions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_caseA
Idempotent

Update an existing test case. Only provided fields are updated; omitted fields remain unchanged. Returns the full updated case object including shareUrl. TIP: Use get_test_case first to see current values.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoReplace all tags with this array
stepsNoReplace all steps with this array
titleNoNew title for the case
caseIdYesInternal numeric test case id to update - NOT the user-facing "TC-N"/caseNumber. If the user gave you a "TC-N", resolve it via list_test_cases(search: "TC-N") first.
statusNoNew status. Typical values: Draft, Ready, Approved, Deprecated. Check list_test_cases meta.availableStatuses for project-specific values.
priorityNoNew priority. Typical values: Low, Medium, High, Critical. Check list_test_cases meta.availablePriorities for project-specific values.
dependsOnNoSet dependency to another numeric case ID, or null to remove dependency
descriptionNoNew description. Supports Markdown: **bold**, *italic*, ~~strikethrough~~, inline `code`, # / ## / ### headings, "- " bullet and "1. " numbered lists, [text](url) links, and "> " blockquotes. Plain text works too. Images and fenced code blocks are not supported.
customFieldsNoCustom field values (replaces all existing custom fields)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

It discloses the partial-update behavior and states that the full updated case object including shareUrl is returned, which is useful beyond the annotations. The annotations already mark it as non-read-only, idempotent, and potentially destructive, and the description does not contradict those hints. It adds context without requiring the agent to infer behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences convey purpose, behavioral nuance, return value, and a useful practical tip. There is no filler or repetition. The format is efficiently front-loaded with the essential action and semantics.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 9 parameters and no output schema, the description includes the key guidance: partial update behavior, current value retrieval, and return shape. Since all parameters are fully documented in the schema, the description does not need to restate them. Nothing critical about calling this tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents each parameter clearly. The description adds no per-parameter details, which is acceptable since the schema is rich. The partial-update note adds some context for all optional fields but does not reduce the need to rely on the schema for param-specific semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the operation ('Update an existing test case') with a specific verb and resource. 'Existing' distinguishes it from create_test_case and delete_test_case, and the scope is unambiguous. No ambiguity remains regarding what the tool acts on.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives practical usage context, especially the tip to call get_test_case first to see current values and the partial-update semantics ('only provided fields are updated'). It does not explicitly contrast with bulk_update_cases or say when to use create instead, but the guidance is clear enough for common use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_collectionA
Idempotent

Update test collection metadata. Only provided fields are updated; omitted fields remain unchanged. Use get_test_collection first to see current values.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew collection name
statusNoNew collection status. NOT project-configurable and matched exactly: the storage layer accepts only these four, and neither a label such as "In Progress" nor a different case such as "IN_PROGRESS" will do.
priorityNoNew priority. NOT project-configurable: the storage layer accepts only these four, in lowercase. Matching ignores case and the value is stored lowercase, so Critical and critical both land as critical.
assignedToNoEmail address of the assigned tester (or empty to unassign)
descriptionNoNew description
environmentNoNew environment
collectionIdYesTest collection UUID to update

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the idempotent and non-destructive annotations, the description discloses an important behavioral trait: 'Only provided fields are updated; omitted fields remain unchanged.' This is critical for correctly invoking a partial update and prevents unintended overwrites. It also implies a stateless merge behavior, adding value over the annotations alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with no filler. The primary action is front-loaded, followed by the update semantics and a helpful get-first instruction, making every sentence valuable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter update tool with 100% schema coverage and meaning-bearing annotations, the description supplies the missing execution model: partial update, preserving omitted values, and for prospective state. It does not describe the response shape, but no output schema is defined and the description already gives enough to invoke the tool safely. A slight esponse-type note would make it fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter already has a clear description, including enum restrictions for status and priority. The description adds that fields are updated only if present, but it does not need to go into parameter details. Given the 100% schema coverage, the baseline is correct.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Update test collection metadata.' It clearly distinguishes from sibling tools like update_folder and update_test_case by honoring collection metadata. It is not a tautology and gives an immediate sense of the tool's purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains when to use the tool and a key precondition: 'Use get_test_collection first to see current values.' It also clarifies that only provided fields are updated, so an agent knows to send only the fields that need changing. It does not explicitly exclude alternatives, but the guidance is sufficient for a read-before-patch workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 18 tool updatesv1.5.3
    • Changedadd_external_link1 field changed
      • changedInput schema / properties / externalUrl / description
        Previous value: -"Full URL to the external ticket. Required for non-Jira providers. Optional for Jira (the canonical URL is fetched from the Jira API)."New value: +"Full URL to the external ticket. Required for non-Jira providers. IGNORED for Jira: the canonical URL is always fetched from the Jira API, and supplying one neither skips the issue lookup nor rescues an unknown key."
    • Changedbulk_update_cases1 field changed
      • changedInput schema / properties / updates / properties / status / description
        Previous value: -"New status for all cases (e.g., \"Ready\", \"Approved\")"New value: +"New status applied to every case in the batch. The vocabulary is per-project - read meta.availableStatuses from list_test_cases, or call get_project, and send a value from that list. An unrecognised value is rejected and NO case is updated."
    • Changedcreate_release3 fields changed
      • removedInput schema / properties / status / default
        Removed value: -"planned"
      • changedInput schema / properties / status / description
        Previous value: -"OPTIONAL - defaults to \"planned\". Valid: planned, in_progress, completed, cancelled."New value: +"Optional. Release status from the project vocabulary, typically planned, in_progress, completed or cancelled, plus any the project added; call get_project for the list. Omit to use whichever value the project marks as its default."
      • removedInput schema / properties / status / enum
        Removed value: -[
        -  "planned",
        -  "in_progress",
        -  "completed",
        -  "cancelled"
        -]
    • Changedcreate_test_case4 fields changed
      • removedInput schema / properties / priority / default
        Removed value: -"Medium"
      • changedInput schema / properties / priority / description
        Previous value: -"OPTIONAL - defaults to \"Medium\". Only specify if user explicitly requests a different priority. Valid values vary by project (typically: Low, Medium, High, Critical)."New value: +"Optional. Omit it and the case takes whichever value the project marks as its default. The vocabulary is per-project, so call get_project for this project list and send a value from it rather than guessing."
      • removedInput schema / properties / status / default
        Removed value: -"Draft"
      • changedInput schema / properties / status / description
        Previous value: -"OPTIONAL - defaults to \"Draft\". Only specify if user explicitly requests a different status. Valid values vary by project (typically: Draft, Ready, Approved, Deprecated)."New value: +"Optional. Omit it and the case takes whichever value the project marks as its default, which is what the UI does. The vocabulary is per-project and a project may have renamed it or added its own, so call get_project for this project list and send a value from it rather than guessing."
    • Changedcreate_test_collection3 fields changed
      • removedInput schema / properties / priority / default
        Removed value: -"medium"
      • changedInput schema / properties / priority / description
        Previous value: -"OPTIONAL - defaults to \"medium\". Valid: low, medium, high, critical."New value: +"Optional, defaults to medium. Unlike a case priority, a collection priority is NOT project-configurable: the storage layer accepts only these four, in lowercase, so a project-custom case priority cannot be used here. Matching ignores case, and the value is stored lowercase."
      • changedInput schema / properties / priority / enum
        Previous value: -[
        -  "low",
        -  "medium",
        -  "high",
        -  "critical"
        -]New value: +[
        +  "critical",
        +  "high",
        +  "medium",
        +  "low"
        +]
    • Changedget_launch_tests1 field changed
      • changedInput schema / properties / statuses / description
        Previous value: -"Filter by test status (default: all statuses)"New value: +"Filter by ATTEMPT-ROW status, not by how a test ended (default: all). A test that failed once and passed on retry still returns its failed row here, so counting the filtered rows, or even counting distinct titlePath over them, answers how many tests had this status at SOME attempt - never how many ended that way. A launch whose own stats.failed is 0 can return 15 failed rows across 13 titles. For tests that ENDED failed, fetch the rows unfiltered and keep the highest-retry row per titlePath, or read stats.failed on the launch itself."
    • Changedget_launches_statistics3 fields changed
      • changedInput schema / properties / search / description
        Previous value: -"Search in launch title/ID/tags"New value: +"Search the launch NAME (the launch field on each row), its id, or its tags. Launches carry no trigger metadata at all - nothing records that a run was nightly, on merge or manual, and there is no branch or CI-job field - so a request naming a run by its schedule can only be resolved through a name or tag the reporter happened to set, or by startedAt. If a name search comes back empty, say the run cannot be identified rather than assuming the newest launch is the one meant: three launches here carry byte-identical stats."
      • changedInput schema / properties / statuses / description
        Previous value: -"Filter by run status"New value: +"Filter by launch status. Set by the reporters through the ingest service, so this list is fixed and not project-configurable. A launch that ended without an explicit status, and whose totals did not reconcile, is stored as finished: that is usually an empty or half-reported launch, and it counts toward totalRuns while sitting in none of the passed, failed or running buckets."
      • changedInput schema / properties / statuses / items / enum
        Previous value: -[
        -  "passed",
        -  "failed",
        -  "running",
        -  "interrupted"
        -]New value: +[
        +  "passed",
        +  "failed",
        +  "running",
        +  "finished",
        +  "interrupted",
        +  "timedOut",
        +  "completed"
        +]
    • Changedget_test_case3 fields changed
      • changedInput schema / properties / includeHistory / description
        Previous value: -"Include edit history (audit log of changes)"New value: +"Include the edit audit trail: field, oldValue, newValue, changeType, userName, userEmail and changedAt per entry. It is NOT a complete account of the case, for two separate reasons. COVERAGE: the trail holds only the changes it audits, and updatedAt can be newer than every entry in it - one case here carries updatedAt two months after its latest recorded change - while the case itself has no updatedBy field. VOLUME: the array is capped at the 50 most recent entries, with no truncation flag, no total and no way to page further back, so a trail of exactly 50 IS a truncated one and its oldest entry is not the oldest change. Two traps inside the rows as well: an entry records that a field was SUBMITTED rather than that its value moved, so oldValue and newValue are often identical - 28 of 50 on one case here - and userName is frequently just the email address again. So this answers what changed and by whom AMONG AUDITED, RECENT changes, and cannot always answer who last touched the case."
      • changedInput schema / properties / includeLinkedAutotests / description
        Previous value: -"Include explicitly linked autotest paths"New value: +"Include autotest links made with link_autotest_to_case. These are the only links reported here, and they are not the only links that exist: a reporter can also bind a test to a case by putting a [TC-N] marker in the test title, which ingest resolves to a case id on the test row itself. get_coverage_gaps counts that second kind as covered in its unlinkedAutotests section ONLY: its summary, automationRate, foldersWithLowCoverage and unlinkedCases all use the same narrow, explicit-link definition this list does. So a test can be absent here and still be linked, and a case can sit in unlinkedCases while a marker-bound test covers it, which makes automationRate a floor rather than the whole picture."
      • changedInput schema / properties / includeRecentExecutions / description
        Previous value: -"Include recent autotest execution results from CI runs"New value: +"Include autotest results for this case from CI. There is NO time window: the latest result per distinct test title is returned, up to 20, with no date cutoff at all - so an empty array is DEFINITIVE and means no automated test has ever reported against this case, rather than none lately. This is also the only place a test bound to the case by a [TC-N] title marker appears, which makes it the arbiter when linkedAutotests comes back empty."
    • Changedlist_executions1 field changed
      • changedInput schema / properties / status / description
        Previous value: -"Optional status filter"New value: +"Optional status filter, matched EXACTLY and case-sensitively against the execution_status vocabulary of the project; call get_project for the values it uses, since a project can rename them or add its own. An unrecognised value is rejected rather than answering with an empty list, which used to be indistinguishable from a genuine nothing-found. Note this matches the label a person set on the EXECUTION, not what is inside it: an execution holding blocked cases usually does not itself read blocked, so to find blocked WORK list without a filter and read progress.blocked."
    • Changedlist_folders1 field changed
      • removedInput schema / properties / includeStats
        Removed value: -{
        -  "default": false,
        -  "description": "Include basic statistics (slower query)",
        -  "type": "boolean"
        -}
    • Changedlist_releases2 fields changed
      • changedInput schema / properties / status / description
        Previous value: -"Filter by release status"New value: +"Filter by release status. Values come from the project vocabulary, typically planned, in_progress, completed or cancelled, plus any the project added; call get_project for the list it uses. An unrecognised value is rejected rather than quietly ignored. Like an execution status and a launch status, a release status is SET BY A PERSON and never derived from the stats beside it: a release can read completed with half its cases unexecuted and its only linked launch failed, and a linked launch can postdate completedAt. Read the rollup for what actually happened."
      • removedInput schema / properties / status / enum
        Removed value: -[
        -  "planned",
        -  "in_progress",
        -  "completed",
        -  "cancelled"
        -]
    • Changedlist_test_cases1 field changed
      • changedInput schema / properties / folderId / description
        Previous value: -"Filter by specific folder UUID. Omit to list all cases in project."New value: +"Filter to cases held DIRECTLY by this folder UUID. It does NOT recurse, so a container folder answers with 0 however many cases sit nested beneath it, and that zero looks exactly like an empty folder. For a subtree count call get_folder with includeStats and read statistics.totalCases; to list the cases, walk the tree with list_folders and ask for each suite. Omit to list every case in the project."
    • Changedlist_test_collections1 field changed
      • changedInput schema / properties / status / description
        Previous value: -"Filter by collection status"New value: +"Filter by collection status. Unlike case and release statuses, a collection status is NOT project-configurable: the storage layer accepts only these four, so ignore whatever test_run_status get_project reports for this purpose. An unrecognised value is rejected rather than quietly ignored."
    • Changedremove_source_from_execution1 field changed
      • changedInput schema / properties / runId / description
        Previous value: -"Source group UUID (execution_test_run id) to remove"New value: +"The source group to remove, which is the id get_execution returns as sources[].id - NOT the id of the suite or Collection the group was created from, and there is no sourceId argument. Confusingly it is spelled runId here because the underlying row is an execution_test_run."
    • Changedreport_case_result1 field changed
      • changedInput schema / properties / status / description
        Previous value: -"Case status: passed/failed/blocked/in_testing/not_executed, or \"auto\" to derive from steps, or omit to leave unchanged."New value: +"Case status from the project vocabulary, typically not_executed, in_testing, passed, failed or blocked, plus any the project added; call get_project for the list it uses. Pass \"auto\" to derive it from the step results, or omit to leave the status unchanged. The per-step status inside steps[] is a different and fixed vocabulary."
    • Changedsearch_tests1 field changed
      • changedInput schema / properties / status / description
        Previous value: -"Filter by recent execution status"New value: +"Filter by what happened in the WINDOW, not by how the test stands now, and the two ends are not symmetrical: failed selects tests with AT LEAST ONE failed attempt, while passed selects only tests where EVERY attempt passed. So a test that failed once a month ago and has been green since is returned by failed and withheld by passed. It cannot answer what is failing now. lastStatus on each row comes closest: it is the FINAL attempt of the most recent launch in the window, so a test that failed and then passed on retry reads passed. But the window is still the window - a test that has not run since it was fixed carries whatever it last did - so for the present state take the newest launch from get_launches_statistics and read it with get_launch_tests. flaky selects tests with both a pass and a failure in the window."
    • Changedupdate_release2 fields changed
      • changedInput schema / properties / status / description
        Previous value: -"New status: planned, in_progress, completed, or cancelled. Setting to \"completed\" auto-sets completed_at timestamp."New value: +"New release status, from the project vocabulary, typically planned, in_progress, completed or cancelled, plus any the project added; call get_project for the list it uses. Setting it to completed also stamps completed_at."
      • removedInput schema / properties / status / enum
        Removed value: -[
        -  "planned",
        -  "in_progress",
        -  "completed",
        -  "cancelled"
        -]
    • Changedupdate_test_collection3 fields changed
      • changedInput schema / properties / priority / description
        Previous value: -"New priority: low, medium, high, or critical."New value: +"New priority. NOT project-configurable: the storage layer accepts only these four, in lowercase. Matching ignores case and the value is stored lowercase, so Critical and critical both land as critical."
      • changedInput schema / properties / priority / enum
        Previous value: -[
        -  "low",
        -  "medium",
        -  "high",
        -  "critical"
        -]New value: +[
        +  "critical",
        +  "high",
        +  "medium",
        +  "low"
        +]
      • changedInput schema / properties / status / description
        Previous value: -"New status: not_started, in_progress, completed, or blocked."New value: +"New collection status. NOT project-configurable and matched exactly: the storage layer accepts only these four, and neither a label such as \"In Progress\" nor a different case such as \"IN_PROGRESS\" will do."
  2. 3 tool updates
    • Changedadd_collections_to_execution1 field changed
      • changedInput schema / properties / collectionIds / description
        Previous value: -"Collection (manual run) UUIDs"New value: +"Test Collection UUIDs"
    • Removedcreate_suite
    • Removedget_folder_statistics
  3. 57 tool updatesv1.5.0
    • First observedadd_cases_to_collection
    • First observedadd_cases_to_execution
    • First observedadd_collections_to_execution
    • First observedadd_external_link
    • First observedadd_suites_to_execution
    • First observedbulk_update_cases
    • First observedcreate_execution
    • First observedcreate_folder
    • First observedcreate_release
    • First observedcreate_suite
    • First observedcreate_test_case
    • First observedcreate_test_collection
    • First observeddelete_execution
    • First observeddelete_folder
    • First observeddelete_release
    • First observeddelete_test_case
    • First observeddelete_test_collection
    • First observedexport_report
    • First observedget_case_execution_history
    • First observedget_coverage_gaps
    • First observedget_execution
    • First observedget_execution_cases
    • First observedget_feature_scan
    • First observedget_folder
    • First observedget_folder_statistics
    • First observedget_launch_tests
    • First observedget_launches_statistics
    • First observedget_launches_trends
    • First observedget_project
    • First observedget_release
    • First observedget_test_case
    • First observedget_test_collection
    • First observedget_test_history
    • First observedlink_autotest_to_case
    • First observedlink_launches_to_release
    • First observedlist_executions
    • First observedlist_external_links
    • First observedlist_folders
    • First observedlist_projects
    • First observedlist_releases
    • First observedlist_test_cases
    • First observedlist_test_collections
    • First observedmove_test_case
    • First observedremove_cases_from_collection
    • First observedremove_external_link
    • First observedremove_source_from_execution
    • First observedreport_case_result
    • First observedrun_health_check
    • First observedsearch_tests
    • First observedset_execution_source_status
    • First observedsuggest_test_cases
    • First observedunlink_launch_from_release
    • First observedupdate_execution
    • First observedupdate_folder
    • First observedupdate_release
    • First observedupdate_test_case
    • First observedupdate_test_collection

TDQS

A4.2/5.0
Disambiguation4/5

Tools are grouped around distinct resources—cases, folders, collections, executions, launches, releases, external links, and analytics—and the descriptions explicitly call out near-misses like test collections vs executions and launch statistics vs launch trends. A few pairs such as add_cases_to_collection vs add_cases_to_execution still need careful reading, but every tool has a distinct purpose.

Naming Consistency5/5

Naming is consistently verb-first snake_case with standard CRUD verbs: get/list/create/update/delete/add/remove/link/link. Even compound tools like bulk_update_cases, run_health_check, and report_case_result follow the same readable pattern with no style mixing.

Tool Count2/5

55 tools is far beyond the 3–15 well-scoped range and clearly falls into the too-many category, even though the underlying domain is broad. The count will overwhelm agents during tool selection and makes the server hard to use effectively.

Completeness4/5

Major lifecycle coverage is strong: cases, folders, suites, collections, executions, releases, launches, external links, and reports all have meaningful create/read/update/delete or equivalent workflows. Minor reverse-operation gaps exist—such as no unlink for autotest-to-case and no way to remove a single loose case from an execution group—but agents can generally work around them.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    F
    maintenance
    Enables management of TestRail projects, test cases, runs, and results directly through MCP-supported clients. It provides a comprehensive set of tools to interact with the TestRail API for seamless test cycle management within AI environments.
    42
    1,580
    44
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    This is an unofficial, community-built MCP server for qTest Manager. Not affiliated with or endorsed by Tricentis. Browse projects, modules, and test cases Create/delete test design modules Create and manage test execution cycles Add test cases into execution suites
    9
    20
    1
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/m00nreport/mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server