TestRail MCP Server
Provides comprehensive TestRail API integration for managing test cases, projects, suites, runs, and tests. Enables updating test cases and runs, adding test results and attachments, and retrieving test data with filtering capabilities.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@TestRail MCP Servershow me the test cases for project 5 that failed in the last run"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
TestRail MCP Server
What is this?
This tool connects your TestRail test management system with AI assistants like Cursor or Claude Desktop. It allows your AI assistant to read and update test cases, add test results, and manage your testing workflow through simple conversations.
Related MCP server: TestRail MCP Server
Features
View Test Cases: Show test cases, projects, and test runs
Update Tests: Modify test case details, titles, and descriptions
Add Results: Record test results and upload screenshots
Search & Filter: Find specific tests or projects quickly
File Attachments: Upload documents and images to test cases
Full Integration: Works with Cursor, Claude Desktop, and other AI tools
Who is this for?
QA Engineers who want to manage tests through AI conversations
Project Managers who need quick access to test information
Developers who want to integrate testing into their AI-assisted workflow
Anyone who uses TestRail and wants to make it more accessible through AI
Quick Start
Install Node.js from nodejs.org
Get TestRail API key from your TestRail settings
Install the server:
npm install -g testrail-mcp-serverConfigure your AI assistant with your TestRail credentials
Restart your AI assistant and start asking questions!
Prerequisites
You need these tools before installing:
Node.js 18.17.0+ - Download from nodejs.org (LTS version)
npm - Comes with Node.js automatically
AI Assistant - Cursor (cursor.sh) or Claude Desktop (claude.ai)
TestRail Account - With API access enabled
Verify installation:
node --version # Should show 18.17.0 or higher
npm --version # Should show a version numberGetting TestRail API Credentials
Log into TestRail at your instance URL (e.g.,
https://yourcompany.testrail.com)Get API Key:
Click your profile picture → "My Settings"
Scroll to "API Keys" section
Click "Add API Key"
Name it (e.g., "MCP Server")
Copy and save the API key securely
Note your credentials:
Username: Your TestRail login (usually email)
URL: Your TestRail web address
Installation
Step 1: Install the Server
Open Command Prompt/Terminal and run:
npm install -g testrail-mcp-serverStep 2: Configure Your AI Assistant
For Cursor Users:
Open Cursor → Settings (
Ctrl + ,orCmd + ,)Find "MCP" settings
Add this configuration:
{
"mcpServers": {
"testrail": {
"command": "npx",
"args": ["testrail-mcp-server"],
"env": {
"TESTRAIL_USERNAME": "your_testrail_username",
"TESTRAIL_API_KEY": "your_testrail_api_key",
"TESTRAIL_URL": "https://your-instance.testrail.com"
}
}
}
}For Claude Desktop Users:
Find config file:
Windows:
%APPDATA%\Claude\claude_desktop_config.jsonMac:
~/Library/Application Support/Claude/claude_desktop_config.json
Add the same configuration as above
Replace these values:
your_testrail_username: Your TestRail login (usually email)your_testrail_api_key: The API key from TestRailhttps://your-instance.testrail.com: Your TestRail URL
Step 3: Restart and Test
Restart your AI assistant completely
Test: Ask "Can you show me my TestRail projects?"
Usage
Once set up, simply talk to your AI assistant naturally:
Viewing Information:
"Show me all my TestRail projects"
"What test cases are in project 1?"
"Show me test case number 123"
Updating Tests:
"Update test case 123 with the title 'Login functionality test'"
"Change the priority of test case 456 to high"
"Add a comment to test case 789"
Adding Results:
"Mark test 456 as passed"
"Add a failed result to test 789 with comment 'Button not clickable'"
"Upload this screenshot to test case 123"
Searching:
"Find all test cases with 'login' in the title"
"Show me failed tests from the last test run"
Troubleshooting
"Command not found" or "npm is not recognized"
Install Node.js from nodejs.org
Restart Command Prompt/Terminal
Check with
node --version
"Authentication failed" or "Invalid credentials"
Verify your TestRail username (usually email)
Copy API key again from TestRail
Check TestRail URL starts with
https://
"Connection timeout" or "Cannot connect to TestRail"
Check internet connection
Verify TestRail URL is correct
Test TestRail in web browser
AI assistant doesn't respond to TestRail questions
Restart your AI assistant completely
Check configuration file format (valid JSON)
Verify all required fields are filled
"Permission denied" error
Run Command Prompt as Administrator (Windows) or with
sudo(Mac/Linux)Update to latest Node.js version
Still having trouble?
Check error messages for clues
Try installation steps again
Create an issue on GitHub with your error details
Security Best Practices
Never share your API key - treat it like a password
Use HTTPS URLs - ensure your TestRail URL starts with
https://Don't put API keys in code - always use configuration files
Rotate API keys regularly - change them every few months
Keep TestRail account secure - use strong passwords and 2FA
What NOT to do:
Don't share configuration files with others
Don't put API keys in public repositories
Don't use HTTP URLs (only HTTPS)
Don't share screenshots showing your API key
Additional Resources
Helpful Links:
Getting Support:
GitHub Issues - Report bugs or ask questions
Contact your TestRail administrator for API access issues
License
MIT License - see LICENSE file for details.
Available Tools
18 toolsadd_attachment_to_caseAdd Attachment to TestRail CaseC
Upload a file attachment to a TestRail test case.
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | TestRail case ID | |
| file_path | Yes | Path to the file to upload as attachment |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Upload a file attachment' implies a write/mutation operation, it lacks critical details: required permissions (e.g., edit access to the case), file size/type limits, whether the attachment is public or private, or how failures are handled. This is inadequate for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core action and target. There's no wasted wording or redundancy, making it easy to parse quickly while conveying the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a file upload operation (mutation with potential side effects), no annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like error handling, response format, or constraints, leaving significant gaps for the agent to navigate blindly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear parameter descriptions in the schema itself (e.g., 'TestRail case ID', 'Path to the file to upload as attachment'). The description adds no additional semantic context beyond what's already in the schema, such as example file paths or case ID formats. This meets the baseline for high schema coverage but doesn't enhance understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Upload a file attachment') and target resource ('to a TestRail test case'), making the purpose immediately understandable. However, it doesn't differentiate from potential sibling tools like 'add_case' or 'add_result' that also involve adding content to TestRail, leaving room for confusion about when to choose this specific tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'add_case' (for creating cases) and 'add_result' (for adding test results), there's no indication of whether this is for supplementary files, required documentation, or specific contexts like bug reports. The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_caseAdd TestRail CaseA
Create a new TestRail test case in a specific section. IMPORTANT: Before creating a case, gather required information using get_projects, get_suites, get_sections, and get_case_fields tools to ensure proper section_id, type_id, and custom field values. Or ask the user to provide the information if not provided.
| Name | Required | Description | Default |
|---|---|---|---|
| title | Yes | Test case title - should be descriptive and unique within the section | |
| section_id | Yes | Section ID where the case will be created. REQUIRED: Use get_sections tool first to find valid section IDs for your project/suite. Different projects have different section structures. | |
| type_id | No | Test case type ID (e.g., 1=Acceptance, 2=Accessibility, 3=Automated, 4=Compatibility, 5=Destructive, 6=Functional, 7=Other, 8=Performance, 9=Regression, 10=Security, 11=Smoke & Sanity, 12=Usability). RECOMMENDED: Use get_cases tool to see what type_id values are used in existing cases in your target section. | |
| priority_id | No | Priority ID (1=Low, 2=Medium, 3=High, 4=Critical). RECOMMENDED: Use get_cases tool to see what priority_id values are used in existing cases. | |
| refs | No | References (e.g., requirement IDs, JIRA tickets, user story numbers). Can be comma-separated for multiple references. | |
| custom | No | Custom fields (key-value pairs). REQUIRED: Use get_case_fields tool first to discover available custom fields and their valid values. Common fields include: custom_automation_type (1=None, 2=Playwright, 3=ChatGPT, 4=Non-Automated, 5=Partial), custom_environment (1=UAT Only, 2=UAT/Prod, 3=Demo UAT, 4=Live UAT), custom_preconds (preconditions text), custom_steps (test steps text), custom_expected (expected results text). Some custom fields are required by the project configuration. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It clearly indicates this is a creation/mutation operation ('Create a new TestRail test case'), mentions important prerequisites (gathering data from other tools), and hints at complexity (custom fields may be required by project configuration). However, it doesn't explicitly mention authentication needs, rate limits, or what happens on failure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized (3 sentences) and front-loaded with the core purpose. Every sentence adds value: first states what the tool does, second provides crucial prerequisite guidance, third offers alternative approach. Could be slightly more concise by combining the last two sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with 6 parameters (including nested objects) and no annotations/output schema, the description does well by emphasizing prerequisites and complexity. However, it doesn't describe what the tool returns (no output schema exists) or potential error conditions, leaving some gaps for a mutation operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description doesn't add significant parameter semantics beyond what's in the schema - it references the same tools (get_sections, get_case_fields) that the schema descriptions mention. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Create a new TestRail test case in a specific section' - a clear verb ('Create') + resource ('TestRail test case') + location constraint ('in a specific section'). This distinguishes it from sibling tools like 'update_case' (modification) and 'get_case' (retrieval).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool: 'Before creating a case, gather required information using get_projects, get_suites, get_sections, and get_case_fields tools' and 'Or ask the user to provide the information if not provided.' It names specific prerequisite tools and offers an alternative approach when data is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_resultAdd TestRail ResultC
Adds a new test result, comment, or assigns a test.
| Name | Required | Description | Default |
|---|---|---|---|
| test_id | Yes | The ID of the test to which the result should be added | |
| status_id | Yes | The ID of the test status (1=Passed, 2=Blocked, 4=Retest, 5=Failed) | |
| comment | No | The comment or description for the test result | |
| version | No | The version or build against which the test was executed | |
| elapsed | No | The time it took to execute the test (e.g., "30s" or "1m 45s") | |
| defects | No | A comma-separated list of defects to link to the test result | |
| assignedto_id | No | The ID of a user to whom the test should be assigned | |
| custom_step_results | No | Array of step results for structured testing | |
| custom | No | Custom fields with custom_ prefix |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'adds' implies a write operation, it doesn't specify permissions required, whether it's idempotent, error handling, or side effects (e.g., notifications). For a mutation tool with 9 parameters, this lack of behavioral context is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core functionality without unnecessary words. Every part ('adds a new test result, comment, or assigns a test') directly contributes to understanding the tool's purpose, making it well-structured and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 9 parameters, no annotations, and no output schema, the description is incomplete. It lacks behavioral details (e.g., permissions, side effects), usage guidelines relative to siblings, and any information about return values or errors, leaving the agent with insufficient context for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description mentions 'test result, comment, or assigns a test', which loosely maps to parameters like status_id, comment, and assignedto_id, but adds minimal semantic value beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('adds', 'assigns') and resources ('test result', 'comment', 'test'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from sibling tools like 'update_test' or 'add_case', which could also modify test-related data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'update_test' and 'add_case' available, there's no indication of whether this is for initial results, updates, or specific scenarios, leaving the agent to guess based on context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_caseGet TestRail CaseC
Fetch a TestRail test case by ID.
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | TestRail case ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'fetch', implying a read operation, but lacks details on permissions, rate limits, error handling, or response format. For a tool with no annotations, this leaves significant gaps in understanding its behavior and constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words. It is front-loaded and appropriately sized, making it easy to parse quickly while conveying the essential action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what data is returned (e.g., case details, fields), potential errors, or how it differs from sibling tools. For a retrieval tool in a context with multiple similar tools, more context is needed to ensure proper usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions 'by ID', which aligns with the single parameter 'case_id' in the schema. With 100% schema description coverage, the schema already documents the parameter as 'TestRail case ID' with type and constraints, so the description adds minimal value beyond reinforcing the ID-based lookup, meeting the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'fetch' and the resource 'TestRail test case by ID', making the purpose specific and understandable. However, it doesn't explicitly distinguish this tool from similar siblings like 'get_cases' (plural) or 'get_test', which also retrieve test-related data, leaving some ambiguity about when to use this specific tool versus others.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_cases', 'get_test', and 'get_case_fields', there is no indication of context, prerequisites, or exclusions, such as whether this is for single-case retrieval versus bulk operations or other related queries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_case_fieldsGet TestRail Case FieldsB
Returns a list of available test case custom fields.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states it 'Returns a list,' implying a read-only operation, but doesn't specify if it requires authentication, has rate limits, returns paginated results, or what format the list takes (e.g., JSON array of field objects). For a tool with zero annotation coverage, this is insufficient transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Returns a list...') with no wasted words. Every part of the sentence earns its place by specifying what is returned and for what resource, making it optimally concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It states what the tool does but lacks context on usage, behavior, or output format. For a read operation with no complex inputs, this is acceptable but leaves gaps that could hinder an agent's effective use, especially without annotations to fill in behavioral details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so the schema fully documents the lack of inputs. The description doesn't need to add parameter details, and it correctly implies no parameters are required by not mentioning any. This meets the baseline for zero-parameter tools, though it doesn't explicitly state 'no parameters needed,' which would have warranted a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Returns') and resource ('list of available test case custom fields'), making the purpose immediately understandable. It distinguishes this from siblings like get_case or get_cases by specifying it returns custom fields rather than cases themselves. However, it doesn't explicitly contrast with all siblings, so it falls short of a perfect 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a project context), when it's appropriate (e.g., before creating cases with custom fields), or what siblings might be better for related tasks (like get_case for case details). This leaves the agent without contextual usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_casesGet TestRail CasesB
Get a list of test cases for a project or specific test suite with optional filtering and pagination.
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | TestRail project ID | |
| suite_id | No | TestRail suite ID (optional if project is in single suite mode) | |
| created_after | No | Only return test cases created after this date (as UNIX timestamp) | |
| created_before | No | Only return test cases created before this date (as UNIX timestamp) | |
| created_by | No | A list of creator user IDs to filter by | |
| filter | No | Only return cases with matching filter string in the case title | |
| limit | No | The number of test cases to return (max 250, default 250) | |
| milestone_id | No | A list of milestone IDs to filter by | |
| offset | No | Where to start counting the test cases from (pagination offset) | |
| priority_id | No | A list of priority IDs to filter by | |
| refs | No | A single Reference ID (e.g. TR-1, 4291, etc.) | |
| section_id | No | The ID of a test case section | |
| template_id | No | A list of template IDs to filter by | |
| type_id | No | A list of case type IDs to filter by | |
| updated_after | No | Only return test cases updated after this date (as UNIX timestamp) | |
| updated_before | No | Only return test cases updated before this date (as UNIX timestamp) | |
| updated_by | No | A user ID who updated test cases to filter by | |
| label_id | No | A list of label IDs to filter by |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions 'optional filtering and pagination,' which hints at behavior, but doesn't disclose critical details like whether this is a read-only operation, potential rate limits, authentication needs, or what the return format looks like (e.g., list structure, error handling). For a tool with 18 parameters and no annotation coverage, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that efficiently conveys the core purpose and key features (filtering, pagination) without unnecessary words. It's front-loaded and every part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (18 parameters, no annotations, no output schema), the description is incomplete. It covers the basic purpose but lacks behavioral details (e.g., read-only nature, response format) and deeper usage guidance. The high schema coverage helps, but for a tool with many parameters and no output schema, more context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 18 parameters thoroughly. The description adds minimal value beyond the schema by mentioning 'optional filtering and pagination,' which loosely maps to some parameters like 'limit' and 'offset,' but doesn't provide additional syntax, format, or usage details. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get a list of test cases for a project or specific test suite with optional filtering and pagination.' It specifies the verb ('Get'), resource ('test cases'), and scope ('project or specific test suite'), but doesn't explicitly distinguish it from sibling tools like 'get_case' (singular) or 'get_sections'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context by mentioning 'project or specific test suite' and 'optional filtering and pagination,' but doesn't provide explicit guidance on when to use this tool versus alternatives like 'get_case' (for a single case) or 'get_sections' (for sections). No exclusions or prerequisites are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_projectGet TestRail ProjectB
Get details for a specific TestRail project by ID.
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | TestRail project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states it 'Get details' but doesn't clarify if this is a read-only operation, what permissions are required, or what the response format includes (e.g., project name, settings). This leaves significant gaps for a tool that likely interacts with a database or API.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It is front-loaded with the core action and resource, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (single parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose but lacks details on behavior, usage context, and output, which are needed for full understanding in the absence of annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the parameter 'project_id' well-documented as a positive integer. The description adds minimal value beyond the schema by specifying 'by ID', but it doesn't explain format constraints or provide examples, so it meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get details') and resource ('specific TestRail project by ID'), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_projects' (plural) or 'get_case', which also retrieve details but for different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_projects' (for listing projects) or specify prerequisites such as needing a valid project ID, leaving usage context implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_projectsGet TestRail ProjectsB
List all TestRail projects.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states the action without behavioral details. It doesn't disclose whether this is paginated, requires authentication, has rate limits, returns structured data, or what happens with large project sets—critical for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste—front-loaded and to the point. Every word contributes directly to stating the tool's purpose without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a simple list operation, the description is incomplete. It lacks details on return format, pagination, error handling, or how it fits with siblings like 'get_project', leaving gaps for an agent to use it effectively in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema coverage, so no parameter documentation is needed. The description doesn't add parameter info, which is appropriate, but it also doesn't imply any hidden parameters or constraints, keeping it straightforward.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List') and resource ('TestRail projects'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_project' (singular) or explain scope beyond 'all', missing full sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'get_project' (singular) or 'get_cases' (which might be project-specific). The description implies a broad listing but offers no context about prerequisites, filtering, or comparison to siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_runGet TestRail RunB
Returns an existing test run. Please see get tests for the list of included tests in this run.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | The ID of the test run |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states this is a read operation ('Returns'), which is clear, but doesn't disclose behavioral traits such as error handling (e.g., what happens if the run_id is invalid), authentication needs, rate limits, or response format. The reference to 'get tests' adds some context but is insufficient for a mutation-free tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose and efficiently references another tool for additional context. Every sentence earns its place with no wasted words, making it appropriately sized and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and points to related information, but as a read tool with no annotations, it should ideally include more on response behavior or error cases to be fully complete for agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'run_id' well-documented in the schema as 'The ID of the test run'. The description doesn't add any meaning beyond this, such as format examples or constraints not in the schema. Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Returns') and resource ('an existing test run'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_runs' (plural) or 'get_test', which could retrieve similar resources. The mention of 'get tests' is helpful but not a direct sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by referencing 'get tests' for related information, suggesting it's for retrieving a specific run's details. However, it lacks explicit guidance on when to use this versus alternatives like 'get_runs' (for multiple runs) or 'get_test' (for individual tests), and doesn't mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_runsGet TestRail RunsA
Get a list of test runs for a project with optional filtering and pagination. Only returns test runs that are not part of a test plan.
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | TestRail project ID | |
| created_after | No | Only return test runs created after this date (as UNIX timestamp) | |
| created_before | No | Only return test runs created before this date (as UNIX timestamp) | |
| created_by | No | A comma-separated list of creators (user IDs) to filter by | |
| is_completed | No | 1 to return completed test runs only. 0 to return active test runs only | |
| limit | No | The number of test runs to return (max 250, default 250) | |
| milestone_id | No | A comma-separated list of milestone IDs to filter by | |
| offset | No | Where to start counting the test runs from (pagination offset) | |
| refs_filter | No | A single Reference ID (e.g. TR-a, 4291, etc.) | |
| suite_id | No | A comma-separated list of test suite IDs to filter by |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'optional filtering and pagination,' which hints at functionality, but lacks details on permissions, rate limits, error handling, or the return format (e.g., structure of the list). For a tool with 10 parameters and no annotations, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Get a list of test runs for a project') and adds key constraints ('with optional filtering and pagination. Only returns test runs that are not part of a test plan.'). Every part earns its place without redundancy or waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (10 parameters, no annotations, no output schema), the description is adequate but incomplete. It covers the basic purpose and a key constraint (excludes test plan runs), but lacks details on behavioral aspects like response format, error cases, or usage nuances. This makes it minimally viable but with clear gaps for effective tool invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, meaning all parameters are documented in the input schema. The description adds minimal value beyond the schema by mentioning 'optional filtering and pagination,' which aligns with parameters like created_after, limit, and offset, but doesn't provide additional syntax or usage details. This meets the baseline of 3 when schema coverage is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get a list of') and resource ('test runs for a project'), making the purpose specific and understandable. It distinguishes this tool from 'get_run' (singular) by indicating it returns multiple runs, but doesn't explicitly differentiate from other list tools like 'get_cases' or 'get_tests' beyond the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by specifying 'Only returns test runs that are not part of a test plan,' which helps guide when to use this tool. However, it doesn't explicitly mention when to use alternatives like 'get_run' (for a single run) or other filtering tools, nor does it state any prerequisites beyond the required project_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_sectionsGet TestRail SectionsC
Get a list of sections for a project and test suite with optional pagination.
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | TestRail project ID | |
| suite_id | No | TestRail suite ID (optional if project is in single suite mode) | |
| limit | No | The number of sections to return (max 250, default 250) | |
| offset | No | Where to start counting the sections from (pagination offset) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'optional pagination' which hints at list behavior, but doesn't describe return format, error conditions, rate limits, authentication needs, or what happens with invalid inputs. For a read operation with 4 parameters, this lacks critical operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that efficiently conveys the core action, scope, and key feature. It's front-loaded with the main purpose and avoids redundancy or unnecessary details. Every word earns its place, making it highly concise and readable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on return values, error handling, and practical usage scenarios. For a tool that fetches structured data, more context is needed to help an agent use it effectively, especially without annotations to fill gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 4 parameters. The description adds minimal value beyond the schema, mentioning 'optional pagination' which loosely relates to 'limit' and 'offset', but doesn't clarify semantics like the relationship between project_id and suite_id or default behaviors. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get a list of') and resource ('sections'), specifying the scope ('for a project and test suite') and optional feature ('with optional pagination'). It distinguishes from siblings like 'get_cases' or 'get_suites' by focusing on sections, but doesn't explicitly contrast with them. The purpose is specific and actionable, though not maximally differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It mentions the context ('for a project and test suite') but doesn't specify use cases, prerequisites, or exclusions. With siblings like 'get_cases' or 'get_suites' available, there's no indication of when sections are needed over those resources, leaving usage ambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_suiteGet TestRail SuiteC
Get details for a specific TestRail test suite by ID.
| Name | Required | Description | Default |
|---|---|---|---|
| suite_id | Yes | TestRail suite ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states it 'gets details' but doesn't specify what details are returned (e.g., suite name, description, project association), whether it's a read-only operation, or any error handling (e.g., for invalid IDs). This leaves significant gaps in understanding the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence that efficiently conveys the core purpose without unnecessary words. It's front-loaded and wastes no space, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'details' are returned, which is critical for a tool with no structured output documentation. For a simple read operation, more context on the response format would help the agent use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions 'by ID', which aligns with the single parameter 'suite_id' in the input schema. Since schema description coverage is 100%, the schema already documents the parameter as 'TestRail suite ID' with type and constraints. The description adds minimal value beyond this, meeting the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get details') and resource ('TestRail test suite by ID'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_suites' (plural) or 'get_case' which suggests this is for a single suite, but this distinction isn't explicitly stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_suites' (for listing multiple suites) or other 'get_' tools for different resources. It lacks context about prerequisites, such as needing a valid suite ID, or exclusions, leaving the agent to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_suitesGet TestRail SuitesB
Get all test suites for a specific TestRail project by ID.
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | TestRail project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the action ('Get all test suites') but lacks behavioral details such as whether this is a read-only operation, if it requires authentication, any rate limits, pagination behavior, or error handling. For a tool with zero annotation coverage, this is a significant gap in disclosing operational traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Get all test suites') and specifies the context ('for a specific TestRail project by ID'). There is zero waste, making it highly concise and well-structured for quick understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (one parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and parameter context, but lacks completeness in behavioral aspects like safety or output details, which are important for a tool with no annotations or output schema to guide the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with the single parameter 'project_id' documented as 'TestRail project ID'. The description adds no additional meaning beyond this, such as format examples or constraints not in the schema. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but doesn't detract either.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('all test suites'), specifying the scope ('for a specific TestRail project by ID'). It distinguishes from siblings like 'get_suite' (singular) by indicating it retrieves multiple suites, though it doesn't explicitly contrast with other list tools like 'get_cases' or 'get_projects' beyond the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing suites for a project, but provides no explicit guidance on when to use this versus alternatives like 'get_suite' (for a single suite) or other list tools. It mentions the required 'project_id' parameter, which hints at prerequisites, but lacks context on exclusions or comparisons with siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_testGet TestRail TestC
Returns an existing test.
| Name | Required | Description | Default |
|---|---|---|---|
| test_id | Yes | The ID of the test | |
| with_data | No | The parameter to get data |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns data (a read operation), but doesn't cover aspects like authentication needs, rate limits, error handling, or what the return format looks like (e.g., JSON structure). For a read tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—a single sentence with zero waste. It's front-loaded with the core action ('Returns'), making it easy to scan. Every word earns its place, though this brevity contributes to gaps in other dimensions like guidelines and transparency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (a read operation with 2 parameters) and lack of annotations and output schema, the description is incomplete. It doesn't explain what 'test' data is returned, how to interpret 'with_data,' or any behavioral traits. For a tool with no structured output and full parameter reliance on the schema, more context is needed to be fully helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning beyond what the input schema provides. Schema description coverage is 100%, with clear documentation for 'test_id' (ID of the test) and 'with_data' (parameter to get data). Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description doesn't compensate or add extra context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Returns an existing test' states the basic action (return) and resource (test), but it's vague—it doesn't specify what a 'test' entails in TestRail (e.g., test case details, results) or differentiate it from sibling tools like 'get_tests' (plural). It avoids tautology by not just restating the name, but lacks specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it doesn't explain if this is for retrieving a single test by ID (vs. 'get_tests' for multiple tests) or mention prerequisites like needing a valid test ID. The description implies usage only through the action 'returns,' but no explicit context or exclusions are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_testsGet TestRail TestsC
Returns a list of tests for a test run.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | The ID of the test run | |
| status_id | No | A comma-separated list of status IDs to filter by | |
| limit | No | The number that sets the limit of tests to be shown on the response (max 250, default 250) | |
| offset | No | The number that sets the position where the response should start from (pagination offset) | |
| label_id | No | IDs of labels as comma separated values to filter by |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure but offers minimal information. It doesn't mention authentication requirements, rate limits, pagination behavior beyond parameters, response format, error conditions, or whether this is a read-only operation (though 'Returns' implies it). For a tool with 5 parameters and no annotation coverage, this is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a list-retrieval tool and front-loads the essential information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 5 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't address key contextual aspects like authentication needs, rate limits, pagination behavior, response format, or error handling. The agent would need to guess about important behavioral characteristics when invoking this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain relationships between parameters (like how status_id and label_id interact) or provide examples. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Returns a list') and resource ('tests for a test run'), making the purpose immediately understandable. It distinguishes from sibling tools like 'get_test' (singular) by specifying it returns multiple tests, but doesn't explicitly differentiate from other list tools like 'get_cases' or 'get_runs' beyond the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when filtering by status or label is appropriate, when pagination is needed, or how this differs from similar tools like 'get_cases' which might return related data. The agent must infer usage from parameter names alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_caseUpdate TestRail CaseC
Update a TestRail test case by ID with new field values.
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | TestRail case ID | |
| title | No | Test case title | |
| section_id | No | Section ID | |
| type_id | No | Test case type ID | |
| priority_id | No | Priority ID | |
| refs | No | References (e.g., requirement IDs) | |
| custom | No | Custom fields (key-value pairs) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is an update operation, implying mutation, but doesn't mention permissions required, whether changes are reversible, rate limits, error handling, or what the response looks like (since there's no output schema). This leaves significant gaps in understanding the tool's behavior beyond the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part ('Update a TestRail test case by ID with new field values') contributes directly to understanding the tool, making it appropriately sized and well-structured for quick comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, mutation operation, no annotations, no output schema), the description is insufficient. It lacks details on behavioral traits (e.g., side effects, authentication), doesn't explain return values or errors, and provides minimal context for usage. For a mutation tool with rich parameters, this leaves the agent under-informed about critical aspects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description adds no additional meaning beyond implying that parameters represent 'new field values,' which is redundant with the schema. This meets the baseline of 3 for high schema coverage, but doesn't compensate with extra insights like field interdependencies or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update'), resource ('TestRail test case'), and mechanism ('by ID with new field values'), which is specific and unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'update_run' or 'update_test', which also update TestRail entities, leaving some ambiguity about when to choose this specific tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'add_case' (for creation) or other update tools (e.g., 'update_run'). It mentions updating by ID but doesn't specify prerequisites (e.g., needing an existing case ID) or exclusions (e.g., not for bulk updates), leaving usage context implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_runUpdate TestRail RunC
Updates an existing test run. Partial updates are supported.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | The ID of the test run to be updated | |
| name | No | The name of the test run | |
| description | No | The description of the test run | |
| milestone_id | No | The ID of the milestone | |
| include_all | No | True for including all test cases and false for a custom case selection | |
| case_ids | No | An array of case IDs for the custom case selection | |
| config | No | A comma-separated list of configuration IDs | |
| config_ids | No | An array of configuration IDs | |
| refs | No | A string of external requirements | |
| start_on | No | The start date (Unix timestamp) | |
| due_on | No | The due date (Unix timestamp) | |
| custom | No | Custom fields (key-value pairs) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions partial updates are supported (useful context), it doesn't address critical aspects like required permissions, whether changes are reversible, rate limits, or what happens to unspecified fields during partial updates. For a mutation tool with 12 parameters, this leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with just two sentences that both earn their place. The first sentence states the core purpose, and the second adds important behavioral context about partial updates. No wasted words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 12 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what the tool returns, error conditions, authentication requirements, or how it differs from similar update tools. The mention of partial updates is helpful but doesn't compensate for the broader gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions 'partial updates are supported' which provides context about how parameters work together, but doesn't add specific meaning to individual parameters beyond what's already in the schema (which has 100% coverage). The baseline of 3 is appropriate since the schema does the heavy lifting of documenting all parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Updates') and resource ('an existing test run'), and specifies that partial updates are supported. However, it doesn't differentiate this tool from sibling tools like 'update_case' or 'update_test' beyond the resource type, which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'update_case' or 'update_test'. It mentions partial updates are supported, but doesn't explain when this might be preferred over creating a new run or using other update tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_testUpdate TestRail TestC
Updates the labels assigned to an existing test.
| Name | Required | Description | Default |
|---|---|---|---|
| test_id | Yes | The ID of the test to be updated | |
| labels | No | The ID of a label, the title of a label or both, in array form | |
| custom | No | Custom fields (key-value pairs) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states this is an update operation (implying mutation) but doesn't cover critical aspects like required permissions, whether changes are reversible, rate limits, or what happens to existing labels not mentioned. For a mutation tool with zero annotation coverage, this is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, with every word earning its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the update operation returns, error conditions, or behavioral constraints. Given the complexity (3 parameters including nested objects) and lack of structured coverage, more context is needed for effective agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all three parameters. The description mentions 'labels' but doesn't add meaningful semantic context beyond what the schema provides (e.g., explaining label types or custom field usage). Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Updates') and resource ('labels assigned to an existing test'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'update_case' or 'update_run', which likely update different aspects of TestRail entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing test), exclusions, or comparisons to sibling tools like 'update_case' or 'update_run', leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
18 tool updates
v1.0.0- First observed
add_attachment_to_case - First observed
add_case - First observed
add_result - First observed
get_case - First observed
get_case_fields - First observed
get_cases - First observed
get_project - First observed
get_projects - First observed
get_run - First observed
get_runs - First observed
get_sections - First observed
get_suite - First observed
get_suites - First observed
get_test - First observed
get_tests - First observed
update_case - First observed
update_run - First observed
update_test
TDQS
Most tools have distinct purposes targeting specific TestRail resources (cases, runs, tests, projects, suites, sections, fields), but there is some overlap between get_case/get_cases and get_test/get_tests that could cause confusion if an agent misinterprets singular vs. plural usage. The descriptions help clarify, but the naming similarity creates minor ambiguity.
All tool names follow a consistent verb_noun pattern with snake_case, using clear action verbs like get, add, and update paired with specific nouns (e.g., get_case, add_attachment_to_case, update_run). This uniformity makes the tool set predictable and easy to navigate.
With 18 tools, the count is slightly high but reasonable for a comprehensive TestRail API coverage, including CRUD operations for cases, runs, tests, and supporting resources like projects and suites. It feels slightly heavy but well-scoped to the domain without being excessive.
The tool set provides complete CRUD/lifecycle coverage for TestRail's core domain, including projects, suites, sections, cases, runs, tests, and attachments. It supports creation (add_case), retrieval (get_* tools), updates (update_* tools), and result management (add_result), with no obvious gaps for essential workflows.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Manage test suites, run tests, view results, and automate QA workflows via AI with testRigor.
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Create and manage MeisterTask projects, tasks, and notes from your AI assistant.
AI QA that runs your app in a browser on every pull request: projects, test targets, test cases.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables AI assistants to interact directly with TestRail instances for managing test projects, suites, cases, runs, results, plans, milestones, and attachments through the TestRail API with secure authentication.772001MIT
- AlicenseAqualityFmaintenanceEnables management of TestRail projects, test cases, runs, and results directly through MCP-supported clients. It provides a comprehensive set of tools to interact with the TestRail API for seamless test cycle management within AI environments.421,58044MIT
- AlicenseBqualityDmaintenanceEnables AI assistants to interact with TestRail test management system, supporting full CRUD operations on projects, suites, sections, test cases, runs, results, plans, and milestones.353,5081MIT
- AlicenseBqualityDmaintenanceIntegrates TestRail with Claude Code to enable AI-assisted test management workflows, including project, suite, test case, test run, and result operations.133,508MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/Derrbal/testrail-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server