Skip to main content
Glama
rudrathkr

MCP Test Failure Analysis Server

by rudrathkr

MCP Server Demo

This project contains Python MCP servers for QA-oriented test failure analysis, built with FastMCP.

Current Structure

mcp-server-demo/
├── FailureAnalysisFromLogs.py
├── README.md
├── logs/
├── pyproject.toml
├── testFailureAnalysis.py
└── uv.lock

Related MCP server: CI Investigator MCP

What This Project Includes

  • testFailureAnalysis.py

    • analyze_test_failure

    • cluster_failures

    • detect_flaky_tests

  • FailureAnalysisFromLogs.py

    • analyze_test_failure

    • Reads and classifies local .log files from the logs/ folder

  • logs/

    • Sample failure logs used by the log-based MCP server

  • pyproject.toml

    • Python version and dependency configuration

  • uv.lock

    • Locked dependency versions for reproducible installs

  • .vscode/mcp.json

    • MCP server entries for test-failure-analysis, test-failure-analysis-from-logs, and mcp-atlassian

Prerequisites

  • Python 3.11 or later

  • uv

  • Internet access for the first dependency install

Installation

From the repository root:

cd mcp-server-demo
uv sync

Running the Server

Start the input-based MCP server with:

uv run python testFailureAnalysis.py

Start the log-based MCP server with:

uv run python FailureAnalysisFromLogs.py

Available Tools

analyze_test_failure

Analyzes a failed test using the test name, stack trace, and logs, then returns a likely failure category and recommendation.

analyze_test_failure in FailureAnalysisFromLogs.py

Analyzes local .log files from the logs/ folder and returns failure classification, likely root cause, recommendation, and important error lines. It can analyze all logs, a specific test name, or an exact log file name.

cluster_failures

Groups similar failures by stack trace signature so repeated patterns are easier to spot.

detect_flaky_tests

Reviews historical pass/fail results and identifies tests that show flaky behavior.

Optional Local MCP Configuration

The repository root contains .vscode/mcp.json, which can be used by MCP-aware tooling for local server setup during development. It includes entries for test-failure-analysis, test-failure-analysis-from-logs, and mcp-atlassian.

Atlassian MCP

The local MCP configuration includes an mcp-atlassian server entry for Jira access.

How to use it

Use the configured mcp-atlassian entry from .vscode/mcp.json in your MCP-aware client.

To run it manually, use:

JIRA_URL=<your-jira-url> \
JIRA_USERNAME=<your-jira-username> \
JIRA_API_TOKEN=<your-jira-api-token> \
uvx mcp-atlassian

Troubleshooting

  • If uv is not available, install it and reopen the terminal.

  • If dependency installation fails, confirm Python 3.11+ is active.

  • If the server does not start, run uv sync again inside mcp-server-demo.

Work Flow Image in Image folder at image/project-workflow.png

Available Tools

3 tools
analyze_test_failureC

Analyze a failed test and suggest likely root cause.

ParametersJSON Schema
NameRequiredDescriptionDefault
test_nameYes
stack_traceYes
logsYes

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, and the description only gives a high-level purpose without detailing behavior (e.g., whether it is safe, requires permissions, or modifies state). The description does not compensate for missing annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short (one sentence), which is concise but lacks necessary detail. It is front-loaded, but could be more informative without losing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 3 required parameters, no output schema, and no annotations, the description is severely lacking. It does not explain return values, how inputs are used, or any other context needed for effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%; the description adds no meaning about the three parameters (test_name, stack_trace, logs). They are just named but not explained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: analyzing a failed test and suggesting a root cause. However, it does not explicitly differentiate from siblings like 'cluster_failures' or 'detect_flaky_tests', though the intent is distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives, no prerequisites or exclusions provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cluster_failuresB
Group failures by similar error signature.
Each failure should contain test_name and stack_trace.
ParametersJSON Schema
NameRequiredDescriptionDefault
failuresYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must cover behavioral traits. It only states grouping by error signature but omits details about output, state modifications, error handling, or performance implications.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, highly concise, and front-loaded with the core action. Every word adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no description of return values. The tool clusters failures but doesn't indicate what the output looks like (e.g., clusters with centroid signatures). This is a significant gap for a grouping tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'failures' is an array of objects without schema descriptions. The description compensates by specifying that each failure should contain 'test_name' and 'stack_trace', adding meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'group' and resource 'failures by similar error signature', and specifies required fields. It distinguishes from sibling tools like analyze_test_failure and detect_flaky_tests, though not explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus its siblings. There is no mention of scenarios where clustering is appropriate or when to use alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

detect_flaky_testsB
Detect flaky tests from historical pass/fail data.
Each item should contain test_name and status.
ParametersJSON Schema
NameRequiredDescriptionDefault
test_historyYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears full responsibility. It does not disclose whether the tool is read-only, destructive, or has side effects. It only hints at input formatting (test_name and status) but omits behavior like output structure or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with two sentences: the first states the tool's purpose, and the second provides an input hint. No unnecessary words; every sentence contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one array parameter and no output schema, the description covers the core purpose and input requirement. However, it lacks details on output (e.g., what the tool returns) and fails to provide context relative to siblings. It is minimally adequate but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description adds that each item should contain 'test_name and status', which provides necessary meaning beyond the schema (which only says array of objects with additionalProperties). However, it does not specify data types or expected values, so it adds limited value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Detect'), the resource ('flaky tests'), and the data source ('historical pass/fail data'). It distinguishes from sibling tools like 'analyze_test_failure' which focuses on analyzing a specific failure, and 'cluster_failures' which clusters failures, making the tool's purpose distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus alternatives ('analyze_test_failure', 'cluster_failures'). It implies usage for historical data but does not set context for selection, leaving the agent to infer.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 3 tool updatesv0.1.0
    • First observedanalyze_test_failure
    • First observedcluster_failures
    • First observeddetect_flaky_tests

TDQS

B3.3/5.0
Disambiguation5/5

Each tool targets a distinct aspect of test failure analysis: root cause analysis for a single failure, grouping failures by error signature, and detecting flaky tests from history. There is no overlap or ambiguity.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern using snake_case (analyze_test_failure, cluster_failures, detect_flaky_tests), making them predictable and easy to understand.

Tool Count5/5

With exactly 3 tools, the server is well-scoped for test failure analysis. This number is sufficient to cover core functionalities without being overwhelming or too sparse.

Completeness4/5

The tool set covers essential operations for analyzing test failures, clustering, and flakiness detection. However, it lacks tools for data retrieval (e.g., fetching test history) or generating summary reports, which are minor gaps.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/rudrathkr/MCPServerCreationPythonSDK'

If you have feedback or need assistance with the MCP directory API, please join our Discord server