Skip to main content
Glama

OrangePro maps every public behavior in your codebase, scores each one by real test evidence, and shows you the structural blind spots before your users find them. Runs locally. Your code never leaves your machine.

npx -y @orangepro/mcp-server@latest start .

Table of Contents


Related MCP server: Pinion

What you get

One command produces an interactive HTML report:

npx -y @orangepro/mcp-server@latest start .
open .orangepro/behavior-coverage.html

The report has two modes: Simple (integration-level blind spots, plain English) and Expert (full behavior list, evidence tiers, flows, system map). Toggle with the pill switch at the top.

β†’ Live example: Twenty CRM (5,237 behaviors mapped)

System map β€” entry lanes (GraphQL, HTTP, Jobs) flowing into services, sized by traffic, colored by evidence tier, red-ringed by risk.

Priority gaps of another open source Project HONO β€” top 20 unproven behaviors ranked by blast radius, with generated test drafts.


Evidence tiers

Every behavior gets exactly one tier. Nothing is labeled "tested" on faith.

Tier

Color

What it means

Dynamically Proven

🟒

A real test kills a targeted mutation of this behavior

Runtime-covered

🟒

Coverage tool executed this code

Statically Linked

🟑

A test imports and calls this code β€” structural link, not proof

Unconfirmed Candidate

βšͺ

A similar test file exists β€” a lead, not evidence

No Signal

πŸ”΄

Nothing tests this behavior

"Dynamically Proven 0" is normal on first run. Proof requires running tests against targeted mutations. That's the trust model.


Quick start

cd /path/to/your/repo
npm install          # install the repo's own dependencies first

npx -y @orangepro/mcp-server@latest start .
open .orangepro/behavior-coverage.html

No API key needed. The report shows your system map, evidence tiers, priority gaps, and delta since last run.

Want test generation? Add a model key (BYOK):

export ANTHROPIC_API_KEY="..."   # or OPENAI_API_KEY / OLLAMA_BASE_URL
npx -y @orangepro/mcp-server@latest start .

AI output never changes evidence tiers. Only the mutation-kill oracle can mint Dynamically Proven.

Output:

.orangepro/
β”œβ”€β”€ behavior-coverage.html   ← open this
β”œβ”€β”€ graph.json               ← deterministic evidence graph
β”œβ”€β”€ COVERAGE_REPORT.md       ← coverage and gap summary
└── ai/                      ← candidate flows (when a key is configured)

orangepro_generated/         ← generated tests; your source files are never touched

Each rerun shows a delta banner: what entered the codebase, what moved up in risk, what got resolved.


Use with your coding agent

OrangePro runs as an MCP server. Add to your client's config:

{
  "mcpServers": {
    "orangepro-local": {
      "command": "npx",
      "args": ["-y", "@orangepro/mcp-server@latest", "mcp"]
    }
  }
}

Client

Where to put it

Claude Code

.mcp.json or ~/.claude.json

Cursor

~/.cursor/mcp.json or Settings β†’ MCP

VS Code / Copilot

MCP settings

Codex / OpenCode

Run npx -y @orangepro/mcp-server@latest agent --client codex

The workflow: Tell your agent:

"Use orangepro_start, then orangepro_generate_tests with base_ref=main. Write each test to its suggested_path, run it, and report pass/fail."

The agent writes the test, runs it, calls orangepro_prove, and the behavior turns Dynamically Proven. One prompt, full loop.


Works with

Any MCP-compatible agent can drive OrangePro. No vendor lock-in.


How it works

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Your Code  β”‚ ──► β”‚  Knowledge   β”‚ ──► β”‚  Evidence   β”‚
β”‚  (any lang) β”‚     β”‚    Graph     β”‚     β”‚   Tiers     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           β”‚
                    β”Œβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”
                    β–Ό             β–Ό
             β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
             β”‚ Gap Reportβ”‚  β”‚ Generate β”‚
             β”‚ + Risks   β”‚  β”‚  Tests   β”‚
             β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Phase

What happens

Needs a model key?

Analyze

AST walk β†’ behaviors, flows, evidence tiers

No

Score

Graph readiness score (0–100)

No

Generate

Grounded tests for top gaps

Yes (BYOK)

Prove

Mutation-kill oracle confirms test breaks if behavior changes

No

Same code = same score. Deterministic. Always.


Language support

Language

Static mapping

Generated tests

Dynamic proof

TypeScript / JavaScript

βœ“

βœ“ Jest / Vitest / Mocha

βœ“

Python

βœ“

βœ“ pytest

βœ“

Go

βœ“

βœ“ *_test.go

βœ“

Java

βœ“

βœ“ JUnit 4/5

βœ“

Kotlin, Rust, PHP, C#, Ruby, Swift, C, C++

βœ“

planned

planned

Static mapping works across many languages via tree-sitter. Dynamic proof is deliberately narrower β€” each language needs a runner, mutation locator, and sandbox profile.


Highest-value local run

Use the repository's own setup and test commands first, and keep unit and integration coverage in separate artifacts. Then run opro start; it performs analysis, ingests the artifacts, attempts targeted proof, generates report-visible drafts, and writes the final report. A separate opro analyze is unnecessary when opro start follows it.

# 1. Install/build exactly as the repository documents.
# 2. Run the repository's unit and integration coverage commands separately.
# 3. Record artifact provenance (example paths and commands):
mkdir -p .orangepro
# create .orangepro/coverage-suites.json using the schema below

opro coverage .                    # optional preflight: discover/generate artifacts
opro start . --proof-limit 5 --generate-limit 20
{
  "artifacts": {
    ".orangepro/coverage/unit.coverprofile": {
      "suite": "unit",
      "command": "make unit-test-coverage"
    },
    ".orangepro/coverage/integration.coverprofile": {
      "suite": "integration",
      "command": "make integration-test-coverage"
    }
  }
}

Without this manifest, OrangePro conservatively infers clear unit/integration names and labels everything else unclassified; it never guesses that an aggregate profile is unit-only. The report shows unit, integration, their overlap, unclassified coverage, and the combined union separately. --proof-limit controls dynamic proof attempts (which may draft a test for proof); --generate-limit independently controls the additional report-visible risk-gap drafting lane. A generation run also records its terminal status and exact reason, so a compiler/import failure is not misreported as a generic dependency problem.


Privacy

  • No stored source. Reads code in-process. Never uploads to an OrangePro server.

  • No existing-source mutation. Never edits your source or test files.

  • Your keys stay yours. Read from env at call time, never persisted.

  • BYOK is direct. Code context goes to the model provider you configure. OrangePro is not in that path.


opro                          # analyze + report + agent next actions
opro start --base main        # same, scoped to a branch diff
opro analyze                  # build the evidence graph
opro score                    # graph readiness (0–100)
opro gaps --limit 10          # top 10 untested behaviors
opro generate --base main     # tests for PR diff
opro generate --single        # top gap, whole repo
opro prove                    # mutation-kill oracle
opro rtm                      # traceability matrix
opro export                   # metadata-only evidence pack
opro mcp                      # run as MCP server (stdio)
opro doctor                   # what evidence to add next
opro coverage                 # discover/generate artifacts; analyze or start ingests them

Add --json to any read command for machine output. Run opro help for the full reference.

Tool

What it does

orangepro_start

One-command setup: analyze + report + next actions

orangepro_analyze_sources

Build/refresh the evidence graph

orangepro_generate_tests

Generate grounded tests for gaps

orangepro_prove

Run mutation-kill oracle on a behavior

orangepro_prove_loop

Setup + dynamic proof + report refresh for one behavior

orangepro_find_test_gaps

List behaviors with weak/missing tests, ranked by risk

orangepro_graph_score

Graph readiness score (0–100)

orangepro_status

Workspace state without generating anything

orangepro_doctor

Recommend next evidence to improve quality

orangepro_rtm

Requirements traceability matrix

orangepro_stats

Aggregate statistics

orangepro_changed_impact

What a diff touches (requires git + base ref)

orangepro_record_run

Record a test run result

orangepro_explain_test

Explain why a test was generated

orangepro_export_evidence_pack

Export metadata-only evidence pack

orangepro_update_graph

Incremental graph update

orangepro_ai_links

Weak behavior→symbol suggestions (optional AI)

orangepro_ai_flows

Candidate flow discovery (optional AI)

opro generate --base main              # tests for what this branch changed
opro generate --pr 1234                # checks out PR #1234
opro generate --changed                # current branch diff vs main

Each generated test includes:

  • Grounding β€” the real files, symbols, and existing tests it cites

  • Run hints β€” where to write it, how to run it

  • Scenario bucket β€” what failure mode it targets

If dependencies aren't installed, tests are kept as Manual tests (Given/When/Then steps with the blocker named). Install dependencies and re-run to convert them to runnable tests.

Generation is evidence-gated. A category is produced only when the graph has supporting evidence.

Category

What it targets

Happy path

Primary expected behavior

Validation error

Bad/invalid input handling

Edge case

Boundaries, empty/null, concurrency, retries

Integration flow

Multi-step behavior across services

Security / privacy

Auth, injection, data leakage

Regression

Pinning a previously-broken behavior

Analysis, scoring, and proof need no model key. Generation does.

Provider

Environment variable

OpenAI-compatible

OPENAI_API_KEY (optional: OPENAI_BASE_URL, OPENAI_MODEL)

Anthropic

ANTHROPIC_API_KEY (optional: ANTHROPIC_MODEL)

Ollama (local, no key)

OLLAMA_BASE_URL (optional: OLLAMA_MODEL)

Auto-detect order: OpenAI β†’ Ollama β†’ Anthropic. Override with --provider and --model. The defaults are gpt-5.3-codex for OpenAI and claude-sonnet-5 for Anthropic.

Run opro setup to configure interactively. Keys stay in your environment β€” never written to graph, config, or artifacts.

With a provider key, OrangePro stages weak AI behavior→symbol links and AI-suggested candidate flows. These are review/generation worklists, not evidence:

  • AI links appear as AI-linked suggestions.

  • AI flows are stored separately from deterministic flows.

  • Neither lane changes evidence tiers or denominator counts.

Use them when you want the agent to find likely service-boundary flows faster; ignore them for a deterministic-only report.


What's on the hosted platform

This repo is the free local tool. The OrangePro platform adds:

  • Persistent knowledge graph across PRs and repos

  • PR/CI policy gates over evidence tiers and risk deltas

  • Jira / Confluence / TestRail / OpenAPI enrichment

  • Cross-repo intelligence and recurring-flow memory

  • Production incident correlation and regression targeting

  • Team dashboards and test lifecycle management


Contributing

git clone https://github.com/OrangeproAI/orangepro-mcp.git
cd orangepro-mcp && npm ci && npm run build
npm test

PRs welcome. Please open an issue first for large changes.


Available Tools

15 tools
analyze_pr_riskAnalyze PR riskA
Read-onlyIdempotent

Analyze a pull request for quality risk. Returns overall risk score (0-100), risk drivers, impacted categories, similar historical bugs, coverage gaps, and recommended tests to run. Use before merging to catch regressions.

ParametersJSON Schema
NameRequiredDescriptionDefault
pr_titleYesPull request title.
changed_filesNoList of changed file paths (e.g., ['src/checkout.ts', 'src/payment.ts']).
pr_descriptionNoPull request body, summary, or description of changes.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so description does not need to disclose safety. Adds output details (risk score, drivers, etc.) but no further behavioral traits like rate limits or auth needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose, second adds usage and output highlights. No redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 3 parameters (all well-documented in schema) and no output schema, the description gives a solid overview of the return value and usage context. Could mention prerequisites like repository access, but overall complete enough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so each parameter is already documented. Description adds no additional parameter meaning beyond what schema provides, but it does describe output structure which is not part of parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states 'Analyze a pull request for quality risk' with a specific verb and resource, and distinguishes from siblings like analyze_release_readiness and generate_missing_coverage by focusing on risk analysis.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states 'Use before merging to catch regressions,' providing clear context. Does not mention when not to use or specific alternatives, but sibling list offers alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_release_readinessAnalyze release readinessA
Read-only

Get a tenant-wide release readiness assessment. Returns a ship/review/block recommendation with confidence score, coverage analysis, execution summary, script readiness, risk areas, recent failures, and recommended actions. Use before deciding whether to release.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=true, so the description properly confirms a read operation. It adds value by detailing the contents of the assessment (coverage analysis, execution summary, etc.), leaving no ambiguity about what the tool returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two efficient sentences: the first defines purpose and output, the second gives usage. No unnecessary words; content is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite lacking an output schema, the description enumerates all key return components (recommendation, confidence score, coverage analysis, etc.), fully informing the agent of what to expect. This compensates for the missing schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and schema coverage is 100%. No parameter information is needed, and the description adds nothing about parameters, which is appropriate. Baseline for 0 parameters is 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get a tenant-wide release readiness assessment.' It lists specific outputs (recommendation, confidence score, coverage analysis, etc.), making it distinct from sibling tools like analyze_pr_risk which focus on individual PR risk.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'Use before deciding whether to release.' This clearly indicates when to use the tool, though it does not explicitly mention when not to use it or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

build_regression_packBuild regression packA
Idempotent

Generate a focused regression test pack for a feature area or recent change. Use after a refactor, migration, or risky change to ensure the area stays stable. Returns a set of test cases targeting the specified area.

ParametersJSON Schema
NameRequiredDescriptionDefault
areaYesFeature area, system, or flow to protect (e.g., 'checkout', 'user authentication', 'payment processing').
contextNoRecent change description or additional risk context (e.g., 'migrated payment provider from Stripe to Adyen').

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate idempotentHint=true and destructiveHint=false. The description adds that the tool 'Returns a set of test cases', which is a fair behavioral note. However, it does not elaborate on permissions, rate limits, or other nuances. With annotations covering safety, a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences with no waste: purpose, usage guideline, and output. Front-loaded with the core action. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has only 2 parameters, no output schema, and annotations present, the description is adequately complete. It explains the output as a 'set of test cases', which is sufficient. Could mention idempotency explicitly, but annotations cover it. Slightly vague on output format, but not a major gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters (area, context) are fully described in the input schema with 100% coverage. The description adds minimal extra meaning beyond repeating schema descriptions. Baseline 3 is correct as the schema handles most of the parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Generate' and the resource 'regression test pack'. It specifies the focus on a feature area or recent change, distinguishing it from sibling tools like 'generate_missing_coverage' or 'analyze_pr_risk' which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: 'after a refactor, migration, or risky change'. Provides clear context with the example. While it doesn't explicitly state when not to use or directly compare with siblings, the guidance is strong and sufficient for most agents.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

convert_bug_to_testsConvert bug to regression testsA
Idempotent

Analyze a bug report and generate durable regression tests to prevent recurrence. Provide a detailed bug description for best results. Returns root cause analysis, affected areas, and generated test cases with steps.

ParametersJSON Schema
NameRequiredDescriptionDefault
severityNoBug severity hint β€” affects test priority ranking.
bug_descriptionYesDetailed bug report: what happened, expected behavior, actual behavior, and reproduction steps.

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate idempotent (idempotentHint=true) and non-destructive (destructiveHint=false). The description adds value by stating the tool returns 'root cause analysis, affected areas, and generated test cases with steps,' clarifying behavioral outcomes beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences, front-loaded with the core action and outputs, with no extraneous words. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the moderate complexity (2 parameters, 100% schema coverage, no output schema), the description sufficiently explains inputs and outputs. It could mention any prerequisites (e.g., existence of a test harness) but is broadly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds semantics: it explains that 'severity' influences 'test priority ranking,' which is not in the schema description. This extra context helps the agent use the parameter effectively.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('analyze' and 'generate') and resource ('bug report' and 'regression tests'), making the tool's purpose clear. It does not explicitly differentiate from siblings like 'build_regression_pack' or 'generate_test_scripts', but the focus on bug-to-test conversion is distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description advises providing a 'detailed bug description for best results,' which gives a usage hint but does not specify when to use this tool versus alternatives (e.g., when to use 'generate_test_scripts' instead) or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

explain_quality_riskExplain quality riskA
Read-only

Get a quality risk assessment using coverage heatmap, execution history, and 30-day trend data. Identifies high-risk and medium-risk areas. Use to answer questions like 'are we safe to ship?' or 'what areas need more tests?'

ParametersJSON Schema
NameRequiredDescriptionDefault
areaNoFocus the risk assessment on categories matching this string.
questionNoSpecific quality question to answer (e.g., 'Is the checkout flow well-tested?').

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=true and destructiveHint=false, so the agent knows this is safe. The description adds behavioral context: it uses coverage heatmap, execution history, and 30-day trend data to identify high- and medium-risk areas. This goes beyond annotations, though it does not detail data sources or outputs fully.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the tool's core action and data sources. Every sentence serves a purpose: the first explains what and how, the second gives usage examples. No extraneous words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description partially compensates by stating it identifies high- and medium-risk areas. Parameters are well-documented. However, it does not specify the output format (e.g., a list, scores, or visualization) or how the risk assessment is presented, which could leave the agent uncertain about return structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with both parameters described sufficiently in the input schema. The description adds example questions but does not elaborate on the 'area' parameter beyond what the schema says. It provides minimal additional semantic value over the schema, so a baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves a quality risk assessment using specific data sources (coverage heatmap, execution history, 30-day trend data) and identifies high/medium risk areas. Example questions like 'are we safe to ship?' further clarify purpose. The name 'explain_quality_risk' matches, and it is distinct from siblings like 'analyze_pr_risk' or 'analyze_release_readiness' which focus on different contexts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance via example questions ('are we safe to ship?', 'what areas need more tests?'), indicating when to use this tool. It does not explicitly state when not to use it or mention alternative tools, but the sibling list implies alternatives exist and the description sets a clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_missing_coverageGenerate missing coverageA

Generate test cases for a user story or feature that needs better coverage. Submits a test generation job and polls for results (up to 2 minutes). Returns categorized test cases with steps and expected results.

ParametersJSON Schema
NameRequiredDescriptionDefault
app_domainNoApplication domain (e.g., 'E-Commerce', 'Banking', 'Healthcare') to tailor test patterns.
user_storyYesUser story, feature description, or coverage gap to generate tests for.
app_contextNoApplication overview or technical context to improve test relevance.

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=false (mutation) and destructiveHint=false. The description adds that the tool submits a job and polls for results up to 2 minutes, which is key behavioral context beyond annotations. It does not cover failure handling or rate limits, but the polling timeout is valuable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences that front-load the core purpose and then describe the process and output. Every word adds value; no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description informs about the return format (categorized tests with steps and expected results). It also mentions the polling timeout. This is fairly complete for a job-submission tool, though it could mention potential error states or job failure behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter already has a description. The tool description does not add new semantics beyond summarizing the overall functionality. Baseline 3 is appropriate as the schema already does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it generates test cases for stories/features needing coverage, specifies the job submission and polling process with a 2-minute timeout, and mentions the output format (categorized tests with steps and expected results). This is specific and distinct from sibling tools like generate_test_scripts or convert_bug_to_tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage 'for a user story or feature that needs better coverage,' but does not explicitly compare to sibling tools or provide when-not-to-use guidance. For example, it doesn't contrast with 'generate_test_scripts' or 'convert_bug_to_tests,' leaving the agent to infer the best context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_test_scriptsGenerate executable test scriptsA

Convert test cases from a completed test generation job into executable test scripts. Requires a source_job_id from a prior generate_missing_coverage or convert_bug_to_tests call. Generates scripts for Playwright, Cypress, Selenium, or Puppeteer. Use this as the second step after generating test cases to get runnable automation code.

ParametersJSON Schema
NameRequiredDescriptionDefault
frameworkNoTarget test framework. 'all' generates for all frameworks. Defaults to playwright.
app_domainNoApplication domain for context (e.g., 'E-Commerce').
source_job_idYesJob ID from a completed test generation run (from generate_missing_coverage output).
test_case_idsNoSpecific test case IDs to convert. If omitted, converts all test cases from the source job.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate a non-readOnly and non-destructive operation. The description adds that it generates runnable automation code and mentions specific frameworks, but does not disclose rate limits, authentication needs, or details about side effects beyond the conversion.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with four sentences, front-loading the purpose. Every sentence adds value without redundancy or wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 4 parameters, 1 required, and no output schema, the description adequately covers the workflow: source job, framework selection, and optional test case filtering. It omits return format but is sufficient for agent invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all 4 parameters. The description adds minimal extra meaning, e.g., tying framework to the four options and mentioning the default. Since schema already does the heavy lifting, a 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool converts test cases into executable test scripts, specifies the required source_job_id, and lists supported frameworks. It distinguishes itself from siblings like generate_missing_coverage and convert_bug_to_tests by positioning itself as the second step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly requires a source_job_id from a prior generate_missing_coverage or convert_bug_to_tests call, and frames this as the second step. It does not explicitly exclude any scenarios or mention alternatives, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_coverage_gapsGet OrangePro coverage gapsA
Read-only

Find application areas lacking test coverage. Returns a heatmap of critical (red), partial (yellow), and healthy (green) coverage zones with test counts. Use to identify where to generate additional tests.

ParametersJSON Schema
NameRequiredDescriptionDefault
areaNoFilter results to categories matching this string (case-insensitive).

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as read-only and non-destructive. The description adds minor behavioral context (color coding, test counts) but does not need to go beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences convey purpose, output, and usage without waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one optional parameter and no output schema, the description fully covers what the tool does and what to expect.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides full description for the single parameter 'area' (case-insensitive filter). The tool description adds no further parameter details, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool finds application areas lacking test coverage and returns a heatmap with colored zones and test counts. It distinguishes from siblings like analyze_pr_risk or generate_missing_coverage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says 'Use to identify where to generate additional tests,' providing clear context. However, it does not mention when not to use or compare directly with sibling tools like generate_missing_coverage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

orangepro_get_agentGet OrangePro agentA
Read-only

Get full detail, configuration, timeline, and recent runs for a specific OrangePro agent. Use after orangepro_list_agents to inspect a particular agent. Returns agent config, run history, and current status.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesThe agent_id from orangepro_list_agents.
tenant_idNoOrangePro tenant id. Defaults to ORANGEPRO_TENANT_ID env var.

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows this is a safe read-only operation. The description adds that it returns 'agent config, run history, and current status,' which is helpful but does not go beyond what annotations guarantee. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, each serving a distinct purpose: first sentence defines the function, second sentence gives usage guidance and return summary. No filler words, front-loaded with the key action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that there is no output schema, the description adequately covers what the tool returns ('agent config, run history, and current status'). The tool is a single-agent getter, and the sibling tools cover more specialized queries, so this description is complete enough for its role.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with both agent_id and tenant_id described. The description adds value by clarifying that agent_id comes 'from orangepro_list_agents' and that tenant_id 'Defaults to ORANGEPRO_TENANT_ID env var,' which is useful context beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get full detail, configuration, timeline, and recent runs for a specific OrangePro agent.' It uses a specific verb ('Get') and a well-defined resource ('OrangePro agent'), and differentiates itself from sibling tools like orangepro_list_agents by focusing on inspecting a single agent in depth.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use after orangepro_list_agents to inspect a particular agent.' This provides clear context for when to use the tool. It does not explicitly state when not to use it or list alternatives, but the sibling tools (e.g., orangepro_get_agent_health, orangepro_get_agent_logs) suggest specialized uses, and the description implies this is a general inspection tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

orangepro_get_agent_healthGet OrangePro agent healthA
Read-only

Read health and connectivity status for an OrangePro agent. Use to diagnose why an agent is failing β€” checks source config, auth, and runtime status.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesThe agent_id to check health for.
tenant_idNoOrangePro tenant id. Defaults to ORANGEPRO_TENANT_ID env var.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint and destructiveHint. The description adds valuable context about what is checked (source config, auth, runtime status), surpassing the annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. First sentence concisely states purpose, second offers usage guidance. Well-structured and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple health check tool with only 2 parameters and no output schema, the description fully covers purpose, usage, and behavior. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters are well-documented. The description does not add parameter-level details but provides context for the tool's overall function. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Read' and resource 'health and connectivity status', and specifies it diagnoses agent failure by checking source config, auth, and runtime status. It effectively distinguishes from siblings like orangepro_get_agent or orangepro_get_agent_logs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states 'Use to diagnose why an agent is failing', providing a clear context. However, it does not explicitly mention when not to use or list alternatives, though sibling tools are available.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

orangepro_get_agent_logsGet OrangePro agent logsA
Read-only

Read recent log lines for an OrangePro agent. Use to debug failures, check processing details, or verify what an agent did during a run. Returns timestamped log lines.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax number of log lines to return. Default 100.
offsetNoNumber of log lines to skip for pagination.
agent_idYesThe agent_id to get logs for.
tenant_idNoOrangePro tenant id. Defaults to ORANGEPRO_TENANT_ID env var.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so description's additional mention of 'timestamped log lines' adds minor value beyond annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with verb and resource. Every sentence is informative and there is no waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations and full schema coverage, the description adequately explains the tool's purpose and behavior. Lacks explicit details on pagination, but schema covers limit/offset. Sufficient for the complexity level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and description does not add significant meaning beyond what the schema already provides for each parameter. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states verb ('Read'), resource ('log lines for an OrangePro agent'), and purpose with specific use cases (debug, check details, verify). Distinguishes from siblings like 'orangepro_get_agent' or 'orangepro_list_agent_runs'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: 'to debug failures, check processing details, or verify what an agent did during a run.' Does not provide explicit exclusions or compare to siblings, but context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

orangepro_list_agent_runsList OrangePro agent runsA
Read-only

List recent runs for a specific OrangePro agent. Use to check run history, find failed runs, or verify a recent run completed. Returns run_id, status, start time, duration, and records processed.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax number of runs to return. Default 20.
offsetNoNumber of runs to skip for pagination.
agent_idYesThe agent_id to list runs for.
tenant_idNoOrangePro tenant id. Defaults to ORANGEPRO_TENANT_ID env var.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so the description adds value by listing return fields (run_id, status, etc.). However, it does not disclose pagination behavior or ordering, which would be useful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with purpose, and no wasted words. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description includes return fields. It covers the main usage but could mention default ordering (e.g., recent first) or pagination hints. Overall adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description does not add significant extra meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List recent runs for a specific OrangePro agent' with a specific verb and resource. It distinguishes from sibling tools like orangepro_list_agents (lists agents) and orangepro_run_agent (run agent).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit use cases: 'check run history, find failed runs, or verify a recent run completed.' It does not explicitly mention when not to use or alternatives, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

orangepro_list_agentsList OrangePro agentsA
Read-only

List all configured OrangePro agents for a tenant. Use this first to discover available agents before getting details or triggering runs. Returns agent_id, name, type, status, and last run timestamp for each agent.

ParametersJSON Schema
NameRequiredDescriptionDefault
tenant_idNoOrangePro tenant id. Defaults to ORANGEPRO_TENANT_ID env var.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, indicating a safe read operation. The description confirms this by stating it returns a list of agents with specific fields, but does not add behavioral context beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the action and purpose, and contains no unnecessary words. It efficiently conveys the primary use case and return fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with no output schema, the description compensates by listing the returned fields (agent_id, name, type, status, last run timestamp). It does not mention pagination or ordering, but for a simple discovery tool, it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage for its single optional parameter (tenant_id) with a description. The tool description does not add additional parameter guidance beyond what is in the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists all configured OrangePro agents for a tenant, and specifies the returned fields (agent_id, name, type, status, last run timestamp). It distinctly differentiates itself from sibling tools like orangepro_get_agent (single agent) and orangepro_list_agent_runs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises to use this tool first to discover available agents before getting details or triggering runs, which provides clear guidance on when to use it. It does not list when not to use it or alternatives, but the context from sibling names implies the workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

orangepro_resolve_storyResolve story in OrangePro KGA
Idempotent

Resolve a user story, requirement, or feature description against the OrangePro Knowledge Graph. Returns grounded entities, matched concepts, and confidence scores. Use to verify story coverage or find KG gaps.

ParametersJSON Schema
NameRequiredDescriptionDefault
top_kNoNumber of top matches to return. Defaults to 5.
tenant_idNoOrangePro tenant id. Defaults to ORANGEPRO_TENANT_ID env var.
input_kindNoInput type: 'story', 'requirement', or 'feature'. Defaults to 'story'.
story_textYesThe user story, requirement, or feature text to resolve against the KG.
source_typeNoSource type: 'manual', 'jira', or 'github'. Defaults to 'manual'.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=true, destructiveHint=false. The description adds that it returns entities and scores, but does not disclose additional traits like required permissions or rate limits. With annotations covering safety, a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loading the action and outcome. Every sentence is informative and there is no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 5 parameters and no output schema, the description mentions return types (entities, concepts, confidence scores) which aids completeness. It could elaborate more on 'resolve' semantics or interpretation of scores, but overall it is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description provides no additional detail on parameters beyond what the schema already describes. No extra meaning is added.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'resolve' and resource 'OrangePro Knowledge Graph', clearly stating the tool's function. It lists return values (grounded entities, matched concepts, confidence scores) and allows it to be distinguished from sibling tools that deal with agents or test generation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use to verify story coverage or find KG gaps', providing clear usage context. However, it does not mention when to avoid this tool or suggest alternatives among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

orangepro_run_agentRun OrangePro agentA
Idempotent

Start an OrangePro agent run. Safe to retry β€” the API returns the active run if one is already in progress. Use this to trigger data ingestion, KG sync, or test generation agents.

ParametersJSON Schema
NameRequiredDescriptionDefault
agent_idYesThe agent_id to run. Get this from orangepro_list_agents.
tenant_idNoOrangePro tenant id. Defaults to ORANGEPRO_TENANT_ID env var.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description confirms the idempotentHint annotation by stating it's safe to retry and explains the API returns the active run if in progress, adding behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the action and safety note. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 2 parameters and no output schema, the description covers purpose, idempotency, common uses, and parameter origins. Nothing is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for both parameters. The description adds valuable context: agent_id is obtained from list_agents, tenant_id defaults to an environment variable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verbs ('Start'), identifies the resource ('OrangePro agent run'), and lists concrete use cases (data ingestion, KG sync, test generation). It clearly distinguishes from sibling tools like orangepro_list_agents.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states it is safe to retry and provides example use cases. While it doesn't list when not to use it or mention alternatives, the context and sibling tool list imply appropriate usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 15 tool updatesv0.2.0
    • First observedanalyze_pr_risk
    • First observedanalyze_release_readiness
    • First observedbuild_regression_pack
    • First observedconvert_bug_to_tests
    • First observedexplain_quality_risk
    • First observedgenerate_missing_coverage
    • First observedgenerate_test_scripts
    • First observedget_coverage_gaps
    • First observedorangepro_get_agent
    • First observedorangepro_get_agent_health
    • First observedorangepro_get_agent_logs
    • First observedorangepro_list_agent_runs
    • First observedorangepro_list_agents
    • First observedorangepro_resolve_story
    • First observedorangepro_run_agent

TDQS

A3.9/5.0
Disambiguation4/5

Tools are mostly distinct with clear purposes, though 'explain_quality_risk' and 'analyze_release_readiness' both address shipping safety and could be confused. The orangepro_ group is well-differentiated.

Naming Consistency3/5

Mixing snake_case verbs (analyze_pr_risk, build_regression_pack) with lowercase phrases (explain_quality_risk) and prefixed names (orangepro_list_agents) creates inconsistency. Some tools lack a consistent verb_noun pattern.

Tool Count4/5

15 tools is slightly above the ideal range but still reasonable for a quality and release management domain. Each tool serves a distinct purpose without unnecessary redundancy.

Completeness3/5

Covers risk analysis, test generation, and agent management, but lacks test execution, result reporting, and agent creation/deletion. Gaps exist in the full lifecycle of quality assurance.

Maintenance

ActivityActive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    Provides comprehensive codebase analysis including project structure evaluation, cross-language duplicate detection, microservices validation, and configuration optimization with AI-powered pattern learning that generates actionable improvement reports.
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    AI-powered characterization test generator that reads Python functions or class methods, synthesizes inputs, captures behavior in a sandbox, and emits pytest files to lock legacy code behavior for safe refactoring.
    4
    Apache 2.0
  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI coding agents to analyze codebase behavior through entrypoints, call paths, and tests, providing impact analysis and context packs for code changes.
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/OrangeproAI/orangepro-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server