sumo-qa
The sumo-qa server provides a comprehensive QA-native environment for analyzing repositories, mapping risks to tests, planning and executing test suites, reviewing diffs before merge, and managing QA knowledge — all grounded in ISTQB principles.
Core QA Workflows
Decide QA Approach – entry point for any QA intent; loads classifications and routes to the right sub-skill
Review Before Merge – assesses changed files, identifies risks, runs tests, and delivers a safe-to-merge verdict only with fresh evidence
Prepare for Work / Create Test Plan – lightweight planning (risks + minimum test set) or formal plans with scope, entry/exit criteria, and residual risks
Implement with TDD – walks red→green TDD cycles with confirmation gates
Strengthen Tests – mutation-testing follow-up; kills weak assertions without touching production code
Close QA Gaps – drives a single uncovered behavior to a regression test with a full evidence loop
Strategise / Plan & Execute QA Rollout – repo-wide audit, test pyramid design, and dispatching plans as parallel subagent tasks
Triage Test Failures – classifies failures (regression, test bug, environment, flakiness) and names the smallest isolation step
Answer Testing Questions / Security Testing – answers generic or security-specific questions by citing loaded principles and techniques
Finish QA Work – captures evidence, produces a PR-ready summary, and closes the QA loop
Repo Analysis
Scan Repo – walks the repository and produces a QA-relevant inventory (repo-map)
Analyze Diff Impact – maps changed files to tests, risk surfaces, and stale-map warnings
Query Repo Map – searches the repo-map for components, tests, CI checks, configs, or commands
Evidence & Reporting
Format Risk Ledger – renders a risk-to-test traceability ledger with uncovered-blocker counts
Format Context Bundle – renders an issue/PR context bundle with freshness warnings
Format QA Scorecard – derives a four-state verdict (ready / ready_with_accepted_residuals / blocked / insufficient_evidence)
Export Test Cases – exports structured test cases to markdown, JSON, or CSV
Generate QA Report – composes all
.sumo-qaartifacts into a self-contained HTML readiness pageRecord Coverage / Mutation – runs tools and persists results into
.sumo-qaartifacts
Test Data Management
Explain data requirements, search a known-good YAML catalogue, validate entries, and register new known-good data
Knowledge & Feedback Management
Load Classifications / Approaches / Principles / Techniques / Standards / Rules – retrieves canonical ISTQB and team-specific QA knowledge
Ingest Knowledge Pack – adds or replaces team QA knowledge from a local file
Capture Review Feedback – manages recurring QA findings as advisory hints for future reviews
Extensibility
Capabilities Discovery – lists all skills with sample prompts and outcomes
Progressive Skill Loading – loads skill slices (manifest/section/module) to stay within token budgets
External Skill Lifecycle – search, install (with user confirmation), and execute external agent skills for capabilities sumo-qa doesn't cover natively
Integrates with JetBrains IDEs to bring QA capabilities such as test planning, TDD, mutation testing, and code review to the development workflow.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@sumo-qaCreate a test plan for the user authentication module"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
sumo-qa MCP
A QA-native MCP server and skills library: analyze a repo, map risks to tests and evidence, plan and scaffold tests, strengthen weak suites, and review diffs before merge with senior-QA discipline.
sumo-qa is an advisor, not an oracle. Like any AI tool it can be wrong. Your judgment and your team's standards are the final word.
🚀 New here? 5-minute demo →
One install line, then the QA loop on your own repo: map it, then review a change for a merge verdict or audit the whole repo for a strategy.
Why it exists
AI writes more of your code every month, and it's confident about all of it: "done", "looks good", "safe to merge". But confidence isn't evidence. It will call a change safe off CI that ran an hour ago, write tests that pass by restating the code they're meant to check, and declare work finished without running anything. You can't tell what was actually verified from what merely sounds verified, and you're still the one who has to answer for what shipped.
sumo-qa exists to make AI-written code something you can stand behind. It holds any agent, in any host and on any stack, to a senior QA's discipline on its own work, and makes it show the evidence: risks tied to specific files and lines, each mapped to a test that ran this turn, and a safe-to-merge verdict you could hand a reviewer instead of "the model said it was fine". The same standard every time, not each model's own idea of "tested".
Ask a stock assistant to QA a change and you get the junior answer: "add unit tests, consider edge cases, maybe test performance." sumo-qa makes it work like the senior who has to sign off on what ships:
Reads the repo first (what it does, which tests cover which code), then names risks tied to specific files and lines, not vague categories
Picks a real design technique per risk from an ISTQB-grounded catalogue, instead of generating tests by vibe
Runs your suite fresh in the current turn before any "safe to merge" claim, and refuses "CI was green earlier"
Holds TDD's red phase before any production code, and keeps production code locked while it strengthens tests against mutation survivors
Won't ship a plan or a verdict without measurable, checkable criteria, and will say "no tests needed here" when that's the honest call
The discipline lives in a library of skill files, each followed literally; every one has an Iron Law and a HARD-GATE callout the LLM can't talk past. Skills route automatically from natural-language prompts; you don't need to remember to invoke them.
Related MCP server: Architect-to-Product (A2P)
Install
pip install sumo-qa && sumo-qa-installThen restart your host or open a fresh chat. sumo-qa-install configures every MCP host it detects. Windows, single-host targeting, the one-command install.sh / install.ps1 wrappers, plugin installs, and updating are all in docs/INSTALL.md.
Verify
sumo-qa-doctorGreen means you're wired. The checks are read-only, and each failure prints the exact Fix: to run. Then ask any host "load the QA classifications"; the canonical names back confirm it end to end. (Troubleshooting →)
Host support
Every host calls the same MCP server and reads the same SKILL.md files. What differs is how each host exposes them. That's a host-API difference, not a sumo-qa choice.
sumo-qa-install sets these hosts up. It wires Claude Code and VS Code directly; for the JetBrains hosts it prints the exact config for you to paste:
Host | Slash | Setup |
Claude Code |
|
|
VS Code + Copilot (Agent mode, Claude Sonnet 4.5 or equivalent) | Natural language |
|
JetBrains AI Assistant |
| One-time UI setup; |
JetBrains Junie | Natural language | Create |
In Claude Code, type / then sumo-qa- to see the skills as hyphenated entries (symlinked into ~/.claude/skills/). The same skills are also registered through MCP with underscores (/sumo_qa_load_classifications, /sumo_qa_find_test_data); both routes call the same SKILL.md.
Natural language works everywhere: "Review my changes", "plan QA for this story", "load the QA classifications" all route by tool description. Slash and natural-language paths produce the same result.
Other MCP hosts (Cursor, Codex, OpenCode, Gemini CLI, etc.): pip install sumo-qa ships a standard stdio MCP server, so it should work with anything that speaks MCP. Follow your host's MCP-server setup docs and point it at the absolute path of the sumo-qa script. These hosts are explicitly not yet verified end-to-end by us, so we don't ship per-host instructions for them.
Host adapter folders
sumo-qa ships generated plugin manifest folders (.claude-plugin/ for Claude Code, .codex-plugin/ for Codex, not yet verified end-to-end), built from a single canonical source in pyproject.toml and drift-checked in CI on every PR. Architecture, per-host install status, and how to add a host: docs/host-adapters.md.
See it in action
Transcripts showing the workflow on real code: diff reviews refusing to call safe-to-merge from stale CI, TDD cycles with the red output surfaced verbatim, mutation survivors walked one at a time, formal test plans gated on entry/exit criteria, and the case where the right answer is "no tests needed, stop here":
tests/scenarios/worked-examples/: start with 02 review my changes for a representative end-to-end
tests/scenarios/SCENARIOS.md: the scenario specs (prompt → expected shape → anti-patterns the skill prevents)
What's included
Three layers: the host LLM follows skills, which cite knowledge and standards to produce the output. The architecture doc has the diagram and the full data flow.
Layer | What |
Skills ( | Iron-Law procedures across the QA lifecycle: deciding approach, preparing for work, TDD scaffolding, diff review, strengthening tests, finding test data, answering testing questions, repo strategy, plus the planning → parallel subagent execution → finishing chain. |
MCP entry points | A thin tool surface: skill tools, knowledge loaders, a capabilities-discovery tool, repo-map tools, test-data tools, an ingestion tool, and external-skill lifecycle tools. Each is file IO or small deterministic logic; no inference. |
Progressive skill loading | A read-only loader that fetches a skill in slices (routing manifest → section → module → full body), so a host pays the routing slice on each revisit, not the whole body. See docs/TOOLS.md and docs/SKILLS.md. |
Knowledge catalogues ( | Classifications, approaches, principles, techniques the agent picks from instead of recalling from training data. Editable as plain markdown. (Specialty-tool picks are deliberately not catalogued; observe the risk surface and web-search current options instead.) |
Run it from the terminal
Beyond the host integration, sumo-qa ships terminal commands for the QA-native repo loop:
sumo-qa analyze # map the current repo into .sumo-qa/repo-map.json
sumo-qa status # is the map present and current against HEAD? what next?
sumo-qa report # compose the .sumo-qa artifacts into qa-report.htmlAll take an optional [path] and --json; report renders honest not-available states for anything missing. (Bare sumo-qa launches the MCP server; sumo-qa-doctor runs diagnostics.)
When sumo-qa doesn't fit
If your QA intent has no native fit (Playwright E2E, accessibility audits, k6 load testing, type checking), sumo-qa searches for an external skill through its MCP server, offers a [y/N] install gate, installs through the Skills CLI, then loads the installed SKILL.md back into the conversation.
The host never runs npx directly; four MCP tools own the lifecycle (search → [y/N] gate → install → load), and search returns the Skills CLI output verbatim, so there's no parser to drift. Node.js is required; if npx is missing the tool returns an actionable error rather than elevating. Any machine-level install the external skill suggests is translated to sumo-qa's repo-pinned, CI-reproducible standard, and sumo-qa keeps its confirmation gates, test evidence, and risk-to-test mapping.
Support
Filing a clear issue gets it fixed faster. Pick the template that matches the problem:
Symptom | Template |
| |
Install worked, but the host (Claude Code, VS Code + Copilot, JetBrains, Cursor, …) does not surface tools or skills correctly | |
The wrong sumo-qa skill ran for a prompt (or none ran when one should have) | |
A skill ran, but its QA output was generic, wrong, or missed something | |
You want a new workflow, skill, or host integration | |
Reproducible defect that does not fit the above |
License
Apache 2.0. See NOTICE for attribution requirements that apply to forks and redistributors.
More docs
AGENTS.md: AI-agent bootstrap and per-host setup
docs/ARCHITECTURE.md: three layers, host delivery, knowledge authority
docs/SKILLS.md: every skill with its Iron Law
docs/TOOLS.md: every MCP entry point
docs/INSTALL.md: per-host install detail and troubleshooting
docs/CONTENT-FORMATS.md: schemas + worked examples for adding team standards, knowledge, change rules, and test data (incl. swapping ISTQB out)
docs/CONFIGURATION.md: env vars
docs/DEVELOPMENT.md: local dev
docs/TEST-DATA.md: known-good test-data catalogue
docs/REPO-MAP.md: QA-native repo-map artifact under
.sumo-qa/: schema, scanner, and the scan / diff-impact / query tools that consume it (issues #155, #156)docs/RISK-LEDGER.md: risk-to-test traceability ledger: the structured appendix to the markdown-first verdict, its row schema and evidence-status vocabulary, and when not to use it (issue #144)
docs/SCORECARD.md: QA readiness scorecard: composes the risk ledger + context bundle + optional coverage/mutation into a derived readiness recommendation (ready / blocked / insufficient_evidence / ready-with-accepted-residuals): an evidence summary, not a predictive quality score (issue #151)
docs/GATE-EVIDENCE.md: evidence-backed gate reporting: the gate-status and evidence-source vocabulary, the schema and validators that reject an unsupported "tests passed" / "safe to merge" claim, and the honesty boundary the skills now carry (issue #213)
docs/RELIABILITY-HARDENING.md: the epic #211 control matrix: for each reliability weakness (prompt dependence, verbosity fatigue, enforcement uncertainty, shallow static analysis) the merged control, its proof artifacts, the backward-compatible default, and the accepted residual limit (issue #211)
docs/EXPORT.md: deterministic export of already-structured QA test cases to versioned JSON, a markdown table, or (flat-only) CSV: the case schema, the format set, the side-effect-free contract, and the import-mapping caveat (issue #148)
docs/QA-REPORT.md: local QA report: the static
.sumo-qa/qa-report.htmlpage composed from the persisted artifacts, the four honest artifact states, and a readiness verdict derived by the #151QaScorecardengine (issue #157)docs/PERSONA.md: optional Sumo-sensei voice (off by default)
Available Tools
48 toolssumo_qa_analyze_diff_impactA
Map a set of changed files onto the repo-map to report which tests likely exercise them, which changed sources have no mapped test (the risk surface), one-hop affected nodes, unmapped files, and whether the map is stale relative to HEAD.
Common natural-language phrasings that map to this tool: "what does this diff affect", "which tests cover my changes", "what's the risk surface of this branch", "what should I re-test after these edits", "analyse the impact of the changes against main".
root is the repository. Supply changed_files (repo-relative
paths) OR base_ref (any git ref; changed files are the diff against
the merge-base of base_ref and HEAD, so changes that landed on the
base after the branch diverged don't leak in). The repo-map is read
from artifact_path when present and falls back to a live scan
otherwise; an artifact for a different project root is ignored. On the
first run of an unmapped repo the live scan is persisted to
artifact_path (reported as persisted_map_path) unless
artifact_path is None. write_overlay writes
.sumo-qa/diff-impact.json under root. When test files exist but
the map has no likely_tests edges, probable_mapping_gap flags the
risk surface as a missed-convention gap rather than true zero coverage.
| Name | Required | Description | Default |
|---|---|---|---|
| root | Yes | ||
| base_ref | No | ||
| artifact_path | No | .sumo-qa/repo-map.json | |
| changed_files | No | ||
| write_overlay | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behavioral traits: writes overlay file, persists map on first run, handles stale maps, and mutual exclusivity of inputs. Annotations are minimal (readOnlyHint=false, etc.), so the description adds significant value beyond them. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is relatively long but well-structured: purpose first, then phrasings, then parameter details. Every sentence adds value. Could be slightly more concise, but front-loaded with core output and efficiently organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, describes the output components (likely tests, risk surface, etc.) and edge cases (stale map, first-run, probable_mapping_gap). Provides a complete picture of what the tool does and returns, sufficient for an AI agent to understand usage and outcomes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0% description coverage, but the tool description thoroughly explains each parameter: root, changed_files, base_ref, artifact_path, write_overlay. It clarifies defaults, mutual exclusivity, and behavior (e.g., artifact path fallback, first-run persistence). This fully compensates for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool maps changed files to tests and reports risk surface, likely tests, etc. Uses specific verbs and resources, and lists common user phrasings, distinguishing it from sibling tools like 'sumo_qa_query_repo_map'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context on when to use (analyze diff impact, find test coverage). Explains the two input modes (changed_files or base_ref) and relationship. Does not explicitly mention when not to use or contrast with alternatives, but the natural language mappings serve as good usage cues.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_answering_testing_questionA
Use when the user asks a generic testing question — "how do I test this?", "what should I check for X?" — that doesn't fit a more specific QA skill. Cites a principle or technique from the loaded catalogue rather than producing generic advice.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description implies it is a read-only retrieval (citing from catalogue) but does not explicitly state non-destructive behavior or other traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise three-line description with front-loaded directive, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Sufficient for a no-parameter, no-output-schema tool; references catalogue which is known from sibling tools, but could mention output format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so description does not need to add parameter meaning. Baseline 4 for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool answers generic testing questions and cites a principle or technique from the catalogue, distinguishing it from more specific QA skills.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly specifies when to use ('generic testing question not fitting a more specific skill') and implies when not to use, providing clear guidance among many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_capabilitiesARead-onlyIdempotent
Return a compact, read-only map of sumo-qa's core QA workflows — each with a sample prompt, the skill it routes to, and a one-line outcome. A discovery aid for "what can sumo-qa do?"; does NOT replace the using-sumo-qa entry router or sumo_qa_deciding_approach.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, non-destructive, idempotent. Description adds that it returns a 'compact map' but doesn't detail exact structure or side effects. No contradictions, and description provides additional context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, clear and front-loaded. Every sentence earns its place, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, description is fully sufficient. It explains purpose, usage boundaries, and return content. Mentions relevant sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, so baseline 4. Description does not need to add param info, but it reinforces the read-only nature. No issues.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns a compact map of core QA workflows with sample prompts, skills, and outcomes. It distinguishes itself from siblings like using_sumo_qa and sumo_qa_deciding_approach by framing itself as a discovery aid.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states it does NOT replace the entry router or deciding_approach tool, providing clear when-not-to-use guidance. Also implies it is for discovering capabilities when asking 'what can sumo-qa do?'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_capture_review_feedbackA
Manage an EXPLICIT, user-confirmed review-feedback memory of recurring QA findings.
Promotes a recurring review lesson (e.g. "we always miss timezone
boundaries in billing") into a local, inspectable, reversible memory that
future planning/review skills consult as an ADVISORY hint — NOT automatic
learning. action selects the operation:
'capture'— add a new lesson (or replace one with the sameid). Requiresentrywithscope,trigger_signal,recommended_probe,source_note, and optionallast_reviewed(ISO-8601; defaults to now).'update'— replace the fields of an existing lesson; needsentry_idplusentry.'delete'— remove a lesson byentry_id.'list'(default) — return stored lessons, advisory-flagged. Thescopedefault is the literal'project', so it lists the current repo; passscope='global'for the cross-repo set. An unrecognisedscopereturns an error envelope (it is never coerced to project).
NEVER persist without explicit user confirmation, and NEVER auto-capture
from a review/prompt/trace. That confirmation gate is the HOST/skill's
responsibility, not enforced by a tool parameter — the deliberate
writer-local data-ownership model shared with the risk-ledger and AC
tools; the sumo-qa-feedback CLI correspondingly exposes only list/delete,
so a capture can never be a fire-and-forget flag. Sensitive input — a raw
diff hunk, a secret, a code snippet, or a pasted full issue/PR body — is
REJECTED; only the user's own summary is stored, and a rejected entry is
never echoed to the debug-capture sink either. Storage reuses the #92 user-writable pack
location (project = /.sumo-qa, global = the user data dir) under
a feedback/ subdir, so it is NOT a second hidden tree. Memory-derived
probes are ADVISORY: cite them SEPARATELY from bundled ISTQB/rules
content; they never override canonical classifications or change-rules.
Common natural-language phrasings that map to this tool: "remember that we always miss X in Y", "save this review lesson", "promote this recurring finding to team memory", "what review lessons have we saved?", "forget the timezone-billing lesson".
| Name | Required | Description | Default |
|---|---|---|---|
| entry | No | ||
| scope | No | project | |
| action | No | list | |
| entry_id | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are sparse (readOnlyHint=false, etc.), but the description adds extensive behavioral context: memory is advisory, not automatic; sensitive input is rejected; storage location is specified; and memory-derived probes must be cited separately. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is thorough but well-structured with clear sections for each action, constraints, and examples. It is front-loaded with a concise purpose statement. A slight reduction because it is longer than necessary, but every sentence provides value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a CRUD-like tool with no output schema, the description fully covers all operations, including what list returns (stored lessons), error cases (unrecognized scope), and safety constraints. It is complete for an AI agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, so the description fully compensates by explaining each action's required fields: capture requires entry with specific subfields; update requires entry_id plus entry; delete needs entry_id; list defaults. It also explains the scope parameter defaults and behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool manages an explicit, user-confirmed review-feedback memory of recurring QA findings. It lists four actions (capture, update, delete, list) and distinguishes this tool from sibling tools by its unique purpose as a memory storage tool for review lessons.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: never persist without explicit user confirmation, never auto-capture, and lists common natural-language phrasings to help the agent map user intent. It also describes error handling for unrecognized scope and differentiates from other tools by mentioning the shared data-ownership model with risk-ledger and AC tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_check_external_skill_installedARead-onlyIdempotent
Locate an installed external SKILL.md file for Codex, Claude, or agents paths.
Returns the first matching path for project or global skill locations, or null when the skill is absent.
| Name | Required | Description | Default |
|---|---|---|---|
| scope | No | auto | |
| skill | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only and non-destructive behavior. The description adds context that it searches project or global skill locations and returns null if absent, which is beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with a clear front-loaded action and return explanation. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple check tool, it covers purpose and return value, but lacks parameter details that would aid correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description does not explain the parameters 'skill' or 'scope' beyond their implicit role. The format or allowed values for 'skill' and the meaning of 'scope' are unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool locates an installed external SKILL.md file and returns the first matching path or null. It distinguishes itself from sibling tools like sumo_qa_install_external_skill or sumo_qa_search_external_skills.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use before executing a skill to check installation, but lacks explicit guidance on when not to use it or mention of alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_closing_qa_gapsA
Use when a review, mutation run, or graded scenario has already named a concrete uncovered behavior gap and the user wants it closed with evidence — e.g. 'close this gap', 'drive this uncovered risk to a regression test', 'fix this surviving mutant end to end'. Drives ONE closed loop per gap — failing test → red evidence → minimal fix → green evidence → risk/ledger update. Pauses when repo context or evidence is insufficient.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full disclosure burden. It transparently outlines the tool's behavioral loop: failing test → red evidence → minimal fix → green evidence → risk/ledger update. Also notes it pauses when insufficient context/evidence. This gives the agent a good understanding of the process, though it could mention that it modifies code and risk ledger explicitly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: starts with usage condition, then workflow summary, then a caveat. It is four sentences, front-loaded. Some redundancy in the examples could be trimmed, but overall every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, the description covers the tool's input assumptions (named gap, evidence) and process steps. It implies the output (updated risk ledger, tests, etc.) but does not explicitly state what the tool returns to the agent. A slight gap in specifying return behavior, but sufficient for an experienced agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the description's role is to clarify what the tool uses from context. It explains that the tool acts on a previously named concrete gap and requires evidence, which is sufficient. No parameter documentation needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the tool's purpose: closing a concrete uncovered behavior gap with evidence (failing test -> red evidence -> minimal fix -> green evidence -> update). It distinguishes from siblings by focusing on a single closed loop per gap and pausing when insufficient, which is distinct from other qa tools like measuring coverage or strengthening tests.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the condition for use: after a review, mutation run, or graded scenario has named a concrete uncovered behavior gap. Provides examples of user requests. Implicitly excludes cases without a named gap, and mentions pausing when repo context or evidence is insufficient. Lacks explicit when-not-to-use or alternatives, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_creating_test_planA
Use when the user asks for a formal test plan, entry/exit criteria, or a phased QA approach for a piece of work. Walk the user through scope → risks → entry criteria → phases → exit criteria → residual risks one section at a time, getting confirmation before each step. Heavier than sumo-qa-preparing-for-work; use when the work is tracked or formally reviewed.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavior: walking user through a step-by-step process (scope → risks → entry criteria → phases → exit criteria → residual risks) with confirmation at each step. It also notes the tool is 'heavier' than another, giving a clear behavioral distinction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences with no wasted words. It front-loads the purpose, then details the process, and ends with a sibling comparison. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and no output schema, the description covers everything needed: what it does, how it behaves (step-by-step with confirmation), and when to use it over alternatives. It is fully complete for its context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so schema coverage is 100%. According to guidelines, baseline is 4. The description correctly omits parameter details as none exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: creating formal test plans, entry/exit criteria, or phased QA approaches. It explicitly differentiates from the sibling 'sumo-qa-preparing-for-work' by noting this tool is heavier and for tracked/formally reviewed work.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use (user asks for formal test plan, etc.) and contrasts with a sibling tool. However, it does not list all possible alternatives from the sibling list, leaving some ambiguity about when other tools might be preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_deciding_approachA
Use as the FIRST step on any QA intent. Loads classifications + approaches (the two needed to route), then reasons over the user's intent to pick the canonical approach and routes to the matching sub-skill (which loads any further catalogues on demand).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses the tool's behavior: loading classifications and approaches, reasoning, picking an approach, routing to sub-skill, and on-demand loading of further catalogues. It does not mention side effects, but the behavior is read-like and well-explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff. The critical 'first step' guidance is front-loaded, and the second sentence provides essential detail on reasoning and routing. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity as a router in a QA workflow, the description lacks explicit mention of return values or state changes. Without an output schema, the agent is left to infer what the tool returns after routing. The description is near-complete but missing return semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters and schema coverage is 100%, so baseline score is 4. The description does not need to add parameter semantics as there are none.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is the first step for any QA intent, loads specific resources (classifications and approaches), reasons over user intent, and routes to the appropriate sub-skill. It distinguishes itself from sibling tools like sumo_qa_load_approaches and sumo_qa_load_classifications by combining loading with reasoning and routing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states 'Use as the FIRST step on any QA intent', providing clear when-to-use guidance. It does not explicitly mention when not to use or list alternatives, but the context of sibling tools suggests specialization.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_execute_external_skillARead-onlyIdempotent
Load an installed external SKILL.md and return the execution handoff.
The payload contains the skill body plus the original intent so the host can follow the external workflow in the current conversation.
| Name | Required | Description | Default |
|---|---|---|---|
| scope | No | auto | |
| skill | Yes | ||
| intent | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the description's statement 'return the execution handoff' aligns. The description adds value by explaining the handoff payload includes 'the skill body plus the original intent', which is useful context beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences. The first sentence front-loads the core purpose, and the second adds detail about the payload. No redundant words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of output schema and incomplete parameter documentation, the description covers the basic purpose and return value but lacks details on parameter semantics and return format. It is minimally adequate for understanding the main function but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description should compensate by explaining the parameters. However, it mentions 'skill body' but does not clarify the meaning of the three parameters (skill, intent, scope). The tool name suggests 'skill' is the file, but intent and scope remain unexplained, leaving a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool loads an installed external SKILL.md and returns an execution handoff. The verb 'Load' and resource 'SKILL.md' are specific. However, it does not explicitly differentiate from sibling tools like sumo_qa_load_skill_context, which might also load skill-related data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when an external skill is available and needs to be executed, but it lacks explicit guidance on when to use this tool versus alternatives like sumo_qa_check_external_skill_installed. No when-not-to-use or exclusion criteria are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_executing_qa_rolloutA
Use after sumo-qa-planning-qa-rollout to dispatch a written QA plan task-by-task. Each task runs in a fresh subagent (parallel where independent); each subagent's output goes through a two-stage review (test-correctness → test-quality) before the task is marked done. Continuous execution — no per-task check-ins. Finishes by routing to sumo-qa-finishing-qa-work.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that tasks run in fresh subagents (parallel where independent), with two-stage review and no per-task check-ins. Adds useful context beyond typical annotations, but lacks details on failure handling or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: first sets purpose and predecessor, second explains execution mechanism, third notes continuous execution and successor. No fluff, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, no-output-schema tool, the description covers workflow position, execution model, and successor. However, it lacks error handling behavior (e.g., what happens if a subagent or review fails), which would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has zero properties, so no parameter information needed. The description naturally omits parameter details, and the baseline for zero parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it dispatches a QA plan task-by-task after planning, runs subagents with parallel execution and two-stage review, and finishes by routing to a finishing tool. It distinguishes itself from siblings by specifying its position in the workflow (after planning, before finishing).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use after sumo-qa-planning-qa-rollout' and mentions routing to sumo-qa-finishing-qa-work, giving clear when-to-use context. It does not provide exclusions or alternatives for when not to use, but the sequential workflow implies usage only in that context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_explain_test_data_requirementsARead-onlyIdempotent
Explain what test data shape and characteristics are needed for a scenario.
Returns: required entity characteristics, resource-state conditions,
scenario preconditions, downstream dependencies, edge cases, and
explicit "what NOT to use" guidance. Domain-neutral by design — works
for any domain (auth, billing, retail, infrastructure, ML, etc.).
Optional environment (e.g. "integration") and domain are folded
into the analysis.
Common natural-language phrasings that map to this tool: "what data do I need to test X", "what test data should I look for to cover X", "what records / accounts / fixtures do I need for X", "what's the minimum data setup for X", "what edge-case data should I test".
| Name | Required | Description | Default |
|---|---|---|---|
| domain | No | ||
| question | Yes | ||
| environment | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds behavioral detail: it returns entity characteristics, conditions, dependencies, edge cases, and 'what NOT to use' guidance, and states it is domain-neutral. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a concise one-sentence summary, a bullet-like list of outputs, a domain-neutrality note, optional parameter clarification, and example phrasings. It is front-loaded and relevant, though slightly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explicitly lists what the tool returns (entity characteristics, conditions, dependencies, edge cases, etc.) and covers optional parameters. It fully explains the tool's functionality and output scope, making it complete for its purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the burden. It explains that `environment` and `domain` are optional and 'folded into the analysis', and implies `question` is the main input. This adds some meaning but does not specify exact formats or constraints, which is adequate but not thorough.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool explains what test data shape and characteristics are needed for a scenario. It lists specific return items. However, it does not explicitly differentiate from sibling tools like sumo_qa_finding_test_data or sumo_qa_validate_test_data, though the purpose is distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides common natural-language phrasings that map to this tool, giving clear use cases. It does not mention when to avoid using the tool or compare with alternatives, but the examples are helpful for appropriate invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_export_test_casesA
Deterministically EXPORT already-structured QA test cases into one
documented machine-readable shape (issue #148). FILE/FORMAT PLUMBING ONLY
— the host LLM identifies the cases; this tool never infers them and never
inspects a repo. By DEFAULT it is side-effect free (it RETURNS the rendered
text and writes nothing); a file is persisted ONLY when an explicit
output_path is supplied.
Each case is a dict with: id (stable within this export), title,
preconditions (ordered list, may be empty), steps (ordered list,
may be empty), expected_result, optional linked_risk_id (a risk id
in a companion risk ledger), priority (one of critical / high / medium
/ low), and evidence_status (one of planned / passing / failing /
stale / accepted_residual — the same vocabulary as the risk ledger).
format is one of: markdown (the DEFAULT human-facing table),
json (a versioned, key-sorted, deterministic document), or csv
(OPTIONAL, and only valid for a flat outline — at most one precondition
and one step per case). An unsupported format, or CSV for a non-flat
export, returns an error envelope naming the supported formats. Tool-
specific import mappings may need local adjustment.
export_title (optional) names the export as a whole — rendered in the
markdown header and the JSON top-level title. (It is named
export_title, not title, so it is distinct from each case's own
title and survives the served-schema title-slimming pass.)
output_path (optional) is the EXPLICIT file-write carve-out. When
omitted (the default) nothing is written. When given, the SAME rendered
bytes are ALSO persisted, confined to the project export root
(<cwd>/.sumo-qa/exports): a relative path resolves under that root, an
absolute path or .. traversal that escapes it is refused, and an
already-existing target is refused rather than silently overwritten. The
write only happens AFTER successful validation+render, so a bad export
never leaves a file. On a successful write written_path carries the
resolved absolute path (else None).
Returns the rendered content, the chosen format, the stamped
schema_version, the validated test_case_count, and written_path
(the persisted location, or None on the default no-write path).
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | markdown | |
| test_cases | Yes | ||
| output_path | No | ||
| export_title | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description thoroughly explains side-effect behavior, file-writing rules, error handling, and the case structure. It goes well beyond the schema and annotations, providing comprehensive behavioral transparency. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is comprehensive but well-structured, starting with core purpose then detailing case structure and parameters. Every sentence adds value, though it is slightly lengthy. It is front-loaded and organized effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (multiple parameters, optional file writing, return structure), the description covers all necessary aspects: input constraints, edge cases, error handling, and return fields. Despite no output schema, return values are explicitly listed. High completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully compensates by exhaustively explaining each parameter: test_cases structure, format enum and constraints, export_title purpose, and output_path file write rules. Adds immense meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly names the verb 'EXPORT' and specifies the resource 'already-structured QA test cases'. It clarifies what the tool does not do (infer or inspect), distinguishing it from other tools in the suite. Purpose is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states it is only for plumbing and that the host LLM identifies cases, implying use when cases are already known. While it does not explicitly compare to sibling tools or provide when-not-to-use guidance, the context is clear enough for an agent to understand when to invoke it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_finding_test_dataA
Use when the user asks about test data — what data to test X, find a known-good record, validate an entry, register new known-good data. Routes between sumo_qa_explain_test_data_requirements, sumo_qa_find_test_data, sumo_qa_validate_test_data, and sumo_qa_register_known_good_test_data.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states that it routes between tools but does not disclose how routing works, what happens after routing, or any behavioral traits like authentication needs or result format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Front-loads use cases and provides key routing information efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and zero parameters, the description lacks explanation of what the tool returns or how the routing is presented. An agent needs to know the output format to use the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters; baseline is 4. The description does not need to add parameter meaning since there are none.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it is a router for test data queries, listing specific scenarios and naming the four sub-tools it routes between. This distinguishes it from sibling tools that are direct executors.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says 'Use when the user asks about test data' and lists example queries, providing clear context. However, it does not mention when not to use it or alternative tools for non-test-data queries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_find_test_dataARead-onlyIdempotent
Search the local known-good test data catalogue for entries that match a scenario.
Returns: ranked matches with confidence, freshness, and suitability
reasons. Reads the local YAML catalogue under knowledge/test_data/
only; no external lookups. Optional scenario_tags and known_valid_for
narrow the search.
Pagination: pass offset to skip the first N matches, and read
total_count, has_more, and next_offset on the response to walk
pages. When has_more is false, next_offset is null.
Common natural-language phrasings that map to this tool: "find me test data for X", "do we have a known-good record for X", "give me an account / fixture / record that does X", "is there a fixture for X", "what test data is available for X".
| Name | Required | Description | Default |
|---|---|---|---|
| sku | No | ||
| limit | No | ||
| domain | No | ||
| offset | No | ||
| product_id | No | ||
| environment | No | ||
| scenario_tags | No | Scenario tags to match against catalogue entries. | |
| known_valid_for | No | Use-case labels the entry has been validated for. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate the tool is read-only, non-destructive, idempotent, and not open-world. The description adds useful behavioral context: it reads only the local YAML catalogue (no external lookups) and explains pagination behavior. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a clear one-sentence summary, followed by return value details, scope limitations, optional filters, pagination instructions, and example phrasings. Every sentence adds value, and the content is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, input parameters, return structure (ranked matches with confidence, freshness, suitability reasons), pagination fields, and typical use cases. It lacks detailed return field descriptions but is reasonably complete given the complexity and absence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With only 25% schema coverage, the description partially compensates by explaining the purpose of `scenario_tags` and `known_valid_for` to narrow the search, and describing `offset` and `limit` for pagination. However, parameters like `environment`, `domain`, `product_id`, and `sku` are not mentioned, leaving gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states clearly that the tool searches the local known-good test data catalogue for scenario matches. It specifies scope ('local YAML catalogue') and differentiates from external lookups, but does not explicitly distinguish it from the sibling tool 'sumo_qa_finding_test_data', which may serve a similar purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides good context on when to use the tool, such as narrowing search with optional tags and labels. It also includes pagination information and common natural-language phrasings. However, it does not explicitly state when not to use this tool or mention alternatives, which would improve guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_finishing_qa_workA
Use at the end of a QA rollout (after sumo-qa-executing-qa-rollout, or after a manual multi-step QA task) to capture evidence, produce a PR-ready summary, and close the loop. Runs the suite one last time, captures coverage / risk-to-test map / open follow-ups, writes a markdown summary to docs/qa/runs/YYYY-MM-DD-.md, and offers to draft the PR description.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and details the tool's actions: runs the suite, captures coverage/risk-to-test map/open follow-ups, writes a markdown summary, and offers to draft a PR description. While it doesn't cover every possible side effect, it provides substantial transparency for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise yet packed with essential information. It front-loads the usage context and clearly lists all key actions without unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description fully explains what the tool does and what it produces (markdown summary and PR draft), which is complete given there is no output schema. It leaves no ambiguity about the tool's role in the QA workflow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description does not need to explain parameters. The baseline for 0 params is 4, and the description adds value by detailing the tool's outputs and workflow beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to finish a QA rollout by capturing evidence, producing a PR-ready summary, and closing the loop. It explicitly mentions preceding actions and distinguishes itself from sibling tools by specifying when it should be used.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use: 'at the end of a QA rollout (after sumo-qa-executing-qa-rollout, or after a manual multi-step QA task)'. This provides clear context, though it does not explicitly mention when not to use or list alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_format_context_bundleARead-onlyIdempotent
Validate and render a host-neutral issue/PR CONTEXT BUNDLE as a compact markdown brief for QA review/planning (issue #149). FILE/FORMAT PLUMBING ONLY — the host gathers the facts; this tool never inspects a repo, makes a network call, or assumes GitHub. A partial/empty bundle is first-class: when little is supplied, the consuming skill falls back to direct repo inspection.
Common natural-language phrasings that map to this tool: "build the review context bundle", "format this PR/issue context for review", "render the context bundle with its freshness", "summarise the diff/CI/test facts I gathered".
bundle is a dict with optional issue_summary, pr_summary,
head_sha, changed_files (each {path, change_kind}),
test_evidence / ci_status (each {result, freshness, source},
plus optional captured_at / detail), and user_constraints.
freshness is one of fresh/stale/unknown/absent; only a FRESH PASS is
safety-supporting — a stale, unknown, or absent fact is rendered with an
explicit "do not claim safety from it" warning. Supply local_head_sha
(the host's live local head) to detect a bundle-vs-local-state conflict;
when the shas differ the brief calls out the divergence instead of
trusting either side. max_files bounds the changed-file list.
| Name | Required | Description | Default |
|---|---|---|---|
| bundle | Yes | ||
| max_files | No | ||
| local_head_sha | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only and idempotent. The description adds that it never inspects repos, never makes network calls, and explains behaviors with partial bundles, freshness semantics, and sha conflict detection, with no contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear front-loaded purpose, followed by constraints, natural-language phrasings, and detailed parameter info. It is slightly verbose but each sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (nested objects, no output schema), the description covers bundle structure, freshness, conflict detection, and fallback behavior. It notes the output is a compact markdown brief, though finer output details are omitted, which is acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description thoroughly explains the 'bundle' object structure, including optional fields and 'freshness' semantics. It also details 'local_head_sha' for conflict detection and 'max_files' for bounding, adding significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool validates and renders a context bundle into a markdown brief for QA review/planning, with clear verb and resource. It distinguishes itself from siblings by noting it never inspects repos or makes network calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides common natural-language phrasings and constraints like 'the host gathers the facts'. It implies when not to use (not for data gathering), but does not explicitly list alternative tools for specific cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_format_qa_scorecardARead-onlyIdempotent
Compose a QA READINESS SCORECARD from already-produced evidence and DERIVE a readiness recommendation (issue #151). EVIDENCE SUMMARY, NOT a predictive quality score — it infers no risk, invents no numeric score, and the host can never assert "ready": the verdict is computed.
Common natural-language phrasings that map to this tool: "is this ready to merge/release", "give me a readiness scorecard", "summarise QA readiness", "release review summary".
Inputs are all optional and reuse the existing artifacts — nothing is re-defined here:
ledger_rows— the #144 risk-to-test rows (same shape assumo_qa_format_risk_ledger); supplies risk coverage + blockers.context_bundle— the #149 bundle (same shape assumo_qa_format_context_bundle); supplies test/CI evidence freshness.coverage/mutation— optional{..., freshness}signals; absent ⇒ reported as "not measured", never assumed passing, and never allowed to outweigh an uncovered high-impact risk.scope— optional label (a PR title, a release name).local_head_sha— optional live local head, to flag a stale bundle.
Returns the four-state recommendation (ready / ready_with_accepted_
residuals / blocked / insufficient_evidence), is_ready (true only for
the two ready states), the uncovered-blocker / residual counts, the
stale-evidence and not-measured dimension lists, the rendered markdown,
a one-line compact_summary to drop inline in short answers, and a
JSON-able serialized snapshot a downstream report (#157) can render.
Readiness is refused whenever a risk is an uncovered blocker or evidence
is stale — that guarantee is structural, not advisory.
| Name | Required | Description | Default |
|---|---|---|---|
| scope | No | ||
| coverage | No | ||
| mutation | No | ||
| ledger_rows | No | ||
| max_reasons | No | ||
| context_bundle | No | ||
| local_head_sha | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds significant behavioral detail beyond annotations: 'infers no risk, invents no numeric score', 'host can never assert ready', structural guarantee on readiness refusal. Annotations already indicate readOnly and idempotent, so description enriches with precise conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with bullet points and clear sections. Slightly verbose in places, but every sentence adds value. Appropriate length for complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers input semantics, output fields (four-state recommendation, counts, rendered markdown), and behavioral guarantees. Despite no output schema, description fully specifies return values. References to other tools provide necessary context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description explains each parameter in detail: ledger_rows, context_bundle, coverage, mutation, scope, local_head_sha, max_reasons. Provides semantics, allowed shapes, and behavior when absent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Compose a QA READINESS SCORECARD' and derives a readiness recommendation, with specific verb and resource. It distinguishes from siblings by referencing their output shapes (risk ledger, context bundle) and natural language phrasings mapping to this tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says inputs are optional and reuse existing artifacts, and provides common phrasings. Does not explicitly exclude alternatives, but context from siblings and description implies this is the final readiness assessment tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_format_risk_ledgerARead-onlyIdempotent
Validate and render a risk-to-test traceability ledger as a markdown appendix (issue #144). FILE/FORMAT PLUMBING ONLY — the host LLM identifies the risks; this tool never infers them.
Each row is a dict with: risk_id (stable within this response),
risk (the statement), source_anchor (file:line or domain term),
test (a test id OR a 'planned: …' check), evidence_status (one of
planned / passing / failing / stale / accepted_residual), residual
(one of open / accepted / mitigated / blocker), and an optional
repo_map_node_id linking to a .sumo-qa/repo-map.json node.
Returns the rendered markdown table (the structured appendix the
markdown-first verdict carries), a one-line compact summary, the row
count, and the count of uncovered blockers (rows that are not passing,
not accepted, and marked residual=blocker — the signal the review
workflow uses to refuse safe-to-merge). The table is bounded by
max_rows so a large ledger stays inside the host token budget.
| Name | Required | Description | Default |
|---|---|---|---|
| rows | Yes | ||
| max_rows | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, non-destructive, idempotent. Description adds details about bounded output (max_rows), return fields (table, summary, counts, blockers), and clarifies it never infers risks, enhancing transparency beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with early purpose statement and capitalized emphasis. Thorough but not bloated; every sentence adds value. Slightly verbose given the detailed row specification, but justifiable for complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema present; description covers return fields (table, summary, row count, uncovered blockers) and row structure. Lacks error conditions or input validation, but sufficient for a format-only tool with robust annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description provides extensive meaning for both parameters: explodes the row object structure (risk_id, risk, source_anchor, test, evidence_status, residual, optional repo_map_node_id) and explains max_rows bounds output. Greatly compensates for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it validates and renders a risk-to-test traceability ledger as a markdown appendix, distinguishing it from sibling tools by emphasizing format plumbing only, no inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implicitly suggests use after risk identification via 'FILE/FORMAT PLUMBING ONLY — the host LLM identifies the risks', but no explicit when-not or alternatives among many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_generate_qa_reportA
Compose the persisted .sumo-qa artifacts (repo map, diff
impact, risk ledger, context bundle) into the local QA report and
return a compact readiness summary. The
rendered HTML body never rides back to the host — pass write_to to
persist the self-contained static page and open it from disk.
Common natural-language phrasings that map to this tool: "generate the QA report", "build the QA dashboard for this repo", "give me the local QA readiness report", "render qa-report.html".
root is the repository to report on (absolute or relative to the
MCP server's working directory). Every artifact is OPTIONAL: a missing,
invalid, or stale source renders an explicit honest state. The readiness
verdict (ready / ready_with_accepted_residuals / blocked /
insufficient_evidence) is derived by the QaScorecard readiness engine
from the risk ledger + context bundle — missing data is never reported
as passing evidence.
risk_ledger_rows / context_bundle are inline overrides for the
chat flow where the ledger/bundle was built in-conversation and never
persisted (the same shapes sumo_qa_format_risk_ledger /
sumo_qa_format_context_bundle accept). They take precedence over
any on-disk file and are validated BEFORE anything is written.
write_to is optional — when set, the page is written there
(relative paths land under the target repo; the conventional value is
.sumo-qa/qa-report.html). Without it the tool writes nothing.
| Name | Required | Description | Default |
|---|---|---|---|
| root | Yes | ||
| write_to | No | ||
| context_bundle | No | ||
| risk_ledger_rows | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the HTML body never rides back to the host, writing only when write_to is set, and overrides are validated. Annotations are minimal, so the description carries the burden well, though it doesn't mention performance implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, natural-language examples, and parameter details, but it is somewhat lengthy and includes redundant explanations (e.g., 'missing data is never reported as passing evidence' is restated).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main use case, parameter behavior, and edge cases like missing artifacts. No output schema exists, but the return value is described as a 'compact readiness summary' with readiness verdict. Could detail the summary format more.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description explains each parameter in detail: root as repository path, write_to as optional output path, and the inline override parameters with their shapes and precedence. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool composes .sumo-qa artifacts into a local QA report and returns a readiness summary. The verb 'compose' and mention of specific artifacts distinguish it from siblings like sumo_qa_format_risk_ledger, which handle individual pieces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides natural-language phrasings and explains when to use the tool (after artifacts exist). It details optional parameters and precedence, but does not explicitly state when not to use it or compare directly with alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_implementing_with_tddA
Use after sumo-qa-deciding-approach picks tdd-scaffold, regression-first, or coverage-first-then-refactor — e.g. "write a regression test for this bug" or "scaffold the failing tests first". Walks plan → name-the-risk-and-test-idea → confirm → red → hand off → green → review, one section per turn with confirmation gates. Don't write the test until the test idea has been agreed.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries burden. It outlines the multi-step process with confirmation gates, and discloses critical behavioral constraint (defer writing until agreement). However, lacks details on failure modes or state changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Front-loads sibling dependency. Could be structured with bullets for clarity, but current form is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and no output schema, description covers usage and process adequately. Lacks mention of return values or next steps, but tool likely yields conversation prompts. Adequate but could be more thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema, so baseline is 4. Description adds no parameter info (unneeded) but provides process context that adds value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states this tool implements TDD after a decision from 'sumo-qa-deciding-approach', with a specific process (plan → risk → test idea → confirm → red → hand off → green → review). It distinguishes itself from siblings by specifying it's used only after a TDD approach is chosen.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use after sumo-qa-deciding-approach picks...', provides example prompts, and warns 'Don't write the test until the test idea has been agreed.' This gives clear context for when and how to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_ingest_knowledge_packA
Adds or replaces team QA knowledge/standards/rules from a local native file.
Accepts a path to a native sumo-qa file or a directory of them:
principles.md, techniques.md, classifications.md, approaches.md, a
standards-pack *.yaml, or change_rules.yaml. Validates the content and
writes a normalized copy into a user-writable pack. The scope argument
selects where it lands: 'project' (/.sumo-qa, the current repo
only) or 'global' (the user data dir, every repo) — the right scope is
a user choice worth confirming. Loader precedence is env var > project >
global > bundled > repo root.
A PDF / PPTX / URL or any other non-native source is not parsed here; it
returns an unsupported_source result that routes through the
sumo-qa-suggesting-external-skill flow to convert the source to
markdown, which is then re-ingested with an explicit content_type.
Common natural-language phrasings that map to this tool: "add this to the knowledge base", "replace our principles", "load our team standards pack", "use these change rules", "ingest this QA pack".
| Name | Required | Description | Default |
|---|---|---|---|
| scope | No | project | |
| source | Yes | ||
| content_type | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (all false) are minimal. The description adds value by explaining validation, normalized copy writing, loader precedence, and the unsupported_source result. However, it could detail more about return values beyond error cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with front-loaded main action, detailed paragraphs, and even natural-language phrasings. Slightly verbose in places (e.g., loader precedence details) but every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and sparse annotations, the description covers all necessary aspects: inputs, behavior, edge cases, and alternative flows. It fully equips the agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description fully explains all three parameters: source (path), scope (project vs global with reasoning), and content_type (for non-native conversion). This compensates completely for the missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a clear verb+resource statement: 'Adds or replaces team QA knowledge/standards/rules from a local native file.' It specifies accepted file types and distinguishes from non-native sources, making it distinct from sibling tools like load_* which are read-only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (for native sumo-qa files) and when not to use (PDF/PPTX/URL), directing to an alternative flow via sumo_qa_suggesting_external_skill. Also clarifies scope choice and loader precedence, providing comprehensive guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_install_external_skillA
Install an external agent skill through the Skills CLI.
The confirmed flag records that the host received explicit user approval before invoking the install operation.
| Name | Required | Description | Default |
|---|---|---|---|
| agent | No | codex | |
| scope | No | project | |
| skill | Yes | ||
| source | No | https://github.com/vercel-labs/skills | |
| confirmed | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and openWorldHint=true. The description adds behavioral context by explaining the confirmed flag 'records that the host received explicit user approval before invoking the install operation', which goes beyond annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two clear, front-loaded sentences with no wasted words efficiently convey the tool's purpose and a key behavioral note.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate given the open-world hint and annotations, but lacks details on success/failure outcomes, side effects, or parameter interactions, which would improve completeness for an install tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. However, it only mentions the 'confirmed' parameter briefly, leaving four other parameters (skill, source, scope, agent) without any explanation of their meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Install an external agent skill through the Skills CLI', using a specific verb and resource that distinguishes it from siblings like check_external_skill_installed or execute_external_skill.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It implies usage for installing skills and mentions the 'confirmed' flag for user approval, but does not explicitly state when to use this tool versus alternatives or provide prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_list_skill_manifestsARead-onlyIdempotent
Return deterministic metadata for every bundled sumo-qa skill as a JSON string — a routing/index aid, NOT the skill bodies.
detail controls how much per-skill index is included (default
"compact"):
"compact" — routing metadata only: skill_name, tool_name (the zero-argument skill tool), description (from frontmatter), content_hash (sha256 of the SKILL.md) and estimated_tokens_full. NO sections[]/modules[] arrays — the cheap all-skill routing slice.
"full_index" — the same metadata PLUS each skill's sections[] (id, heading, level, estimated_tokens, required) and modules[] (id, path, estimated_tokens) index arrays. Section ids are stable heading slugs (duplicates get
-2/-3suffixes); required marks the structural sections (frontmatter, Iron Law, Checklist, Flow, Red Flags, HARD-GATE) when present.
Once routing has chosen one skill, fetch that skill's section/module
index with sumo_qa_load_skill_context(skill_name, mode="manifest"),
then a single slice via mode="section"/"module"/"full".
An unrecognised detail returns a JSON error envelope listing the valid
values rather than raising. Read-only and local-only: no network, no
extraction, no caching. The existing zero-argument skill tools still
return full bodies unchanged.
| Name | Required | Description | Default |
|---|---|---|---|
| detail | No | compact |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (readOnlyHint, idempotentHint), the description adds important behavioral details: local-only, no network/extraction/caching, deterministic, and error handling for unrecognised detail values.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded, but slightly verbose with repeated mentions of skill index arrays. However, every sentence adds value and the organization is logical.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description fully describes the return format for both detail options, explains error handling, and provides context for how this tool fits with others (e.g., sumo_qa_load_skill_context).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage, the description exhaustively explains the single parameter 'detail' with two options ('compact' and 'full_index'), detailing what each returns, and notes behavior for unrecognised values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it returns deterministic metadata for every bundled skill as a JSON string and clarifies it is a routing/index aid, not skill bodies. This clearly distinguishes its purpose from sibling tools like sumo_qa_load_skill_context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use this tool (for routing/index) and provides explicit alternatives: 'Once routing has chosen one skill, fetch that skill's section/module index with sumo_qa_load_skill_context(...)' and mentions that zero-argument skill tools still return full bodies unchanged.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_load_approachesARead-onlyIdempotent
Return the canonical QA approaches as plain text. The host LLM picks which approach fits a given piece of work.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the tool is safe and idempotent. The description adds value by specifying that output is plain text, which helps set expectations. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundancy. The verb is front-loaded, and every sentence adds value. The description is efficiently structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters, annotations, and no output schema, the description is quite complete. It explains what the tool returns and why it is used. A slight improvement could be to define 'canonical QA approaches' or mention related tools, but it's adequate for a simple retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and the input schema coverage is 100%. The description does not need to explain parameters, and its focus on the return value is appropriate. It adds meaning beyond the schema by explaining the purpose of the output.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool returns canonical QA approaches as plain text. It uses a specific verb ('Return') and identifies the resource ('canonical QA approaches'). While it doesn't explicitly differentiate from siblings like sumo_qa_deciding_approach or other load tools, the purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions that the host LLM picks which approach fits a piece of work, implying usage for selecting approaches. However, it lacks explicit guidance on when not to use this tool or alternatives (e.g., sumo_qa_deciding_approach). Usage is implied but not clearly delineated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_load_catalogue_entryARead-onlyIdempotent
Load a single catalogue entry, or a whole catalogue in compact
form, as a JSON string — a lighter alternative to the full-text
loaders for one of the four prose catalogues: classifications,
approaches, principles, techniques.
With
nameset: return one entry.namematches the stable slug id (api_contract_change,equivalence-partitioning) or the verbatim heading text (case-insensitive).With
nameomitted: return the whole catalogue.format="full"(default) returns the verbatim catalogue text;format="compact"returns one lead-line summary per entry.
format: "full" (default) returns verbatim entry text marked
canonical=true — safe to cite. "compact" returns a truncated
summary marked canonical=false — a navigation/recall aid, NOT a
citation replacement; load the full form (or the zero-argument
sumo_qa_load_* loader) when exact wording matters.
Never raises: an unknown catalogue, name, or format returns a JSON
error envelope listing the valid choices. The existing
zero-argument sumo_qa_load_* loaders are unchanged. Read-only and
local-only.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| format | No | full | |
| catalogue | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral details beyond annotations: it is read-only and local-only, never raises errors (returns error envelope), and marks output with canonical flags. This complements the readOnlyHint and idempotentHint annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with bullet points and clear sections, but is slightly verbose. Every sentence adds value, though some redundancy could be trimmed without loss of meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given three parameters, no output schema, and many sibling tools, the description is thorough: it covers input combinations, output behavior (canonical flags), error handling, and ties to other loaders. The agent has all needed information to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully explains each parameter: 'catalogue' must be one of four values, 'name' matches slug or heading text, 'format' has detailed semantics including defaults and canonical marking. This adds essential context missing from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it loads a single catalogue entry or whole catalogue as JSON, and distinguishes itself as 'a lighter alternative to the full-text loaders' for four specific prose catalogues. It explicitly names the catalogues and contrasts with sibling zero-argument loaders.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance: use with 'name' for a single entry, omit for whole catalogue; choose 'full' format for safe citation, 'compact' for navigation only. It also notes that the existing zero-argument loaders remain unchanged, helping agents decide between alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_load_classificationsARead-onlyIdempotent
Return the canonical change classifications as plain text. The host LLM picks which apply to a given change.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds that it returns plain text and that the host LLM picks applicable classifications, providing context beyond annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences front-load the purpose and succinctly cover return type and usage. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no parameters and no output schema, the description completely covers purpose, return format, and usage context. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, schema coverage is 100%, so the description adds value by explaining the return format and usage, meeting the baseline for zero-param tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns 'canonical change classifications as plain text', with a specific verb and resource. It distinguishes from sibling load_* tools that load other entities like approaches, rules, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage: when the host LLM needs to pick classifications for a change. It doesn't explicitly state when not to use, but the context is clear among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_load_principlesARead-onlyIdempotent
Return ISTQB Foundation + Advanced + ISO 25010 grounding as plain text. The host LLM cites principles when shaping recommendations.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds minimal behavioral context beyond the usage hint ('cites principles') and does not disclose any additional behavioral traits such as output size or performance implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the action, and contains no redundant information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters, no output schema, and a straightforward purpose, the description adequately explains what the tool returns. However, it could be more complete by specifying the format or length of the output, or by contrasting with similar load tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and 100% schema coverage, the baseline is 4. The description does not need to add parameter information, but it does clarify what the output contains, which is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns ISTQB Foundation, Advanced, and ISO 25010 grounding as plain text, using a specific verb 'Return' and resource. It distinguishes from sibling load tools that handle different content types (approaches, classifications, etc.).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used when citing principles in recommendations, but provides no explicit guidance on when to use this versus alternative load tools (e.g., load_standards) or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_load_rulesARead-onlyIdempotent
Return the team's loaded change rules as plain text. Optional classification filter accepts single or comma-separated values and returns matching rule entries; no keyword inference.
| Name | Required | Description | Default |
|---|---|---|---|
| classification | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive. The description adds that it returns plain text and that 'no keyword inference' occurs, but could mention default behavior when no filter is applied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main purpose, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one optional param, the description covers the main behavior and return format. Minor gap: no mention of what happens if no classification is provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% coverage for the classification parameter, but the description adds that it accepts 'single or comma-separated values', which is crucial for correct usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns 'the team's loaded change rules as plain text' and specifies an optional filter, distinguishing it from sibling tools like sumo_qa_load_classifications.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the filter but lacks explicit guidance on when to use this tool versus alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_load_skill_contextARead-onlyIdempotent
Load just one slice of a skill's context as a JSON string, instead of the whole SKILL.md body.
mode:
"manifest" — routing summary + section list + module list;
"section" — one section's text (pass
section, an id from the manifest);"module" — one module's text (pass
module, an id from the manifest);"full" — the entire SKILL.md body, byte-for-byte identical to the existing zero-argument skill tool for
skill_name; a body over the host's per-response token cap is returned as anoversizepointer to the manifest/section/module slices instead of failing (#393).
The section/module/full slices each return content_hash (sha256 of the
returned text) and estimated_tokens. Pass known_hash to ask "has this
slice changed since hash X?": a match returns changed=false with the
body omitted (saving the re-send), a mismatch returns changed=true with
the body. This is derived per call — there is NO hidden session cache, so
it is safe across hosts regardless of MCP session identity.
Never raises: an unknown skill_name/mode/section/module, a missing required arg, or a path-traversal attempt returns a JSON error envelope listing the valid choices. Read-only and local-only.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | ||
| module | No | ||
| section | No | ||
| known_hash | No | ||
| skill_name | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite annotations already indicating readOnlyHint=true, idempotentHint=true, the description adds significant value: it explains the known_hash change detection mechanism, error handling (never raises, returns JSON error envelope), and guarantees about being local-only and safe across hosts. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with bullet points for modes and clear separation of concepts. It is slightly lengthy but every sentence adds value, and it is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (5 parameters, no output schema), the description is thorough: it covers modes, parameter usage, error handling, change detection, and safety. It does not formally describe the response structure but mentions key fields like content_hash and estimated_tokens, which is sufficient for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully compensates by detailing the allowed values for mode (manifest, section, module, full) and how section and module parameters correspond to manifest IDs. It also explains known_hash usage. All 5 parameters are meaningfully described.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool loads a slice of a skill's context as a JSON string, explicitly listing modes (manifest, section, module, full). It distinguishes this from loading the whole SKILL.md body and from sibling tools like sumo_qa_load_approaches.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use each mode, especially noting that 'full' mode is identical to the existing zero-argument skill tool and that oversize bodies return a pointer to slices. It provides clear context for when to use this tool over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_load_standardsARead-onlyIdempotent
Return the team's loaded standards packs as plain text. Optional classification filter is metadata-based and accepts comma-separated values (packs whose frontmatter declares any requested classification); no keyword inference.
| Name | Required | Description | Default |
|---|---|---|---|
| classification | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds behavioral context: the filter is metadata-based (frontmatter) and returns plain text, which is not in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no fluff. The most important information (what it returns and the filter behavior) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description says 'as plain text' but doesn't specify the exact format or if there are size limits. While readOnlyHint ensures safety, more detail on the output structure would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully explains the classification parameter: it accepts comma-separated values, is metadata-based, and does no keyword inference, which is essential for correct usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb 'Return' and noun 'loaded standards packs as plain text', clearly distinguishing from sibling tools like sumo_qa_load_approaches or sumo_qa_load_classifications which load different content types.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the optional classification filter and explicitly states 'no keyword inference', setting clear expectations. While it doesn't explicitly contrast with alternatives, the tool name and context make the use case obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_load_techniquesARead-onlyIdempotent
Return the test design technique catalogue (black-box, white-box, experience-based, static, property-based, mutation) as plain text. The host LLM picks one technique per named risk.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint, idempotentHint. Description adds that output is plain text and describes usage pattern, which is beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. Information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only tool with rich annotations, the description fully covers purpose and usage context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters; baseline 4 as schema coverage is 100% and description doesn't need to add param info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it returns the test design technique catalogue as plain text, listing example techniques. This distinguishes it from sibling 'load_' tools like load_approaches or load_classifications.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
States 'The host LLM picks one technique per named risk', giving context for when to use it. However, no explicit when-not-to-use or alternatives mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_measuring_coverageA
Use when the user wants coverage or mutation numbers in the local QA report — "run coverage", "measure coverage", "run mutation testing", "record the survivors". Runs the repo's configured coverage/mutation tool, reads its output (any format), and persists a compact summary into the .sumo-qa artifacts the report loads.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears full burden. It discloses that the tool runs the configured coverage/mutation tool, reads its output, and persists a summary. This is transparent but omits potential failure modes or configuration requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with usage guidance, and every sentence adds distinct value. No redundant or extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters or output schema, the description adequately covers when and how to use it. It explains the process and artifact location, fitting the complexity of the tool well.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and the schema coverage is 100%. The description adds value by explaining the tool's actions but does not need to elaborate on parameters. Baseline of 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the verb (runs, reads, persists) and resource (coverage/mutation tool, .sumo-qa artifacts). However, it does not explicitly distinguish itself from sibling tools like sumo_qa_record_coverage and sumo_qa_record_mutation, which may overlap in purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage context ('Use when the user wants coverage or mutation numbers') and lists example phrases. However, it lacks guidance on when not to use this tool or mention of alternative tools for similar tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_planning_qa_rolloutA
Use when you have a chunk of QA work (a story, a PR, a strategy phase) that needs to be turned into a written plan of bite-sized, independently-executable tasks before any test code is written. Walks scope → file structure → bite-sized tasks → confirm, one section per turn. Produces docs/qa/plans/YYYY-MM-DD-.md ready for subagent dispatch via sumo-qa-executing-qa-rollout.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the burden. It explains the step-by-step process ('Walks scope → file structure → bite-sized tasks → confirm, one section per turn') and the output file creation. This provides good behavioral context, though it omits details like whether it modifies existing files or requires specific permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (3 sentences), front-loaded with the use condition, and every sentence serves a purpose: when to use, what it does, and what the output is. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, no output schema, and no annotations, the description provides a comprehensive overview: use case, process, output path, and relation to a sibling tool. It could be slightly more explicit about the tool's behavior if the conversation lacks the needed context, but overall it is complete enough for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and full schema coverage, the baseline is 4. The description adds value by explaining that the tool expects context from the conversation (the 'chunk of QA work') rather than formal parameters, which is meaningful for the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it converts QA work into a written plan of bite-sized tasks, with a specific output file path. It distinguishes itself from sibling tools like sumo_qa_executing_qa_rollout by being the planning step before execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use when you have a chunk of QA work...' and mentions the prerequisite of no test code written. It also notes the file is ready for subagent dispatch via a sibling tool. However, it does not explicitly state when not to use it or list alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_preparing_for_workA
Use when the user asks to plan QA for a story, ticket, or piece of work before coding starts. Identifies named risks anchored in the change shape, then proposes a smallest useful test set tied to those risks. Lighter-weight than sumo-qa-creating-test-plan; no formal entry/exit criteria.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explains the tool produces risk identification and a test proposal. While no annotations are available, the description implies no destructive actions, which is appropriate for a planning tool. A slightly more explicit statement about being non-destructive would enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two sentences that front-load the usage scenario and core functionality, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately explains what the tool does and its context (pre-coding planning, risk identification, test set proposal). It could note the output format, but it is sufficient for a lightweight analysis tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters, the description does not need to augment parameter meanings. The baseline for zero parameters is 4, and the description adds value by explaining the tool's context and output.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to plan QA for a story/ticket before coding starts, identifying risks and proposing a test set. It distinguishes from the sibling 'sumo-qa-creating-test-plan' by noting it is lighter-weight and lacks formal criteria.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use ('when user asks to plan QA... before coding starts') and contrasts with the heavier sibling tool, providing clear guidance on when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_query_repo_mapARead-onlyIdempotent
Search the repo-map for the components, tests, CI checks, configs, or commands that match a query, returning a bounded, ranked list with enough metadata (id, path, type, tags, match reason) to open the files directly — never the full artifact.
Common natural-language phrasings that map to this tool: "find the repo-map node for X", "which tests are mapped to the billing module", "list the CI workflows in the map", "what commands does the repo-map know about", "search the map for files tagged mcp".
root is the repository. query matches case-insensitively across
node id, path, file name, type, category, and tags, and across command
names and kinds; results rank exact identity above substring hits.
limit caps the returned matches (total_matches still reports the
full count). types restricts the search to given node types and/or
the literal "command". The repo-map is read from artifact_path
when present and falls back to a live scan otherwise; an artifact for a
different project root is ignored.
| Name | Required | Description | Default |
|---|---|---|---|
| root | Yes | ||
| limit | No | ||
| query | Yes | ||
| types | No | ||
| artifact_path | No | .sumo-qa/repo-map.json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds significant behavioral details beyond annotations: ranking (exact identity above substring hits), limit behavior (total_matches still reports full count), types restriction, and fallback logic for artifact_path (live scan if missing, ignores wrong project root). This fully discloses operational behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is approximately 150 words, well-organized into paragraphs and a bullet-like list of examples. It is front-loaded with the main purpose and each sentence adds value, with no redundancy or filler. The structure makes it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, behavior, parameters, and output (metadata fields like id, path, type, tags, match reason). It lacks details on response structure or pagination for large result sets, but the limit parameter and mention of bounded ranked list suffice. Annotations cover safety and idempotency. Overall, it is well-rounded and sufficient for the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the full burden. It explains all five parameters: root, query (case-insensitive matching across multiple fields), limit (caps matches), types (restricts to node types or 'command'), and artifact_path (default, fallback, and ignore condition). Additionally, it describes default values and behavior, compensating fully for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it searches the repo-map for specific items (components, tests, CI checks, etc.) and returns metadata to open files directly, never the full artifact. It includes natural language examples and distinguishes from potential siblings by specifying the bounded, ranked list output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides natural-language phrasings that map to the tool, implying appropriate usage contexts. However, it does not explicitly state when not to use it or compare it to alternatives like sumo_qa_scan_repo, leaving the agent to infer usage boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_record_coverageA
Validate a host-collected coverage summary and persist it as the
.sumo-qa/coverage.json artifact the QA report loads (issue #147
follow-up). FILE/FORMAT PLUMBING ONLY — the host skill runs the coverage
tool and the LLM reads its output (any format); this tool runs nothing
and infers nothing.
Common natural-language phrasings that map to this tool: "record the coverage result", "save coverage into the QA report", "persist the coverage summary".
coverage is a dict with optional line_percent (0–100),
freshness (fresh/stale/unknown/absent), detail (e.g. uncovered
changed files), plus source_tool and generated_at provenance.
Omit line_percent for a not-measured signal. Validation fails BEFORE
any write. Coverage is REPORTED, never gated — it cannot flip a verdict.
write_to defaults to the conventional .sumo-qa/coverage.json
under the target repo; a relative path is confined to root.
| Name | Required | Description | Default |
|---|---|---|---|
| root | Yes | ||
| coverage | Yes | ||
| write_to | No | .sumo-qa/coverage.json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description fully discloses behavioral traits beyond annotations: validation fails before write, coverage is reported not gated, and the tool's role as plumbing. Annotations are all false and consistent with the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections (purpose, phrasings, parameter details, behavior) and front-loaded. It is slightly lengthy but every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter tool with nested objects and no output schema, the description is quite complete. It covers inputs, validation behavior, and special cases (omit line_percent). A minor gap is the lack of return value information, but the tool likely has no significant output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description thoroughly explains the 'coverage' dict structure (including optional fields and semantics) and the 'write_to' default and path constraint. 'root' is implied but not deeply detailed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates and persists a coverage summary as a specific artifact (.sumo-qa/coverage.json). It explicitly distinguishes this as 'plumbing only' and differentiates from siblings by focusing on recording rather than measuring coverage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit common phrasings mapping to the tool and clarifies what it does not do (runs nothing, infers nothing). However, it does not explicitly list alternatives or when not to use it, though the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_record_mutationA
Validate a host-collected mutation summary and persist it as the
.sumo-qa/mutation.json artifact the QA report loads (issue #147
follow-up). FILE/FORMAT PLUMBING ONLY — the host skill runs the mutation
tool and the LLM reads its output (any format); this tool runs nothing
and infers nothing.
Common natural-language phrasings that map to this tool: "record the mutation result", "save the survivors into the QA report", "persist the mutation summary".
mutation is a dict with optional survivors (>= 0), killed
(>= 0), freshness (fresh/stale/unknown/absent), detail (e.g.
where survivors live), plus source_tool and generated_at
provenance. Omit the counts for a not-measured signal. Validation fails
BEFORE any write. Mutation evidence is REPORTED, never gated.
write_to defaults to the conventional .sumo-qa/mutation.json
under the target repo; a relative path is confined to root.
| Name | Required | Description | Default |
|---|---|---|---|
| root | Yes | ||
| mutation | Yes | ||
| write_to | No | .sumo-qa/mutation.json |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are minimal (readOnlyHint=false, etc.), but the description adds significant behavioral detail: validation occurs before write, evidence is reported not gated, and it runs nothing. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the main action, includes common phrasings, and details parameters concisely. Every sentence adds value, and the structure is logical and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's role, parameter details, and validation behavior. It could mention what happens on validation failure or error cases, but for a plumbing tool with good coverage of inputs, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage in the description, it explains the `mutation` parameter's structure (optional fields, counts, provenance) and `write_to` defaults/path confinement. `root` is not explained but is a common path parameter. This compensates well for the lack of formal param descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates and persists a mutation summary to a specific artifact (`.sumo-qa/mutation.json`). It provides common phrasings that map to the tool, distinguishing it from siblings that perform other QA tasks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states this tool is for 'FILE/FORMAT PLUMBING ONLY' and that the host skill runs the mutation tool, providing clear context. It does not, however, explicitly list when not to use it or name alternative tools among the siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_register_known_good_test_dataA
Add or update a known-good test data entry in the local YAML catalogue.
Detects duplicates by environment + domain + product/SKU + scenario
overlap. Writes to knowledge/test_data/<domain>/known_good.yaml.
Arg shape — pass entry as a literal dict, NOT a YAML string. Example:
sumo_qa_register_known_good_test_data(entry={
"id": "billing-overdue-invoice-001",
"environment": "staging",
"domain": "billing",
"scenario_tags": ["overdue_invoice", "dunning_eligible"],
"known_valid_for": ["dunning workflow testing"],
"constraints": ["Reset overdue flag after test."],
"owner": "billing-platform",
"last_validated_at": "2026-05-16T09:00:00Z",
"confidence": "high",
"source": "qa-curated",
"notes": "Overdue invoice usable for dunning-flow testing.",
})Common natural-language phrasings that map to this tool: "save this as known-good test data", "register this fixture so the team can reuse it", "promote this record to known-good", "update the validated timestamp on entry X", "add this record to the catalogue".
| Name | Required | Description | Default |
|---|---|---|---|
| entry | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, consistent with the mutation described. The description adds that it writes to a specific YAML file and detects duplicates, but does not detail permissions, side effects, or error behavior. This adds moderate context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is well-structured with a clear first sentence, details, an example, and common phrasings. It is front-loaded with the purpose. Slightly long but every part adds value; could be trimmed slightly but overall good.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has one complex parameter and no output schema. Description covers the parameter thoroughly and explains behavior (duplicate detection, file path). It does not mention return value or error cases, but the example implies success. Given constraints, it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has only one parameter 'entry' with no property descriptions and 0% schema coverage. The description compensates excellently by providing a full example dict with all common fields (id, environment, domain, etc.) and explicitly instructs to pass as a literal dict, not a YAML string. This adds crucial meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states 'Add or update a known-good test data entry in the local YAML catalogue', clearly indicating the action and resource. It also explains duplicate detection and file location, distinguishing it from sibling tools like sumo_qa_find_test_data and sumo_qa_validate_test_data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description includes common natural language phrasings that map to this tool, helping the agent recognize when to use it. It also describes duplicate detection logic. However, it does not explicitly state when not to use this tool or provide alternatives among siblings, so usage guidance is good but not fully comprehensive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_reviewing_before_mergeA
Use when the user asks "review my changes" / "is this safe to merge" / "what could break". Reads the diff and the changed files first, surfaces what was found + named risks, runs tests, then delivers the verdict — section by section with confirmation gates, not as one dump. Refuses to claim safe-to-merge without fresh verification evidence.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses the multi-step process (reads diff, surfaces risks, runs tests, delivers verdict section by section with confirmation gates) and a key behavioral constraint (refuses to claim safe-to-merge without verification). This gives reasonable insight into operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph of ~4 sentences, front-loading the usage trigger. It is efficient but could be broken into bullet points for readability. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters, no output schema, and no annotations, the description covers the essential workflow and constraints. It explains what the tool does, how it delivers results (section by section with gates), and its refusal behavior. While more details on test execution feedback could be added, it is complete enough for an agent to decide.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the description does not need to add parameter meaning. According to scoring guidelines, 0 parameters gives a baseline of 4. The description provides context about what the tool acts on (diff, files) but does not mislead.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with 'Use when the user asks...' which clearly identifies the trigger and action. It specifies the key steps: reading diff, surfaced findings, running tests, delivering verdict. This distinguishes it from sibling tools like sumo_qa_analyze_diff_impact that may only analyze without the full review cycle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use ('review my changes', 'is this safe to merge', 'what could break') and what the tool does (reads diff, tests, delivers verdict). It also notes what it refuses to do (claim safe-to-merge without fresh evidence). However, it does not explicitly contrast with all siblings or provide when-not-to-use scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_scan_repoA
Walk a repository and return a compact summary of its QA-relevant
shape: per-type node counts, likely_tests edge counts by confidence,
command counts by kind, warning counts by kind. Optionally writes the
full schema-validated .sumo-qa/repo-map.json artifact to disk via
write_to.
Common natural-language phrasings that map to this tool: "map this repo", "scan the repo and tell me what's here", "give me a QA-shaped inventory of this project", "what tests exercise what sources in this repo", "generate the repo-map artifact for X".
root is the repository to scan (absolute or relative to the MCP
server's working directory). generator_version defaults to the
installed sumo-qa version. write_to is optional — when set, the
full artifact is written to that JSON path, deterministic on the
same repo state except for project.generated_at.
| Name | Required | Description | Default |
|---|---|---|---|
| root | Yes | ||
| write_to | No | ||
| generator_version | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are all false, so the description carries full burden. It discloses that the tool reads the repository and optionally writes an artifact to disk via `write_to`. It notes that the artifact is deterministic except for a timestamp field. It does not mention permissions or side effects (no destructive hint, so writing is not destructive). Adequate transparency for a non-destructive scan tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the core purpose. The natural-language phrasings are helpful but add length. Each sentence earns its place, though the description could be slightly tighter. Still efficient overall.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description explains what the tool returns (compact summary with various counts) and what the optional artifact contains. It covers the key aspects. For a scan tool with 3 parameters, the description is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description fully explains parameters. It describes `root` as the repository to scan, `generator_version` defaults to installed version, and `write_to` as optional path for writing the artifact. It adds determinism context for the output. This fully compensates for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool walks a repository and returns a compact QA-relevant summary, listing specific outputs (per-type node counts, edge counts, command counts, warning counts). It distinguishes from siblings like sumo_qa_query_repo_map by noting it generates the artifact, while query_repo_map likely queries it. The purpose is specific and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides common natural-language phrasings that trigger this tool, such as 'map this repo' or 'scan the repo'. This helps agents select it. However, it does not explicitly state when not to use it or contrast with siblings beyond implicit behavior. Clear context for use is given, but exclusions are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_search_external_skillsARead-only
Search the Skills CLI registry for external agent skills.
Returns the ANSI-stripped CLI output verbatim plus a one-line hint on how to read it. No structured parsing — the host LLM interprets the raw text so format drift in the Skills CLI doesn't break the flow.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description explains return format (raw CLI output with hint) and rationale for raw output to avoid breaking on format drift. No contradiction with annotations (readOnlyHint: true, etc.).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first clearly states purpose, second adds critical behavioral context without unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers return format and rationale well, but missing edge cases like empty results or error handling. Given simplicity, nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only parameter 'query' has no description in schema or tool description. No format, example, or constraints provided, leaving the agent to infer.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb 'search' and resource 'Skills CLI registry for external agent skills' clearly state the tool's function, distinguishing it from siblings like check, install, execute, and suggest.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage for searching skills but no explicit guidance on when to use vs alternatives or any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_security_testingA
Use when explicit security testing or a grounded security gap needs deeper evidence selection.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so description must disclose behavioral traits. It mentions 'deeper evidence selection' but does not explain what the tool does concretely (e.g., returns evidence? selects from a pool?), nor does it address side effects, authorization, or state changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence front-loaded with the usage cue 'Use when.' Every word serves a purpose, no fluff. Appropriate length for a zero-parameter tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description provides essential purpose and usage context but lacks details on return value or procedural behavior. 'Deeper evidence selection' remains ambiguous, leaving gaps for an agent deciding invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so baseline is 4. Description adds no parameter info, but that is acceptable since schema coverage is 100% and there are no parameters to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states this tool is for explicit security testing or grounded security gaps requiring deeper evidence selection. It distinguishes from sibling tools that focus on general QA processes like plan creation, gap analysis, or test execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit condition: 'Use when explicit security testing or a grounded security gap needs deeper evidence selection.' Does not list when to avoid or alternatives, but the condition is specific enough to guide selection among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_strategisingA
Use for repo-wide / policy-shaped asks — "audit our test coverage", "design our QA strategy from scratch", "where should we invest QA effort first", "design our test pyramid". Walks repo inventory → per-area risks → specialty fit → prioritisation → pyramid → phased rollout → residual risks, one section at a time with confirmation gates. Walks the repo with the host's file tools first.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description outlines the step-by-step process (from repo inventory to residual risks) and mentions the use of 'host's file tools', providing full transparency about the tool's behavior without any annotations needing disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first clearly states purpose with examples, the second details the process. It is concise, front-loaded, and every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters, no output schema, and no annotations, the description provides a thorough explanation of the tool's purpose, usage, and behavior, making it fully complete for an agent to decide when to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description need not explain them. The 100% schema coverage is trivial, and the description adds value by explaining the tool's operation, consistent with the baseline score of 4 for no-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool is for 'repo-wide / policy-shaped asks' and gives concrete examples like 'audit our test coverage', which clearly defines its purpose and distinguishes it from siblings focused on specific tasks like diff impact analysis or test plan creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use the tool (for strategic, repo-wide questions) and includes example queries, but does not explicitly state when not to use it or compare with alternatives like sumo_qa_creating_test_plan or sumo_qa_deciding_approach.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_strengthening_testsA
Use after sumo-qa-deciding-approach picks strengthen-test-coverage. Mutation-testing follow-up, raise-coverage tasks, killing weak assertions. Walks survivor → tautology check → technique → strengthening test, one mutant at a time with confirmation gates. Production code STAYS UNCHANGED.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden. It discloses that production code stays unchanged and outlines the step-by-step process with confirmation gates. However, it does not detail whether tests are automatically modified or what specifically changes in test files (e.g., assertions).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with zero waste. Front-loaded with usage context, followed by workflow steps, and ends with a key behavioral constraint. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters, no output schema, and no annotations, description adequately covers purpose, usage, and workflow. It mentions what stays unchanged (production code) but could elaborate on effects on test files. Overall, it is sufficiently complete for a zero-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Tool has zero parameters, so baseline is 4. Description correctly omits any parameter details as none exist. No information is needed beyond what is already provided by schema (100% coverage of empty params).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool's purpose: strengthening tests after sumo-qa-deciding-approach picks strengthen-test-coverage. It specifies a workflow (survivor → tautology check → technique → strengthening test) and explicitly distinguishes itself from siblings by referencing a specific decision outcome.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: 'Use after sumo-qa-deciding-approach picks strengthen-test-coverage.' Provides context (mutation-testing follow-up, raise-coverage tasks) but does not list explicit exclusions or alternatives. Still, it offers clear guidance for the intended use case.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_suggesting_external_skillA
Use when sumo-qa-deciding-approach routes here (no native sumo-qa sub-skill fits a QA surface) OR when an ingestion source needs converting to markdown before it can be ingested. Finds, installs, and executes an external skill for any capability sumo-qa lacks natively, through sumo-qa MCP tools, with [y/N] confirmation before each install and fallback to the next candidate on failure. Never invoked cold — always via the deciding-approach fallback or the ingestion conversion entry.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description reveals key behaviors: confirmation before install, fallback on failure, and invocation constraints. However, it omits details about output, persistence of installation, authentication needs, or other side effects, leaving gaps in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a concise single paragraph that front-loads usage scenarios. Minor redundancy ('through sumo-qa MCP tools') exists, but overall it is efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description fails to explain what the tool returns (e.g., success, result, or error). It also assumes domain knowledge and does not reference sibling tools within the suite, leaving the agent with incomplete context for a complex tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so schema description coverage is 100%. The description does not add parameter semantics because none exist, meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's role: it finds, installs, and executes external skills when no native sumo-qa sub-skill fits or for ingestion conversion. It distinguishes itself from siblings by specifying the exact entry points (deciding-approach fallback or ingestion conversion) and the actions performed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool (as a fallback or for ingestion conversion) and notes it is never invoked cold. However, it does not explicitly exclude other scenarios or directly compare to sibling tools like sumo_qa_install_external_skill, though the dual install/execute behavior is implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_triaging_test_failuresA
Use when a test is failing or flaky and the cause is not yet known — e.g. 'this test keeps failing', 'red in CI but passes locally', 'this test is flaky', 'a test failed, what now?'. Reads the failure output, classifies the likely cause (product regression, test bug, fixture/data, environment/dependency, timing/order, external service), and names the smallest next isolation step BEFORE any fix. Diagnosis only — it does not patch; it routes to a fixing skill only once a concrete product-behaviour gap is confirmed.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description fully discloses behavior: reads failure output, classifies cause, names next isolation step, and explicitly states it does not patch. Provides complete behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise, front-loaded with usage condition, and every sentence adds value. No redundant or vague language.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters, no output schema, and no annotations, the description is complete: it explains input (user query), process, output (next step), and boundaries (no patching).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0 parameters with 100% coverage, so baseline is 4. Description adds no parameter info since none exist, but it explains the process sufficiently.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool is for diagnosing test failures or flakiness when cause is unknown. It uses specific verb 'triages' and resource 'test failures', and distinguishes from siblings by focusing on the triage phase.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use when a test is failing or flaky and the cause is not yet known', gives example user queries, and clarifies it does not patch but only diagnoses and routes to fixing skill when appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sumo_qa_validate_test_dataARead-onlyIdempotent
Validate a test data entry without provisioning or mutating downstream systems.
Returns: validation result with confidence level, freshness status, and
an explained reason. Accepts either entry_id (looked up in the
catalogue) or entry (a full record dict).
Common natural-language phrasings that map to this tool: "is this test data still valid", "validate this record", "is entry X still good", "check if X is fresh".
| Name | Required | Description | Default |
|---|---|---|---|
| entry | No | ||
| entry_id | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that no downstream mutation occurs and describes the return payload (confidence, freshness, reason), enhancing behavioral understanding beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences plus a bulleted list of example phrasings. Every sentence adds value, with the main verb and scope stated first. No redundant details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple validation tool with no output schema, the description covers purpose, parameter options, return contents, and usage context. It is fully sufficient for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description explains the two parameters: entry_id (looked up in catalogue) and entry (full record dict). This adds critical meaning that the schema alone lacks, effectively compensating for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool validates test data entries without side effects, distinguishing it from mutation tools. It provides specific verb+resource ('Validate a test data entry') and includes common natural-language queries, making the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes the tool does not provision or mutate, indicating appropriate use for read-only validation. While it lacks explicit comparison to sibling tools, the natural-language phrasings guide usage context. No false exclusions present.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
using_sumo_qaA
MUST be called first for any QA-shaped request. Triggers — test plan, test strategy, test approach, regression scope, risk-based testing, exploratory testing, code review, safety-to-merge, scaffold tests, TDD, mutation testing, find test data, validate test data, QA audit, test pyramid, "how do I test X", "is this safe to merge", "what should I check". Entry router for all sumo-qa work. Establishes the global discipline that every sub-skill inherits. Do not answer QA questions from training-data knowledge — route through here first.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but the description explains that it triggers sub-skills and establishes global discipline, indicating its role as a router without side effects. Could mention if any state is modified, but likely none.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is somewhat verbose with many examples, but it front-loads the critical instruction. Could be more succinct by grouping examples.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description adequately explains the tool's role as a router and lists many sub-tasks, making it complete enough for an agent to understand its function.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0 parameters and schema description coverage is 100%, so description does not need to add parameter meaning. Adding parameter info would be redundant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: it is the mandatory entry router for all sumo-qa work, listing many example QA requests. It distinguishes itself from siblings by being the first tool to call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'MUST be called first for any QA-shaped request' and instructs not to answer QA questions from training-data knowledge, directing to route through this tool, providing clear when and when-not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
1 tool update
v0.50.0- Added
sumo_qa_security_testing
7 tool updates
v0.48.0- Added
sumo_qa_closing_qa_gaps - Added
sumo_qa_format_qa_scorecard - Added
sumo_qa_generate_qa_report - Added
sumo_qa_measuring_coverage - Added
sumo_qa_record_coverage - Added
sumo_qa_record_mutation - Added
sumo_qa_triaging_test_failures
40 tool updates
v0.42.0- First observed
sumo_qa_analyze_diff_impact - First observed
sumo_qa_answering_testing_question - First observed
sumo_qa_capabilities - First observed
sumo_qa_capture_review_feedback - First observed
sumo_qa_check_external_skill_installed - First observed
sumo_qa_creating_test_plan - First observed
sumo_qa_deciding_approach - First observed
sumo_qa_execute_external_skill - First observed
sumo_qa_executing_qa_rollout - First observed
sumo_qa_explain_test_data_requirements - First observed
sumo_qa_export_test_cases - First observed
sumo_qa_find_test_data - First observed
sumo_qa_finding_test_data - First observed
sumo_qa_finishing_qa_work - First observed
sumo_qa_format_context_bundle - First observed
sumo_qa_format_risk_ledger - First observed
sumo_qa_implementing_with_tdd - First observed
sumo_qa_ingest_knowledge_pack - First observed
sumo_qa_install_external_skill - First observed
sumo_qa_list_skill_manifests - First observed
sumo_qa_load_approaches - First observed
sumo_qa_load_catalogue_entry - First observed
sumo_qa_load_classifications - First observed
sumo_qa_load_principles - First observed
sumo_qa_load_rules - First observed
sumo_qa_load_skill_context - First observed
sumo_qa_load_standards - First observed
sumo_qa_load_techniques - First observed
sumo_qa_planning_qa_rollout - First observed
sumo_qa_preparing_for_work - First observed
sumo_qa_query_repo_map - First observed
sumo_qa_register_known_good_test_data - First observed
sumo_qa_reviewing_before_merge - First observed
sumo_qa_scan_repo - First observed
sumo_qa_search_external_skills - First observed
sumo_qa_strategising - First observed
sumo_qa_strengthening_tests - First observed
sumo_qa_suggesting_external_skill - First observed
sumo_qa_validate_test_data - First observed
using_sumo_qa
TDQS
Many tools have distinct purposes, but there is overlap between routers and specific tools (e.g., sumo_qa_finding_test_data routes to sumo_qa_find_test_data), and several load/format/execute tools could be confused. The entry router using_sumo_qa helps, but the agent may still select the wrong tool without it.
Tool names mix gerunds (finding, creating, executing) and imperatives (analyze, capture, load) inconsistently, and some concepts appear in both forms (finding_test_data vs find_test_data). The pattern is not predictable across the set.
48 tools is significantly above the typical well-scoped range of 3-15. While the server aims to cover a broad QA domain, the large number risks overwhelming the agent and suggests insufficient consolidation.
The tool set covers most QA lifecycle activities: impact analysis, test planning, TDD, coverage, mutation, test data management, review, reporting, and external skill integration. Minor gaps like direct CI integration exist, but overall it is comprehensive.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Official MCP server for Qase — manage test cases, runs, suites, defects via AI tools.
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
MCP server for Mint — AI-powered QA that runs your app in a real browser on every PR.
Related MCP Servers
- FlicenseAqualityDmaintenanceAn MCP server that automates the full software development lifecycle through an AI-driven TDD state machine. It handles everything from task decomposition and test-driven development to integration testing and automated pull request creation.4-
- AlicenseNot gradedqualityCmaintenanceAn MCP server that adds engineering discipline to AI-assisted development, enforcing evidence-gated TDD, security review, backup strategy, and deployment generation to turn AI-generated code into production-ready software.2110MIT
- -licenseNot gradedqualityNot gradedmaintenanceA universal MCP server that acts as a code quality gate for AI assistants, providing pre-generation guidance, post-generation review, and root cause analysis to improve code quality.-
- FlicenseNot gradedqualityBmaintenanceA Python-based MCP server that enforces configurable Quality Assurance automation strategies for AI coding agents, ensuring generated tests adhere to strict architectural and testing standards.-
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/sumithr/sumo-qa'
If you have feedback or need assistance with the MCP directory API, please join our Discord server