opa-mcp-server
Server Quality Checklist
Latest release: v0.5.0
- Disambiguation3/5
The domain prefixes (rego_, opa_, conftest_) help separate broad areas, and descriptions are detailed. However, many tools are near-neighbors: rego_eval variants, rego_explain_decision vs. rego_eval_with_explain, rego_format vs. rego_format_write vs. rego_fix, and opa_status vs. opa_config returning the same underlying document. An agent can usually disambiguate with careful reading, but misselection risk is notable.
Naming Consistency3/5All names are snake_case and consistently prefixed by rego_, opa_, or conftest_, which is readable. However, the verb/noun ordering is mixed: rego_parse_ast and opa_delete_policy are verb-first, while opa_bundle_build and rego_playground_share are noun-first, and several tools are bare nouns like rego_capabilities, opa_status, and opa_config. The convention is predictable at the prefix level but not a uniform verb_noun pattern.
Tool Count2/552 tools is a very heavy MCP surface, well above the range where an agent can quickly select the right tool. Many tools are variants of the same core operation, such as the multiple rego_eval_* forms, the format/fix/write cluster, and the status/config pair. The broad OPA/Conftest scope partially justifies the size, but the count still feels excessive and burdensome.
Completeness4/5The toolset is remarkably complete for the OPA/Rego ecosystem: authoring, checking, linting, testing, benchmarking, schema inference, policy/data CRUD against a server, bundle build/sign/verify, and conftest integration are all present. Minor gaps exist, such as no explicit remote bundle upload/activation workflow, but most core workflows have no dead ends.
Average 4.3/5 across 52 of 52 tools scored. Lowest: 3.4/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 64 commits in the last 12 weeks
- Last stable release on
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
This server has been verified by its author.
How to sync the server with GitHub?
Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.
To manually sync the server, click the "Sync Server" button in the MCP server admin interface.
How is the quality score calculated?
The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).
Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.
Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).
Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.
Tool Scores
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds the output shape 'standard {result: [...]} shape', which is useful but goes no further. No additional behavioral traits like auth or rate limits are mentioned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, followed by essential output shape and positioning. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given schema coverage and annotations, the description is mostly complete. It mentions the output shape and positions the tool as central. However, it could briefly note that many parameters like 'source' and 'paths' are mutually exclusive, though those details are in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so per guidelines baseline is 3. The description does not add any meaning beyond what the schema already provides for parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as evaluating a Rego query with specific resources (policy, input document) and output shape. It uses the verb 'Evaluate' and the resource 'Rego query against a policy and an input document'. However, it does not explicitly differentiate from siblings like rego_eval_with_coverage, though the term 'bread-and-butter' implies primacy.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., rego_eval_with_coverage for coverage tracking). The phrase 'bread-and-butter' hints at default usage but lacks explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, making the delete behavior clear. The description adds minimal extra context ('from the running OPA server') but does not disclose potential side effects, authentication needs, or constraints beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence with no wasted words. Every word is necessary and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple one-parameter schema and no output schema, the description is mostly complete. However, it could mention error handling (e.g., policy not found) or that deletion is permanent, which would raise completeness to 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with 'Policy ID to delete.' in the parameter description. The tool description adds no further meaning, meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete', the resource 'a policy', and specificity 'by ID from the running OPA server'. This distinguishes it from sibling tools like opa_get_policy or opa_delete_data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., opa_put_policy to update) or prerequisites like ensuring the policy exists. The description only states the action without contextual usage advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructive and idempotent behavior. The description adds that the policy is uploaded as raw text/plain and parsed server-side, and that it replaces any existing policy with that ID, providing useful behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no redundant information. It is concise and front-loaded with the key action. Could potentially be structured as a brief paragraph but still efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description covers core behavior (replace, raw text). However, it does not mention return values or error conditions, which would be helpful for completeness given the tool's destructive nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions already define 'id' and 'source' adequately. The description adds that the source is raw text/plain, which is helpful but not extensive. With 100% schema coverage, the description does not significantly enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Upload' and resource 'Rego policy' with a given ID. It distinguishes from sibling tools like opa_get_policy and opa_delete_policy by specifying the upload/replace action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as opa_put_data or opa_bundle_build. There is no mention of prerequisites or context where this tool is preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that the tool uses 'opa parse' and returns a tree of specific node types, but does not disclose additional behavioral traits like error handling, output format details, or performance characteristics. The added value beyond annotations is modest.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: the first clearly states the action and implementation, the second provides usage context. No extraneous information; every sentence is purposeful. Highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple input (one string parameter), annotations covering safety/idempotency, and no output schema, the description is fairly complete. It explains what the tool does, how it works (opa parse), and the general output structure. Minor gap: no detail on error cases or output format beyond node types, but sufficient for a parse tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description for the 'source' parameter. The tool description adds little beyond the schema, stating it parses Rego to AST and mentioning the output structure, but does not enrich parameter semantics further. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool parses Rego source to a JSON AST using 'opa parse' and lists the AST node types. However, it does not distinguish from sibling tools like rego_check or rego_eval, though the purpose is unique among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a use case ('when you need to introspect policy structure programmatically') but does not provide explicit guidance on when not to use this tool or which alternatives exist. It implies usage context but lacks exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, readOnlyHint=false, and idempotentHint=true, covering the safety profile. The description adds that the output is a .tar.gz archive and that signing is optional, but it does not disclose additional behavioral details such as overwriting existing outputs or the fact that signing implies bundle mode (though the schema covers the latter). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no wasted words: purpose first, then output format, then supported features. The description is front-loaded and every sentence contributes useful information. The mention of `opa build` clarifies the underlying command without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (16 parameters, 2 required, no output schema), the description is a useful but high-level summary. It does not guide the agent on when to choose this tool over opa_bundle_sign/verify, nor does it highlight important constraints like entrypoints being required for wasm/optimize; the schema covers those details, but the description alone is not fully complete for a tool of this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 16 parameters. The description adds a high-level summary of key capabilities (optimization, revision strings, WASM target) that maps to parameters, but it does not provide meaningful new meaning beyond what the schema already states. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb ('Build') and resource ('deployable bundle from policy / data paths') and adds concrete output details (.tar.gz, optional signing, WASM target). This distinguishes it from related siblings like opa_bundle_sign and opa_bundle_verify, which handle signing/verification rather than building.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for building deployable bundles from policy/data paths, but it does not explicitly contrast this with related tools such as opa_bundle_sign, opa_bundle_verify, or rego_eval. There is no explicit when-to-use or when-not-to-use guidance, so the agent must infer the boundary from the description and sibling names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a destructive, non-idempotent write operation. The description adds useful behavioral context by explaining the operation format and that omitting path and segments patches the root to add a new top-level document. It does not go into further side effects, but the annotation covers the main destructive risk.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences with no wasted words. It front-loads the core action, then gives the operation shape, then handles the important root-patch special case. Each sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a three-parameter tool with annotations covering the destructive nature and a schema covering all parameters, the description provides the remaining key context: how operations are structured and how to target the root. There is no output schema, but return-value details are not critical for invoking this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by clarifying the JSON Patch operation shape and the special root-patching behavior when both path and segments are omitted. This is meaningful parameter-level guidance an agent would not get from the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the action: applying an RFC 6902 JSON Patch to the OPA data document. It names a specific verb and resource and is distinct from sibling tools like opa_put_data and opa_delete_data, though it does not explicitly differentiate itself from them in the description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when this tool is useful by defining it as the JSON Patch mechanism for data, and it gives a concrete usage tip about omitting path/segments to patch the root. However, it does not explicitly state when to use this tool instead of alternatives such as opa_put_data or opa_delete_data.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true and idempotentHint=true. The description adds value by specifying the exact HTTP endpoint and explaining the concept of partial evaluation (substituting knowns). This provides behavioral context beyond annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently deliver the action and result. No extraneous text. The first sentence is front-loaded with the verb 'compile' and endpoint, making it immediately actionable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's core behavior and return value (residual query) without an output schema. It assumes familiarity with OPA concepts but is sufficient for an agent. Could add more on use cases or prerequisites, but is adequate for the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for all three parameters. The tool description reinforces the purpose of partial evaluation but does not add new parameter-specific details beyond the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sends a query to the OPA server's /v1/compile endpoint for partial evaluation and returns the residual query. This distinguishes it from evaluation tools like rego_eval or opa_query_decision, showing a specific verb and resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool vs alternatives such as rego_eval or opa_query_decision. It only mentions partial evaluation but gives no guidance on scenarios or exclusions, leaving the agent to infer usage context from the tool name and siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the safe, read-only nature is clear. The description adds behavioral context by detailing the return data (manifest, namespaces, annotations, signature metadata) and the accepted input types, which goes beyond the annotation flags.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences that succinctly state the action and the outputs. Every word adds value; no redundancy or verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description adequately covers what the tool does and what it returns. Minor missing details like error behavior or format specifics are not critical given the tool's straightforward nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers the single 'target' parameter completely with a clear description of possible values. The tool description echoes this but adds no new semantic information beyond the schema. With 100% schema coverage, the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Inspect') and resource ('OPA bundle, policy directory, or single Rego file') and lists the specific outputs (manifest, namespaces, rule annotations, signature metadata). It distinguishes from siblings like rego_check or rego_eval by focusing on structural inspection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit guidance on when to use this tool versus alternatives. While the purpose implies it's for inspection of bundle structure, there is no mention of when not to use it or how it differs from similar sibling tools like rego_deps or rego_describe_policy.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds behavioral context: it skips existing test rules, infers input shape from AST, and with tableStyle generates array-based stubs. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear first sentence stating purpose, followed by process details. It is moderately concise; every sentence contributes value (AST parsing, skipping rules, input inference, tableStyle behavior). Could potentially trim some elaboration but not wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but description covers the output: stub tests, inferredInputShape response field, and behavior for tableStyle. For a 2-parameter tool with no enums, this is sufficient. Explains what the agent can expect from the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds meaning to tableStyle (explains table-driven stubs with 'every tc in cases') and source (mentions AST parsing), but the schema already adequately describes each parameter. Description provides useful context but not essential beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it generates a '*_test.rego' skeleton from a policy, with specific verb 'Generate' and resource 'Rego test skeleton'. It details the process: parsing AST, finding non-test rules, emitting stubs. This distinguishes it from sibling tools like rego_test (testing) and rego_format (formatting).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for generating test skeletons from policy source, but does not explicitly state when to use vs alternatives, when not to use, or prerequisites. No guidance on choosing this over other code generation or testing tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the read-only and idempotent annotations by explaining what the tool returns: per-file pass/fail results, NO_TESTS_FOUND when no test rules exist, and CONFTEST_NOT_FOUND when the binary is unavailable. This gives an agent concrete expectations about success, failure, and external dependency behavior without contradicting the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences and each one earns its place: first the core action, then the conceptual equivalence, then return/error behavior. It front-loades the purpose and avoids repetition or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With three optional parameters, 100% schema coverage, and no output schema, the description supplies the key operational details an agent needs: what results to expect, the no-tests case, and the required external dependency. The only meaningful gap is that it does not explicitly route away from similar sibling tools, which keeps it just short of fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all three parameters with detailed descriptions, including allowed roots, defaults, and namespace behavior, so the schema coverage is 100%. The description adds only general context about the policy directory and test files, which aligns with the `policy` parameter but does not materially increase parameter-level understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Run') and names the exact resource (`test_*` rules in `*_test.rego` files) with the goal of verifying that conftest policies are correct. It also contrasts with `opa test` and conftest's loading machinery, but it does not explicitly distinguish itself from sibling tools like `conftest_test`, `rego_test`, or `rego_verify`.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description makes the domain clear — a conftest policy directory — and notes an equivalent behavior (`opa test`), which implies when the tool might be appropriate. It also states a prerequisite (`conftest` on PATH or `CONFTEST_BINARY` set), but gives no explicit when-to-use or when-not-to-use guidance against similarly named sibling tools, so the usage guidance is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the description's mention of 'static analysis' adds context but does not disclose additional behavioral traits like performance or side effects beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the purpose and key details. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description explains what the tool returns (base and virtual document references, transitively). It covers purpose, parameters, and output sufficiently for a static analysis tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. The description adds meaning by explaining the ref format (e.g., 'data.example.allow') and the paths constraint (must be inside allowed root), which adds value beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs static dependency analysis for a Rego reference, specifying the target ref format and what it returns (base and virtual document references). This distinguishes it from sibling tools like rego_check or rego_eval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for dependency analysis but does not explicitly state when to use this tool versus alternatives like rego_eval or rego_explain_decision. No when-not or alternative guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. Description adds that it returns findings grouped by severity with remediation guidance and requires regal, which provides useful context beyond annotations but does not contradict them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences: purpose, usage guidance, prerequisite. Front-loaded with key information. No unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description explains output format (findings by severity with remediation). Missing details on error handling or invalid paths, but overall sufficient given annotations and schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter. Description adds minimal extra meaning (e.g., 'Pass the root of your policy fleet to scan everything at once' for paths), but does not significantly supplement the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it runs 'regal lint restricted to the security and bugs categories' across policy directories, which is a specific verb-resource combination. It distinguishes from sibling 'rego_lint' by focusing on security and bugs categories and mentioning fleet-wide sweep.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage guidance: 'Use this for a periodic fleet-wide security sweep rather than per-file style review.' Also states prerequisite 'Requires regal.' Does not explicitly name alternatives but context implies different tool for per-file review.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive. The description adds that it returns one suggestion per diagnostic and confidence levels (high/medium/low). This provides useful behavioral context beyond annotations, though it does not detail the output structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and no output schema, the description explains input source, output quantity, and confidence levels. It does not describe the suggestion structure, but for a low-complexity tool, this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all fields. The description only adds that diagnostics should come from rego_check or rego_lint, which is helpful but minimal. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it maps compile errors and lint findings to fix suggestions, and specifies the source diagnostics. However, it does not explicitly differentiate from sibling tool rego_fix, which may apply fixes, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs to pass diagnostics from rego_check or rego_lint, providing clear usage context. Does not mention when not to use or alternatives, but the context is sufficient for an AI agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds context by explaining the default for 'unknowns' and the meaning of the residual policy, which is not covered by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the action of partial evaluation, and no unnecessary words. The structure is clear and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, no output schema), the description provides a high-level understanding but does not mention what the output format is (e.g., JSON or Rego string). It is adequate but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description does not need to add much. It does highlight the default for 'unknowns', but other parameters are not elaborated beyond their schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs partial evaluation on a Rego query, substituting known values and returning a residual policy. It distinguishes from siblings like rego_eval by specifying the partial evaluation nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states use cases: 'offline policy slicing or pre-computing decision sets.' It does not mention when not to use it, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint, idempotentHint, and destructiveHint, which the description aligns with by stating it 'parse[s] a Rego policy and return[s] a structured summary.' The description adds useful behavioral detail beyond annotations, such as listing specific output fields (clauseCount, isDefault, etc.).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of three sentences that flow logically: what the tool does, details about what it returns, and a use case. Each sentence adds value without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema, the description adequately describes the return value (package, imports, rules, and rule details) and covers the tool's functionality for a single-input, simple tool. It is complete enough for an agent to understand the tool's purpose and output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage for the single parameter 'source' with description 'Rego source to describe.' The tool description does not add significant new meaning, as the schema already explains the parameter adequately. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool parses a Rego policy and returns a structured summary including package, imports, rules, and detailed rule attributes. It distinguishes itself from sibling tools like rego_eval, rego_check, and rego_inspect by focusing purely on structural description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly suggests using this tool as 'the first step in any 'what does this policy do' workflow,' providing clear context for when to use it. It does not explicitly mention when not to use it or contrast with alternatives, but the guidance is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnly=true, destructive=false, idempotent=true. The description adds that it returns per-line coverage data, which is useful behavioral detail. No contradictions. It could mention that evaluation is side-effect-free, but annotations cover that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states the action and output, second gives usage context. No redundant words, front-loaded with key information. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters and no output schema, the description is brief. It does not explain the output format (e.g., structure of coverage data), error handling, or performance implications. While the schema covers parameters, additional context about return values would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description does not elaborate on parameters beyond what the schema already provides (e.g., mutual exclusivity of source/paths, input/inputPath). No added value for parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool evaluates with --coverage and returns per-line coverage data. This distinguishes it from siblings like rego_eval, rego_eval_with_explain, rego_eval_with_profile, and rego_coverage_gaps by specifying the exact feature (coverage).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says 'Useful for verifying that tests actually exercise the rules they're meant to,' which gives clear context for when to use it (for test coverage). However, it lacks explicit guidance on when not to use it or how it compares to alternatives like rego_test or rego_coverage_gaps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructive and idempotent hints; the description adds non-obvious runtime behavior: the body is JSON, path separator parsing switches between dots and slashes, and dot-containing keys can be addressed via slash-separated paths. This is meaningful context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: purpose, body format, separator rule, and segments fallback. The most important verb-first statement is front-loaded and the paragraph is dense without padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tricky path-encoding behavior is fully explained, and annotations cover the destructive/idempotent safety profile. There is no output schema and no response description, but for a write operation the essential calling requirements are covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description elevates it by explaining how `path` is parsed, why `hosts/example.com` works, and when `segments` is the right parameter. It adds practical meaning not fully present in the schema's field descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Write or replace a value') on a specific resource (OPA data path), which clearly distinguishes it from siblings like opa_patch_data and opa_delete_data. The 'replace' wording communicates full overwrite rather than merge or delete.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives useful in-tool guidance for choosing path versus segments, but it never addresses when to use opa_put_data instead of opa_patch_data or opa_delete_data. Tool-vs-alternative selection is therefore left mostly implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as read-only, idempotent, and non-destructive. The description adds valuable behavioral context by specifying the POST method, the data-path endpoint, and that the response is 'whatever the rule produces'. It does not detail error or undefined-rule behavior, but the annotations lower the burden.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no wasted words. It front-loads the action and endpoint, then gives a concrete example in the second sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the phrase 'returns whatever the rule produces' gives useful response expectations, and annotations cover the safety profile. The need to provide a path or segments is implied but not explicit, which is a minor gap given the schema hints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all five parameters with clear descriptions. The description reinforces the meaning of `path` and `input` through the data.X.allow example, but it does not add significant meaning beyond the schema, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Evaluate'), a specific resource ('the running OPA server'), and the mechanism ('POSTs to the data path'). The quoted example, 'given this input, what does data.X.allow say?', clearly differentiates this from local evaluation siblings like rego_eval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear context for when to use this tool: querying a running OPA server with an input document. It implicitly distinguishes from local rego evaluation tools, but it does not explicitly name alternatives or state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish that the tool is read-only, non-destructive, and idempotent. The description adds useful behavioral information beyond that: it invokes `opa bench` and returns timing statistics rather than ordinary evaluation results. This helps the agent predict observable behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences with no filler: the first states the action, the second enumerates the return data, and the third gives the use case. Every sentence earns its place and the content is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description's explicit mention of iterations, ns/op, and allocation counts is valuable. The schema fully documents all five parameters. The description could add explicit comparisons to profiling-related siblings, but it is otherwise complete enough for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and parameter descriptions are already detailed, including count semantics, paths, and inputPath. The tool description adds only the high-level 'policy + input' framing, so it does not need to compensate for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb ('Benchmark'), a specific resource ('a Rego query against a policy + input'), and the implementation (`opa bench`). The output statistics are also listed, making it easy to distinguish this from sibling tools like rego_eval or rego_test.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The final sentence explicitly tells the agent when to use this tool: 'Use this to spot slow rules.' It does not explicitly state when to prefer alternatives such as rego_eval_with_profile, but the benchmark framing provides clear selection context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, etc. The description adds that it uses --explain=full and returns a structured trace, which provides behavioral context beyond the annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences, front-loading the main behavior and use case. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters and no output schema, the description explains the core functionality and use case. It could be more complete by describing the trace structure, but the reference to rego_explain_decision partially compensates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so each parameter already has a description. The tool description does not add additional semantics beyond the schema, meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it evaluates Rego with '--explain=full' and returns a structured trace, distinguishing it from rego_eval which likely returns only results. It specifies the use case for understanding rule firing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this when an agent needs to see why a rule fired (or didn't)', providing clear context. It mentions the trace is the basis for rego_explain_decision, implying an alternative for further analysis, though it could more directly name rego_eval for cases without trace.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only and idempotent behavior. The description adds valuable context about the profiling output (per-rule timing and counts) and the purpose. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the key information. Every word is meaningful with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters and no output schema, the description adequately explains the tool's purpose but lacks detail on the output structure (e.g., format of timing and counts). Annotations compensate for safety, so completeness is acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents each parameter. The description does not add extra detail beyond the overall purpose, but the baseline is 3 given full coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Evaluate with --profile and return per-rule timing and evaluation counts' and 'Use this to find hot rules in slow policies', providing a specific verb-resource combination and clear use case that distinguishes it from siblings like rego_eval or rego_eval_with_coverage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly indicates when to use: for profiling to find performance bottlenecks. However, it does not explicitly mention when not to use or list alternatives, though the sibling tools provide context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, non-destructive, idempotent behavior. The description adds that evaluation is with full tracing and the output is a structured trace plus summary, and that the agent narrates without re-implementing parsing. This goes beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states what the tool does (verb+resource+output), second gives usage and agent behavior. No wasted words, front-loaded with key info.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters and 100% schema coverage, the description explains the output and usage context. However, it could briefly mention mutual exclusivity of source/paths or partial vs full evaluation, but schema already does that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description doesn't add parameter-specific details beyond what the schema already provides. It mentions 'full tracing' but that's about the tool's mode, not parameter meanings.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it evaluates a Rego query with full tracing and returns a structured trace plus per-rule summary, specifically for answering 'why was this denied?'. This verb+resource combination is distinct from siblings like rego_eval (no tracing) or rego_explain_undefined (focused on undefined).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use this when you need to answer "why was this denied?"', providing clear context. While it doesn't list when not to use, the surrounding sibling tools imply alternatives (e.g., rego_eval for normal evaluation).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint/idempotentHint/destructiveHint, and the description adds substantial non-obvious behavior: `errored` tests are neither pass nor fail and a suite with any is not passing; enabling `coverage`/`threshold` switches to coverage-report mode which drops per-test counts; repeated runs stop at the first failing repetition; and parameterized `test_x[case]` rules are reported as a single test unless disambiguated via `parameterizedGroups`. These behaviors are invisible in the schema and are exactly what an agent needs to interpret results correctly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded correctly with purpose and result shape in the first two sentences, and every subsequent clause carries real operational meaning. The length is mostly earned given 12 parameters, hidden mode interactions, and no output schema; however, several clauses duplicate schema text (timeout default, coverage-mode switch, threshold return value), and the single unbroken prose block scans harder than bulleted parameter guidance would.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of explaining return values and does so well: errored counts, `parameterizedGroups`, `caseCounts`, `coverage`/`coveragePct`, `repetitions`, and `trace` are all named. Remaining gaps are minor: no guidance on when `v1Compatible` matters and no explicit note that multi-root suites belong to `rego_test_multiroot`.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds genuine value beyond the schema: the table-driven-test debugging rationale for `varValues`, the error-hint-includes-pattern behavior for `runPattern`, and the `parameterizedGroups`/`caseCounts` interpretation for parameterized tests. Some overlap exists — `threshold`'s COVERAGE_BELOW_THRESHOLD return and the coverage-mode switch are restated from the schema — but the net addition justifies above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource statement — 'Run Rego unit tests with `opa test`' — and immediately specifies the result shape (aggregate pass/fail/skip/error counts plus per-test records). It does not, however, distinguish itself from the closely overlapping sibling `rego_test_multiroot`, so an agent selecting between the two must infer the difference from the name alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Rich conditional guidance at the parameter level: 'Use `runPattern` to filter by name regex', 'Use `bundle: true` when testing bundle-structured policy directories', 'Use `varValues: true` with `verbose: true`' for debugging table-driven tests, and 'Use `threshold` to gate on minimum coverage'. What's absent is tool-selection guidance — when to choose this over `rego_test_multiroot`, `rego_eval`, or `conftest_test`.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as non-read-only and destructive; the description adds meaningful context beyond that: it never handles credentials, registry auth must be pre-configured externally, the policy path must be inside allowed roots, and `conftest` must be on PATH or `CONFTEST_BINARY` set. It could add overwrite/tag-replacement semantics, but the description meaningfully enriches the annotation profile.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each load-bearing: the main action, the credential-handling caveat, and the path/binary prerequisites. Information is front-loaded and there is no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive push operation with no output schema, the description covers the command, target registry, credential model, path restrictions, and binary prerequisite. An agent has enough information to decide whether it can invoke the tool and what side effects to expect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and both parameter descriptions already document path constraints, fallback behavior, allowed roots, and registry credential requirements. The tool description mostly restates these, adding only environment-level context like `CONFTEST_BINARY` rather than new parameter-level meaning, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource pair: package the local Rego policy directory as an OCI artifact and push it to a registry using `conftest push`. This clearly distinguishes it from siblings like `conftest_pull`, `conftest_test`, and the various rego_ inspection tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It clearly states when to use the tool (when publishing a policy directory as an OCI artifact) and lays out prerequisites: pre-configured registry credentials, the policy path inside OPA_MCP_ALLOWED_PATHS, and `conftest` availability. It does not explicitly name alternatives or exclusions, but the push-scope and inverse sibling `conftest_pull` make the intended use obvious.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds valuable behavioral detail about path interpretation: dotted notation versus slash-only separator, and how a key containing a dot can still be addressed. This goes beyond what annotations and schema alone provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three tightly written sentences with no filler. The core action is front-loaded, and the necessary path-format nuances are packed efficiently into the remaining sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a low-complexity read tool with strong annotations, the description is nearly complete. It covers the trickiest part: path formatting and segments selection. A minor gap is that it does not state what happens when neither `path` nor `segments` is provided, even though the schema allows zero required parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3, but the description substantially enriches parameter understanding. It clarifies the dotted-path rule, the slash-only fallback, the `example.com` addressing case, and the exact condition for using `segments` instead of `path`. This resolves real ambiguity in how to invoke the tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Read a path from OPA's data hierarchy,' which names a specific verb, resource, and scope. This clearly differentiates it from siblings like opa_get_policy (policies) and opa_query_decision (decision evaluation) by targeting the data hierarchy specifically.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives strong internal guidance on when to use `path` vs `segments`, but it never names alternative tools or states when this tool should be preferred over opa_get_policy or opa_query_decision. Tool-selection context is implied by 'data hierarchy' but not made explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safe-read profile (readOnlyHint, idempotentHint, non-destructive), so the description's job was to add behavioral depth beyond that, and it delivers: the exact endpoint, 200-vs-otherwise result semantics, the key gotcha that an unactivated bundle yields { healthy: false } rather than a tool error, and the OPA_UNREACHABLE failure mode. These are precisely the interpretation cues an agent needs and cannot derive from annotations or the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, roughly 70 words, with no filler. The endpoint is front-loaded, followed by result interpretation, the unreachable edge case, and finally the flags — a logical order where every sentence earns its place. Nothing is redundant with the schema or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-required-param, read-only health check with no output schema, the description is fully sufficient: it names the endpoint, defines both success and failure result shapes, covers the edge cases (unactivated bundle, unreachable server), and documents both optional flags. There is nothing an agent needs in order to call this tool correctly that is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% — both bundles and plugins already carry adequate descriptions. The description's phrase 'query flags to require those subsystems to also be healthy' adds a small amount of meaning by tying the booleans to the subsystem-health concept, which aligns with and slightly reinforces the schema. Since the schema does the heavy lifting, the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Names the specific resource (`/health` endpoint) with a clear verb ('Hit'), and then defines the expected response semantics. This makes the tool immediately distinguishable from the many siblings in the namespace, especially opa_status and opa_config, without needing to open their schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description conveys useful context about when to use the tool — checking whether the OPA server (and optionally its subsystems) is healthy — and clarifies that an unactivated bundle appears as a health result rather than a tool error, which affects result interpretation. However, it never explicitly names alternatives or gives when-to-use / when-not-to-use conditions, so routing among overlapping siblings like opa_status and opa_config is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnly, destructive false, idempotent. Description adds behavioral context: runs opa test, sorts output, threshold usage. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
3-4 sentences, front-loaded with key action, no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, sorting, threshold. Omitted runPattern parameter. No output schema, so description could be more specific about output structure. Adequate for most use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 3 params with descriptions. Description repeats threshold's purpose but doesn't add new meaning beyond schema. runPattern not mentioned in description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb+resource+scope: runs opa test --coverage, returns per-file breakdown of uncovered line ranges, sorts by coverage ascending. Distinguishes from sibling rego_test by focusing on gaps, not test pass/fail.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
States purpose (coverage gap analysis) and mentions threshold filtering but does not explicitly compare to siblings like rego_test or rego_eval_with_coverage. Agent can infer use case but no direct when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the annotations, revealing the internal strategy (plain eval + full-trace eval + per-condition AST analysis), the two handled failure modes (runtime failures and indexer elimination), the behavior for default rules (`queryResult` reports `default`), and the return shape (structured breakdown plus human-readable summary). Annotations already mark it read-only and idempotent, and nothing in the description contradicts them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, then explains the method, the default-rule edge case, and finally the return value. Every sentence contributes useful context, though the technical method details and the restatement of the 'why is allow false' question add slight extra length. Overall it remains well-structured and readable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the critical information an agent needs: query format, diagnostic behavior, special handling of default rules, and output contents, which matters because no output schema exists. It does not explicitly discuss the input/inputPath/paths/source parameters, but the schema fully describes those with 100% coverage, so this is not a significant gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so each parameter is already documented with meaningful detail. The tool description adds minimal parameter-specific information beyond the fully-qualified query concept, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Diagnose why a fully-qualified Rego query ... produces no value, or falls back to its default.' It clearly distinguishes this diagnostic tool from eval/explain siblings by describing the combined methodology (plain eval, full-trace eval, per-condition AST analysis) and explicitly addressing the default-rule case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use the tool: when a fully-qualified Rego query is undefined or falls back to a default, and explicitly frames the question 'why is allow false' as the target. It does not explicitly name alternatives or state when not to use it, but the diagnostic intent is evident and the query path requirement is reinforced by the schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, etc. The description adds value by specifying what the tool returns (name, version, runtime details) and that it checks binary reachability. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states purpose, second provides usage guidance. No wasted words, front-loaded with key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers return values (name, version, runtime details, binary status) adequately. No output schema, but the description provides sufficient context for a simple info tool. Minor gap: 'runtime details' is vague, but overall complete enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and schema coverage is 100%. The description does not need to add parameter info. Baseline for 0 parameters is 4, and the description adds no unnecessary detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns name, version, and runtime details of the opa-mcp server. The verb 'Return' and resource 'opa-mcp server instance' are specific. Among siblings which are mostly OPA/Conftest/Rego manipulation tools, this is the only info tool about the server itself, so differentiation is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states two use cases: confirming the version of opa-mcp and verifying reachability of OPA, Regal, and Conftest binaries. While it doesn't mention when not to use it, the context is clear and no alternatives are needed as the tool is unique among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile comprehensively (readOnlyHint=true, idempotentHint=true, openWorldHint=true, destructiveHint=false), so the bar for the description is lower. The description adds genuine behavioral value beyond annotations by disclosing the response-cap behavior: enabling either include flag can cause list responses to exceed the cap. This is exactly the kind of operational trait an agent needs to anticipate failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero filler. The first sentence front-loads the action and return value; the second handles the optional parameters and the reason for the defaults. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with 0 required parameters, rich annotations, and no output schema, the description is nearly complete: it states the return value at a useful level ('policy IDs and a count') and explains both flags. It could marginally improve by describing the response envelope or ordering, but nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with unusually rich per-parameter descriptions (size ratios, response-cap warnings, alternative-tool routing), which sets the baseline at 3. The description adds meaning on top by distinguishing the two flags at a semantic level — 'Rego text' vs 'parsed AST' — and stating the shared default-off behavior and its rationale, which is not fully redundant with the schema text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource — 'List policies registered on the running OPA server' — and goes beyond that to specify the return value ('policy IDs and a count'). The phrase 'running OPA server' clearly differentiates this from the many rego_* sibling tools that operate on static policy files, and from opa_get_policy/opa_put_policy/opa_delete_policy which target individual policies.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: use this to enumerate registered policies, and the optional include flags are discouraged by default because they 'push a list of any real size past the response cap.' This effectively tells an agent when NOT to set the flags. It does not explicitly name opa_get_policy as the alternative for fetching a single policy's source in the description body — that routing lives in the schema — so it stops just short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses important behavioral traits beyond the annotations: it explains the response size limit rationale for names_only default, what data is returned in each mode, and the trade-off of setting names_only to false. This adds substantial context the annotations alone do not provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (3 sentences) and front-loaded with the core purpose in the first sentence. Each subsequent sentence earns its place by describing parameter modes and default behavior. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given three optional parameters and no output schema, the description provides enough context: it explains the return content, the version-specific behavior, and the names_only trade-off. It does not detail the output format, but this is not critical given the clarity of the content description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all three parameters with detailed descriptions (mutual exclusivity, default behavior, response size rationale). The description repeats much of this information without adding new meaning, so it neither enhances nor detracts from the schema's coverage. Baseline 3 is appropriate since schema coverage is 100%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Return OPA capabilities' followed by a clear breakdown of contents (builtins, future keywords, features, WASM ABI versions). This clearly distinguishes it from sibling Rego/OPA tools, which focus on formatting, parsing, testing, or policy operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit conditional usage for each parameter combination: 'With current: true...', 'With version...', 'With neither...'. It also explains the names_only default and how to override it. While it does not explicitly name alternatives or exclusions, the parameter-driven guidance is clear and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint=true and idempotentHint=true. Description adds details: writes to disk, dryRun preview, abort on parse failure. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Concise, front-loaded with main action, then key features (dryRun, return value, sibling differentiation, flags, error behavior). Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 5 params and no output schema, description covers return format, error behavior, version flags, and safety. Could mention idempotency or permissions, but redundant with annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%. Description adds meaning: paths must be within allowed root, dryRun for preview, regoV1 adds import rego.v1, v0Compatible/v1Compatible for version-specific formatting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool runs `opa fmt --write` to format Rego files in place. Distinguishes from sibling `rego_format` by noting this writes to disk vs returning a string. Lists version flags.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends using `dryRun: true` for preview and distinguishes from `rego_format`. Mentions abort on parse failure. Could explicitly state when not to use, but differentiation is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, but the description adds context: uses AST-level analysis via opa parse, no OPA server needed, and accepts inline source, files, or directories. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence adds value: purpose, method, use cases, input formats. Well-structured with no fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Though no output schema, the tool returns a JSON Schema object which is self-describing. Description covers input modes thoroughly. Slight lack of output format details is acceptable given the tool's nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds value by explaining mutual exclusivity of source and paths, allowed root constraint for paths, and recursive directory walking, beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it statically analyzes Rego policies to return a JSON Schema of input.* fields. It uses specific verbs ('analyse', 'return') and resource ('input.* fields'), differentiating it from siblings like rego_eval or opa_query_decision.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions it's a 'correct starting point' for integration tests, schema validation, or documenting policy API, implying when to use. It does not explicitly state when not to use or name alternatives, but the context is reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description discloses that the target directory is emptied before the pull, warns that pulled policies are arbitrary executable Rego with exfiltration/network risks, and notes the external binary dependency and path restrictions. This is substantial value added on top of destructiveHint and readOnlyHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the main action and each subsequent sentence covers a distinct aspect: use case, prerequisite, path constraint, and security warning. No filler or redundancy; the security warning earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a side-effectful tool with no output schema, the description covers purpose, prerequisites, destructive side effects, security implications, and path constraints. An agent has everything needed to decide whether to call it and to call it safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema already documents the url schemes and the policy directory fallback and emptying behavior. The tool description doesn't add new parameter-level meaning beyond restating that the policy directory must be inside allowed paths, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific action: 'Download Rego policies from an OCI registry or Git repository into a local directory using conftest pull.' It identifies the resource, destination, and direction, and it distinguishes itself from the sibling conftest_push by direction and from conftest_test by sequencing ('hydrate ... before running conftest_test').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit usage context: 'Use this to hydrate a local policy/ directory before running conftest_test' and states prerequisites (conftest on PATH or CONFTEST_BINARY, OPA_MCP_ALLOWED_PATHS). It does not explicitly name an alternative to use instead, but the download-vs-push contrast and the sequencing with conftest_test provide clear context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover readOnly/openWorld/idempotent/destructive, and the description adds genuinely new behavioral facts: the external conftest binary requirement with the CONFTEST_NOT_FOUND failure mode, per-file/per-namespace result granularity, and the notable trait that policies can invoke OPA built-ins such as http.send, implying network side effects — which aligns with and extends the openWorldHint. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose is front-loaded in the first sentence, and every subsequent sentence covers a distinct topic: result format, dependency and failure mode, input options, defaults, and policy capabilities. At roughly 130 words it is dense but has no filler and nothing that merely repeats the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex 11-parameter tool with no output schema, the description compensates well: it states the return shape (per-file, per-namespace pass/fail/warn), the external dependency, defaults, and the mutual-exclusion pairs echoed in the schema. Nothing an agent needs to call it correctly and interpret results is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 11 parameters are already documented with types, mutual-exclusion constraints, and defaults. The description reinforces the two-axis mental model (config via files/inlineConfig, policy via policy/inlinePolicy) and the default policy directory, but adds little per-parameter meaning beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a precise verb+resource: 'Evaluate configuration files ... against Rego policies using `conftest test`', enumerating the accepted formats (Kubernetes manifests, Terraform plans, Dockerfiles, Helm charts, etc.). This clearly differentiates the tool from the rego_* siblings, which operate on Rego source, and from conftest_verify/conftest_pull/conftest_push, which address other conftest workflows.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear operational context: two input axes (files vs inlineConfig, policy vs inlinePolicy), the mutually exclusive pairs, and the default `./policy` directory behavior when both policy options are omitted. However, it never names an alternative tool or states when not to use this tool, so an agent must infer differentiation from sibling names rather than explicit guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses non-obvious behavior well beyond the annotations: OPA drops the `credentials` block, returns `services.*.headers` verbatim, and redacts header values while keeping header names. This is exactly the kind of behavioral context that helps an agent anticipate the returned data and security implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose and followed by a high-value behavioral caveat. Every clause earns its place; there is no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a simple zero-parameter read operation. The description states what is returned, where it comes from, and the important redaction behavior. Annotations already convey read-only and idempotent safety. No critical information is missing for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100%, so there are no parameter semantics for the description to clarify. Per the rubric, a zero-parameter tool gets a baseline of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Return') and a precise resource ('the running OPA server configuration from `GET /v1/config`'). This clearly identifies what the tool does and separates it from sibling tools like opa_status or opa_health, which concern server health rather than configuration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The use case is implied: if an agent needs the running OPA server configuration, this is the tool. However, the description does not explicitly state when to prefer this over alternatives or mention any exclusions, so guidance is present only by inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint, destructiveHint, and idempotentHint. The description adds that the tool runs 'opa check' and returns structured diagnostics with locations, providing behavioral context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the verb and resource. Every sentence adds essential information without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the return format (valid/errors with diagnostics) and explains the two input modes. No output schema is provided, but the description adequately describes the output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents each parameter. The description adds value by explaining the mutual exclusivity of source/paths and the purpose of maxErrors, going beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies the verb 'type-check' and the resource 'Rego', explicitly invoking 'opa check'. It distinguishes from siblings like rego_lint and rego_test by focusing on type-checking.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states the two mutually exclusive usage modes: inline source or file/directory paths. It does not explicitly list when not to use this tool vs alternatives, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes beyond annotations by detailing return values (formatted source and changed flag) and warning about a specific OPA bug that can corrupt escape sequences. This level of detail is valuable for agent decision-making.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first states core operation, second adds critical edge case. Extremely concise and well front-loaded with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite lacking an output schema, the description explains what is returned (formatted source + changed flag). Handles the single parameter fully and addresses version-specific behavior. Complete for its simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter 'source' having a clear description. The tool description adds no extra parameter context beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Format Rego source code using `opa fmt`', specifying the verb (format), resource (Rego source code), and method. Distinguishes from siblings like rego_check and rego_lint which serve different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context for when to use (formatting) and includes important caveats about OPA version and string interpolation bugs. Does not explicitly mention alternatives or when not to use, but given the distinct purpose, it's still effective.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive behavior, and the description goes well beyond that by disclosing the temp-file mechanism, the file-by-file verification order, digest-vs-parsed-value differences, failure reason enumerations, and the precise condition for returning `verified: true`. It also surfaces the `verificationKeyId` nuance.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every sentence carries substantive behavioral or edge-case information for a complex tool. It is front-loaded with the core purpose and implementation, then proceeds into verification details and error conditions. Some schema repetition exists, such as the `scope` instruction, but overall it earns its length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description thoroughly documents return behavior: `INVALID_BUNDLE` with an enumerated `details.reason`, `INVALID_INPUT` for unusable keys or algorithms, and the exclusive condition for `verified: true`. It also covers failure ordering, v0 compatibility, directory verification convention, and key-ID behavior, making the tool self-sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so a baseline of 3 applies, but the description adds meaningful semantics: `scope` is emphasized as needing to match the signing value exactly, `v0Compatible` is tied to post-signature failure behavior, and `verificationKeyId` is explained as rarely needing to be set with a single key. This enriches the schema without being redundant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence names a specific verb and resource: 'Verify the signature of a signed bundle directory or `.tar.gz` archive with the public key.' It also clarifies the implementation mechanism and the matching relationship to `opa_bundle_sign`, which distinguishes it from general Rego verification siblings like `rego_verify` and `conftest_verify`.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives strong usage context: it explains that OPA has no standalone verify command, how the verification is performed, what inputs are required, and how `scope` must exactly match the signing value. It does not explicitly name alternative tools or say when not to use this tool, but the guidance is clear enough to invoke correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint, idempotentHint), description discloses that directory-package-mismatch moves files, requires force for uncommitted changes, and suggests dryRun for safety. Fully transparent about mutating behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences plus a note, all essential. Front-loaded with main action and critical usage guidance. No fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but the description covers what the tool does and key behavioral notes. Lacks explicit mention of return value (e.g., success/error messages), but for a fix tool this is minor. Overall sufficiently complete with given annotations and schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. The description adds value by explaining the practical implications of key parameters (dryRun, force, disable) in context, aiding correct usage beyond schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs 'regal fix' to apply mechanical fixes and lists the five specific rules supported, distinguishing it from sibling tools like rego_lint or rego_format.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on using dryRun for preview, force for uncommitted files, and disabling directory-package-mismatch. Lacks explicit when-not-to-use advice, but the context is clear enough for correct invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that each call creates a new Gist (non-idempotent), requires GITHUB_TOKEN with 'gist' scope, and explains the return values. Annotations provide readOnlyHint=false and destructiveHint=false, which are consistent with creating external resources. The description adds crucial behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a moderate-length paragraph that front-loads the purpose and then provides supporting details. Every sentence contributes useful information (return format, usage with OPA, metadata bundling, authentication). It could be slightly more concise but remains clear and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, 1 required, no output schema), the description effectively covers invocation, return values, authentication requirements, and how to use the results. It provides sufficient context for correct tool selection and usage without relying on additional documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents each parameter. However, the description adds value by explaining that query, input, and data are bundled into metadata.json for reproducibility, and that rawPolicyUrl can be used directly with OPA or Conftest. This enhances practical understanding beyond schema definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: sharing a Rego policy as a public GitHub Gist for team collaboration or reproducible examples. It distinguishes itself from sibling tools (like rego_eval or rego_test) by focusing on sharing rather than evaluation or testing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (share policy, create reproducible example) and provides context for optional parameters (query, input, data bundle into metadata.json). It also notes the prerequisite GITHUB_TOKEN and what happens if missing. However, it does not explicitly state when not to use it or compare directly to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the tool is safe. The description adds that evaluations run in parallel and details the return format ('equal: true/false', raw results, 'changedPaths'). This goes well beyond the annotations, providing full behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loading the core purpose and then detailing behavior and parameters. Every sentence adds value with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters, 1 required, and no output schema, the description covers return values, parameter relationships, and key behavioral traits. It lacks error conditions or edge cases but is sufficient for a diff tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds meaning by explaining mutual exclusivity between source/path pairs and that the same input and query are used for both evaluations. This enriches understanding beyond the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool evaluates the same query against two policies and compares results. It specifies the verb ('Evaluate' and 'compare'), the resources ('two policies or two versions'), and distinguishes from siblings by focusing on diffing rather than single evaluation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit use cases ('verifying that a refactor preserves behavior' and 'understanding exactly where two policies diverge') and explains parameter relationships (mutual exclusivity of sourceA/pathA). However, it does not explicitly state when not to use this tool or list alternatives among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (destructiveHint=true, idempotentHint=true), the description discloses the exact mutation semantics: a directory gets `.signatures.json` written into it in place, while an archive is not modified and receives a sidecar signature in `outputDir`. It also reveals the portability caveat — files are recorded as `<directory name>/<file>` so the directory must keep its name — which no annotation could convey. No contradiction with annotations; the destructive hint is consistent with the in-place write.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core action and then builds logically: directory behavior, archive behavior, key format, claims, return values. Each sentence carries unique operational information — no filler, no restatement of the title — and the length is justified by the tool's two distinct input modes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description states the return values (path written, algorithm, file count) explicitly. It covers both input types, the mutation differences, key material requirements, claims injection, and names the sibling tools that complete the signing/verification workflow. Minor edge cases like overwriting an existing `.signatures.json` are not covered, but nothing essential to calling the tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter is already documented. The description adds interaction-level meaning: it explains that `bundle` may be a directory or archive, that `outputDir` applies only to archives, and that `signingKey` can be a PEM private key (RSA/ECDSA) or an HMAC secret file depending on `signingAlg`. It also ties `claimsFile` to concrete examples like `keyid` and `scope`, going beyond the schema's bare description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource — "Sign a bundle directory or `.tar.gz` archive with `opa sign`" — and immediately distinguishes the tool from its siblings by naming `opa_bundle_verify` (verification) and `opa_bundle_build` (build). It is not a tautology and leaves no ambiguity about what operation the tool performs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear workflow context: directories are signed in place and verified with `opa_bundle_verify` or via `opa build`/`opa run --bundle`, while signed archives come from `opa_bundle_build` with `signingKey`. The archive-versus-directory distinction tells an agent which invocation form applies. It stops short of explicit when-not-to-use exclusions, but the alternatives are named and the selection conditions are inferable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/idempotent/non-destructive, so the bar for added context is lower. The description adds real behavior: the tool returns Rego source, omits the AST by default, explains the size tradeoff, and notes the includeAst alternative. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, all substantive, with the main purpose in the first clause. No filler or duplication of schema fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-required-parameter read tool, the description covers what the caller gets (Rego source), the optional behavior (includeAst), and the alternative for AST. Annotations cover safety and idempotency, and schema covers parameters, so nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are fully documented; the description adds little beyond what the schema already provides. The mention that AST is omitted 'unless asked for' aligns with includeAst's schema description, so no additional compensation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description opens with a specific verb+resource: 'Fetch a single policy by ID from the running OPA server.' It clearly scopes to one policy, distinguishes from list/put/delete siblings, and differentiates from rego_parse_ast by stating this returns Rego source and AST is optional.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states that the AST is omitted unless asked and directs the agent to use `rego_parse_ast` when an AST is wanted, giving a concrete when-not. It also implies the primary use case—getting the Rego source for one policy—without ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=false. The description adds valuable behavioral context: it returns structured diagnostics, uses `opa check --schema`, and explains how mismatches are reported. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is 120 words, front-loaded with purpose, then behavior, return format, and parameters. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters (some nested), no output schema, and complexity of mutex constraints, the description covers all necessary details: purpose, return format, schema sources, path restrictions, and strict mode. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds meaning by explaining the mutual exclusivity of source/paths and inlineSchema/schemaPath, and that schemaPath must be within allowed roots. This goes beyond the schema's own descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb 'Validate' and resource 'Rego policy's input.* field references against a JSON Schema'. It clearly distinguishes from siblings by specifying the use of `opa check --schema`, which is unique among rego_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (for schema validation) and provides details on how to provide the schema (inline or path). However, it does not explicitly state when not to use it (e.g., for general syntax checks), though the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (readOnly, idempotent), the description details error handling (returns REGAL_NOT_FOUND), inline source behavior (auto-disable directory-package-mismatch, mangled file location), and re-enabling via 'enable'. This adds significant behavioral value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, front-loaded with purpose, no waste. Efficiently conveys critical information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers return format, error condition, tricky inline behavior, and re-enabling rules. Given 9 parameters (all schema-described) and no output schema, the description is sufficiently complete for an LLM to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. The description adds value by explaining global behaviors (e.g., inline source handling) that relate to parameters, but does not detail each parameter individually. Moderate added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Lint Rego source with the Regal linter' and specifies the output format. It distinguishes itself from siblings (e.g., rego_check, rego_format) as the only lint tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides prerequisites ('Requires regal on PATH or REGAL_BINARY set') and explains when to use inline vs paths, including auto-disabling of location-bound rules. It lacks explicit 'when not to use' but covers key usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond annotations: it details the two-phase process, explains that the tool returns migrated source and a changed flag even when errors remain, and specifies handling of invalid input (INVALID_REGO). This aligns with annotations (readOnlyHint, idempotentHint) without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (5 sentences) and front-loaded with the purpose. Each sentence adds distinct information: purpose, phases, return values, edge case. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity and no output schema, the description covers all essential aspects: input, process, output (migrated source, errors, INVALID_REGO), and behavior when errors remain. An agent has enough context to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, baseline is 3. The description adds value by explaining the parameter's purpose in context of migration, though the schema description already repeats tool behavior. The single parameter is well-documented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: migrate Rego v0 to v1 syntax. It specifies the two-phase process (auto-fix with opa fmt --rego-v1 and validation with opa check --v1-compatible) and differentiates from other rego tools by covering both formatting and checking.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use this tool (for migration) and what happens in each phase. It implicitly contrasts with siblings like rego_format and rego_check by describing combined functionality. However, it does not explicitly state when not to use it or list alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description extensively discloses behaviors beyond the annotations: it returns concrete counterexamples, reports INCONCLUSIVE for specific unsupported constructs, treats absent fields as undefined, and explains the empty-input caveat for always_true. Annotations only declare read-only/idempotent, so this adds substantial value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but information-dense; every clause contributes a necessary limitation or behavioral detail. It is front-loaded with the core verification promise before moving to edge cases, making it efficiently scannable for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema, the description covers all major outcomes: proof, counterexample, INCONCLUSIVE, unsupported language features, and undefined-field semantics. An agent has enough information to invoke the tool correctly and interpret likely results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds semantic depth beyond the schema by clarifying what always_true means for absent fields and how a rule requiring input.x yields the counterexample {}, which enriches the enum definitions in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: formally verify a property about a Rego rule using SMT solving. It clearly distinguishes itself from testing by checking ALL possible inputs and from sibling eval/test tools by emphasizing proof or counterexample output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context for when to use this tool: when exhaustive verification is desired instead of testing, and it explains when results will be INCONCLUSIVE due to unsupported constructs. It does not explicitly name sibling alternatives, but the 'Unlike testing' contrast and limitation list provide practical usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (destructiveHint=true, idempotentHint=false), the description discloses the 204 success response, the 404-to-DATA_NOT_FOUND mapping, and the intentional root-path exclusion. These behaviors directly inform an agent about success, failure, and edge cases, which is exactly the kind of context annotations alone cannot provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is information-dense but every sentence earns its place: the core action, path syntax rules, fallback parameter guidance, and edge-case behavior are all stated with no filler. The most important context is front-loaded and the formatting is easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter destructive operation with no output schema, the description is complete: it covers success codes, error mapping, path constraints, and the root-path edge case. The annotations already mark the destructive nature, and the description fills the remaining behavioral gaps an agent would need to safely invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema already covers 100% of parameters, the description adds crucial semantics: dotted vs slash-only path parsing, how to address keys containing dots or slashes, and when to switch from `path` to `segments`. This resolves ambiguous inputs that the schema descriptions only partially convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Remove') with a clear resource ('document from OPA's data store') and the path-based scope. It naturally distinguishes itself from sibling tools like opa_delete_policy by explicitly targeting the data store rather than policy.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear operational context: it explains when to use `path` vs `segments`, notes the root-path deletion exclusion, and requires at least one segment. It does not explicitly compare itself to alternative data-management tools like opa_patch_data or opa_put_data, but the conditional guidance is strong and the tool's purpose is unmistakable among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Although annotations already declare readOnlyHint=true and idempotentHint=true, the description adds substantial behavior beyond that: per-file independent evaluation, error fields instead of result fields for failing files, CI-gate behavior reporting failed: true instead of erroring, and the decision-name normalization that prevents undefined results. No contradiction with annotations exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place: the core operation and tool name are front-loaded, the sibling comparison follows immediately, then per-file behavior, CI gates, and mutual exclusions are covered. The length is justified by the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 9-parameter tool with no output schema, the description covers invocation details, mutual exclusions, gate semantics, timeout bounds, and decision-name conversion. It could go slightly further by sketching the successful result envelope shape, but it adequately supports correct selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter already has a solid description, so the baseline is 3. The description adds aggregate meaning by clarifying the interaction between fail/failDefined/failNonEmpty and the overall CI-gate result, plus the semantic difference between bundle and dataPaths. It adds value beyond the schema without replacing it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Evaluate a policy decision against one or more input files using opa exec --format=json.' It immediately distinguishes itself from the sibling tool rego_eval by noting the single-input vs. batch-input difference, so an agent can tell them apart without inspecting schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly names rego_eval as the alternative for single inputs and says opa exec is 'ideal for CI pipelines that check many config files against a policy in one call.' It also explains when to use the fail gates and that bundle and dataPaths are mutually exclusive, giving clear selection and invocation guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds extensive behavioral detail beyond annotations, including the leaf discovery rule, coverage mode switching, per-root threshold behavior, and interaction of parameters like `varValues` requiring `verbose`. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise for the complexity, covering essential details in a few sentences. Could benefit from bullet points for the two modes, but the prose is clear and front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (11 parameters, two modes, scan logic, coverage nuances) and lack of output schema, the description thoroughly explains all necessary context for correct invocation, including edge cases like the leaf rule and mutual exclusivity of `roots` and `scanDir`.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 11 parameters have schema descriptions (100% coverage). The overall description adds contextual understanding of parameter interactions (e.g., `coverage` and `threshold` affecting output mode, `sharedPaths` excluded from scan). Slightly more structured parameter grouping would improve, but still adds value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs `opa test` once per root and aggregates results, solving the package-conflict problem (OPA issue #4724). It distinguishes itself from sibling tools like `rego_test` by specifying multi-root handling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly describes two modes (`explicit` and `scan`) with their use cases, when each should be used, and mentions when to use alternative like `rego_test` for single roots. Provides clear context for `sharedPaths` and coverage options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint/idempotentHint annotations, the description discloses meaningful behavioral details: the response reflects startup-reported configuration, includes bundle/decision-log/plugin settings, and service header values are redacted because they may contain API keys. This adds genuine transparency beyond what structured annotations already convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: the first states the action and endpoint, the second clarifies the difference from a sibling tool, and the third covers response contents and a security-relevant redaction. The description is front-loaded and compact with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although there is no output schema, the description compensates by enumerating response categories, clarifying the relationship to opa_config, and warning about redacted header values. For a zero-parameter read-only status tool, this is sufficient context for an agent to invoke it correctly and interpret its result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so schema coverage is complete by definition and the description need not explain parameters. It still adds useful context about what the returned configuration document contains and the redaction policy, which is more than the empty schema provides. Baseline 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource ('Return the running OPA server configuration via GET /v1/config') and precisely distinguishes this tool from its sibling opa_config by noting the 'status' key presentation and the intent to check 'what is running' vs 'what was configured'. This gives an agent a clear, unambiguous purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly names the alternative tool opa_config and states the selection criterion: use this when the agent wants 'what is running' rather than 'what was the server configured with'. This is direct routing guidance with no ambiguity about when to prefer this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
GitHub Badge
Glama performs regular codebase and documentation scans to:
- Confirm that the MCP server is working as expected.
- Confirm that there are no obvious security issues.
- Evaluate tool definition quality.
Our badge communicates server capabilities, safety, and installation instructions.
Card Badge
Copy to your README.md:
Score Badge
Copy to your README.md:
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/OrygnsCode/opa-mcp-server'
If you have feedback or need assistance with the MCP directory API, please join our Discord server