Server Quality Checklist

Tool Scores

• recallC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the description need not repeat that. It adds the concepts of 'bounded' and relevance to the current task, which are minor behavioral cues, but it does not disclose retrieval mechanism, result format, or error handling. The added value is modest.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words and the primary action is front-loaded. It is efficient, though its brevity may contribute to the lack of parameter and usage clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters (2 required), low schema coverage, and no output schema, the description is insufficient. It does not explain how to construct the query, what 'limit' or 'max_tokens' control, or what the response contains. An agent would need to consult other sources to call this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33% (only bank and workspace have descriptions). The tool description neither explains the remaining parameters (query, as_of, limit, max_tokens) nor provides any parameter-level guidance. The phrase 'bounded source observations' hints at limits but does not map to specific parameters, failing to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('Retrieve') and a resource ('bounded source observations'), which distinguishes it from sibling tools like 'record' and 'forget'. However, it does not explicitly mention the 'bank' parameter or clarify the meaning of 'bounded', leaving some ambiguity about scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'explain' or 'correct'. The only contextual hint is 'relevant to the current task', which is vague and does not explain prerequisites, exclusions, or when to prefer recall over other memory operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• correctB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true, which covers the mutation aspect. The description adds 'with auditable evidence,' implying that actions are logged or supported by evidence—information not present in annotations. However, it does not elaborate on the nature of the audit trail or what happens to the superseded/retracted claim, so the added value is modest.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, compact sentence conveys the core purpose without wasted words. The action and key qualifier are front-loaded, making it easy to scan. No redundant phrases or unnecessary detail are present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with 14 parameters, a one-line description is severely insufficient. It omits operational context (e.g., what constitutes a 'current claim', how to specify the target, acceptable evidence types, return behavior, or error conditions). Combined with no output schema and low parameter coverage, the description leaves critical gaps that will impede correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With only 14% schema description coverage and 14 parameters, the description carries the burden of explaining key parameters. It does not elaborate on claim_id, operation, rationale, valid_from, valid_to, or other fields. The phrase 'auditable evidence' hints at rationale/excerpt but does not map to any parameter, leaving the agent without semantic guidance for required inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Supersede or retract') and the resource ('a current claim'), and includes the qualifier 'with auditable evidence' that signals a specific audit-trail behavior. It implicitly distinguishes itself from siblings like record (creating new claims) and forget (possibly deletion) by focusing on correction of existing claims.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given on when to use this tool versus alternatives. The description implies correction of existing claims but does not state conditions, prerequisites, or contrast with siblings. An agent must infer usage from the operation names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explainB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description adds context about the kind of information returned (lifecycle stages, relations, evidence granularity), which goes beyond bare annotations. However, it doesn't describe output format, pagination, or any side effects beyond what annotations imply, so it adds moderate value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It efficiently conveys the core purpose without redundancy, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description doesn't explain the return structure or how the 'explain' information is presented (e.g., nested JSON, human-readable text). It also doesn't mention any constraints like required permissions or workspace handling. For a tool that likely returns complex claim details, this is a notable gap, though the basic purpose is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67%: 'bank' and 'workspace' have descriptions, but 'claim_id' does not. The tool description doesn't clarify any parameters further, especially the undocumented claim_id. It adds no meaning beyond what the schema already provides, leaving the unidentified parameter ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'show' and the resource 'claim' with specific aspects: lifecycle, relations, and exact supporting evidence. This distinguishes it from siblings like recall (retrieve), record (create), correct (modify), and forget (delete) by focusing on detailed visualization rather than simple retrieval or mutation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor any conditions for exclusion. It doesn't mention whether it's preferred for debugging, auditing, or investigating claim states, leaving the agent to infer from the name and siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recordB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false. The description adds context about user approval and non-inference of claims, which are beyond the annotations. But it doesn't disclose other behaviors like error handling, rate limits, or side effects beyond the basic write operation, so it adds some value but not a lot.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero fluff. It front-loads the core purpose, then gives a focused, actionable instruction for the most complex parameter combination. Every word earns its place, and the layout makes it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 12 parameters, nested objects, and no output schema, the description is insufficient. It only addresses the 'attempt' kind and does not cover other potential uses or explain most parameters. There is no mention of return values, error cases, or examples. While annotations cover idempotency and destruction, the description leaves too much to inference for a complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 17%, so the description carries considerable burden. It clarifies kind, content, and metadata specifically for the 'attempt' case, which is valuable. However, it leaves the majority of the 12 parameters (e.g., actor_id, actor_type, sensitivity, trust_class) unexplained, and provides no general semantics for other kinds. It partially compensates for the low coverage but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Store') and resource ('raw observation'), with a qualifier ('user-approved') and a constraint ('without inferring claims'). It is unambiguous, though it doesn't explicitly distinguish from sibling tools like 'explain' or 'recall', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance for a specific scenario (setting kind to 'attempt') and outlines required metadata fields. However, it lacks any guidance on when this tool should be used compared to siblings, no exclusions or alternatives are mentioned, so it's partially helpful but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forgetA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and idempotentHint=true, and the description adds value by explaining the 'privacy-delete' nature (implying data sanitization) and the propagation of retractions. This goes beyond the bare annotation flags and gives a clearer picture of side effects, though it doesn't detail the propagation mechanism itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that front-loads the core action ('Privacy-delete') and immediately follows with the key side-effect. There is zero fluff; every word carries functional meaning, making it an exemplary concise definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple deletion tool with 3 parameters and no output schema, the description covers the essential action and side effects. It does not mention return values, error handling, or prerequisites, but given the low complexity and annotations covering idempotency and destructiveness, it is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67%, meaning two of three parameters have descriptions in the schema. The tool description does not add any semantic detail about parameters (e.g., what 'bank' or 'observation_id' specifically mean in this context). Since coverage is not high (>80%) but not low (<50%), the baseline 3 applies, and the description makes no attempt to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Privacy-delete') and resource ('one source observation') and adds a distinguishing side-effect ('propagate direct retractions'). This clearly sets it apart from siblings like 'record' or 'explain', making the tool's purpose instantly recognizable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given for when to use this tool versus alternatives. It does not mention conditions, exclusions, or scenarios where a different sibling (e.g., 'correct') would be more appropriate. The usage context is entirely left to the agent to infer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.