Server Quality Checklist

Tool Scores

• get_trial_balanceC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full behavioral disclosure burden. 'Read' signals a non-mutating operation, but the description gives no detail about return contents, error behavior, data scope, or defaults such as as_of behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence with no wasted words. The key action and target resource are front-loaded, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description is responsible for explaining what the agent can expect. It does not describe the trial balance contents, return format, or behavior of the optional as_of parameter, leaving notable gaps for a tool this minimal.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 50%, with engagement_id left undocumented. The description adds no parameter-level meaning beyond implying engagement context, so it does not compensate for the missing schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the verb ('Read') and resource ('a trial balance for an engagement'), so an agent can understand the core function. It stops short of a 5 because it does not explicitly distinguish itself from sibling tools like get_capabilities or list_engagements.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to choose this tool over alternatives, nor any exclusions or prerequisites. The intended use is inferable from the name and phrasing, but the tool itself does not state it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_review_notesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates a read operation, but does not disclose ordering, pagination, permission needs, error behavior, or what happens with empty results. This is minimal and leaves the agent with limited expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. The verb and object are front-loaded, and every word contributes to the meaning. It could not be meaningfully shorter without losing information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read tool, the description is minimally viable: it states the action and target object. However, with no annotations, no output schema, and no behavioral details, an agent is left without information about return format, ordering, or edge cases. It is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description needs to compensate for the engagement_id parameter. It only says 'on an engagement,' which weakly ties the parameter to the resource, but it does not explicitly explain what engagement_id is, its format, or its role in the request. This is insufficient for a parameter with no schema-level documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Read') and a clear resource ('review notes on an engagement'), so the core purpose is unambiguous. It does not explicitly differentiate from sibling tools, but the operation is distinct enough from list_engagements, get_trial_balance, and propose_adjustment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There is no discussion of prerequisites, when to prefer it over another tool, or when not to use it. The engagement context is implied but no explicit routing is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_engagementsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states that engagements are 'visible to the adapter' but does not clarify read-only behavior, output shape, pagination, ordering, or failure modes. 'List' implies reading, but little else is disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence that front-loads the verb and resource with no filler. Every word contributes to the meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list operation, the description is minimally sufficient to invoke the tool correctly. However, because there is no output schema and no annotations, the return format and the precise meaning of 'engagements' remain unexplained, leaving the description only partially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema is fully empty, so there is no parameter burden for the description to carry. The baseline of 4 for a no-parameter tool applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and names the resource 'engagements', adding a scope qualifier 'visible to the adapter'. It is distinguishable from the siblings by name and scope, though 'engagements' is not defined further.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus siblings like get_capabilities, get_trial_balance, list_review_notes, or propose_adjustment. The agent must infer usage entirely from the tool name and a one-line description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• propose_adjustmentA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly discloses the critical side-effect behavior: 'nothing is written unless mode is commit.' Since no annotations are provided, this is essential and well-handled. It does not disclose other behavioral aspects like validation errors or idempotency behavior, but the core write-safety concern is addressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence, front-loading the tool's purpose and following with the key safety behavior. Every word earns its place; there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters, no annotations, and no output schema, the description is too sparse to be complete. It covers the central write-safety behavior but omits context around required inputs, validation semantics, idempotency, and what a successful plan or commit returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, and the description adds no parameter-level meaning beyond the mode default already present in the schema. The 'nothing is written unless mode is commit' line clarifies mode's behavioral effect, but engagement_id, description, rationale, and idempotency_key remain semantically undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Plan') and resource ('an adjusting entry'), and the qualifier about validate mode makes clear this is a planning/write tool rather than a read tool. This differentiates it from the sibling tools, which are all read-oriented (get, list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives, nor does it mention prerequisites like an existing engagement or trial balance. The only contextual hint is the planning-vs-commit distinction, which is more behavioral than usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_capabilitiesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does indicate a read-only introspection action ('Describe') with no side effects, which is helpful. However, it does not disclose the output format, whether any configuration or permissions are needed, or what exactly will be listed in 'what it supports'. The behavior is not contradictory, just underexplained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, compact sentence with no filler. The action ('Describe the configured adapter') is front-loaded, and the second clause ('what it supports') adds relevant scope without redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description is the only source of behavioral context. It states the tool's purpose but leaves the return value's shape and the precise extent of 'supports' unspecified. An agent could call the tool and be surprised by the response structure, but for a simple introspection call this is an acceptable gap rather than a fatal one.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema, so there is nothing for the description to add. Per the rubric, zero parameters earn a baseline of 4. The description correctly avoids inventing parameter details that do not exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Describe') and resource ('configured adapter and what it supports'), making the tool's purpose understandable. It is distinct from sibling tools like list_engagements and propose_adjustment, which clearly concern different resources. However, the phrase 'what it supports' is somewhat vague and could be more explicit about the kind of capabilities being described.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used to inspect adapter capabilities, but it gives no explicit guidance on when to use it versus alternatives, nor does it mention exclusions or prerequisites. The sibling tools are unrelated enough that confusion is unlikely, but the lack of any 'use this when' statement leaves the guidance at the level of implication.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.