Server Quality Checklist

Tool Scores

• savingsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It indicates a read-only reporting behavior by saying 'Show' and names the output categories, but it does not disclose data sources, freshness, or what exactly 'kept off your premium quota' means. This is acceptable but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that leads with the action and object, then adds a clarifying parenthetical. Every word contributes meaning, and there is no wasted or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only savings overview with an output schema available, the description conveys enough purpose and scope to guide an agent. It could elaborate on what 'premium quota' encompasses or what estimates/list prices derive from, but these are minor gaps for such a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter documentation burden. The description still adds semantic context by explaining what the savings refer to: delegation effects on premium quota, estimates, and list prices.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: showing the savings that delegating has produced, specifically in terms of premium quota, estimates, and list prices. It uses a specific verb and resource, making it distinguishable from sibling tools like delegate and list_models, though it does not explicitly name them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for viewing delegation-related savings, but it gives no explicit guidance on when to use this tool versus delegate or list_models. No when-not-to-use conditions or alternative recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. 'List' indicates a read-only, side-effect-free operation, and the output content is specified as aliases plus their best use. No mutation or destructive behavior is suggested.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One compact sentence that front-loads the key information with no filler. Every part of the description earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero-parameter, read-only, output-schema-bearing nature of the tool, the description fully covers what the agent needs: what it returns and why it matters for delegate().

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description correctly avoids inventing parameter details and focuses on what the tool returns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a precise verb ('List'), a concrete resource ('model aliases available to delegate()'), and the value add ('with their best use'). This clearly distinguishes the tool from delegate and the unrelated savings sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'available to delegate()' tells the agent this is the lookup step before selecting a delegate model, so the usage context is clear. It does not explicitly state when not to use it or compare it with savings, so it stops short of full explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delegateA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it delivers: it discloses that the model sees only the task string, has no conversation access, blocks for ~10- 60s, routes models by task, and returns an error-prefixed string on failure. It also explains the max_tokens rationale, making runtime behavior predictable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence carries information: usage boundaries, parameters, latency, failure mode, and return value. It is front-loaded with the core purpose and uses clear section labels (WHEN TO USE, Args, Note, Returns) for scannability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers what the tool does, when to use it, what each parameter means, what the model sees, how long it takes, and what is returned on both success and failure. Nothing essential is missing for safe and correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain the parameters. It defines task as self-contained, explains the auto routing behavior for model with specific model aliases, and gives the default and purpose of max_tokens. This is more than enough to invoke the tool correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: offloading a self-contained subtask to a cheaper model and returning its output. It clearly distinguishes itself from premium-token work and from the unrelated siblings (savings, list_models) by defining its exact role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides an explicit WHEN TO USE section with concrete examples of acceptable grunt work and an explicit DO NOT delegate list for judgment work. This gives an agent clear criteria for choosing this tool over doing the work itself or using another tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.