Server Quality Checklist

Tool Scores

• get_build_waitC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals that the tool is non-blocking and returns a state or final result, but it does not explain poll behavior, what happens before completion, possible states, error behavior, or whether the result is one-time or repeatable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single tight sentence with no redundant phrasing and the core action is front-loaded. However, it is so sparse that it borders on under-specification rather than efficient completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, no annotations, and only one parameter that lacks semantic explanation. The description does not explain return values, possible states, id meaning, or how this differs from the sibling tools, leaving an agent without enough context to invoke it correctly with confidence.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description adds no meaning about the 'id' parameter. The schema only says it is a UUID; the tool description never clarifies whether the id identifies a wait, a build, or some other entity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and names a resource ('state or final result of a non-blocking build wait'). It is understandable on its own, but 'state or final result' is somewhat ambiguous and it does not explicitly distinguish itself from siblings like get_build_status or get_build.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'non-blocking build wait' implies it relates to start_build_wait and contrasts with await_build, but the description never says when to use this tool versus get_build_status, get_build, or await_build. No explicit when-to-use or alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_build_waitC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior, but it only mentions non-blocking. It does not explain what the tool returns, whether a wait handle is created, how to retrieve the result later, or what happens if no completed version appears.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler, but the phrase 'after afterVersion' is awkward and potentially confusing. It is concise but not as polished or clear as it could be.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description leaves out important operational details: timeout semantics, expected return value, and how the non-blocking wait is later consumed. An agent would likely need additional tool definitions or trial-and-error to use it confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description gives meaning to afterVersion by explaining that the wait is for the first completed version after that version. However, timeoutMs is completely ignored, and schema description coverage is 0%, so the description only partially compensates for the missing parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: starting a non-blocking wait for the first completed version after a given version. The 'non-blocking' qualifier helps distinguish it from likely blocking alternatives like await_build, though it does not name the sibling explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'non-blocking' wording implies when to use this tool (when you don't want to block on a build wait), but it gives no explicit guidance on how it relates to siblings like await_build or get_build_wait. Usage context is only implied, not clearly spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• await_buildC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral disclosure burden. It reveals the tool waits for a build result but does not explain blocking behavior, timeout handling, polling mechanics, or what happens if no version after afterVersion completes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single terse sentence with no fluff. However, 'after afterVersion' is slightly awkward and repetitive, though it does not materially hurt clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a blocking/waiting operation with no output schema and no annotations, the description omits critical details: what the return value looks like, timeout/error behavior, and how this relates to sibling wait/status tools. An agent could invoke it but would not know what result to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%: afterVersion has a schema description, timeoutMs only has default/min/max. The description adds relational meaning to afterVersion ('first completed version after...') but does not explain how timeoutMs affects the wait or what it measures.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Await'), a resource ('build result'), and a selection rule ('first completed version after afterVersion'). It is clear about the core operation, though it does not explicitly distinguish it from sibling wait/status tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance about when to use this tool instead of get_build_wait, start_build_wait, get_build_status, or get_build. There is no mention of preferred scenarios, preconditions, or exclusions, leaving the agent to infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_buildC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not explain what 'build result' includes, whether this operation is read-only, how errors are surfaced, or how it differs from checking build status. The description mostly restates the tool name's obvious meaning.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that names the verb and object immediately. It is appropriately concise for a simple tool, though it sacrifices behavioral and comparative detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and has no required parameters, so the schema covers invocation adequately. However, with no output schema and no explanation of what 'build result' contains, an agent may still be unsure what it will receive or how this differs from similar sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents the only parameter 'version', including its type, default, and lower bound. Since schema coverage is 100%, the description adding little parameter detail is acceptable; the phrase 'specified version' aligns with the schema but adds no new meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Return') on a specific resource ('build result') for a specified version. It is reasonably clear, though it does not explicitly differentiate itself from get_build_status, which could plausibly return similar information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives like get_build_status or await_build. The description implies a retrieval purpose by saying 'Return the build result', but it provides no exclusions, conditions, or sibling comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_build_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears the full burden of disclosing behavior. It only says 'Return', which implies a read-only snapshot, but it does not clarify blocking behavior, staleness, absence of a build, or how state is represented. The presence of await_build and get_build_wait siblings makes the missing blocking distinction significant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler or repetition. Every word contributes to defining what the tool returns, which is appropriately concise for a zero-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, no annotations, and several closely related sibling tools, the description is under-specified. An agent cannot tell whether the call returns immediately or waits, what the response structure looks like, or when to choose this over the sibling tools. A few additional sentences on these points would be needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so there is no parameter semantic burden; the empty input schema already covers everything. The description does not need to explain parameters, making the zero-parameter baseline appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and a concrete resource ('build process state and latest completed version'), so the core purpose is clear. It does not explicitly distinguish itself from siblings like get_build or await_build, but the resource is specific enough to be understood.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to call this tool versus get_build, await_build, get_build_wait, or start_build_wait. It does not state whether this is a non-blocking status check or whether it should follow a build start, leaving the agent to infer from the name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• debug_get_live_outputA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral burden. It does convey that the tool returns recent raw stream output and is intended only for debugging, but it does not clarify what 'recent' means, whether the call is read-only/non-blocking, or how it behaves for finished or nonexistent builds. This is useful but incomplete transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler. The core behavior is front-loaded, and the usage restriction is a natural second sentence. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only debug tool with one optional parameter, the description is mostly sufficient: it names the output, the source, and the intended scenario. Gaps like the precise meaning of 'recent' and behavior across build states keep it from being fully complete, but an agent can likely select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and the tool description does not mention tailChars at all. The parameter name and JSON-schema constraints (default, maximum, exclusiveMinimum) provide some meaning, but the description adds no guidance on how to choose or omit the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and a specific resource ('recent raw stdout and stderr from build processes'), which clearly differentiates it from sibling tools about build status or waiting. The phrase 'Meant only for debugging stuck builds' sharpens the intended scope further.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear use context: debugging stuck builds, and even adds an exclusion with 'Meant only.' However, it does not explicitly name sibling alternatives or state when to choose another tool like get_build_status or await_build, leaving some routing to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.