Server Quality Checklist

Tool Scores

• brancher_createC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It mentions waiting, sanitizing, and reporting, hinting at a non-instant operation, but fails to explain what 'sanitize' means, whether it's destructive, what permissions are needed, or what 'report it ready' entails. The description is too vague to make the tool's behavior predictable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise in length, but it packs multiple actions without clear separation or explanation. It is not well-structured for quick comprehension of the tool's purpose and behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters (2 required), no schema descriptions, and an output schema (content unknown), the description is severely incomplete. It does not explain the parameters, the return value, or the actual behavior beyond vague steps. The agent cannot reliably invoke this tool based on the description alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention any of the three parameters (appname, labels, clear_services). The description adds no meaning beyond the schema, leaving the agent without any guidance on how to populate the inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Create' and the resource 'Brancher node', distinguishing it from sibling tools like list, delete, exec, put, and ssh_info. It adds procedural steps (wait, sanitize, report ready) which, while vague, still clarify the tool's multi-step nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor any prerequisites, exclusions, or context. The description only states what it does, not when it should be chosen.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• brancher_ssh_infoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for behavioral disclosure. It only states the return value, omitting whether the operation is read-only, requires authentication, what happens if the node does not exist, or any error conditions. This is insufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. However, it is too brief to cover necessary details, making it merely adequate rather than excellent. It earns its place but misses opportunities to add value without much extra length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple info retrieval tool with an existing output schema, the description covers the essential return fields (host, user, port). However, it does not address error scenarios, preconditions (node existence), or side effects. Annotations are absent, leaving behavioral gaps. Completeness is acceptable but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage for the required node_name parameter. The description adds only 'for a Brancher node', implying node_name identifies a node but failing to explain valid values, case sensitivity, or where to obtain the name. It does not compensate for the schema's lack of documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns SSH connection details (host, user, port) for a Brancher node, which is a specific verb and resource. It differentiates well from siblings like brancher_list (listing) or brancher_exec (executing commands), leaving no ambiguity about this tool's role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., use it after listing nodes to get connection info, or before executing SSH commands). No when-not-to-use or exclusion criteria are mentioned, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• brancher_putB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It only mentions 'via rsync', but does not disclose overwrite behavior, directory creation, error handling, or any side effects. The agent is left guessing about important safety-relevant behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no fluff or repetition. It is front-loaded and efficient, containing exactly the core information without wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema (content unknown), the description fails to address many aspects relevant to a file sync tool: return values, error conditions, whether directories are created, handling of existing files, permission requirements, or rsync flags. The brevity leaves significant gaps for practical usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% — none of the parameters have descriptions in the schema. The tool description merely restates the role of local_path and remote_path ('sync local_path to remote_path') but does not clarify their format, constraints, or the purpose of node_name and port. The linking of path parameters is the only semantic addition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('sync'), the source ('local_path'), destination ('remote_path'), the mechanism ('via rsync'), and the target ('on a Brancher node'). This distinguishes it from sibling tools like brancher_list, brancher_delete, and brancher_exec, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. For example, it doesn't mention when to use brancher_put instead of brancher_exec for file transfer, or if there are size or permission limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• brancher_listB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. 'List active Brancher nodes' suggests a read-only operation, but does not clarify if the list is paginated, limited, or includes metadata (e.g., status, uptime). The description minimally conveys safety (read-only) but omits specifics like authentication needs or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just one line with no wasted words. It appropriately front-loads the action and target resource, making it easy for an AI agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is only one parameter and an output schema exists, the description is somewhat complete for a simple listing tool. However, it lacks details on what the list contains (e.g., node IDs, IPs, status) and does not clarify if the tool returns only active nodes or all nodes filtered by activity. With no annotations, more context on behavior would be valuable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes only one parameter (`appname`) with 0% schema description coverage, meaning the description must add meaning. However, the description only mentions `appname` in context without elaborating on its format, acceptable values, or examples. It merely restates that `appname` is needed, adding little beyond the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and the resource (active Brancher nodes), and it specifies the required parameter `appname`. However, it does not differentiate from sibling tools like `brancher_ssh_info` or `brancher_create` in terms of what makes this specific listing distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by requiring `appname`, but provides no explicit guidance on when to use this tool versus alternatives (e.g., `brancher_ssh_info` for SSH info or `brancher_delete` for deletion). There is no mention of prerequisites or conditions for using the list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• brancher_execA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description effectively discloses the tool's behavior: it executes a command via SSH on a specific node and returns standard output, error, and exit code. It implicitly informs the agent that this is a potentially impactful action (remote command execution) and that it requires SSH access, which is transparent enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, just one sentence with 11 words. It front-loads the main action and return value, leaving no wasted words. Every element (execute, command, SSH node, return) is necessary and adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple tool with a clear action and an output schema (implied return of stdout/stderr/exit_code), the description covers the essential purpose and behavior. It does not specify failure modes or SSH configuration requirements, but given the context (no nested objects, few parameters) and the presence of an output schema, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 0%, the description briefly adds context by naming the two required parameters (command, node_name) within its purpose. However, it does not explain the optional timeout parameter (default 30 seconds) or provide details on valid formats or constraints for the parameters beyond what is in the schema. Given low coverage, the description compensates somewhat.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Execute over SSH), the target (Brancher node), and the return values (stdout/stderr/exit_code). It effectively distinguishes from sibling tools like brancher_list, brancher_put, etc., which are about managing files or listing, not executing commands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when or when not to use this tool versus alternatives. Since there are siblings like brancher_ssh_info which might provide connection info, but no instructions on when to prefer one over the other or any prerequisites (e.g., SSH setup) are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• brancher_deleteA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral disclosure. It reveals that deletion is not immediate but requires a second call with confirm=True, which is a critical behavioral trait. However, it does not elaborate on what happens on the first call (e.g., no-op or preview) or whether deletion is reversible, which keeps it from being a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core action ('Delete a Brancher node') and immediately follows with the critical behavioral constraint. Every word earns its place; there is no redundancy or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 parameters, no nested objects) and the presence of an output schema, the description is nearly adequate but lacks clarity on what the first call (without confirm=True) does. This omission could confuse an agent about the tool's behavior. The description is otherwise sufficient for the basic delete operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only partially addresses the confirm parameter by explaining its role in the gating mechanism. The node_name parameter is not described at all. This leaves a significant gap for the required parameter, limiting the agent's ability to correctly invoke the tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Delete a Brancher node') with a specific verb and resource. It also distinguishes from sibling tools (brancher_create, brancher_list, etc.) by implying deletion rather than creation, listing, or execution. The mention of the confirmation gating adds specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when a node needs to be deleted, but it does not provide explicit guidance on when to use it versus alternatives (e.g., when not to delete, or that brancher_create might be needed to recreate). No exclusions or alternative tools are mentioned, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• brancher_appsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly indicates this is a read-only listing operation, but it does not disclose any potential edge cases (e.g., output size, pagination, or error behavior). The mention of 'every' suggests comprehensiveness, but no additional behavioral traits are described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that precisely conveys the tool's purpose without any redundant information. It is perfectly front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with no parameters and an output schema available, the description is complete. It specifies exactly what is listed (Hypernode app names) and the filtering condition (with a configured API token). No further details are necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing for the description to clarify. The baseline score of 4 applies, and the description correctly avoids any unnecessary parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (List), the resource (Hypernode appname), and a specific condition (with a configured API token). It effectively distinguishes from sibling tools like brancher_list by specifying the token requirement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool—when you need to list Hypernode apps that have API tokens. It does not explicitly mention alternatives or exclusions, but the purpose is self-evident enough for basic selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.