Server Quality Checklist

Tool Scores

• get_eventsC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the operation is a read, which implies non-destructive behavior, but it doesn't confirm read-only status, how limit interacts with pagination, or whether results are ordered. It also doesn't mention any error conditions, rate limits, or what happens when after_event_id exceeds the latest event. The description is minimal and leaves key behavioral aspects unexplained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no redundancy, and the core purpose is front-loaded. However, it is under-specified: it omits essential usage details that a tool of even moderate complexity should include. While conciseness is good, it sacrifices too much information, making it only minimally viable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (two optional parameters, no required fields) and the presence of an output schema, the description needn't cover return format. But it still lacks context on how to use the cursor for multi-page reads, whether events are returned in ascending order, and how this relates to watch_events. The 'after an event ID' phrase provides a hint, but without explicit guidance on pagination or usage patterns, the description is incomplete for an agent that needs to use this correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented parameters. It only hints at the meaning of after_event_id ('after an event ID') but says nothing about limit, its default, or the format/range of after_event_id. Without this, an agent cannot reliably construct a correct call beyond guessing at the defaults. The description adds almost no semantic value beyond what the property names inherently suggest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Read' and the resource 'durable task/artifact lifecycle events', with a specific qualifier 'after an event ID' that defines the cursor. While it doesn't explicitly name sibling tools, the phrasing distinguishes it from a streaming tool like watch_events by indicating it reads a past snapshot. It's specific enough for an agent to infer its core purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives like watch_events or list_tasks. The description implies incremental fetching via 'after an event ID' but doesn't state prerequisites, typical usage patterns, or when to prefer a different tool. An agent is left to infer that this is for retrieving historical events, but the absence of explicit context or exclusions is a notable gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_graphC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that the tool 'return[s]' data, implying a read-only operation, but does not explicitly mention that it has no side effects, does not mutate state, or any other behavioral traits. There is no mention of authentication, permissions, or consequences, which is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, which is efficient and front-loaded with the core purpose. However, it is so sparse that it omits necessary details, making it under-specified rather than appropriately concise. It earns a middle score because it is not verbose, but the brevity comes at the cost of completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists and can document return values, the description still lacks context about parameter usage, edge cases, or how this tool fits into the workflow. With two parameters and zero semantic explanation, plus no annotations, the description is insufficient for an agent to confidently use the tool. It leaves major gaps in understanding the tool's behavior and invocation requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning the description must explain parameter meaning. However, the description makes no mention of 'task_id' or 'depth' at all. It provides no context for what these parameters represent, how they affect the returned graph, or any constraints. This is a critical omission for an agent to understand how to invoke the tool correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a 'persisted dependency neighborhood around a task,' which conveys a specific resource and operation. It distinguishes from get_task (which likely returns task details) and submit_graph (which submits a graph) through the phrase 'dependency neighborhood.' However, it does not explicitly name alternatives, so it falls short of a top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description simply states what it does without any context about scenarios, prerequisites, or cases where another tool would be more appropriate. This leaves the agent to infer usage from the tool name and siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tasksC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral aspects itself. It implies a read-only listing, but doesn't explicitly state that it's non-destructive, nor does it mention ordering, pagination, default behavior, or any side effects. The lack of detail around 'recent' and how filtering works leaves the agent guessing about the actual behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no fluff. It communicates the core purpose efficiently. However, it might be too brief to be helpful, but conciseness itself is good—there are no redundant phrases or unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and schema descriptions, the description carries the full burden of contextual completeness. It omits crucial information like sorting order, pagination, valid state values, and any limits or constraints. For a listing tool with optional parameters and no documented output schema details visible, this is insufficient for an agent to confidently decide when to call this tool and how to fill parameters correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must clarify parameter meaning. It explains that 'state' filters the results, but provides no guidance on valid state values or how 'limit' behaves (e.g., maximum, default is 100 from schema). The description fails to cover the 'limit' parameter entirely, making it inadequate for understanding parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and the resource 'tasks', differentiating it from singular operations like 'get_task'. It also mentions an optional state filter, which adds some specificity. However, the term 'recent' is vague and could refer to different ordering criteria (creation time, last update), and there's no mention of sorting or pagination, so clarity is good but not perfect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as 'get_task' or 'submit_task'. The description simply states what it does without indicating contexts, prerequisites, or when not to use it. Given the sibling tools, an agent would need to infer usage, which is insufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_artifactsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries full responsibility for behavioral disclosure. It mentions that artifacts are 'immutable' but does not state return format, pagination behavior, error handling, or any side effects. For a list operation, key behavioral details like potential large result sets or ordering are absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. The verb and resource are placed first, and every word contributes to understanding the core operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and an output schema, it is reasonably simple, but the description lacks usage guidelines and behavioral transparency (e.g., whether it returns all artifacts or paginated results). The absence of any mention of output structure or edge cases leaves an agent uncertain about what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only implies that task_id identifies the task, but does not clarify whether it is the task's ID, its format, or that it is required (though schema marks it required, but that's not in the description). The description adds minimal value beyond the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (list) and resource (artifacts) with a clear scope (committed by a task). It implicitly distinguishes from get_artifact (singular vs. plural) and list_tasks (different resource). The phrase 'immutable artifacts' adds specificity, though it could be more explicit about the distinction from get_artifact.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like get_artifact or list_tasks. The description does not mention any conditions, prerequisites, or exclusion criteria. An agent must infer the appropriate context from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_graphC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It does state that registration is atomic, which is a useful behavioral trait, but it omits other critical aspects such as side effects, permission requirements, error handling, or what the tool returns. Given the lack of annotations and the significance of atomicity, this is incomplete coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that leads with the core action and key constraint ('atomically'). Every word earns its place; there is no redundancy or filler. It is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description is extremely sparse for a tool that accepts a free-form graph object. It doesn't define the expected structure of the DAG beyond two hints, doesn't mention what the output represents, and provides no information about validations, constraints, or failure modes. An agent would struggle to construct a correct graph argument with only this description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description for the 'graph' parameter (coverage 0%), so the description must compensate. It does add some semantics by explaining how to express local refs and control dependencies, which gives meaning to the free-form object. However, this is only a partial explanation and does not cover other aspects like how to define tasks, outputs, or dependencies beyond the two hints, leaving the parameter under-specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Register') and resource ('batch DAG') and explicitly mentions atomicity, which distinguishes it from sibling tools like submit_task (which likely handles single tasks). However, it does not elaborate on what constitutes a 'batch DAG' or why it differs from other submission tools, so it's clear but not fully specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It briefly hints at graph syntax ('Local refs use @task/artifact and control deps use @task') but does not mention any selection criteria, exclusions, or scenarios where a different tool (e.g., submit_task) would be preferable. This leaves the agent to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_taskC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the full burden. 'Register' implies a write/create operation, but the description does not disclose whether submission triggers execution, is asynchronous, requires special permissions, or returns a task ID. The input-ref format note is useful but describes parameter syntax, not behavioral side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero wasted words. The purpose is front-loaded and the single additional sentence conveys actionable input-reference semantics. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  A submission tool with 8 parameters, no annotations, and only a two-sentence description is under-specified. The presence of an output schema covers return values, but the behavioral ambiguity (queue vs. execute), the relationship to submit_graph, and the command-array format are all unexplained, leaving an agent uncertain about consequences of calling it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema coverage at 0%, the description must compensate, and it partially does: the 'task:<id>/artifact:<name>' and 'artifact:<id>' guidance genuinely adds meaning for the inputs/after parameters that the schema's generic string-object type does not convey. However, the remaining six parameters (name, command, env, outputs, metadata, resources) receive no semantic elaboration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Register one task'), making the core action clear. The word 'one' implicitly distinguishes it from the sibling submit_graph, which registers a graph of tasks, though it does not explicitly name that sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is provided. The phrase 'one task' weakly implies the alternative submit_graph for multiple related tasks, but the description never names the alternative or gives the decision rule, and no guidance distinguishes it from get_task/list_tasks/cancel_task.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watch_eventsC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does add some behavior: it is a long-poll (efficient, not busy waiting) and supports reconnection using the last event ID (stateful). However, it does not disclose error handling, rate limits, or what happens on timeout. Given the lack of annotations, it is somewhat useful but still incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that packs two key pieces of information: long-polling without busy waiting, and reconnection using the last event ID. It is concise and front-loads the core purpose. However, it could be slightly more structured to elaborate on parameters while maintaining brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of three parameters with no descriptions, an output schema that is not described, and only a brief mention of reconnection, the description is insufficient for an agent to call the tool correctly. The agent knows it is a long-poll but lacks details on parameter usage, response format, or how to obtain the 'last event ID' from the output. These gaps make it incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not explicitly explain any parameters. It implicitly refers to 'last event ID' which maps to after_event_id, but it does not clarify the meaning of limit or timeout_seconds. Since schema description coverage is 0%, the description needs to compensate but only does so for one parameter indirectly. The parameter semantics are largely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool watches events via long-polling, which is a specific verb+resource. It mentions 'durable events' and 'reconnect' which gives a sense of its function. However, it does not explicitly differentiate itself from the sibling tool 'get_events', which likely provides a comparable event-fetching capability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without busy waiting' implies it is for continuous streaming compared to polling, but there is no explicit guidance on when to use this tool versus alternatives like get_events. The description does not mention when it is appropriate to call this tool over others, nor does it state any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_artifactB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions the artifact is 'immutable' and the return includes a local storage path, but it does not state whether the operation is read-only, what errors occur on invalid input, or any side effects. The description is silent on preconditions (e.g., task completion) and failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the purpose. There is no filler or redundant wording, making it highly concise and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists (which may document return fields), the description lacks essential context for correct invocation: it does not explain the meaning of logical_name, any prerequisites, or how this tool differs from list_artifacts. An agent would struggle to know what to pass for logical_name without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must clarify parameter meanings. It implies logical_name identifies a specific output of the task, but it does not explain what logical_name looks like, how to discover valid values, or the expected format for task_id. It adds minimal value beyond the parameter names themselves.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear, specific action: 'Resolve one task output to its immutable artifact record and local storage path.' It identifies the resource (task output) and the result (artifact record + path), and distinguishes from siblings like list_artifacts (which lists artifacts) and get_task (which gets a task).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. It does not mention that it is for a single artifact retrieval as opposed to list_artifacts, nor does it provide context on when it is appropriate over other task-related tools. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_taskB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. The description only lists what is returned but does not state that this operation is read-only, has no side effects, or requires specific permissions. It offers no context about side effects, rate limits, or error behavior beyond the implicit 'get' semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the primary action and the specific information returned. There is no fluff or repetition; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately summarizes the high-level return content (task state, attempts, dependencies, artifacts) for a simple get-by-ID tool. Since an output schema exists (as indicated in the context), the exact structure is presumably documented there. However, it lacks explicit guidance on when to use this tool over siblings, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate for the lack of parameter documentation. However, the description does not mention task_id at all, leaving its role to be inferred from the name. The description adds no meaning beyond what the schema already provides, and fails to compensate for the missing schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'task', and enumerates the specific data returned (state, attempts, dependencies, committed artifacts). This distinguishes it from siblings like list_tasks (which lists tasks) and get_artifact (which retrieves a specific artifact).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as list_tasks or get_artifact. It does not state prerequisites, exclusions, or conditions. The agent is left to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_taskB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does disclose a key behavioral trait: invalidation of the fencing generation, which hints at side effects beyond simple cancellation. However, it does not explain failure conditions, reversibility, permission requirements, or what happens if the task is already completed. The specific invalidation detail adds some value, but coverage remains incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no fluff. It states the verb, object, and effect immediately, making it efficient for an agent to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter, one action) and has an output schema, so return format is covered. However, it lacks usage guidance and more behavioral detail (e.g., edge cases, failure modes). For a simple cancel operation, these are moderate gaps, making it adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter documentation. It does not explain what task_id means beyond the obvious (a task identifier), nor any format, constraints, or how it maps to the action. For a single required parameter, the agent can infer meaning, but the description adds zero semantic value beyond the schema field name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the action (cancel), the target (pending or running task), and the specific side effect (invalidate its current fencing generation). This is precise and clearly distinguishes it from siblings like submit_task or retry_task, even without naming them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of conditions like 'use this to stop a task,' nor any exclusions or trade-offs compared to retry_task or get_task. The context implies cancellation is the purpose, but it doesn't help an agent decide between cancel and retry.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retry_taskA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The description discloses that this is a mutation operation ('Create'), which implies a side effect. It does not mention permissions, idempotency, or what happens to the original task. While the core mutation is clear, additional behavioral details (e.g., whether the retry replaces the original execution or adds a new one) are absent. This is a basic, but not rich, disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no filler. It front-loads the core action and condition. Every word earns its place, and it is appropriately sized for a tool with one parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retry tool with one parameter and an output schema (not shown but present), the description covers the essential purpose and condition. It does not mention side effects or prerequisites beyond the task state, but given the simple complexity and available output schema, it is largely complete. A small gap is the lack of explicit clarification on what 'new execution generation' entails (e.g., new task ID), which might be relevant to an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes only task_id with no description (0% coverage). The description does not explicitly detail the parameter, but the tool name and description (retry a task) make it clear that task_id identifies the task to retry. The meaning is strongly implied, but the description does not add explicit elaboration beyond the schema, such as expected format or constraints. Given the low schema coverage, the description partially compensates through context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Create a new execution generation') on a specific resource (a task), and narrows the scope to failed/canceled/blocked tasks. This clearly distinguishes it from siblings like submit_task (which submits a new task) and cancel_task (which cancels). The verb+resource+condition is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly defines the condition for use: 'for a failed/canceled/blocked task.' This gives clear context on when to invoke this tool. However, it does not explicitly name alternatives (e.g., submit_task for new tasks or get_task to inspect before retry), nor does it state when NOT to use it. The guidance is strong but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resource_statusA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the tool shows capacity and reservations and explicitly disclaims prediction, which is useful. However, it omits details such as whether the data is live or cached, whether any permissions are required, or any side effects. For a read-only status tool, this level of disclosure is adequate but not rich; a 3 reflects the absence of annotations without being overly punitive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that leads with the primary action ('Show configured capacity and current reservations') and then appends a clarifying limitation ('no resource prediction is performed'). It is front-loaded, efficient, and contains no filler. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no parameters and an existing output schema, the description is complete. It explains what the tool returns (capacity and reservations) and what it does not (prediction). An agent can correctly invoke the tool without further information, as there are no required arguments and the purpose is fully described. The output schema covers return details, so the description need not elaborate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so per the baseline rule for 0 parameters, the score is 4. There is nothing for the description to add about parameters, and the schema coverage is trivially 100%. The description correctly implies that no arguments are needed to invoke the tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Show') and a clear resource ('configured capacity and current reservations'), making the tool's function unmistakable. It also distinguishes itself from all siblings, which are task/graph/artifact/event oriented, by focusing on resource-level status. The purpose is precise and immediately identifiable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context that this tool reports resource capacity and reservations, and explicitly states what it does not do ('no resource prediction'). While it doesn't name alternative tools, the sibling set contains no resource-related tools, so the usage context is implicitly clear. The negative constraint 'no resource prediction' provides a boundary for when not to use it, meeting the 'clear context, no exclusions' criterion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.