Server Quality Checklist

Tool Scores

• create_agentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the disclosure burden. It discloses the side effect ('writes the in-memory process table'), notes that it does not execute a goal, and mentions the default model. It could add more about process-table volatility or idempotency, but the key behavioral traits are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences front-load the purpose, then cover return value, side effect, and default model. There is no filler, and the most decision-relevant information appears early.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers what the tool returns, what it does not do, and its side effect, while the output schema and fully documented parameters fill remaining details. It is sufficient for an agent to invoke the tool correctly, though it does not explicitly route to create_workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description echoes intent, USD budget, capabilities, and default model but adds no meaning beyond what the input schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific operation: 'Create an agent process with intent, USD budget, and capabilities.' It also clarifies that the tool returns an agent_id for later run_task/list_agents and does not execute a goal, which distinguishes it from sibling execution tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clearly implies the correct workflow: create the agent first, then use the returned agent_id with run_task or list_agents. It also gives an explicit exclusion ('Does not execute a goal'), though it does not directly mention when to use create_workflow instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_metricsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden. It clearly states 'Read-only,' which is an important behavioral trait, and specifies exclusions that clarify the tool's scope beyond a simple return-value statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The primary action is front-loaded, and the usage and exclusions follow immediately, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description covers what it returns, when to use it, its read-only nature, and what it excludes. No critical information for invoking the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description need not explain parameter behavior. The baseline of 4 applies because there is nothing missing or misleading regarding parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Return runtime counters') and identifies the resource ('agents created, tasks'). It also distinguishes itself from siblings by explicitly excluding cost rows and policy events, separating it from get_cost_ledger and get_audit_log.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use for health checks,' providing a clear use case. It also notes what the tool does not include, which helps route agents away from it for cost or policy data, though it does not name the alternative tools directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_available_toolsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It explicitly states 'Read-only,' which is a meaningful behavioral trait for an agent to know. It does not cover auth or rate limits, but for a simple list operation this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with zero wasted words. The core action is front-loaded, followed by a concrete usage instruction and a read-only note. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and an output schema, the description is fully complete for an agent to select and invoke this tool correctly. It explains what it lists, why an agent would use it, and that it has no side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is 100%, so the baseline is 4. The description adds relevant context by explaining that the output is meant for the 'capabilities argument' in create_agent, which is helpful even though there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and resource ('capability names that can be granted to agents'), which clearly distinguishes it from siblings like list_agents and create_agent. The purpose is immediately understandable without needing to inspect schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool before create_agent to choose the capabilities argument, giving clear contextual guidance. It does not mention when not to use it or compare it to alternatives, but the intended workflow is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_workflowA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It clearly lists side effects: 'creates agents + tasks and marks the workflow completed', and explains default role behavior and budget sharing. It could mention reversibility or permission requirements, but the key side effects are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four short sentences with no fluff. The core purpose is front-loaded, followed by default behavior, usage guidance, and side effects. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a workflow-creation tool, the description covers purpose, defaults, side effects, and when not to use it. An output schema exists, so return value documentation is not required here. It lacks some details like idempotency or auth, but those are not essential for initial selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds meaningful context beyond the schema by explaining that 'agents' defaults to planner then worker and that 'budget_usd' is shared across roles. This helps the agent understand how the parameters relate to each other.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Create specialist agents and run a sequential multi-agent workflow') and clearly identifies the resource. It further distinguishes itself from run_task, making it easy for an agent to know this is the multi-agent orchestration tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use when a goal needs more than one agent' and 'Do not use to run a single existing agent (run_task)'. This gives both positive and negative usage conditions and names the alternative tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_audit_logA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It clearly states the tool is 'Read-only,' which is a key safety trait for an agent. It also clarifies the scope of the data ('governance allow/deny records from AgentFence') and the expected context ('after a blocked run_task'), providing meaningful behavior beyond the tool name. It does not detail retention, auth, or rate limits, but for a zero-parameter read-only tool with an output schema, this is adequate and valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with zero filler. The first sentence front-loads the core purpose and resource, the second adds usage context, read-only status, and an exclusion. Every word earns its place, and the structure makes it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has no parameters and the output shape is covered by the output schema, the description provides everything an agent needs: what the tool returns, when to use it, its read-only nature, and what it is not. The explicit use-after-blocked-run_task context ties it to the surrounding workflow, making the definition complete for a simple read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter ambiguity to resolve. With 0 parameters, the baseline is 4, and the description does not need to add parameter details. The input schema confirms no parameters, making the description complete in this regard.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Return governance allow/deny records from AgentFence.' It also explicitly names the sibling tool it is not ('Not the dollar ledger (get_cost_ledger)'), which differentiates it from a related tool. This is far beyond a tautology and clearly identifies the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Use after a blocked run_task' gives a concrete trigger condition. It also names an alternative/adjacent tool ('get_cost_ledger') and explicitly excludes it, telling the agent what not to use this tool for. This is direct and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_cost_ledgerA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states 'Read-only', which is helpful for safety assessment. It also scopes the resource to 'this process.' It does not mention potential side effects, but for a read-only ledger operation this is adequate. The statement is truthful and adds context beyond what a schema would show.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences with no filler. The core purpose is front-loaded, followed by usage guidance and sibling exclusions. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool with an output schema, the description is complete. It states what it returns, when to use it, and which siblings it is not. Nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there are no parameter semantics to explain. The baseline for no parameters is 4, and the description adds context about the tool's scope ('for this process') without being redundant.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Return') and resource ('token/$ cost ledger for this process'). It explicitly distinguishes itself from sibling tools by stating 'Not get_audit_log or get_metrics', eliminating ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance on when to use the tool: 'Use to audit spend after run_task'. It also names the alternative tools it is not, helping the agent choose correctly without needing to inspect other definitions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_agentsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by declaring 'Read-only snapshot of this process' and 'Does not create agents or start work.' This gives an agent important safety and side-effect information beyond what structured metadata would provide, though it could also mention potential limitations like pagination or empty results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences, front-loaded with the core action and scope. The usage guidance and exclusions are packed in without any redundancy, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a zero-parameter, read-only list tool. It explains what it lists, why an agent would use it, the operational scope, and the key exclusions. Since an output schema exists, the description does not need to enumerate return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing for the description to clarify. The listed fields (id, status, spend, budget, capabilities) are return values rather than inputs, but still give useful context that complements the output schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and names the exact resource ('in-process agent records') with expected fields. It also distinguishes itself from sibling tools by explicitly stating it does not create agents or start work, so an agent can clearly separate this from create_agent and run_task.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Use before run_task to discover agent_id values.' It also states what the tool does not do, naming the relevant alternatives (create_agent, run_task), which is strong routing information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_taskA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals blocking up to 120s, the error behavior for a missing agent_id, and side effects ('task row + spend'), which is unusually transparent for a tool definition.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with every sentence earning its place: purpose, timeout/error behavior, prerequisite, alternative tool, and side effects. No filler or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return value details are unnecessary. The description covers purpose, prerequisites, alternatives, timeout, failure mode, and side effects, making it complete enough for an agent to invoke correctly without additional documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both goal and agent_id accurately. The description adds some context about prerequisites and budget/policy behavior, but it does not substantially enhance the parameter-level semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Run one goal on an existing agent until completion, policy deny, or budget stop.' It clearly defines the tool's scope and termination conditions, and distinguishes it from multi-agent orchestration by naming create_workflow as the alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: use create_agent first to obtain an agent_id, and use create_workflow instead for multi-agent orchestration. It also implies this tool is for single-goal execution on an existing agent, giving an agent enough context to select it correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• x402_payment_quoteA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It clearly states the tool is offline, read-only, does not charge or call a facilitator, and returns an error object for missing services. This is unusually complete and directly addresses the key risks an agent needs to know before invoking.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero filler. The purpose is front-loaded, followed by the most important behavioral caveats and a sibling exclusion. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with a rich schema (100% coverage), an output schema, and clear behavioral caveats, this description is complete. It covers what the tool does, how it behaves (read-only, offline), error handling, and sibling differentiation. Nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the 'service' parameter at 100% coverage, so the baseline is 3. The description adds value beyond the schema by stating that missing services return an error object, which tells the agent the service must be a known paywalled service name. It also reaffirms 'named service,' reinforcing the parameter's role.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pair: 'Return an offline x402 payment quote for a named service.' It also explicitly distinguishes itself from siblings with 'Not create_agent or run_task,' so an agent can tell exactly what this tool does and what it does not.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description names two sibling tools it is not ('Not create_agent or run_task') and states 'Read-only simulation,' which implies it should be used when a quote is needed without an actual charge or facilitator call. It does not provide a full when-to-use versus alternatives matrix, but the context is clear for a simple one-parameter tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.