Server Quality Checklist

Tool Scores

• execute_toolA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the basic behavior (executes a named tool with parameters), but does not disclose that executing arbitrary tools may have side effects, require permissions, or fail. No details are given about error handling, output, or consequential actions, leaving the agent without critical safety context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first precisely states the tool's function, the second provides essential workflow guidance. Every sentence earns its place, and the core purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential use case—execute a named tool with params—and mentions the need to discover tools via search_tools. However, it omits guidance on using get_tool_schema to validate parameters, what happens on invalid tool names or schemas, and whether execution results are returned. Given the tool's role as a generic dispatcher, these are meaningful gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters meaningfully. The description adds no extra semantic detail beyond reinforcing that parameters are passed to the named tool. According to the baseline for high schema coverage, a 3 is appropriate because the description does not need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('execute'), a resource ('a specific tool by name'), and the action taken with params. It clearly distinguishes the tool from siblings by instructing the agent to use search_tools first for discovery, positioning execute_tool as the execution step in the workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs the agent to use search_tools first, establishing a clear precondition and workflow. It does not explicitly mention get_tool_schema or state when not to use execute_tool, but the 'search first' guidance provides clear context for the intended usage sequence.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tool_schemaA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. 'Get' implies a read-only operation, and 'complete or filtered' adds some scoping detail. However, it does not explicitly state that the tool does not execute anything, does not require special auth, or describe the return format, leaving some room for ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the core action first and the usage condition second. There is no fluff or redundant detail, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple schema-retrieval tool with fully documented parameters, the description is largely sufficient. It does not describe the exact return structure, but the tool's name and purpose make the return type fairly predictable. It also does not mention using search_tools for unknown tools, though the sibling context partially covers this.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both tool_name and query. The description adds the 'complete or filtered' concept, which loosely maps to the query parameter, but does not provide new parameter-level meaning beyond what the schema already offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets a complete or filtered schema for a specific tool, which is a specific verb plus resource. It does not explicitly name or contrast with siblings like search_tools or execute_tool, though the 'known tool' phrasing hints at differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage condition: 'Use this when you need detailed schema information for a known tool.' It does not explicitly mention alternatives or exclusions, but the context of a known tool implies this is not for discovery or execution.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_toolsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It does state that the tool returns matching tools and relevant schema fields based on the query, which is the core behavior. However, it does not explain relevance ranking, result limits, or behavior when no matches are found—though these are partially inferable from the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The first sentence states scope and return value, and the second provides the intended use case. Every word contributes to the agent's understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a relatively simple meta-search utility with no output schema, so the description must convey what results look like; it does so by saying it returns matching tools with relevant schema fields. It also communicates the size of the searchable space (174 tools) and the intended workflow. A small gap is that it does not describe the shape of the returned tool objects, but the core invocation context is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all five parameters thoroughly. The description adds only a high-level statement that results are based on the query, which is useful but not necessary. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search through') and a specific resource ('174 available SOR CRUD tools'), then states the output: matching tools with relevant schema fields. It clearly distinguishes itself from siblings by positioning itself as the discovery tool, while execute_tool and get_tool_schema serve execution and schema retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to discover which tool to use for your task,' which gives the agent a clear trigger condition. It does not explicitly name alternatives or say when not to use it, but the discovery framing is sufficient context given the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.