Server Quality Checklist

Tool Scores

• list_dirC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It states what the tool supports but not how it behaves in practice—such as default recursion behavior, whether hidden files are excluded by default, error handling, or what the response contains. The read-only nature is implied by the name but never stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with the core purpose. The second sentence is a compact feature list. It is slightly redundant with the schema's parameter descriptions but does not waste words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return values do not need to be described. However, the description lacks usage context, such as how recursion depth interacts with recursive, whether glob matching is relative to the listed directory, and when this tool should be preferred over the glob sibling. It is adequate for a basic listing tool but leaves some call-related ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents path, max_depth, recursive, glob_pattern, and include_hidden. The description's feature list roughly maps to these parameters but adds no new semantic depth beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: "List directory contents." It also enumerates supported filtering capabilities, which distinguishes it from file content operations like read_file and write_file. However, it does not explicitly differentiate itself from the sibling glob tool, which also deals with path-matching filters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to choose list_dir over glob, search_files, or other siblings. The description only lists features; it does not state scenarios, exclusions, or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_filesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It usefully discloses the ripgrep dependency and supported features, but it does not explain behavior such as what happens if ripgrep is missing, search recursion scope, or how results are returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, front-loaded with the core purpose, and each sentence contributes meaningful information: what it does, the system prerequisite, and the supported options. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate given the rich schema and existing output schema, but it omits practical context like the default search path, how the path relates to a root directory, and when to prefer this over glob. It covers the basics but leaves room for more operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema descriptions already cover all parameters (100% coverage), and the description mostly summarizes those same features (regex patterns, glob filtering, case sensitivity, context lines, result limiting). It adds no deeper meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches file contents using ripgrep, identifying the verb, resource, and implementation. It does not explicitly contrast with sibling tools like glob, but 'file contents' helps distinguish it from file-name-based search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (searching file contents) and lists supported capabilities, but it does not explicitly state when not to use it or how it compares to alternatives like glob or list_dir. Usage context is inferable rather than directly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• globA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It adds useful details such as using pathlib's glob matching and supporting recursive patterns, hidden-file filtering, and result limiting. It does not disclose edge-case behaviors like symlink handling or error conditions, so it is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two compact sentences with no filler. The core purpose is front-loaded, and the implementation detail and feature highlights are presented efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the fully documented input schema and the presence of an output schema, the description is largely complete for a glob-finding tool. It adds the key implementation note about pathlib glob semantics, though a small amount of additional usage guidance relative to sibling tools would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for parameters, so the schema already documents path, pattern, recursive, max_results, and include_hidden. The description only lightly reinforces these concepts, which meets the baseline but adds no significant new parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool finds files matching a glob pattern, using a specific verb and resource. It distinguishes itself from file content tools and directory listing tools, though it does not explicitly contrast with the sibling search_files tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when a glob pattern is needed to locate files. However, it does not explicitly state when to prefer an alternative such as search_files or list_dir, leaving some routing decisions to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden, and it does well by disclosing atomic temp-file-plus-rename behavior, default parent-directory creation, and configurable encoding. It does not mention overwrite behavior on an existing file or failure modes, so it is informative but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, with the core action front-loaded and every sentence adding relevant behavior. No repetition of schema boilerplate and no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the nested request schema, complete parameter descriptions, and presence of an output schema, the description provides enough for a capable agent to invoke the tool correctly. It could be more complete with an explicit statement about replacing existing files and when to use patch_file instead, which keeps it from a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents path, content, atomic, encoding, and create_dirs. The description adds the rationale 'to prevent partial writes' and confirms defaults, but most parameter meaning is already carried by the schema; this is a solid baseline rather than a standout contribution.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb and resource: 'Write content to a file atomically,' which makes the tool's core purpose immediately identifiable. It adds useful behavioral details (atomic writes, parent dir creation) but does not explicitly differentiate from patch_file, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to prefer write_file over the sibling patch_file, or what conditions make full-file writes appropriate. The description states behavior rather than selection criteria or exclusions, leaving the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• patch_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It proactively discloses three important traits: all occurrences are replaced, the operation uses atomic write, and the tool returns the number of replacements made. It does not spell out no-match or file-not-found behavior, but the core side effects and return signal are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, front-loaded with the core action followed by the behavioral guarantee and return value. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations, the description covers the main behavioral contract and return semantics, and the input schema supplies the nested request fields and encoding default. The main gaps are explicit failure modes and when-not-to-use guidance, which are useful but not critical for invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining that the patch targets every occurrence of old_str, not just the first one, which is a semantic guarantee not explicit in the schema. Path and encoding are already fully described in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Apply a targeted patch to a file,' identifying a specific verb, target, and scope. It further defines the behavior as replacing all occurrences of old_str with new_str, which clearly distinguishes it from write_file (full-file overwrite) and read_file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied by the phrase 'targeted patch' and the find/replace semantics, but the description never explicitly says when to prefer patch_file over write_file or states any exclusions. An agent can infer the use case, but the guidance is not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full behavioral disclosure burden. It does this well by explaining return behavior for text vs binary files, mentioning the `is_binary=True` flag, and noting configurable size limits and encoding. It does not cover error cases like missing files or permission issues, but the core behavioral traits are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no redundant words. The primary action is front-loaded, followed by essential return-format details and configurable options. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema (per context signals), the description adequately covers what the agent needs: return text, binary representation, and configurable limits/encoding. It could mention error behavior but is otherwise complete for a read-only file operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100%, so the schema already documents path, encoding, and max_size thoroughly. The description's reference to 'configurable size limits and encoding' merely echoes the schema without adding deeper meaning, keeping this at the baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the exact action ('Read a file from the filesystem') with a specific verb and resource. It is clearly distinct from sibling tools like list_dir (directory listing), search_files (searching), and write_file/patch_file (modification).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the tool's use case clear: retrieve file content from the filesystem. While it doesn't explicitly mention alternatives or exclusions, the context is direct enough that an agent can infer when to use read_file versus siblings. It lacks explicit 'use this instead of...' guidance, so it doesn't earn a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.