Server Quality Checklist

Tool Scores

• enable_docset_ftsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states the action and gives no indication of side effects, persistence, idempotency, required permissions, or what changes occur within the docset. This is a meaningful gap for a tool that enables a feature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear sentence with no wasted words. It is concise and front-loaded with the core action and target. It could have added more behavioral or usage context without losing conciseness, but as written it is tightly structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one well-documented parameter and no output schema, the description is minimally adequate. However, it lacks context about when enabling FTS is useful, what it changes, and whether there are prerequisites beyond obtaining the identifier. These gaps make it only partially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and there is only one parameter, so the schema already provides the semantic baseline. The property description adds the useful origin of the identifier ('from list_installed_docsets'), but the main description does not add much beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Enable full-text search') and a specific resource ('a specific docset'). It is clearly distinct from the sibling tools like list_installed_docsets or search_documentation, though it does not explicitly contrast itself with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives, such as explaining that it should be called after list_installed_docsets and before search_documentation. The parameter description hints at a prerequisite by saying the identifier comes from list_installed_docsets, but the main description provides no usage context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentationB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It does add one useful behavioral fact: 'Results are automatically truncated if they would exceed 25,000 tokens.' However, it omits other relevant behavior such as whether the search is read-only, whether FTS must be enabled first, or what the result structure looks like.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The core purpose is front-loaded, and the truncation warning is a concise, valuable behavioral detail that earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The schema covers all parameters, and the truncation behavior is mentioned, which helps invocation. However, there is no output schema and no mention of the expected result shape, the relationship to FTS-enabled docsets, or when to prefer sibling tools, leaving moderate gaps for a tool with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description mentions 'docset identifiers and snippets' but does not add meaningful semantics beyond what the schema already provides for query, max_results, search_snippets, or docset_identifiers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear verb and object: 'Search for documentation across docset identifiers and snippets.' It is distinct from siblings like list_installed_docsets and load_documentation_page because it clearly names the search action, though it does not explicitly contrast itself with those alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus load_documentation_page or enable_docset_fts. The description neither states the appropriate context for searching documentation nor excludes alternative tools, leaving the agent to infer when this tool is the right choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_installed_docsetsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It discloses two important behaviors: an empty list is returned when no docsets are installed, and results are auto-truncated at 25,000 tokens. This is solid context for a simple read-only list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences with no filler. The main action is front-loaded, and the edge-case behaviors are stated efficiently. Every sentence contributes useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with no output schema, the description covers the essential return behaviors: empty-list handling and truncation. It does not describe the exact shape of each docset item, but this is a minor gap given the simplicity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter semantics burden. The description appropriately does not invent parameter details, and no additional meaning is needed beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair ('List all installed documentation sets in Dash') that clearly states what the tool does. It is immediately distinguishable from sibling tools like search_documentation and load_documentation_page.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this is the tool for enumerating installed docsets, and the sibling names suggest alternatives for searching and loading pages. However, it does not explicitly state when to prefer this tool over the siblings or mention any prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• load_documentation_pageA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It clearly states the output behavior: page content returned as plain text with markdown-style links. 'Load' and 'Returns' imply a read-only operation, and there are no destructive or side-effect behaviors that need disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences deliver the purpose, input source, and output format with no filler. Every clause earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description is complete: it explains where the input comes from, what the tool does, and what the caller will receive. No critical operational detail is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameter is already fully documented in the schema. The description reaffirms that load_url must come from search_documentation but does not add new validation or format details beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Load'), a specific resource ('documentation page'), and the exact source of the input ('load_url returned by search_documentation'). It clearly differentiates this tool from search_documentation and the other siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly frames usage as a follow-up to search_documentation, establishing a clear search-then-load workflow. It does not explicitly discuss when not to use list_installed_docsets or enable_docset_fts, but the referenced workflow provides sufficient context for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.