Server Quality Checklist

Tool Scores

• set_project_pathA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations; description adds context about security boundaries and being the first step, but does not detail side effects, idempotency, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-loading purpose and usage. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple setter tool: explains why it's needed (security, file resolution) and gives usage instruction. Lacks return value info but acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with 'Absolute path to the project root'. Description repeats this without adding new constraints, format, or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Set the project root path for security boundaries' and identifies it as the first step, distinguishing it from sibling tools which are likely not initialization tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call this before any other tool', providing clear when-to-use guidance, but lacks when-not-to-use or alternative tool mentions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• healthA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, idempotent, and not open-world. The description adds context about the tool being 'codegraph-compatible' and details the behavior of each action (e.g., 'unreferenced functions / unused imports' for dead action). No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with bullet points for each action and sub-options. It front-loads the overall purpose and then systematically lists actions. Every sentence adds value; no wasted words. Could be slightly more concise but appropriate given the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (12 actions, 10 parameters, many sub-modes), the description is comprehensive. It covers all actions and their parameters, mentions alternatives (viz), and describes behavioral outcomes. Missing explicit return format descriptions, but overall complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage, but descriptions are terse (e.g., 'Target file path'). The tool description extensively lists parameters per action, often with specifics like 'min_grade, max_files' that are not in the schema's explicit properties (since schema uses additionalProperties true). This adds meaning beyond the schema but also introduces inconsistency since those parameters are undocumented in the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it's a 'Code-intelligence (codegraph-compatible) health and analysis facade' and lists 12 distinct actions, each with a specific verb (e.g., 'dead', 'deps', 'file'). It explicitly distinguishes from the sibling tool 'viz' by stating 'For UML diagrams... use the viz facade instead.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'Pick a capability via action' and details each action with parameters. It explicitly says when not to use this tool ('For UML diagrams... use the viz facade instead'). However, it does not guide on when to use this tool over other siblings like 'project' or 'search', though the scope is well-defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• indexA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are all false and provide no detail, so the description carries the full burden. It does this exceptionally well by separating READ-ONLY operations from 'WRITES ON-DISK INDEX' operations, disclosing that cache also exposes mutating modes, mentioning force reindexing, invalidate, and background auto-indexing, and flagging slow vs fast operations. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although long, the description is appropriately sized for a multi-action hub and is well structured with READ-ONLY and WRITES sections, bullets, and inline parameter lists. It front-loads the core concept and every section earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, seven actions, ten parameters, and no output schema, the description covers action selection, side effects, performance characteristics, and status return information well. It does not describe return shapes for most actions or error behavior, and some schema parameters are not mentioned, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so the baseline is 3, but the description adds real value by mapping parameters per action, stating defaults like mode (default search/stats), and naming additional params absent from the schema such as kind, enable, watch, paths, backend, max_files, max_edges, and include_docs. Some schema params like scope, symbol, language, and output_format are left unmapped, preventing a perfect score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a code-intelligence index lifecycle hub and enumerates seven distinct capabilities via the action parameter, with concrete verbs like check, query, build, sync, and materialize. It does not explicitly differentiate from sibling tools such as search, nav, or health, so it misses the top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives strong in-tool routing guidance, such as 'use when index is absent or corrupt' for build, 'fast; use after editing files' for sync, and explicit read-only vs mutating modes for cache. It lacks explicit when-not-to-use guidance against sibling tools, but within the tool the action selection conditions are clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vizA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only, idempotent, non-destructive behavior. The description adds meaningful behavioral detail beyond those annotations: similarity defaults to a summary map with no code bodies, include_bodies=true is required for snippets, and the knowledge action exports a materialized graph for Sigma.js/Graphology. This goes beyond what the structured annotations alone provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a one-sentence summary of the tool's role, then organized into four compact action bullets. The length is justified by the multi-capability facade nature of the tool. There is no filler or tautology; every clause adds routing or parameter information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex tool with four action modes, 14+ parameters, an open additionalProperties schema, and no output schema. The description covers action selection, per-action parameter sets, and at least one key output behavior (similarity's summary map default). Still, it does not describe output shapes or formats for UML, graph, or knowledge actions, which an agent would need to confidently interpret results. That gap prevents a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description earns a 4 by listing action-specific parameters that are not present in the schema (e.g., diagram, source, target, max_depth, visualization_format, include_bodies, export_format, lod, focus), which is critical because additionalProperties is true. It also embeds defaults and behavioral semantics like min_lines defaulting to 5, max_groups defaulting to 20, and include_bodies controlling snippet inclusion. However, many listed parameters are not individually explained, so it isn't a 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific statement of purpose: 'Code-intelligence (codegraph-compatible) visualization and similarity facade.' It enumerates four distinct capabilities (UML, graph visualization, similarity, knowledge graph) and maps each to concrete action values. This clearly distinguishes the tool from search/nav/structure siblings by framing it as a visualization and similarity facade.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides strong internal routing guidance: 'Pick a capability via action' and then itemizes exactly which parameters apply to which action. However, it does not explicitly contrast this tool with sibling tools like search or structure, nor state when not to use viz. The usage context is clear, but exclusions and external alternatives are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• editA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds value beyond annotations by explaining the mixed nature of the actions (some read-only, some non-writing). Annotations set readOnlyHint=false overall, but the description clarifies which specific actions are safe. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well-structured, with action definitions clearly separated. It front-loads the overall purpose and uses bullet-style formatting. For the complexity of 8 actions and 11 parameters, it is appropriately sized, though slightly verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complex multi-action facade with no output schema, the description covers each action's purpose, parameters, and behavior comprehensively. It notes data formats and override behaviors, making it complete for the tool's scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description significantly enriches the input schema by detailing required parameters per action (e.g., 'symbol* (required)' for guard) and adds context on how parameters like 'mode' or 'output_format' are used. Schema coverage is 100%, but the description provides the 'why' behind parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool as a 'safety and change-management facade' for code intelligence, listing 8 distinct actions with specific verbs and resources (e.g., 'action=safe — pre-edit safety gate'). It distinguishes itself from sibling tools like 'health' or 'index' by being a composite facade covering multiple capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance for each action, stating when to use them (e.g., 'Call after every non-trivial edit' for impact) and notes that some actions are read-only while others suggest changes. However, it does not explicitly state when not to use this tool in favor of alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• projectA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses behavioral traits beyond annotations: it separates actions into read-only and may-write categories, indicates that overview is ideal for first use, and advises using a different tool for index lifecycle. Annotations (readOnlyHint=false) are consistent with the mixed read/write nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear overview followed by grouped actions in bullet points. It is front-loaded with the purpose and every sentence adds value. No unnecessary text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 10 parameters and no output schema, the description covers all actions, their parameters, usage hints, and cross-tool guidance. It is fully self-contained for an agent to understand when and how to invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, the description adds value by grouping parameters per action and explaining their role in context (e.g., 'Params: path, extensions, limit, format' for files). This provides semantic context beyond the schema's property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool as a 'project-intelligence hub' with specific capabilities. It enumerates all actions and distinguishes from the sibling tool 'index' for lifecycle operations. The purpose is specific and not a tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance for when to use each action, e.g., 'Best first call on an unfamiliar repo' for overview, and recommends the 'index' facade for index lifecycle. However, it does not compare against all siblings like 'edit' or 'nav'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• structureA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and idempotentHint=true. The description adds rich behavioral context: notes about signatures being lightweight, sitemap taking directory not file_path, and language auto-detection. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with bullet points and clear action listings. It is long but each sentence adds value. Slight redundancy (e.g., repeating 'codegraph_* equivalent') but overall efficient for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 11 parameters, 9 actions, no output schema, and no annotations beyond safety hints, the description is remarkably complete. It covers all actions, parameter details, and usage tips, leaving no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds significant meaning beyond parameter names. For each action, it explains how parameters are used (e.g., for signatures: 'Params: file_path[, language] (language auto-detected from file extension when omitted)'). Also clarifies that directory is optional for sitemap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a multi-action facade for code intelligence. It enumerates each action (outline, analyze, signatures, etc.) with specific verbs and resources, distinguishing it from sibling tools like edit or search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance for when to use each action (e.g., 'Use FIRST for large files (>500 lines) to pick methods by name, then action=read'). It does not cover when to avoid this tool in favor of siblings, but the guidance is clear for the tool's own actions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• navA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark it as read-only and idempotent. The description adds significant behavioral details beyond annotations, such as risk score computation from production edges only, test bucket always present, and nested tree capabilities.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While well-structured with bullet points, the description is very long (over 400 words). It could be more concise by combining similar actions or reducing redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 14 actions and 10 parameters, the description covers all actions, parameter combinations, usage patterns, and edge cases. No output schema, but return values are described textually. Complete given complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema provides minimal descriptions, but the tool's description adds rich detail for each action's parameters, including dependencies, defaults, and behavior differences (e.g., scope=point vs graph). This greatly exceeds the schema's coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a code-intelligence navigation facade covering multiple codegraph actions. It distinguishes itself from sibling tools like search and structure by positioning itself as the starting point for code understanding tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to start with action=context, then use tree actions. Warns against looping search or per-symbol navigate, and tells when to use alternatives like callee_tree over looping callees.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only, idempotent, non-destructive. The description adds behavioral context: explains each action's output (e.g., subscribe returns {sub_id, resource_uri}), the chain DSL syntax, and select DSL syntax. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-organized with bullet points for each action. It front-loads the overall purpose and action list. However, some redundancy (e.g., repeating 'Params:' for each action) could be trimmed. Still clear and usable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (11 parameters, 9 actions) and no output schema, the description is thoroughly complete: it covers all actions, their parameters, and expected outputs (e.g., subscribe returns sub_id and resource_uri). No gaps for an AI agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% description coverage, but the description adds significant context: e.g., for 'chain' action, explains the query parameter format with examples; for 'batch', explains the 'queries' array structure; for 'select', explains the CSS-selector-like syntax. This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a 'Code-intelligence (codegraph-compatible) search facade' covering multiple search capabilities, and lists all 9 actions with specific purposes. It distinguishes itself from siblings (e.g., edit, nav) by defining its search role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use each action (e.g., 'action=symbol for fast "where is X defined"', 'action=content for ripgrep text/regex search') and contrasts similar actions (e.g., 'action=query is NOT the same as symbol'). Offers enough detail for an agent to select the right action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.