Server Quality Checklist

Tool Scores

• impactA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It discloses that results are grounded in resolved call relationships, includes depth and resolution semantics, mentions truncation via truncated/truncated_count, and states that results reflect the last corbel index run. This is rich, non-obvious behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but every sentence adds substantive value, including use case, output field meanings, truncation behavior, and index freshness. It is front-loaded with the core purpose; minor redundancy exists around the comparison to text search, but it is not excessive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a multi-hop reverse-call-graph traversal, no output schema, and no annotations, the description is exceptionally complete. It explains what the tool returns, how results are resolved, when truncation occurs, and what data source backs the analysis, leaving few operational ambiguities.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline of 3 applies. The description reinforces that 'name' is the starting symbol and explains why 'file' matters for disambiguation, but it does not add new parameter-level meaning beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a concrete verb and resource: 'Trace the blast radius of changing a symbol' by walking the reverse call graph across multiple hops. It clearly distinguishes itself from a shallow text search and from single-hop 'find references' tools, so an agent can tell it apart from siblings like find and get_symbol.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this tool before refactoring' and gives concrete scenarios such as changing a function's signature or behavior. It also warns against using a text search or single-hop find references for this purpose, which provides clear when-not-to-use guidance relative to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_symbolA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it does so thoroughly. It discloses that the index is built ahead of time by `corbel index` and may be stale, that `resolution` reflects index-wide lookup rather than text matching, explains the difference between `scoped` and `global-unique`, and details truncation behavior (`truncated`, `truncated_count`) and token-budget allocation. This is a very transparent description of behavior and limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and then scaffolds into disambiguation, truncation, budget, and staleness, which is logical. It is longer than average, but every sentence earns its place given there are no annotations and no output schema. A small deduction because the budget explanation is somewhat wordy and could be tightened without losing meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description is highly complete: it names the return components (definition, callers, callees, `resolation`), explains how disambiguation works, covers truncation and budget behavior, and notes index staleness. An agent has enough context to invoke the tool correctly and interpret its results without further lookup.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3, but the description adds substantial meaning beyond the schema. It explains that `file` narrows the lookup, that `line` requires `file` and is for overloaded declarations, and it provides a detailed account of how `token_budgets` is divided across matches and then between callers and callees. This goes well beyond the parameter descriptions in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Look up a single symbol by name in the local corbel index and return where it is defined (file, line, signature), everything that calls it (callers), and everything it calls (callees).' It clearly distinguishes itself from grep/text search and implies a difference from sibling `find` by noting the exact name/file/line triple needed to pin down one match. This is more than adequate for an agent to understand what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description contains explicit usage guidance: 'Use this tool when you need to jump to a function's or type's definition, inspect its signature, or see who calls it and what it calls, before editing it.' It also advises passing `line` when a name is ambiguous and references the `find` tool's output as a source for the disambiguation triple. It stops short of explicitly stating when NOT to use this tool versus `find` or `impact`, so it earns a 4 rather than a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• findA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it delivers: it discloses substring matching, ASCII-only case folding, the ranking scheme, stable ordering, truncation behavior with truncated/truncated_count, and the fact that results depend on the last corbel index run. This is unusually transparent behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence carries useful information that is not derivable from the schema or annotations. It is front-loaded with the core purpose and differentiator before diving into ranking and truncation details. It could be tightened slightly, but the density justifies the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema and no annotations, the description is remarkably complete: it specifies the match fields (name, file, line) and how they map to get_symbol's needs, explains truncation flags, covers index freshness, and clarifies what the tool does not return. An agent has enough context to invoke it correctly and interpret the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the input schema already covers all three parameters (100% coverage), the description adds substantial meaning beyond it: the hard maximum of 200 for limit, rejection behavior above it, the built-in defaults, how token_budget constrains the response, and how query matching and ranking actually behave. This exceeds the baseline 3 for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search'), a specific resource ('the local corbel index'), and a precise condition ('for when you don't know the exact symbol name to pass to get_symbol'). It clearly distinguishes itself from siblings by naming get_symbol and impact and explaining what this tool does not do.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool ('when you don't know the exact symbol name'), and explicitly defines when not to use it: 'This tool does not resolve or return call relationships — use get_symbol or impact on a specific match for that.' This gives an agent direct routing guidance against sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.