Server Quality Checklist

Tool Scores

• graphex_queryD
  1.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It doesn't mention whether the operation is read-only, what kind of output to expect, or any side effects. 'Tier-filtered' hints at a behavior but leaves it unexplained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely short, but this is under-specification rather than conciseness. A single vague phrase conveys almost no actionable information and does not earn its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a query tool with two optional parameters and no output schema, this description provides nowhere near enough context to invoke it correctly. It fails to explain the purpose, parameters, or expected behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema defines two parameters (budget and pattern) with 0% description coverage, and the tool description says nothing about them. Agents are left with no idea what these parameters control or how to use them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Project knowledge-graph context' but doesn't specify the action or resource. 'Tier-filtered' is ambiguous, and there's no differentiation from siblings like graphex_glossary or graphex_conventions. An agent cannot tell what this tool actually does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus any of the eight siblings. No context, no exclusions, no alternatives. The description offers zero help in selecting the right tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• propose_changesetC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. The verb 'promote' implies a state-changing operation, but the description never states what actually happens to the timeline entries (are they consumed, deleted, marked?), whether changes are reversible, whether authorization is required, or what the tool returns. Consequences of the mutation are entirely undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The single sentence is grammatically efficient and front-loads the core action with no wasted words. However, it is under-specified: for a tool with an undocumented required parameter, this brevity crosses from conciseness into incompleteness. It reads cleanly but does too little documentation work.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description is the only documentation for a state-changing tool. It fails to explain the `message`/`intent` parameters, the side effects of promotion, or what a changeset is. Notably the JSON response wasn't part of this review, but without a schema defining the return, an agent cannot know what to expect after invoking it. Inadequate for a mutation with undocumented parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not compensate at all — it never mentions `message` (required) or `intent` (optional). The agent has no idea what these values should contain (e.g., is `message` a commit note? does `intent` describe the change's purpose?). With zero coverage and zero description help, both parameters are effectively opaque.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: 'Promote pending timeline entries into a changeset.' The verb+resource is clear, but it does not distinguish itself from its 'propose_' siblings (propose_graph_update, propose_belief). Also relies on undefined domain jargon ('timeline entries', 'changeset') which assumes context. Clear but no sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus propose_graph_update or propose_belief. There is no stated condition, prerequisite, or when-not-to-use guidance. The reader can only infer usage from the name, which is not sufficient routing information among three similarly-named siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• propose_graph_updateC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden. It discloses that the proposal lands for human approval (a state-change), which is useful. However, it does not explain what happens after proposal, whether it can be undone, permissions required, or what the response looks like. For a mutating tool with no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with no fluff. It front-loads the action and outcome. However, it is almost too brief—it sacrifices needed behavioral context for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema, no annotations, and no parameter documentation, this description is insufficient for an agent to call the tool correctly. The tool involves proposing a node with multiple parameters, yet only the high-level purpose is stated. The agent would need to infer parameter semantics from names alone, which is risky. Sibling differentiation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions none of the four parameters (name, tier, node_type, description). The description gives no meaning beyond what the schema types imply (strings). With zero coverage and no parameter details, an agent is left guessing about required values, constraints, or relationships.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb (Propose) and resource (graph node), with a distinctive outcome (lands as Proposed for human approval). It does not explicitly differentiate from sibling tools like propose_changeset or propose_belief, though the phrase 'graph node' hints at a different domain. It is not a tautology but could be more specific about what 'graph node' entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus alternatives. It implies a workflow of proposing for human approval, but does not mention exclusions or alternatives. Sibling tools like propose_changeset, propose_belief, and reaffirm_belief exist in the same namespace, and without any usage guidance an agent might select the wrong one.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graphex_glossaryC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full responsibility for behavioral disclosure. It only states the action with zero additional context about side effects, read-only nature, error behavior, or output characteristics, failing to inform the agent beyond the basic verb.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the core action without any extraneous words. Every character earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks essential details such as return format, behavior when the optional parameter is omitted, or any constraints. Given the absence of an output schema and the sparse parameter info, the description is inadequate for an agent to confidently invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one optional parameter 'term' with no description, and the tool description does not explain its meaning or behavior (e.g., what happens when omitted). With 0% schema description coverage, the description must compensate but does not, leaving parameter semantics entirely undefined.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'look up' and a clear resource 'domain glossary terms', making the tool's purpose unambiguous. It is readily distinguishable from sibling tools like graphex_query or graphex_conventions without needing their schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, and no exclusions or prerequisites are mentioned. While the purpose implies its use, there is no explicit context for selection, leaving agents to infer conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graphex_conventionsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only says 'List applicable coding conventions' with no indication of the output format, whether it returns a list, or any side effects (none expected). It does not say what 'applicable' means or what constitutes a convention. This is minimal and not transparent beyond the action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the action and object. There is no fluff, and it is appropriately sized for a tool that takes no parameters. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless tool with no output schema, the description is minimally adequate. It tells the agent what it does. However, it leaves open questions about the nature of the output (e.g., plain text vs. structured list) and what 'applicable' means in the current context. Given that there is no output schema or annotations, a bit more detail—such as 'returns a list of convention identifiers'—would improve completeness. Still, it is not severely lacking.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters and the schema is empty (100% coverage trivially). According to the scoring guidelines, a baseline of 4 is appropriate for 0-parameter tools. The description does not need to explain parameters because none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and a clear resource ('applicable coding conventions'). It is unambiguous and does not conflate with siblings like graphex_query (query) or graphex_glossary (glossary). However, it does not explicitly differentiate from them, though the resource itself is distinct enough.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It merely states the action without context such as 'use this when you need the list before proposing changes' or any exclusions. An agent would have to infer when it is applicable, which is a gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• propose_beliefB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does disclose meaningful behavior: the record lands as 'Proposed for human approval' and is 'staleness-checked against history from the moment it exists.' This adds real value. However, it does not state the return/response shape, failure behavior, or any side effects beyond the proposal status — a notable gap for a write operation with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences with the core purpose front-loaded, followed by format specs and the constraint. The format specifications are packed tightly and each sentence earns its place. Slightly heavy on the type-notation details, but nothing is wasted — appropriately concise for the information conveyed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given five parameters, no annotations, and no output schema, the description covers the critical input semantics and constraint well but leaves gaps: 'claim' and 'tier' are unexplained, the response/proposal-approval flow is only hinted at, and the 'staleness' mechanism is mentioned without elaboration. It is workable for a competent agent but not fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate — and it largely does. It explains evidence format ('path:start-end', 1-based inclusive lines), paths ('repo-relative paths or globs'), symbols ('path:Name'), and the mutual-requirement constraint. This adds substantial meaning beyond the bare schema. However, the 'claim' and 'tier' parameters receive no explanation, which keeps this from a 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Record an evidence-anchored belief about the code') and adds a distinguishing detail: the belief 'lands as Proposed for human approval'. This differentiates it from siblings like reaffirm_belief (which presumably updates an existing belief) and propose_changeset, but it does not explicitly name the sibling it is not, so a small inference gap remains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit when-to-use vs when-not-to guidance and names no alternative tool. While 'At least one of evidence/paths/symbols is required' clarifies a precondition, it never routes the agent to reaffirm_belief for updating existing beliefs or explains when proposal vs. direct record is appropriate. The contrast with reaffirm_belief is left entirely implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graphex_statusA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It mentions 'statistics' but does not explicitly state that it is a read-only operation, does not describe output format, potential errors, or any side effects. The description is too thin to give an agent confidence about what to expect beyond the basic counts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is minimal—a single colon-separated phrase—with no wasted words. It is front-loaded with the core idea and is appropriately sized for a tool with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no annotations, no output schema), the description is nearly complete. It tells an agent what the tool does and what it returns conceptually (node/edge counts). However, it omits details like whether the counts reflect the entire graph or a current state, and it does not mention if there is any filtering or scope. Still, for a trivial status tool, the coverage is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline is 4 per the rubric. The schema is empty, and the description adds no parameter information, but none is needed. The description adequately communicates that no inputs are required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Graph statistics' with the detail 'node/edge counts.' It clearly identifies what the tool does and is distinct from sibling tools like graphex_query or graphex_glossary, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus siblings. The description does not mention alternatives, prerequisites, or context—it simply states what it does. For a tool that provides stats, an agent might need to know when to call it instead of a query, but no such direction is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• report_costA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that reports accumulate (additive behavior) and clarifies the unit of cost_microdollars. However, it does not mention whether the operation is a write or if it has side effects beyond attribution, nor does it describe error behavior or reversibility. It adds some context but not comprehensive transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose, and includes crucial unit clarification. No extraneous words. It could be more structured (e.g., separating parameter notes), but it is efficient and well-ordered.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple reporting tool, the description provides key facts: the source of the changeset id and the accumulation behavior. However, with no annotations and no output schema, it omits details like what happens on success/failure, whether all token fields are required together, or how to handle partial reports. The absence of any explanation of the remaining numeric fields leaves gaps for an agent trying to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all six parameters. It only explains changeset (source of value) and cost_microdollars (unit). The other parameters (model, input_tokens, cached_tokens, output_tokens) are left to their names, which are not fully self-explanatory (e.g., what counts as cached_tokens?). This falls short of adequate compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Attribute') and a clear resource ('token/compute usage to a changeset'), and precisely identifies the source of the changeset id ('returned by propose_changeset'). This leaves no doubt what the tool does and distinguishes it from sibling tools like propose_changeset or propose_graph_update.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives a clear prerequisite: the changeset must be the id returned by propose_changeset, which tells the agent when this tool is appropriate. It also states that reports accumulate, implying multiple calls are allowed. However, it does not explicitly list when NOT to use it or mention alternative tools for cost tracking, so it's slightly short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reaffirm_beliefA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that it re-anchors at the current changeset, that new evidence replaces old ones, and that it rejects invalidated beliefs. It doesn't state side effects like whether it creates a new changeset or overwrites prior anchors, but the core behavior is adequately transparent for a belief-management operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The purpose is front-loaded, followed by a clarifying example and a crisp restriction. Every sentence carries essential information, making it efficient and easily parseable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two simple parameters, no output schema, and no nested objects, the description covers the essential aspects: action, target, optional behavior, and an explicit exclusion. It doesn't address edge cases like invalid belief IDs or failure modes, but those are not critical given the tool's simplicity and the presence of sibling tools for broader context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50% (belief has a schema description, evidence does not). The description adds crucial meaning by explaining that 'evidence' is optional and that new evidence spans replace old ones, compensating for the schema's lack of description on that parameter. The belief parameter is reinforced by the overall context, though it doesn't add new syntax-level detail beyond 'id or exact claim'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Re-anchor a belief') on a specific resource ('belief') and gives a concrete use case (stale-candidate whose claim still holds). It also contrasts with an alternative ('propose a new belief... instead'), which distinguishes it from siblings like propose_belief without needing to inspect schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use it ('after you have re-verified it against the code'), when not allowed ('Not allowed for invalidated beliefs'), and points to the alternative ('propose a new belief with current evidence instead'). This gives clear, actionable guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.