Server Quality Checklist

Tool Scores

• forget_memoryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states that the tool deletes a memory, making destruction obvious, but it does not mention whether deletion is permanent, reversible, or scoped in any way. The description lacks depth about consequences beyond the basic verb.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: a single leading sentence followed by a compact argument list. Every word earns its place, and the action is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter delete, the description provides the core operation and parameter meaning. However, the lack of any behavioral caveats (e.g., irreversibility) and the absence of guidance on sourcing a memory ID make it only partially complete. The presence of an output schema lightens the need to describe return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. 'memory_id: The ID of the memory to delete' adds only marginal meaning over the parameter name and integer type; it does not explain where the ID comes from (e.g., recall_memories) or what happens if an invalid or nonexistent ID is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Delete a memory' — a specific verb and resource that clearly identifies the operation. It is distinct from sibling tools like recall_memories, save_memory, and update_memory, though it does not explicitly name them as alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'that's no longer relevant' provides a condition for when to use the tool. However, it does not explicitly state when not to use it or mention alternatives such as update_memory for memories that still matter but need modification.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_statsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. 'Get' implies a read-only, non-destructive operation, which is helpful, but the description does not disclose what kind of statistics are returned or how this behaves relative to memory retrieval. The output schema may cover return shape, but the description itself adds limited behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences and front-loads the core action. The second sentence adds a usage cue, though it is somewhat generic and could be sharpened. No wasted words, but the second sentence is marginal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter tool with an output schema, the description is mostly complete. It tells the agent why to call it and what it does. The main gap is the lack of explicit guidance on how this differs from recall_memories, which could lead to the wrong choice when the agent wants actual memory contents.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are not a challenge. The schema already covers 100% of the parameter surface, and the description correctly avoids inventing parameter details. Baseline 4 is appropriate for a no-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource ('Get statistics about stored memories') and implies a distinction from content retrieval tools like recall_memories. However, it stops short of explicitly differentiating itself from siblings: 'understand what's in your knowledge base' could be interpreted as retrieving actual memory contents rather than aggregate stats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives a clear calling context: use this when you need an understanding of the knowledge base. But it does not state when not to use it or name alternatives such as recall_memories for detailed content, leaving some routing judgment to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_memoryA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavior disclosure. It explains the merge-like behavior of leaving fields empty to keep existing values, but it does not disclose what happens on invalid memory_id, whether changes are reversible, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-sentence purpose, a one-sentence usage rule, then a clear parameter list. Every line earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple four-parameter tool with an output schema, the description covers purpose, usage, and parameter semantics adequately. The main gaps are behavioral details, such as missing-memory handling and how to clear a field, but these are not critical for basic correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates well by explaining each parameter beyond the schema's type/title/default fields. 'Leave empty to keep existing' adds real semantic value, though it leaves unspecified how one would intentionally clear a field.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action, 'Update an existing memory,' identifying both the verb and resource. The use case in the next sentence helps distinguish this from save_memory and recall_memories, though it does not explicitly name the alternative tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear guidance on when to use the tool: when a previous memory is outdated or needs correction. It does not mention exclusions or explicitly compare with save_memory, but the provided context is sufficient for most decisions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_memoryA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does disclose that the write is persistent and that the tool stores knowledge, but it does not describe duplicate handling, overwrite behavior, or effects on existing memories. This is a noticeable gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well organized: purpose, trigger examples, then argument details. It is front-loaded and every section earns its place without filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers what the tool does, when to use it, and all parameters, while return behavior is left to the output schema. It is slightly incomplete because it does not address existing memory entries or differentiate sibling update/forget scenarios, but for a simple save-style tool this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though the schema has no parameter descriptions, the Args section documents all three parameters: content guidance, valid category values, and tag format with an example. This fully compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Save a piece of knowledge to persistent memory.' This clearly communicates what the tool does and distinguishes it from sibling memory tools like recall_memories and forget_memory.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit trigger: 'Call this when you learn something worth remembering,' followed by concrete categories. It does not mention when to avoid using it or when to use update_memory or forget_memory instead, so it stops short of full decision-rule coverage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watch_fileA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full behavioral disclosure burden. It does a solid job by explaining stop conditions (pattern found, or any change when pattern is empty), timeout default and max, polling interval, and tail_lines returned. It does not explicitly describe a timeout error scenario or how it handles a non-existent file, which are mild gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is wll-organized: a one-sentence purpose, a brief use-case list, three illustrative examples, and a compact Args block. Every sentence contributes useful information, and the layout makes defaults and constraints easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, use cases, and each parameter with defaults/limits, while the output schema covers return values. It does not explicitly state timeout/error behavior or relationship to sibling monitoring tools, but for a file-watch tool with strong parameter documentation it is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does fully. The Args section explains each parameter: path, regex pattern semantics and empty-pattern behavior, timeout defaults/max, interval defaults/min, and the purpose of tail_lines. This adds materially beyond the schema's types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Watch a file for changes or a specific pattern', naming a specific verb and resource, and the examples clarify the intended file-focused scope. It does not explicitly contrast with sibling tools like monitor_command or poll_url, so differentiation is implicit rather than named.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'Useful for monitoring log files, build outputs, job status files' and gives three concrete usage examples, which gives the agent clear context for when to call it. However, it does not state when not to use it or name alternatives such as poll_url for URLs, so it lacks explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• monitor_commandA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, and it delivers: recurring execution, early stopping via regex, change detection, exit-code stopping, timeouts, intervals, and snapshot dropping. Minor gaps such as error handling for failed commands and whether stderr is captured prevent a perfect score, but the core behavior is clearly portrayed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description opens with a one-sentence purpose, follows with an emphasized usage guideline, then gives four helpful examples, and closes with per-parameter explanations. Every section earns its place, and the structure is skimmable and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description covers all seven parameters, gives use cases, and explains stopping behavior thoroughly. Since an output schema exists, omitting return-format details is acceptable. The only shortfall is not mentioning shell environment, time-out behavior details, or error cases, which would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero descriptions, but the Args block provides meaningful semantics for every parameter, including defaults, min/max bounds, and the meaning of stop_on_change, stop_on_exit_code, and max_snapshots. This fully compensates for the schema's lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence states a specific verb and resource: 'Run a shell command repeatedly and collect output over time.' This clearly distinguishes the tool from siblings like watch_file, poll_url, and run_long_command, which do not generically repeat arbitrary shell commands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs agents to 'USE THIS instead of running commands in a loop yourself — it handles the polling so you don't burn through your iteration limit.' This gives a strong when-to-use signal, and the examples reinforce realistic scenarios. It does not, however, name sibling alternatives like watch_file or poll_url or state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• poll_urlA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses the polling loop, stop conditions (status or body pattern), defaults for timeout and interval, and header format. It does not state what happens when the timeout expires or on network errors, which is a notable gap, but the core retry behavior is clearly described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized with a one-sentence summary, use cases, illustrative examples, and a compact argument list. Every section adds functional value—examples make the polling semantics concrete, and the Args block mirrors the schema without padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return values need not be explained. All input parameters and their interactions are covered, plus examples. The main omission is failure/timeout behavior, which matters for a polling tool, but overall the definition provides enough context for correct invocation in typical deployment and health-check workflows.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description is solely responsible for parameter meaning. It documents all seven parameters with practical details, including the regex semantics for expected_body_pattern, the exact header format, and the min/max/default values for timing parameters. This exceeds what the bare schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a precise action: 'Poll a URL repeatedly until it returns expected status/content.' This clearly identifies the resource (URL), the operation (polling), and the completion condition, distinguishing it from the unrelated memory and monitoring siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description names concrete use cases ('waiting on deployments, health checks, APIs') which tells an agent when to apply the tool. It does not explicitly discuss when not to use it or mention alternatives like monitor_command or watch_file, but the scenarios are clear enough for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recall_memoriesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses that query uses full-text search, that an empty query returns recent memories, and that category is an optional filter with an enumerated list. This is sufficient for a read-only recall tool, though it doesn't describe ordering or empty-result behavior explicitly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tightly structured: one-sentence purpose, direct usage instruction, then a clean Args list. Every sentence provides necessary information, and the most important usage directive is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a memory-retrieval tool with an existing output schema, this is complete. It tells the agent when to invoke, how to form the query, what filters are available, and the limit behavior. No critical operational aspect is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description fully compensates. It explains each parameter: query search semantics and empty-query behavior, category filter with all allowed values, and limit with its default of 20. This far exceeds the bare schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Search and recall past memories,' which names a specific verb and resource. It clearly differentiates from sibling write tools like save_memory and forget_memory, so an agent can identify this as the retrieval tool without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit call-time guidance: 'Call this at the START of every session' and 'whenever you need to remember something from the past.' It doesn't name alternatives explicitly, but the read-versus-write distinction across siblings is implicit, so the context is clear even without formal exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_long_commandA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral disclosure burden. It covers one-shot execution, streaming until completion or timeout, progress-check intervals, early termination via stop_pattern, and tail output. However, 'capture its full output' vs. 'tail_lines' returning only the end of output creates a minor ambiguity about what exactly is returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a one-line summary, a comparison with the sibling, a use-case list, concrete examples, and a clear Args section. Every sentence is informative; the examples use realistic values, and the section headers make scanning easy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no annotations, and an output schema handling the return shape, this description covers everything an agent needs to invoke the tool correctly: what it does, when to use it, how it differs from the closest sibling, parameter details with defaults, and example usage. There are no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description alone must explain all 5 parameters. It does so thoroughly: each arg gets a meaning, default value, and in some cases a max or format (e.g., timeout_seconds default 600/max 3600, stop_pattern as regex that kills the process). Example command pairings make the semantics even clearer.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb+resource statement: 'Run a long-running shell command and capture its full output.' It explicitly distinguishes itself from monitor_command, so an agent can immediately recognize this tool's unique role among siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It names the specific alternative (monitor_command) and explains the key difference: this runs ONE command, whereas that re-runs repeatedly. It also lists concrete use cases (builds, installs, migrations, test suites), providing clear when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.