Server Quality Checklist

Tool Scores

• json_mergeB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral disclosure burden. It does state dotted-path traversal and empty-path root behavior, but it does not say whether the operation mutates the input, how merge conflicts are resolved, or what is returned. This leaves a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence with no filler. The parenthetical about empty path adds a valuable edge case without bloating the text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three required string parameters and no annotations, the description is too sparse. It does not define the data and value parameters, describe merge behavior, or clarify return semantics, so an agent cannot confidently construct a correct invocation without additional assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all three parameters. It adds meaning for path via 'dotted path' and 'empty path', but data and value remain ambiguous — it is unclear whether they should be JSON strings, parsed objects, or something else. This is insufficient for a 3-parameter required tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific operation (merge) and resource (JSON object at a dotted path), and the parenthetical clarifies root merging. It is clear enough to distinguish from json_get and json_set, but it does not explicitly reference those siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The empty-path special case gives useful contextual guidance for when to merge at the root. However, there is no explicit comparison to alternatives like json_get, json_set, or write_file, so the decision of when to choose json_merge is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_workflowB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that the tool executes a workflow in the server workspace, but it does not mention potential side effects, permissions, state changes, or error behavior—critical details for a 'run' operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence with no filler. It front-loads the action and resource and economically includes the workspace scope, achieving high information density with minimal words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description leaves important execution semantics unexplained, such as what `inputs_json` does, how paths are resolved, and whether running a workflow causes irreversible side effects. This is insufficient for a tool that actually runs code.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not explain the purpose or format of the `path` and `inputs_json` parameters. The description never maps 'workflow file' to `path` or clarifies that `inputs_json` is a JSON string, leaving the agent to infer meaning from parameter names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a strong verb ('Run') and names a specific resource ('ReadyAgents workflow file'), plus adds scope ('under the server workspace'). None of the sibling tools perform this action, so it is cleanly distinguished.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for executing workflow files and no sibling tool overlaps. However, it does not explicitly state when to use it instead of alternatives or mention any exclusions or prerequisites, so the context is clear but not fully guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• http_getB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden for behavioral disclosure. It correctly states the core behavior (an HTTP GET) and the allow_http restriction, but it does not mention error handling, timeouts, redirects, or response behavior. The disclosure is adequate but thin.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with zero filler. It front-loads the action and adds a critical availability constraint, so every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and an output schema likely documents return values, lowering the burden on the description. Still, the description doesn't cover expected behavior on non-200 responses, error cases, or practical usage context. It is minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter, url, with 0% description coverage. The description's phrase 'HTTP GET a URL' confirms that the parameter is the target URL, adding some meaning beyond the schema's bare 'Url' title. However, it does not specify URL format, supported schemes, or other constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the operation with a specific verb and resource: 'HTTP GET a URL.' This is enough to understand what the tool does and is implicitly distinct from siblings like json_get or read_file, though it lacks an explicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives such as json_get or read_file. The only usage-related note, 'Disabled unless allow_http is enabled,' is a configuration prerequisite rather than a selection guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• json_setB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the main behavior (set a path, return the document), which is useful with no annotations present. However, it does not clarify whether the input is mutated in place, what happens if the path does not exist, how invalid JSON is handled, or how the returned document is formatted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. It front-loads the action and resource, and every word contributes to the tool's meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has three required parameters, no annotations, and no parameter descriptions, but the description only covers data and path semantics at a high level. Missing details about value encoding, error behavior, and output format leave the agent under-informed for a reliable call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must clarify parameters. It does explain that 'path' is a dotted path and 'data' is JSON text or an object, but 'value' is left ambiguous: it is unclear whether value should be a raw JSON string, a quoted string, or a serialized JSON literal. This is a critical ambiguity for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Set'), a precise resource ('a dotted path in JSON text or an object'), and the result ('return the document'). This makes the tool's core operation unmistakable and distinguishes it from siblings like json_get, which presumably reads, and json_merge, which combines JSON.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus json_get, json_merge, or any alternative. The agent must infer usage from the name and short description, and there are no exclusions or conditions given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_dirA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does disclose sandboxing and dotfile skipping, which are important traits. However, it leaves ambiguity around the include_hidden parameter: saying 'dotfiles skipped' without qualification could mislead an agent into thinking hidden files can never be listed, when the parameter suggests otherwise.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that communicates the tool's purpose and key constraints without wasted words. Every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and zero schema description coverage, the description is somewhat thin. It covers the core listing behavior and sandbox boundary, but omits details about entry limits, sorting, and the interaction between include_hidden and the dotfile-skipping behavior. Still, the tool is simple and an output schema exists, so the gap is moderate rather than severe.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it does not explain any parameter semantics. The word 'directory' loosely maps to path, and the parameter names are mildly self-explanatory, but max_entries and include_hidden receive no clarification in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and resource ('a directory'), and adds meaningful scope constraints: sandboxed to the workflow workspace and dotfiles skipped. This clearly distinguishes it from sibling tools like read_file and write_file, which operate on file content rather than directory listings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for exploring workspace directories, especially given the sandboxing note. However, it does not explicitly state when to prefer this tool over alternatives or when not to use it, such as when a full file listing with hidden files is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds useful constraints: the file must be UTF-8 text and access is restricted to the workflow workspace. However, it does not describe error handling, size limits, or return behavior (though an output schema exists). This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that immediately states the action and key constraints. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with one parameter and an output schema, the description covers the core behavior and the critical sandbox restriction. It is missing path semantics (relative vs absolute), but the low parameter count and existing output schema keep the gap moderate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only identifies the parameter's referent as a UTF-8 text file in the workspace, without specifying path format, relative/absolute resolution, or defaults. This is minimal compensation for a single required parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (read), the resource (UTF-8 text file), and the scope (sandboxed to the workflow workspace). This makes the tool's purpose unambiguous and distinct from siblings like write_file and list_dir.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool—for reading UTF-8 text files within the workspace—but does not explicitly mention alternatives or exclusion cases (e.g., use json_get for structured JSON). This is implied usage, not explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calcA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It adds a key trait ('safe'), but it does not disclose error handling, supported operators, or return behavior beyond the verb 'Evaluate'. Adequate for a simple calculator but with gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the action and resource, and the example is directly illustrative. No redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with an output schema, the description is mostly sufficient. However, it lacks explicit notes on expression limitations and error behavior, which could matter for an agent choosing to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The example '2 + 2 * 10' clarifies the expected string format beyond the schema's bare type, but it does not define the complete grammar of allowed expressions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Evaluate' with a clear resource 'arithmetic expression', and provides an example. This distinguishes it from all siblings, none of which perform calculation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear: use this tool when an arithmetic expression needs to be evaluated. No exclusions are stated, but no similar sibling tool exists, so explicit when-not-to-use guidance is unnecessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• json_getA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and only says it extracts a dotted path. It conveys a non-mutating read operation and notes that input can be JSON text or an object, but discloses nothing about behavior on invalid JSON, missing keys, or failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with every word earning its place. No filler, no redundancy; structure directly supports skimmability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter getter with an output schema, the core invocation is mostly captured. However, the description's 'or an object' conflicts with the schema's string-only 'data' type, and there is no guidance on error handling or unsupported paths, leaving minor but real gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies that 'path' uses dot notation and that 'data' can be JSON text or an object, adding meaning to the bare schema properties. It leaves out examples and edge-case semantics, but covers the essential role of both parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Extract' and names the resource ('a dotted path') plus acceptable input ('JSON text or an object'). This makes it immediately distinguishable from sibling write tools like json_set and json_merge.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a read-only use case but never states when to choose json_get over siblings or when not to use it. There are no explicit exclusions or alternative routing, so the agent must infer its role from the name and extraction semantics.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosing side effects. It reveals that the file will be UTF-8 text and sandboxed to the workspace, but it does not state whether an existing file is overwritten, whether parent directories are created, or what happens on failure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single focused sentence that clearly states the operation, the file type, and the workspace constraint, with no unnecessary words. It is appropriately concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only two simple parameters and an output schema exists, so the description does not need to explain return values. The sandbox and encoding information give an agent enough context to make a first correct call, though overwrite and directory behavior are still unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not compensate for missing parameter details. It gives a small hint that content should be UTF-8 text, but it says nothing about path semantics, path format, content size limits, or how the two parameters relate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Write') and a clear resource ('a UTF-8 text file') and adds an important scope constraint ('sandboxed to the workflow workspace'). It is clearly distinct from siblings like read_file and list_dir, which represent read operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the intended use clear: writing text files within the workflow workspace. It does not explicitly name alternatives or state when not to use it, but the 'sandboxed to the workflow workspace' phrase provides helpful context for selecting the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nowA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It discloses timezone (UTC) and format (ISO-8601), and 'current' implies a stateless read. It does not mention precision, potential latency, or response structure, but for a trivial time tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no filler. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complexity is minimal, an output schema exists, and the description covers the essential behavior. It lacks explicit alternative routing, but that is not material for a zero-parameter UTC clock.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so there is nothing to document. Schema coverage is complete and the description's format note adds useful context beyond the empty input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States exactly what the tool returns (current UTC time) and the format (ISO-8601). It is clearly distinct from sibling tools like calc, json_get, or read_file, which operate on different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear: call when a current UTC timestamp is needed. It does not explicitly name alternatives or exclusions, but with zero parameters and a dedicated time resource, the usage is self-evident and no sibling competes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.