Server Quality Checklist

Tool Scores

• scrape_webA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It does disclose key behavior: it fetches external web content, returns markdown, uses Jina Reader, and requires http/https URLs. However, it does not mention errors, access limitations, size limits, or that this is a read-only network operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and includes clear Args/Returns sections, but it is somewhat redundant: 'Scrape the content of a web page using Jina Reader' and 'fetches the content of any web page... using the Jina Reader service' overlap. It could be tightened without losing information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with an output schema, the description covers the essential contract: input URL must be http/https, output is the page content as markdown. It lacks failure-mode detail, but the tool is low-complexity and the return format is explicitly stated.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides zero description coverage, but the tool description compensates well for the single parameter: it identifies 'url' as the page to scrape and imposes the http/https constraint. This gives an agent meaningful information beyond the bare schema definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb and resource: scrape the content of a web page using Jina Reader, returning markdown. It does not explicitly compare against siblings such as search_docs, but the action and resource are clear enough to distinguish it.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when raw web page content is needed, and states it can handle 'any web page'. It gives no explicit guidance about when not to use it or what alternatives (e.g., search_docs, count_word_occurrences) are better suited for other tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• count_word_occurrencesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses that the tool scrapes a web page and returns a dictionary, which is useful, but it omits caveats about network dependency, page size limits, site accessibility, or what the vague 'additional info' actually contains.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a one-line summary, brief explanation, Args section, and Returns section. However, the first two sentences are somewhat redundant ('Count occurrences...' and 'This tool scrapes... counts...'), which slightly reduces tightness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with three parameters, the description covers inputs, behavior, and return type adequately. It does not specify the exact structure of the returned dictionary or note edge cases such as unavailable pages, but given the tool's simplicity, the remaining gaps are relatively minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate for the schema's bare types. The Args section clearly explains all three parameters, including the url to analyze, the word to count, and the case_insensitive behavior with its default. It fully adds meaning beyond the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb-resource combination ('Count occurrences of a specific word on a web page') and clearly distinguishes the tool from siblings: scrape_web (raw content retrieval) and search_docs (searching documentation). The purpose is immediately obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose makes the tool's usage context clear: use it when you need word frequency from a web page. However, it does not explicitly contrast with alternatives or state exclusions for when another tool would be better, so it stops short of full usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It transparently states that the tool downloads a zip, indexes markdown files, caches the index, and returns document previews. It does not mention network failure modes or cache invalidation, but the core side effects are clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a summary, Args, Returns, and Examples sections. Every section contributes useful information, and the core purpose is front-loaded in the first sentence without unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and moderate complexity, the description covers the full workflow, parameter semantics, defaults, return shape, and practical examples. The presence of an output schema covers the return format details, so nothing critical is missing for an agent to invoke this correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the description fully compensates by explaining each parameter: query as a search string, zip_url with a default and required URL format, and num_results as a maximum count. This adds substantial meaning beyond the raw schema types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches documentation from a GitHub repository zip file, with a specific verb, resource, and workflow (download, index, return relevant documents). It implicitly differentiates from siblings like scrape_web and count_word_occurrences by focusing specifically on GitHub-hosted documentation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied through examples and the default zip_url, showing when to call the tool directly. However, there is no explicit guidance about when to prefer this tool over alternatives or any exclusions for edge cases like searching non-markdown documentation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.