Server Quality Checklist

Tool Scores

• vision_detectA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations give basic hints, but description adds substantial context: cost (1 credit per 5 calls), that detection only reads page 1, the meaning of 'recommended' as exactly what preset:'auto' runs, and fallback behavior as an honest 'unknown'. No contradictions with annotations; readOnlyHint false is clarified by the read-only nature described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with front-loaded purpose and cost, but is relatively long. Every sentence contributes value, yet it could be tightened without losing meaning, especially the repeated emphasis on cost and the comparison with vision_analyze.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description conveys enough about output structure (presets ranked with confidence and reason, recommended field). It also covers cost, fallback, and usage. However, it leaves ambiguity about file parameter requirement (both optional in schema) and does not explicitly state that at least one file source is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions cover all 4 parameters at 100% coverage, so the baseline is 3. The description adds no parameter-level semantics beyond the schema; it does not explain that at least one of file_url/file_path must be provided, nor how to choose between them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states the tool identifies what a file is without extraction, explicitly contrasting with vision_analyze. It also specifies output: presets ranked with confidence and reason. This distinguishes it from sibling tools and provides a clear verb+resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use ('when the TYPE is the decision') and when-not-to-use ('do NOT reach for it as a step before every vision_analyze call'), naming the alternative preset:"auto" within vision_analyze. Includes concrete scenarios like routing a mixed inbox and avoiding costs on large PDFs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vision_askA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations, the description explains cost behavior (per image/per page, failures cost nothing), the verdict semantics ('yes'/'no'/'uncertain'/'n-a'), confidence markers, document-level answers, and the auto/sync/async fallback behavior. This gives a clear mental model of what will happen when the tool runs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is lengthy but well structured with distinct sections for purpose, cost, usage guidance, and reading the output. Every paragraph earns its place, though the sentence on max_chars duplicates the schema's message and could be trimmed without meaningful loss.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Even without an output schema, the description fully explains the response format and semantics: branch on verdict, the meaning of each verdict value, confidence markers, and document-level scoping. It also covers cost, page selection, and long-document modes, making the tool self-contained for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with thorough parameter descriptions. The tool-level description mainly restates information already present in the schema, such as pages being charged per selected page and questions being free. It adds almost no new parameter-level meaning beyond what the input schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Ask up to 5 plain-language questions about ONE image or PDF and get answers with a verdict.' This is a specific verb+resource+scope statement. It also explicitly contrasts with vision_analyze by naming that tool, so sibling differentiation is clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Reach for this over vision_analyze when the answer is a judgement rather than a field' and the reverse. It also gives concrete context for mode and pages, such as passing 'async' up front past ~10 pages.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vision_get_presetA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only and idempotent, but the description adds critical behavioral semantics: fields marked ★ are always present, others appear only when the document carries them, and null means absence rather than failure. This goes beyond the annotation hints and clarifies output interpretation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized, with the purpose front-loaded and each paragraph adding distinct value: field list definition, free access, correctness warning, and interpretation rules. It is concise, with zero fluff, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Even without an output schema, the description fully explains what the response will contain (field names, types, descriptions, presence markers), the meaning of null, and how the format parameter affects the output. It is complete for a simple read-only getter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% description coverage for both parameters (name and format), including detailed enum descriptions. The tool description adds no new parameter information beyond what the schema already supplies, so the baseline score of 3 is justified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches the full field list for one preset, with every field name, type, description, and presence marker. It explicitly calls itself the 'ONLY correct source' for a preset's field names, distinguishing it from sibling tools like vision_list_presets and from guessing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the agent when to use this tool (to obtain authoritative field names) and when not to (never write from memory), while also noting it is free and requires no API key. It explains how to interpret the response (★ markers, null meaning) and warns against using guessed names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vision_get_taskA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true and openWorldHint=true, but the description adds important behavior: it is free (no credits consumed), results are kept for 7 days, and after that only metadata survives. It also explains that a queued state is working, not failed. However, it does not specify details like the exact structure of the result or how errors are reported.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. It leads with a clear purpose sentence, then provides key behavioral details and usage notes in short paragraphs. Every sentence adds value, and the formatting makes it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema, the description compensates by explaining what the result contains ('Status, and once finished the result'). It covers usage context, cost, data retention, and the meaning of intermediate states. For a tool with three simple parameters and clear behavior, this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for parameters, so the baseline is 3. The description adds value by explaining the format parameter with details on each enum value (markdown, compact_json, json) and when to use each. It also provides guidance on max_chars: 'Raise it only if you truly need more than 20 000 characters.' This adds semantic context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Status, and once finished the result, of a queued task.' It identifies the specific resource (a queued Vision API task) and the action (get status/result). It distinguishes itself from siblings by referencing queued tasks and explaining the context of async calls or polling timeouts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: when to call it (with an id from a queued call, either from mode:"async" or when polling ran out of time), and what not to do (re-submitting the file would be a second charge). It also clarifies that queued/processing states are not failures, which helps the agent avoid unnecessary re-submission.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vision_list_presetsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, openWorld, and idempotent, but the description adds valuable context: it's free with no credits or API key, and the names are catalogue entries not field names (cautioning against misuse). No contradictions with annotations. This adds meaningful behavioral safety and cost information beyond the structured metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: it opens with what the tool does, then cost, then usage and anti-usage, then a caution. Every sentence earns its place and adds distinct value. It's compact yet comprehensive, avoiding fluff while covering all necessary aspects.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a simple listing tool with good annotations. It explains the purpose, cost, usage contexts, and a critical caution (catalogue entries vs field names). Without an output schema, it gives enough about the content (field count and purpose). The combination of annotations and description offers a full picture for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of the parameter (format) with a detailed description of each enum value. The tool description does not add extra parameter detail, but since schema coverage is complete, the baseline of 3 is appropriate. The description doesn't need to repeat schema info; it focuses on when to use the tool, which is fine.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists the preset catalogue with field count and purpose. It distinguishes itself from siblings by specifying exactly what it provides (catalogue entries) and explicitly contrasts with vision_get_preset (which retrieves a single preset) and other tools. This is a specific verb+resource with clear scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use scenarios (user asks about document types, needs preset name for code) and when-not-to-use scenarios (avoid for preset selection before extraction, with alternatives like preset:'auto' and vision_detect). It names specific alternative tools and explains why they are better in those situations. This is exemplary usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vision_creditsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral details beyond the readOnlyHint annotation: it is free (consumes no credits), and it explains the order of bucket spending. This gives the agent a deeper understanding of what to expect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the main purpose. Each sentence adds unique information: the resource, the cost, the usage scenario, and the bucket order. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters), the existing annotations, and the presence of an output schema, the description fully covers the tool's context. It explains when and why to use it, and the cost implication.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no input parameters, so the description has nothing to add. Per the rubric, 0 params gets a baseline of 4; no further clarification is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing the account's credit balance and per-bucket breakdown, which is distinct from sibling tools focused on presets and analysis. The 'Worth checking before a large batch' phrasing reinforces its purpose as a balance check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use the tool ('before a large batch') and explains the benefit (avoiding insufficient_credits errors). It also provides the bucket consumption order, giving the agent a clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vision_analyzeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only flag readOnlyHint=false, non-idempotent, and non-destructive; the description adds substantial behavior beyond that: credit costs, failure refunds with no cleanup, magic-byte detection, 60-second timeout with queue resubmission, absent-field reporting, and confidence markers. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but deliberately organized into labeled sections: cost, selection modes, reading output, long documents, and per-file limits. Every sentence carries operational information; there is no filler or repetition beyond what is needed for a 14-parameter tool with no output schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 14-param, 0-required tool with no output schema, the description covers input selection, costs, failure/refund semantics, output interpretation (Not found, mid/low confidence), timeout behavior, page selection, formatting choices, and folder-processing guidance. An agent has enough to call it correctly without external docs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents parameters, but the description adds critical semantics: the exclusivity rule ('Choose ONE way'), schema_name not being combinable, schema descriptions acting as prompts, mode fallback/refund behavior, and the meaning of the format options. This goes well beyond the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with 'Extract structured data from ONE image or PDF' — a specific verb plus objects and file types. It distinguishes the tool's role from vision_list_presets by telling users not to call that sibling just to guess a preset, and clearly separates the preset/schema/schema_name configuration paths.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit when-to-use guidance: use 'auto' when the document type is unknown, call vision_get_preset before hardcoding field names, pass async for anything over ~10 pages, and process one file per call while waiting on too_many_tasks. It also names exclusions, such as 'Do not call vision_list_presets just to guess a preset.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.