Server Quality Checklist

Tool Scores

• create_batch_jobA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully covers behavioral traits: it returns a job id, runs rendering in the background, and poll get_job is required. It also discloses the side-effect that filename_template and group_by are saved to the template/project, altering defaults for future runs—critical for an agent to know.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: it opens with a one-sentence system context, then the tool's specific action, followed by usage conditions, a behavioral note, and a parameter warning. Every sentence serves a purpose and the information is front-loaded with the core purpose. While not extremely short, it is efficient and free of fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains the return value (job id) and how to retrieve results via get_job. It covers prerequisites, background execution, and side-effects, making it fully self-contained for an agent to correctly invoke and interpret the results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides complete descriptions for all four parameters, including the persistence side-effect for group_by and filename_template. The description adds usage guidance ('Only pass them when...'), but does not introduce new semantic meaning about what each parameter does beyond the schema. Given 100% schema coverage, the description meets but does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: it queues a batch job that renders one PDF per row of an uploaded dataset and returns a job id. It also distinguishes itself from siblings by mentioning the batch scenario ('many documents at once') versus single renders, making it specific and differentiating.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use it ('Use it when the user wants many documents at once'), provides a prerequisite (dataset must already be uploaded, cannot upload), and directs the agent to poll get_job for progress. This is clear guidance on when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It discloses that the return includes a temp-file path and size, that PDFs under 512 KB are attached inline, and warns about performance when fetching many documents. It also notes the provenance of document_id. While it doesn't mention error handling or permissions, it covers the essential behavior for a download tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: the first sentence states the core purpose, followed by id provenance, usage guidance, and return behavior. Every sentence adds value with no fluff, and the most critical information (spoiler: the id source) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers all necessary aspects: what it does, how to obtain the id, when to use it vs alternatives, and what the return includes. Nothing essential is missing for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes document_id with high coverage, but the description adds crucial disambiguation: it explains the id comes specifically from get_job's document list and clarifies it is not a template or job id. This goes beyond the schema's baseline, making parameter meaning clearer for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the specific verb (download), resource (PDF from a batch job), and distinguishes it from siblings like render_pdf and get_job. It also clarifies that document_id comes from get_job, making the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use the tool (fetch a specific document or spot-check a batch) and when to instead direct the user to the SheetRender web app for bulk downloads. It also references get_job for obtaining valid IDs, providing clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_jobA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With zero annotations, the description carries the full behavioral burden and succeeds. It discloses the asynchronous polling nature, enumerates the complete state machine (queued, retry_queued, running, succeeded, partial, failed, cancelled), explains that the document list is empty during in-progress states, and specifies what becomes available once finished. This is exactly the behavioral context an agent needs to avoid misinterpreting empty results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short paragraphs, zero filler. The purpose and polling caveat are front-loaded; the SheetRender context sentence and the get_document routing both earn their place. The state enumeration is necessary, not redundant, given the target states versus document-list-availability states must be distinguished.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema and no annotations, the description fully compensates: it explains the returned fields (status, rows done/failed, document ids/filenames), the conditional availability of results, and the follow-up action. For a single-parameter polling tool, nothing an agent needs to call it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% — job_id is documented as 'Job id returned by create_batch_job.' The description reinforces but does not extend this: it references 'the id' of every rendered document but adds no new formatting or validation details beyond the schema. Baseline 3 is appropriate since the schema already carries the semantic weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: 'reports the progress of a batch job started by create_batch_job' — it monitors a job rather than producing content. It distinguishes itself from siblings by explaining the workflow position (reports on create_batch_job output, feeds get_document). The title 'Check a batch PDF job' reinforces the purpose without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description names the prerequisite sibling (create_batch_job), the successor (get_document for downloading), and gives explicit call-timing guidance: poll again while status is in 'queued', 'retry_queued' or 'running' rather than treating an empty document list as zero results. This is concrete, actionable when-to-use guidance that maps the tool into a larger workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_templatesA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It clearly states it is a listing operation that returns template ids, which is the core behavior. It adds context about being a prerequisite mapping step, which helps the agent understand its role. It does not explicitly state it is read-only or non-mutating, but that is strongly implied. It also does not mention return format or pagination, but for a simple list tool this is a minor omission. The description is transparent enough for correct invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the domain context and the tool's purpose, then immediately gives the when-to-use guidance and the arg note. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with no output schema, the description covers the essential points: what it does, what it returns (ids), and when to call it. It does not describe the exact response structure or pagination, but those are often inferred and not critical for a simple list operation. The context about SheetRender and the mapping step adds completeness. Minor gap: it does not mention how many templates might be returned or any limits, but this is a minor omission given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero properties, and the description explicitly says 'Takes no arguments.' This goes beyond the schema by removing any possible ambiguity about whether arguments might be accepted (an empty schema could be misread). The explicit statement adds value and fully clarifies parameter expectations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb-resource pair ('lists the templates saved in the user's account') and clarifies the value it provides (the id each one needs). It also distinguishes itself from siblings by positioning it as the listing step against the rendering/job tools (render_template, create_batch_job). This is unambiguous and effectively differentiates the tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to call it ('Call it first whenever the user refers to a template by name') and points to the follow-up tools (render_template, create_batch_job). It also notes the tool takes no arguments, which is a useful usage hint. No ambiguity about when or how to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• render_templateA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry behavioral disclosure. It covers the output (temp-file path, size, inline attachment under 512 KB), the page_settings override behavior, and the free-plan footer. It does not mention error cases or side effects, but the core mutating behavior (creating a PDF) is transparently described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized and front-loaded with purpose. It conveys all necessary context in a compact flow: what the tool does, when to use it, how data works, page_settings caveat, free-plan note, and output details. No superfluous sentences; every line earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the nested page_settings object and no output schema, the description covers everything needed to call the tool correctly: how to obtain template_id, the data shape, optional page_settings behavior, return value specifics, and the free-plan footer. It also routes to alternatives, making it complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed property descriptions, so baseline is 3. The description adds value by explaining that omitting page_settings keeps the template's saved setup and that data supplies one row of variables. It also clarifies the 'overrides for this render only' nuance not present in the schema, pushing it above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool renders one PDF from a saved template and returns the file path. It distinguishes from siblings by naming render_pdf (for custom HTML) and create_batch_job (for multiple rows), and directs to list_templates for the ID. The verb+resource+scope is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given for when to use this vs alternatives: use it for existing designs, render_pdf when writing HTML yourself, and create_batch_job for rendering many rows. It also instructs to get template_id from list_templates. This leaves no ambiguity about selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• render_pdfA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It transparently discloses that external resources are not fetched, that HTML over 2MB is rejected (including after substitution), that free accounts get a footer, and that return includes path/size plus inline attachment for small PDFs. This exceeds expectations for behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear paragraphs. Each sentence earns its place: purpose, usage (with alternative), HTML requirements, data behavior, server limits, and return format. It is detailed but not repetitive; complexity is handled without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all necessary aspects for correct use: input requirements, template variable mechanics, output behavior, error conditions (2MB limit), and an environmental quirk (free-plan footer). It is fully self-sufficient even without annotations or output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema coverage, the description adds substantial meaning beyond the schema: it explains Jinja variable substitution with a concrete example, confirms loops/conditionals work, clarifies that data can be omitted, and details page_settings usage in plain terms. This significantly aids correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: rendering a single PDF from user-supplied HTML and returning the saved file path. It also distinguishes itself from render_template by explicitly saying when to use which, making it easy for an agent to select correctly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('one-off documents... writing the markup yourself') and when to use the alternative (render_template when a saved template exists). It also warns about size limits and the free-plan footer, covering both usage context and potential pitfalls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.