Server Quality Checklist

Tool Scores

• check_tariff_updatesD
  1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description fails to reveal any behavioral aspects, and the annotation readOnlyHint=true suggests safe operation, but the description adds no clarity on side effects or authorization needs. Without a description, behavioral transparency is entirely absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness1/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  With zero words, it is concise but lacks substance, making the criterion 'appropriate length relative to purpose'. The description has no structure to evaluate, so it cannot be effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 1-parameter helper tool, the description is empty, failing to cover required behaviors like whether it mutates state or how to use it. The description is absent, so it cannot be complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema includes one parameter, but the description gives no insight into its meaning beyond the schema itself. The schema description is decent, but the description should complement it. No added value is provided beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is absent, so it cannot describe the purpose with a verb and resource. It does not differentiate the tool from siblings. The title and name provide some clues, but the description itself is missing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives is provided. The description is empty, leaving the AI without instructions on when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_tariff_scenarioA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish readOnlyHint=true. The description adds important behavior beyond that: it returns chapter-99 rule text verbatim rather than a final computed rate, instructs the LLM to walk exception chains itself, and requires the output to be labeled as an estimate. This is useful and non-obvious behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: the first sentence enumerates return values, and the second captures the agent workflow and estimate caveat. Every sentence adds meaningful guidance without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, but the description explains the output categories and tells the LLM how to use and present the results. It does not specify an exact response shape, but that is not essential for correct invocation given the parameter schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all six parameters adequately. The description adds no parameter-level detail beyond mentioning origin and HTS code, which keeps this at the baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Returns') and names concrete outputs: MFN/FTA base rate, active chapter 99 rules, and MPF/HMF fees. This clearly distinguishes the tool from sibling tools like search_hs_candidates, watch_tariff_changes, and check_tariff_updates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context that the tool is for computing a tariff scenario from an HTS code and origin, and includes explicit procedural guidance for the LLM. It does not explicitly say when NOT to use it or name alternatives, so it misses the highest bar.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• watch_tariff_changesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate it's a write operation (readOnlyHint=false) and idempotent. The description adds the workflow context (that rates are tracked across dataset refreshes) and specifies the local watchlist, going beyond what the annotations alone convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences. The first sentence states the action; the second sentence provides the workflow and context. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 params, no output schema), and the description covers the purpose, the workflow (call check_tariff_updates after refresh), and the context (tracking tariff volatility). It doesn't discuss edge cases, but those aren't expected for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents both parameters with descriptions (100% coverage). The description's mention of 'optional origin' mirrors the schema description for origin_country, adding little novelty. It doesn't compensate beyond the existing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Saves') with a clear resource (HTS codes to a watchlist) and scope. It differentiates from siblings like check_tariff_updates by stating this tool does the saving while the sibling does the checking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states the workflow: save codes first, then call check_tariff_updates after each dataset refresh. This gives clear context for when to use the tool, though it does not explicitly state when not to use it or mention alternative tools for code search/calculation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dataset_infoA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already signals this is a safe read operation, and the description aligns with that by saying 'Fetch.' It adds modest context by listing the returned dataset attributes (date, source, license, row counts), but does not go beyond that with additional behavioral details such as output formatting or freshness semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence with no filler. The main purpose and the usage guidance are both front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple, zero-parameter metadata retrieval tool. The description covers what it returns and why to use it, and the readOnlyHint covers safety. Nothing essential is missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. There is nothing for the description to clarify about parameters, and the schema already confirms an empty parameter set.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and identifies the exact resource contents: date, source, license, and row counts. This clearly distinguishes it from sibling tools like search_hs_candidates and calculate_tariff_scenario, which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete usage context: 'use to check data freshness.' It does not explicitly state exclusions or compare against alternatives, but the intended scenario is clear enough for an agent to decide when to call it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_hs_candidatesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and openWorldHint=false, and the description adds substantial behavioral context beyond that: it cautions that results are 'CANDIDATES' and not legal rulings, explicitly warns about the HTS's use of 'legal terminology,' and describes the expected retry behavior on zero results. This transparency goes beyond just the read-only nature and helps the agent manage expectations about output reliability.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tightly written, using sentence case and a clear structure: what it does, a caution, a concrete instruction, and a fallback. The 'IMPORTANT:' emphasis and examples are efficient, though the paragraph could slightly overwhelm new users. Every sentence adds value without fluff, so a 4 is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two parameters and no output schema, the description covers all necessary context: input format, expected output, limitations, and error handling. It prepares the agent for edge cases (0 results) and clarifies the tool's role within the larger suite. No critical information is missing for it to be invoked correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents both parameters with 100% coverage (product_description and limit). The description adds high-value guidance for product_description by giving concrete rewriting examples ('pink kids backpack' → 'travel bags of man-made fibers') and explaining the need for legal/material-based nouns. This extra semantic layer justifies a score above the baseline of 3. The limit parameter is fully specified in the schema, so no additional info needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Returns top 8-10 digit HTS code candidates … for a product description.' It clearly differentiates itself from a 'classification ruling' and includes concrete details (MFN/FTA rates, CBP CROSS ruling links). This distinguishes it well from sibling tools like 'calculate_tariff_scenario' or 'watch_tariff_changes' by focusing on the candidate search use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is given on how to prepare input ('rewrite the product description into HTS-style material + use nouns') and how to handle empty results ('retry with 2-3 different phrasings'). It also clarifies the tool is for 'pre-broker screening, not a classification ruling,' which sets expectations. However, it doesn't explicitly name when to prefer a sibling tool, and the 'when not' is only implied by the disclaimer, not fully elaborated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.