Server Quality Checklist

Tool Scores

• get_historical_ratesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses that sampling is fixed per period (1d=hourly, 7d/30d=daily, 1y=weekly), the window always ends NOW, and it provides the return shape. It does not mention rate limits or auth, but for a read-only historical data tool the disclosed behavior is substantial and accurate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the primary use case, followed by the return format, key behavioral constraints, and sibling routing. Every sentence earns its place and there is no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Even without an output schema, the description gives the full return structure, sampling rules, window semantics, and explicit alternative tools. Combined with fully documented parameters, an agent has everything needed to call this tool correctly in the intended scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema already describes source, target, and period in detail, including defaults and exact point counts. The description mostly restates the period sampling behavior and adds use-case flavor, but does not add significant new meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific operation ('get historical rates') with a clear resource and scope: fixed-window time-series. It gives concrete example queries and explicitly distinguishes itself from sibling tools, so an agent can tell exactly what this tool does without inspecting others.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states exactly when to use this tool ('fixed-window time-series questions') and when not to: use get_rates for specific past datetimes or custom ranges, and get_exchange_rate for a single live rate. This explicit routing to alternatives leaves no ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_ratesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral disclosure, and it does: it reveals the response is an array with one row per target×time bucket, defines the shape [{ rate, source, target, time }], and implies the operation is read-only. It does not cover potential rate limits or error behavior, but those are secondary for a straightforward retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence carries information: use cases, output shape, and routing to siblings. It is front-loaded with the most important selection criteria and contains no filler or repetition of schema text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a six-parameter tool with no output schema, the description covers the core usage modes, mutual exclusivity of parameters, grouping behavior, and response cardinality. It could also state ordering or time format of returned rows, but enough is present for an agent to call the tool correctly in the main scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents all six parameters at 100% coverage, so the baseline is 3. The description adds meaningful semantics on top: it links the target multiplicity to row count, explains the output row format, and frames parameters in concrete usage scenarios like 'USD vs EUR, GBP, JPY'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('get rates') and enumerates three distinct use cases with concrete examples: multi-target queries, point-in-time rates, and custom date ranges. It also distinguishes itself from siblings by explicitly covering what this tool is for, rather than restating the tool's name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage guidance is explicit and actionable: single live pairs should use get_exchange_rate, and fixed lookback windows should use get_historical_rates. It also references list_currencies for code validation, giving clear when-to-use and when-not-to-use direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_exchange_rateA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it delivers: it discloses the return shape as a JSON object with rate/source/target/time, the semantic meaning (1 source = X target), and the important behavioral limitation that crypto, commodities, and arithmetic are not supported. This goes well beyond a basic restatement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with usage triggers, then the return format, then exclusions, then sibling routing. Every sentence adds distinct value, and there is no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description is complete: it explains when to use it, what the output looks like, what it explicitly cannot do, and which sibling tools cover adjacent cases. An agent has everything needed to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and both source and target parameters already have detailed schema descriptions including ISO 4217 format, case, examples, rate-direction semantics, fiat-only constraint, and a pointer to list_currencies. The tool description repeats the rate direction but adds little that is not already in the schema, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the exact operation: returning the latest mid-market rate for a single fiat-to-fiat pair. It gives concrete user phrasings and explicitly distinguishes itself from sibling tools like get_rates and get_historical_rates. Nothing about the tool's core purpose is ambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states clear when-to-use conditions ('single fiat-to-fiat live exchange rate question'), explicit exclusions (crypto, commodities, arithmetic), and direct alternatives (get_rates for multiple targets, get_historical_rates for past dates/lookbacks). This gives an agent strong routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_currenciesA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full burden and does so well: it discloses the exact return shape, includes a count field, states the currency universe (160+ ISO 4217 fiat), notes the exclusion of crypto, and flags that it is cheap to call. This gives the agent a clear behavioral model for invoking it safely.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the most important usage instruction ('Call this BEFORE other tools'), and every sentence earns its keep: when to use, return shape, coverage, exclusion, and downstream value. No filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool with no output schema, the description fully compensates by defining the response structure and enumerating coverage. It also answers likely agent questions about validity, symbols, quantity, and relationship to sibling rate tools. Nothing needed for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description cannot add meaning beyond the schema; baseline 4 applies. It still usefully explains what the returned codes represent and how they relate to other tools, but there is no parameter-specific detail needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the specific resource (supported currencies and their metadata) and the action (list/validate), and clearly differentiates itself from the rate-focused sibling tools. It also adds scope details like '160+ ISO 4217 fiat currencies' and explicitly excludes cryptocurrencies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use triggers ('before other tools... when unsure whether a currency code is supported'), user-phrase examples, and tells the agent to use it to prevent downstream errors in get_exchange_rate, get_rates, and get_historical_rates. It also states what it does NOT cover (cryptocurrencies), leaving no ambiguity about its role versus siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.