Server Quality Checklist

Tool Scores

• get_rate_freshnessC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The word 'Return' implies a read-only operation, but there are no annotations to confirm this and no description of behavioral traits such as filtering behavior, error cases, or what is meant by 'freshness'. The description provides only a high-level result, not enough behavioral context for an agent without annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence with the action front-loaded and no filler. It is structurally clean, though the brevity sacrifices necessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations, no output schema, and an undocumented parameter, the description is too thin. It does not clarify what 'freshness status' looks like, what 'routes' are affected, or how the optional 'category' parameter influences the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema defines a single 'category' property with no description, and schema coverage is 0%. The description does not mention 'category', its purpose, accepted values, or whether it filters by data source. The agent cannot determine what value, if any, to provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and resource ('public data source freshness status and affected routes'), which distinguishes it from the rate-oriented sibling tools. However, 'freshness status' and 'affected routes' are not elaborated, leaving some ambiguity about exactly what is returned.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as get_top_rates or get_reality_score. No exclusions, prerequisites, or contextual triggers are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_switch_savingsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It states that a comparison is made, but does not disclose whether the top APY is fetched live, how the comparison result is expressed, whether there are side effects, or what failure modes exist. This is a notable gap for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence with no filler. The core operation is front-loaded and every word contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description should clarify what the tool returns or how the comparison is presented. It does neither. Given the presence of sibling rate-lookup tools, an agent may not know whether this returns a difference, a recommendation, or a boolean, leaving an important gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add parameter meaning. It does map currentApy to 'user's current APY' and balance to 'for a balance', which helps, but it does not explain units, interpretation, or how balance affects the comparison beyond that hint.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific operation (compare) and the resources involved (user's current APY vs SwitchWize's top savings APY). It is distinguishable from the sibling get_* tools because this is a comparison, not a simple rate lookup, though it does not explicitly name an alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when comparing a user's APY to the top rate) but provides no explicit guidance on when not to use it or how it relates to sibling tools like get_top_rates or get_top_hysa_rates. There are no stated prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reality_scoreC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states what is returned, not how the score is calculated, what output format to expect, or any other behavioral traits. This is a notable gap for a tool with no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that front-loads the core purpose. It has minimal waste, though the word 'premium' adds little substantive information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with fully documented schema, the description is adequate for basic selection and invocation. However, since there is no output schema, the description should ideally clarify what the returned Reality Score represents (e.g., range, meaning), which is currently missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and both parameters already have clear descriptions with examples. The tool description only loosely restates 'card and consumer segment' without adding extra meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Return'), a specific resource ('premium credit-card Reality Score'), and the inputs ('a card and consumer segment'). It is distinct from the sibling tools, which focus on bank rates and index datasets. However, the term 'premium' is vague and the exact nature of a Reality Score is not explained.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention any conditions, exclusions, or related sibling tools, leaving the agent to infer usage solely from the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_index_datasetB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral disclosure burden. The verb 'Return' implies a read-only operation, and the phrase 'summary and recent series' gives a basic sense of output shape. However, it does not disclose how 'recent' is defined, how the optional limit affects results, or any freshness or data-source caveats.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler or redundant wording. Every word contributes to identifying the tool's basic function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema or annotations, the description is too sparse to fully prepare an agent to call the tool correctly. It does not explain the return structure beyond 'summary and recent series,' does not clarify limit semantics, and offers no differentiation from similarly named siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 50%, so the description should compensate for undocumented parameters. It adds no meaning for the 'limit' parameter, leaving its role ambiguous, and only the schema examples for 'indexId' provide semantic grounding. The phrase 'recent series' hints at limiting but does not explicitly explain the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Return') and resource ('SwitchWize index dataset summary and recent series'), so an agent can tell this is a read operation for index dataset data. However, 'index dataset' is somewhat vague and it does not distinguish this tool from siblings like get_bank_gap_index, which could plausibly serve a similar purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidance is provided. The description does not state when to prefer this tool over alternatives, when not to use it, or any prerequisites or context for selecting it among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_top_hysa_ratesB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It does disclose that the output includes freshness and attribution metadata, which gives some behavioral context beyond a simple 'return rates'. However, it omits details like whether the operation is purely read-only, how 'top' is determined, or any response shape.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the core action and resource, with no filler. Every word adds meaning, and the sentence is appropriately sized for the described tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and multiple sibling tools, the description is too sparse. It does not explain the output structure, the meaning of 'top', the data source, or how this tool differs from get_top_rates, so an agent lacks enough context to invoke it correctly and confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention the 'limit' parameter at all. The schema's minimum/maximum constraints help, but the description fails to explain that limit controls the number of rates returned or how it interacts with the output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: return top high-yield savings rates, with explicit output metadata (freshness and attribution). It is clear, but it does not distinguish itself from sibling tool get_top_rates, which may represent the general rates tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus alternatives such as get_top_rates or get_reality_score. No context, exclusions, or conditional conditions are provided; the agent is left to infer the appropriate use case entirely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_top_ratesB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It does disclose that results include 'freshness and attribution metadata', which is useful, and the verb 'Return' implies a read-only operation. However, it does not explain ordering, default limit behavior, or what qualifies as a 'supported category', which would make the behavior fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence with no filler. It front-loads the core action and result, then appends the metadata detail, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, this description is too thin for full standalone understanding. It does not enumerate supported categories, define 'top' ranking criteria, mention the default limit, or distinguish its behavior from closely related sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only 50% of parameters are described in the schema (category has examples, limit does not). The description adds minimal meaning for category by framing it as 'supported category', but provides no semantic detail for limit, such as whether it controls the number of returned rates or how truncation works.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb ('Return'), resource ('top SwitchWize rates'), and scope ('for a supported category'), which is clear and actionable. It does not explicitly compare against sibling tools like get_top_hysa_rates, but the category-driven scope gives reasonable differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance about when to use this tool versus siblings such as get_top_hysa_rates or get_rate_freshness. It does not state exclusions, fallback conditions, or scenarios where another tool would be preferable, leaving selection to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bank_gapB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It does disclose the nature of the operation ('Estimate' suggests computation), the time horizon ('annual'), and the dependency on 'current' index assumptions. However, it does not say whether the tool is read-only, how the estimate is calculated, what happens for a zero balance, or what the output format is. Adequate but shallow for an unannotated tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single 18-word sentence with no filler. The verb and resource are front-loaded, and the qualifier 'using the current Bank Gap Index assumptions' earns its place by grounding the estimation basis. Every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-param tool with no required parameters, the core is covered. But there is no output schema and no annotations, so the description should hint at the return value and clarify how this differs from get_bank_gap_index; it does neither. The meaning of 'savings gap' is left undefined, which is a real gap for agent comprehension.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 50% (balance lacks any schema description), so the description must compensate. It does add partial meaning by identifying balance as the subject of the estimation and implies bank is peripheral context. But it does not clarify balance units, the meaning of the gap, or how bank influences the estimate beyond the schema's 'user-facing context' note.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Estimate') with a clear resource ('annual savings gap for a balance') and a distinguishing qualifier ('using the current Bank Gap Index assumptions'). This separates it from get_bank_gap_index (which likely returns the index data itself) and compare_switch_savings (comparison-focused). However, it never defines what a 'savings gap' is or what the returned value represents, leaving mild ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given for when to use this tool versus alternatives. The phrase 'current Bank Gap Index assumptions' implies a contrast with get_bank_gap_index, but that is left to inference. There are no explicit conditions, exclusions, or mentions of sibling tools, so the agent must guess which tool fits a given task.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bank_gap_indexA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits itself. It does state the output contents, which is useful, but it does not mention data freshness guarantees, whether the value is cached or real-time, any access requirements, or what happens if no release exists. The minimal nature of the operation limits the gap, but the description is still somewhat bare.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that immediately names the action and the three concrete deliverables: headline value, latest release, and methodology URL. Every word earns its place, and there is no fluff or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool, the description is nearly complete. It names all return components, but since there is no output schema, it could be clearer about the shape or format of the response (e.g., whether the release is a date, version string, or object). Still, the operation's simplicity means this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are irrelevant. The baseline of 4 applies because there is nothing for the description to compensate for; the schema already fully defines the input as an empty object.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Return' and clearly identifies the resource: the current SwitchWize Bank Gap Index headline value, latest release, and methodology URL. It enumerates exactly what outputs to expect, and this specificity distinguishes it from siblings like get_index_dataset or get_bank_gap, which likely cover broader or different data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives. With seven sibling tools, the description does not state whether to prefer this for quick headline data or get_index_dataset for full historical data. The intended usage is only implied by the tool name and listed outputs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.