Server Quality Checklist

Tool Scores

• tables_schemaC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. It does not state whether the operation is read-only, what the returned schema structure looks like, or how errors are handled. The description is not misleading, but it reveals almost no behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short phrase with no filler or redundancy. It is easy to parse and front-loads the core subject, though it is minimal enough that brevity is not a substitute for missing detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool the description states the core subject, but with no annotations, no output schema, and no parameter clarification, it leaves the agent to infer both how to call it and what to expect in return. It is not fully inadequate, but it has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter table_s has 0% schema description coverage, and the description only implies through 'multiple tables' that more than one table may be involved. It does not explicitly clarify that table_s contains table names, how names should be formatted, or what happens for unknown or invalid tables.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the resource ('schema') and scope ('a table or multiple tables'), and it is distinguishable from sibling tools that handle sample data, table listing, and query execution. However, it lacks an explicit verb such as 'returns' or 'fetches', so it is clear but not fully specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus list_tables, get_sample_data, or run_a_query. It does not mention prerequisites, limitations, or expected input context, leaving the agent to infer usage entirely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It clearly communicates that this is a read-only listing operation and mentions the optional filter scope, but it does not disclose return format, ordering, or pagination behavior. For a simple list tool this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the verb and resource, states the optional filter, and wastes no words. Every element contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description provides enough information to call it correctly. The main gap is the lack of explicit routing versus sibling tools, but this is minor given the simplicity of the operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining that table_type filters results and enumerating the valid categories ('marts, facts, dims'). The word 'optionally' signals that null means unfiltered, though the exact default behavior is not explicitly stated.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List'), a clear resource ('Gold Schema'), and a scope ('optionally filtered by type (marts, facts, dims)'). It is clear enough to distinguish from siblings like tables_schema, run_a_query, and get_sample_data, but it does not explicitly name the alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the usage context: list tables in Gold Schema, optionally narrowing by table type. However, it provides no explicit guidance on when to choose this tool over tables_schema or run_a_query, nor any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sample_dataA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It does disclose useful behavior: the result is capped at 10 rows, filtering is optional, and join_logic combines filters with AND/OR. However, it does not state whether the operation is read-only, what columns are returned, or how errors are handled, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The one-sentence purpose is front-loaded, followed by a compact Args block where each parameter earns its place. There is no redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple preview tool, the description covers the essential call mechanics: target table, filter shape, and join behavior. It is a little thin on the output side—no explicit statement of returned columns or empty-result behavior—and it does not mention the sibling run_a_query alternative, but the row cap and filter semantics make the tool usable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no property descriptions and only 0% coverage, but the description documents all three parameters: table_name, filters (with a concrete JSON format and a value-type warning), and join_logic (with allowed AND/OR values). This fully compensates for the schema's silence.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening line, 'Get sample data (top 10 rows) from a table with optional filters,' names a specific action, a resource (table), and an explicit row cap. This distinguishes it from sibling tools like list_tables and tables_schema, which return metadata, and from run_a_query, which implies a broader query rather than a capped preview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a quick-preview use case but never explicitly says when to choose this over run_a_query or how it differs from list_tables/tables_schema. There are no when-to-use or when-not-to-use statements, so an agent must infer tool selection from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_a_queryA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It reveals constraints such as 'Main table must be included', value formatting requirements, and a default limit of 20, but it does not explicitly state whether the operation is read-only, what the response format is, or what 'max' means in the limit line.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear summary followed by an organized Args list. It includes valuable examples and constraints without excessive prose. Minor issues like the typo 'Dictonary' and the ambiguous 'limit: default: 20, max' keep it from being perfect.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The parameter coverage is strong and the nested structures are explained, but the absence of an output schema and annotations leaves gaps: no description of the returned data shape, no clarity on whether queries are read-only, and the limit maximum value is unspecified. These are meaningful gaps for a query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so parameter documentation falls entirely on the description. It provides concrete formats and examples for all six parameters, including nested join/filter structures and the special between-operator syntax, fully compensating for the schema's silence.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: 'Execute custom queries... on F1 data tables.' It also names the key capabilities (joins and filters) that set it apart from sibling tools like list_tables and get_sample_data, making its role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains how to use the tool through its args list, but it does not explicitly state when to choose this tool over the siblings or when it is not appropriate. Usage is implied ('custom queries'), with no exclusions or alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.