Server Quality Checklist

Tool Scores

• query_documentation_index_toolA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains that the tool retrieves and formats documentation snippets, enumerates the exact output fields, and instructs the agent to tell the user which documentation URL to open. It does not cover edge cases such as no matches found, but the core behavior is clear and accurate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description front-loads the core behavior and output format before moving to user-facing instructions and examples. The list of examples is long but earns its place by serving as parameter guidance; there is no filler or redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and annotations, the description covers what the tool returns, how the agent should present the result, and how to phrase queries. The main gap is the lack of explicit guidance for choosing between this tool and its siblings, which is already reflected in the usage guidelines score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description for the 'query' parameter, so the description must compensate. It does so through a detailed 'Input query examples' section that illustrates the expected natural-language phrasing. For a single parameter, this gives an agent sufficient understanding of what to pass, even though explicit constraints like length or format are not stated.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Retrieves and formats') and clearly names the resource ('Tecton documentation snippets'). It also describes exactly what each snippet contains (Source URL, section header, relevant text chunk), which makes the tool's function unambiguous and distinguishable from sibling code-example and SDK-reference tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides ten concrete input query examples that establish when the tool is appropriate, such as 'How do I unit test a Feature View?' and 'Explain Batch Feature Views.' It gives clear context on the kind of natural-language documentation questions to use, but it does not explicitly name alternatives or state when not to use this tool versus its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_example_code_snippet_index_toolA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden, and it does disclose the retrieval mechanism ('vector database'), the output form ('collection of python code examples'), and the ranking ('ranked by relevance'). It could add failure or freshness caveats, but for a simple retrieval tool the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The main description is front-loaded and the output format is stated in one sentence. The 13 examples are long but earn their place because the schema provides no query guidance; little in the text is redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter retrieval tool with no output schema, the description covers what to send and what will come back. It lacks only edge-case behavior (e.g., empty results or non-Tecton queries), which is minor for this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema gives only a bare 'query' string with no description (0% coverage), so the list of 13 concrete query examples is essential and largely compensates. It shows the expected phrasing and scope of queries, though it does not state an explicit 'describe the Tecton construct you need examples of' rule.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence uses a specific verb ('Finds'), a specific resource ('relevant Tecton code examples'), and a mechanism ('vector database'). Paired with sibling names like query_documentation_index_tool, this clearly marks the tool as the code-example retriever rather than a docs or SDK reference lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit condition: query the examples retriever before generating Tecton code. It does not explicitly name when to prefer documentation or SDK-reference siblings, so it stops short of full when-to-use/when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_full_tecton_sdk_reference_toolA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. 'Fetches' implies a read operation and 'full SDK reference' indicates scope, but the description does not mention possible response size, structure, or any access requirements. This is acceptable but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences each serve a purpose: what the tool does, when to use it, and when to use the alternative instead. There is minimal redundancy and the main point is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-argument fetch-all tool, the description clearly states the use case and directs subset users to the right sibling. It would be slightly more complete if it noted the likely size or format of the returned reference, but nothing critical is missing for invocation decisions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so there is nothing for the schema or description to explain about arguments. The description adds useful contextual scope by contrasting full and subset behavior, matching the baseline for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Fetches the full Tecton SDK reference') and clarifies the scope as 'all classes/functions.' It differentiates from the sibling query tool by explicitly contrasting full versus subset access.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this only if you need to get the full SDK reference' and names the alternative for subset use, giving an agent a clear decision rule.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_tecton_sdk_reference_toolA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full burden, and it does this well. It warns that invalid names will 'result in an error or empty output,' and it details exactly what the returned output will include: a bulleted list, type, name, import path, definition header, and full docstring. This makes the tool's behavior predictable to an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The prose is concise and well-structured, with the critical constraint front-loaded and output format clearly sectioned. The description is long because of the extensive allowed classes/functions list, but that list is necessary to prevent invalid calls, so the length is justified.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Since there is no output schema, the description appropriately documents the return structure, including details like import path and definition header. It also handles the main failure mode. However, it does not provide any guidance about close alternatives or mention the full-reference sibling, which would have made the context complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only the parameter name and type, with 0% schema description coverage, so the description must fully explain class_names. It does, by requiring names to come from the provided 'Available classes/functions' list and by describing the consequence of violating that constraint. The exhaustive allowed-value list adds substantial semantic meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Fetches the Tecton SDK reference for a specific list of classes/functions.' This clearly distinguishes it from the sibling get_full_tecton_sdk_reference_tool, since this tool is scoped to a provided list rather than returning the entire reference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use it: 'Use this tool when you need information about specific Tecton components from the allowed list.' It does not explicitly mention when not to use it or name alternatives like query_documentation_index_tool, so it stops short of full routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.