Server Quality Checklist

Tool Scores

• nextjs_buildB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are completely absent, so the description must disclose behavioral traits. It mentions running a build and returning errors, but omits side effects like the creation/modification of build artifacts (e.g., .next directory), potential long execution time (timeout parameter exists), and that a failed build may still return partial output. The description does not fully cover the tool's impact or failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, concise sentence that front-loads the core command and lists key outputs with no filler. Efficiently communicates essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides some return-value detail (structured error types and page list) but lacks full context: no output schema, no mention of side effects, environment requirements, or how to interpret the results. It gives a reasonable overview but is incomplete for a tool that runs a full build without annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 67% of parameters (env and project_dir have descriptions; timeout_seconds does not). The description adds no extra parameter context beyond what the schema provides, though the missing timeout parameter is self-explanatory. Since coverage is moderate but the description adds no value, a score of 3 is appropriate—adequate but not insightful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Runs next build') and the specific outcomes (structured errors and generated pages list). It distinguishes itself from siblings like typecheck and npm_run by being Next.js-specific and producing build validation output. No ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives (e.g., typecheck, npm_run). It does not state prerequisites, like that a Next.js project must exist, or when it should be preferred over other validation tools. The agent is left to infer usage from the name and output description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• typecheckA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses the read-only nature of tsc --noEmit implicitly and describes the return format, which is valuable given no output schema. However, it omits timeout behavior, exit-code/error-handling semantics, and the side-effect-free guarantee explicitly — gaps that annotations would otherwise cover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that packs the action, the exact command, and the output structure with zero filler. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters, no annotations, and no output schema, so the description must compensate. It compensates well for the missing output schema by spelling out error fields, but it leaves gaps: timeout_seconds is undocumented in both schema and description, the files/tsconfig interplay is unaddressed, and failure behavior is unstated. Adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 75%, so the schema already documents files, tsconfig, and project_dir. The description adds the 'tsc --noEmit' context, which clarifies how tsconfig and project_dir are consumed, but it doesn't explain timeout_seconds (the one undocumented parameter) or the files-overrides-tsconfig behavior. Value added is marginal over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Runs'), a specific resource ('tsc --noEmit'), and states the output shape (structured errors with file, line, column, error code, and message). This clearly distinguishes it from siblings like run_e2e_tests, npm_run, and nextjs_build, which target different workflows.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no when-to-use or when-not-to-use guidance, and never mentions sibling tools. Context is only implied by the tool's name and the 'tsc --noEmit' detail; an agent receives no explicit direction on choosing it over the build/test siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_sql_as_roleA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavior. It explicitly states the tool runs inside a ROLLBACK transaction and never mutates data—a critical safety guarantee. It also mentions setting JWT claims. It does not cover error handling or timeout behavior, but the core safety trait is transparently disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff: the first states what and why, the second highlights the safety behavior. The purpose is front-loaded, making it easy for an agent to quickly grasp the tool's value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (role switching, nested claims) and no output schema, the description covers purpose and safety well. It does not mention return values (e.g., query results) or prerequisites like a running database, but these are implied. Slightly more context on expected output would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 83%, so most parameters are already documented. The description adds no extra parameter meaning beyond the schema. The uncovered parameter (timeout_seconds) is not mentioned, but with high schema coverage, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (Executes), the resource (SQL query), and the purpose (verify RLS policies). It adds specificity with role and JWT claims, and the purpose distinguishes it from unrelated sibling tools like run_e2e_tests or typecheck.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied: 'so you can verify RLS policies' indicates when to use. The ROLLBACK safety note suggests it's safe for testing. However, it does not explicitly state when not to use it or mention alternatives, though siblings are irrelevant. The guidance is clear but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_e2e_testsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral disclosure. It transparently states that it starts a dev server if configured and returns structured results, which is the core behavior. However, it does not disclose potential side effects such as file modifications, server teardown, or environmental requirements like locally installed Playwright, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the main action and then details the conditional server startup and output format. There is no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a test runner with five parameters, the description covers the overall flow and return format sufficiently. The schema handles parameter details, and the description clarifies the conditional server startup. It lacks some environmental prerequisites, but overall it is adequate for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Because the input schema already provides descriptions for all five parameters, the baseline is 3. The description adds a little context by linking the dev server startup to the Playwright config, but it does not add new parameter-specific semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool runs Playwright tests and returns structured pass/fail results, with a clear verb and resource. It also mentions spinning up a dev server, which helps distinguish it from siblings like typecheck or npm_run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its usage by identifying Playwright E2E testing as the core function. However, it does not explicitly name alternatives or specify when not to use it, leaving some room for interpretation. The context is clear enough that an agent would know to use it for running E2E tests.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• npm_runA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly discloses that Vitest/Jest output is parsed into structured per-file results and falls back to raw output for non-test scripts, which shapes agent expectations. It omits side effects like dependency installation or exit-code behavior, but the core behavior is adequately revealed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose and output behavior, with zero redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description's explanation of structured output and raw fallback provides sufficient context for invocation. It could elaborate on error handling or exit codes, but the essential information for calling the tool correctly is present, and sibling tools are clearly separated by their purposes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 80% (4/5 parameters have descriptions), so the schema already documents the parameters well. The description adds no additional meaning beyond the schema, meeting the baseline for high coverage without compensating for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (runs) and resource (npm script) and explicitly differentiates from siblings like run_e2e_tests and typecheck by focusing on arbitrary npm scripts. The mention of parsing test output adds specificity, making it unmistakable which tool to use for unit tests.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the tool is for running any npm script, particularly tests, and distinguishes it from e2e, typecheck, and build tools. However, it does not explicitly name alternatives or state when not to use this tool, leaving exclusions to inference rather than explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.