Server Quality Checklist

Tool Scores

• get_solveA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark it read-only and open-world, but the description adds valuable behavioral context: it is free and never re-bills, and a 404 is scoped to the current API key's workspace. This goes beyond structured metadata without contradicting it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no filler. The main purpose is front-loaded, followed by a concrete use case and an error interpretation; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter retrieval tool with an output schema and safety annotations, the description fully covers when to call it, what the id means, cost implications, and error semantics. Nothing essential is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the only parameter, id, is already clearly described with a format example. The description adds the useful hint that the id may come from a timeout error, but does not need to elaborate further since the schema carries the parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Fetch a solve... by id') and distinguishes itself from siblings by describing exactly when it is the right tool. The mention of solve_turnstile and 404 behavior makes its role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells the agent when to use this tool: after solve_turnstile times out while the solve is pending, with the id coming from that error message. It also interprets a 404 response, giving clear operational context beyond the schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_pageA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint, it adds meaningful behavioral specifics: it performs an HTML inspection rather than solving a challenge, requires no API key, is free, and has a specific hard limitation around widget mode. These details substantially describe side effects and boundaries beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although the description is long, each clause delivers a distinct, useful fact: what is fetched, what is reported, cost/auth requirements, usage timing, and a critical limitation. The most important action and resource are front-loaded, and no sentence is slack.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single required parameter, an output schema, and annotations, the description covers everything an agent needs: when to call it, what it returns, its constraints, and what it cannot infer. The mitigation of return format details is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the single 'url' parameter with an example, so parameter documentation coverage is 100%. The description does not add much beyond the schema, but the schema already carries the necessary semantic weight. This aligns with the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Fetch') and a concrete result ('report what its HTML says about Cloudflare Turnstile') and enumerates exact outputs: widget presence, sitekey, render method, data-* configuration, test keys, and full-page WAF challenge. This clearly separates it from sibling solving/result-retrieval tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Use this before solving when the sitekey is unknown,' giving a direct invocation condition. It also provides an exclusion: it cannot determine the widget's mode because that data is not in the page, so agents know when this tool is not sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solve_turnstileA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate non-read-only and non-idempotent behavior, and the description adds meaningful operational context: tokens are single-use and expire in minutes, failed solves are never billed, SOLVEGATE_API_KEY is required, and sitekeys are bound to hostnames. This goes well beyond the annotations and clarifies important side effects and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences with no filler: purpose is front-loaded, then parameter guidance, then token lifecycle and error/cost behavior. Every sentence adds useful information and the structure is easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema and useful annotations present, the description supplies the critical missing context: token expiry timing, no-billing on failed solves, API key requirement, and hostname binding. Nothing an agent needs to invoke this tool correctly appears to be missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds extra semantic value by explaining the `gate` enum values, reinforcing that `sitekey` and `url` are both required, and explaining why `url` must be the actual rendering page. Some parameters like `proxy` and `action` receive no additional description, but the schema already documents them adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pair ('Clear a Cloudflare challenge and return a token') and then defines the two `gate` modes, clearly distinguishing an embedded widget from a full-page interstitial. This makes the tool's purpose immediately understandable and separates it from generic 'solve' tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives actionable selection guidance: it explains when to use 'turnstile' versus 'waf', and instructs the agent to request tokens at submission time due to expiry. It does not explicitly name sibling tools as alternatives or state when not to use this tool, so it stops short of full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.