Server Quality Checklist

Tool Scores

• search_endpointsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds useful behavioral context beyond annotations by clarifying that results are endpoint_id candidates only and that get_endpoint_schema should be called next, setting expectations about the return value and workflow.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler. The primary action is front-loaded, and the second sentence provides an actionable next step, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema, annotations, and schema-provided defaults, the description covers the main workflow adequately. It clearly states what the tool returns and what to do next, though it leaves some parameter semantics implicit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 40%, so the description needs to compensate for undocumented parameters like limit, include_hidden, and include_deprecated. It does not mention any parameter behavior or filtering semantics, leaving meaningful gaps for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find') and resource ('JustOneAPI endpoint candidates') and clarifies that the query is natural language. It also states the output nature ('Returns endpoint_id candidates'), which distinguishes it from siblings like get_endpoint_schema and call_endpoint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when you need to discover endpoint candidates from natural language. However, it does not explicitly state when not to use it or how it compares to siblings, aside from pointing to get_endpoint_schema as the next step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usage_summaryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful scoping context by specifying 'current JustOneAPI token' and the included trend information, but it does not disclose details like the exact time window or any operational limits. This is acceptable given the annotations carry the main behavioral burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose and then adds the key included items. Every word contributes value, with no repetition of the tool name or title beyond what is natural.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only tool with an output schema and strong annotations, the description is largely complete. It states the resource scope and the content of the summary. The only minor gap is that 'recent' is not precisely defined, but the output schema likely clarifies the returned structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially 100% and there are no parameter semantics to document. Per the rubric, zero parameters receives a baseline of 4; the description appropriately focuses on what the returned summary contains rather than parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: getting the current JustOneAPI token's API usage and spending summary. It explicitly mentions included content (recent call trends and spending trends), which clearly differentiates it from siblings like get_account_balance, which is about balance rather than usage/spending summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool should be used when an agent needs usage and spending summary information for the current token. However, it does not explicitly contrast it with alternatives such as get_account_balance or state when not to use it, so the usage guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_catalogA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already flag destructive behavior, and the description adds meaningful context by specifying admin-only access and the source of catalog data. It does not repeat or contradict the annotations, though it could mention that the active catalog is replaced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short front-loaded sentences place the most operationally important constraint (admin-only) first and add a useful source detail without filler. Everything else is left to the schema and annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With full parameter documentation, an output schema, and destructive annotation present, the description is nearly sufficient. It lacks explicit side-effect context about replacing the active catalog, but rollback behavior is captured in the parameter schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions, including the force compatibility caveat and rollback semantics. The description itself adds no parameter detail, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States an explicit verb (Refresh), a clear resource (endpoint catalog), and a source (JustOneAPI OpenAPI documents). The admin-only scoping further distinguishes it from read/query siblings like search_endpoints and call_endpoint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The only usage guidance is the admin-only precondition; it does not state when to refresh or explicitly contrast with siblings such as 'use search_endpoints to query the catalog.' Usage is implied rather than fully specified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_endpoint_schemaA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds no behavioral details beyond retrieving schema, but nothing more is needed given the strong annotation coverage and presence of an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words. It efficiently conveys the action, the resource, and the source of the required parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter, read-only lookup tool with a complete input schema, strong annotations, and an output schema, the description provides sufficient context. It also ties the parameter to its producer tool, making invocation straightforward.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema already documents endpoint_id as 'Endpoint id returned by search_endpoints.' The description essentially repeats this relationship without adding new parameter meaning, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource: the full schema and parameter contract for an endpoint_id. It also references the sibling tool search_endpoints as the source of the endpoint_id, distinguishing this lookup operation from call_endpoint and other siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes it clear the endpoint_id must come from search_endpoints, which implies the correct sequencing. It doesn't explicitly mention not to use call_endpoint for this purpose, but the context plus the tool name make the intended use unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_balanceA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool readOnly, idempotent, and non-destructive, so the safety profile is covered. The description adds the semantic detail that the balance indicates whether the token can continue calling APIs, which is useful context, but it does not disclose any additional behavioral traits such as freshness, rate limits, or response shape. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences deliver the function first and usage triggers second, with no filler or repetition. Every clause earns its place; the description is short but information-dense.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless, read-only tool with an output schema and full annotation coverage, this description is complete. It names what is returned (balance and currency), explains when to invoke it, and addresses the likely ambiguity of 'balance' by linking it to continued API calling. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema declares zero parameters, so there is no ambiguity to resolve. The schema description coverage is 100% by construction, and the description correctly avoids inventing parameters. Baseline of 4 is appropriate for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') and identifies the exact resource: the current JustOneAPI token's available balance and currency. This clearly distinguishes it from siblings like call_endpoint or get_usage_summary, none of which target account balance. The stated use cases reinforce the tool's identity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states explicit trigger conditions: when the user asks about account balance, remaining balance, or whether the token can continue calling APIs. This gives clear when-to-use context. However, it does not mention when not to use it or alternative tools, so it stops short of full routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• call_endpointA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations already signal readOnlyHint=false and destructiveHint=true, but the description adds meaningful behavioral context: the call may incur charges under the bound API Token's pricing and budget. It also clarifies that exactly one call is made, which is useful given openWorldHint=true. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the main action is stated first, followed by the charge warning, then the parameter naming guideline. Every sentence earns its place with no filler or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a dynamic endpoint-calling tool, the description covers the essential operational concerns: validation, single-call behavior, charge risk, and parameter source. The output schema exists, so return-value details are not required here. The reference to get_endpoint_schema fills the main knowledge gap for constructing valid calls.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so by explaining that params should use snake_case names from get_endpoint_schema, directly addressing the opaque 'params' object. This is essential guidance for an otherwise undocumented parameter structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair: 'make one JustOneAPI endpoint call by endpoint_id.' It clearly distinguishes call_endpoint from the sibling tools (search_endpoints, get_endpoint_schema, get_account_balance, etc.) by stating the action is an actual endpoint invocation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when you need to execute an endpoint call. It also gives an important usage prerequisite: 'Params should use snake_case names from get_endpoint_schema,' which routes the agent to a sibling tool for parameter discovery. It does not explicitly list when-not-to-use scenarios, but the context is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_platformsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool as readOnly, idempotent, and non-destructive. The description adds useful context about what the response covers (supported platforms and endpoint counts). There is no hidden mutation or side-effect risk, and the description does not contradict the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single focused sentence that states the action, the object, and the key output detail. Every word earns its place, and the core purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has no parameters, is a safe read-only operation per annotations, and has an output schema available, the description is fully sufficient. An agent has everything needed to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description does not need to explain parameter behavior because there are none to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') with a clear resource ('supported JustOneAPI platforms') and adds the distinctive result content ('endpoint counts'). This is clearly distinct from sibling tools like search_endpoints or get_endpoint_schema, which operate on endpoints rather than platforms.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the use case clear: an agent would call this when it needs the set of supported platforms and their endpoint counts. While it does not explicitly name alternatives or exclusions, the purpose is unambiguous and no sibling tool overlaps with this listing function.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.