Server Quality Checklist

Tool Scores

• infobroker_get_citationsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint. The description adds meaningful behavioral context: it can operate without an API key under a certain condition and returns a JSON envelope prefixed with [OK] or [ERROR]. It doesn't go into error scenarios or rate limits, but it adds value beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each earning its place: output definition, usage guidance, exclusions with alternatives, and auth/response format. The most important information is front-loaded, with no redundant or vague phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with two parameters and no output schema, the description covers the return format, citation fields, auth condition, and routing to alternatives. Nothing essential is missing for an agent to decide whether to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters (query, max_results). The description does not add extra parameter-level meaning beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Return academic references for a query as BibTeX citations with title, authors, year, venue, and URL.' It also explicitly distinguishes itself from sibling tools by excluding general web search and claim verification, so an agent can uniquely identify this tool's purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states when to use ('Use when scholarly writing needs a reference list') and when not to use it, naming alternatives ('use web_search', 'use verify_claims'). Minor deduction because the alternatives are shortened names rather than the exact sibling tool names (infobroker_web_search, infobroker_verify_claims), which could cause an agent to look for a nonexistent tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infobroker_manage_kbA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already flag destructive behavior, and the description adds crucial specifics: delete cannot be undone, a lost encryption key makes the store unrecoverable by design, and all responses are wrapped in a [OK]/[ERROR] JSON envelope. This goes meaningfully beyond the annotation flags and matches them without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core operation list and every sentence serves a purpose: usage, exclusions, destructive warnings, parameter routing, and output format. It is dense and moderately long, but this is justified given 15 parameters and multiple actions; still, the density requires careful parsing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a high-complexity tool with no output schema, the description covers operations, when to use them, forbidden uses, destructive consequences, encryption specifics, parameter mapping, and the response envelope. It does not specify the exact response shape for list/stats/get, and the source_url/url naming inconsistency remains a small gap. Overall, quite complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds useful action-to-parameter routing (query for search, text for ingest, source_url for get/delete) and report-ingest guidance, but it also references a 'url' parameter for ingest that does not actually exist in the schema—ingest URLs are handled by source_url. This minor inaccuracy undercuts the added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description begins with a specific verb ('Manage') and a clear resource ('local knowledge base'), then enumerates the exact operations: search, ingest, list, get, stats, delete, and encryption. It distinguishes itself from external-tool siblings by explicitly scoping to 'local' content and later naming web_search and fetch_page as the external alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'Use when...' guidance for archiving reports, revisiting stored content, and managing encryption keys. It also gives a clear 'Do NOT use for...' exclusion with named sibling tools (web_search and fetch_page), so an agent knows when to route elsewhere.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infobroker_verify_claimsA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the sparse annotations: it discloses multiple searches bounded by max_iterations, provider rate limits, knowledge base recall and indexing side effects, and the [OK]/[ERROR] envelope. This meaningfully informs an agent about cost, non-idempotence, and side effects, and does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but mostly purposeful, front-loading the core function and return value before usage guidance and limits. A minor redundancy exists between 'multi-pass' and 'Makes multiple searches,' but overall every sentence contributes meaningful decision or behavioral context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, but the description covers return format, confidence scoring, source attribution, side effects, and failure prefixing. For a complex, multi-call tool with no structured output definition, this is sufficient for an agent to understand what it will receive and what the tool will do.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds interpretive context for max_iterations and confidence_threshold, but does not add meaning beyond the schema for query, priority, or providers. This is adequate but not exceptional.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a rich, specific operation: a multi-pass truth-finding loop that searches providers, reconciles claims, and returns confidence-scored, source-attributed findings. This clearly distinguishes it from simpler sibling search or citation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (high-stakes or contested claims needing agreement, contradiction, gaps) and when not to use simple lookups or citation formatting, naming web_search and get_citations as alternatives. This is ideal routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infobroker_fetch_pageA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already state readOnly, openWorld, and idempotent hints. The description adds meaningful behavioral context beyond that: external HTTP calls, Jina throttling fallback to native HTTP, truncation of long pages, no API key required, and the '[OK]'/'[ERROR]' JSON envelope. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four dense sentences, each earning its place: core function, renderer details, usage/exclusions, behavioral notes. Information is front-loaded with the main purpose, then routed to alternatives, then caveats. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple renderers, question mode, date detection, parallel fetches, fallback behavior, output envelope) and absence of an output schema, the description covers all major behavioral aspects an agent needs to call correctly. Nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds slight value by clarifying that question mode returns ranked passages and mentioning detect_date, but these are also present in the schema. It does not deeply enrich parameter understanding beyond what the schema already conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pair ('Fetch a URL and extract clean content') and immediately distinguishes itself from siblings by naming web_search and verify_claims as alternatives. An agent can tell exactly what this tool does and what it is not for.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'Use when' conditions are given (need readable text, ask a page a question, detect a last-updated date) and explicit 'Do NOT use' exclusions with named alternatives (web_search, verify_claims). This is textbook guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infobroker_inspect_providersA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, and the description reinforces read-only behavior and states it never modifies configuration, providers, or stored data. It adds useful detail about the JSON envelope prefix ([OK] or [ERROR]) and the meaning of actions, exceeding what annotations alone convey, though it does not discuss authentication or external side effects of a live health check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose, then flows logically into usage guidance, exclusions, safety, and parameter specifics. Every sentence earns its place, with no filler or repetition of the title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only inspection tool with three well-covered parameters and no output schema, this description is complete. It explains all three actions, the return envelope, conditional parameter requirements, and both positive and negative usage cases. An agent has enough context to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already explains each parameter. The description adds meaningful relationships: provider is required for the health action and status filters list. This goes beyond the schema's generic field descriptions by clarifying conditional usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Inspect') and resource ('configured search providers'), then enumerates the concrete actions (list state, live health check, build/spec identity). It explicitly contrasts with sibling tools, naming web_search and fetch_page as the tools this is not, so an agent can distinguish it without inspecting other schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use context: when searches return empty or slow results and provider status, quota, or latency matter, or when choosing a trusted backend. It also provides a clear 'Do NOT use' directive with named alternatives, leaving no ambiguity about routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infobroker_reload_configA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description reveals key behaviors beyond annotations: active connections are preserved, an invalid new configuration leaves the previous one active and returns an error, and the response is a JSON envelope prefixed with [OK] or [ERROR]. These are all useful operational details that the empty schema and annotations do not capture. There is no contradiction with readOnlyHint=false or idempotentHint=true.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four focused sentences: main function and effect, when to use, when not to use with alternative, and failure behavior/return format. Every sentence carries distinct, valuable guidance, and the most important operational facts are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no input parameters and no output schema, the description is complete: it explains what the tool does, when to invoke it, the alternative, the failure safety, and the response envelope format. An agent has everything needed to call and interpret the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are trivially complete; the schema already covers everything with an empty properties object. The description adds no parameter-specific detail because none is needed. This aligns with the 0-parameter baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Re-read') and resource ('configuration file') and clearly states what changes are applied (provider, rate-limit, knowledge-base) without restarting. It also explicitly distinguishes itself from the sibling inspect_providers by saying what it is NOT for, so an agent can select it correctly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states precisely when to use the tool ('when you have edited config.json or config.local.json and want the changes applied immediately') and when not to use it ('Do NOT use to inspect configuration or provider state'), naming the alternative inspect_providers. This leaves no ambiguity about tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infobroker_web_searchA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Goes beyond the readOnly/openWorld/idempotent annotations by disclosing automatic provider selection, fallback chain, local caching, per-provider rate limits, no API key requirement, and the [OK]/[ERROR] JSON envelope. These details accurately describe observable behavior without contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core function, then gives usage conditions, alternatives, behavioral notes, and return format in a compact paragraph. Every sentence contributes distinct information with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 12-parameter tool with no output schema, the description covers return envelope, modes, cachiing, rate limits, authentication, and sibling routing. The completeness is high enough for an agent to select and invoke the tool correctly without opening the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Though schema coverage is 100%, the description adds semantic meaning beyond the schema by mapping modes to parameters: batched search for query arrays, suggest for autocomplete, expand for query expansion, and deep for ranked passages. It also connects automatic provider selection to provider/priority routing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific action and resource: search web, encyclopedia, academic, and code sources through one interface. It further differentiates from siblings by explicitly naming what it is not for (URLs, high-stakes verification, stored reports, academic citations), making purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit positive conditions ('Use when you need broad or batched search...') and explicit exclusions with named alternatives ('Do NOT use for a URL... use fetch_page'). This gives an agent clear decisional criteria for routing to the correct sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.