Server Quality Checklist

Tool Scores

• agent_disclosure_taxonomyA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only, non-destructive, and closed-world. The description adds a useful behavioral signal, 'Offline,' indicating no live retrieval, and clarifies the scope includes both positive dimensions and negative exclusions. This goes beyond annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one dense, front-loaded sentence with useful exclusion examples and no filler. The standalone 'Offline' fragment could be integrated more smoothly, but the overall length and organization are efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter, read-only reference tool, the description adequately conveys what the taxonomy contains: six disclosure dimensions, negative scope, and illustrative excluded items. It does not state the return format, but the static, offline nature makes this a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100%, so there is nothing for the description to add about parameter meaning. Per the baseline for zero-parameter tools, a 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names concrete content: the six Article 50(1) disclosure dimensions and the negative scope of what is not covered, with specific examples. It is clearly a taxonomy/reference tool, distinguishable from siblings like classify or explain_obligation, though it lacks an explicit action verb like 'returns'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus the sibling tools such as explain_obligation, generate_disclosure, or discover_ai_surfaces. The content implies a reference lookup, but no conditions, exclusions, or alternatives are stated, leaving the agent to infer the appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_agent_disclosureA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover readOnlyHint and destructiveHint, so the description adds real value by disclosing three behavioral traits: conformity to the Article 50 Guidelines, refusing rather than fabricating a principal, and operating offline. These go beyond what annotations or the schema express, though the description does not detail refusal output or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first sentence states the core purpose, the second establishes regulatory provenance, and the final fragments add decisive behavioral constraints without filler. Every sentence earns its place, and no content unnecessarily duplicates schema fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the detailed schema for all five parameters and annotations that establish a read-only, non-destructive profile, the description supplies enough contextual behavior for an agent to call the tool correctly. The main gap is the absence of an output schema combined with no explicit statement about what the returned disclosure artifact looks like, but for a drafting tool this is a modest omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3 even without additional parameter prose. The description loosely aligns with parameters via 'what it is, who it acts for, and limits of delegated authority,' but it does not add parameter-level detail beyond the schema, which already covers ask-the-user behavior and null handling.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb ('Draft'), a precise regulated artifact ('Article 50(1) disclosure an AI agent owes'), and the three core content elements: what it is, who it acts for, and the limits of delegated authority. It is clearly not a tautology, and its legal specificity helps separate it from generic disclosure tools, though it does not explicitly contrast with the sibling 'generate_disclosure'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the intended scenario: use this when drafting an AI agent's Article 50(1) disclosure. It also communicates an important boundary through 'Refuses rather than inventing a principal.' However, it does not explicitly state when not to use the tool or how to choose between this and sibling tools such as 'generate_disclosure'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_recordA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds behavioral context beyond annotations by stating the tool is 'Online' and performs a network read, and it restricts to 'published' records, which gives helpful operational expectations. There is no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core purpose and followed by a useful behavioral note. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read-only tool with strong annotations and full schema coverage, the description is nearly complete. It communicates the resource, the input, and the network behavior. It does not detail the return format, but that is not strictly necessary given the simplicity and the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema documentation covers 100% of the parameter meaning, with detailed description for 'slug' including format and example. The main description merely repeats 'by org slug', adding no additional semantic value beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Fetch') and a specific resource ('published Legalithm Trust Center compliance record') with the required input ('by org slug'). This clearly distinguishes it from sibling tools like classify or generate_disclosure, which imply different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context ('Online (reads the public API)') and implies this is for retrieving published records, but does not explicitly state when to use this tool versus alternatives or when not to use it. No exclusions or alternative tool names are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• classifyA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds valuable behavioral context: 'Offline' (no network calls), 'checked against Regulation (EU) 2024/1689' (authoritative source), 'not legal advice' (limitation disclaimer), and it describes the output as 'risk tier + cited rationale'. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence starting with the verb and resource, followed by two terse context clauses. Every element earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with no output schema, the description covers the return contract (risk tier + rationale), the legal basis, offline behavior, and disclaimer. Combined with the comprehensive schema descriptions, it is largely complete; the only gap is explicit sibling differentiation, which is covered under usage guidelines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter having thorough meaning including enum semantics and special guidance (e.g., domain breadth, audience selection, use_case phrasing). The tool description itself adds no further parameter information, but the schema carries the full burden, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'classify' with a clear resource ('an AI use case under the EU AI Act') and outcome ('risk tier + cited rationale'). It is immediately clear what the tool does and is distinct from sibling tools like explain_obligation or generate_disclosure.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage whenever classification under the EU AI Act is needed, but does not explicitly state when to use this tool instead of siblings like explain_obligation or check_record. The 'Offline' and 'checked against Regulation' context hints at behavior, but there are no exclusions or alternative references, so guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_disclosureA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the read-only nature is known. The description adds the 'Offline' trait, which is useful context, but it does not describe the output format, any prerequisites, or other behavioral details. With annotations covering safety, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core action, the categories, the languages, and a key behavioral trait ('Offline'). Every element earns its place with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple tool with only two parameters, both fully described in the schema. The description succinctly states the tool's purpose and adds the 'Offline' detail. It does not explicitly mention the return format, but for a 'snippet' generator this is largely implied and not a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and both parameters (locale and scenario) have enums with detailed descriptions explaining the relevant Article 50 duties. The tool description itself adds no parameter-specific meaning beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Generate') and specifies the exact resource ('Article 50 transparency disclosure snippet'), plus the relevant categories (chatbot, genai-content, deepfake, emotion) and languages (EN/DE). This clearly distinguishes it from sibling tools like classify or explain_obligation, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use case by naming the disclosure types and languages, and the parameter schema further clarifies when each scenario applies. However, it does not explicitly contrast with sibling tools (e.g., explain_obligation) or state when not to use it, so it lacks explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_ai_surfacesA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Goes well beyond the readOnly/destructive annotations by disclosing that outputs are hypotheses, not findings, and that the tool is offline with nothing stored. This is valuable behavioral context that prevents misuse and over-interpretation of results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences with no filler. The most important behavioral caveat (hypotheses, never findings) is prominent, and the operational constraints (offline, no storage) are packed efficiently into a short closing phrase.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers input expectations, analysis flow, evidence logic, output status, and operational constraints at a high level. Since there is no output schema, a bit more detail on the shape of returned limbs/dates/duty-bearers would improve completeness, but the essentials are present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% description coverage, so the schema carries the parameter meaning. The description reinforces the files parameter and the possible-vs-likely evidence distinction, but does not add new semantic information about deploys, own_brand, or the date flag beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states a specific action (read file contents, propose AI capabilities, resolve into Article 50 limbs) and names the concrete input types. It does not explicitly differentiate itself from siblings like classify or explain_obligation, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on when to use the tool: when the user can supply file contents and wants AI-capability analysis mapped to Article 50 obligations. It does not mention alternative sibling tools or exclusion cases, so guidance on choosing among siblings is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_obligationA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the read-only/destructive annotations, the description adds that every obligation is returned with its Article citation and that the tool is offline. There is no contradiction with annotations, and no destructive or open-world behavior is hidden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences carry the core purpose, optional-data behavior, and the offline constraint, with no filler. The most important information is front-loaded before the optional parameter note.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description states the response shape (obligations with Article citations, plus competent authority when country is passed). Combined with the richly documented input schema and read-only annotations, nothing essential is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents role, risk, sector, and country in detail. The description's mention of passing country/sector to get the regulator restates the schema rather than adding new parameter meaning; baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a precise resource ('EU AI Act obligations for a role + risk tier'), and explicitly notes the Article citations and optional competent-authority output. This clearly differentiates it from siblings like classify (risk scoring) and generate_disclosure (document generation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear conditional usage for country and sector ('Pass country (and sector) to also get the competent national authority') and notes the tool is offline. It does not explicitly state when not to use it or name a sibling alternative, but the required role + risk inputs and optional extensions give a usable selection context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.