Server Quality Checklist

Tool Scores

• searchB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says the tool 'Returns movies, shows, episodes, and more,' which is minimal. It does not mention whether the operation is read-only, whether authentication is required, how matching works, or behavior on empty/no results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant filler. Every word contributes to the core meaning, though 'and more' is slightly vague. It is concise and appropriately scoped for a simple search tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter search tool, the description plus input schema provide a minimally viable picture: a required query and an optional limit with default. However, it lacks information about query syntax, search scope details, and when to prefer a sibling tool. The presence of an output schema reduces the need to explain return values, but usage and parameter guidance remain thin.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the schema's silence. It adds some meaning by indicating the query is a title search, but it does not explain the 'limit' parameter, acceptable query formats, search behavior, or result ordering. The description only partially clarifies the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search') and resource ('the Jellyfin library') and clarifies that search is by title. It also indicates the result types (movies, shows, episodes), which helps distinguish it from sibling tools like info, sessions, libraries, and next_up. It is clear, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when an agent needs to find library items by title. However, it provides no explicit guidance about when not to use it or which sibling tool might be a better fit for non-title searches. Usage context is inferable but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• next_upB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It communicates a non-mutating 'list' operation and the scope of results, which is basic transparency. However, it does not disclose how 'next' is determined, whether the result is one episode per show, or any session/authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no redundant words. Every part of the description contributes meaning, making it an excellent example of concise tool documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and has an output schema, so the description covers the core purpose adequately. However, the complete absence of parameter guidance and usage alternatives leaves gaps that an agent would need to resolve through inference or trial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention the 'limit' parameter at all. Since the description was expected to compensate for the schema's lack of parameter documentation, this is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a precise resource ('the next unwatched episode for shows you are following'). This clearly distinguishes it from siblings like search, info, sessions, and libraries, none of which cover this specific function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states what the tool does but gives no guidance on when to prefer it over alternatives, nor any exclusions or context about typical scenarios. The intended use case is only implied by the description itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infoA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It clearly states the operation is a metadata retrieval, but it does not mention behavior on invalid IDs, authentication requirements, or what 'full metadata' includes. The output schema covers return shape, so the gap is moderate rather than severe.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that communicates the action, target, and required identifier with no wasted words. Every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get-by-ID tool with one required parameter and an output schema, the description covers the essential invocation context. It does not discuss error handling or exclusions, but given the low complexity and existing output schema, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does by clarifying that item_id is the Jellyfin item ID and that it identifies a single library item, adding meaning beyond the schema's minimal 'Item Id' title. It does not specify format details, but for a single simple parameter this is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb ('Return') with a clear resource ('full metadata for a single library item') and the exact identifier ('Jellyfin item ID'). It clearly differentiates from siblings such as search, sessions, libraries, and next_up, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the appropriate use case: call this when you already have a Jellyfin item ID and need full metadata. However, it does not explicitly state when not to use it or mention alternatives such as search for finding IDs first.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sessionsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'List' clearly signals a read-only operation, and the scope is explicitly defined. However, it does not disclose potential limitations such as filtering scope, real-time freshness, or behavior when no sessions are active.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no filler. The verb is front-loaded, and the descriptive qualifiers add meaning without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple parameterless listing tool with an output schema available, so the description does not need to document return values. It provides enough context to invoke the tool, though slightly more usage context would improve it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100%, so there is nothing for the description to explain. The baseline of 4 applies because parameter semantics are irrelevant here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('List') and a specific resource ('active playback sessions'), then adds useful detail about what is included: who is watching, what they are watching, and on which device. This clearly distinguishes it from sibling tools like search or info.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The wording implies use when an agent needs current playback session information, but it does not explicitly say when to prefer this over sibling tools or when not to use it. No alternatives or exclusions are mentioned, leaving some selection to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• librariesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the behavioral burden. It clarifies that only top-level libraries are returned, avoiding confusion with nested items. The verb 'List' implies a read-only, non-destructive operation. No side effects or permissions are mentioned, but none are needed for such a straightforward listing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence. It includes a specific verb, the target resource, and clarifying examples without any filler. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-argument tool with an output schema present, the description is functionally complete. It defines the operation and scope clearly. It could optionally mention that the returned libraries can be used as inputs to search or info, but this is not necessary for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, and schema description coverage is 100% (vacuously). Per the baseline for 0-parameter tools, a score of 4 is appropriate because there are no parameter semantics to add.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and a clear resource ('top-level media libraries') with illustrative examples (Movies, TV Shows). It is immediately distinguishable from sibling tools like search, info, sessions, and next_up, which all describe different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes it clear that this tool is used when you need the list of top-level media libraries. There is no explicit mention of alternatives or when-not-to-use, but the imperative phrasing and resource specificity provide clear context without ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.