Server Quality Checklist

Tool Scores

• get_tocB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It states that the tool extracts the TOC and returns hierarchical structure with page numbers, which implies a read-only operation and provides some output expectations. However, it does not disclose what happens when a PDF has no bookmarks, whether it scans the text if bookmarks are absent, or any error behavior. This is a meaningful gap for a tool with no annotation safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly worded sentence that front-loads the core action (extract TOC) and immediately gives the key output detail (hierarchical structure with page numbers). Every part adds value, and there is no repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one well-documented parameter and no output schema, the description adequately conveys the purpose and the general shape of the return value. It does not mention edge cases like PDFs without bookmarks, but the core usage scenario is clear enough for an agent to make a confident call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the only parameter, file_path, already clearly documented as 'Absolute path to the PDF file'. The tool description adds no additional meaning or nuance to the parameter. Baseline score of 3 is appropriate since the schema handles this dimension completely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb 'Extract' and the resource 'table of contents from a PDF', and it states the return value as a hierarchical chapter structure with page numbers. However, it does not explicitly distinguish itself from the sibling tool list_chapters, which may perform a similar function. The level of specificity is high, but lacking direct sibling differentiation prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance about when to use this tool versus its siblings. The description neither mentions alternatives nor provides context such as 'use this for embedded bookmarks' or 'prefer list_chapters when only chapter names are needed.' The agent is left to infer appropriate usage from the name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_chaptersA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It does disclose meaningful behavior: 'Uses TOC if available, otherwise splits by pages' and lists return fields. However, it does not mention side effects, permissions, failure modes, or the implications of the optional level parameter, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly written sentences with no filler. The first sentence states the primary action and the second explains the fallback behavior and return fields. Everything included is relevant and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description defines the tool's core function and return content, but it lacks guidance on sibling tool distinctions and has no annotations. Given the sibling set includes get_toc and detect_headings which may overlap in purpose, the description is not fully sufficient for correct selection in all cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both file_path and level. The description adds no additional parameter-level guidance beyond the schema; it merely restates the overall goal. Baseline 3 is appropriate because the schema handles the details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'List chapters/sections detected from the PDF.' It also distinguishes itself from siblings like get_toc and detect_headings by describing the output as chapter index, title, page range, and estimated line count. This is unambiguous and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not state when to use this tool versus alternatives such as get_toc, detect_headings, or read_chapter. It implies usage through the resource type but offers no explicit context, exclusions, or routing guidance. An agent would have to infer the appropriate tool from vague naming and behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pdf_infoA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. The verb 'Get' implies a read-only operation and the scope is limited to metadata, which is useful, but the description does not explicitly state that the file is not modified or that page content is not extracted. There is no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It states the action and the returned metadata fields in minimal space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description provides the essential return fields, which gives the agent a good sense of the result. It slightly lacks an explicit statement about read-only behavior, but this is largely inferable from the verb 'Get' and the nature of metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for its single parameter, file_path, which is clearly documented as 'Absolute path to the PDF file'. The tool description adds no additional parameter semantics, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' with the resource 'PDF metadata' and lists concrete fields (title, author, page count, file size). This clearly distinguishes it from content-oriented siblings like get_toc, read_chapter, and read_pages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The implied usage is that this tool is for document-level metadata rather than content extraction, but no explicit when-to-use or when-not-to-use guidance is given. It does not name alternatives or exclusion conditions, leaving the choice to the agent's inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_chapterA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that content is automatically truncated to fit context limits, which is a distinct behavioral trait and useful warning. It does not disclose whether truncation affects the output format or whether the full content is unavailable due to truncation, leaving some ambiguity about result completeness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, both purposeful: the first states the action and source, the second warns about context truncation. It is front-loaded with the primary purpose and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For reading a chapter, it explains the truncation behavior and source of indices, which is sufficient for a simple read operation. It does not mention the return format or how truncation manifests in the output, and with no output schema the description could have added a bit more on what the agent should expect back. Still, the tool is narrow in scope and the essentials are covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already has 100% coverage on parameters, so the baseline is 3. The description adds little beyond referencing list_chapters and the level-matching constraint, which is already in the schema. That connection is valuable but does not dramatically elevate semantic clarity beyond schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it reads text content of a specific chapter by index, with the source from list_chapters. The verb 'read' and resource 'chapter' distinguish it from siblings like get_pdf_info or read_pages, and the reference to list_chapters anchors its use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Confirms the chapter_index comes from list_chapters and that the level must match the level used in list_chapters, which guides correct invocation. It does not explicitly list when not to use it versus read_pages or when to prefer alternatives, but the context is clear given the sibling set.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detect_headingsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It discloses that results are heuristic 'candidates' derived from font sizes, which is useful. It does not describe output shape, limitations, or side effects, but for a read-like detection tool this is a reasonable level of transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The first sentence gives the action and method, and the second gives the practical use case. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description is largely complete: it explains what the tool does, how it does it, and when to use it. The main gap is that it does not hint at what the returned heading candidates look like, but this is not essential for selecting and invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameter is already well documented as an absolute PDF path. The description adds no new parameter-level detail beyond repeating the PDF context, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Detect'), a specific resource ('heading candidates'), and the method ('analyzing font sizes'). It also distinguishes itself from TOC/bookmark-based tools by explicitly targeting PDFs without TOC/bookmarks, which separates it from siblings like get_toc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear when-to-use guidance: use it for PDFs without a TOC/bookmarks, before reading, to understand document structure. However, it does not explicitly name alternative tools or state when not to use it beyond the 'without TOC/bookmarks' condition.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_pagesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It discloses the core behavior (reads text) and implies a non-mutating operation through the word 'Read', but it does not mention return format, error handling, or page-boundary behavior. It is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. The first sentence states the purpose clearly, and the second provides usage guidance. The content is front-loaded and every word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with three fully documented parameters and no output schema, the description plus schema supplies the essential information: what the tool does, when to use it, and the page-range arguments. The lack of return-format details and edge-case behavior is a minor gap, not a serious omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all three parameters (file_path, start_page, end_page) already described in the schema. The description only mentions 'page range' and adds no extra meaning beyond what the schema provides, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Read'), a clear resource ('text content from a specific page range'), and scope. The phrase 'when chapters are too large' implicitly differentiates this from sibling chapter-level tools like read_chapter, making it easy for an agent to distinguish the operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Use this for fine-grained control or when chapters are too large.' It clearly implies an alternative (chapter-based reading) and provides context for selecting this tool, though it doesn't explicitly name the sibling or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.