Server Quality Checklist

Tool Scores

• room_infoA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral disclosure burden. It does disclose that the operation retrieves readable info and defines the default behavior when chat_id is omitted, which is useful. However, it does not explicitly state that the operation has no side effects, what happens on invalid room IDs, or any permission requirements — gaps that matter for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The first sentence states the tool's purpose and outputs, the second gives the use case and default behavior. Critical information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple getter with one optional parameter, the description is mostly complete: it lists return fields, the use case, and the fallback behavior. Since there is no output schema, describing the expected fields is important and done here. Minor omissions around error/not-found behavior prevent a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value beyond the schema by clarifying that 'recent conversation' specifically means the most recently forwarded room, and by explaining why chat_id is optional in terms of the mapping use case.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a clear verb ('获取' / get) and resource ('指定 Matrix 房间'), and lists the exact data returned (name, topic, member nicknames and IDs). It does not explicitly contrast itself with siblings such as list_rooms or room_members, but the resource scope and output fields are specific enough to distinguish it without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states a concrete use case: configuring and checking the 'room ID ↔ name' mapping. It also explains the optional chat_id behavior (fallback to the most recently forwarded room), which guides when to call the tool without an argument. No alternatives or when-not-to-use conditions are mentioned, hence not a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_approvalA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains that the task is escalated, that it waits for approval, and that it will be replayed and executed after the manager replies 'approve'. It does not mention rejection behavior, but the core lifecycle is clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence that front-loads the trigger condition, then states the action and outcome. Every part is informative and there is no redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single required parameter and no output schema, the description covers the key context: when to call, what happens, and what triggers execution. It does not specify behavior on rejection, but the overall guidance is sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers the single parameter 'reason' with 100% coverage, including its meaning as the approval reason or plan summary. The description adds no further parameter-level detail, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific action: escalating the current task to a manager control room for approval. It also gives the triggering condition ('unfamiliar source or high risk'), which distinguishes it from the sibling tools like reply, health_check, and list_rooms.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: when the task is from an unfamiliar source or carries high risk. It does not explicitly list alternatives or exclusions, but the condition is clear enough that an agent can decide appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• room_membersA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  无 annotations，描述承担行为说明职责。它说明了工具是只读查询、返回成员列表并解释 chat_id 缺省行为，但未披露任何可能的权限要求、失败情形或数据即时性等行为特征。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  两句话完整说明用途、返回值与缺省行为，信息密度高且无冗余，属于精炼的合格描述。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  对单参数只读列表工具，描述已说明返回字段、缺省行为和使用场景；缺少输出结构样例或错误条件，但整体足以让 agent 正确调用。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  参数仅有 chat_id，schema 已包含描述；描述额外说明了省略 chat_id 时的默认行为（取最近对话房间），为参数语义增加了超出 schema 的价值。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述使用明确动词「获取」与资源「指定 Matrix 房间（群聊）的在场成员列表」，并具体说明返回 user_id 与 display_name，足以与 siblings 如 list_rooms、room_info 区分。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  描述说明 chat_id 省略时取最近转发进来的房间，提供了明确的使用上下文；但未明确说明何时不使用本工具（例如需要房间详情时用 room_info），因此未达满分。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_roomsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and discloses the scope (only rooms the avatar has joined) and the output content (room_id + title). For a read-only, zero-parameter listing tool this is adequate, though it omits minor traits like ordering, pagination, and whether the owner room is guaranteed to exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with zero filler. The first sentence front-loads the core purpose and output fields; the second adds a practical decision aid without repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 0-parameter list tool with no output schema, the description covers what is listed, what fields come back, and when to use it. Minor gaps (e.g., behavior when no rooms exist, pagination) are low-stakes for a read-only listing, so nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially 100% and there is nothing to document. The baseline of 4 applies because no parameter semantics are needed; the description appropriately spends its effort on output and scope instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb (列出/list) and resource (all Matrix rooms this avatar has joined), and explicitly states the returned fields (room_id + title). The scope '本分身已加入' clearly separates it from per-room siblings like room_info and room_members, so an agent can tell them apart without opening another schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a concrete usage scenario: when unsure which room to reply to, use this tool to locate the owner's private chat room. It does not, however, name alternatives or state when not to use it (e.g., room_info for a single room's details), so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool performs a diagnostic self-check and returns connection and encryption status, which implies a read-only probe. It does not explicitly say it has no side effects, but the self-check framing makes the behavior sufficiently transparent for a zero-parameter tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one compact sentence that front-loads the action and scope, then adds the return value and invocation context. Every clause earns its place; there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter diagnostic tool with no output schema, the description covers purpose, method, return value, and invocation context. Nothing essential is missing for an agent to select and call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters and schema description coverage is 100%, so there is nothing for the description to clarify. Baseline for zero parameters is 4, and the description adds no conflicting or misleading parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: it runs a connection/E2EE self-check equivalent to 'npm run doctor' and returns Matrix connection and encryption status. It also distinguishes this health-check tool from the messaging/approval-oriented sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear usage context by stating it can be invoked during /mcp health checks and is for checking connection/encryption status. It does not explicitly name when-not-to-use conditions, but the sibling tools are clearly different in purpose, so the distinction is implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• replyA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  没有 annotations，因此描述承担了全部行为披露责任。描述主动说明了自动重试机制、失败后的默认回复兜底行为（发往控制室/owner 私聊），以及原路返回的逻辑，这些都是非显而易见的关键行为。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  描述短小精悍，三句话分别说明核心操作、chat_id 省略行为和重试/兜底机制，信息密度高且没有冗余。关键行为前置，结构清晰。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  对于发送消息类工具，描述配合完整 schema 已覆盖调用所需信息：目标房间、默认回退行为、重试和失败兜底。没有输出 schema 也不影响理解，整体上下文完整。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema 描述覆盖率为 100%，所有参数在 schema 中已有说明。描述额外强调了 chat_id 省略时的行为，但 schema 中同样包含该信息，因此描述在参数语义上未显著超出 schema 提供的价值。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述明确说明该工具通过 Matrix 发送回复到指定房间，并解释了省略 chat_id 时的原路返回行为。动词和资源都很具体，与 list_rooms、room_info 等只读查询类兄弟工具形成清晰区分。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  描述清晰说明了何时使用该工具（发送回复）以及省略 chat_id 时的行为，但未明确提及何时不应使用或列举替代工具。整体使用场景清楚，只是缺少显式的排他性说明。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.