Server Quality Checklist

Tool Scores

• execute_javascriptB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. It only mentions the sandbox environment and does not state whether execution returns output, how stdout/stderr are handled, whether state persists, or what side effects may occur. This leaves the agent guessing about execution semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no waste. It efficiently conveys the core operation and environment, though it sacrifices helpful context for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description should clarify what the caller receives and how the sandbox behaves. It omits return format, timeout behavior, and environmental constraints, making it insufficient for confident invocation beyond passing arbitrary code.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both parameters and their purpose. The description adds no additional semantic value beyond what the schema provides, meeting the baseline for a fully documented schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Execute'), a specific resource ('JavaScript/Node.js code'), and a specific environment ('E2B sandbox'). This clearly distinguishes the tool from its sibling execute_python by language and from file operations, leaving no ambiguity about what resource it acts on.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to prefer this tool over execute_python or when not to use it. There are no exclusions, prerequisites, or alternative routing hints. Usage must be inferred entirely from the tool name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_pythonB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden of behavioral disclosure. It only says 'execute Python code in a sandbox', without mentioning side effects, sandbox persistence, execution timeouts, network/filesystem access, or whether the environment is ephemeral. For an arbitrary code execution tool this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single clear, front-loaded sentence with no wasted words. It is concise, though it could arguably include more behavioral context without becoming bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that executes arbitrary code, this description is too minimal. With no annotations and no output schema, the agent is left unaware of return values, execution limits, environment behavior, or risk considerations, making the definition incomplete for safe and correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters with clear descriptions, so the schema already provides the needed semantic meaning. The description adds no extra parameter detail, which is acceptable under the high schema coverage baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Execute') and resource ('Python code in an E2B sandbox environment'). It clearly distinguishes this from sibling execute_javascript by language, though it does not elaborate on broader sandbox capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for Python execution, which differentiates it from execute_javascript, but it provides no explicit guidance on when to choose this tool over alternatives like create_file, list_files, or install_packages. Context is present, but exclusions and alternative routing are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the full behavioral burden. It only restates the basic operation and adds no context about return format, recursion, hidden behavior, or how sandbox_id affects listing. This adds little beyond the tool name itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no wasted words. The verb and object are front-loaded, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool, this is minimally adequate, but with no output schema and no annotations, an agent is left unsure whether the return includes file names only, full paths, or directories. The description lacks enough detail for fully confident invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters. The description adds no extra parameter-level meaning, which is acceptable given the high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and resource (files in a directory) within the sandbox environment. It is distinguishable from siblings like read_file and create_file, though it does not explicitly name or differentiate from them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives such as read_file or get_sandbox_info. The intended usage is only implied by the tool name and short description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_fileB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('create a file') but does not mention whether the file is overwritten if it already exists, whether parent directories are created automatically, or any permission or sandbox constraints. This is minimal behavioral transparency for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no filler. It is appropriately sized for a simple tool and immediately communicates the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a file-creating tool with no annotations and no output schema, the description omits important behavioral details such as overwrite behavior, path handling relative to the sandbox root, and whether content is written exactly as provided. The low complexity keeps this from being a 1, but the description is not complete enough for an agent to predict side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for all three parameters, so the schema already documents the parameters. The description adds no parameter-specific information, but the baseline of 3 applies because the schema covers the needed semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Create') and resource ('file in the sandbox environment'), which distinguishes it from sibling tools like read_file and list_files. It lacks detail about the file path semantics, but the core purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implicitly communicates when to use the tool: whenever a file needs to be created in the sandbox. However, there is no explicit guidance about when not to use it or how it compares to related operations like writing files via execute_python or creating files through package installation. The usage context is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'read', which implies no side effects, but it does not explain what is returned, how missing files are handled, or how sandbox_id affects behavior. This is a minimal disclosure for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler or redundant detail. Every word contributes meaning, and the core action is immediately clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read operation with only two parameters, the description is largely sufficient. It lacks an explicit statement of return value and error behavior, but given the tool's simplicity and the fully covered schema, the remaining gaps are minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully documents the two parameters, so the baseline is 3. The description adds useful context by specifying that the path refers to a file inside the sandbox environment, which is not stated in the schema's 'File path to read' property. This small semantic addition raises it to 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('read') and resource ('file from the sandbox environment'), making it easy to distinguish from siblings like create_file and list_files. It does not explicitly name the sibling alternatives, so it stops short of a 5, but the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as list_files or execute_python. The description implies reading a file's contents, but it gives no exclusions, prerequisites, or comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sandbox_infoA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the transparency burden. 'Get information' signals a read-only operation, but it does not disclose what specific fields are returned, whether the call can be expensive, or what happens when sandbox_id is omitted. It's adequate but lacks behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no filler. It conveys the core purpose in the fewest possible words and does not repeat schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, has one optional parameter, and the schema covers it. However, with no output schema, the description does not explain what the returned information looks like or the meaning of 'resource usage', leaving some agent-facing ambiguity about what to expect from the call.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents the only parameter fully (optional sandbox ID for a specific sandbox), giving 100% schema coverage. The description adds no extra parameter semantics, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('Get') and names the resource ('sandbox status and resource usage'), which distinguishes it from the sibling file/execution tools. It stops short of a fully specific outcome like a list of returned fields, so it's a 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use case is implied by the description: call this when you need sandbox status or resource usage. However, there is no explicit guidance on when not to use it, when the optional sandbox_id should be supplied, or comparison to alternatives, so it earns only an implied-use score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• install_packagesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states that packages are installed via pip/npm, without mentioning side effects, persistence, failure behavior, or return values. This is a significant gap for a tool that mutates the sandbox environment.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the verb and resource, then adds the ecosystem detail efficiently. There is zero filler and no redundant repetition of schema information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The core call can be constructed from schema plus description, but with no annotations and no output schema, an agent is left guessing about return values, failure handling, and when to prefer this over the execute_* siblings. This is minimal but not entirely inadequate for a straightforward install operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by mapping the language enum to actual package managers (pip for Python, npm for Node.js) and clarifying that installation targets the sandbox, which helps the agent interpret the parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states a specific action (install packages), a target resource (sandbox environment), and the two ecosystems involved (Python pip or Node.js npm). This distinguishes it from sibling tools like execute_python/execute_javascript and file operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (when dependencies are needed in the sandbox) but does not explicitly compare it to alternatives like execute_python/execute_javascript or mention when not to use it. The usage context is left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.