Agent Vending Factory

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and excels: it discloses the isolated subprocess, hard timeout, curated pre-loaded modules, blocked operations (import, open, eval, exec, os, sys, network, classes, dunders), stdout cap, error_type enum values, and per-call cost. This is exceptionally transparent about behavior and safety boundaries.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but well-structured with labeled sections (Input, Output, Pre-loaded, Blocked, error_type, Cost). Every sentence carries critical information, and the format makes it easy to scan. It packs a lot of details without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a code-execution tool with no output schema, the description is remarkably complete. It covers input format, output shape, error types, timeout, resource limits, allowed modules, forbidden operations, and cost. An agent has all the information needed to decide whether to use the tool and how to interpret its results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics beyond the schema by explaining that input_data is bound to a variable named 'input_data', that the return value must be assigned to 'result', and by enumerating the output fields (success, result, stdout, execution_time_ms, error_type). This extra context helps the agent use the parameters correctly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Executes') and a precise resource ('Python 3.12 code in an isolated subprocess'), plus a 5-second timeout. This clearly differentiates the tool from siblings like tool_data_transformer and tool_zar_prices, which are not code-execution tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by detailing execution constraints, blocked operations, and cost, but it never explicitly states when to use this tool versus alternatives. There is no mention of 'use this when you need to run arbitrary Python' or any exclusions. The purpose is clear, but usage guidance is only implicitly conveyed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full responsibility and excels: it discloses that no LLM is involved, lists the extraction_method enum, mentions type coercion, describes the output structure, and even states the cost per call. Limitations about nested schemas add critical expectations. This is far beyond typical descriptions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence provides distinct value: endpoint, input, output, methods, behavior, limitations, and cost. It is structured logically with clear separators and avoids fluff or repetition of schema content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite lacking annotations and an output schema, the description compensates by fully specifying the output fields (success, extracted_data, extraction_method, validation_passed, error) and covering operational details (type coercion, flat-schema preference, cost). This is a complete picture for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed descriptions, so the baseline is 3. The description adds extra value by specifying that target_json_schema follows JSON Schema draft-07 and by framing the parameters as caller-supplied, which clarifies their role in the pipeline. This nuance earns a 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('extracts') and resource ('structured JSON from raw text') and clearly distinguishes the tool's function from the sibling tools, none of which suggest data extraction. The inclusion of the endpoint and input/output structure reinforces the purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context on when to use the tool (for extracting structured data without an LLM) and notes a limitation ('Works best with flat schemas; deeply nested structures extract less reliably'), serving as a when-not. However, it does not explicitly name alternative tools or provide a clear 'use this instead of X' directive, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full behavioral disclosure. It reveals read-only nature, cost per call, error_type values with retry guidance, flight_number normalization, and a precondition health check. This level of transparency is exceptional.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence adds value. It is front-loaded with the core purpose and then systematically covers input, output, coverage, data window, health check, error handling, and cost. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema, the description compensates by listing all output fields and error types. It also covers prerequisites, restrictions, retry logic, and pricing. For a tool of this complexity, the description is fully self-contained.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description adds meaningful semantics: it shows the exact input structure, explains flight_number case-insensitivity and normalization, lists output fields, and details error_type meanings that affect retry behavior. This far exceeds the schema's own descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Returns live flight status from ACSA (airports.co.za)'. It clearly distinguishes itself from sibling tools by naming the coverage (JNB, CPT, DUR) and the data source, making the tool's unique purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use context: limited to three South African airports, 48-hour data window, and read-only (no booking/ticketing). It also instructs to call the /health endpoint first and not proceed if structure_valid=false. This goes beyond mere context to actionable guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It adds important context: sources (VALR, Luno), per-exchange pair coverage, concurrent fetching, ISO-8601 UTC timestamps, and cost per call. However, it omits potential failure modes, rate limits, and authentication requirements, which would make it fully transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact yet information-dense, covering endpoint, input, output, sources, concurrency, timestamp format, and cost. Every sentence provides value without redundancy. It is well-structured and front-loaded with the primary purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description adequately explains the return format (array of objects with exchange, pair, price, bid, ask, timestamp). It also covers source availability, concurrency, and cost. The context is sufficiently complete for an agent to invoke and interpret results correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already covers the parameter 100% with enum values, default, and description. The tool description adds meaningful semantic enrichment by explaining which exchanges support each pair (e.g., 'SOL/ZAR and USDC/ZAR are VALR-only') and clarifies 'all' fetches concurrently. This goes beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Returns live bid/ask/last prices for crypto/ZAR pairs.' It identifies the specific resource (crypto/ZAR pairs) and the action (fetch prices), and distinguishes it from sibling tools which are unrelated (compute, data transformer, airport oracle). The endpoint is also specified.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on when to use the tool (to get live prices for specified pairs) and what inputs to provide. While it doesn't explicitly mention alternatives or when-not-to-use, the scope is well-defined and no competing sibling tools exist, making the usage context unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.