@argosvix/mcp-server
The @argosvix/mcp-server is a comprehensive LLM observability and operations MCP server that lets AI agents query, manage, and act on LLM call data, costs, alerts, evals, and runtime controls directly from conversation. It provides 87 tools, 3 resources, 8 resource templates, and 3 prompts.
Query & Analytics
Retrieve, filter, and export LLM call records (up to 50,000 on Pro) by provider, model, time range, and tags
Aggregate cost, latency, and token metrics; get p50/p95/p99 percentiles as single values or time series
One-call account health summary (error rate, latency, budget, recent events) with ok/warn/critical status
Anomaly detection comparing current window vs. baseline across cost, latency, error rate, and call volume
Alerts
List, create, update, and delete alerts (cost threshold, error rate, latency, monthly budget, anomaly, eval score, guardian finding)
Silence/unsilence, acknowledge, and view alert event history
AI-generated alert rule proposals based on historical patterns; bulk-silence noisy alerts
Evals & Quality
Manage LLM-as-judge eval criteria (global defaults + custom, including deterministic evaluators)
Run, list, and compare eval runs on recent calls or golden datasets; detect regressions
AI-generated eval criteria suggestions from a use-case hint and sample calls
Annotations
Create, read, update, and delete human reviews (rating, label, comment) on individual LLM calls
Prompt Management
List, create, update, rename, and delete versioned prompt templates
Deploy/rollback prompt versions to environments (production/staging); resolve the active deployed prompt
Runtime Control Plane
Budget gates: monthly LLM spend limits with SDK-enforced pre-execution blocking (fail_open/fail_closed)
Policy gates: model allowlists, PII blocking, and secret blocking rules
Human approval gates: request, poll, and list approvals before dangerous operations (e.g., bulk delete, purge plaintext)
Safety & Compliance
List/get safety assessments (OpenAI Moderation results); batch-classify unclassified calls on demand
Audit log with filters (event type, actor, time range)
Bulk-purge expired plaintext records (with dry-run and approval gate)
Proposals (Guardian)
View, read, and reply to AI-generated improvement proposals (quality drift, anomalies, cost switching, safety, noisy alerts)
Webhooks
List, create, update, delete, and test outbound webhooks for account events
Account & Project Management
List, create, rename, and delete projects; list team members and roles
Manage saved filter views for the calls dashboard
Integrates with OpenAI's Moderation API for safety classification of LLM calls and uses GPT-4o-mini as a judge for evaluating call quality and proposing eval criteria.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@@argosvix/mcp-serverwhat was my total cost from yesterday?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
@argosvix/mcp-server
Argosvix MCP server lets AI agents (Claude Desktop, Cursor, Codex CLI, custom MCP clients) query, manage, and operate their LLM observability data directly from the conversation. Supports both stdio (subprocess) and HTTP (remote / self-host) transports.
Surface: 89 tools (86 generally available + 3 internal operations tools that return 403 for customer accounts) / 3 resources / 8 resource templates / 3 prompts. Health and anomaly endpoints (get_account_health / detect_anomaly / propose_alert_rules / classify_calls_batch / propose_eval_criteria) plus a runtime control plane (budget gates / policy gates / human-approval gates) let an agent both observe and act. Release history is available on npm.
Why
You're already sending LLM calls through @argosvix/sdk or the Python SDK. Now ask Claude / Cursor questions like:
"What was my OpenAI cost over the past 24h?"
"Which alerts are firing right now?"
"Show me the 20 most expensive calls today."
No dashboard tab-switching. The agent fetches the data via this MCP server using your Argosvix API key.
Related MCP server: CosTrack MCP
Install
One-click (starts keyless; set ARGOSVIX_API_KEY afterwards to use the tools for your own account):
Claude Code users can install the argosvix plugin instead — it bundles this server plus setup skills:
/plugin marketplace add argosvix/claude-plugin
/plugin install argosvix@argosvix
/reload-pluginsManual:
npm install -g @argosvix/mcp-serverConfigure (Claude Desktop)
Edit your Claude Desktop config:
macOS:
~/Library/Application Support/Claude/claude_desktop_config.jsonWindows:
%APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"argosvix": {
"command": "argosvix-mcp",
"env": {
"ARGOSVIX_API_KEY": "argk_..."
}
}
}
}Restart Claude Desktop. All 89 tools appear under the argosvix__ prefix (e.g. argosvix__query_calls, argosvix__get_account_health, argosvix__create_budget_gate).
No API key yet? The server also starts without
ARGOSVIX_API_KEYin introspection-only mode: all 89 tools are listed so you can evaluate the surface, and two public data tools (get_synth_daily/get_daily_readthrough— Argosvix's own daily AI digest and OSS readthrough) work with no key at all. Every other tool call returns instructions for getting a key at https://dashboard.argosvix.com/api-keys.Notes on the public data tools: they are part of the default
fullprofile only (not inARGOSVIX_MCP_PROFILE=core), and like every other tool they honorARGOSVIX_API_BASE— if you point the server at a different base, they fetch/v1/synth-daily//v1/readthroughfrom that base without authentication.
Configure (Cursor)
Edit ~/.cursor/mcp.json:
{
"mcpServers": {
"argosvix": {
"command": "argosvix-mcp",
"env": {
"ARGOSVIX_API_KEY": "argk_..."
}
}
}
}Tool profile (ARGOSVIX_MCP_PROFILE)
The default profile is full (all 89 tools). If your MCP client's context budget is tight, set ARGOSVIX_MCP_PROFILE=core to expose only the 11 essentials for day-to-day operations:
query_calls · aggregate_calls · get_cost_summary · get_percentiles · get_account_health · detect_anomaly · list_alerts · create_alert · silence_alert · unsilence_alert · get_deployed_prompt
{
"mcpServers": {
"argosvix": {
"command": "argosvix-mcp",
"env": {
"ARGOSVIX_API_KEY": "argk_...",
"ARGOSVIX_MCP_PROFILE": "core"
}
}
}
}Works in both stdio and HTTP transports. Unknown values fall back to full with a warning on stderr.
Description language (ARGOSVIX_MCP_LANG)
Tool, resource, and prompt descriptions returned by tools/list / resources/list /
prompts/list are in English by default. Set ARGOSVIX_MCP_LANG=ja to get the original
Japanese descriptions instead. Unset or unknown values fall back to en (with a warning
on stderr for unknown values). Tool names, input schemas, and behavior are identical in
both languages — only the human/agent-facing description text changes.
{
"mcpServers": {
"argosvix": {
"command": "argosvix-mcp",
"env": {
"ARGOSVIX_API_KEY": "argk_...",
"ARGOSVIX_MCP_LANG": "ja"
}
}
}
}Works in both stdio and HTTP transports.
Tools (highlights)
89 tools in total — the full list is returned by tools/list. A sample of the core read / write surface:
Tool | Purpose | Type |
| Recent LLM call records, filterable by provider / model / time range | read |
| Aggregate cost / calls / tokens by provider or model | read |
| Configured alerts + recent trigger status | read |
| Detail of a specific alert + recent trigger history | read |
| Alert trigger events across the account (notification history) | read |
| Mute a specific alert (= temporary notification stop, default 24h) | write |
| Resume notifications for a previously muted alert | write |
| Create a new alert rule (cost / error rate / latency / anomaly) | write |
| Mark a specific alert event as acknowledged (idempotent, orthogonal to silence) | write |
Resources
Resources expose read-only snapshots that AI agents can pull into context without an explicit tool call.
URI | Purpose |
| Plan / quota / current-month record usage / retention snapshot (non-sensitive, Bearer-only) |
| Snapshot of currently enabled alerts |
| Last-24h cost breakdown by provider (with |
Resource templates
Resource templates let agents construct dynamic URIs from a known id. The server enforces account scope via PK lookup on the backend, so cross-account ids return 404.
URI template | Purpose |
| Single LLM call record (provider / model / tokens / cost / latency / tags / error / trace_id). Plug in any id from |
| Single alert rule (name / type / threshold / window / channelKinds / sleep / enabled / silencedUntil) + recent 20 trigger events. Plug in any id from |
| Single trace = all spans grouped by |
Prompts
Prompts are reusable templates the user can launch as slash commands.
Name | Purpose |
| Compare 24h / 7d / 30d cost trends and flag anomalies |
| Audit current alert rules and propose improvements |
| Investigate recent error / latency anomalies (default last 24h) |
Subscriptions (= v0.10, stdio only)
resources.subscribe capability is declared in stdio mode. Clients can subscribe to one or more of the static resources below; the server polls each subscribed resource every 60 seconds and emits notifications/resources/updated when the content hash changes.
Subscribable URIs (resource templates such as argosvix://calls/{id} are not subscribable):
argosvix://accountargosvix://alerts/activeargosvix://cost/today
HTTP transport (= argosvix-mcp --http) does not declare subscribe and rejects subscribe requests, since per-request stateless mode cannot keep a subscription set or deliver server-initiated notifications. Use stdio for live updates.
listChanged is intentionally not declared: the resource list is fixed for the server lifetime.
Implementation notes (v0.10):
Single-flight guard: a slow polling cycle does not start a concurrent next cycle (= no overlapping
readResourcecalls).Unsubscribe / shutdown race: per-URI re-check before fetch and before notify, plus an
isShuttingDownflag, so notifications are not emitted for URIs removed mid-cycle.Failure handling: fetch and notify failures are silent (next cycle retries). With
ARGOSVIX_MCP_DEBUG=1the server logs{ uri, errorClass }to stderr — error messages are intentionally not logged to avoid leaking upstream payload text.Tests: 150 unit tests (incl. overlap single-flight + unsubscribe / shutdown race + invalid URI →
McpError(InvalidParams)+ axis 4 Tier 1 dispatcher coverage).
Privacy
The MCP server sends queries to https://ingest.argosvix.com using your API key. No prompts or completions are exposed — only metadata (tokens, cost, latency, model name, your tags).
Development
npm install
npm run build
npm testLicense
MIT © Yuto Makihara (Argosvix). See LICENSE.
HTTP transport
The server can also run as a remote MCP endpoint over HTTP, suitable for self-hosting or multi-tenant scenarios where API key is supplied per request.
# Start HTTP transport (default: 127.0.0.1:3000)
argosvix-mcp --http
# Bind to all interfaces with custom port + allowed Host headers
MCP_HTTP_HOST=0.0.0.0 \
MCP_HTTP_PORT=4000 \
MCP_HTTP_ALLOWED_HOSTS="mcp.example.com,localhost:4000" \
argosvix-mcp --httpEndpoints
GET /health→200 OKwith server name/version (no auth)POST /mcp→ MCP JSON-RPC endpoint (auth required)
Client example
curl -X POST http://localhost:3000/mcp \
-H "Authorization: Bearer argk_..." \
-H "Content-Type: application/json" \
-H "Accept: application/json, text/event-stream" \
-d '{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/list",
"params": {}
}'Auth model
Each request must include Authorization: Bearer <api-key> with a valid
Argosvix API key (issue at https://dashboard.argosvix.com/api-keys). The
server uses stateless mode (no session ID), so each request is independent
and may carry a different key.
Security notes
Default bind is
127.0.0.1(localhost only). UseMCP_HTTP_HOST=0.0.0.0to expose externally, but pair with a reverse proxy + TLS in production.DNS rebinding protection is applied for localhost binds by checking the
Hostheader against a known-localhost allow list. UseMCP_HTTP_ALLOWED_HOSTS(comma-separated) when binding non-locally.Body size is capped at 1 MiB.
Debug logging
Backend error response bodies are not logged by default to keep production log aggregators free of backend-internal payloads. To enable raw body logging for a debugging session:
ARGOSVIX_MCP_DEBUG=1 argosvix-mcp # stdio mode
ARGOSVIX_MCP_DEBUG=1 argosvix-mcp --http # HTTP modeWithout the env var, error logs only include path, status, and x-request-id.
Available Tools
87 toolsacknowledge_alertA
Mark an individual alert firing (event) as handled / acknowledged. Unlike silence_alert (which temporarily mutes the whole alert rule), ack is a per-event receipt — future firings of the same rule are still delivered as usual. Pass the id obtained from list_alert_events as eventId. Re-acking an already acknowledged event does not overwrite the existing ack info (the first acknowledgedAt / acknowledgedBy) and returns 200 (idempotent; distinguishable via the alreadyAcknowledged flag).
| Name | Required | Description | Default |
|---|---|---|---|
| eventId | Yes | Event id to acknowledge (list_alert_events.events[].id) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: per-event action, idempotent (does not overwrite ack info), return 200 with flag. No hidden traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with main action, then contrast, then details. Every sentence earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description explains return behavior (200, flag) and idempotence. Sufficient for a simple acknowledge operation. Sibling tools cover related alert actions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes eventId with origin note; description adds 'Pass the id obtained from list_alert_events as eventId', reinforcing correct usage. High schema coverage (100%) plus extra context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it marks an alert event as acknowledged, and explicitly distinguishes from sibling silence_alert by explaining per-event vs rule-level behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on when to use this tool (per-event ack) versus silence_alert (rule-level mute), and explains idempotent behavior with the alreadyAcknowledged flag.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
aggregate_callsA
Get an aggregation cube over calls (POST /v1/query/aggregate). groupBy (provider / model / day / hour / minute / tag / error) x metric (cost / latency / tokens / input_tokens / output_tokens / cached_tokens / cache_savings / count / error_rate) — e.g. "aggregate this month's cost by model" in one call. tag mode requires tagKey (alphanumerics plus _ - only, e.g. 'env' / 'feature'). error mode aggregates only error rows by error string (which errors, how many; metric=count recommended). hour mode caps at 168h / minute mode at 60min (400 beyond). cost = SUM(cost_usd) / latency = AVG(latency_ms) / tokens = SUM(total_tokens) / input_tokens = SUM(prompt_tokens) / output_tokens = SUM(completion_tokens) / cached_tokens = SUM(cached_read_tokens) / cache_savings = SUM(cache_savings_usd) / count = COUNT(*) / error_rate = errors / total. Returns { groups: [{key, value, count}], total: {value, count} }.
| Name | Required | Description | Default |
|---|---|---|---|
| metric | No | Metric kind ('cost' / 'latency' / 'tokens' / 'input_tokens' / 'output_tokens' / 'cached_tokens' / 'cache_savings' / 'reasoning_tokens' / 'audio_tokens' / 'ttft' / 'count' / 'error_rate'), default = 'cost'. cached_tokens = SUM(cached_read_tokens), cache_savings = SUM(cache_savings_usd) (prompt-cache savings), reasoning_tokens = SUM(reasoning tokens), audio_tokens = SUM(audio tokens), ttft = AVG(ms to first token) | cost |
| tagKey | No | Required when groupBy='tag'. Key name inside the tags JSON (alphanumerics plus _- only, 1-64 chars) | |
| endTime | No | Range end ISO timestamp (UTC; omit = now) | |
| groupBy | No | Aggregation axis ('provider' / 'model' / 'day' / 'hour' / 'minute' / 'tag' / 'error'), default = 'provider'. hour caps at 168h / minute at 60min. error aggregates only error rows by kind | provider |
| provider | No | Provider filter ('openai' / 'anthropic' etc.); omit = all providers | |
| startTime | No | Range start ISO timestamp (UTC; omit = all time) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description fully discloses behavioral traits: limits on hour/minute modes (168h, 60min), required tagKey pattern, error mode behavior, and exact formulas for each metric. It also describes the return format. Since no annotations are present, this burden is met excellently.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise yet comprehensive: it opens with the core purpose, then systematically covers dimensions, special modes, metric definitions, and output format. Every sentence adds value, and the structure is front-loaded and logical.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no output schema, no annotations), the description is remarkably complete. It covers purpose, usage example, constraints, metric semantics, and output shape. Only missing explicit sibling comparisons, but these are covered by purpose clarity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds significant value beyond schema by explaining metric formulas (e.g., cost = SUM(cost_usd)), caps for hour/minute, tagKey requirements, and return structure. This exceeds the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool computes an aggregation cube over calls, specifying groupBy dimensions and metrics with concrete examples. It distinguishes itself from siblings like query_calls or get_cost_summary by focusing on multi-dimensional aggregation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
While the description provides a usage example ('aggregate this month's cost by model'), it does not explicitly state when to use this tool versus alternatives such as query_calls, get_cost_summary, or get_percentiles. The guidance is implied but not precise.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
apply_promo_code_to_customerA
Apply a user-facing promotion code already registered in Stripe (e.g. 'LAUNCH50') to your account's Stripe subscription (POST /v1/tier2/promo/apply). Founder-operations only (an internal support tool; general accounts get 403). It will not be opened up without terms covering economically impactful operations (timing undecided). 409 if an active discount already exists (structural defense against stacking), and 409 when the status is canceled / incomplete_expired. Redemption is delegated to Stripe via promotion_code (applying coupons directly is forbidden as a constraint bypass); dryRun must be passed explicitly, and idempotencyKey is required when dryRun=false. Re-calls with the same key return the cached result via the tier2_idempotency table, structurally serializing concurrent applies. dryRun=true previews resolution + the active-discount check + the estimated discount only (no Stripe mutation); dryRun=false applies the promotion code.
| Name | Required | Description | Default |
|---|---|---|---|
| dryRun | Yes | Must be passed explicitly. true = preview only; false = actual promotion-code application + Stripe mutation | |
| reason | Yes | Reason for applying it (recorded in the audit log; required, 200 chars max) | |
| promoCode | Yes | Promotion code already registered in Stripe (e.g. 'LAUNCH50'; alphanumerics plus '_-', 64 chars max) | |
| approvalId | No | Approval id granted via request_approval (apr_ + 32 hex; create with action 'apply_promo_code_to_customer'). Server-side verification + atomic consumption on a fresh execution (1 approval = 1 execution chain; retries with the same idempotencyKey do not re-consume). dryRun only verifies | |
| idempotencyKey | No | Required when dryRun=false. 16-128 chars alphanumeric plus '_-'; re-calls with the same key return the cached result | |
| targetAccountId | Yes | Target account id (your own account only for now; specifying another user gets 403) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behaviors: error codes (403, 409), idempotency via tier2_idempotency table, delegation to Stripe, atomic consumption of approvalId, and dryRun semantics. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with purpose and restrictions, then details parameters and behaviors. It is well-structured but slightly verbose; some sentences could be merged. However, every part adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description covers purpose, error conditions, parameter details, and idempotency. It lacks explicit return format but mentions preview output for dryRun. Adequate for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While schema coverage is 100%, the description adds significant context beyond schema: dryRun effects, idempotencyKey requirement, approvalId consumption pattern, targetAccountId restriction. It explains parameter interactions and constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool applies a promotion code to a Stripe subscription, specifying the resource (promotion code), action (apply), and constraints (registered in Stripe, user-facing). It distinguishes from siblings as there is no other promo-related tool in the list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly limits usage to founder-operations (403 for others), explains when not to use (active discount or canceled status), and provides guidance on dryRun vs actual application. No comparison to siblings needed as no alternative exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
auto_silence_noisy_alertA
Bulk-silence noisy alerts that fired repeatedly in the past hour (POST /v1/tier2/alerts/auto-silence). Finishes "this alert fired 50 times in the past hour — silence it for an hour" in one prompt. Specify exactly one of alertId (silence a single alert) or byVolumeThreshold (all alerts with N+ firings in the past hour). silenceDurationMinutes is 5-1440 (5 minutes to 24 hours), default 60. An optional reason can be attached. dryRun=true previews the targets with fireCount; dryRun=false UPDATEs alerts.silenced_until and emits an audit event per alert (tier2.auto_silence_noisy_alert). Because this is a reversible mutation (the existing unsilence_alert can undo it) with strict per-account scoping (other accounts' alerts are unaffected), there is no founder gate — paid Pro+ users can call it directly. Returns (dryRun=true) { dryRun: true, targetCount, silenceUntil, silenceDurationMinutes, lookbackStart, targets: [{alertId, name, fireCount}] }; (dryRun=false) { dryRun: false, targetCount, silenceUntil, silenceDurationMinutes, succeeded: string[], failed: [{alertId, reason}], skipped: string[], reason }. idempotencyId = sha1(endpoint+accountId+alertId+silenceUntil truncated to the minute), coalescing duplicate runs within the same minute.
| Name | Required | Description | Default |
|---|---|---|---|
| dryRun | No | true = preview only (zero mutation); false = actual silence. Default false | |
| reason | No | Silence reason (recorded in the audit log, 200 chars max) | |
| alertId | No | Alert id for single-alert silencing (alrt_xxx format). Cannot be combined with byVolumeThreshold | |
| approvalId | No | Approval id granted via request_approval (apr_ + 32 hex; create with action 'auto_silence_noisy_alert'). Server-side verification + atomic consumption on the actual silence (1 approval = 1 execution). dryRun only verifies | |
| byVolumeThreshold | No | Batch-silence all alerts with N+ firings in the past hour. Cannot be combined with alertId | |
| silenceDurationMinutes | No | Silence duration (minutes, 5-1440 = 5 minutes to 24 hours), default 60 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavioral traits: it is a mutation that updates alerts.silenced_until and emits audit events, details dryRun vs. actual execution, idempotency mechanism, per-account scoping, and the absence of a founder gate for paid users. This is comprehensive and adds significant context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively long but efficiently structured with clear sections for purpose, parameters, behavior, and output. Every sentence adds value, though some minor redundancy exists (e.g., repeating endpoint details). It is front-loaded with the core use case.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, no output schema), the description is very complete: it explains both dryRun and actual outputs, idempotency, permissions, and relationships with sibling tools. Minor omissions include explicit error conditions or rate limits, but overall it provides sufficient context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all 6 parameters with descriptions (100% coverage), but the description adds important context beyond the schema: it explains the mutual exclusivity of alertId and byVolumeThreshold, default and range for silenceDurationMinutes, optional reason, dryRun behavior, and approvalId usage. It also describes the idempotency mechanism, which is not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool is for bulk-silencing noisy alerts that fired repeatedly in the past hour, using specific verbs and resources. It distinguishes itself from sibling tools like silence_alert and unsilence_alert by emphasizing the automated, volume-based triggering and the reversal capability.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit guidance on when to use the tool (e.g., finishing the prompt about repeated alerts) and how to choose between alertId and byVolumeThreshold. It mentions that unsilence_alert can undo the action and that Pro+ users can call it directly, but it could be improved by explicitly stating when not to use it (e.g., for single alert without volume criteria).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
bulk_delete_callsA
Bulk-delete the given call ids (max 100), scoped to your account (POST /v1/calls/bulk-delete). Useful for cleaning up garbage calls accumulated by dogfooding / dev tests. dryRun=true returns the matched count before deleting. The delete is one atomic SQL statement; a bulk_deleted event is recorded in the audit log. Per FK constraints, related traces / annotations / scores are cascade-deleted via ON DELETE.
| Name | Required | Description | Default |
|---|---|---|---|
| dryRun | No | true returns only the matched count without deleting (confirmation UX) | |
| callIds | Yes | Array of call ids to delete (1-100 entries, each 1-128 chars) | |
| approvalId | No | Approval id granted via request_approval (apr_ + 32 hex; create with action 'bulk_delete_calls'). Server-side verification + atomic consumption on actual deletion (1 approval = 1 execution). dryRun only verifies |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description fully covers behavioral traits: atomic SQL statement, audit logging (bulk_deleted event), cascade delete of related entities via FK constraints, dryRun behavior, and approval flow. Comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with main action, every sentence adds unique information. No repetition or fluff. Efficiently covers purpose, usage, behavior, and side effects.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is remarkably complete: it explains what the tool does, when to use it, how it works internally, side effects, and parameter nuances. Leaves no ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds value by explaining dryRun's confirmation purpose and approvalId's server-side verification and atomic consumption, beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states bulk-delete of call ids, scoped to account, with a max of 100. Distinguishes itself via 'garbage calls' context and no sibling tool offers bulk deletion of calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear use case ('cleaning up garbage calls from dogfooding/dev tests') and mentions dryRun for safe verification. Does not explicitly state when not to use, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
classify_calls_batchA
Batch safety-classify unclassified calls on demand (via OpenAI Moderation, POST /v1/safety-assessments/scan-batch). Complements the cron (every 15 min, 50 records) with a "right now" path — an AI agent can finish "classify all of last week's calls" in one prompt. Pro+ only (Free relies on the cron); the backend enforces the plan gate and budget gate. maxRecords (1-100, default 50); returns { scanned, assessed, flagged, failures, skipped }. Recorded with source='mcp' (distinguished from cron entries, so the dashboard can visualize on-demand classification). Audit: emits a safety.scan_batch_run event to the audit log.
| Name | Required | Description | Default |
|---|---|---|---|
| maxRecords | No | Max records scanned per request (1-100, default 50). Capped considering OpenAI's 1000 RPM limit and the 10s CPU/IO limit per worker request |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must fully disclose behavior. It details the backend enforcement (plan gate, budget gate), limitations (maxRecords with 1-100 range and default 50), return format (scanned, assessed, flagged, failures, skipped), data lineage (source='mcp'), and audit event emission. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately concise, with each sentence providing distinct value. It is front-loaded with the core purpose. A minor redundancy exists with the parameter default mentioned both in the description and schema, but overall well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description thoroughly explains the return values and side effects (audit event, source tracking). With only one parameter and clear usage context, it covers all necessary information for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter maxRecords, with a detailed description including min/max/default and technical notes. The main description restates the default value but adds no significant new semantics beyond the schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs batch safety classification of unclassified calls on demand, specifying the verb 'classify', resource 'unclassified calls', and method 'batch on demand'. It distinguishes itself from the cron job by offering a real-time path, making the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool ('right now' path for immediate classification) versus the cron job (every 15 min). Notes Pro+ plan requirement and that Free plan relies on cron. Provides a concrete example: 'classify all of last week's calls in one prompt', guiding agents on applicable scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_eval_runsA
Compare two eval runs (baseline / candidate) and return per-criterion mean score deltas + the failed count delta + a verdict (GET /v1/eval-runs/compare). Lets an AI agent grasp "how did the candidate change relative to the baseline" in one call, for prompt-improvement measurement and regression detection. verdict = improved / regressed / mixed / unchanged. The failed count treats scores <= 2 as "failed". Same account only.
| Name | Required | Description | Default |
|---|---|---|---|
| baselineRunId | Yes | Id of the baseline run (list_eval_runs.runs[].id) | |
| candidateRunId | Yes | Id of the candidate run (same source); must differ from the baseline |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses return format (deltas, verdict), verdict options, failed count definition (scores <= 2), and account constraint. With no annotations, this covers key behavioral traits adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph with clear progression: action, value, details. Efficient but could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Explains key outputs (deltas, verdict) and failed count rule. No output schema, so description carries load. Adequately complete for a comparison tool, though could mention edge cases like identical runs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds context beyond schema: baselineRunId references list_eval_runs.runs[].id, candidateRunId must be from same source and different from baseline. Schema coverage is 100%, so description adds meaningful usage guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it compares two eval runs, returning per-criterion mean score deltas, failed count delta, and a verdict. Distinguishes from siblings like list_eval_runs and get_eval_run by focusing on comparison.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Describes use for measuring prompt improvement and regression detection. Implicitly contrasts with single-run access tools, but does not explicitly exclude cases where alternative tools are more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_alertA
Create a new alert rule. Watches for cost / error rate / latency / anomaly threshold breaches and notifies the specified channels. Example: "notify me by email when daily cost exceeds $10". channelKinds is an array of channel kinds to enable; channelTargets is an object keyed by those kinds holding the destinations (e.g. channelKinds:["email"], channelTargets:{"email":"dev@example.com"}). Every kind listed in channelKinds must have a destination in channelTargets. anomaly_* types interpret thresholdValue as a standard-deviation multiplier (0.5-10, e.g. 3 = 3 sigma). The Free plan allows the email channel only and up to 3 alerts (the backend returns 403 beyond that).
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Display name of the alert (1-100 chars, no line breaks) | |
| enabled | No | Whether to enable immediately after creation. Default true. | |
| alertType | Yes | Metric to watch. cost_threshold = one-shot cost threshold (USD) / monthly_budget = monthly budget (USD) / error_rate = error rate (%) / latency_degradation = latency degradation (ms) / anomaly_cost / anomaly_latency / anomaly_error_rate = anomaly detection (windowMinutes is fixed at 60) / eval_score = quality SLO (evalCriterionId required; fires when the mean score in the recent window drops below thresholdValue) / guardian_findings = external notification of findings (notifies new inbox findings; thresholdValue / windowMinutes are unused — pass 0 and 60) | |
| conditions | No | v1.5 multi-condition alert (composite conditions). When specified, single-metric evaluation via alertType + thresholdValue + windowMinutes is ignored and the conditions JSON switches to AND/OR aggregation. Example: {"operator":"AND","conditions":[{"metric":"cost_threshold","threshold":100,"windowMinutes":60,"comparator":">"},{"metric":"error_rate","threshold":0.05,"windowMinutes":60,"comparator":">"}]}. The backend validates the shape via parseConditionsJson (operator AND/OR, 1-8 conditions, each requiring metric/threshold/windowMinutes/comparator). Omit (null) to stay on the single-metric path. | |
| filterModel | No | Restrict to this model name only (substring match). Omit for all models | |
| channelKinds | Yes | Array of notification channel kinds to enable (each kind needs a destination under the same key in channelTargets). The Free plan can use email only. | |
| sleepMinutes | No | Suppression window for repeated notifications (minutes, 5-10080). After firing once, no re-notification during this window. Default 60. | |
| windowMinutes | No | Aggregation window (minutes, 5-43200). Default 60. Ignored for anomaly types (fixed at 60). | |
| channelTargets | Yes | Object keyed by channel kind with the destination as the value (must include a destination for every kind listed in channelKinds). Example: {"email": "dev@example.com"}. email takes an email address; slack/discord/teams/webhook take the service's webhook URL. | |
| filterProvider | No | Restrict to this provider only (openai / anthropic / gemini / mistral). Omit for all providers | |
| thresholdValue | Yes | Threshold (>= 0). USD for cost types, % for error_rate, ms for latency_degradation. For anomaly types, a standard-deviation multiplier (e.g. 3 = 3 sigma) | |
| evalCriterionId | No | Required when alertType=eval_score. The id of the eval criterion to watch (list_eval_criteria.criteria[].id). Fires when the mean score in the recent window drops below thresholdValue. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description carries the transparency burden. It discloses plan limitations, the special interpretation of thresholdValue for anomaly types as a sigma multiplier, and the required relationship between channelKinds and channelTargets. However, it does not mention authentication needs, rate limits, or any destructive behavior (though it's a creation).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that efficiently conveys the purpose, main features, and an example. It avoids redundancy and is front-loaded with the core action. However, it could be slightly more structured with bullet points for multiple behavior details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (12 parameters, nested objects, multiple alert types) and no output schema, the description is incomplete. It does not cover the conditions parameter for composite alerts, nor does it mention the evalCriterionId or guardian_findings use cases. Also, it does not hint at the return value (the created alert object). While the schema covers these, the description should provide a high-level summary of all major features.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents each parameter. The description adds value by explaining the relationship between channelKinds and channelTargets with an example, the sigma multiplier range for anomaly types, and an illustrative example of usage. This goes beyond the schema's individual field descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new alert rule, specifying the types of breaches it watches and the notification action. It distinguishes from siblings like update_alert and delete_alert by using 'create' and describing the creation process.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly compare to alternatives like propose_alert_rules or when not to use this tool. It implies usage for creating alerts but lacks usage guidance. It mentions a plan limitation (Free plan only email, up to 3 alerts) which is a constraint but not a usage alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_annotationA
Create a new annotation (human review / labeling) for an LLM call. Specify at least one of annotationText / label / qualityScore (an "empty annotation" gets 400 from the backend). Example phrasing: "Claude, label this call 'badly-summarized' with quality 2", or bulk-apply positive / negative labels for an eval loop. Combined with the eval baseline runner (run_eval), annotations can calibrate eval criteria as ground truth.
| Name | Required | Description | Default |
|---|---|---|---|
| label | No | Label (0-50 chars, alphanumerics plus _ - only). Usable as a dashboard filter | |
| callId | Yes | Target call id (query_calls.records[].id) | |
| qualityScore | No | Quality score (integer 1-5). Omit for NULL | |
| annotationText | No | Free-form comment (0-2000 chars). Length is validated by the backend |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses the empty-annotation error condition but could mention success response structure or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Compact paragraph with front-loaded purpose. All sentences add value, though could be slightly more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, constraints, and tool synergy (run_eval). Lacks return value description but output schema absent makes this acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage, but description adds critical constraint ('at least one of') and clarifies label pattern and lengths, exceeding schema baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create a new annotation' with specific resource and context (LLM call). Examples distinguish from sibling tools like update_annotation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states required combination of optional fields to avoid 400 error. Provides concrete examples and mentions integration with eval pipeline.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_budget_gateA
Create a runtime budget gate (Pro+ only). Sets a monthly LLM spend limit (USD) for the account; the SDK (budgetGate opt-in) blocks over-limit calls before execution. Enforcement is optimistic (spend is cached for 60 seconds and in-flight calls pass, so the limit is a guideline that can be exceeded, not a strict hard cap). enforceMode = fail_open (default; calls pass when the backend is unreachable) / fail_closed (calls are blocked when unreachable; a cold start where the SDK has never fetched the config additionally requires the SDK-side failClosed opt-in). Omitting projectId = an account-wide gate (only one; 409 if one exists). Specifying projectId = a gate for that project only (ANDed with the account gate — the strictest limit wins; one per project). Specifying tagKey + tagValue = a gate for calls carrying that tag (e.g. tagKey=service / tagValue=checkout caps the monthly spend of service=checkout. ANDed with the account gate; one per (tagKey,tagValue)). tagKey/tagValue must be specified together and are mutually exclusive with projectId. Example phrasing: "create a budget gate at $50/month" / "cap project X at $10/month" / "cap the service=checkout tag at $20/month"
| Name | Required | Description | Default |
|---|---|---|---|
| tagKey | No | Tag key for a per-tag gate (e.g. service). Must be specified together with tagValue; mutually exclusive with projectId. 1-128 chars | |
| enabled | No | Whether the gate is enabled (default true) | |
| tagValue | No | Tag value for a per-tag gate (e.g. checkout). Must be specified together with tagKey. 1-128 chars | |
| projectId | No | proj_-style project ID (omit for an account-wide gate). The project must belong to your account and not be archived | |
| enforceMode | No | Behavior when the backend is unreachable (default fail_open) | |
| monthlyLimitUsd | Yes | Monthly limit in USD (0.01 - 1000000, $0.01 increments). E.g. 50 / 100.5 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully bears the burden of behavioral disclosure. It explains optimistic enforcement, caching, in-flight pass, enforceMode behaviors, and scoping rules—all critical for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Packs extensive detail into a single paragraph without fluff. Every sentence adds value: scope variants, enforcement modes, error conditions, and example phrasings. Front-loaded with core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 params, scoping, enforcement, caching) and no output schema, the description covers all necessary aspects: side effects, error cases, inter-dependencies, and operational semantics. Nothing obvious is omitted.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, the description enriches each parameter with context: inter-parameter relationships (tagKey/tagValue mutual exclusivity, projectId omission for account-wide), example values, and implicit behavior (e.g., 409 for duplicate account gate). This goes well beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a runtime budget gate, specifying the verb and resource. It distinguishes from siblings like delete/update/get budget gates by explaining the creation-specific details, but does not explicitly contrast with them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides extensive context on when to use (creating a budget gate with various scopes) and constraints (e.g., duplicate account gate causes 409, tag params mutually exclusive with projectId). However, lacks explicit directive on when not to use (e.g., to update an existing gate).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_eval_criterionA
Create one custom eval criterion in your account (Pro+ only). name + rubric + scaleMin + scaleMax are required. Same name already existing in the account = 409. A name matching a global default is structurally allowed (UNIQUE (account_id, name) separates it from account_id IS NULL). type defaults to 'llm_judge' (judge LLM scoring). Specifying a deterministic evaluator type (exact_match / contains / regex / json_schema / json_path) scores without calling an LLM — free and instant (pass -> scaleMax / fail -> scaleMin). Deterministic types require config. The path an AI agent takes when it decides "add this criterion" during dogfood evals.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Criterion name (1-50 chars, starts with an alphanumeric, [A-Za-z0-9 _\-.] only). E.g. 'helpfulness' / 'concise' | |
| type | No | Evaluator type (default 'llm_judge'). Deterministic evaluators score without an LLM call — free and instant: 'exact_match' / 'contains' / 'regex' / 'json_schema' / 'json_path' | |
| scope | No | Evaluation scope (default call). call = per call; trajectory = scores multiple calls + steps in the same trace as one trajectory (llm_judge only) | |
| config | No | Type-specific settings (not needed for llm_judge). exact_match: {expectedOutput}, contains: {substring, caseSensitive?}, regex: {pattern, flags?}, json_schema: {schema}, json_path: {path, expectedValue?}. Categorical scoring also requires config.categories (2-10 entries, worst to best). | |
| rubric | Yes | Scoring rubric text (10-2000 chars; the narrative the judge LLM bases scores on. Required as a human-readable explanation even for deterministic evaluators) | |
| scaleMax | Yes | Score upper bound (1-100, must be greater than scaleMin) | |
| scaleMin | Yes | Score lower bound (1-100, must be less than scaleMax) | |
| scoreType | No | Scoring type (default numeric). boolean = pass/fail; categorical requires config.categories (llm_judge only) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses default type/scope, error conditions, and deterministic evaluator behavior (free/instant). It does not detail rate limits or auth but covers key behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Multiple sentences, some verbose (e.g., final sentence about AI agent path). Core purpose is front-loaded but could be tightened. Still organized and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 params, no output schema, and nested config, the description covers required fields, constraints, types, defaults, error behavior, and deterministic evaluator details. Missing return format but otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. Description adds context: deterministic types require config, rubric needed even for deterministic, examples for config fields, and default type/scope.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create one custom eval criterion' and specifies the scope 'in your account (Pro+ only)'. It distinguishes from siblings like get_eval_criterion, list, update, delete by focusing on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It mentions required fields, uniqueness constraint causing 409, and when to use deterministic types vs llm_judge. However, it does not explicitly contrast with propose_eval_criteria or other creation tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_eval_datasetA
Create a golden dataset (POST /v1/eval-datasets, Pro+ only). items can carry up to 20 test cases with expected outputs. Up to 50 datasets per account. frozen=true freezes the population (items can no longer be changed or unfrozen — fixing comparability for regression verdicts).
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Dataset name (1-100 chars, unique within the account) | |
| items | No | Test cases (up to 20). Each inputText is fed to the target model, and expectedOutput is used as the judge's [REFERENCE ANSWER] for scoring. | |
| frozen | No | true = freeze the population (items can no longer be changed or unfrozen). Omit = false. | |
| description | No | Optional description (<= 500 chars) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that frozen=true makes items permanently immutable, which is a critical behavioral detail. It also mentions the item limit. However, it does not describe the return value or error conditions, missing some behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: first defines the action, second covers limits, third explains frozen. All sentences are essential and front-loaded. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers key constraints and behavioral aspects. Lacks return value description and prerequisites (e.g., authentication), but for a creation tool with no output schema, it is mostly complete. The sibling tools provide context for retrieval.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value beyond the schema by explaining that inputText is fed to the model and expectedOutput is used as a judge reference, and that name must be unique within the account. This enriches parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action 'Create a golden dataset' with the HTTP method and resource, and includes the 'Pro+ only' restriction, distinguishing it from other dataset-related tools like run_eval_dataset or get_eval_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides important usage constraints: Pro+ only, up to 20 test cases, up to 50 datasets per account, and the effect of frozen=true. However, it does not explicitly mention when to use this tool versus alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_policy_gateA
Create a runtime policy gate (Pro+ only). Configures an account-wide model allowlist / PII block / secret block; the SDK (policyGate opt-in) blocks violating calls before execution. At least one rule (modelAllowlist / blockPii / blockSecrets) is required. One per account (409 if one exists). A redact mode is not supported (block only). Example phrasing: "only allow gpt-5.5 and claude-fable-5" / "block calls containing PII"
| Name | Required | Description | Default |
|---|---|---|---|
| enabled | No | Whether the gate is enabled (default true) | |
| blockPii | No | Block on PII detection. Coverage = email / card numbers (Luhn-verified) / delimited phone numbers / delimited national ID numbers / IPv4 / IPv6 (full and common compressed forms). Undelimited digit runs for phone / national ID numbers are excluded to avoid false blocking | |
| enforceMode | No | Behavior when the backend is unreachable (default fail_open) | |
| blockSecrets | No | Block on detection of API-key / private-key-like tokens | |
| modelAllowlist | No | Array of allowed model names (1-100 entries, exact match). Omit = no model restriction |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that redact mode is not supported, SDK blocks calls before execution, and at least one rule is required. No annotations provided, so description carries full burden and does so well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Messages are front-loaded and information-dense, but includes a longer example phrasing. Could be slightly trimmed without losing clarity, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description explains constraints, account limits, and SDK interaction. Does not mention return value, but likely expected for a creation tool. Adequately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. Description adds value by summarizing parameter purposes (modelAllowlist, blockPii, blockSecrets), gives examples, and explains PII coverage details beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Create' and resource 'runtime policy gate', describes its function (model allowlist/PII/secret block), and distinguishes from sibling tools like delete/update/get policy gate. Includes example phrasing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Specifies Pro+ only, required rules (at least one), account limit (one), and SDK opt-in. Does not explicitly state when not to use, but constraints are clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_projectA
Create a new project (POST /v1/projects). name = display name; slug = a short URL-safe identifier (/^[a-z][a-z0-9-]{0,31}$/). Pro caps at 5 projects, Team unlimited, Free cannot create (403). As a mutation, session-authenticated requests enforce Origin/Referer (dashboard-driven).
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Project display name (1-64 chars) | |
| slug | Yes | Short URL-safe identifier (/^[a-z][a-z0-9-]{0,31}$/, up to 32 chars, starts with a lowercase letter, hyphens allowed) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses mutation behavior, session authentication with CSRF protection (Origin/Referer), and plan-based restrictions. With no annotations provided, the description fully covers behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise single sentence that first states the core purpose, then efficiently adds plan details and auth requirements. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Fully covers all relevant context: action, parameters, plan restrictions, and authentication behavior. No output schema needed for this mutation tool; description is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good descriptions. The description adds clarity by explaining 'name = display name' and 'slug = a short URL-safe identifier', but mostly echoes schema info. Slight added value over schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action 'Create a new project' and specifies the HTTP method. Distinguishes from sibling create tools by focusing on project-specific details like plan limitations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage context including plan caps (Pro 5, Team unlimited, Free cannot create) and authentication requirements. This helps the agent decide when to use this tool and when not to.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_promptA
Register one new prompt template (Pro+ only). name + version + template are required; variables / labels / description are optional. An existing (name, version) pair returns 409 (UNIQUE constraint). Used when an AI agent auto-registers templates for dogfood evals / experiments.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Prompt name (series identifier, [A-Za-z0-9][A-Za-z0-9_-]{0,63}). E.g. 'customer_support' | |
| labels | No | Array of labels (up to 8, each [A-Za-z0-9][A-Za-z0-9_-]{0,31}). E.g. ['production', 'staging']. | |
| version | Yes | Version identifier ([A-Za-z0-9][A-Za-z0-9._-]{0,63}). E.g. 'v1' / '1.0.2' / '2026-06-03' | |
| template | Yes | Prompt body (non-empty, up to 50000 chars). {{var}} placeholders are filled from variables. | |
| variables | No | Default values for {{var}} placeholders in the template (plain object, 4096 bytes max after JSON serialization). Optional. | |
| description | No | Description (up to 500 chars). Optional. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, so description carries full burden. Discloses unique constraint (409 error) and Pro+ restriction. Does not mention auth needs, rate limits, or response format. Adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with purpose front-loaded. No filler, but could be slightly more structured (e.g., bullet points for use cases). Efficient overall.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 params, no output schema, and schema covering all fields, the description is fairly complete. Covers required fields, duplicate behavior, and use case. Missing return value description is acceptable without output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage 100% with descriptions. Description adds no extra semantics beyond restating required/optional fields. Baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool registers a new prompt template (Pro+ only). It specifies required and optional fields, and mentions auto-registration for dogfood evals/experiments, distinguishing it from siblings like update_prompt and delete_prompt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context: 'Used when an AI agent auto-registers templates for dogfood evals / experiments' and warns about duplicate (name, version) returning 409. Does not explicitly list alternatives but context implies creating vs updating.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_saved_viewA
Create a new saved view, or overwrite when the name exists (POST /v1/saved-views). name is unique within the account. filter follows the SavedViewFilter shape (startDate / endDate / provider / model / limit / preset / sortBy? / sortOrder?). Lets an AI agent save frequently used filters under a name — e.g. create a "last 7 days, GPT-4 only" view and recall it later.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Name of the saved view (1-80 chars, no line breaks). Overwrites an existing view with the same name | |
| filter | Yes | Filter shape = startDate (ISO) + endDate (ISO) + provider (may be empty) + model (may be empty) + limit (number) + preset (string|null) + sortBy? + sortOrder? |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the critical behavior of overwriting when the name exists ('overwrite when the name exists'), which is a key trait for an upsert operation. It also mentions name uniqueness and filter structure. No contradictions exist because no annotations are present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with three sentences. The first sentence states the main action and overwrite behavior. The second details the filter shape, and the third provides an example. No extraneous information; every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has two parameters (one nested), no output schema, and moderate complexity. The description adequately covers the main purpose and parameter meaning. However, it does not mention the return value of the tool, which would be helpful for an agent to know what to expect upon creation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds some value by explaining the naming uniqueness and providing an example use case ('last 7 days, GPT-4 only'), but it does not add significant parameter-level detail beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool's action: 'Create a new saved view, or overwrite when the name exists'. It specifies the resource (saved view) and the verb (create/overwrite), and the example ('last 7 days, GPT-4 only') clarifies the purpose. Among siblings like list_saved_views and delete_saved_view, this tool is clearly distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool: 'Lets an AI agent save frequently used filters under a name'. It provides context for use and mentions name uniqueness and overwrite behavior. However, it does not explicitly state when not to use or compare with alternatives, though the sibling list provides enough differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_webhookA
Register one outbound event webhook (Pro+ only, POST /v1/webhooks). url (HTTPS required; SSRF defense rejects private/loopback) + optional secret (HMAC-SHA256 signing key) + eventTypes (array of event kinds to subscribe to; omitted / empty = subscribe to everything). Up to 10 per account. Delivery payload = { event, eventId, occurredAt, accountId, data }; with a secret set, an X-Argosvix-Signature header is attached.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Destination URL (HTTPS required; private/loopback hosts are rejected) | |
| secret | No | Optional. HMAC-SHA256 signing key (when set, X-Argosvix-Signature is attached) | |
| enabled | No | Enabled flag (default true) | |
| eventTypes | No | Array of event kinds to subscribe to (omitted / empty array = all events). Available: approval.requested / proposal.executed / proposal.reversed | |
| description | No | Display memo (optional, up to 200 chars) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses: Pro+ requirement, SSRF defense, URL must be HTTPS, optional secret behavior, eventTypes behavior (omitted/empty = all events), limit of 10 per account, delivery payload format, and signature header. This is highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (3 sentences) and front-loaded with the purpose and constraints. Every sentence provides necessary information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is thorough, covering constraints, parameter behaviors, limits, and delivery format. However, with no output schema, it does not explicitly state what the API returns upon success (e.g., webhook ID). This is a minor gap for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining that eventTypes omitted/empty means subscribe to all (though schema also says that) and mentions the account limit of 10, which is not in the schema. It also describes the delivery payload format, which adds context beyond parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool registers an outbound event webhook, using the verb 'Register' and specifying 'Pro+ only, POST /v1/webhooks'. It distinguishes from sibling tools like delete_webhook, update_webhook, list_webhooks, test_webhook by focusing solely on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool (creating a webhook) but does not explicitly state when not to use it or list alternative tools. However, the context is sufficient for an agent to infer usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_alertA
Delete an alert (DELETE /v1/alerts/:id). Related alert_events are CASCADE-deleted too. To guard against accidental deletion, checking the details with get_alert first is recommended. If you only want to pause an alert, prefer silence_alert (mute) or update_alert with enabled=false instead of delete (both are recoverable).
| Name | Required | Description | Default |
|---|---|---|---|
| alertId | Yes | Target alert ID (from list_alerts) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses cascading delete of related alert_events and irreversibility compared to recoverable alternatives. No annotations provided, so description fully covers behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with main action, then cascade info, then alternatives and precaution. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter delete tool with good schema, description covers relevance: irreversibility, cascading effect, alternatives, and recommended precaution. Complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (alertId) with complete schema including pattern and description. Description adds 'from list_alerts' context, but schema already has high coverage. Baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Delete an alert' with HTTP method and endpoint, and distinguishes from sibling tools by mentioning alternatives like silence_alert and update_alert.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly describes when to not use (for pausing, prefer silence_alert or update_alert) and recommends checking details with get_alert first. Strong guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_annotationA
Delete an annotation (DELETE /v1/annotations/:id). No other rows depend on it, so there is no CASCADE impact. To guard against accidental deletion, checking the details with get_annotation first is recommended.
| Name | Required | Description | Default |
|---|---|---|---|
| annotationId | Yes | Target annotation id (AUTOINCREMENT integer) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description discloses key behaviors: the HTTP DELETE method, no cascading dependencies, and a safety recommendation. It does not mention irreversibility or auth requirements, but for a simple delete operation this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no redundant information. Every sentence adds value: the first states the action, the second provides safety context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has one parameter and no output schema. The description fully covers the purpose, behavior, and a usage recommendation, making it complete for its simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add extra meaning beyond the schema's description of annotationId as an AUTOINCREMENT integer. No parameter-level details are missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Delete an annotation' with the HTTP method and resource identifier. It distinguishes from sibling tools like create_annotation and update_annotation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states no CASCADE impact and recommends verifying with get_annotation before deletion. Provides clear context for safe usage, though does not explicitly mention when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_budget_gateA
Delete a runtime budget gate (Pro+ only). After deletion the SDK's pre-execution enforcement is disabled. To pause temporarily, prefer update_budget_gate with enabled: false.
| Name | Required | Description | Default |
|---|---|---|---|
| gateId | Yes | Target gate id (from get_budget_gate, starts with bg_) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that deletion disables SDK pre-execution enforcement and that it is Pro+ only. Does not contradict annotations (none provided). Lacks details on reversibility or other side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded, no unnecessary words. Every sentence provides value: action, consequence, and alternative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With only one parameter and no output schema, the description is complete. It explains the tool's effect, the only parameter, and provides usage context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a description of gateId. The description adds no new information beyond the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a runtime budget gate' with a specific verb and resource. It distinguishes from sibling tool 'update_budget_gate' by mentioning an alternative for temporary pausing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: 'To pause temporarily, prefer update_budget_gate with enabled: false', indicating when not to use delete and offering an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_eval_criterionA
Delete a custom criterion in your account (Pro+ only, DELETE /v1/eval-criteria/:id, 204). Global defaults (account_id IS NULL) are structurally out of scope = 404; other accounts are 404 too. WARNING: all past eval_run score rows for this criterion (eval_scores) are physically deleted at the same time via ON DELETE CASCADE — historical comparisons and score trend analysis become permanently impossible. This is not a tool for an AI agent to call casually while tidying up criteria; only proceed when the user has explicitly confirmed the past run scores are not needed. If you only want to rename, using update_eval_criterion (full replace) with name + rubric + scaleMin + scaleMax preserves the history.
| Name | Required | Description | Default |
|---|---|---|---|
| criterionId | Yes | Target criterion id (list_eval_criteria.criteria[].id) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the cascading delete of eval_scores, permanent loss of historical analysis, scope constraints (Pro+ only, 404 for global/other accounts), and the DELETE method and status code. No annotations present, so description fully covers behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose and method, then provides warnings and alternative. Efficient but slightly lengthy due to necessary warnings; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given destructive nature, no output schema, and single parameter, the description fully covers side effects, prerequisites, and behavior. Complete for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% so baseline is 3. The description does not add additional meaning beyond the schema's description of criterionId.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool deletes a custom criterion, distinguishes it from update_eval_criterion for renaming, and specifies the verb 'delete' and resource 'custom criterion'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns against casual use, advises only when user confirms not needing history, and directs to update_eval_criterion for renaming. Provides when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_eval_datasetA
Delete a golden dataset (DELETE /v1/eval-datasets/:id, Pro+ only). Items are cascade-deleted. Past eval runs / scores remain.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetId | Yes | Id of the dataset to delete |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description effectively discloses key behaviors: cascade deletion of items and preservation of past eval runs/scores. It could additionally note irreversibility but is otherwise transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the core purpose, endpoint, and tier. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and no output schema, the description covers the essential behavioral context (cascade, data retention). It could mention irreversibility but remains adequately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully covers the single parameter (datasetId) with a description. The tool description adds no additional meaning beyond what the schema provides, so baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('delete a golden dataset'), specifies the HTTP method and endpoint, and mentions the Pro+ tier restriction. It distinguishes from sibling tools like create_eval_dataset and list_eval_datasets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when wanting to delete a dataset but does not explicitly state when to use this tool instead of alternatives, nor does it provide exclusions or prerequisites beyond the tier.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_policy_gateA
Delete a runtime policy gate (Pro+ only). To pause temporarily, prefer update_policy_gate with enabled: false.
| Name | Required | Description | Default |
|---|---|---|---|
| policyId | Yes | Target policy id (from get_policy_gate, starts with pg_) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Although no annotations exist, the description adds the Pro+ edition restriction. The destructive nature is implied by 'delete', but additional details on permissions or reversibility would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with just two sentences, no unnecessary words, and perfectly front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and no output schema, the description covers the action, edition requirement, and an alternative. Could mention permissions or consequences, but overall adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter, and the description does not add new information about the parameter beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it deletes a runtime policy gate and specifies the Pro+ edition requirement. It distinguishes from update_policy_gate by suggesting an alternative for temporary pausing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends using update_policy_gate with enabled: false for temporary pausing instead of deleting, providing clear when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_projectA
Soft-delete a project (DELETE /v1/projects/:id; sets archived_at for a logical delete). The default project cannot be deleted (400, keeping accounts.default_project_id referentially consistent). After archiving, calls / alerts remain as-is (past observations are kept); route new records to another project.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | Id of the project to delete (via list_projects) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clearly explains the soft-delete mechanism (`archived_at`), the restriction on the default project, and the impact on related data (calls/alerts remain). This is transparent for a deletion tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, consisting of two sentences with no unnecessary words. It front-loads the key action and endpoint, then covers important details efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, the description explains the consequences of deletion (soft-delete, data retention, next steps). It is mostly complete but could mention undoability or more detailed consequences.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter with a description that matches the tool's description. The tool-level description adds behavioral context about the parameter (e.g., the default project cannot be deleted), providing extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs a soft-delete on a project, specifies the HTTP endpoint, and distinguishes itself from other delete tools by focusing on projects. The caveat about the default project being undeletable adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly indicates when to use the tool (to delete a project) and provides important context about limitations (default project cannot be deleted) and post-deletion behavior. However, it does not explicitly state when to avoid using it or suggest alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_promptA
Delete an existing prompt (Pro+ only, DELETE /v1/prompts/:id, 204 No Content). Scoped to your account (other accounts' ids return 404). WARNING: physical delete with no restore; past eval_runs' prompt_registry_id is SET NULL, losing the trace of which prompt template each run used (history comparisons can no longer be linked). Even when sunsetting an old version in a rotation, while past run traces remain it is safer to do a logical sunset via update_prompt with labels such as 'sunset'.
| Name | Required | Description | Default |
|---|---|---|---|
| promptId | Yes | Target prompt id (list_prompts.prompts[].id) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses the physical deletion, no restore, and the effect on eval_runs (setting prompt_registry_id to NULL, losing trace). Provides comprehensive behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with vital information front-loaded. Every sentence adds value; no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description explains the 204 response and side effects on eval_runs. Adequately covers all aspects given the tool's complexity and context from sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter description, but the description adds contextual semantics like 'Scoped to your account' and references list_prompts, adding value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Delete an existing prompt' and specifies the REST endpoint and response code. Distinguishes from siblings by emphasizing physical deletion and comparing to update_prompt for logical sunset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns that deletion is physical and irreversible, and advises using update_prompt with labels for safer sunsetting. Also notes scoping to user's account.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_saved_viewA
Delete the saved view with the given id (DELETE /v1/saved-views/:id). Scoped to your account.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Id of the saved view to delete (UUID) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully bears the transparency burden. It only states deletion without mentioning reversibility, permissions needed, side effects, or response details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence plus the endpoint and scope, conveying essential information with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description covers the basic purpose and scope, but lacks behavioral details like deletion permanence or success confirmation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (id parameter described with UUID). The description adds no additional meaning beyond what the schema already provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'saved view with the given id', and distinguishes from sibling tools by being deletion-specific and scoped to account.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It mentions 'Scoped to your account', providing context for when to use, but does not explicitly state when not to use or name alternatives among the many sibling delete tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_webhookA
Delete an outbound event webhook (Pro+ only, DELETE /v1/webhooks/:id). Other accounts' webhooks return 404. webhookId is the id from list_webhooks.
| Name | Required | Description | Default |
|---|---|---|---|
| webhookId | Yes | Target webhook id (owh_...) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given no annotations, the description carries full burden. It discloses the destructive nature, the 404 error for unauthorized webhooks, and the parameter source. However, it does not explicitly state irreversibility or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no extraneous words. All information is front-loaded and essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and no output schema, the description covers the action, endpoint, plan requirement, error case, and parameter source. It is nearly complete, though could mention finality.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by specifying that webhookId comes from list_webhooks, which aids in parameter population beyond the schema's basic description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it deletes an outbound event webhook, specifies the HTTP method and endpoint, and differentiates from sibling tools like create, list, update, test, and retry by focusing on deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides usage context by noting it's Pro+ only and that other accounts' webhooks return 404, but does not explicitly compare with alternatives or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deploy_promptA
Deploy a specific prompt version to a label (an environment such as production / staging) (Pro+ only, POST /v1/prompts/:id/deploy). If a deployment already exists, the prior version is kept as previous and rollback_prompt can revert in one step. Re-deploying the same version does not create a previous entry. Labels are per prompt name (each name has its own production version).
| Name | Required | Description | Default |
|---|---|---|---|
| label | Yes | Environment label (alphanumerics plus _ -, 1-32 chars. E.g. 'production' / 'staging') | |
| promptId | Yes | Id of the version to deploy (list_prompts.prompts[].id) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must cover behavioral traits. It discloses key behaviors (deployment overwrite logic, per-label namespacing) and mentions the 'Pro+ only' restriction. However, it does not describe the return value, error conditions, authentication requirements, or rate limits, leaving some transparency gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences that each add significant value. The first sentence states the purpose and constraints, the second explains critical behavioral details. No superfluous words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given two parameters, no output schema, and moderate complexity, the description covers deployment behavior and references rollback_prompt. However, it lacks information about the response format (e.g., what is returned upon success/failure) and does not explicitly state that prompt IDs come from list_prompts (though schema says so). It is nearly complete for an action tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% coverage with descriptions for both parameters ('label' with pattern and example, 'promptId' with type and source). The description adds minimal extra meaning beyond restating the deployment context and referencing 'rollback_prompt', so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Deploy a specific prompt version to a label'), identifies the resource ('prompt version'), and provides context ('environment such as production/staging'). It also mentions the HTTP endpoint and distinguishes from sibling 'rollback_prompt'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the behavior when a deployment already exists (prior version kept) and when re-deploying the same version (no new previous). It explicitly names 'rollback_prompt' as an alternative for reverting, but does not provide guidance on when not to use the tool or contrast with other siblings like 'get_deployed_prompt'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
detect_anomalyA
Compare the current window against a baseline window (the immediately preceding window of the same length) and detect anomalies across 4 axes: cost / latency / error_rate / call_volume — lets an AI grasp "is anything off?" in one prompt. Sensitivity is tunable via threshold: sensitive (1.5x) / normal (2x, default) / conservative (3x). 0-4 detections, each with a narrative. Returns { window, threshold, current: {...}, baseline: {...}, anomalies: [{ axis, severity: 'minor'|'major'|'critical', current, baseline, ratio, narrative }] }. errorRate is evaluated and displayed as a percent (0-100), matching the backend aggregate unit. Insufficient baseline data (fewer than 10 records in the period) yields anomalies: [] plus a warning message.
| Name | Required | Description | Default |
|---|---|---|---|
| window | No | Observation window ('1h' / '24h' / '7d', default '24h') | 24h |
| threshold | No | Sensitivity ('sensitive' 1.5x / 'normal' 2x / 'conservative' 3x, default 'normal') | normal |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully shoulders the burden. It details the baseline window logic, tunable sensitivity with explicit thresholds (1.5x, 2x, 3x), the 4 axes, output structure, errorRate formatting as percent, and an edge case for insufficient data. This is comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that efficiently covers purpose, parameters, output, and edge cases without redundancy. It is front-loaded with the core comparative logic. Minor room for restructuring but overall compact and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description fully specifies the return structure including nested objects, severity levels, ratio, and narrative. It also covers the edge case of insufficient baseline data. For a tool with 2 parameters and a well-defined output, this is complete and self-contained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters documented, but the description adds meaning: it maps threshold enum values to multiplier meanings (sensitive=1.5x, etc.) and explains that window is the observation period. It also clarifies the 4 axes of detection, which is not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it compares a current window to a baseline to detect anomalies across 4 specific axes (cost, latency, error_rate, call_volume). This is unique among the sibling tools, which focus on CRUD, alerts, evaluations, etc., making it easily distinguishable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says it lets an AI grasp 'is anything off?' in one prompt, indicating usage for anomaly detection. While it doesn't list when not to use or alternatives, the context is clear and no similar sibling tool exists, making usage intuitive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
export_callsA
Large-batch export of calls (POST /v1/query/export). Higher limit than query_calls (per-plan max records: Free 1000 / Pro 50000); available on all plans. Filter axes = startTime / endTime / provider / model plus limit. Example phrasing: "pull all of last month's GPT-4 calls and analyze the trends" — one call. The result format is the same JSON as query_calls (the AI can feed it straight into CSV / statistics).
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Cap on returned records. Passed through when within the plan max; clamped to the plan max beyond it | |
| model | No | Model name filter (exact match, no substring matching; e.g. 'gpt-4o-mini') | |
| endTime | No | Range end ISO timestamp (UTC; omit = now) | |
| provider | No | Provider filter (openai / anthropic / google / azure / cohere) | |
| startTime | No | Range start ISO timestamp (UTC; omit = all time) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully handles behavioral disclosure. It explains the export operation, plan-specific limits, limit clamping behavior, and that the result format matches query_calls. This provides adequate transparency without contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise—three sentences with no wasted words. It front-loads the purpose and key differentiator, then adds necessary details (limits, filters, example). Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, 100% schema coverage, no output schema, and sibling tools like query_calls, the description is sufficiently complete. It mentions the result format and provides guidance on limits and filters, covering essential context for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all parameters described). The description adds value by explaining the 'limit' parameter's clamping behavior and summarizing filter axes, which goes beyond the schema's individual descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Large-batch export of calls (POST /v1/query/export).' It distinguishes from the sibling query_calls by highlighting a higher limit and per-plan maximums, providing a specific verb+resource+scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly compares to query_calls, noting 'Higher limit than query_calls' and per-plan record limits. It provides filtering axes and an example usage, offering clear context for when to use this tool instead of alternatives. No explicit when-not-to-use, but the comparison suffices.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
extend_customer_trialA
Extend your account's Stripe subscription trial by 1-30 days (POST /v1/tier2/trial/extend). Founder-operations only (an internal support tool; general accounts get 403). Trial extension directly affects revenue, so there is no plan to open it up. Cumulative cap of 60 days (aggregated from the last 30 days of audit logs); 409 unless status='trialing'. dryRun must be passed explicitly (guards against accidental mutation via an implicit false); when dryRun=false, idempotencyKey is also required (16-128 alphanumeric plus '_-'). Re-calling with the same key returns the cached result via the tier2_idempotency table (structurally preventing retry double-extends). dryRun=true previews previousTrialEnd / newTrialEnd / the cumulative total only (no Stripe call); dryRun=false performs the actual Stripe mutation plus the accounts_subscription sync update.
| Name | Required | Description | Default |
|---|---|---|---|
| dryRun | Yes | Must be passed explicitly. true = preview only; false = actual trial extension + Stripe mutation | |
| reason | Yes | Reason for the extension (recorded in the audit log; required, 200 chars max) | |
| approvalId | No | Approval id granted via request_approval (apr_ + 32 hex; create with action 'extend_customer_trial'). Server-side verification + atomic consumption on a fresh execution (1 approval = 1 execution chain; retries with the same idempotencyKey do not re-consume). dryRun only verifies | |
| extendDays | Yes | Days to extend (1-30, cumulative cap 60 days) | |
| idempotencyKey | No | Required when dryRun=false. 16-128 chars alphanumeric plus '_-'; re-calls with the same key return the cached result | |
| targetAccountId | Yes | Target account id (your own account only for now; specifying another user gets 403) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It details the dryRun behavior (preview vs mutation), idempotency key caching, cumulative cap, Stripe mutation side effects, and approvalId atomic consumption. Comprehensive disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense single paragraph with all essential information front-loaded. Every sentence adds value. Slightly long but appropriate for the complexity; could be broken into bullet points for readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, yet the description explains return values for dryRun (preview of dates and cumulative total) and the actual mutation effects. Covers error conditions (403, 409) and status checks. Complete for a sensitive financial tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds critical context beyond schema: dryRun must be explicit, idempotencyKey required when dryRun=false, cumulative cap explanation, and atomic approvalId consumption. Enriches all parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'extend' and the resource 'Stripe subscription trial' with a specific range (1-30 days). Includes the HTTP endpoint and distinguishes itself from sibling tools by noting it's an internal support tool for founder-operations only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (founder-operations, status='trialing'), when not to use (general accounts get 403), and limitations (cumulative cap of 60 days). Provides clear context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_account_healthA
Get a health summary of your LLM infrastructure in one call. Fetches 4 existing endpoints in parallel (aggregate_calls / get_percentiles / get_llm_budget / list_audit_log) and compresses them into one response. Returns { window, totals: {calls, costUsd, errorRate (percent 0-100)}, latency: {p50, p95, p99 (ms)}, budget: {used, limit, percentUsed (0-100)}, recentEvents: count, summary: 'ok' | 'warn' | 'critical' }. critical = errorRate>=10% / budget>=90% / p95>=10s; warn = >=3% / >=70% / >=3s. Example phrasing: "how is our LLM infra doing right now?" — answered in one prompt. Pure read aggregator (no new backend endpoint); individual endpoint failures return partial results (one axis timing out does not block the summary).
| Name | Required | Description | Default |
|---|---|---|---|
| window | No | Observation window ('1h' / '24h' / '7d', default '24h') | 24h |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: it is a pure read aggregator that fetches endpoints in parallel, returns partial results if one endpoint fails, and defines clear thresholds for 'critical' and 'warn' states. This exceeds expectations for transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is detailed but well-structured, starting with purpose, then return type, thresholds, example, and behavior. It could be slightly more concise, but every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description fully explains the return structure (including exact field names and types) and behavior on partial failures. This is complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter 'window', and the schema already includes a description. The tool description does not add additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it gets a health summary of LLM infrastructure by fetching 4 existing endpoints in parallel, distinguishing it from sibling tools like aggregate_calls, get_percentiles, etc. The verb 'get' and resource 'account health' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides an example query ('how is our LLM infra doing right now?') and explains the tool returns a compressed summary, implying it should be used for quick health checks. However, it lacks explicit when-not-to-use guidance or direct comparisons to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_alertA
Return the detailed configuration of an alert and its recent trigger history. Pass the alertId obtained from list_alerts. Use it to check the threshold / notification channels / silence state / when it fired.
| Name | Required | Description | Default |
|---|---|---|---|
| alertId | Yes | Target alert ID (from list_alerts) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It implies a read operation (returning data) but does not explicitly state it is read-only, nor does it mention authorization needs, rate limits, or other side effects. Adequate but could be more transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, front-loading the core purpose in the first sentence and providing usage guidance in the second. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter and no output schema, the description adequately explains what is returned (configuration and history) and how to get the ID. It covers the essential information for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the alertId parameter. The description adds value by specifying the origin of the ID ('from list_alerts') and listing what to check, which goes beyond the schema's basic 'Target alert ID'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns the detailed configuration and recent trigger history of an alert. It lists specific attributes (threshold, notification channels, silence state, when it fired), distinguishing it from sibling tools like list_alerts or list_alert_events.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the agent to pass the alertId obtained from list_alerts and to use it for checking specific details. While it doesn't state when not to use it, the context among siblings (create, update, delete) makes the use case clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_annotationA
Fetch one annotation by id (obtained from list_annotations_*). Includes annotationText / label / qualityScore / callId / createdAt / updatedAt / createdByUserId. Ids belonging to other accounts return 404 (structural defense).
| Name | Required | Description | Default |
|---|---|---|---|
| annotationId | Yes | Target annotation id (AUTOINCREMENT integer) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full behavioral disclosure. It mentions the structural defense (404 for other accounts) and lists returned fields. It could add more details like authentication requirements or error states, but it covers the key behaviors for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no extra words. First sentence gives purpose and id source, second lists data and added behavior. Information is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple GET by ID tool with one parameter and no output schema, the description is complete. It mentions the source of the id, the fields returned, and a security behavior. No missing critical information for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by stating that the id comes from list_annotations_*, providing context beyond the schema's 'Target annotation id' description. This helps the agent understand the correct source for the id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch one annotation by id', specifying the verb and resource. It distinguishes from sibling tools like list_annotations_* that retrieve multiple, and create/update/delete tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies that the id should be obtained from list_annotations_*, guiding the agent on prerequisite steps. It also notes that ids from other accounts return 404, a usage constraint. However, it does not explicitly exclude use cases like updates or deletions, but the context from sibling names makes that clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_approvalA
Get the current state of an approval request. status = pending / approved / denied / expired. Do not perform the target operation unless the status is approved (default-deny). Dangerous mutation tools also support server-side consumption via their approvalId param (see the request_approval description).
| Name | Required | Description | Default |
|---|---|---|---|
| approvalId | Yes | The id returned by request_approval (starts with apr_) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses the returned status values and the default-deny security model, and clarifies the relationship with mutation tools. Could mention idempotency or rate limits, but for a simple read tool it is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, entirely relevant, no filler. The key purpose and security note are front-loaded. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with one parameter and no output schema, the description is fully complete. It covers the return values, the required condition for proceeding, and references the associated request_approval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and schema already describes approvalId. The description adds helpful context that the ID starts with 'apr_', which aids proper parameter usage. This goes beyond the schema, so above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves the current state of an approval request, listing all possible status values and referencing the companion request_approval tool, which distinguishes it from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns not to perform the target operation unless status is approved (default-deny), and explains how the approvalId can be used server-side by dangerous mutation tools. This provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_budget_gateA
Get the runtime budget gate settings (runtime control plane Phase 1) plus this month's LLM spend. Response = { gates: [{ id, projectId, monthlyLimitUsd, enforceMode, enabled, ... }], spentUsdThisMonth, monthStart, ttlSeconds }. monthStart is the UTC month start. The same source the SDK's budgetGate opt-in evaluates before execution. Distinct from get_llm_budget (which caps Argosvix's internal AI feature costs) — this one is a monthly cap on your own LLM spend. Example phrasing: "how much budget gate headroom is left this month?" / "is the gate set to fail_open?"
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the returned fields (gates, spentUsdThisMonth, monthStart, ttlSeconds) and notes that monthStart is UTC. It also mentions the tool's relationship to the SDK's budgetGate opt-in. Lacks details on freshness/caching or authorization, but is fairly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concisely structured: first sentence summarizes purpose, then response format, then contextual details, then differentiation, then example queries. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description sufficiently explains the response fields and behavior. It covers the core functionality and distinguishes from related tools. Could potentially mention if multiple gates are expected, but overall it is complete for a straightforward retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, and the description appropriately indicates no inputs are needed. Schema coverage is 100% (empty schema), so no additional parameter details are required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves budget gate settings and monthly LLM spend. It specifies the response structure and explicitly distinguishes from the sibling get_llm_budget tool, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use this tool ('how much budget gate headroom is left this month?') and distinguishes it from get_llm_budget. It could be improved by explicitly stating scenarios where it should not be used, but the distinction and examples are helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_cost_summaryA
Return cost / call count / token aggregates per time range, with a per-provider breakdown. When groupBy="none" is specified, a per-provider breakdown is still returned for backend compatibility (check the response.total field for the overall sum).
| Name | Required | Description | Default |
|---|---|---|---|
| groupBy | No | Aggregation axis (overall sum / per provider / per model). Default provider | provider |
| rangePreset | No | Aggregation range. Default 7d | 7d |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the behavioral nuance that groupBy='none' still returns per-provider breakdown with overall sum in response.total, but it does not mention whether the tool is read-only, any authentication requirements, or potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences with no wasted words. The first sentence clearly states the primary purpose, and the second provides a critical behavioral note. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has only two enum parameters and is fairly simple, but the description omits details about the response structure (besides mentioning response.total) and any error conditions. Given no output schema, a more complete description would include typical return fields or constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions, setting a baseline of 3. The description adds extra meaning beyond the schema by explaining the backend compatibility behavior when groupBy='none' and pointing to the response.total field, which is not in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Return') and resource ('cost / call count / token aggregates per time range') and distinguishes itself from siblings like get_llm_budget or get_percentiles by clearly stating it provides aggregates with per-provider breakdown.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving cost summaries, and the note about groupBy='none' clarifies a special case. However, it does not explicitly state when to use this tool over alternatives such as get_llm_budget or get_percentiles, though the purpose is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_deployed_promptA
Resolve and return the prompt version currently deployed to the given environment (name + label) (GET /v1/prompts/resolve, available on Free). The main runtime path for an agent to fetch the "production prompt". Returns that version's template / variables / labels / version. 404 when nothing is deployed.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Prompt name | |
| label | Yes | Environment label |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses HTTP method, endpoint, availability (Free), return fields (template/variables/labels/version), and error case (404 when nothing deployed). No annotations to contradict.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with main action, no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with 2 parameters and no output schema, the description covers key aspects: purpose, return fields, error condition, and endpoint info.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both name and label parameters. Description adds minimal value by grouping them as 'environment (name + label)', but does not provide additional details beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Resolve and return' and resource 'prompt version currently deployed', and distinguishes from siblings like get_prompt by specifying 'deployed' and 'main runtime path for an agent to fetch the production prompt'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context 'main runtime path for an agent to fetch the production prompt' implying when to use, but does not explicitly state when not to use or directly contrast with alternative tools like get_prompt.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_eval_criterionA
Fetch one criterion's detail (name / rubric / scaleMin / scaleMax / createdAt) by id. The id comes from list_eval_criteria.criteria[].id. Both global defaults (accountId NULL) and your account's custom criteria are accepted; other accounts' customs return 404 (structural defense).
| Name | Required | Description | Default |
|---|---|---|---|
| criterionId | Yes | Target criterion id (AUTOINCREMENT integer) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description effectively discloses the tool's behavior: it accepts global and own custom criteria, rejects others with 404, and mentions 'structural defense'. It could further detail response format or auth requirements, but it's sufficient for a simple fetch operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The purpose is front-loaded, and the contextual guidance follows efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter, no output schema, and no annotations, the description covers all essential aspects: what it returns, where the ID comes from, and the scope of valid IDs. It is complete and self-contained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description of criterionId. The tool description adds value by explaining the ID's origin (from list_eval_criteria) and the acceptance rules, which goes beyond the schema's autoincrement note.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Fetch' and the resource 'criterion's detail', listing specific fields (name, rubric, scaleMin, scaleMax, createdAt). It distinguishes from sibling tools like list_eval_criteria and update_eval_criterion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly tells where to get the ID ('from list_eval_criteria.criteria[].id') and explains which criteria are accepted (global defaults and own account's customs) and what happens with other accounts' customs (404). This provides clear when-to-use and exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_eval_datasetA
Fetch one dataset's detail plus all of its items (GET /v1/eval-datasets/:id). datasetId is list_eval_datasets.datasets[].id.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetId | Yes | Target dataset id (list_eval_datasets.datasets[].id) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes return content (detail plus all items) and idempotent fetch behavior. No annotations provided, so description carries full burden. Could mention that it is read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb, no extraneous words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately describes purpose, endpoint, and parameter source for a simple fetch tool. Lacks details on response format, but no output schema exists. Could mention potential limitations like pagination.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers parameter with name, type, and basic description. Description adds value by specifying the source (list_eval_datasets.datasets[].id), aiding correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (Fetch), resource (one dataset's detail plus all items), and explicit endpoint (GET /v1/eval-datasets/:id). It distinguishes from siblings like list_eval_datasets and delete_eval_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. Implicitly indicates dataset ID comes from list_eval_datasets, but does not compare with siblings like get_eval_run or get_eval_criterion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_eval_runA
Fetch one eval run's detail plus the list of per-(criterion x call) scores. Use runs[].id from list_eval_runs as-is. The scores array includes score (an integer within the criterion's scale) + reasoning (the judge's rationale). Same endpoint as the argosvix://eval-runs/{id} resource template.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | Target eval run id (AUTOINCREMENT integer) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Describes output structure (scores array with integer and reasoning) and references endpoint. Lacks details on error handling or existence checks, but adequate for read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each with distinct purpose: action, input sourcing, output details. Front-loaded and no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description details key parts of return value (scores array structure). Lacks description of 'detail' object, but sufficient for a fetch tool with clear input pattern.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (runId) with full schema coverage. Description adds value by instructing to use ID from list_eval_runs 'as-is', providing semantic context beyond the schema's 'Target eval run id'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Fetch' and resource 'eval run's detail plus scores'. It distinguishes from sibling 'list_eval_runs' by instructing to use its ID.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance to use 'runs[].id from list_eval_runs as-is', indicating when to use this tool after listing. Does not mention when not to use or alternatives like compare_eval_runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_llm_budgetA
Get the current monthly LLM feature budget (the LLM cost cap covering the 3 axes: safety classifier + secondary PII audit + eval baseline runner). Response = { budgetUsd, spentUsd, remainingUsd, periodStart, defaultBudgetUsd, minBudgetUsd, maxBudgetUsd }. Readable on Free and Pro+ alike; used when an AI agent decides "have we hit 80% of budget?" / "should we raise it?". Default $5/month; auto-resets at month boundaries (per YYYY-MM).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully covers behavior: it is a read-only operation (implicit by 'Get'), returns specific fields, mentions auto-reset at month boundaries, and states availability to all tiers. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the purpose and response fields, and provides usage context efficiently. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description thoroughly explains the response fields and includes important context like the default budget, auto-reset behavior, and usage scenarios. It is complete for an AI agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters (100% coverage), so the description does not need to document parameters. It adds value by explaining the response structure and usage context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves the current monthly LLM feature budget, specifies the three axes it covers, and lists the response fields. It distinguishes from sibling tools like 'raise_llm_budget' and 'get_budget_gate' by explicitly stating its read-only nature and purpose for checking budget thresholds.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when the tool is used: by an AI agent deciding if the budget has hit 80% or if it should be raised. It notes availability across tiers. However, it does not explicitly mention when not to use it or compare with alternatives like 'raise_llm_budget' or 'get_budget_gate'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_percentilesA
Get percentile metrics over calls (POST /v1/query/percentiles). metric = 'latency' (ms) or 'cost' (USD); either a single value for the whole range, or a time series with groupBy='day'/'hour'/'minute'. Example phrasing: "daily p95 latency trend for last week". Computed with the nearest-rank method via window functions (D1 SQLite has no percentile_cont).
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | Model filter | |
| metric | No | Metric kind, default = 'latency' | latency |
| endTime | No | Range end ISO timestamp | |
| groupBy | No | Time-series bucketing (omit = one value for the whole range, 'day' = daily, 'hour' = hourly, 'minute' = per minute) | |
| provider | No | Provider filter | |
| startTime | No | Range start ISO timestamp (UTC; omit = all time) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It explains the computation method (nearest-rank via window functions) and notes implementation specifics (D1 SQLite limitation). It also clarifies output behavior (single value vs. time series). Some details about response format are missing, but the key behaviors are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences convey the core purpose, available options, and implementation detail. No wasted words; every sentence adds meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 optional parameters and no output schema, the description covers essential behavior (metric types, time grouping, computation). It lacks explicit mention of return structure, but the example phrasing and method hint at the output. Overall, it is sufficiently complete for an agent to understand the tool's function.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by explaining the metric and groupBy semantics in natural language, including defaults and example. It goes beyond the schema by describing how to interpret groupBy (time bucketing) and provides a concrete usage example.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves percentile metrics over calls, specifies metric options (latency/cost) and time series capability with groupBy. It references the specific API endpoint and provides an example phrase, making the purpose distinct from sibling tools like query_calls or get_cost_summary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use the tool (for percentile queries) and describes the groupBy options for time series. It does not explicitly state when not to use it or compare to alternatives, but the context is clear enough for an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_policy_gateA
Get the runtime policy gate settings (runtime control plane Phase 2). Response = { policy: { id, modelAllowlist, blockPii, blockSecrets, enforceMode, enabled, ... } | null }. The config the SDK's policyGate opt-in evaluates locally before each LLM call (exact-match model allowlist + blocking on PII / secret detection). Example phrasing: "what model restrictions are active right now?" / "is PII blocking enabled?"
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It explains the response format (policy object or null), mentions SDK local evaluation, and gives behavioral context. It lacks disclosure of authentication needs or rate limits, but for a zero-parameter read operation this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized with each sentence adding value: purpose, response shape, SDK context, example queries. It could be slightly more concise but is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and no output schema, the description fully explains the return value and usage context. It includes example phrasings and mentions the relationship to SDK configuration, making it complete for a simple getter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so baseline is 4. The description adds no parameter information, but none is needed since schema coverage is 100% (vacuously).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get the runtime policy gate settings', using a specific verb (get) and resource (policy gate). It distinguishes itself from sibling tools like create_policy_gate, update_policy_gate, and delete_policy_gate by focusing on reading configuration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides example phrasing ('what model restrictions are active right now?', 'is PII blocking enabled?') that tells the agent when to use this tool. It mentions runtime control plane context but does not explicitly state when not to use it or list alternatives, though the sibling list implies other actions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_promptA
Fetch one prompt's detail by id. Use prompts[].id from list_prompts as-is. Includes template + variables + labels + description; scoped to your account (structurally enforced by a backend WHERE clause — other accounts' ids return 404). Same endpoint as the argosvix://prompts/{id} resource template.
| Name | Required | Description | Default |
|---|---|---|---|
| promptId | Yes | Target prompt id (AUTOINCREMENT integer) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses account-scoping and 404 behavior, but does not explicitly state read-only nature, idempotency, or permissions. Adequate but could be more thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. Front-loaded purpose, then usage and behavioral details. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, return contents (template, variables, labels, description), id provenance, scoping, and error behavior. For a simple get-by-id tool without output schema, this is fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by instructing to use prompts[].id from list_prompts as-is, which is beyond the schema's description. This provides practical context for the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch one prompt's detail by id,' specifying the verb, resource, and method. It distinguishes from siblings like list_prompts (multiple) and get_deployed_prompt (deployed version) by mentioning the id usage and what is included.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly advises using prompts[].id from list_prompts, providing a direct link to the source of the id. It also notes account scoping and 404 behavior for invalid ids, guiding proper usage. Lacks explicit mention of alternatives like get_deployed_prompt but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_proposal_threadA
Return the thread for a proposal (the questions asked so far and the AI's replies). Get proposalId from list_proposals.
| Name | Required | Description | Default |
|---|---|---|---|
| proposalId | Yes | Target proposal ID (from list_proposals, starts with prp_) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description implies a read-only operation by stating 'return', but does not explicitly confirm safety, permissions, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, both essential: first defines purpose, second gives usage guidance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description fully covers what the agent needs to know.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description of proposalId. The description adds value by specifying the source (list_proposals) and format (starts with prp_).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it returns the thread for a proposal (questions and AI replies), differentiating from sibling tools like list_proposals or reply_proposal.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description tells the user to get proposalId from list_proposals, providing a clear prerequisite. It does not explicitly exclude other uses, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_safety_assessmentA
Fetch one assessment's detail by id. Use assessments[].id from list_safety_assessments as-is. Includes labels (array of flagged categories) + score (max category score 0-1) + reasoning + classifier_id + source. Same endpoint as the argosvix://safety-assessments/{id} resource template.
| Name | Required | Description | Default |
|---|---|---|---|
| assessmentId | Yes | Target assessment id (AUTOINCREMENT integer) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description compensates by listing exact return fields (labels, score, reasoning, classifier_id, source) and noting the endpoint resource template, indicating a read-only GET operation with no side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: purpose, usage hint, and return details. Front-loaded and free of redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter tool with no output schema, description fully covers purpose, input sourcing, and output structure. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already describes assessmentId as 'AUTOINCREMENT integer'. Description adds value by specifying where to obtain the id (from list_safety_assessments), providing usage context beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Fetch one assessment's detail by id', specifies verb and resource, and distinguishes from list counterpart by referencing list_safety_assessments. It also outlines return fields, providing complete purpose clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit instruction to use assessments[].id from list_safety_assessments, leaving no ambiguity about input source. Does not explicitly state when not to use, but given the singular nature of the tool, guidance is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_alert_eventsA
Return alert trigger events, newest first. Account-wide (recent firings of all alerts) by default; pass alertId to narrow to one alert. Use for questions like "which alerts fired recently and how often?" or "when did the cost alert go off?". Each event's id can be passed directly to the acknowledge_alert tool. acknowledgedAt / acknowledgedBy are null if not yet acknowledged. Each event includes a snapshot of thresholdValue / windowMinutes / alertType at firing time (so the firing-time conditions survive later rule edits). For the next page, pass the last event's triggeredAt + id as beforeTriggeredAt + beforeId (keyset cursor).
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Number of events to return (1-100, default 20) | |
| alertId | No | ID to narrow to a specific alert. Omit for all alerts' trigger history | |
| beforeId | No | Pagination cursor (id of the last event on the previous page). Must be passed together with beforeTriggeredAt | |
| beforeTriggeredAt | No | Pagination cursor (triggeredAt of the last event on the previous page). Must be passed together with beforeId |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: pagination mechanism (keyset cursor with beforeTriggeredAt + beforeId), null values for unacknowledged events, inclusion of snapshot fields (thresholdValue, windowMinutes, alertType), and that event ids can be passed to acknowledge_alert. This is highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately sized and front-loads the core action. Every sentence adds distinct information (pagination, snapshot, acknowledgment). Slightly longer than necessary but not wasteful. Could potentially be tightened but remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description covers key return fields (acknowledgedAt/By, snapshot values) and pagination. It explains account-wide vs narrow, the use for acknowledgment, and how to paginate. This is complete for a list tool with no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining the pagination parameters (must be passed together), clarifying that alertId narrows to one alert vs all, and stating the default limit. This enriches the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Return alert trigger events, newest first' and distinguishes account-wide vs narrowed by alertId. It is a specific verb+resource with scope differentiation, clearly distinguishing from sibling tools like list_alerts (which lists alert definitions) and acknowledge_alert.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides example questions that the tool answers ('which alerts fired recently and how often?' and 'when did the cost alert go off?'), giving clear context. It does not explicitly state when not to use it or mention alternatives, but the provided examples effectively guide usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_alertsA
Return the list of configured alerts plus trigger history within the last 24 hours.
| Name | Required | Description | Default |
|---|---|---|---|
| includeTriggered | No | true = include trigger history (triggered_at within the last 24h) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It mentions the 24-hour window for trigger history, which is a key behavioral trait, but lacks details on permissions, safety, or response format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the core action, no filler. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one optional parameter and no output schema, the description covers the essential context: what is returned and the time constraint. Minor omission of pagination or ordering but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the parameter description already covers the trigger history inclusion. The tool description mostly echoes the schema, adding no new semantic context beyond the time range.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Return' and clearly identifies the resource as 'configured alerts' and 'trigger history.' It distinguishes from siblings like get_alert (singular) and create_alert by explicitly stating it returns a list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance is provided. It implies usage for listing alerts and recent history, but does not compare with alternatives like list_alert_events or acknowledge_alert.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_annotations_by_labelA
Return annotations carrying the given label, newest first (account-wide, up to 100). Use for things like collecting calls rated "good" or listing human reviews labeled "bug". Labels are ASCII letters / digits / underscore / hyphen only ([a-zA-Z0-9_-], up to 64 chars).
| Name | Required | Description | Default |
|---|---|---|---|
| label | Yes | Target label (exact match with the label set at annotation creation) | |
| limit | No | Number of annotations to return (1-100, default 20) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full behavioral burden. It discloses sorting order, account-wide scope, and maximum return count, plus label format constraints. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences: first states core function, second adds examples and format constraints. No wasted words, information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all essential aspects: what it returns (annotations by label), ordering, scope, limit. Could mention pagination or response structure but not critical for a simple list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers both parameters fully. Description adds value via usage examples for the label parameter, clarifying its intent beyond the schema's technical description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns annotations by label, with scope 'account-wide' and order 'newest first', distinguishing it from sibling list_annotations_for_call which is per-call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides concrete examples ('collecting calls rated good', 'listing human reviews labeled bug') that clarify when to use. Lacks explicit exclusion of alternatives, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_annotations_for_callA
Return the annotations attached to an LLM call (records[].id from query_calls). An annotation is a user-authored evaluation (rating / comment / label); each annotation includes annotationText / label / qualityScore / createdAt / updatedAt. Use to check whether a call has human review attached or what past reviews said. Independent of the Pro+ plaintext feature (annotations work without plaintext storage enabled).
| Name | Required | Description | Default |
|---|---|---|---|
| callId | Yes | Target call id (query_calls.records[].id) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes what an annotation contains and clarifies independence from plaintext feature, but lacks behavioral details like pagination, ordering, or auth requirements. Without annotations, some burden is carried, but still gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, each purposeful. Front-loaded with purpose, then defines annotations, then use case, then clarifies a nuance. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Explains return fields and usage context, but missing details on pagination, error handling, and explicit mention that it returns a list. Simple tool but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter (callId) is well-described in the schema (100% coverage). The description repeats the source ('records[].id from query_calls') which adds no new meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Return') and resource ('annotations attached to an LLM call'), distinguishing it from siblings like list_annotations_by_label (by label) and get_annotation (single annotation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: 'to check whether a call has human review attached or what past reviews said.' Provides context but does not explicitly mention when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_approvalsA
List approval requests (latest 50). status filter = pending (default) / approved / denied / expired / all.
| Name | Required | Description | Default |
|---|---|---|---|
| status | No | Status filter (default pending) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the limit of 50 and valid status values, but no annotations exist. Does not mention ordering, pagination, or whether it is read-only. The description partially informs behavior but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, direct sentences. Front-loaded with action and key constraint. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple list tool but lacks description of output fields and pagination beyond the limit. No output schema exists, so the description should cover return value details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear enum and default. Description only restates the schema values, adding no new meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (List) and resource (approval requests) with a constraint (latest 50) and filter options. Distinguishes from siblings like create_approval or get_approval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for listing approvals with status filter, but lacks explicit guidance on when to use this tool versus other list tools (e.g., list_proposals) or conditions for using non-default status filters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_audit_logA
List the audit log (GET /v1/audit-log). Scoped to your account; admin role only (viewer/member get 403). Lets an AI agent autonomously review recent operation history such as invitations / API key revocations / project changes. Filters = eventType ('invitation.created' / 'api_key.revoked' etc.) / targetKind / actorUserId / from / to. Supports cursor pagination (nextCursor format = 'created_at|id'), max limit 200.
| Name | Required | Description | Default |
|---|---|---|---|
| to | No | Range end ISO timestamp (UTC) | |
| from | No | Range start ISO timestamp (UTC) | |
| limit | No | Number of results (1-200, default 50) | |
| cursor | No | Pagination cursor (pass the previous response's nextCursor as-is, 'created_at|id' format) | |
| eventType | No | Exact-match event_type filter ('invitation.created' / 'api_key.revoked' / 'membership.removed' etc.) | |
| targetKind | No | target_kind filter ('invitation' / 'api_key' / 'membership' etc.) | |
| actorUserId | No | actor_user_id filter (only a specific user's operations) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses admin scoping, cursor pagination format (nextCursor = 'created_at|id'), max limit 200, and available filters. Without annotations, it covers essential behavioral traits well. Could add that it is read-only, but 'list' implies that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, front-loaded with purpose and scoping. No redundant information; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description still explains pagination format and max limit. Parameters are fully covered in schema and description. For a list tool with 0 required params and 7 optional, it is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds examples for eventType, explains cursor format, and clarifies scope. This provides meaningful additional context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists the audit log, scoped to account, and specifies the REST endpoint. No sibling tool duplicates this functionality, so it's well-distinguished.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions admin role requirement and that viewer/member get 403, giving clear context. Does not explicitly state when not to use or provide alternatives, but usage is well-understood.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_eval_criteriaA
Return the list of LLM-as-judge evaluation criteria. Includes the 5 global defaults (helpfulness / accuracy / relevance / safety / conciseness) plus the custom criteria created in your account. Each criterion has id / name / rubric (the instruction text for the judge) / scaleMin / scaleMax. Use before running an eval to see which axes are available. The Free plan can read all criteria (creating custom ones is Pro+ only, but existing rows stay visible after downgrade).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses that Free plan can read all criteria but creating custom ones is Pro+, and existing rows remain visible after downgrade. No annotations exist, so description fully carries the burden. For a read-only listing tool with no parameters, this is highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no fluff. Front-loaded with main purpose, then details and usage guidance. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, the description is complete: explains output fields, global vs custom, plan limitations, and usage context. Nothing missing for this simple listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% but there are no parameters. The description adds meaning by detailing the output fields (id, name, rubric, scaleMin, scaleMax) and the distinction between global and custom criteria. This compensates for the lack of parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns the list of LLM-as-judge evaluation criteria, including 5 global defaults and custom criteria, with specific fields (id, name, rubric, scaleMin, scaleMax). This distinguishes it from sibling tools like create_eval_criterion, get_eval_criterion, and others.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage guidance: 'Use before running an eval to see which axes are available.' Also notes plan limitations (Free vs Pro+), helping set expectations. No explicit exclusion of alternative tools, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_eval_datasetsA
List your account's golden datasets (GET /v1/eval-datasets). Each dataset has a name / description / item count / frozen state. A golden dataset is a fixed test set with expected outputs — the population run_eval_dataset pushes through a target model to measure regression A/B.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full behavioral burden. It indicates this is a read-only GET operation and describes the response fields. While it does not mention auth or rate limits, the simplicity of the operation makes this adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: first gives purpose and endpoint, second details return fields and context. No extraneous content; every sentence serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description sufficiently explains the return value structure and the semantic meaning of golden datasets. It is complete for a parameterless list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and schema coverage is 100%. The description adds value by explaining the returned data (name, description, item count, frozen state) and the concept of a golden dataset, so it exceeds the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists golden datasets using a GET endpoint, specifies the returned fields (name, description, item count, frozen state), and provides context explaining what a golden dataset is. This differentiates it from sibling tools like list_eval_runs or list_eval_criteria.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly defines when to use this tool (to view all golden datasets) and hints at related actions (run_eval_dataset). However, it lacks explicit when-not or alternative usage instructions, which would elevate it to a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_eval_runsA
List the eval baseline runner's run history. Scoped to your account, most recent first. Includes summary.scoredCount / failedCount / meanScoreByCriterion, so an AI agent can grasp recent eval result summaries and per-criterion score trends in one call. Free users can read past runs too.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Number of results (1-50, default 20) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It discloses ordering, output fields (scoredCount, failedCount, meanScoreByCriterion), scope to account, and free user access.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with purpose, then ordering and output fields, then access note. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Completes the picture for a listing tool: tells what output includes, ordering, scope. No missing critical info given single parameter and no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear description for the limit parameter. Tool description adds nothing beyond schema, so baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists eval run history, scoped to account, most recent first. It distinguishes from siblings like list_eval_datasets and compare_eval_runs by explicitly mentioning the summary fields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use: to get recent eval summaries and trends. However, it does not explicitly exclude alternatives or mention when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_membersA
List a Team account's members (GET /v1/memberships, removed members excluded). Read-only tool returning each member's email / role (admin/member/viewer) / status / joined-at. Invitations, role changes, and removals are privilege operations and intentionally not exposed over MCP (use the dashboard, or a future approval-gate flow).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: read-only, HTTP method and path, return fields (email, role, status, joined-at), and exclusion of removed members. No hidden side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences: first states purpose and endpoint, second details return fields, third clarifies excluded operations. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Reasonable completeness given no parameters and no output schema: describes return fields and scope. Lacks mention of pagination or limits, but acceptable for a simple list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so baseline 4 applies. Description adds value by explaining tool scope and return data, though no parameter details are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists a Team account's members, specifies the API endpoint, and excludes removed members. Differentiates from siblings by focusing on members and mentioning exclusive operations, though not explicitly contrasting with other list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the tool is read-only and that invitations, role changes, and removals are not exposed via MCP, directing users to the dashboard. This provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_projectsA
List your account's active projects (GET /v1/projects, archived excluded). Supports per-environment observation such as dev / staging / prod. Pro allows 5 projects / Team unlimited; Free has the default project only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full burden. It reveals the HTTP method (GET), that archived projects are excluded, and plan restrictions, making behavior fairly transparent for a read-only list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that front-load the core purpose and add additional context without redundancy. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks output format details (e.g., response fields). While adequate for a simple list, providing expected return fields would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema, so baseline is 4. The description adds relevant context about per-environment filtering and plan limitations, which is useful beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists active projects, excluding archived, and specifies the API endpoint. It distinguishes from other project tools like create_project and delete_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context for per-environment observation and plan limitations, but does not explicitly state when not to use alternatives like list_prompts. Overall clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_prompt_deploymentsA
List the current deployment states (GET /v1/prompts/deployments, available on Free). Each row = { promptName, label, currentVersion, canRollback, deployedAt }. Narrow by name / label (omit both for everything).
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Narrow by prompt name (optional) | |
| label | No | Narrow by label (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It reveals the endpoint and row structure but fails to state it is a read-only operation, lacks safety guarantees, and does not mention rate limits or authorization needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two precise sentences covering purpose, endpoint, row structure, and filtering. No extraneous information, making it highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although lacking output schema, the description details the fields in each row. It omits pagination, sorting, or limits, but for a likely small deployment list, this may be sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions already present. The description reinforces the filtering intent without adding new semantic value, thus meeting the baseline for this dimension.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List the current deployment states') and specifies the endpoint and the structure of each row. It distinguishes this tool from siblings like deploy_prompt or rollback_prompt by its listing nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to filter using name/label, including the note to omit both for all entries. However, it does not explicitly mention when not to use this tool or suggest alternatives like get_deployed_prompt for a single deployment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_promptsA
List the prompt templates the user has registered. Each prompt includes id / name / version / template / variables / labels / description / createdAt. Filter by a label such as "production" (?label=xxx), or fetch all versions of one name (?name=xxx). Up to 200 entries; sort = name ASC + created_at DESC. The main path for an AI agent to read and use prompts the user registered in the dashboard.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Name filter (fetches all versions of that name). Exact match. | |
| label | No | Label filter (e.g. 'production' / 'staging' / 'experiment'). Exact match. | |
| limit | No | Number of prompts to return (1-200, default 200) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description carries the full burden. Describes listing behavior, returned fields, filtering, limit, and sort order. Implicitly read-only (listing templates), but does not explicitly state no side effects or permission requirements. Still, the behavior is well-covered for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph with 3 well-structured sentences. Front-loaded with purpose, then filtering details, then limit/sort info. Every sentence adds necessary information; no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with 3 optional params, no output schema, and no annotations, the description covers all essential aspects: purpose, filters, limit, sort order, returned fields, and its role in the dashboard. No critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all 3 parameters documented). Description adds value by explaining how to use filters (e.g., label example 'production'), that name fetches all versions, and adds context about sort order and limit. This goes beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool lists prompt templates, enumerates included fields (id/name/version/template/variables/labels/description/createdAt), and notes it is the main path for an AI agent to read prompts, distinguishing it from sibling tools like get_prompt or create_prompt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear usage context: filtering by label or name, with example values and exact match semantics. Also specifies limit (up to 200) and sort order (name ASC + created_at DESC). Does not explicitly exclude scenarios or mention alternatives, but gives sufficient guidance for typical use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_proposalsA
Return the unresolved improvement proposals found by the Argosvix guardian (quality drift / reliability anomalies / cost switching / safety / silencing noisy alerts). Approving, dismissing, and executing happen in the dashboard inbox (agents can only read and converse).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It discloses read-only access and scope, but does not detail auth needs, side effects, or output format. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose. Every sentence adds value, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no parameters and no output schema, the description explains what it returns and what it cannot do. It is sufficient, though missing details on sorting or pagination.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema coverage is 100%. The description adds no parameter info because none exist, which is appropriate. Baseline for zero params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns unresolved improvement proposals from the Argosvix guardian, listing specific categories. It uses a specific verb and resource, distinguishing it from siblings like reply_proposal.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states that approvals, dismissals, and executions are handled in the dashboard inbox, and agents can only read and converse. This sets clear boundaries, but does not name alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_safety_assessmentsA
List the assessments written by the safety classifier (OpenAI Moderation). source includes 'cron' (periodic batch) / 'mcp' (classify_calls_batch on-demand) / 'human_override' / 'api' / 'auto'. With callId = all classifier assessments for that call; without callId = account-wide, flagged first then most recent. In environments without OPENAI_API_KEY provisioned the cron does not run and this returns an empty array (classify_calls_batch also returns 503). Precondition: safety classification is disabled by default (founder-scoped / off-by-default); no assessments are generated until it is enabled. An empty array means "not enabled / nothing flagged", not a failure. AI agents use this to review recently flagged calls or to check policy-violation candidates for a specific call.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Number of results (1-200, default 50) | |
| callId | No | Target call id (llm_calls.id, [A-Za-z0-9_-]{1,128}). Omit for the whole account. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description fully bears the burden. Discloses source types, behavior with/without callId, case of missing API key, precondition, and empty array semantics. Clearly read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Relatively verbose but each sentence adds unique information. Front-loaded with core action. Could be slightly trimmed but effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, so description should explain return structure. Mentions ordering and empty array meaning, but does not describe fields of each assessment (e.g., content, severity). Gap in completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but description adds meaning: ordering when no callId (flagged first then most recent), clarifies callId targets llm_calls.id, and provides context like default/max already in schema. Adds value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it lists safety classifier assessments (OpenAI Moderation), specifies filtering by callId or account-wide, and distinguishes from sibling tool classify_calls_batch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains when to use with/without callId, precondition that safety classification is disabled by default, meaning of empty array, and use case for AI agents. Does not explicitly list when not to use, but contrasts with classify_calls_batch.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_saved_viewsA
List the saved views (GET /v1/saved-views). A saved view is a named combination of frequently used /calls page filters (startDate/endDate/provider/model/limit). Enables phrasing like "show calls with my usual last-week OpenAI filter". Per account, max 20.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description discloses the max limit of 20 per account and indicates it's a list (read) operation. Could be more explicit about read-only behavior, but sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise two sentences: first sentence states the action, second explains the concept and constraint. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list tool with no output schema, the description provides enough context: what it returns, the max limit, and the concept. Could include expected return format, but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist; schema coverage is 100%. Description adds context about saved views, which adds value beyond the empty schema. Baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'List' and the resource 'saved views'. Explains what a saved view is with an example, and differentiates from siblings like create_saved_view and delete_saved_view.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implicitly suggests use when wanting to see existing saved views, but lacks explicit when-to-use vs alternatives or when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_webhooksA
List the registered outbound event webhooks (GET /v1/webhooks). Each webhook includes id / url / hasSecret / enabled / eventTypes / lastStatus / consecutiveFailures etc. (the secret itself is never returned). This is the subscription surface that notifies external endpoints of account events (approval requests, proposal execution / reversal) via signed POSTs. Readable on Free.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full burden. It discloses that the secret is never returned and that webhooks use signed POSTs. It does not mention pagination or rate limits, but for a parameterless read operation, the transparency is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two sentences) and front-loaded with the main action. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, the description fully explains the tool's purpose, return fields, and the broader context of webhooks. It is complete for an agent to decide when to call it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema coverage is 100%. The description adds context about the return fields, which is useful but not required for a no-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists registered webhooks, specifies the HTTP method and path, and enumerates returned fields. It distinguishes itself from sibling tools like create_webhook, delete_webhook, and update_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use the tool (to list webhooks) and notes availability on Free tier. No explicit when-not-to-use or alternatives are provided, but it's clear from context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
propose_alert_rulesA
Analyze the call patterns over the past lookbackDays (7-30, default 14) and propose recommended alert rules for cost / latency / error_rate / anomaly as JSON. Applying them is a separate step via create_alert after customer confirmation (propose only — zero side effects). Only rules that do not overlap existing alerts are proposed (existing types are fetched via list_alerts). Returns { lookbackDays, baseline: {meanDailyCost (USD), p95Latency (ms), errorRate (percent 0-100), dailyCalls, totalCalls}, proposals: [{ name, alertType, thresholdValue, windowMinutes, reasoning }], skipped: [{ alertType, reason }] }. The thresholdValue of an error_rate proposal is also a percent (consistent with backend create_alert). What big-vendor dashboards show in UI, done MCP-first in one prompt.
| Name | Required | Description | Default |
|---|---|---|---|
| lookbackDays | No | Lookback days for computing the baseline (7-30, default 14) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes zero side effects, data source (call patterns over lookbackDays), and output structure. No annotations, so description carries burden; it covers key behavioral traits adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Relatively concise for the information provided. Front-loaded with main action and distinction from create_alert. Each sentence adds value, though could be slightly trimmed without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description fully explains the return object including fields like baseline, proposals, skipped. Parameter is fully described. No missing context for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single parameter with 100% schema coverage. Description adds range, default, and context: 'analyze the call patterns over the past lookbackDays (7-30, default 14).' Also explains how lookbackDays appears in output.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it analyzes call patterns and proposes alert rules for cost/latency/error_rate/anomaly. Distinguishes from sibling tools like create_alert and list_alerts by emphasizing the propose-only nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly separates proposal from creation: 'Applying them is a separate step via create_alert after customer confirmation.' Also notes that only non-overlapping rules are proposed, referencing list_alerts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
propose_eval_criteriaA
Have an LLM judge (gpt-4o-mini) propose eval criterion candidates from a one-line useCaseHint (e.g. "customer support bot") and optional sampleCallIds (representative calls from your account, up to 5) (POST /v1/eval-criteria/propose). An AI agent can finish "propose criteria to measure our prompt quality" in one prompt. Pro+ only (the backend enforces the plan gate + budget gate); nothing is INSERTed (propose only — adoption is a separate step via create_eval_criterion, structurally limiting LLM-hallucination impact). Decrypt failures for sampleCallIds are reported in partialFailures (the LLM call still runs without samples). Privacy note for prompt samples: with sampleCallIds, the backend decrypts those calls' prompt/response and sends excerpts (1500 chars each) to OpenAI gpt-4o-mini. This re-sends data your SDK originally sent to OpenAI/Anthropic, so no new vendor is added, but be aware it may reach an OpenAI model different from your own LLM calls. If that is a concern, run with useCaseHint only. Results are advisory: the returned criteria are LLM proposals and may include semantically weak rubrics (overuse of "helpful", duplicates) even when structurally valid. User review before adoption is recommended; do not feed them blindly into create_eval_criterion. Returns { criteria: [{ name (snake_case 32 chars), rubric (1-200), scaleMin (=1), scaleMax (5 or 10), reasoning (1-200) }], partialFailures: string[], budgetSpentUsd, proposedRawCount (raw count returned by the LLM), droppedCount (entries removed by the validator) }. Audit: emits an eval.propose_criteria event to the audit log.
| Name | Required | Description | Default |
|---|---|---|---|
| maxCriteria | No | Max number of criteria to return (1-10, default 5) | |
| useCaseHint | Yes | 1-2 line description of the intended use case (e.g. "customer support bot for e-commerce returns + refunds"; 1-500 chars, required) | |
| sampleCallIds | No | Array of call_ids from your account passed as context (optional, up to 5, [A-Za-z0-9_-]{1,128}). Grounds the LLM's proposals in your own data |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description fully covers behavior: nothing is inserted, decrypts samples, sends excerpts to OpenAI, results are advisory, includes return structure and audit event.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is detailed but front-loaded with key information. Each sentence adds value; however, it could be slightly more concise without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description thoroughly explains return structure, including partialFailures and budgetSpentUsd. Covers edge cases like decrypt failures. Complete for a 3-param tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds value by explaining the purpose of each parameter and behavioral aspects (e.g., sampleCallIds grounds proposals).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool proposes eval criterion candidates using an LLM judge from a useCaseHint and optional sampleCallIds. It distinguishes from sibling create_eval_criterion by noting adoption is a separate step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use ('propose criteria to measure our prompt quality'), mentions prerequisites (Pro+ only), provides alternative for privacy (useCaseHint only), and recommends user review before adoption.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
purge_expired_plaintextA
Bulk-purge your account's plaintext records older than olderThanDays (POST /v1/tier2/plaintext/purge-expired). Consistent with the Terms of Service v2.1 "retainable up to 90 days" (automatic retention); an AI agent can finish "auto-purge plaintext older than 30 days" in one prompt. dryRun=true (the safe default to reach for) returns the count plus 5 sample call_ids; dryRun=false performs the actual UPDATE. Emit-then-UPDATE ordering plus a deterministic idempotencyId (sha1(endpoint+accountId+olderThanDays+cutoff_date)) gives webhook-retry-equivalent semantics. Pro+ plan only (Free gets 403). An actual purge (dryRun=false) requires approvalId — obtain human approval via request_approval (action: 'purge_expired_plaintext') first, because NULLing plaintext is irreversible. Only your own account is purged. Returns (dryRun=true) { dryRun: true, targetCount, cutoffTimestamp, olderThanDays, sampleTargetCallIds }; (dryRun=false) { dryRun: false, purgedCount, cutoffTimestamp, olderThanDays, purgedAt }. Audit: emits tier2.purge_expired_plaintext.
| Name | Required | Description | Default |
|---|---|---|---|
| dryRun | No | true = preview only (zero mutation); false = actual UPDATE. Default false (MCP discipline recommends passing dryRun explicitly) | |
| approvalId | No | Approval id granted via request_approval (apr_ + 32 hex; create with action 'purge_expired_plaintext'). When passed, the server verifies approved + within expiry + action match + unconsumed, and atomically consumes it on the actual purge (1 approval = 1 execution). dryRun only verifies without consuming | |
| olderThanDays | No | Age threshold in days for purging (1-365, default 30; consistent with the Terms of Service v2.1) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description fully discloses behavioral traits: irreversibility (NULLing plaintext), need for human approval via request_approval, idempotency semantics with hash, dryRun behavior, and audit event emission. This is comprehensive given no annotations are provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose and then provides supporting detail. It is slightly verbose with technical details (e.g., hashing algorithm) but remains efficient overall without unnecessary repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 parameters, no output schema, destructive action) and 45 sibling tools, the description covers all essential aspects: purpose, parameters, behavioral traits, prerequisites, error conditions, return values, and audit. It leaves no major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema description coverage, the description adds significant value: it explains the safe default for dryRun, the pattern and creation of approvalId, the default and range for olderThanDays, and the interplay between parameters. This goes beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool bulk-purges plaintext records older than a specified number of days, with the specific verb 'purge' and resource 'plaintext records'. It distinguishes from sibling tools by focusing on plaintext retention policies and mentioning account-level operation, which is distinct from other deletion tools like bulk_delete_calls or delete_alert.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance, including the example of auto-purge older than 30 days, and when-not-to-use for Free plan users (403). It also recommends using dryRun=true for preview. However, it does not explicitly contrast with other deletion tools, though the unique approval requirement implicitly differentiates.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
query_callsA
Retrieve recent LLM call records captured by Argosvix. Filterable by provider / model / time range / tag. Defaults to the last 24 hours, 100 records.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Number of records to return (1-500, default 100) | |
| model | No | Model name to filter by (substring match). Omit for all models | |
| beforeId | No | Keyset pagination cursor (id of the last row on the previous page). Must be used together with beforeTimestamp | |
| provider | No | Provider to filter by (openai / anthropic / gemini / mistral). Omit for all providers | |
| latencyMax | No | Upper bound on response latency (ms, >= 0). Combine with latencyMin for a range | |
| latencyMin | No | Lower bound on response latency (ms, >= 0). For outlier drill-down (e.g. "only calls over 2 seconds") | |
| rangePreset | No | Time range preset. Default 24h | 24h |
| beforeTimestamp | No | Keyset pagination cursor (timestamp of the last row on the previous page, ISO-8601). Must be used together with beforeId. Descending timestamp order only |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It correctly labels the operation as 'Retrieve' (read-only) and mentions filtering capabilities and default parameters. However, it omits details like pagination behavior (though present in schema) and potential limits, but overall it provides sufficient transparency for a read tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence but includes an undefined reference to 'tag' (not in input schema), which wastes space. It could be more structured (e.g., separating default behavior from filters). The front-loaded information is adequate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters and no output schema, the description is moderately complete. It covers the core purpose and defaults but does not mention pagination, ordering, or error conditions. The schema fills in many gaps, but the description could be more self-contained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds no semantic value beyond what the schema already provides for each parameter. It mentions 'tag' (not in schema), which is slightly misleading, but the defaults are reiterated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves recent LLM call records. It specifies the resource ('call records captured by Argosvix'), the action ('Retrieve'), and key filtering dimensions (provider/model/time range/tag). It also distinguishes from sibling tools like aggregate_calls or export_calls by focusing on retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving recent call records with filters, but it lacks explicit guidance on when to use this tool versus siblings (e.g., export_calls for bulk export, aggregate_calls for statistics). No when-not-to-use or alternative recommendations are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
raise_llm_budgetA
Raise or lower the monthly LLM feature budget (Pro+ only). Range $5 - $500 (hard cap against runaway spend), in $0.01 increments. Existing spend carries over; auto-resets at month boundaries. Example phrasing: "we hit 80% of the budget — raise it to $30 just for this month" / "we overspent — lower next month to $10". A new value below current spend is accepted (remaining simply becomes 0; counting restarts from 0 next month).
| Name | Required | Description | Default |
|---|---|---|---|
| budgetUsd | Yes | New monthly budget in USD (5-500, $0.01 increments). E.g. 30 / 50.5 / 100 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations exist, the description carries full burden. It discloses the range ($5-$500), increments, carry-over of existing spend, monthly resets, and behavior when setting below current spend. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: purpose first, then details, then examples. Every sentence adds value with no redundancy. Slightly longer but appropriate for the content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter tool with no output schema, the description covers purpose, constraints, behavioral nuances, and usage examples. It is sufficient for an AI agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a description for budgetUsd. The description adds context like 'hard cap against runaway spend', carry-over behavior, and concrete examples, which enrich understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Raise or lower') and the resource ('monthly LLM feature budget'). It distinguishes from siblings like 'get_llm_budget' by specifying it modifies the budget, and the 'Pro+ only' constraint adds specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context with examples ('we hit 80% of the budget') and mentions the 'Pro+ only' eligibility. It lacks explicit when-not-to-use but the examples guide appropriate scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rename_projectA
Update an existing project's name / slug (PATCH /v1/projects/:id). Specify either or both. slug keeps the URL-safe constraint (/^[a-z][a-z0-9-]{0,31}$/). Renaming the default project is allowed.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | New display name (unchanged when omitted) | |
| slug | No | New slug (unchanged when omitted, /^[a-z][a-z0-9-]{0,31}$/) | |
| projectId | Yes | Target project id (the UUID obtained from list_projects) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must carry the full burden. It discloses that it is a PATCH endpoint (partial update) and mentions the URL-safe constraint for slug. However, it does not describe permissions, side effects, conflict behavior, or return value, leaving gaps for an AI agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is exceptionally concise with three short sentences. It front-loads the core action, then covers additional details. No fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple rename tool, the description covers core functionality. However, with no output schema, it omits the return value and error conditions. It also does not address idempotency or concurrency, which are relevant for update tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds value by clarifying optionality ('Specify either or both') and the permissibility of renaming the default project, but these are minor enhancements beyond the schema's own descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('Update an existing project's name / slug') and specifies the resource (project). It distinguishes from siblings like create_project and delete_project by focusing on renaming, and includes a specific note about default projects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description provides clear guidance on parameter usage ('Specify either or both') and mentions a special case ('Renaming the default project is allowed'). However, it lacks explicit when-not-to-use instructions or alternatives beyond the sibling context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rename_promptA
Change an existing prompt's name + version (Pro+ only, POST /v1/prompts/:id/rename). Main use is typo fixes ('customer_supprt' to 'customer_support'). Collision with an existing (name, version) in the account = 409. Since update_prompt never changes name/version by contract, rename is a separate tool for semantic separation.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | New name ([A-Za-z0-9][A-Za-z0-9_-]{0,63}) | |
| version | Yes | New version ([A-Za-z0-9][A-Za-z0-9._-]{0,63}) | |
| promptId | Yes | Target prompt id (list_prompts.prompts[].id) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that it's Pro+ only, uses POST, changes both name and version, and can cause 409 collision. However, it does not describe the response format or any side effects (e.g., on deployments). Still, key behavioral traits are covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, with each sentence adding value: purpose, use case, and differentiation. It is well-structured and front-loaded but could be slightly more concise (e.g., merging the first two sentences).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, the description covers licensing, collision behavior, and distinguishes from related tools. It could mention what the API returns (e.g., the renamed prompt) but is otherwise sufficient for a straightforward rename operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all three parameters. The description adds context about typo fixes and collision but does not add parameter-specific semantics beyond what the schema already provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Change' and resource 'existing prompt's name + version', and distinguishes from update_prompt by noting that update_prompt never changes name/version. The example and explicit reference to POST /v1/prompts/:id/rename provide specific purpose clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Main use is typo fixes' and contrasts with update_prompt, explaining that rename is a separate tool for semantic separation. It also mentions collision with existing (name, version) results in 409 error, providing clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
reply_proposalA
Post a question about a proposal and get the AI's reply (same as the inbox conversation). Explanation only — nothing is executed. Get proposalId from list_proposals.
| Name | Required | Description | Default |
|---|---|---|---|
| body | Yes | Question about the proposal (e.g. why did it degrade? should we fix it?) | |
| proposalId | Yes | Target proposal ID (from list_proposals) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the tool is non-executing ('nothing is executed'), which is key. However, it omits other details like authentication needs, rate limits, or side effects, leaving some transparency gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two sentences and a brief instruction. No redundant words, front-loaded with the main action and key differentiator ('explanation only'). Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description is vague about the return value ('get the AI's reply'). Mentioning 'same as the inbox conversation' helps but doesn't specify format or structure. For a simple query tool, it is adequate but could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds value by providing an example for 'body' (e.g., 'why did it degrade?') and a source hint for 'proposalId' ('from list_proposals'), improving agent understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action ('post a question about a proposal') and the resource ('proposal'), with a specific outcome ('get the AI's reply'). The 'same as the inbox conversation' and 'explanation only' differentiate it from sibling tools like get_proposal_thread and propose_eval_criteria, though not fully explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a prerequisite ('Get proposalId from list_proposals') and a behavioral note ('nothing is executed'), but lacks explicit guidance on when to use or avoid this tool versus alternatives (e.g., get_proposal_thread for just viewing).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
request_approvalA
Create an approval request in the human approval gate (runtime control plane Phase 3; Pro+ only). Call it before dangerous operations (deletion / money transfer / account closure etc.); the account owner gets an email notification and a human approves or denies via the dashboard or the email link. Important: no MCP tool exists to approve or deny (an AI agent cannot self-approve its own request). Poll the result with get_approval. Expiry after timeoutSeconds (default 3600) counts as denied. Server-side consumption: passing approvalId to a dangerous mutation tool (bulk_delete_calls / purge_expired_plaintext / retry_failed_webhook / auto_silence_noisy_alert / extend_customer_trial / apply_promo_code_to_customer) makes the backend verify action match + approved + within expiry + unconsumed, and consume it on execution (1 approval = 1 execution). In that case create the request with an action exactly matching the target tool name. Example phrasing: "deleting user usr_123 is a dangerous operation — get human approval first"
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | Operation identifier (1-128 chars; alphanumerics, ._:-, and spaces). E.g. delete_user | |
| summary | Yes | One-line human-readable description (1-500 chars; appears verbatim in the approval email) | |
| metadata | No | Supplementary JSON object (up to 4KB, optional) | |
| timeoutSeconds | No | Approval deadline in seconds (60-86400, default 3600) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses creation of request, human approval via email/dashboard, timeout expiry counted as denied, server-side consumption with specific mutation tools, action matching, and one-time use. Fully transparent about side effects and constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with purpose, usage example, key notes, and consumption details. Every sentence is informative, though slightly lengthy. Front-loaded with core action and context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with 4 params, nested objects, and no output schema, the description is remarkably complete. Covers workflow, expiration, consumption, action matching, and dependencies. Leaves no critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, but description adds significant value: explains action should match target tool name for automatic consumption, summary appears verbatim in email, timeoutSeconds default 3600, and metadata usage. This extra context goes beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create an approval request in the human approval gate' with context (Phase 3, Pro+ only) and examples. Distinguishes from siblings by mentioning polling with get_approval and listing dangerous operations that require it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Call it before dangerous operations' and lists examples. Warns that no MCP tool exists for self-approval and instructs to poll with get_approval. Provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
retry_failed_webhookA
Mark failed Stripe webhook events (the billing_dead_letter table) for reprocessing in the audit log (POST /v1/tier2/webhook-events/retry). Finishes "retry all the Stripe webhooks that failed transiently last week" in one prompt. Select targets by eventIds (specific events, up to 100) or fromTimestamp/toTimestamp (range, 7-day cap). dryRun=true previews the list; dryRun=false records a 'marked_for_manual_redispatch' entry per event in the audit log (the founder performs the actual retry via wrangler / the Stripe dashboard; fully automatic re-dispatch is a later phase). Emits use a deterministic idempotencyId (sha1(endpoint+accountId+eventId)); duplicate runs with the same args are silently skipped. Founder-operations only (an internal billing-webhook recovery tool; general accounts get 403). billing_dead_letter is an internal cross-account table and actual re-dispatch stays manual, so there is no plan to open this up. Returns (dryRun=true) { dryRun: true, targetCount, events: [{eventId, eventType, reason, receivedAt}] }; (dryRun=false) { dryRun: false, targetCount, succeeded: string[], failed: [{eventId, reason}], skipped: string[], narrative, retriedAt }. Audit: emits tier2.retry_failed_webhook per event.
| Name | Required | Description | Default |
|---|---|---|---|
| dryRun | No | true = preview only; false = actually emit the markers. Default false | |
| eventIds | No | Array of Stripe event ids to retry (evt_xxx format, up to 100). Can be combined with fromTimestamp | |
| approvalId | No | Approval id granted via request_approval (apr_ + 32 hex; create with action 'retry_failed_webhook'). Server-side verification + atomic consumption on actual execution (1 approval = 1 execution). dryRun only verifies | |
| maxRetries | No | Per-request cap (1-100, default 10) | |
| toTimestamp | No | Range end (ISO-8601, optional) | |
| fromTimestamp | No | Range start (ISO-8601; more than 7 days ago is rejected with 400) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses multiple behavioral traits: it only marks events for retry (actual re-dispatch is manual), uses deterministic idempotency for duplicate detection, requires approval for actual execution, returns specific shapes, emits audit events, and has access restrictions. This is comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with the purpose in the first sentence, followed by selection criteria, behavior, and return shapes. It is somewhat verbose but every sentence adds necessary information. Could be more concise, but it is not overly long for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema and six parameters, the description provides complete context: input semantics, behavioral nuances (dry run, duplicates, access control), explicit return shapes for both dryRun true/false, audit details, and idempotency. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% parameter description coverage, but the description adds value by explaining how parameters interact (e.g., combining eventIds with fromTimestamp, 7-day cap on range, dryRun behavior, approvalId usage, maxRetries cap). It goes beyond the schema's individual descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool marks failed Stripe webhook events for reprocessing, with specific verb 'retry' and resource 'failed Stripe webhook events'. It distinguishes itself from sibling tools like test_webhook by noting it is an internal founder-operations-only recovery tool, and focuses on a narrow use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states when to use the tool: to retry failed Stripe webhooks, even providing a common prompt phrase. It mentions access restrictions (403 for general accounts) and internal nature, but does not explicitly list alternative tools or scenarios where it should not be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rollback_promptA
Revert the prompt deployed to a label to the previous version (Pro+ only, POST /v1/prompts/deployments/rollback). 409 when there is no previous version (first deployment only), and 409 when the previous version has already been deleted. After reverting, another rollback toggles back (current / previous swap).
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Target prompt name | |
| label | Yes | Environment label |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: toggling between versions on repeated rollback, 409 errors for first deployment or deleted previous version, and Pro+ requirement. No annotation provided, so description carries full burden and does well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences covering purpose, error conditions, and toggle behavior. No redundant information. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers behavior, errors, and toggle effect. Missing any example or mention of return value (no output schema). Fairly complete for a rollback tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema provides both parameter names and descriptions with 100% coverage. Description adds no additional meaning beyond schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states tool reverts a deployed prompt to previous version, specifying verb (revert), resource (prompt deployed to label), and adds detail about versioning and toggle behavior. Distinguishes from siblings like deploy_prompt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implicitly clear when to use (to undo a deployment), but lacks explicit guidance on when not to use or comparison to alternatives. No exclusions or alternatives mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_evalA
Start a new eval run immediately (POST /v1/eval-runs). Scores the most recent N calls against the 5 default criteria (plus up to 8 custom criteria) using gpt-4o-mini. Pro+ only (Free gets 403); environments without OPENAI_API_KEY provisioned return 500 from the backend. Precondition: only calls with plaintext storage (the content-storage opt-in) ON are scored. With the opt-in OFF (the default) there are zero candidates and the run returns summary.scoredCount=0 with reason='no_plaintext_calls' (gating, not a failure). Cost: about $0.01 per run (20 calls x 5 criteria = 100 LLM calls); around 30 runs/month = $0.30 at founder-dogfood scale.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Free-form run name (1-100 chars, e.g. 'weekly-prod-eval-2026-06-02') | |
| label | No | Label filter (substring match within tags). Omit = all calls. | |
| recentCount | No | Number of calls to evaluate (1-20, default 10). The most recent N calls are passed to the judge. | |
| idempotencyKey | No | Opaque key for retry dedup (UUID recommended, 64 char cap). Re-POSTing the same key within 60 minutes returns the existing run. | |
| promptRegistryId | No | Target prompt registry id (list_prompts.prompts[].id). Omit = ad-hoc run. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries full burden. Discloses mutating nature (POST), cost (~$0.01/run), scoring algorithm, and return behavior (summary with scoredCount and reason). Lacks mention of idempotency handling or full response structure, but adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph, dense but efficient. Front-loaded with the main action and endpoint. Could benefit from bullet points for preconditions, but still clear and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 5 parameters, no output schema, and important preconditions, the description covers almost everything: failure modes, costing, preconditions, and partial return structure. Missing full response shape but sufficient for agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good descriptions. The tool description adds context linking parameters to behavior (e.g., recentCount defines how many calls, label as substring filter, idempotencyKey for dedup). Adds value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb (Start a new eval run immediately), resource (eval-runs via POST), and what it does (scores recent calls against default and custom criteria). Distinct from siblings like compare_eval_runs or run_eval_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit conditions: Pro+ only, needs OPENAI_API_KEY, requires plaintext storage opt-in. Gives failure modes (403, 500, zero scored). Does not explicitly contrast with run_eval_dataset, but context is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_eval_datasetA
Run a golden dataset against a target model and produce a regression verdict (POST /v1/eval-datasets/:id/run, Pro+ only). Feeds each item's inputText to targetModel, has gpt-4o-mini score the outputs against the default criteria plus expectedOutput, and records eval_scores (regression A/B). Results can be compared across runs with compare_eval_runs. Run records are excluded from production cost / analytics / alert aggregation. Cost: item count x criteria count LLM calls. 503 in environments without OPENAI_API_KEY provisioned.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetId | Yes | Id of the dataset to run (list_eval_datasets.datasets[].id) | |
| judgeModel | No | Judge model (omit = gpt-4o-mini). Only OpenAI models present in the pricing table. | |
| targetModel | Yes | Model to measure regressions for (only OpenAI models present in the pricing table, e.g. 'gpt-4o-mini'). Unknown models get 400. | |
| idempotencyKey | No | Opaque key for retry dedup (UUID recommended, 200 char cap). Re-POSTing the same key returns the existing run (double-billing prevention). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description fully discloses behavioral traits: feeds inputText to targetModel, uses gpt-4o-mini for scoring against criteria, records eval_scores, and explains idempotencyKey for dedup. It also notes cost implications and environment dependencies.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (about 100 words) and well-structured, with front-loaded main action. Every sentence adds unique information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description covers essential aspects: what the tool does, how it works, cost model, idempotency, error conditions, and comparison with sibling tools. It is complete for a mutation tool with 4 parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by explaining default for judgeModel, restrictions for targetModel (OpenAI models only), and the purpose of idempotencyKey (retry dedup and double-billing prevention).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs a golden dataset against a target model to produce a regression verdict. It specifies the HTTP method and endpoint, distinguishes from siblings like compare_eval_runs, and mentions the Pro+ plan restriction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit context: results can be compared with compare_eval_runs, run records are excluded from production cost/analytics/alert aggregation, and it warns about cost and 503 errors without API key. It doesn't explicitly state when not to use, but offers strong guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
silence_alertA
Temporarily mute an alert (stops notification delivery). Defaults to 24 hours; pass an ISO-8601 timestamp as until for a custom expiry. Pass the alertId obtained from list_alerts.
| Name | Required | Description | Default |
|---|---|---|---|
| until | No | Unmute time in ISO-8601 (e.g. 2026-06-01T00:00:00Z). Omit for 24 hours from now | |
| alertId | Yes | Target alert ID (from list_alerts) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the burden. It states the effect (stops notification delivery) and default duration. However, it does not disclose return behavior, idempotency, or reversibility (though unsilence_alert exists). Adequate but could be more explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: purpose, parameter details, instruction. It is concise and front-loaded with no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (2 parameters, no output schema), the description covers essential behaviors. It lacks mention of the return value but is otherwise complete for a straightforward mute operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, providing a baseline of 3. The description adds value by explaining the default for 'until' and instructing to get alertId from list_alerts, which provides context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Temporarily mute an alert (stops notification delivery)' using a specific verb and resource. It distinguishes from siblings like unsilence_alert and acknowledge_alert.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description tells when to use the tool by mentioning the default 24-hour expiry and custom until timestamp. It also instructs to pass alertId from list_alerts, implying a prerequisite. However, it does not explicitly mention when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
test_webhookA
Send one fabricated alert to the given URL as a test delivery (Pro+ only). Main use: checking that a webhook URL is reachable before registering it. SSRF defense requires https and rejects private / loopback / cloud-metadata IPs. When secret is provided, an HMAC-SHA256 signature (X-Argosvix-Signature) is attached. Rate limit = 5/min per account (60s sliding window; may be exceeded across worker instances). response.delivered = whether the receiver returned 2xx within 5s; false means an invalid URL / timeout / 5xx / network error.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Destination webhook URL (https, SSRF-guarded, 1-500 chars) | |
| secret | No | Secret for the HMAC-SHA256 signature (optional, 1-256 chars; verify X-Argosvix-Signature on the receiver side) | |
| alertName | No | Name for the fabricated alert (optional, 1-64 chars, [A-Za-z0-9 _\-.] only). Defaults to 'argosvix test alert' |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavioral traits: SSRF defense, HMAC-SHA256 signing when secret is provided, rate limit (5/min sliding window, possible exceed), and response field meaning (delivered). No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense paragraph covering all key aspects without excess. Could be slightly more structured (e.g., bullet points), but every sentence adds necessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains the key response field (delivered). All aspects are covered: purpose, constraints, behavior, rate limits, and result interpretation. Complete for a test tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions. The description adds value by explaining the purpose of the secret (signature) and the default for alertName, going beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action ('Send one fabricated alert'), the resource ('to the given URL'), and the purpose ('checking that a webhook URL is reachable before registering it'). It clearly distinguishes from sibling tools like create_webhook or update_webhook.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies the main use case and includes important prerequisites: Pro+ tier, SSRF defense requiring https and rejecting private IPs. It does not explicitly state when not to use, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
unsilence_alertA
Unmute a currently silenced alert.
| Name | Required | Description | Default |
|---|---|---|---|
| alertId | Yes | Target alert ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It states the tool reverses a silenced state, but does not disclose permissions, side effects, or behavior if the alert is not silenced. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence with no redundant information. It is front-loaded and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description covers the core action adequately. However, it lacks context on post-unmuting behavior (e.g., notifications, state change effects).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers the single parameter 'alertId' with a description. The description does not add additional meaning beyond the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (unmute) and resource (silenced alert). It distinguishes from siblings like 'silence_alert' and 'acknowledge_alert' by using specific verb and context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives, such as 'silence_alert' or 'acknowledge_alert'. It does not provide prerequisites (e.g., alert must be silenced) or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_alertA
Update an existing alert's settings (PATCH /v1/alerts/:id). alertType (the watched metric type) is immutable — to change it, create a new alert and then delete the old one (completing the alert lifecycle). Threshold / evaluation window / notification channels / name / enabled flag / composite conditions can be partially updated (all fields optional). Example phrasing: "lower the monthly budget alert threshold from $100 to $50" / "add Slack as a notification channel".
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Display name of the alert (1-100 chars, no line breaks). Omit to keep the current value | |
| alertId | Yes | Target alert ID (from list_alerts) | |
| enabled | No | Alert enabled flag. false pauses evaluation (unlike silence, re-enabling requires another PATCH) | |
| conditions | No | Update of the v1.5 multi-condition alert. When specified, the single-metric path is ignored in favor of AND/OR aggregation. Can overwrite either an existing single-metric or multi-condition alert. | |
| filterModel | No | Target model (substring match). Omit to keep the current value; pass an explicit null to reset to all models | |
| channelKinds | No | Notification channel kinds to enable. Omit to keep the current value. Updating together with channelTargets is recommended | |
| sleepMinutes | No | Suppression window for repeated notifications (minutes, 5-10080). Omit to keep the current value | |
| windowMinutes | No | Aggregation window (minutes, 5-43200). Omit to keep the current value | |
| channelTargets | No | Destination object for every kind listed in channelKinds. Omit to keep the current value | |
| filterProvider | No | Target provider. Omit to keep the current value; pass an explicit null to reset to all providers | |
| thresholdValue | No | Threshold (>= 0). Omit to keep the current value |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: PATCH method, partial updates (all fields optional), enabled flag behavior (pauses vs. silence), and conditions override behavior. Covers mutation safety and lifecycle.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise single paragraph (~70 words) front-loading the verb+resource, then key constraints, then examples. No wasted words, efficient structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all critical aspects: what updates, what is immutable, optionality, field behaviors, and example usage. Given 11 parameters and nested objects without output schema, this is complete and actionable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed descriptions. The description adds value by highlighting immutability of alertType, optionality of all fields, and the conditions override. While schema is already strong, the description provides context beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates an existing alert's settings, identifies the immutable alertType, and lists mutable fields. It distinguishes from sibling tools like create_alert and delete_alert by specifying the update lifecycle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: alertType is immutable, instructing to create a new alert and delete the old one. Offers example phrasing for typical use cases like lowering a threshold or adding a notification channel.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_annotationA
Partially update an annotation's annotationText / label / qualityScore (PATCH /v1/annotations/:id). callId is immutable. For fixing a label or re-scoring quality from 4 to 5, etc. Pass annotations[].id obtained from list_annotations_for_call as annotationId.
| Name | Required | Description | Default |
|---|---|---|---|
| label | No | New label (0-50 chars, alphanumerics plus _ - only). Omit to keep the current value | |
| annotationId | Yes | Target annotation id (AUTOINCREMENT integer) | |
| qualityScore | No | New quality score (1-5). Omit to keep the current value | |
| annotationText | No | New comment (0-2000 chars). Omit to keep the current value |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral context. It discloses that the update is partial (PATCH), callId is immutable, and the annotationId must come from list_annotations_for_call. This is valuable beyond the schema, though it doesn't cover idempotency or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no redundant information. It front-loads the action and specific fields, then provides examples and sourcing instructions. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, parameters, and usage context well. It is sufficient for a simple partial update tool with 4 parameters and no output schema. Missing explicit error handling or permissions, but not critical for basic usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds significant value by explaining how to source annotationId and reaffirming that omitted fields retain current values. This goes beyond the schema's parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs a partial update on an annotation's annotationText, label, or qualityScore, using the PATCH method. It distinguishes from sibling tools like create_annotation or delete_annotation by specifying 'update' and giving examples like 'fixing a label or re-scoring quality'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides usage context by instructing to obtain annotationId from list_annotations_for_call and noting that callId is immutable. It gives examples of when to use the tool (fixing label, rescoring quality). However, it does not explicitly exclude alternative tools (e.g., delete_annotation) beyond the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_budget_gateA
Update a runtime budget gate (Pro+ only). Partially updates any of monthlyLimitUsd / enforceMode / enabled. Example phrasing: "raise the limit to $100" / "disable the gate temporarily" / "switch to fail_closed"
| Name | Required | Description | Default |
|---|---|---|---|
| gateId | Yes | Target gate id (from get_budget_gate, starts with bg_) | |
| enabled | No | Whether the gate is enabled | |
| enforceMode | No | Behavior when the backend is unreachable | |
| monthlyLimitUsd | No | New monthly limit in USD (0.01 - 1000000, $0.01 increments) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It states 'Partially updates' which implies it is not destructive, but does not disclose any other behavioral traits such as auth requirements (Pro+ only is a plan restriction, not direct auth), rate limits, or idempotency. Somewhat adequate but could be more transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences plus an example list. Every sentence adds value, and the information is front-loaded. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, 1 required, no output schema), the description adequately covers what the tool does and how to use it with concrete examples. It is complete for an agent to understand and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter having a description. The tool description adds value by providing usage examples mapping to parameters (e.g., 'raise the limit to $100' to monthlyLimitUsd). However, it does not substantially enhance understanding beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the action 'Update' and the resource 'runtime budget gate'. It mentions 'Partially updates any of monthlyLimitUsd / enforceMode / enabled', which distinguishes it from sibling tools like create, delete, or get budget gates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Examples are provided ('raise the limit to $100', 'disable the gate temporarily', 'switch to fail_closed') that illustrate typical use cases. However, there is no explicit guidance on when not to use this tool or alternatives, such as when to use create_budget_gate instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_eval_criterionA
Update a custom criterion in your account with a full replace (Pro+ only, PATCH /v1/eval-criteria/:id). name + rubric + scaleMin + scaleMax are required (not a partial update — all fields are overwritten). type / config are also fully replaced (omitting them reverts to 'llm_judge' / no config). Deterministic types require config. Global defaults (account_id IS NULL) are structurally out of scope (404); other accounts' customs are 404 too. Name collision within the account = 409.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | New name (1-50 chars, starts with an alphanumeric, [A-Za-z0-9 _\-.] only) | |
| type | No | Evaluator type (defaults to 'llm_judge' when omitted). Deterministic: 'exact_match' / 'contains' / 'regex' / 'json_schema' / 'json_path' | |
| scope | No | Evaluation scope (default call). call = per call; trajectory = per trajectory (llm_judge only) | |
| config | No | Type-specific settings (not needed for llm_judge). exact_match: {expectedOutput}, contains: {substring, caseSensitive?}, regex: {pattern, flags?}, json_schema: {schema}, json_path: {path, expectedValue?}. Categorical scoring also requires config.categories (2-10 entries, worst to best). | |
| rubric | Yes | New rubric (10-2000 chars) | |
| scaleMax | Yes | New scaleMax (1-100, greater than scaleMin) | |
| scaleMin | Yes | New scaleMin (1-100, less than scaleMax) | |
| scoreType | No | Scoring type (default numeric). boolean = pass/fail; categorical requires config.categories (llm_judge only) | |
| criterionId | Yes | Target criterion id (list_eval_criteria.criteria[].id) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It discloses full replace behavior, required fields, consequences of omitting type/config (revert to defaults), and necessary config for deterministic types. It also covers error scenarios, providing good transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured paragraph that front-loads the main action. It is concise, covering key details without wordiness. While it could be broken into bullet points, it remains efficient and informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 params, nested objects, no output schema), the description is remarkably complete. It explains the full replace nature, required fields, default behaviors, and error conditions. This is sufficient for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value beyond the schema by explaining the full replace semantics, required vs optional fields, and how type/config are overwritten. It also clarifies the impact of omissions and error conditions, enhancing parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates a custom criterion with a full replace, specifying it is Pro+ only and uses PATCH /v1/eval-criteria/:id. It differentiates from siblings like create_eval_criterion and delete_eval_criterion by emphasizing the full replace behavior and the requirement of all fields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use: for full updates of custom criteria, not partial. It also gives error conditions (404 for out-of-scope criteria, 409 for name collisions). However, it does not explicitly state when not to use or compare to alternative tools, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_policy_gateA
Update a runtime policy gate (Pro+ only). Partially updates modelAllowlist (null clears the restriction) / blockPii / blockSecrets / enforceMode / enabled. Example phrasing: "add gpt-4o-mini to the allowlist" / "enable secret blocking"
| Name | Required | Description | Default |
|---|---|---|---|
| enabled | No | ||
| blockPii | No | ||
| policyId | Yes | Target policy id (from get_policy_gate, starts with pg_) | |
| enforceMode | No | ||
| blockSecrets | No | ||
| modelAllowlist | No | New allowlist (null clears the model restriction) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses 'Partially updates' and that null clears the restriction for modelAllowlist. However, it does not mention permissions, idempotency, return value, or side effects, limiting transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences plus examples, front-loaded with purpose and scope, and contains no fluff. Every part adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers fields and gives usage examples but lacks details on return value, prerequisites, or implications of settings (e.g., enforceMode). For a mutation tool with no output schema or annotations, more context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 33% schema description coverage, the description adds value by listing all modifiable fields and explaining the null behavior for modelAllowlist. It does not provide detailed semantics for other fields, but compensates for the low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and resource 'runtime policy gate', lists the modifiable fields, and provides examples. This distinguishes it from siblings like create, delete, and get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'Pro+ only' as a licensing restriction but does not explicitly state when not to use this tool or alternatives like create_policy_gate. Examples give usage context, but no exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_promptA
Partially update an existing prompt's template / variables / labels / description (Pro+ only, PATCH /v1/prompts/:id). name + version are immutable (change them via rename_prompt). promptId is required; only the fields you pass are updated. Used by AI agents for label moves (promoting 'staging' to 'production') and small patch edits.
| Name | Required | Description | Default |
|---|---|---|---|
| labels | No | New labels (full replacement, up to 8, each [A-Za-z0-9][A-Za-z0-9_-]{0,31}). | |
| promptId | Yes | Target prompt id (list_prompts.prompts[].id) | |
| template | No | New template body (non-empty, up to 50000 chars). | |
| variables | No | New variables (plain object; null clears them all). | |
| description | No | New description (1-500 chars). To explicitly clear the existing description, PATCH other fields without this one (an empty string '' is rejected by the schema — prevents an LLM hallucination from wiping the existing description). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses partial update behavior (only passed fields updated), immutability constraints, and the nuance about description field (rejecting empty string to prevent accidental wipe). No annotations exist, so description fully addresses behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two well-structured sentences, front-loaded with key information, no wasted words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given complexity (5 params, nested objects) and absence of output schema, the description covers operation, restrictions, usage scenarios, and important edge cases (description clearing). It is complete for an AI agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good parameter descriptions. The tool description adds context like 'variables (null clears them all)' and 'promptId is required', which supplements the schema meaningfully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it's for partially updating an existing prompt, lists the updatable fields (template, variables, labels, description), and distinguishes from rename_prompt. It specifies the HTTP method and Pro+ requirement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly warns that name and version are immutable and must be changed via rename_prompt. Provides usage examples like label moves and small patch edits, guiding when to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_webhookA
Partially update an outbound event webhook (Pro+ only, PATCH /v1/webhooks/:id). Only the specified fields change (url / secret / eventTypes / description / enabled). Sending secret as null removes the signature; re-enabling with enabled=true also resets the consecutive-failure counter. Other accounts' webhooks return 404. webhookId is the id from list_webhooks.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | New URL (HTTPS required) | |
| secret | No | New secret (empty / null removes it) | |
| enabled | No | Enabled flag | |
| webhookId | Yes | Target webhook id (owh_...) | |
| eventTypes | No | Array of subscribed event kinds (empty = all) | |
| description | No | Display memo |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden. It discloses partial update behavior, effect of null secret (removes signature), re-enabling resets failure counter, and 404 for other accounts' webhooks. Lacks details on idempotency or rate limits, but covers key behavioral traits well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is three sentences, front-loaded with core action and HTTP verb, and includes key behavioral notes without unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters and no output schema, description explains partial update, special behaviors, ownership, and id source. Lacks mention of response format, but overall sufficient for agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters (100% coverage), so baseline is 3. Description adds value by clarifying that only specified fields change, and provides special behaviors for 'secret' and 'enabled'. This goes beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it partially updates an outbound event webhook, specifies the HTTP method and endpoint, lists the editable fields, and distinguishes from create/delete/tool siblings. The purpose is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explains when to use (update webhook) and ownership restriction, but does not explicitly guide when NOT to use it or mention alternatives like test_webhook or retry_failed_webhook. Implicitly suggests using list_webhooks first to get the id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
87 tool updates
v0.30.0-alpha.14- First observed
acknowledge_alert - First observed
aggregate_calls - First observed
apply_promo_code_to_customer - First observed
auto_silence_noisy_alert - First observed
bulk_delete_calls - First observed
classify_calls_batch - First observed
compare_eval_runs - First observed
create_alert - First observed
create_annotation - First observed
create_budget_gate - First observed
create_eval_criterion - First observed
create_eval_dataset - First observed
create_policy_gate - First observed
create_project - First observed
create_prompt - First observed
create_saved_view - First observed
create_webhook - First observed
delete_alert - First observed
delete_annotation - First observed
delete_budget_gate - First observed
delete_eval_criterion - First observed
delete_eval_dataset - First observed
delete_policy_gate - First observed
delete_project - First observed
delete_prompt - First observed
delete_saved_view - First observed
delete_webhook - First observed
deploy_prompt - First observed
detect_anomaly - First observed
export_calls - First observed
extend_customer_trial - First observed
get_account_health - First observed
get_alert - First observed
get_annotation - First observed
get_approval - First observed
get_budget_gate - First observed
get_cost_summary - First observed
get_deployed_prompt - First observed
get_eval_criterion - First observed
get_eval_dataset - First observed
get_eval_run - First observed
get_llm_budget - First observed
get_percentiles - First observed
get_policy_gate - First observed
get_prompt - First observed
get_proposal_thread - First observed
get_safety_assessment - First observed
list_alert_events - First observed
list_alerts - First observed
list_annotations_by_label - First observed
list_annotations_for_call - First observed
list_approvals - First observed
list_audit_log - First observed
list_eval_criteria - First observed
list_eval_datasets - First observed
list_eval_runs - First observed
list_members - First observed
list_projects - First observed
list_prompt_deployments - First observed
list_prompts - First observed
list_proposals - First observed
list_safety_assessments - First observed
list_saved_views - First observed
list_webhooks - First observed
propose_alert_rules - First observed
propose_eval_criteria - First observed
purge_expired_plaintext - First observed
query_calls - First observed
raise_llm_budget - First observed
rename_project - First observed
rename_prompt - First observed
reply_proposal - First observed
request_approval - First observed
retry_failed_webhook - First observed
rollback_prompt - First observed
run_eval - First observed
run_eval_dataset - First observed
silence_alert - First observed
test_webhook - First observed
unsilence_alert - First observed
update_alert - First observed
update_annotation - First observed
update_budget_gate - First observed
update_eval_criterion - First observed
update_policy_gate - First observed
update_prompt - First observed
update_webhook
TDQS
Every tool has a clearly distinct purpose. For example, there are multiple alert-related tools (acknowledge_alert, auto_silence_noisy_alert, silence_alert, etc.) but each targets a unique action like acknowledging a single event, bulk-silencing, or muting. Descriptions are detailed enough to avoid ambiguity.
All 87 tool names follow a consistent verb_noun pattern in snake_case (e.g., create_alert, list_alerts, delete_alert). Verbs like get, list, create, update, delete, propose, etc. are used uniformly, making naming predictable.
With 87 tools, the count is very high and exceeds typical well-scoped ranges (3-15). However, the server covers a broad domain (calls, alerts, budgets, evaluations, prompts, projects, etc.) as a comprehensive admin interface, so the count is somewhat justified but still feels heavy.
The tool set covers core CRUD/lifecycle operations for each subdomain (alerts, calls, eval, prompts, webhooks, budgets, policies, projects). There are no obvious gaps; even advanced operations like automated silencing, proposal reply, and approval request are included.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
See, price, and control every tool call your AI agents make: policy checks, cost, and audit tools.
AI agent observability for production traces, natural-language insights, and improvement loops.
Data + AI observability — monitor and troubleshoot production-grade agents and the context they use.
Discover, inspect and run 63,000+ agent tools from one balance. Pay per call, no subscriptions.
1
Related MCP Servers
- AlicenseAqualityCmaintenanceConnects AI assistants to Warpmetrics telemetry data to monitor AI agent performance, execution runs, and LLM costs. It allows users to query success rates, latency, and spend metrics directly through natural language interfaces.2059MIT
- FlicenseNot gradedqualityDmaintenanceEnables AI agents to track LLM costs, enforce budgets, compare models, and estimate expenses through simple tool calls.-
- AlicenseNot gradedqualityDmaintenanceTracks and analyzes AI agent tool calls with event logging, dashboard, per-tool and per-agent analytics, and error monitoring.MIT
- FlicenseNot gradedqualityAmaintenanceProvides real-time monitoring of AI agents, context, usage limits, workflows, files, Git, tests, builds, errors, secrets, and model-economy advice for tools like Claude Code, Codex, and Cursor, with 30 MCP tools for comprehensive observability.1-
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/argosvix/mcp-server'
If you have feedback or need assistance with the MCP directory API, please join our Discord server