Skip to main content
Glama

The Portkey Admin API as an MCP server — 178 tools across prompts, configs, keys, analytics, governance, deployments, and more.

IMPORTANT

Active compatibility development. Palo Alto Networks completed its Portkey acquisition on 2026‑05‑29 and now presents Portkey as the core of Prisma AIRS AI Gateway. The Portkey Admin API remains live, and its official OpenAPI and product changelog continued adding control-plane surfaces through August 2026; this project has therefore resumed API-coverage work. It targets the Portkey-compatible API (x-portkey-api-key), not Prisma AIRS/Strata Cloud Manager directly. Prisma AIRS AI Gateway currently has a different management and authentication surface, so it is not a PORTKEY_BASE_URL swap. See the short Prisma AIRS interoperability guide for the supported side-by-side model and adapter criteria.

You need a Portkey API key with appropriate scopes. Get one from your Portkey dashboard under API Keys.

Claude Code

claude mcp add -e PORTKEY_API_KEY=your_key portkey-admin -- npx -y portkey-admin-mcp

Cursor / Windsurf / VS Code

Add to your MCP config (.cursor/mcp.json, .windsurf/mcp.json, or .vscode/mcp.json):

{
  "mcpServers": {
    "portkey-admin": {
      "command": "npx",
      "args": ["-y", "portkey-admin-mcp"],
      "env": {
        "PORTKEY_API_KEY": "your_api_key"
      }
    }
  }
}

Run directly

PORTKEY_API_KEY=your_key npx -y portkey-admin-mcp

To expose only a focused subset of tools in stdio clients, set PORTKEY_TOOL_DOMAINS:

PORTKEY_API_KEY=your_key \
PORTKEY_TOOL_DOMAINS=prompts,analytics \
npx -y portkey-admin-mcp

Scoping domains is also the biggest lever on context cost, not just access. tools/list is paginated, and the complete 178-tool catalog is roughly 400 KB once a client follows nextCursor through every page. Narrowing to the domains a client actually needs cuts that roughly proportionally.

git clone https://github.com/CodesWhat/portkey-admin-mcp.git
cd portkey-admin-mcp
npm install && npm run build

Then use this config:

{
  "mcpServers": {
    "portkey-admin": {
      "command": "node",
      "args": ["/path/to/portkey-admin-mcp/build/index.js"],
      "env": {
        "PORTKEY_API_KEY": "your_api_key"
      }
    }
  }
}

Category

Tools

Examples

Prompts

14

Create, version, render, execute, migrate, promote prompts

Prompt Partials

7

Reusable prompt fragments with versioning

Prompt Labels

5

Organize prompt versions (production, staging, dev)

Configs

6

Gateway routing, caching, retry, loadbalancing

Deployments

5

Register, inspect, update, and archive self-hosted Gateways

API Keys

6

Create, rotate, and manage scoped API keys

Secret References

5

Manage AWS, Azure, and HashiCorp external-secret references

Virtual Keys

5

Manage provider access keys

Collections

5

Group prompts by app or project

Providers

5

Manage AI provider configurations

Integrations

11

Provider integrations, model pricing, models, workspace access

MCP Integrations

10

External MCP tool integrations

MCP Servers

12

MCP server registry, capabilities, and live connections

Guardrails

11

Content safety policies, organisation defaults, workspace exclusions

Usage Limits

7

Cost and token consumption limits

Rate Limits

5

Request frequency controls

Analytics

22

Cost, latency, errors, tokens, cache, feedback, provider groups

Logging

10

Log retrieval, ingestion, export, and field restrictions

Tracing

2

Feedback creation and updates on traces

Users & Workspaces

24

User management, invites, workspace members, SCIM group mappings

Audit

1

Audit log access

178 tools total across 20 tool domains. See ENDPOINTS.md for the full list with descriptions.

Portkey's newer product language increasingly presents provider credentials as Providers, while the current Admin API still exposes both /virtual-keys and /providers. This server keeps both domains: Virtual Keys manage provider access credentials, and Providers manage workspace provider configurations.

Most tools work with a workspace-scoped service key that has Select All permissions enabled. That covers prompts, configs, virtual/API keys, providers, guardrails, workspace integrations, MCP servers, rate/usage limits, logs, prompt completions, and workspace user management.

If a tool returns a 403 with Portkey error AB03, it means missing scopes — not a broken endpoint.

Enterprise-gated tools (53)

The following tools require an organisation-level scope that is only available on Portkey Enterprise plans. They return 403 You do not have enough permissions to execute this request on workspace plans. Their descriptions include an Enterprise-gated. Returns 403 on non-Enterprise Portkey plans. suffix so MCP clients know upfront.

Area

Tools

Required scope

Analytics (22)

get_cost_analytics, get_request_analytics, get_token_analytics, get_latency_analytics, get_error_analytics, get_error_rate_analytics, get_cache_hit_latency, get_cache_hit_rate, get_cache_summary, get_users_analytics, get_error_stacks_analytics, get_error_status_codes_analytics, get_user_requests_analytics, get_rescued_requests_analytics, get_feedback_analytics, get_feedback_models_analytics, get_feedback_scores_analytics, get_feedback_weighted_analytics, get_analytics_group_users, get_analytics_group_models, get_analytics_group_metadata, get_analytics_group_providers

org-level analytics.view

Deployments (5)

list_deployments, register_deployment, get_deployment, update_deployment, archive_deployment

Enterprise deployment administration

Audit

list_audit_logs

audit_logs.list

Org-level integrations

get_integration, list_integration_models, list_integration_workspaces

organisation_integrations.read

Org-level users

list_all_users, get_user, get_user_stats, list_user_invites

organisation_users.list / organisation_users.read

Organisation guardrails (6)

get_organisation_defaults, update_organisation_defaults, list_input_guardrail_workspace_exclusions, update_input_guardrail_workspace_exclusions, list_output_guardrail_workspace_exclusions, update_output_guardrail_workspace_exclusions

organisation_settings.read/update and organisation_exclusions.list/update

Log exports (8)

create_log_export, list_log_exports, get_log_export, start_log_export, cancel_log_export, download_log_export, update_log_export, get_log_export_field_restrictions

Enterprise Logs Export + logs.export

SCIM groups (4)

list_scim_groups, list_scim_workspace_mappings, create_scim_workspace_mapping, delete_scim_workspace_mapping

SCIM enabled + organisation admin access

Other scope requirements

Feature

Required

Prompt completions (run_prompt_completion)

completions.write scope + billing metadata (app, env)

Org-level service API key creation via create_api_key

organisation_service_api_keys.create (Enterprise)

Status: The HTTP transport works locally and is covered by the integration test suite, but it is a proof of concept — there is no hosted version of this server, and hosted deployment is not currently a goal. Use stdio (npx) as the supported transport.

The server supports Streamable HTTP for remote access:

HTTP authentication controls access to the server, but it does not impersonate separate Portkey tenants. All authenticated principals use the same configured PORTKEY_API_KEY and can invoke any enabled tool and scope that credential grants. Run separate instances or deployments with separately scoped Portkey credentials and PORTKEY_TOOL_DOMAINS allowlists for different trust levels.

PORTKEY_API_KEY=your_key \
MCP_HOST=127.0.0.1 \
MCP_PORT=3000 \
MCP_PUBLIC_BASE_URL=https://mcp.example.com \
MCP_AUTH_MODE=bearer \
MCP_AUTH_TOKEN=your_secret \
node build/server.js

Or via npx (the portkey-admin-mcp package includes the HTTP binary):

PORTKEY_API_KEY=your_key MCP_AUTH_MODE=bearer MCP_AUTH_TOKEN=your_secret \
  npx -y -p portkey-admin-mcp portkey-admin-mcp-http

For local-only HTTP use, leave MCP_HOST at its default 127.0.0.1. Set MCP_HOST=0.0.0.0 only when you intentionally need to accept connections from outside the local machine, such as Docker or a reverse proxy on another interface.

Variable

Default

Description

PORTKEY_API_KEY

(required)

Your Portkey API key

PORTKEY_BASE_URL

https://api.portkey.ai/v1

Portkey Admin API base URL. Prisma AIRS/SCM URLs are not compatible; credentialed requests never auto-follow redirects

PORTKEY_ALLOW_PRIVATE_BASE_URL

—

Set to true to allow a literal loopback/private PORTKEY_BASE_URL

PORTKEY_ALLOW_INSECURE_HTTP

—

Separately set to true only when a trusted self-hosted gateway cannot use HTTPS

PORTKEY_TOOL_DOMAINS

—

Server-side allowlist of the 20 domains: users, workspaces, configs, deployments, keys, collections, prompts, analytics, guardrails, limits, audit, labels, partials, tracing, logging, providers, secret-references, integrations, mcp-integrations, mcp-servers. HTTP ?tools= may narrow it but cannot expand it

MCP_HOST

127.0.0.1

Bind address

MCP_PORT

3000

Port

MCP_PUBLIC_BASE_URL

—

Public absolute base URL to advertise from /auth/info and the status page; recommended for hosted deployments

MCP_AUTH_MODE

none

none, bearer, or clerk (none is blocked for HTTP unless explicitly overridden)

MCP_AUTH_TOKEN

—

Secret for bearer auth

CLERK_ISSUER / CLERK_AUDIENCE

—

Required issuer and audience when MCP_AUTH_MODE=clerk

CLERK_ALLOWED_SUBJECTS

—

Optional CSV subject allowlist for Clerk; at least one Clerk authorization policy is required

CLERK_ALLOWED_ORGANIZATION_IDS / CLERK_ALLOWED_ROLES

—

Optional CSV organization and role constraints; every configured constraint must match

CLERK_REQUIRED_PERMISSIONS

—

Optional CSV permissions that must all be present in the verified Clerk JWT

MCP_ALLOW_UNAUTHENTICATED_HTTP

—

Set to true only for intentional local unauthenticated HTTP debugging

MCP_SESSION_MODE

stateful

stateful or stateless

MCP_MAX_SESSIONS

100

Maximum concurrent stateful sessions or active stateless request handlers

MCP_EVENT_STORE

off

off, memory, or redis; stateless GET /mcp replay requires memory or redis

MCP_EVENT_TTL_SECONDS

300

Replay retention in seconds

MCP_EVENT_STORE_MAX_EVENTS

10000

Maximum events retained by the in-memory replay store; oldest events are evicted first

MCP_EVENT_STORE_MAX_BYTES

67108864

Approximate maximum serialized bytes retained by the in-memory replay store

MCP_EVENT_STORE_COMMAND_TIMEOUT_MS

5000

Redis command timeout for the event store, in milliseconds; 0 disables the timeout (restores unbounded pre-v6 behavior)

MCP_REDIS_URL

—

Redis URL for shared event store; production requires rediss:// and ACL-scoped credentials

MCP_EVENT_ENCRYPTION_KEY

—

Required 32-byte base64 AES key for Redis replay payloads; generate with openssl rand -base64 32

MCP_REDIS_KEY_PREFIX

mcp:event-store

Dedicated Redis namespace for replay data

MCP_TLS_KEY_PATH

—

TLS key for native HTTPS

MCP_TLS_CERT_PATH

—

TLS cert for native HTTPS

ALLOWED_ORIGINS

—

CORS allow-list; also used to validate the Host header (DNS-rebinding protection) when MCP_AUTH_MODE=none

MCP_TRUST_PROXY

loopback

Express trust-proxy policy. Use an exact nonnegative hop count or trusted proxy subnet; true is rejected because it trusts forwarding headers from every peer

RATE_LIMIT_STORE

memory

redis for multi-instance/serverless deployments; production memory mode requires RATE_LIMIT_SINGLE_PROCESS=true

RATE_LIMIT_REDIS_URL

—

Shared limiter Redis URL, falling back to MCP_REDIS_URL/REDIS_URL; production requires rediss://

RATE_LIMIT_REDIS_KEY_PREFIX

mcp:rate-limit

Redis namespace for atomic pre-authentication IP and principal-plus-IP token buckets

RATE_LIMIT_MAX_BUCKETS

10000

Maximum local buckets in explicit memory mode before new clients share overflow capacity

Production containers must choose their rate-limit topology explicitly: set RATE_LIMIT_STORE=redis for multi-instance deployments, or set RATE_LIMIT_SINGLE_PROCESS=true only for a single long-lived process.

Vercel support is kept as a reference proof of concept — we do not run a hosted deployment. See docs/VERCEL_DEPLOYMENT.md if you want to self-deploy.

Key points:

  • Uses stateless request handling with encrypted, principal-bound Redis replay and a shared atomic Redis rate limiter

  • Requires Clerk or bearer auth

  • Leave MCP_TLS_* unset (Vercel terminates HTTPS)

  • Set MCP_PUBLIC_BASE_URL to your deployment URL so advertised MCP endpoints never depend on request headers

  • Vercel does not support WebSockets — Streamable HTTP/SSE only

docker build -t portkey-admin-mcp .
docker run \
  -e PORTKEY_API_KEY=your_key \
  -e MCP_TRANSPORT=http \
  -e MCP_HOST=0.0.0.0 \
  -e MCP_PORT=3000 \
  -e MCP_AUTH_MODE=bearer \
  -e MCP_AUTH_TOKEN=your_secret \
  -p 3000:3000 \
  portkey-admin-mcp

Health Endpoints

Path

Purpose

GET /health

Server liveness

GET /ready

Readiness (includes optional Portkey connectivity check)

GET /auth/info

Auth configuration metadata

npm run dev           # stdio with hot reload
npm run dev:http      # HTTP with hot reload
npm test              # unit + contract tests
npm run test:coverage # unit + contract tests with the enforced 80% line floor
npm run test:e2e      # MCP protocol tests
npm run test:http     # HTTP endpoint smoke test
npm run smoke         # credentialed read-only Portkey API smoke suite
npm run ci            # full pipeline (lint + typecheck + coverage + build + e2e + verify)

The live smoke suite reports expected credential-scope denials and the explicitly tracked hosted control-plane route gaps as skips. Unexpected HTTP responses, network errors, and response-contract failures still fail the run.

The required CI and release gates measure every TypeScript source file and fail below 80% line coverage. The current full report is 98.19% lines, 91.92% branches, and 98.58% functions.

npm run dev:http now requires MCP_AUTH_MODE=bearer or MCP_AUTH_MODE=clerk by default. For deliberate local-only unauthenticated testing, set MCP_ALLOW_UNAUTHENTICATED_HTTP=true.

Contributions use pull requests and the checks documented in CONTRIBUTING.md. Project decisions and maintainer roles are documented in GOVERNANCE.md. Report vulnerabilities through SECURITY.md, and see SECURITY-ASSURANCE.md for the public threat model and assurance case.

Questions and bug reports belong in Issues; broader discussion, ideas, and help using the server belong in Discussions.


Built With

TypeScript MCP SDK Zod 4 Biome Node 24 Anthropic

SemVer Conventional Commits Keep a Changelog

MIT License Ā· Inspired by r-huijts/portkey-admin-mcp-server

Also listed on LobeHub and Glama:

LobeHub MCP

Back to top

Available Tools

171 tools
add_workspace_memberA

Add an existing org user to a workspace with a role. Requires a UUID user_id; use list_all_users to find it, and invite_user first if the person is not yet in the org.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesRole in the workspace
user_idYesThe user ID to add (must be a valid UUID from list_all_users, not an email address)
workspace_idYesThe workspace ID to add the member to

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate a mutation tool (readOnlyHint=false). Description adds precondition that user must exist in org, but does not disclose potential error cases like duplicate membership or permission requirements. Good but could be more thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: purpose and usage guidance. No redundant information, well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given moderate complexity and presence of output schema, description covers key preconditions and usage steps. Could mention error handling for duplicate members, but overall sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers all parameters with descriptions. Description adds value by clarifying user_id must be UUID (not email) and referencing list_all_users, which goes beyond schema details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Add an existing org user to a workspace with a role,' specifying verb, resource, and constraints. It distinguishes from sibling tools like invite_user and list_all_users by referencing them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly provides prerequisites: requires UUID user_id, use list_all_users to find it, and invite_user if not in org. This gives clear when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cancel_log_exportA
Destructive

Cancel a pending or running log export job, unlike start_log_export which queues one or delete_integration which removes the source. This permanently stops that export, takes effect immediately, and does not roll back already-processed rows; call create_log_export and start_log_export again to retry. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export to cancel

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description discloses key behaviors: the export is 'permanently stops', takes effect 'immediately', and 'does not roll back already-processed rows'. It also states the 403 response on non-Enterprise plans, which is valuable context not in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the purpose. Each sentence adds distinct value: what it does, behavioral consequences/retry, and access limitations. No fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the destructive nature, the description covers the essential context: what it cancels, permanence, no rollback, how to retry, and access restrictions. With an output schema present, return values need not be explained. This is complete for a cancellation mutation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% coverage with a description for export_id ('The unique ID of the log export to cancel'). The tool description adds no additional parameter semantics, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource: 'Cancel a pending or running log export job.' It also distinguishes from siblings by explicitly naming start_log_export and delete_integration, making the tool's unique role clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states when to use this tool versus alternatives: 'unlike start_log_export which queues one or delete_integration which removes the source.' It also provides the retry workflow (create_log_export and start_log_export again) and enterprise access expectations, offering clear guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_api_keyA

Create a Portkey API key for auth. Org keys grant broader access; workspace keys are scoped. WARNING: The key secret is returned ONCE in the tool result and will be visible in MCP transcripts and LLM context — store it securely immediately. Using the key grants access immediately according to its scopes, defaults, and limits. Workspace keys require workspace_id and user keys require user_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the API key
typeYesKey type: 'organisation' for org-wide access or 'workspace' for workspace-scoped
scopesYesPermission scopes for the key (e.g., ['logs.read', 'analytics.read'])
user_idNoUser ID (required for user sub-type keys)
sub_typeYesSub-type: 'user' for user-associated keys or 'service' for service accounts
expires_atNoExpiration date in ISO 8601 format
descriptionNoOptional description for the key
alert_emailsNoEmail addresses for alerts
credit_limitNoCredit limit for usage
workspace_idNoWorkspace ID (required for workspace-type keys)
rate_limit_rpmNoRate limit in requests per minute
alert_thresholdNoAlert threshold percentage (0-100)
default_metadataNoDefault metadata key-value pairs
default_config_idNoDefault configuration ID to use with this key

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds critical behavioral details beyond annotations: warns that the key secret is returned only once and visible in transcripts, and states that the key grants immediate access. No contradiction with annotations (readOnlyHint=false, etc.).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences with clear structure: purpose, type distinction, security warning, activation behavior, and requirements. Front-loaded with most important info.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, security, activation, and key requirements. With output schema present and 14 parameters, the description is adequate but could mention optional fields like rate limits or defaults.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with clear descriptions for each parameter. The description reinforces key constraints (workspace_id required for workspace type, user_id for user sub-type) and scopes, adding marginal value over schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Explicitly states 'Create a Portkey API key for auth' and distinguishes between org and workspace keys. Clearly differentiates from sibling tools like delete_api_key and update_api_key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides context on when to use each key type (org vs workspace) and prerequisites (workspace_id for workspace keys, user_id for user sub-type). However, does not contrast with alternatives like create_virtual_key.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_collectionA

Create a new prompt collection for organizing prompts by app. Use this when you need a new namespace before create_prompt; returns the collection id and slug, and does not move any prompts.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesCollection name (e.g., 'hourlink', 'apizone', 'research-pilot')
workspace_idNoWorkspace ID to create collection in

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint=false, destructiveHint=false) indicate write operation that is not destructive. Description adds that it returns collection id and slug and does not move prompts, providing useful behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Description is concise, one sentence with two clauses. Front-loaded with main purpose. Could be slightly more structured but is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given simple tool with output schema, description covers key aspects: purpose, usage context, return values. No missing critical details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so schema already describes both parameters. Description adds marginal value: it relates 'name' to namespace concept, but doesn't add semantic detail beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb 'Create' and resource 'prompt collection' with explicit purpose 'organizing prompts by app'. Distinguishes from sibling create_prompt by noting it's a prerequisite.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states 'Use this when you need a new namespace before create_prompt', providing clear context. Does not explicitly state when not to use, but context implies it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_configA

Create a config that defines routing, cache, retry, and targets for requests; use update_config to modify an existing one and list_config_versions for history. At least one setting is required, new configs become active immediately once referenced by a key or prompt, and the call returns the new id and version_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName for the new configuration
targetsNoArray of target providers with virtual keys
cache_modeNoCache mode: 'simple' or 'semantic'
workspace_idNoWorkspace ID to create config in
cache_max_ageNoCache max age in seconds
strategy_modeNoRouting strategy: 'loadbalance' or 'fallback'
retry_attemptsNoNumber of retry attempts (1-5)
retry_on_status_codesNoHTTP status codes to retry on (e.g., [429, 500, 502, 503])

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses important behavioral traits beyond annotations: 'At least one setting is required, new configs become active immediately once referenced by a key or prompt, and the call returns the new id and version_id.' This adds activation semantics and return value details, giving the agent clear expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, and every clause earns its place. The structure is efficient with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 8-parameter schema with full descriptions and an output schema (so return values are already documented), the description covers all necessary contextual aspects: what it does, when to use it, key behavioral rules, and activation implications. Nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds a non-obvious constraint ('At least one setting is required') not enforced by the schema (which only requires name), and mentions return fields (id, version_id), providing extra semantic context beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Create a config that defines routing, cache, retry, and targets for requests.' It uses a specific verb (create) and resource (config), and distinguishes itself from siblings by referencing update_config and list_config_versions as alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides usage guidance: 'use update_config to modify an existing one and list_config_versions for history.' This tells the agent when to use create_config versus related tools, and also implies it is for new configs only.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_feedbackA

Create feedback for a trace or request. Writes a new feedback record linked by trace_id, returns the created feedback IDs and status, and takes effect immediately; use update_feedback when correcting an existing record.

ParametersJSON Schema
NameRequiredDescriptionDefault
valueYesFeedback value/rating. Common patterns: 1 for positive (thumbs up), 0 for negative (thumbs down), or use a scale like 1-5.
weightNoOptional weighting factor for the feedback. Use to give more importance to certain feedback.
metadataNoOptional custom metadata for categorization and analysis (e.g., feedback_source, category, user_segment).
trace_idYesThe trace ID to associate the feedback with. This links feedback to a specific request/generation.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate write operation (readOnlyHint false) and non-destructive (destructiveHint false). The description adds behavioral context: 'takes effect immediately' and 'returns the created feedback IDs and status', which go beyond basic annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that communicates the core action, return value, and alternative. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the moderate complexity (4 parameters, one nested object) and presence of an output schema, the description covers purpose, usage, and behavioral impact. It could mention that weight/metadata are optional, but schema already does so. Almost complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (all parameters have descriptions). The description does not add new semantic meaning beyond the schema; it only reinforces the trace_id link already documented. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('create feedback'), resource ('for a trace or request'), and the linking mechanism ('by trace_id'). It explicitly distinguishes itself from the sibling tool 'update_feedback' by mentioning its use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'use update_feedback when correcting an existing record', which is a clear when-to-use vs when-not-to-use instruction. No other usage guidelines needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_guardrailA

Create a guardrail with checks and actions for request filtering. Create it first, then reference it from configs; the new version becomes the policy anchor for downstream use.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the guardrail
checksYesArray of checks to apply (at least one required)
actionsYesActions to take when guardrail checks pass or fail
workspace_idNoWorkspace ID to create the guardrail in
organisation_idNoOrganisation ID (required if workspace_id not provided)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=false (write operation) and destructiveHint=false. The description adds behavioral context: 'the new version becomes the policy anchor for downstream use', implying versioning and dependency. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core purpose. Every sentence adds value: first defines the tool, second explains the usage pattern. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the essential aspects: creation, contents (checks and actions), and lifecycle hint. Given the tool's complexity (nested objects, output schema), it provides sufficient context for an agent to use it appropriately. Could mention required fields but schema already handles that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add extra meaning beyond what the schema already provides for parameters like 'checks' and 'actions'. It focuses on purpose rather than parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a guardrail with checks and actions for request filtering', providing a specific verb and resource. It distinguishes from 'update_guardrail' and 'delete_guardrail' among siblings. The phrase 'policy anchor for downstream use' adds further context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises 'Create it first, then reference it from configs', giving a clear usage pattern. It does not mention alternatives or when not to use, but the context is sufficient for an agent to infer the appropriate scenario.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_integrationA

Create an AI-provider integration that becomes the source for workspace providers. ai_provider_id identifies the backend; provider-specific fields configure Azure, Bedrock, Vertex, or custom hosts. For workspace-scoped integrations, create_default_provider controls automatic provider creation. key is write-only, but secret_mappings can resolve it or configuration fields from Secret References at runtime. pricing_adjustments apply negotiated discounts or markups to cost accounting. Use update_integration_models and update_integration_workspaces after creation; returns the new integration id and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyNoAPI key for the provider (if required)
nameYesHuman-readable name for the integration
slugNoURL-friendly identifier (auto-generated from name if not provided)
aws_regionNoAWS region (for AWS Bedrock)
api_versionNoAPI version (for Azure OpenAI)
custom_hostNoCustom base URL for the provider
descriptionNoOptional description of the integration
workspace_idNoWorkspace ID for workspace-scoped integrations
resource_nameNoResource name (for Azure OpenAI)
vertex_regionNoGCP region (for Vertex AI)
ai_provider_idYesID of the AI provider (e.g., 'openai', 'anthropic', 'azure-openai', 'aws-bedrock', 'vertex-ai')
deployment_nameNoDeployment name (for Azure OpenAI)
secret_mappingsNoUnique runtime Secret Reference mappings; mapping key allows the key field to be omitted
aws_access_key_idNoAWS access key ID (for AWS Bedrock)
vertex_project_idNoGCP project ID (for Vertex AI)
pricing_adjustmentsNoNegotiated discount or markup multipliers for cost accounting
aws_secret_access_keyNoAWS secret access key (for AWS Bedrock)
default_provider_slugNoCustom slug for the automatically created workspace provider
create_default_providerNoCreate a provider automatically for a workspace-scoped integration; defaults to true

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds significant behavioral context beyond annotations: key is write-only, secret_mappings can resolve keys or configuration fields at runtime, and pricing_adjustments affect cost accounting. It also discloses that the integration becomes the source for workspace providers and returns the new integration id and slug. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise yet comprehensive, with each sentence serving a distinct purpose: purpose, key field explanations, behavior controls, security notes, and follow-up steps. It is front-loaded with the core action and avoids redundancy, making it effective for a tool with 19 parameters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex creation tool, the description covers essential aspects: purpose, key parameters, conditional behavior, follow-up actions, and return value. The output schema further defines the return. It lacks explicit prerequisites (e.g., existence of ai_provider_id) but overall provides sufficient context for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

While the schema already covers all 19 parameters (100% coverage), the description adds valuable semantic grouping: ai_provider_id identifies the backend, provider-specific fields configure cloud providers, and create_default_provider controls automatic provider creation. It also explains the relationship between key and secret_mappings, going beyond individual parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's primary function: 'Create an AI-provider integration that becomes the source for workspace providers.' It distinguishes this from sibling tools like create_provider and create_mcp_integration by specifying the integration's role. The mention of provider-specific fields for Azure, Bedrock, Vertex, and custom hosts further clarifies its scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance on when to use the tool, including context for workspace-scoped integrations via create_default_provider. It also directs users to use update_integration_models and update_integration_workspaces after creation. However, it does not explicitly state when not to use this tool or mention alternatives like create_provider for different scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_log_exportA

Create a log export definition with filters and requested fields. This only sets up the export and does not start processing; call start_log_export next, then use get_log_export or download_log_export to inspect or retrieve the finished result. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
ai_modelNoFilter by specific AI model names
cost_maxNoMaximum cost filter
cost_minNoMinimum cost filter
time_maxYesMaximum time filter in date format (e.g., '2024-01-31' or ISO 8601)
time_minYesMinimum time filter in date format (e.g., '2024-01-01' or ISO 8601)
descriptionNoHuman-readable description for the export job
workspace_idNoWorkspace ID for the export
total_units_maxNoMaximum total units (tokens) filter
total_units_minNoMinimum total units (tokens) filter
requested_fieldsYesFields to include in export: id, trace_id, created_at, request, response, is_success, ai_org, ai_model, req_units, res_units, total_units, request_url, cost, cost_currency, response_time, response_status_code, mode, config, prompt_slug, metadata

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses a key behavioral trait—that creation does not start processing—and the expected follow-up workflow. Also notes the 403 error on non-Enterprise plans. While annotations indicate readOnlyHint=false, the description adds meaningful context about the asynchronous workflow and access restriction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each with a distinct purpose: action, workflow steps, and access caveat. Information is front-loaded and no redundant wording appears.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderately complex tool with 10 parameters and an output schema, the description covers the critical workflow (create→start→retrieve) and enterprise restriction. Return values are already documented by the output schema, so no additional explanation is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides 100% coverage with detailed descriptions for all 10 parameters, including enums for requested_fields. The description only adds generic 'filters and requested fields', so it does not need to elaborate further. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a log export definition with filters and requested fields' with a specific verb and resource. It further distinguishes the tool from siblings by explicitly noting that it 'only sets up the export and does not start processing', preventing confusion with start_log_export or download_log_export.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit workflow guidance: 'call start_log_export next, then use get_log_export or download_log_export'. Also mentions Enterprise-gating and 403 behavior, which helps decide when to use this tool versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_mcp_integrationA

Create a Portkey integration for an external MCP server URL. For headers auth, provide custom_headers or a Secret Reference mapping targeting configurations.custom_headers; secret_mappings resolve protected values at runtime without storing them in the tool call. Organisation admin keys normally need workspace_id. After creation, create_mcp_server and configure capabilities/access; returns the integration id and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesURL endpoint of the MCP server to integrate
nameYesDisplay name for the MCP integration
slugNoCustom slug. Auto-generated if omitted
auth_typeYesAuthentication type: 'none', 'headers' (custom headers), or 'oauth_auto' (OAuth)
transportYesMCP transport protocol: 'http' (streamable HTTP) or 'sse' (server-sent events)
descriptionNoDescription of the MCP integration
workspace_idNoWorkspace ID — required when using organization admin API keys
custom_headersNoCustom headers for authentication (e.g. { "Authorization": "Bearer xxx" }). Sent via configurations.custom_headers
secret_mappingsNoRuntime Secret Reference mappings; every configurations.<field> target must be unique

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate this is a non-read, non-idempotent, non-destructive operation, and the description aligns with these hints. It adds meaningful context beyond the annotations: secret_mappings resolve protected values at runtime without storing them in the tool call, and workspace_id is needed for org admin keys. It also clarifies that this tool alone does not complete the MCP server setup, implying a multi-step workflow.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, front-loads the core purpose, and every sentence carries distinct information: the action, auth handling, prerequisite, and next steps. There is no redundancy or filler, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a creation tool with 9 parameters, nested objects (secret_mappings), and an output schema, the description covers the main workflow, auth security, and the necessary follow-up (create_mcp_server). It does not explain every enum value or optional parameter, but the schema covers those. The description is sufficiently complete for an agent to understand the tool's role and key constraints.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, so the baseline is 3. The description adds extra meaning by explaining the relationship between custom_headers and secret_mappings (targeting configurations.custom_headers) and highlighting that secret_mappings avoid storing secrets in the call. It also specifies when workspace_id is required. This goes beyond mere schema enumeration, warranting a 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Create a Portkey integration for an external MCP server URL', which clearly specifies the action and target resource. It also distinguishes itself from the sibling create_mcp_server by stating that after integration creation, one must call create_mcp_server to configure capabilities and access, making it unambiguous which tool does what.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly instructs to use create_mcp_server after creating the integration, which sets the boundary between the two tools. It also notes that organization admin keys normally require workspace_id, providing a prerequisite. However, it does not explicitly enumerate cases where this tool should not be used or contrast it with other integration tools, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_mcp_serverA

Create an MCP server under an existing integration. Registers the server and returns the new id and slug; use list_mcp_integrations first to find the parent integration, then capabilities or access tools to configure it.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the MCP server
slugNoCustom slug. Auto-generated if omitted
descriptionNoDescription of the MCP server
mcp_integration_idYesID or slug of the MCP integration this server belongs to

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is a write operation (readOnlyHint=false) and non-destructive (destructiveHint=false). The description adds that it registers the server and returns id/slug, but does not elaborate on potential side effects or additional behavior beyond creation. With minimal annotation context, the description provides some added value but not extensive behavioral detail.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, consisting of two focused sentences. The first sentence states the core purpose, and the second provides contextual workflow advice. While efficient, the second sentence could be slightly more compact, but no unnecessary information is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema (stated in context), the description appropriately omits detailed return value documentation. It mentions the returned id and slug. The description also advises on a necessary prerequisite step. For a creation tool with good parameter documentation, this is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all 4 parameters. The description mentions the required parameters (name, mcp_integration_id) and notes slug and description are optional, but does not add new constraints or semantics beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool creates an MCP server under an existing integration. It specifies the resource ('MCP server') and action ('Create'), and distinguishes from sibling tools by mentioning the prerequisite of using list_mcp_integrations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: use list_mcp_integrations first to find the parent integration, then use this tool to create the server, and later use capabilities or access tools for configuration. This clearly indicates when to use this tool versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_promptA

Create a new prompt template and initial version. Use this for first-time setup; use migrate_prompt for idempotent CI/CD flows. Accepts plain text or structured chat messages, creates a new version immediately, and returns the prompt id, slug, and version id. For multi-message chat prompts pass messages (preferred) or a JSON-encoded array as string.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the prompt
modelNoModel identifier (e.g., 'gpt-4', 'claude-3-opus'). Required unless ai_model_id or finetune_id is provided
toolsNoTool definitions for tool use
stringNoLegacy prompt template string. Use plain text for single-message prompts, or a JSON-encoded messages array string for multi-message chat prompts.
dry_runNoWhen true, validate without creating
messagesNoStructured chat template alias. Serialized to the legacy string format before creation.
functionsNoFunction definitions for function calling
parametersYesDefault values for template variables
ai_model_idNoAI model ID (alternative to model). Required unless model or finetune_id is provided
finetune_idNoFine-tune ID (alternative to model). Required unless model or ai_model_id is provided
tool_choiceNoTool choice strategy
virtual_keyYesVirtual key slug for model access
collection_idYesCollection ID to organize the prompt in (use list_collections to find)
template_metadataNoCustom metadata (app, env, source_file, etc.)
version_descriptionNoDescription for this prompt version

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Description adds behavioral context beyond annotations: it creates a version immediately and returns ids. Annotations already indicate non-readonly, non-destructive, non-idempotent, so no contradiction. Some specifics like 'creates a new version immediately' are helpful but not exhaustive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences are compact and front-loaded with purpose, then usage guidance, then parameter details. Every sentence adds unique value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (15 parameters, nested objects), the description covers core workflow, tool selection, parameter guidance, and return values. Output schema exists, so return details are sufficient. No gaps for an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Despite 100% schema coverage, description adds valuable meaning: distinguishes between 'string' (legacy) and 'messages' (structured), recommends 'messages' for multi-message prompts, and explains that messages are serialized. This aids correct parameter selection beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Create a new prompt template and initial version' and distinguishes from sibling 'migrate_prompt' for CI/CD flows. The verb 'create' with resource 'prompt template' is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises use for first-time setup versus migrate_prompt for idempotent flows. Also explains when to use plain text vs structured messages, providing clear context for parameter selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_prompt_labelA

Create a prompt label for tagging prompt versions such as production, staging, or experiment. Requires either organisation_id or workspace_id to set scope, returns the new label id, and does not assign it to any versions yet.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the label
color_codeNoHex format, e.g. '#FF5733'. Optional — omit for default color.
descriptionNoDescription of the label
workspace_idNoWorkspace ID to create the label in
organisation_idNoOrganisation ID to create the label in

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate it is not read-only (write operation), not destructive, and open world. The description adds that it returns the new label id and does not assign to any versions. This provides useful behavioral context beyond annotations, but could mention if there are any side effects on existing labels or other entities.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the purpose, and every word adds value. No redundant or vague phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema exists, the description does not need to detail return structure but mentions the id. It covers scope requirement, lack of assignment, and optional color. For a creation tool with 5 parameters, this is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers all parameters with descriptions. The description adds that either organisation_id or workspace_id is required (though schema marks only name as required), which is a critical constraint. It also clarifies that color_code is optional and defaults to a default color. This adds meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a prompt label' and gives concrete examples like 'production, staging, or experiment'. It distinguishes the tool from siblings by noting that it does not assign the label to any versions yet, which sets it apart from assignment or update actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description specifies that either organisation_id or workspace_id is required to set scope, providing clear guidance on parameter usage. It also notes that the label is not assigned to versions, implying a subsequent step. However, it does not explicitly state when to use this tool versus alternatives like update_prompt_label.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_prompt_partialA

Create a reusable prompt partial for inclusion with {{> partial_name}}. Use this for shared snippets or macros; returns the partial id, slug, and version id, and the new version stays inactive until published.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the partial
stringYesThe partial content/template string
workspace_idNoWorkspace ID to create partial in (required for org-level API keys)
version_descriptionNoDescription for this version

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide basic hints (readOnlyHint=false), and the description adds behavioral context: returns specific IDs (partial id, slug, version id) and notes the version remains inactive until published. This goes beyond annotations without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. The verb and resource are front-loaded, and the key return values and state are succinctly covered.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the existence of an output schema, the description appropriately focuses on key return fields and version state. It omits details like workspace_id requirements for org-level keys, but overall it is sufficiently complete for a creation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage with descriptions for all 4 parameters. The description does not add extra meaning beyond the schema, so baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create') and the resource ('reusable prompt partial'), and explains its purpose for shared snippets/macros. It distinguishes itself from siblings like 'create_prompt' by emphasizing reusability and the inactive status until publication.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear use case ('for shared snippets or macros') and mentions that the version stays inactive until published, which guides when to use. However, it does not explicitly state when not to use this tool or mention alternatives like 'publish_partial' for activation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_providerA

Create a workspace provider backed by an org integration. The provider inherits the integration key, but its limits and expiration are enforced independently for that workspace. Returns the new provider id and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesDisplay name for the provider
noteNoOptional note or description for the provider
slugNoCustom slug for the provider. Auto-generated with random suffix if omitted
expires_atNoExpiration date in ISO 8601 format
credit_limitNoCredit limit for usage
workspace_idNoWorkspace ID - required when using organization admin API keys
integration_idYesIntegration slug for the provider (e.g., 'openai', 'anthropic', 'azure-openai')
periodic_resetNoPeriod for resetting usage limits: 'monthly' or 'weekly'. Defaults to 'monthly'.
alert_thresholdNoAlert threshold percentage (0-100)
rate_limit_unitNoMust be provided together with rate_limit_value. Values: 'rpm' (requests/min), 'rph' (requests/hour), or 'rpd' (requests/day).
rate_limit_valueNoMust be provided together with rate_limit_unit.
usage_limit_typeNoType of usage limit: 'cost' (monetary) or 'tokens' (token count). Defaults to 'cost'.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate a write operation (readOnlyHint=false) that is not destructive (destructiveHint=false). The description adds behavioral detail that the provider inherits the integration key but enforces limits and expiration independently per workspace, providing nuance beyond annotations. It could mention error conditions like duplicate names.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences efficiently cover purpose, behavioral context, and return value with no wasted words. Very concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 12 parameters and the presence of an output schema, the description adequately covers creation and the return of id and slug. It could mention the prerequisite of an existing integration, but overall it is complete for a creation tool with annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, providing a baseline of 3. The description adds context about how integration_id and workspace_id interact (inherits key but independent limits), which adds value beyond individual parameter descriptions. No redundancy.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Create' and the resource 'workspace provider backed by an org integration', distinguishing it from sibling tools like create_integration and update_provider. It also mentions the return value.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when a provider tied to an integration is needed, but does not explicitly state when to use or not use this tool versus siblings like create_integration or create_virtual_key. No exclusions or alternatives are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_rate_limitA

Create a request or token throttle with conditions, group_by, type, unit, and value. conditions and group_by are required; use usage limits when you need a cumulative budget instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoOptional name for the rate limit
typeYesWhat to rate limit: 'requests' or 'tokens'
unitYesTime unit: 'rpm' (per minute), 'rph' (per hour), or 'rpd' (per day)
valueYesThe maximum allowed value per unit (e.g., 100 rpm)
group_byYesArray of fields to group the rate limit by (e.g., ['virtual_key'], ['api_key', 'user_id'])
conditionsYesArray of conditions that determine which requests this rate limit applies to
workspace_idNoWorkspace ID to scope the limit to
organisation_idNoOrganisation ID to scope the limit to

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide basic behavioral cues (readOnlyHint false, destructiveHint false, idempotentHint false, openWorldHint true). The description adds no further behavioral context beyond what is already in annotations, but does not contradict them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with an additional sentence for alternative guidance. It is concise, front-loaded, and contains no redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (8 parameters, 5 required) and the presence of an output schema, the description is mostly complete. It could mention the possibility of duplicate creation, but the annotations and sibling tools provide sufficient context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% description coverage for all 8 parameters, so the description adds no additional meaning beyond listing some parameter names and noting that conditions and group_by are required, which is already in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool creates a request or token throttle, listing key parameters. It distinguishes itself from the sibling tool create_usage_limit, which is used for cumulative budgets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool (for rate limiting) and when not (use usage limits for cumulative budgets). Also notes that conditions and group_by are required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_scim_workspace_mappingA

Map one identity-provider SCIM group to a Portkey workspace role so current and future group members receive access automatically. Provide exactly one of scim_group_id or scim_group_name; a name can pre-create the Portkey SCIM group before the IdP provisions it. Use list_scim_groups to discover existing groups and list_workspaces for the workspace ID. This changes access provisioning and is distinct from add_workspace_member, which grants one user directly. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesWorkspace role automatically granted to group members
workspace_idYesPortkey workspace ID that the SCIM group should access
scim_group_idNoExisting identity-provider SCIM group ID; provide this or scim_group_name, but not both
scim_group_nameNoExisting SCIM group display name; provide this or scim_group_id, but not both

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate a mutating operation (readOnlyHint=false), but the description adds side-effect context: it provisions access for current and future members, can pre-create the group via scim_group_name, and may return 403 on non-Enterprise plans. This goes well beyond the structured fields and provides essential behavioral expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each adding distinctive information: core purpose, parameter constraint/pre-creation, discovery tool references, side-effect/enterprise-403 warning. No redundant filler or repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex SCIM mapping operation with an output schema present, the description covers purpose, prerequisites, parameter selection rule, side effects, and error condition. It leaves return-value documentation to the output schema, which is acceptable. No significant gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all four parameters with descriptions covering the one-of constraint. The description reinforces the exactly-one rule and adds the pre-creation behavior for scim_group_name, which is not in the schema. Given 100% schema coverage, this extra nuance elevates the score above baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+object+effect: 'Map one identity-provider SCIM group to a Portkey workspace role so current and future group members receive access automatically.' It clearly distinguishes from sibling `add_workspace_member` by stating it grants access via group rather than individually.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names prerequisite discovery tools (list_scim_groups, list_workspaces) and compares to the alternative add_workspace_member, stating when this tool is appropriate ('distinct from... grants one user directly'). Also notes Enterprise-gated plans.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_secret_referenceA

Create a reference to a secret stored in AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault. Authentication credentials are sent to Portkey and exposed to this MCP transcript; use short-lived or workload identity modes when possible. Returns the new UUID and slug, never the resolved external secret.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesHuman-readable name
slugNoOptional stable slug containing only letters, numbers, underscores, or hyphens; Portkey generates one when omitted
tagsNoOptional string key-value tags for filtering and organization
secret_keyNoOptional key within a structured external secret; null uses the whole secret value
auth_configYesChoose exactly one documented authentication shape matching manager_type. Credentials are sent to Portkey and visible in the MCP transcript; prefer workload identity or short-lived credentials when available.
descriptionNoOptional description; use null to leave it empty
secret_pathYesPath of the secret in the external secret manager
manager_typeYesExternal secret manager: aws_sm for AWS Secrets Manager, azure_kv for Azure Key Vault, or hashicorp_vault for HashiCorp Vault; must match the auth_config field family
organisation_idNoOrganisation UUID; required when the MCP server is not authenticated with a Portkey API key
allowed_workspacesNoWorkspace UUIDs or slugs allowed to use this reference; omit when allow_all_workspaces is true
allow_all_workspacesNoWhether every workspace may use this reference; defaults to true and cannot be true with allowed_workspaces

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that authentication credentials are sent to Portkey and exposed in the transcript, and clarifies that the return value is a UUID and slug, not the actual secret. This adds value beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with no filler; front-loaded with purpose and followed by important behavioral notes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (11 parameters, auth patterns) and the existence of an output schema, the description covers the essentials: creation, security implications, and return type. Could mention the relationship between manager_type and auth_config, but schema descriptions compensate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameters with descriptions; the tool description does not add new parameter-level details beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it creates a reference to external secrets from three named providers and explicitly distinguishes itself from other tools by mentioning it never returns the resolved secret.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides security guidance to prefer short-lived or workload identity modes, but does not explicitly compare against sibling tools like update_secret_reference or delete_secret_reference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_usage_limitA

Create a cumulative budget for cost or tokens with optional alerts and weekly or monthly resets. conditions and group_by are required; use rate limits when you need request throttling.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoOptional name for the usage limit
typeYesWhat to limit: 'cost' (in dollars) or 'tokens'
group_byYesArray of fields to group the usage limit by (e.g., ['virtual_key'], ['api_key', 'user_id'])
conditionsYesArray of conditions that determine which requests this usage limit applies to
credit_limitYesThe maximum allowed usage (cost in dollars or token count)
workspace_idNoWorkspace ID to scope the limit to
periodic_resetNoAutomatically reset usage counters on this schedule
alert_thresholdNoPercentage threshold (0-100) at which to send an alert
organisation_idNoOrganisation ID to scope the limit to

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds context beyond annotations: cumulative budget, optional alerts, weekly/monthly resets. Annotations already show it's a write operation (readOnlyHint false) and not destructive, so the description complements without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no wasted words. The first sentence conveys purpose and key features; the second adds usage guidance and an alternative. Info is front-loaded and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 params, output schema exists), the description covers the core purpose, required fields, and alternative. It omits explicit mention of return value or scoping parameters, but the schema handles those. Slightly better could mention that it creates a usage limit entity with an ID.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed descriptions for each parameter. The description adds little beyond stating that conditions and group_by are required, which is already in the schema. It provides no extra format or examples, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it creates a cumulative budget for cost or tokens, with optional alerts and resets. It distinguishes itself from the sibling 'create_rate_limit' by specifying that rate limits are for request throttling, not budgets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'use rate limits when you need request throttling,' providing a clear alternative. It also notes that 'conditions and group_by are required,' guiding the agent on mandatory fields. However, it could further clarify scenarios like per-user vs per-key limits.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_virtual_keyA

Store a provider API key as a virtual key. The raw key is encrypted and only returned at creation time, so save the returned slug and use it in prompts/configs. Optional usage and rate limits apply immediately, and the tool returns the new slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyYesThe actual provider API key to store
nameYesDisplay name for the virtual key
noteNoOptional note or description for this key
providerYesProvider slug (e.g., 'openai', 'anthropic', 'azure-openai', 'google')
api_versionNoAPI version (for Azure OpenAI)
credit_limitNoMaximum usage cost threshold
workspace_idNoWorkspace ID to create the key in
resource_nameNoResource name (for Azure OpenAI)
rate_limit_rpmNoRequests per minute limit
alert_thresholdNoPercentage of credit_limit at which to send alert emails (0-100)
deployment_nameNoDeployment name (for Azure OpenAI)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide safety hints, but description adds value: encryption, one-time return of raw key, immediate application of limits. These behaviors are not in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences, action first, no wasted words. Efficiently communicates key facts.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With output schema present, return value explanation is sufficient. Description covers security, slug usage, and immediate limit enforcement. Minor gap: no error handling info, but completeness is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, baseline 3. Description clarifies that optional usage/rate limits apply immediately, adding behavioral context beyond schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool stores a provider API key as a virtual key, specifying encryption, slug return, and optional limits. It distinguishes from siblings like create_api_key by focusing on virtual keys and encryption.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description advises to save the returned slug for use in prompts/configs, which is helpful but lacks explicit when-to-use vs alternatives (e.g., create_api_key). No exclusion or alternative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_workspaceA

Create a workspace to isolate resources, API keys, and team members. If slug is omitted it is auto-generated from the name; returns the new workspace id, name, and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName of the workspace
slugNoURL-friendly slug (auto-generated if not provided)
metadataNoCustom metadata key-value pairs
is_defaultNoSet as default workspace (1 = yes, 0 = no)
descriptionNoDescription of the workspace

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate non-read-only, non-destructive, non-idempotent behavior. Description adds return value info (id, name, slug) and slug auto-generation, but does not disclose additional behavioral traits beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. First sentence states purpose, second provides key behavioral detail. Information is front-loaded and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 5 parameters (1 required), nested objects, and no output schema, the description covers creation purpose, slug behavior, and return fields. Schema covers the rest. Could briefly mention metadata or default usage but not necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% description coverage, so baseline is 3. Description adds value by explaining slug auto-generation behavior and mentioning return fields, which clarifies parameter usage beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Create a workspace to isolate resources, API keys, and team members', specifying the verb and resource. Differentiates from siblings like create_api_key or create_collection by naming the workspace entity and its isolation purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains slug auto-generation behavior ('If slug is omitted it is auto-generated from the name'), which guides usage. However, does not explicitly mention when to use vs. alternatives like update_workspace or delete_workspace.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_api_keyA
Destructive

Delete an API key by UUID. This cannot be undone, revokes access immediately, and can break active sessions using the key. Returns success after revocation.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe UUID of the API key to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds behavioral details beyond annotations: states it cannot be undone, revokes immediately, can break sessions, and returns success after revocation. Annotations only provide destructiveHint=true.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, all information relevant and front-loaded. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the effect and response, but does not mention prerequisites like permissions. For a simple delete, it is still fairly complete given annotations and output schema exist.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of the single parameter with description. Description adds no extra param details beyond mentioning UUID, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states specific action 'Delete an API key by UUID', clearly distinguishing from siblings like create_api_key and update_api_key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly mentions permanence, immediate revocation, and potential impact on active sessions. Could be improved by contrasting with alternatives, but the destructive context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_collectionA
Destructive

Delete a prompt collection by ID. This cannot be undone; prompts stay in the workspace but lose their collection grouping, so reassign them first if organization matters.

ParametersJSON Schema
NameRequiredDescriptionDefault
collection_idYesCollection ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate destructiveHint=true. The description adds context about irreversibility and the effect on prompts (lose grouping), going beyond annotations. It does not contradict any annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: first states the purpose, second adds behavioral context. Every sentence adds value, and it is front-loaded. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core action and its effects. It is complete for a simple delete operation with one parameter. Could mention the output schema, but not necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a description for collection_id. The tool description does not add additional meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Delete a prompt collection by ID', specifying the verb and resource. It distinguishes itself from siblings like delete_prompt by focusing on collections and their effect on prompt grouping.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides guidance on when to use (delete collection) and warns about consequences ('cannot be undone'). It suggests reassigning prompts first if organization matters, but does not explicitly list alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_configA
Destructive

Delete a config by slug. This is permanent, removes all versions, and breaks anything still pointing at that slug; check list_config_versions first.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesConfiguration slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description adds key details: permanent deletion, removal of all versions, and breaking anything pointing to the slug. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with two concise clauses, front-loaded with the action. Every word adds value; no unnecessary text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with an output schema (exists elsewhere), the description covers the action, permanence, and a crucial precaution. It is fully adequate for an agent to use without ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides 100% coverage for the single parameter 'slug' with description 'Configuration slug to delete'. The description does not add additional parameter details, meeting the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Delete a config by slug.' It explicitly names the resource and action, and the sibling tools (create_config, update_config, get_config, list_configs) make it distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description warns of permanence and breaks dependencies, and suggests checking list_config_versions first. It provides clear when-to-use/not-use context, but lacks explicit mention of alternatives like update_config for deprecation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_guardrailA
Destructive

Delete a guardrail by id or slug. This is irreversible and removes the check from any configs that reference it, so review dependent configs first.

ParametersJSON Schema
NameRequiredDescriptionDefault
guardrail_idYesThe guardrail UUID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes beyond the annotations (destructiveHint=true) by detailing the irreversible nature and the cascading effect on configs. This adds valuable behavioral context about consequences, complementing the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences. The first sentence states the purpose, and the second provides a critical warning. No unnecessary words, earning its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple nature of a delete operation with one required parameter and an existing output schema, the description covers all necessary context: what it does, how to identify the guardrail, and the irreversible consequences. Complete for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'guardrail_id' is already well-described in the schema as 'The guardrail UUID or slug to delete'. The description does not add additional meaning beyond what the schema provides, meeting the baseline for 100% coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Delete' and the resource 'guardrail', specifying identification by 'id or slug'. It distinguishes itself from sibling tools like 'create_guardrail' or 'update_guardrail'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: it warns that the action is irreversible and removes checks from dependent configs, advising to review those first. While it doesn't explicitly list when not to use, the warning implies careful consideration. It offers clear context for usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_integrationA
Destructive

Delete an integration by slug. This is irreversible and stops the org-level connection, which will break dependent virtual keys, providers, and workspace access.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already set destructiveHint=true and readOnlyHint=false. The description adds value by specifying irreversibility and cascading effects on virtual keys, providers, and workspace access.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the primary action, and every sentence adds critical information without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter delete operation, the description covers purpose, consequences, and parameter. It is nearly complete, though the output schema is not described.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage, the description adds no extra meaning beyond the schema's explanation of the 'slug' parameter. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Delete an integration by slug,' providing a specific verb and resource. It distinguishes this tool from siblings like delete_api_key or delete_provider, which target different entities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains that deletion is irreversible and breaks dependent entities, giving clear context for when to use the tool. However, it does not explicitly mention when not to use it or suggest alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_integration_modelA
Destructive

Delete a custom model from an integration. Built-in models should be disabled instead, because deletion only applies to custom entries. Returns success after the custom model is removed.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
model_slugYesThe slug of the model to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint=true), the description explains that deletion only applies to custom entries and that success is returned after removal. This adds behavioral context not fully captured by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words, front-loaded with action and key constraint. Highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with full schema coverage and an output schema, the description covers purpose, usage, behavioral traits, and return behavior completely. Annotations and schema handle the rest.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description does not add additional meaning to the parameters beyond their names and definitions in the schema. Baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete a custom model from an integration') and distinguishes between custom and built-in models, implying that built-ins should be handled differently. This differentiates it from sibling tools like 'update_integration_models'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use (custom models) and when not to (built-in models should be disabled instead), providing clear usage guidelines. It does not name the alternative tool but implies it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_mcp_integrationA
Destructive

Delete an MCP integration and all servers beneath it. This is irreversible, removes connected access immediately, and should only be used after confirming nothing depends on the integration.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds behavioral context beyond annotations: irreversible, removes access immediately, cascades to servers. Annotations only mark destructiveHint=true; the description fills in the details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. Action and implications front-loaded, making it easy to quickly understand.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive delete operation, the description covers irreversibility, immediate impact, cascade, and a usage precaution. Output schema exists, so return values need not be described. Comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides a description for the single parameter 'id' (the integration ID or slug). The tool description does not add further parameter information, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Delete an MCP integration and all servers beneath it', using a specific verb and resource. It distinguishes from sibling delete operations like delete_mcp_server by noting the cascade effect.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: after confirming nothing depends on the integration, and warns of irreversibility and immediate access removal. No alternative tools are named, but the caution is sufficient for an irreversible delete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_mcp_serverA
Destructive

Delete an MCP server instance. This is irreversible, removes connected users' access immediately, and should be used only after confirming no workflows depend on the server.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true, but the description adds critical behavioral context: the operation is irreversible and immediately removes connected users' access. This goes beyond the annotation's binary hint. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, providing all necessary information without redundancy. The first sentence states the action, and subsequent sentences add essential warnings. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is an output schema, the description does not need to describe return values. It covers the irreversible nature, immediate user impact, and the caution about workflows. It is sufficient for the agent to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The parameter 'id' is fully described in the schema ('The MCP server ID or slug to delete'). The description does not add any additional semantic meaning beyond what the schema already provides. With 100% schema coverage, a score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete') and the resource ('MCP server instance'), making the purpose immediately clear. Among many sibling delete tools, the tool name 'delete_mcp_server' combined with the description sufficiently differentiates it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes an important usage condition: 'should be used only after confirming no workflows depend on the server.' This advises the agent on when it is appropriate to invoke the tool. However, it does not explicitly mention alternatives or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_promptA
Destructive

Delete a prompt and all its versions by id. This cannot be undone, immediately breaks callers using the slug, and should only be used after checking list_prompt_versions or confirming you do not need an audit trail.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark destructiveHint=true and readOnlyHint=false. The description adds specific behavioral context: 'This cannot be undone, immediately breaks callers using the slug,' which goes beyond annotations. However, it does not detail any rate limits or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no unnecessary words. It front-loads the core action ('Delete a prompt...'), then adds critical caveats. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive one-parameter tool with an output schema, the description covers what is deleted, irreversibility, impact on callers, and prerequisite checks. It is complete given the tool's simplicity and the presence of annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage for the single parameter 'prompt_id' with a clear description ('Prompt ID or slug to delete'). The tool description adds no additional meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Delete a prompt and all its versions by id.' It uses a specific verb ('Delete') and resource ('prompt and all its versions'), distinguishing it from sibling tools like 'delete_prompt_label' or 'delete_prompt_partial'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'should only be used after checking list_prompt_versions or confirming you do not need an audit trail.' It warns about irreversibility and impact on callers ('immediately breaks callers using the slug'), helping decide when to use this tool versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_prompt_labelA
Destructive

Delete a prompt label by ID. This cannot be undone; versions carrying the label lose it, and any workflow resolving by that label will need a replacement.

ParametersJSON Schema
NameRequiredDescriptionDefault
label_idYesLabel ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes beyond annotations by detailing the irreversible nature and downstream effects on versions and workflows. Annotations only flag destructiveHint=true, but the description explains what gets destroyed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states the action, second explains key consequences. Every word is necessary, no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has one parameter and an output schema (though not shown, it exists). The description covers the essential behavioral impact, making it complete for this simple tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage and only one parameter, the schema already describes the parameter as 'Label ID to delete.' The description does not add additional semantics, so baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Delete a prompt label by ID.' It uses a specific verb and resource, distinguishing it from sibling tools like update_prompt_label or create_prompt_label.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit consequences: 'cannot be undone', 'versions carrying the label lose it', and 'any workflow resolving by that label will need a replacement.' This helps the agent decide when to use it cautiously, though it does not explicitly name alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_prompt_partialA
Destructive

Delete a prompt partial by ID. This cannot be undone, and prompts that reference it with {{> name}} will fail to render until you replace the reference.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_partial_idYesPrompt partial ID or slug to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint=true), the description adds that deletion is irreversible and affects dependent prompts. This clarifies the side-effect and non-idempotent nature beyond the structured annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states the action, second adds the critical consequence. No unnecessary words, well front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter tool and existence of an output schema (per context signals), the description adequately covers the action and side effects. It is complete for the complexity level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter with 100% schema coverage. The schema already describes it as 'Prompt partial ID or slug to delete.' The description adds no extra details beyond 'by ID,' so baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Delete a prompt partial by ID.' It specifies the resource and identifier, distinguishing it from siblings like 'update_prompt_partial' and 'create_prompt_partial.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the consequence (permanent deletion, prompt render failures) but does not explicitly state when to use this vs. alternatives like updating or disabling. The warning about reference replacement provides practical guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_providerA
Destructive

Delete a workspace provider by slug. This is irreversible and will break prompts, configs, and virtual keys that reference it; use delete_integration for the org source instead. Returns success after the provider is removed.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the provider to delete
workspace_idNoWorkspace ID - required when using organization admin keys

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true. The description goes further by specifying that deletion is irreversible and will break prompts, configs, and virtual keys, adding valuable behavioral details beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with action, then consequences and alternative. Every word is necessary and there is no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With destructiveHint=true and clear description of consequences, the tool is well-specified. However, it lacks details about permissions or rate limits, which would make it fully complete. Output schema existence is implied but not detailed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% description coverage for both parameters (slug, workspace_id). The description does not add extra meaning beyond what the schema already provides, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Delete' and the resource 'workspace provider by slug'. It also distinguishes itself from the sibling tool 'delete_integration' for the org source, eliminating ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use (delete a workspace provider) and when not to (use delete_integration for org source). Also warns about irreversibility and impacts on other entities, providing clear context for decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_rate_limitA
Destructive

Delete a rate limit by id. This is permanent and removes throttling immediately; review dependent configs and virtual keys before deleting.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe unique identifier of the rate limit

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Description adds context beyond annotations by stating 'permanent and removes throttling immediately' and the caution about dependencies. Annotations already indicate destructive behavior, so the description adds useful nuance without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences: first states the action and permanence, second provides a critical caution. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (delete by id) and presence of an output schema, the description covers the essential purpose and behavioral implications. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'id' is fully described in the input schema (100% coverage), so the description adds no additional meaning. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the action 'Delete a rate limit by id' and emphasizes permanence and immediate effect. Distinguishes from sibling tools like create_rate_limit, get_rate_limit, update_rate_limit, and list_rate_limits.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit caution to 'review dependent configs and virtual keys before deleting', guiding the agent on prerequisites. However, it does not explicitly state when not to use or compare to alternatives like updating.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_scim_workspace_mappingA
DestructiveIdempotent

Delete a SCIM group-to-workspace mapping by mapping_id and stop future group updates from affecting that workspace. Existing provisioned members remain in the workspace and must be managed separately. Inspect list_scim_workspace_mappings first; this does not delete the identity-provider group or the Portkey workspace. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
mapping_idYesSCIM workspace mapping ID from list_scim_workspace_mappings

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint, idempotentHint), the description adds critical behavior: existing provisioned members remain, Enterprise-gating, and 403 error on non-Enterprise plans. This gives the agent important context to set expectations and handle errors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is tightly written in three sentences, each providing unique, necessary information. It is front-loaded with the primary action, followed by consequential details, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter, rich annotations, and presence of an output schema, the description covers all relevant context: purpose, side effects, prerequisites, exclusions, gating, and error behavior. It is fully self-sufficient for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds value by specifying that mapping_id comes from list_scim_workspace_mappings. This reinforces the parameter's origin, though it could include more details about required format or validation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Delete') and resource ('SCIM group-to-workspace mapping'), clearly distinguishing it from sibling tools like create_scim_workspace_mapping and list_scim_workspace_mappings. It also states the key side effect (stopping future group updates), making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly instructs to 'Inspect list_scim_workspace_mappings first', providing a clear prerequisite and source for the mapping_id. It also clarifies exclusions (does not delete the identity-provider group or the Portkey workspace), helping the agent avoid misuse.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_secret_referenceA
DestructiveIdempotent

Permanently delete a Secret Reference. This is irreversible and integrations or virtual keys using it can fail to resolve credentials immediately; confirm dependencies before deletion.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesSecret Reference UUID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true and idempotentHint=true; description adds detail about irreversible nature and immediate impact on dependent integrations/virtual keys, providing context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with action and permanence, then dependency warning. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter destructive operation with full annotations and an output schema, description comprehensively covers consequences and prerequisites.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameter with description 'Secret Reference UUID or slug'. Description adds no further parameter meaning beyond what schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Permanently delete a Secret Reference' with specific verb and resource. Distinguishes from sibling delete tools by specifying the entity type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly warns to confirm dependencies before deletion and describes consequences, but does not state when to avoid using or suggest alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_usage_limitA
Destructive

Delete a usage limit by id. This is permanent, removes the budget immediately, and clears tracked usage state; check list_usage_limit_entities first if you need impact.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe unique identifier of the usage limit

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Description adds important behavioral context beyond annotations: permanence, immediate budget removal, and clearing tracked usage state. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose, second adds critical behavioral context. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter and an output schema, the description is complete: it covers purpose, permanence, budget removal, and suggests a precheck. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with the 'id' parameter described. The description does not add extra meaning beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Delete a usage limit by id.' It uses a specific verb and resource, and distinguishes itself from other delete tools on the server.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description explains when to use (delete a usage limit permanently) and advises checking list_usage_limit_entities first for impact, implicitly stating alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_userA
Destructive

Delete a user from the org by id. This is permanent, removes org and workspace memberships, revokes API keys, and ends active sessions; use delete_user_invite for pending invites instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesThe user ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (destructiveHint=true), the description details the permanent nature of deletion and its consequences: removes memberships, revokes keys, ends sessions. This provides comprehensive behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first delivers the core purpose and effects, the second provides an alternative. It is concise and front-loaded with essential information, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter, the description covers purpose, behavioral effects, and an alternative. The output schema exists but does not require additional description. The tool is fully specified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers the single parameter 'user_id' with a description. The tool description does not add additional semantics beyond the schema. Since schema coverage is 100%, the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'delete', the resource 'user', and the method 'by id'. It differentiates from the sibling tool 'delete_user_invite' by specifying that tool is for pending invites. The scope of deletion is thoroughly explained.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool (to delete a user by ID) and provides a direct alternative for a specific case ('use delete_user_invite for pending invites instead'). This is clear guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_user_inviteA
Destructive

Delete a pending invite and revoke its invite link. This does not affect existing users; use delete_user for full user removal.

ParametersJSON Schema
NameRequiredDescriptionDefault
invite_idYesThe invite ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations include destructiveHint=true, so the safety profile is already indicated. The description adds that the invite link is revoked and that existing users are unaffected, providing extra context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no redundant information. Every phrase adds value, and the structure is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter and an existing output schema, the description covers purpose, scope, and exclusions thoroughly, leaving no ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with one parameter (invite_id) already documented. The description does not elaborate on parameter meaning beyond schema, but the schema itself is clear.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Delete a pending invite and revoke its invite link,' specifying the action and resource. Distinguishes from 'delete_user' by noting it does not affect existing users, helping the agent differentiate among sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides when to use (delete pending invite) and when not to (does not affect existing users), and directs to 'delete_user' for full removal, offering a clear alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_virtual_keyA
Destructive

Delete a virtual key by slug. This is irreversible and will break prompts and configs that reference the slug, so confirm no active dependencies first. Returns success after removal.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the virtual key to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Description adds valuable context beyond annotations (destructiveHint=true), explaining irreversible nature and secondary effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences, front-loaded with purpose, no extraneous content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Simple tool with one parameter, description covers purpose, side effects, and return value adequately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with adequate description for the single parameter; description adds no additional meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool deletes a virtual key by slug, distinguishing it from sibling delete tools for other resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly warns about irreversibility and impact on prompts/configs, advising to confirm no active dependencies first.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_workspaceA
Destructive

Delete a workspace by id. This is permanent and removes the workspace, its members, configs, API keys, and resources.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesThe workspace ID to delete

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint: true), the description provides concrete details: 'permanent and removes the workspace, its members, configs, API keys, and resources.' This fully discloses the consequences.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with two sentences: the first states the action, the second the consequences. No unnecessary words, and critical information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter and an output schema (not shown but present), the description fully covers the effect and scope. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage (workspace_id described as 'The workspace ID to delete'), the tool description adds no new meaning beyond 'by id', which is redundant. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Delete' and resource 'workspace by id', making the purpose unambiguous. It distinguishes from other delete tools by specifying 'workspace'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool should be used when permanent deletion is intended, but does not explicitly state when to use it over alternatives (e.g., no mention of archiving or deactivation). It also lacks guidance on prerequisites or caution.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

disconnect_mcp_server_connectionA
DestructiveIdempotent

Disconnect one user's active session from a Portkey-managed MCP server. This immediately ends that connection but does not revoke future access; use update_mcp_server_user_access when access itself should be removed. Provide user_id with service keys (user keys derive it), and workspace_id with organisation admin keys. Inspect list_mcp_server_connections first when the target is uncertain.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesMCP server ID or slug containing the active connection
user_idNoUser UUID to disconnect; required with service keys and derived from user keys when omitted
workspace_idNoWorkspace ID or slug; required with an organisation admin API key

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral context beyond annotations: it states the action 'immediately ends that connection but does not revoke future access,' clarifying the scope of destructiveness. It also explains key derivation for user keys, supplementing the destructiveHint and idempotentHint annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: core action, scope/alternative, and parameter guidance plus a best-practice tip. No redundancy, well front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with three parameters (one required) and an output schema, the description fully covers purpose, exclusions, auth requirements, and a pre-check suggestion. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds valuable context: 'Provide user_id with service keys (user keys derive it), and workspace_id with organisation admin keys.' This explains the conditional necessity of optional parameters, going beyond the static schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description begins with a specific verb and resource: 'Disconnect one user's active session from a Portkey-managed MCP server.' It clearly distinguishes from the sibling tool update_mcp_server_user_access, which is for removing access rather than just ending an active session.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance is given: 'use update_mcp_server_user_access when access itself should be removed' and 'Inspect list_mcp_server_connections first when the target is uncertain.' It also specifies key-type-dependent parameter usage, making it clear when to provide user_id vs workspace_id.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

download_log_exportA
Read-onlyIdempotent

Get a signed URL for downloading a completed log export. The export must already be finished; use get_log_export to confirm readiness and start_log_export if it has not run yet. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export to download

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, but the description adds context beyond these: the export must be completed, and non-Enterprise plans receive a 403. This is valuable behavioral information not present in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main purpose, and followed by necessary conditions and alternative tool guidance. Every sentence contributes value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with an output schema, the description covers prerequisites, alternative tools, error conditions (403), and access restrictions, making it fully complete for selecting and invoking the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters with a clear description for export_id. The tool description adds no additional parameter-level semantics, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Get a signed URL for downloading a completed log export.' It distinguishes from sibling tools by specifying the export must be completed and referencing related tools like get_log_export and start_log_export.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance on when to use: the export must already be finished; directs the user to get_log_export to confirm readiness and start_log_export if it has not run yet. Also notes the Enterprise-gating prerequisite and potential 403 error.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_analytics_group_metadataA
Read-onlyIdempotent

Get a paginated metadata breakdown with total_groups, group_count, and a metadata_groups array grouped by the required metadata_key. Use this for custom breakdowns like per-environment or per-feature analysis; pass metadata_key in addition to the time window. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
page_sizeNoResults per page (max 100)
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
current_pageNoPage number for pagination
metadata_keyYesThe metadata key to group by (e.g., 'env', 'app', 'client_id')
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Description adds value beyond annotations by noting pagination, the return structure, and that it's enterprise-gated (returns 403 on non-enterprise plans). Annotations already cover readOnly, idempotent, and non-destructive hints, so the description fills in operational behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the main purpose and return fields. Includes a usage tip and a critical restriction (enterprise-gated). No fluff; every sentence serves a purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the large number of parameters (32) and existence of an output schema, the description covers the key aspects: pagination, grouping, required inputs, and enterprise restriction. It could mention that the many filter parameters are for narrowing results, but the schema already handles that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the description does not need to elaborate on parameters. It mentions the required metadata_key and time window but adds no new semantic information beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool retrieves a paginated metadata breakdown grouped by a required metadata_key, listing specific return fields (total_groups, group_count, metadata_groups). It gives concrete use cases (per-environment, per-feature) and distinguishes from siblings by emphasizing the grouping by metadata_key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use this for custom breakdowns like per-environment or per-feature analysis' and requires passing metadata_key with the time window. No explicit alternatives or when-not-to-use, but the context is clear enough for an AI agent to decide appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_analytics_group_modelsA
Read-onlyIdempotent

Get a paginated per-model breakdown with total_groups, group_count, and a models array containing request count, cost, and token usage. Use this to compare model cost, popularity, and efficiency; use get_token_analytics or get_cost_analytics for time-series trends instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
page_sizeNoResults per page (max 100)
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
current_pageNoPage number for pagination
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds important behavioral context: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.' This discloses access restrictions and error outcomes. It also mentions pagination, which is not in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, each serving a distinct purpose: what the tool does, when to use it vs alternatives, and enterprise restriction. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (31 params, 2 required, nested objects) and presence of an output schema, the description adequately covers return fields and pagination. It also mentions the enterprise gating, which is critical for usage. The description is sufficient for an agent to understand the tool's behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all 31 parameters have descriptions in the input schema. The description does not add parameter-specific meaning beyond summarizing the high-level purpose. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'paginated per-model breakdown'. It specifies the returned fields (total_groups, group_count, models array with request count, cost, token usage). It also distinguishes from siblings get_token_analytics and get_cost_analytics by mentioning their alternative use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says when to use this tool ('compare model cost, popularity, and efficiency') and when not to ('use get_token_analytics or get_cost_analytics for time-series trends instead'). It names specific alternative tools, providing clear guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_analytics_group_usersA
Read-onlyIdempotent

Get a paginated per-user breakdown with total_groups, group_count, and a users array containing request count, cost, and token usage. Use this for billing, audits, or top-consumer analysis; use get_users_analytics for aggregate active and new user trends. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
page_sizeNoResults per page (max 100)
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
current_pageNoPage number for pagination
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so safety is clear. The description adds value by specifying pagination, output format, and enterprise access restriction, which are beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core functionality, followed by usage guidance and access restrictions. Every sentence adds unique value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (31 parameters, output schema, nested objects), the description covers key aspects: pagination, output fields, use cases, and access control. It could mention pagination parameters explicitly, but overall it is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed descriptions for each of the 31 parameters. The description does not add parameter-specific meaning beyond what the schema provides, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves a paginated per-user breakdown with specific fields (total_groups, group_count, users array). It distinguishes itself from the sibling tool get_users_analytics by noting the latter is for aggregate trends.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool ('billing, audits, or top-consumer analysis') and when to use an alternative ('use get_users_analytics for aggregate active and new user trends'). Also notes enterprise-gating and 403 response for non-Enterprise plans.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_api_keyA
Read-onlyIdempotent

Fetch one API key by UUID without revealing the secret. Use this to inspect scopes, defaults, limits, expiration, and reset state before changing access.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe UUID of the API key to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the description adds the behavioral trait of not revealing the secret, which is beyond annotations. It does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the main purpose, followed by usage guidance. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers what the tool does, what it inspects, and when to use it. With output schema present, return values are not required. Context is complete for a safe read operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a description for the 'id' parameter. The description does not add additional meaning beyond what the schema provides, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Fetch one API key by UUID' with a specific verb and resource, and distinguishes from siblings like list_api_keys by specifying 'one API key' and 'without revealing the secret'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use this to inspect scopes, defaults, limits, expiration, and reset state before changing access,' providing clear context for when to use it, though it does not explicitly mention when not to use it or list alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cache_hit_latencyA
Read-onlyIdempotent

Get cache-hit-only latency time-series data with summary.total_latency, summary.avg_latency, and per-bucket total/avg latency. Use this to evaluate cached-response speed; use get_latency_analytics for all requests. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds that it returns 403 on non-Enterprise plans and that it provides time-series data, which is useful behavioral context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences covering purpose, output fields, alternative use case, and enterprise restriction. No wasted words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has an output schema, annotations, and sibling tools, the description is complete: explains what it returns, when to use it, and the enterprise limitation. No critical gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 29 parameters. The description does not add new meaning beyond summarizing the output fields; it does not augment parameter understanding, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it retrieves cache-hit-only latency time-series data with specific fields (summary.total_latency, etc.) and explicitly contrasts with get_latency_analytics for all requests. This distinguishes it well from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to use (evaluate cached-response speed) and when not (use get_latency_analytics for all requests). Also mentions Enterprise-gating and 403 response for non-Enterprise plans, providing clear usage context and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cache_hit_rateA
Read-onlyIdempotent

Get cache-effectiveness time-series data with summary.hit_rate, summary.total_hits, summary.total_misses, and per-bucket hits/misses/rate. Use this to measure cache effectiveness; use get_cache_hit_latency for speed rather than hit/miss ratio. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, idempotentHint, etc. Description adds enterprise gating and output structure (summary and per-bucket). No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences plus enterprise gating note. Front-loaded with purpose and output fields. No redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite 29 parameters and many filtering options, description doesn't mention filtering capabilities. Output schema exists, but for a complex tool, more detail could help. Adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline 3. Description does not add parameter-specific details beyond schema, which is acceptable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'get cache-effectiveness time-series data' and lists returned fields (hit_rate, total_hits, total_misses, per-bucket data). It distinguishes from sibling get_cache_hit_latency for speed vs hit/miss ratio.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises 'Use this to measure cache effectiveness; use get_cache_hit_latency for speed rather than hit/miss ratio.' Also mentions enterprise gating and 403 response for non-Enterprise plans.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_collectionA
Read-onlyIdempotent

Fetch one collection by id or slug and return its name, slug, workspace, and timestamps. Use list_collections when browsing and get_collection when you already know the target.

ParametersJSON Schema
NameRequiredDescriptionDefault
collection_idYesCollection ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, covering the safety profile. The description adds value by specifying the retrieval method (by id or slug) and the returned fields, which are not in the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loads the verb and resource, and every sentence serves a clear purpose. No unnecessary words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, safe read operation, existing output schema), the description provides all necessary context: what it does, how to use it, and when to prefer it over siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a description for 'collection_id' that already mentions 'Collection ID or slug'. The description does not add new semantic meaning beyond the schema, so baseline is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Fetch', the resource 'one collection by id or slug', and specifies the return fields. It distinguishes from the sibling tool 'list_collections' by mentioning the browsing vs. targeted retrieval use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises when to use this tool ('when you already know the target') versus the alternative 'list_collections' ('when browsing'), providing clear guidance on selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_configA
Read-onlyIdempotent

Get one config by slug and return its routing, cache, retry, and target settings. Requires a known slug; use list_configs to discover one before editing.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe unique identifier (slug) of the configuration to retrieve. This can be found in the configuration's URL or from the list_configs tool response

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, non-destructive, idempotent, and open-world. Description adds value by specifying what settings are returned, which goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. Essential information is front-loaded: action, resource, returned fields, and prerequisite.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, annotations, and output schema presence, the description covers all necessary aspects. It explains what the tool returns and how to obtain the required parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter (slug) with 100% schema description coverage. Description adds context about where to find the slug (URL or list_configs response), enhancing the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('config by slug'), and specifies the returned data: 'routing, cache, retry, and target settings'. It differentiates from sibling tools like list_configs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states prerequisite: 'Requires a known slug' and directs to list_configs for discovery. Provides clear when-to-use and alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cost_analyticsA
Read-onlyIdempotent

Get cost time-series data with summary.total_cost, summary.average_cost_per_request, and per-bucket total/avg cost. Use this for spend analysis and spike detection; use get_token_analytics when you need token volume instead of monetary cost. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/intent, so burden is low. Description adds Enterprise-gated constraint and error detail. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose, no superfluous text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers return fields, usage context, and access restrictions. Output schema exists. Could mention aggregation granularity but not required.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. Description doesn't add parameter-level meaning beyond schema, which is acceptable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Get cost time-series data' with specific fields (summary.total_cost, etc.). Distinguishes from sibling get_token_analytics by monetary vs token focus.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use for spend analysis and spike detection' and 'use get_token_analytics when you need token volume'. Also notes Enterprise gating and 403 response.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_error_analyticsA
Read-onlyIdempotent

Get error-count time-series data with summary.total_errors and per-bucket counts. Use this for high-level error trends; use get_error_rate_analytics for percentages, or get_error_status_codes_analytics and get_error_stacks_analytics for breakdowns. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, idempotentHint, and destructiveHint=false. The description adds important behavioral context: the tool is enterprise-gated and returns a 403 error on non-enterprise plans. It also describes the output structure (summary.total_errors and per-bucket counts). This adds meaningful value beyond annotations, though it does not cover all potential behaviors (e.g., pagination).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences: first states the purpose, second gives usage guidance with siblings, third warns about enterprise gating. It is front-loaded with the tool's core functionality and uses no unnecessary words. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (29 parameters, many legacy/structured pairs) and the presence of an output schema, the description covers the essential context: purpose, usage guidelines, and a critical restriction (enterprise-gated). It does not explain bucket granularity or aggregation details, but the output schema likely covers those. The description is sufficiently complete for an AI agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and each parameter already has a detailed description in the schema. The tool description itself does not add parameter-specific semantics, but with full schema coverage, the baseline of 3 is appropriate. The description does not need to repeat param info.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Get error-count time-series data with summary.total_errors and per-bucket counts', clearly specifying the verb, resource, and output structure. It distinguishes from siblings by naming get_error_rate_analytics, get_error_status_codes_analytics, and get_error_stacks_analytics for different use cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states 'Use this for high-level error trends; use get_error_rate_analytics for percentages, or get_error_status_codes_analytics and get_error_stacks_analytics for breakdowns.' It also notes the tool is enterprise-gated and returns 403 on non-enterprise plans, providing clear when-to-use and when-not-to-use guidance with alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_error_rate_analyticsA
Read-onlyIdempotent

Get error-rate time-series data with summary.error_rate_percent and per-bucket percentages of total requests. Use this for reliability and SLA trends; use get_error_analytics for absolute error counts instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint. The description adds meaningful context: the tool is enterprise-gated and returns 403 on non-Enterprise plans. This is a behavioral trait beyond what annotations provide, and there is no contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences: first states purpose, second gives usage differentiation, third notes enterprise restriction. Every sentence adds value, no redundancy. Front-loaded with the primary action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given high parameter count (29), rich schema, and presence of output schema, the description covers the main output elements, sibling differentiation, and access restriction. Could mention aggregation granularity or time-series specifics, but output schema likely covers that. Slightly incomplete but mostly sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description does not provide additional parameter-level details beyond the schema, but the schema already documents all 29 parameters. No extra value added for parameters, but baseline is acceptable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it retrieves error-rate time-series data with summary and per-bucket percentages, and explicitly distinguishes from sibling get_error_analytics by noting that tool gives absolute counts. The verb-resource combination is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context: use for reliability and SLA trends, and suggests an alternative (get_error_analytics) for absolute error counts. Also notes enterprise gating and potential 403 error. Could add more explicit when-not-to-use scenarios, but the sibling comparison is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_error_stacks_analyticsA
Read-onlyIdempotent

Get stacked error-series data grouped by HTTP status code over time, with summary and per-code series. Use this to see which error classes dominate; use get_error_status_codes_analytics for distinct-code distribution instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only and idempotent behavior. The description adds important access control information: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.' This goes beyond annotations without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is only three sentences, each providing essential information: purpose, usage comparison, and enterprise restriction. No redundant or irrelevant content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 29-parameter schema and output schema, the description is sufficiently complete. It explains the aggregation and grouping, and the alternative tool. Minor lack of detail about the output structure is mitigated by the output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the schema descriptions are detailed. The tool description does not add additional meaning to parameters beyond the schema, so it meets the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool's purpose: 'Get stacked error-series data grouped by HTTP status code over time, with summary and per-code series.' It distinguishes from the sibling tool get_error_status_codes_analytics by explaining when to use each.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly contrasts with get_error_status_codes_analytics and mentions enterprise gating and 403 return, providing clear usage context. However, it lacks broader guidance on when not to use this tool relative to other sibling analytics tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_error_status_codes_analyticsA
Read-onlyIdempotent

Get HTTP error-code distribution time-series data with summary and per-code series. Use this to see which codes occur most often; use get_error_stacks_analytics for stacked or cumulative breakdowns. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, openWorldHint=true, destructiveHint=false. The description adds the enterprise restriction and 403 response, which are beyond annotations. No contradiction. However, additional behavioral details like rate limits or pagination could be mentioned, but given annotation coverage, this is sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Description is very concise: two sentences plus a one-sentence enterprise note. It is front-loaded with the core purpose in the first sentence, and every sentence adds value. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (29 parameters, output schema exists), the description adequately covers the tool's purpose, output type (time-series, summary, per-code series), and usage context. The output schema handles return value details, so description does not need to elaborate. The sibling distinction completes the context. Minor gap: could mention that required parameters are time bounds, but schema handles that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description itself does not add any parameter-level details; all parameter semantics are already provided in the input schema. The description's mention of 'time-series data' implies temporal parameters, but the schema explicitly documents them. Hence, no additional value beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool retrieves HTTP error-code distribution time-series data with summary and per-code series. It uses specific verbs ('Get') and resource ('HTTP error-code distribution time-series data'), and explicitly distinguishes from sibling get_error_stacks_analytics by specifying that tool is for stacked/cumulative breakdowns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance on when to use this tool ('see which codes occur most often') and when to use the alternative ('get_error_stacks_analytics for stacked or cumulative breakdowns'). Also states that it's Enterprise-gated and returns 403 on non-Enterprise plans, which informs usage conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feedback_analyticsA
Read-onlyIdempotent

Get feedback-submission time-series data with summary totals and per-bucket counts. Use this as the top-level feedback trend view; use get_feedback_models_analytics, get_feedback_scores_analytics, or get_feedback_weighted_analytics for breakdowns. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, so the safety profile is clear. The description adds behavioral context: enterprise-gating and 403 response, which are beyond annotations. It does not contradict annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no filler: first states purpose, second gives usage alternatives, third mentions enterprise-gating and error. Front-loaded and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 29 parameters, 2 required, and an output schema exists, the description provides a high-level overview (time-series, summary, per-bucket counts) and error condition. It covers essential context without needing to detail all parameters, as the schema handles that. Slightly more could be added about optional filtering, but it's sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters sufficiently. The description adds no additional parameter-specific meaning beyond what the schema provides, resulting in a baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'Get' and identifies the resource as 'feedback-submission time-series data' with outputs 'summary totals and per-bucket counts'. It also distinguishes this tool from siblings by naming specific alternatives for breakdowns, ensuring clear differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool ('top-level feedback trend view') and when to use alternatives ('get_feedback_models_analytics, get_feedback_scores_analytics, or get_feedback_weighted_analytics for breakdowns'). Also notes enterprise-gating and 403 error for non-Enterprise plans, providing clear usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feedback_models_analyticsA
Read-onlyIdempotent

Get feedback time-series data grouped by model, with per-model counts over time. Use this to compare feedback volume and satisfaction across models; use get_feedback_analytics for the overall total instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds that the tool is enterprise-gated and returns 403 for non-Enterprise plans, which is valuable behavioral context beyond annotations. However, it does not mention rate limits, pagination, or other potential behaviors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences that front-load the core purpose and immediately provide usage guidance. Every sentence adds value with no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has an output schema (covering return values), 29 parameters (2 required), and annotations, the description is complete enough. It explains the tool's role, when to use it, and Access restrictions. No additional information seems necessary for an analytics retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific meaning beyond what the schema already provides; it simply states the overall purpose. No additional guidance on how to use parameters is given.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get feedback time-series data grouped by model, with per-model counts over time.' It uses a specific verb ('Get') and resource ('feedback time-series data grouped by model'), and distinguishes from the sibling tool 'get_feedback_analytics' by noting that it provides per-model breakdowns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use this to compare feedback volume and satisfaction across models; use get_feedback_analytics for the overall total instead.' It also mentions that the tool is 'Enterprise-gated' and 'Returns 403 on non-Enterprise Portkey plans,' providing clear when-to-use and alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feedback_scores_analyticsA
Read-onlyIdempotent

Get raw feedback-score distribution time-series data with per-score buckets. Use this to understand sentiment mix; use get_feedback_weighted_analytics for calibrated scores with weighting. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds Enterprise-gating behavior and 403 error response, which is valuable beyond annotations. However, it does not discuss rate limits, pagination, or other behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states the core functionality, second provides usage guidance and enterprise note. No filler, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

While the output schema exists (reducing need to describe return values), the description does not mention the extensive filter parameters available in the schema, which is a gap for a tool with 29 parameters. The description is adequate but could be more complete regarding filtering options.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The tool description does not add any additional meaning or context to the parameters, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it retrieves 'raw feedback-score distribution time-series data with per-score buckets' and distinguishes itself from get_feedback_weighted_analytics, which handles calibrated scores. The verb 'Get' and resource 'feedback-score distribution' are specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells when to use this tool ('understand sentiment mix') and when to use the alternative ('get_feedback_weighted_analytics for calibrated scores'). Also notes Enterprise-gating and 403 error on non-Enterprise plans, providing clear context for usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_feedback_weighted_analyticsA
Read-onlyIdempotent

Get weighted feedback-score time-series data using the weight recorded at feedback creation. Use this for calibrated quality metrics; use get_feedback_scores_analytics for the raw unweighted distribution. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations include readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds the enterprise gating constraint and the 403 response on non-Enterprise plans, which is behavioral context beyond annotations. No contradictions. A 4 is appropriate as the description enriches transparency but the annotations already cover most behavioral aspects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the core purpose, and includes the critical usage distinction and gating info. Every sentence adds value with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema (not shown but indicated as present), so return value details are not needed. The description covers purpose, usage distinction, and access restrictions. It is complete for a read-only analytics tool with 100% schema coverage and strong annotations. Slight gap: it doesn't mention the read-only nature explicitly, but annotations cover that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all 29 parameters, including legacy/preferred pairs and formatting notes. The description itself does not add parameter-specific information; it only contextualizes the output as time-series data. Baseline 3 is correct because the schema already provides comprehensive parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves weighted feedback-score time-series data. It explicitly distinguishes from the sibling tool get_feedback_scores_analytics by noting it uses the weight recorded at creation for calibrated metrics. The verb 'Get' and resource 'feedback-weighted-analytics' are specific and unique among siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use this for calibrated quality metrics; use get_feedback_scores_analytics for the raw unweighted distribution.' It also warns about an enterprise gating mechanism: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.' This clearly communicates when to use and when not to use, with a named alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_guardrailA
Read-onlyIdempotent

Fetch one guardrail by id or slug with its full checks and actions; use list_guardrails to discover ids first. Use before update_guardrail or delete_guardrail when you need the exact enforcement policy, and returns the full check and action configuration alongside status and ownership.

ParametersJSON Schema
NameRequiredDescriptionDefault
guardrail_idYesThe guardrail UUID or slug (with guard_ prefix) to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, non-destructive, idempotent behavior. The description adds that it returns full check and action configuration, status, and ownership, which is valuable beyond annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states the purpose, second provides usage guidance. No unnecessary words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and an output schema (present but not shown), the description covers the purpose, usage context, and return content well. Slight deduction for not mentioning pagination or filtering, but not needed given the fetch-by-ID nature.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a clear description of guardrail_id. The description mentions 'by id or slug', which matches but does not add significant new semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies 'Fetch one guardrail' by id or slug, clearly defining the verb and resource. It distinguishes from sibling tool list_guardrails by mentioning to use that for discovery first.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states to use this tool before update_guardrail or delete_guardrail when the exact enforcement policy is needed, providing clear context and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_integrationA
Read-onlyIdempotent

Fetch one integration by slug, including masked key, workspace access, allowed models, and configuration metadata. Use this before editing provider-specific settings or auditing access. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe unique slug identifier of the integration to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds useful context about returned fields (masked key, workspace access, etc.) and enterprise restriction. No contradiction with annotations; additional transparency provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each earning its place: purpose+details, usage context, enterprise restriction, error note. Front-loaded and concise with zero waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, usage, enterprise gating, and error codes. Has output schema, so return values not needed. Complete for a simple retrieval tool; could mention authentication but annotations cover readOnly. Sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage for the single parameter 'slug' with a complete description. The tool description only repeats 'by slug' without adding format or source details. Baseline 3 appropriate as description adds no extra value beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb 'Fetch' and specific resource 'one integration by slug', listing included data (masked key, workspace access, allowed models, configuration metadata). Differentiates from siblings like list_integrations, create_integration, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises use 'before editing provider-specific settings or auditing access', providing clear context. Also warns about Enterprise gating and 403 error, implying when not to use. No alternative tool named, but guidance is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_latency_analyticsA
Read-onlyIdempotent

Get latency time-series data with summary.avg_latency_ms, summary.p50_latency_ms, summary.p90_latency_ms, summary.p99_latency_ms, and per-bucket latency percentiles in ms. Use this to spot slowdowns and regressions; use get_cache_hit_latency when you only want cache-hit latency. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds critical behavioral context: Enterprise-gated and returns 403 on non-Enterprise plans, which annotations do not cover.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very concise: three sentences, front-loaded with what the tool does, then usage guidance, then behavioral note. Every sentence adds value, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (29 parameters, legacy/structured aliases, output schema exists), the description covers output details, usage context, and access restrictions. Missing explicit mention that output is time-series, but implied. Slightly short of 5 due to lack of more detail on time-series structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description does not add any parameter-level information beyond the schema; it only mentions output fields. Thus, no additional value for parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb 'get' and resource 'latency time-series data', lists output fields (summary.avg_latency_ms, etc.), and distinguishes from sibling 'get_cache_hit_latency' by stating when to use each.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: 'to spot slowdowns and regressions'; provides alternative: 'use get_cache_hit_latency when you only want cache-hit latency'; also notes Enterprise-gating, guiding non-Enterprise users away.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_logA
Read-onlyIdempotent

Get one gateway request log by ID, including its request, response, usage, cost, and metadata payload when available. For path_format v2, also provide the log's created_at timestamp.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoSet to hooks when retrieving a hook execution log
log_idYesGateway log ID to retrieve
created_atNoLog creation timestamp in ISO 8601 format; required for v2
path_formatNoLog storage path format; defaults to v1

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds meaningful context: the included fields are conditional ('when available'), and for path_format v2 the caller must provide created_at. No contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, and every clause contributes relevant information. It is appropriately concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-log retrieval with an output schema and strong annotations, the description covers essential behavior and the key path_format caveat. The output schema handles return format, so no further detail is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, providing a baseline of 3. The description adds the critical relationship that path_format v2 requires created_at, which is not marked required in the schema. It does not cover the 'type' parameter, but the schema fully documents it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get one gateway request log by ID' with a specific verb, resource, and scope, and lists the included payload elements (request, response, usage, cost, metadata). It distinguishes from sibling log-export tools by focusing on single-log retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: when retrieving a specific gateway log by ID with full details. However, it does not explicitly name alternatives or provide exclusions, so it falls short of a perfect score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_log_exportA
Read-onlyIdempotent

Fetch one log export job by export_id and return its status, filters, requested fields, and file metadata. Use this when you already know the target; use list_log_exports for a workspace-wide overview. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds useful context beyond annotations: it discloses the return payload fields and the Enterprise plan requirement that may cause a 403. This gives the agent important behavioral and error-handling context without contradicting the structured hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, each earning its place: function and return value, usage distinction, and access restriction. It is front-loaded with the primary action and avoids any filler. This is a model of concise, high-signal writing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one required parameter, a well-populated schema, an output schema, and rich annotations, this description provides all necessary context: what it retrieves, when to use it, and what plan is required. There are no significant gaps given the tool's low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema fully covers the single parameter export_id with a description ('The unique ID of the log export'). The description adds no additional parameter-level syntax or format detail, but because schema coverage is 100%, the baseline of 3 is appropriate; the tool's simple structure does not require further elaboration.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Fetch one log export job by export_id') and the resource (a single log export job), and lists what it returns (status, filters, requested fields, file metadata). This distinguishes it from the sibling tool list_log_exports, which is explicitly named as an alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use guidance: 'Use this when you already know the target; use list_log_exports for a workspace-wide overview.' This clearly separates it from the alternative and also notes the Enterprise-gated access and 403 behavior, which helps set expectations for use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_log_export_field_restrictionsA
Read-onlyIdempotent

Get the organisation-managed fields that a workspace is restricted from including in log exports. Use before create_log_export or update_log_export to avoid requesting disallowed fields. Requires an API key with logs.export scope; completion-log fields may additionally require completion scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesWorkspace ID whose export field restrictions should be read

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint=false, but the description adds significant behavioral context beyond those: required API key scopes (logs.export, completion scope for some fields), enterprise gating, and the 403 error on non-Enterprise plans. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each earning its place: primary purpose, usage directive, and necessary caveats. Front-loaded with the core function, no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema is present and annotations cover operational hints, the description fully rounds out context: it explains when to call (before exports), what scopes are needed, and enterprise gating. This is complete for a read-only, single-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter workspace_id is fully described in the schema ('Workspace ID whose export field restrictions should be read'), with 100% schema description coverage. The description does not add syntax or additional meaning to the parameter beyond what the schema already provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Get') and a precise resource ('organisation-managed fields that a workspace is restricted from including in log exports'). It clearly distinguishes the tool from sibling tools by referencing create_log_export and update_log_export, making its unique function obvious.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs when to use: 'Use before create_log_export or update_log_export to avoid requesting disallowed fields.' It also provides contextual caveats (API key scope, enterprise gating, 403 on non-Enterprise), giving clear guidance on prerequisites and expected failures.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_mcp_integrationA
Read-onlyIdempotent

Retrieve one MCP integration by id or slug. Returns the full Portkey-side config, including auth type, transport, and masked configuration keys; use get_mcp_integration_metadata for the server's self-reported metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. The description adds valuable behavioral context by specifying what is returned (full Portkey-side config including auth type, transport, masked keys) without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main purpose, and every word contributes. No extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one required parameter, output schema present), the description fully covers what the tool does, what it returns, and how it relates to a sibling tool. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already has 100% coverage with a description including 'ID or slug'. The description reiterates the same information without adding new parameter semantics beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Retrieve' and the resource 'one MCP integration' with identifier id or slug. It distinguishes from the sibling tool 'get_mcp_integration_metadata' by specifically mentioning that alternative for server-reported metadata.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance is provided: 'use get_mcp_integration_metadata for the server's self-reported metadata'. This clearly tells when to use this tool versus an alternative, making the usage context unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_mcp_integration_metadataA
Read-onlyIdempotent

Retrieve the external MCP server's self-reported metadata for an integration. Returns name, version, protocol, capability flags, and sync status; use get_mcp_integration for the Portkey-side connection config.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive. Description adds that it returns self-reported metadata with specific fields, enhancing agent understanding beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff, efficient communication of purpose and distinction.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple retrieval tool with output schema and clear annotations, the description covers purpose, return values, and sibling differentiation. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and description does not add any additional meaning to the single 'id' parameter beyond what the schema already provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it retrieves external MCP server metadata, lists returned fields, and explicitly distinguishes from sibling tool get_mcp_integration for connection config.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Directly tells when to use this tool vs. get_mcp_integration, providing a clear alternative for a different purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_mcp_serverA
Read-onlyIdempotent

Retrieve one MCP server by id or slug. Returns server details including the parent integration, status, and created time; use get_mcp_server when you need the server record rather than the integration config.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the description does not need to repeat these. It adds value by specifying the return fields (parent integration, status, created time), which gives behavioral insight beyond the schema. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two sentences, each earning its place. The first sentence states purpose and return info, the second provides usage guidance. No extraneous words, front-loaded effectively.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (single parameter, output schema exists), the description sufficiently covers what the tool does, what it returns, and when to use it. Annotations and output schema cover the rest, so no gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (the parameter 'id' has a clear description). The description adds only marginal value by mentioning 'by id or slug', which is already in the parameter description. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verb 'Retrieve' and resource 'one MCP server by id or slug', clearly stating the action and input. It distinguishes itself from get_mcp_integration by specifying when to use this tool for server records rather than integration config.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool vs. the alternative get_mcp_integration: 'use get_mcp_server when you need the server record rather than the integration config.' This provides clear context and an alternative, meeting the when/alternatives criterion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_model_pricingA
Read-onlyIdempotent

Get Portkey's current public pricing configuration for one exact provider/model pair. Prices are returned in USD cents per token or provider-specific unit and may include cache, audio, image, fine-tuning, and calculation metadata. Use this read-only catalog lookup before setting integration pricing_adjustments or custom-model pricing; it does not return your negotiated integration multiplier or require Portkey authentication.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelYesExact provider model identifier, such as gpt-4o
providerYesLowercase Portkey provider identifier, such as openai, anthropic, bedrock, or x-ai

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as read-only, open-world, and idempotent. The description adds valuable context on return semantics (USD cents per token or provider-specific unit), the types of metadata included (cache, audio, image, fine-tuning, calculation), and the exclusion of negotiated pricing. This goes beyond the structured annotations and helps the agent set expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences long, front-loaded with the core purpose, and every sentence adds critical information. There is no redundancy or filler, making it easy for an agent to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (2 params, read-only, output schema present), the description provides complete context: what it returns, its units, content categories, what it excludes, and when to use it. The output schema covers return structure, so no further detail is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific meaning beyond the schema, but it reinforces the 'exact pair' requirement. Since the schema already handles parameter semantics fully, this is sufficient.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Get Portkey's current public pricing configuration') and clearly scopes it to one exact provider/model pair. This distinguishes it from any analytics tools or other pricing-related operations. The purpose is unambiguous and immediately understandably.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says when to use the tool ('before setting integration pricing_adjustments or custom-model pricing') and what it does not do ('does not return your negotiated integration multiplier or require Portkey authentication'). This gives clear usage boundaries and prevents misuse, even without naming alternative tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_organisation_defaultsA
Read-onlyIdempotent

Get the organisation-wide input and output guardrails that workspaces inherit by default. Use this before update_organisation_defaults or when auditing baseline enforcement; it does not include per-workspace exclusions, which are available from the directional exclusion list tools. Requires an organisation service API key with organisation_settings.read scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the safety profile is covered. The description adds valuable behavioral context by specifying auth scope, enterprise-gating, and the 403 response on non-Enterprise plans, plus clarifying that per-workspace exclusions are not included. It doesn't contradict annotations and adds useful information beyond them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, front-loaded with the core purpose, and each sentence earns its place: purpose, usage, exclusions, auth, and error behavior. No redundant or extraneous content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (no parameters) and presence of an output schema, the description is complete: it specifies purpose, usage context, exclusions, access prerequisites, and error behavior. It fully equips an agent to select and invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so per rubric the baseline is 4. The description doesn't need to add parameter details since there are none, and the schema coverage is effectively complete for an empty parameter set.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get the organisation-wide input and output guardrails that workspaces inherit by default' with a specific verb and resource, and explicitly distinguishes from related tools like update_organisation_defaults and exclusion list tools. It fully clarifies scope, making it unmistakable what this tool does relative to siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Use this before update_organisation_defaults or when auditing baseline enforcement' and states what is excluded and where to find it ('directional exclusion list tools'). Also includes required scope and enterprise-gating requirements, giving clear when-to-use and when-not-to-use context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_promptA
Read-onlyIdempotent

Fetch a prompt's full definition, active version, and version history. Use this before updating, publishing, rendering, or copying a prompt when you need the stored template and metadata. For multi-message chat prompts pass messages (preferred) or a JSON-encoded array as string.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnly, non-destructive, idempotent. The description adds that it returns active version and history, and gives specific advice for multi-message chat prompts, which goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and output, second provides usage guidance and a specific edge case. No redundant information, perfectly front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter tool with output schema and annotations, the description fully covers what the tool returns and when to use it, including a notable edge case for multi-message chats.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the baseline is 3. The description does not add additional semantic meaning to the prompt_id parameter beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it fetches a prompt's full definition, active version, and version history. This distinguishes it from sibling tools like get_prompt_version (specific version) and get_prompt_label (label metadata).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly recommends using this tool before updating, publishing, rendering, or copying a prompt. It provides clear when-to-use context and an alternative approach for multi-message chats, though it does not explicitly state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prompt_labelA
Read-onlyIdempotent

Fetch one label's full definition, including scope, color, and status. Use this when you already know the label_id; list_prompt_labels is better for browsing candidates.

ParametersJSON Schema
NameRequiredDescriptionDefault
label_idYesLabel ID to retrieve
workspace_idNoWorkspace ID for filtering
organisation_idNoOrganisation ID for filtering

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, destructiveHint, idempotentHint, openWorldHint, so safety is clear. Description adds no new behavioral traits beyond return content, which is expected for a fetch tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, then usage guidance. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given output schema exists and annotations are rich, the description is sufficient for a simple fetch tool. Minor omission: optional filter parameters not elaborated, but schema covers them.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so description adds no meaning beyond what schema already provides for parameters. It mentions needing label_id but that is already required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the verb 'Fetch' and the resource 'one label's full definition', and lists included fields (scope, color, status). It explicitly distinguishes from sibling list_prompt_labels.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to use this tool (when you already know the label_id) and when to use the alternative (list_prompt_labels for browsing).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prompt_partialA
Read-onlyIdempotent

Fetch a partial's content and current version details. Use this before embedding, updating, or checking what {{> partial_name}} resolves to; returns the stored string plus version metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_partial_idYesPrompt partial ID or slug to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds value beyond the annotations by specifying what the tool returns ('the stored string plus version metadata') and its purpose (pre-embedding/update check). Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, so safety is clear. The description does not contradict any annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two sentences, both front-loaded with key information. The first sentence states the core action and objective. The second sentence provides usage guidance and return details. Every phrase earns its place; there is no wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has one required parameter and an output schema. The description covers the main use case (fetching content and version details) and advises when to use it. Given the low complexity and the presence of an output schema, the description is sufficient. It could briefly mention that the partial is from a prompt_template, but that is not necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has one parameter (prompt_partial_id) with a description already provided ('Prompt partial ID or slug to retrieve'). Schema description coverage is 100%. The tool description does not add additional meaning beyond what the schema already provides, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Fetch a partial's content and current version details') and specifies the resource (a partial). It distinguishes itself from siblings like list_prompt_partials (which lists all partials) and get_prompt (which retrieves a prompt). The verb 'Fetch' is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises when to use this tool: 'Use this before embedding, updating, or checking what {{> partial_name}} resolves to'. This provides clear context for invocation. It does not explicitly mention alternatives, but the context implies that for listing all partials one would use list_prompt_partials, and for mutations one would use create/update_prompt_partial.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prompt_versionA
Read-onlyIdempotent

Retrieve a specific prompt version by its version UUID. Use list_prompt_versions to find the id first; returns the template, parameters, and model config for that version.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug
version_idYesVersion UUID to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so description needs little behavioral context. It adds value by stating what fields are returned (template, parameters, model config), which is helpful but not critical.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff. Every word earns its place, front-loaded with the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that output schema exists, the description appropriately highlights key return fields. It covers purpose, prerequisite steps, and output summary, making it complete for a retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters are fully described in the schema (100% coverage). The description adds no additional meaning to the parameters beyond usage guidance, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it retrieves a specific prompt version by UUID, distinguishing it from list_prompt_versions and get_prompt. The verb 'Retrieve' and resource 'prompt version' are specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs to use list_prompt_versions first to find the ID, providing clear when-to-use guidance and a direct reference to a sibling tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_providerA
Read-onlyIdempotent

Fetch one provider by slug, including limits, rate settings, expiration, and reset status. Use this to check consumption or audit configuration before updating.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe unique slug identifier of the provider to retrieve
workspace_idNoWorkspace ID - required when using organization admin keys

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds valuable context beyond annotations by specifying that the response includes 'limits, rate settings, expiration, and reset status,' which informs the agent about the richness of the data. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise sentences: the first states the action and included data, the second provides usage guidance. No extraneous words; every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema (not shown but indicated), the description is complete: it explains the tool's function, included data, and use case. No gaps given the low complexity of a single-resource fetch operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage with descriptions for 'slug' and 'workspace_id'. The description adds the phrase 'by slug', but does not provide additional semantic detail beyond what the schema already states. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Fetch one provider by slug' – a specific verb+resource. It details the data returned (limits, rate settings, expiration, reset status) and distinguishes from sibling tools like list_providers and update_provider by focusing on a single entity and use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear usage context: 'Use this to check consumption or audit configuration before updating.' This guides the agent to use it for auditing or pre-update checks. While it doesn't explicitly name alternative tools, the context is sufficient to differentiate from list_providers or update_provider.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_rate_limitA
Read-onlyIdempotent

Get one rate limit by id and return its full conditions and grouping definition. Use list_rate_limits to discover ids or compare policies first.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe unique identifier of the rate limit

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already define safety profile (readOnly, idempotent, non-destructive). Description adds detail about return content (conditions and grouping), which is useful but doesn't introduce new behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and return, second gives usage guidance. No redundant words, efficient and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With output schema present, description doesn't need to detail return values. It covers purpose, return content, and usage context fully.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema describes 'id' parameter fully (100% coverage). Description only repeats 'by id', adding no extra semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states verb 'Get', resource 'rate limit', and scope 'by id' while specifying return content (full conditions and grouping). Mention of sibling tool list_rate_limits provides differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to use list_rate_limits first to discover ids, giving clear when-to-use guidance and an alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_request_analyticsA
Read-onlyIdempotent

Get request-volume time-series data with summary.total_requests, summary.successful_requests, summary.failed_requests, and per-bucket total/success/failed counts. Use this for traffic and reliability trends; use get_error_analytics when you only need error counts. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive. Description adds enterprise-gating and 403 error response, plus outlines response structure (summary and per-bucket counts). No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each serving a distinct purpose: response fields, usage guidance, enterprise constraint. No redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 29 params and output schema, description covers purpose, usage, and enterprise constraint. Could mention aggregation granularity, but output schema likely covers response shape. Adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so description need not repeat parameter details. Description does not add new parameter semantics beyond implicit date range from required params. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it gets request-volume time-series data with specific fields (summary.total_requests, etc.). Distinguishes from sibling get_error_analytics by specifying use case. Also mentions enterprise gating.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use this for traffic and reliability trends; use get_error_analytics when you only need error counts.' Provides clear guidance on when to use this tool versus an alternative. Also implies enterprise plan required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_rescued_requests_analyticsA
Read-onlyIdempotent

Get rescued-request time-series data showing requests recovered by retry or fallback handling. Use this only when your configs include resilience features, and use it to measure how often recovery logic saved requests. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false. The description adds enterprise gating and a 403 error response for non-Enterprise plans, which is valuable behavioral context not covered by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences totaling ~40 words. The first sentence states the purpose, the second provides usage guidance and a critical constraint. No redundant or extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 29 parameters and an output schema, the description provides essential context: what it does, when to use it, and a key authorization constraint. The output schema handles return value details. Acceptable completeness for a tool with rich structured metadata.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all 29 parameters, so the baseline is 3. The tool description adds no additional parameter meaning beyond the high-level purpose. It does not elaborate on any specific parameters, but the schema fully documents them.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves rescued-request time-series data for requests recovered by retry or fallback handling. It uses a specific verb ('Get') and specifies the resource ('rescued-requests analytics'). This distinguishes it from sibling analytics tools like get_cost_analytics or get_error_analytics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use this only when your configs include resilience features, and use it to measure how often recovery logic saved requests.' It does not explicitly mention alternatives or when not to use, but the context is clear and helpful.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_secret_referenceA
Read-onlyIdempotent

Retrieve one Secret Reference by UUID or slug. Portkey masks sensitive authentication fields for non-system users; the tool returns metadata and the masked auth configuration, not the resolved external secret value.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesSecret Reference UUID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds important context beyond annotations: masking behavior for non-system users, returns metadata and masked auth, not the actual secret value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two precise sentences: first states action and identifier, second adds critical behavioral nuance. No redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers retrieval purpose, identification method, return content (metadata + masked auth), and what is NOT returned (secret value). Output schema handles return structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only parameter 'id' is fully documented in schema (100% coverage). Description reiterates 'UUID or slug' but adds no new semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Retrieve one Secret Reference' with identifier type (UUID or slug), distinguishing it from sibling list/create/update/delete tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies singular retrieval vs list by using 'one', but lacks explicit when-to-use vs alternatives. Context signals and siblings help differentiate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_token_analyticsA
Read-onlyIdempotent

Get token-usage time-series data with summary.total_tokens, summary.prompt_tokens, summary.completion_tokens, and per-bucket total/prompt/completion counts. Use this for consumption trends; use get_cost_analytics when you need spend instead of token volume. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false, establishing safety. The description adds value by disclosing the enterprise gating and the 403 error response on failure, which goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, each serving a distinct purpose: the first defines the tool's function and output; the second provides usage guidance, sibling distinction, and access constraints. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 29 parameters, 100% schema coverage, output schema, and rich annotations, the description covers core purpose and a key sibling. However, it lacks details on time-series bucket granularity, pagination, or ordering. Still, the structured fields and schema compensate partially.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are already well-documented. The description does not add extra meaning to specific parameters but clarifies the overall output structure (summary fields, per-bucket counts). Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves token-usage time-series data with specific metrics (summary.total_tokens, summary.prompt_tokens, summary.completion_tokens, per-bucket counts). It also distinguishes from the sibling 'get_cost_analytics' by noting the difference between token volume and spend.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says 'Use this for consumption trends' and suggests 'use get_cost_analytics when you need spend instead of token volume'. It also notes the enterprise gating and 403 error for non-enterprise plans, providing clear context for when the tool is applicable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_usage_limitA
Read-onlyIdempotent

Get one usage limit by id and return its full budget, threshold, grouping, and reset details. Use list_usage_limits to discover ids or compare policies first.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe unique identifier of the usage limit

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds value by specifying the exact return fields (budget, threshold, grouping, reset details), making the tool's behavior fully transparent without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: the first front-loads purpose and return details, the second provides usage guidance. No unnecessary words; every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (1 parameter, no nested objects) and the existence of an output schema, the description adequately covers what the tool does, what it returns, and when to use it. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter 'id'. The description does not add any new meaning beyond what the schema already provides, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get one usage limit by id') and specifies the returned data ('full budget, threshold, grouping, and reset details'). It distinguishes itself from sibling tool list_usage_limits by referencing it for discovery.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use list_usage_limits to discover ids or compare policies first,' providing clear when-to-use and when-not-to-use guidance with a named alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_userA
Read-onlyIdempotent

Get one accepted user by id and return their profile, role, and timestamps. Use list_all_users to find the id if you only have a name or email, and get_user_invite for pending invitations. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesThe user ID to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, destructiveHint false, etc. The description adds context that the tool only returns 'accepted' users and is Enterprise-gated, providing useful behavioral detail beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose, no redundant information. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get-by-id tool with one parameter and an output schema, the description covers the return fields, how to find the id, alternatives for different states, and access restrictions. Complete for its complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The sole parameter user_id is described in schema as 'The user ID to retrieve'. The description adds no new semantic meaning to the parameter itself, but does provide context on obtaining the id from list_all_users, which is more about usage than parameter semantics. Schema coverage is 100%, so baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and resource 'one accepted user', and distinguishes from siblings like list_all_users and get_user_invite by specifying the scope and alternative uses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly guides when to use list_all_users (to find id from name/email) and get_user_invite (for pending invitations), and warns about Enterprise gating and 403 response.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_inviteA
Read-onlyIdempotent

Get one invitation by invite id and return its email, role, status, and expiry. Use this for pending invites only; use get_user for accepted users.

ParametersJSON Schema
NameRequiredDescriptionDefault
invite_idYesThe invite ID to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds that it returns specific fields (email, role, status, expiry) and that it's for pending invites. There is no contradiction, and the behavior is well disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, no unnecessary words. It front-loads the purpose and includes usage guidance in a compact form.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a single parameter, strong annotations, and the presence of an output schema (context shows has output schema: true), the description is sufficient. It covers what the tool does, what it returns, and when to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter is invite_id, which has a clear schema description ('The invite ID to retrieve'). The tool description does not add further semantic detail beyond what the schema already provides. With 100% schema coverage, the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves a single invitation by ID and lists the returned fields (email, role, status, expiry). It also distinguishes from get_user by explicitly stating 'Use this for pending invites only; use get_user for accepted users.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use this for pending invites only; use get_user for accepted users.' This tells the agent when to use this tool vs. an alternative sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_requests_analyticsA
Read-onlyIdempotent

Get per-user request-count time-series data with counts grouped by user. Use this to find heavy users and traffic concentration; use get_users_analytics for aggregate active and new user trends instead. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering safety. The description adds the crucial behavioral detail that the tool is Enterprise-gated and returns 403 on non-Enterprise plans, which is not in annotations. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is only three sentences: purpose, usage guidance with alternative, and access constraint. Every sentence is essential, no redundancy, and the most critical information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the high parameter count (29) and the presence of an output schema, the description suffices by covering purpose, usage context, and access limitations. It does not detail return values (handled by output schema) or every parameter (handled by input schema), but this is appropriate for a tool with rich structured metadata.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, so the schema fully documents all 29 parameters. The tool description does not add any parameter explanations beyond what the schema provides, which is acceptable per the baseline rule for high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies 'Get per-user request-count time-series data with counts grouped by user', which clearly states the verb, resource, and grouping. It explicitly distinguishes from the sibling 'get_users_analytics' by contrasting use cases (heavy users vs aggregate trends).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use this to find heavy users and traffic concentration; use get_users_analytics for aggregate active and new user trends instead.' It also discloses that the tool is 'Enterprise-gated' and returns 403 on non-Enterprise plans, helping the agent avoid errors.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_users_analyticsA
Read-onlyIdempotent

Get user-growth time-series data with summary.total_active_users, summary.total_new_users, and per-bucket active/new user counts. Use this for growth and adoption trends; use get_user_requests_analytics for per-user traffic or get_analytics_group_users for per-user cost and token detail. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
configsNoLegacy Portkey query param for config slugs. Comma-separated string; prefer config_slugs for structured inputs.
span_idNoLegacy Portkey query param for span IDs. Comma-separated string; prefer span_ids for structured inputs.
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
metadataNoLegacy Portkey query param for metadata filtering. Stringified JSON object, e.g. '{"env":"prod","app":"myapp"}'; prefer metadata_filter for structured inputs.
span_idsNoStructured alias for span_id. Use an array of span IDs; normalized to the legacy comma-separated Portkey query param.
trace_idNoLegacy Portkey query param for trace IDs. Comma-separated string; prefer trace_ids for structured inputs.
trace_idsNoStructured alias for trace_id. Use an array of trace IDs; normalized to the legacy comma-separated Portkey query param.
api_key_idsNoLegacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form.
prompt_slugNoFilter by prompt slug
status_codeNoLegacy Portkey query param for HTTP status codes. Comma-separated string; prefer status_codes for structured inputs.
ai_org_modelNoLegacy Portkey query param for provider/model pairs. Format: 'provider__model' with double underscore, e.g. 'openai__gpt-4' or 'anthropic__claude-3-opus'. Comma-separated string; prefer provider_models for structured inputs.
config_slugsNoStructured alias for configs. Use an array of config slugs; normalized to the legacy comma-separated Portkey query param.
status_codesNoStructured alias for status_code. Use an array of HTTP status codes; normalized to the legacy comma-separated Portkey query param.
virtual_keysNoLegacy Portkey query param for virtual key slugs. Comma-separated string; prefer virtual_key_slugs for structured inputs.
workspace_slugNoFilter by specific workspace
metadata_filterNoStructured alias for metadata. Use an object such as { env: 'prod' }; normalized to a JSON string before the request is sent.
provider_modelsNoStructured alias for ai_org_model. Use provider__model strings in an array; normalized to the legacy comma-separated Portkey query param.
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
prompt_token_maxNoMaximum number of prompt tokens
prompt_token_minNoMinimum number of prompt tokens
virtual_key_slugsNoStructured alias for virtual_keys. Use an array of virtual key slugs; normalized to the legacy comma-separated Portkey query param.
completion_token_maxNoMaximum number of completion tokens
completion_token_minNoMinimum number of completion tokens
weighted_feedback_maxNoMaximum weighted feedback score (-10 to 10)
weighted_feedback_minNoMinimum weighted feedback score (-10 to 10)
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint. Description adds enterprise restriction and 403 behavior, which is useful beyond annotations. However, no mention of pagination or other side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences plus a warning, no extra words. Front-loaded with core purpose. Efficient and clear.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given output schema exists, description adequately explains key return fields and enterprise gating. For a 29-parameter tool, it covers the essential context without overloading.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. Description does not add parameter-specific meaning beyond what schema provides; it focuses on return values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves user-growth time-series data with specific fields (summary.total_active_users, summary.total_new_users, per-bucket counts). It also distinguishes from sibling tools by naming alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance on when to use (growth and adoption trends) and when not (per-user traffic, per-user cost/token detail) with named alternatives. Also warns about enterprise gating and 403 error.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_statsA
Read-onlyIdempotent

Return per-user request and cost analytics for a required time range. This is usage-by-user, not population metrics; use get_users_analytics for active-user or cohort trends. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
cost_maxNoMaximum cost in cents to filter by
cost_minNoMinimum cost in cents to filter by
page_sizeNoNumber of results per page (max 100)
status_codeNoFilter by specific HTTP status codes (comma-separated)
virtual_keysNoFilter by specific virtual key slugs (comma-separated)
total_units_maxNoMaximum number of total tokens to filter by
total_units_minNoMinimum number of total tokens to filter by
time_of_generation_maxYesEnd time for the analytics period (ISO8601 format, e.g., '2024-02-01T00:00:00Z')
time_of_generation_minYesStart time for the analytics period (ISO8601 format, e.g., '2024-01-01T00:00:00Z')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. The description adds valuable behavioral context not covered by annotations, such as the Enterprise-gated access (returns 403 on non-Enterprise plans) and the distinction from population metrics, which clarifies the tool's scope and limitations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is highly concise with three sentences, each serving a distinct purpose: stating the function, differentiating from a sibling, and noting access restrictions. The key information is front-loaded, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of a rich input schema, annotations, and an output schema, the description provides sufficient context. It covers purpose, usage differentiation, access constraints, and required parameters. However, it could briefly mention that the output is per-user, but the high schema coverage compensates, so it is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% parameter description coverage, so the baseline is 3. The tool description mentions the required time range but does not add new semantic information beyond what is already in the schema. Hence, it provides no additional parameter insights.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns per-user request and cost analytics for a required time range, using specific verbs ('Return') and resource ('per-user request and cost analytics'). It also distinguishes from the sibling tool get_users_analytics by specifying 'This is usage-by-user, not population metrics'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool (for per-user analytics) and when not to (for population metrics, use get_users_analytics). It also provides access constraint information ('Enterprise-gated. Returns 403 on non-Enterprise Portkey plans'), guiding the agent on proper usage and error handling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_virtual_keyA
Read-onlyIdempotent

Fetch one virtual key by slug, including metadata, a masked secret, limits, status, and model config. Use this before updating or to inspect the current configuration.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe unique slug identifier of the virtual key to retrieve

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool read-only and idempotent. The description adds value by specifying exactly what data is returned (e.g., masked secret, limits), which is not in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no redundancy. First sentence states purpose, second gives usage context. Efficient and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter, full schema coverage, and output schema present, the description is complete. It includes usage guidance and return field highlights.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers the single parameter 'slug' with a description. The description's mention of fetching by slug adds little beyond the schema, achieving the baseline for full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool fetches a virtual key by slug, listing the returned fields (metadata, masked secret, limits, status, model config). It distinguishes itself from sibling getters like get_api_key.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'Use this before updating or to inspect the current configuration.' It does not mention alternatives but gives clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_workspaceA
Read-onlyIdempotent

Get one workspace by id and return its full details, including defaults and the complete member list. Use this when you need membership detail; use list_workspaces for an overview.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesThe unique identifier of the workspace to retrieve. This can be found in the workspace's URL or from the list_workspaces tool response

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint, destructiveHint, idempotentHint, openWorldHint. Description adds that it returns 'full details, including defaults and the complete member list', providing useful context beyond the annotations. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and what it returns, second gives usage guidance. Front-loaded and efficient with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, and description adequately states what is returned (full details including defaults and member list). Sufficient for a simple retrieval tool with one parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 1 parameter with 100% description coverage, so baseline is 3. Description does not add additional parameter semantics beyond what schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Get one workspace by id' with specific verb and resource, and distinguishes from sibling tool list_workspaces by mentioning 'return its full details, including defaults and the complete member list'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Contains explicit guidance: 'Use this when you need membership detail; use list_workspaces for an overview.' Clearly tells when to use this tool vs alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_workspace_memberA
Read-onlyIdempotent

Get one workspace member by workspace_id and user_id. Use this when you already know both IDs; use list_workspace_members to browse the full roster.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesThe user ID to retrieve (must be a valid UUID from list_all_users, not an email address)
workspace_idYesThe workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds a minimal behavioral fact: it returns a single member and requires both IDs. However, it does not disclose additional behavioral traits such as error handling, rate limits, or authentication requirements. With annotations carrying most of the burden, this description adds only marginal transparency beyond them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences: the first states the purpose and the second gives usage guidance. It is front-loaded with the action and resource, contains zero filler, and every word earns its place. This is a model of conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple lookup tool, the description is complete. The input schema fully documents the two required parameters, an output schema exists to define the return value, and annotations cover safety. The description adds the critical context about when to use it versus list_workspace_members. There is no missing information that would prevent correct use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% since both parameters have descriptions. The user_id description is detailed (valid UUID, not email, from list_all_users), while workspace_id's is generic. The tool description restates the parameters but does not add new semantic meaning beyond the schema, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Get one workspace member by workspace_id and user_id.' This specific verb + resource combination distinguishes it from sibling tools like list_workspace_members, add_workspace_member, and remove_workspace_member. It also explicitly names list_workspace_members as the alternative for browsing, reinforcing differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'Use this when you already know both IDs; use list_workspace_members to browse the full roster.' This tells the agent exactly when to use this tool and points to the appropriate alternative, making the choice unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

insert_logA

Insert log records for requests that bypassed the gateway. This writes request, response, and trace metadata into Portkey immediately, and the call will fail if request_provider does not match a configured integration. Use the span fields to stitch trace hierarchies together.

ParametersJSON Schema
NameRequiredDescriptionDefault
request_urlNoThe endpoint URL being called
request_bodyNoRequest payload/body
metadata_userNoUser identifier for the log
response_bodyNoResponse payload/body
response_timeNoResponse latency in milliseconds
request_methodNoHTTP method used (defaults to 'post')post
streaming_modeNoWhether the response was streamed
metadata_customNoAdditional custom metadata key-value pairs
request_headersNoRequest headers as key-value pairs
response_statusNoHTTP response status code (defaults to 200)
metadata_span_idNoSpan ID for tracing
request_providerNoAI provider name (e.g., 'openai', 'anthropic')
response_headersNoResponse headers as key-value pairs
metadata_trace_idNoTrace ID for distributed tracing
metadata_span_nameNoSpan name for tracing
metadata_organizationNoOrganization identifier for the log
metadata_parent_span_idNoParent span ID for tracing

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint=false, destructiveHint=false) indicate a write operation. The description adds that writes are immediate, fail on unmatching provider, and span fields enable trace hierarchy. All behaviors are disclosed without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences front-loaded with purpose and context. Every sentence provides meaningful information without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool complexity (17 parameters) and existing schema/annotations/output schema, the description covers purpose, usage condition, failure case, and tracing guidance. It could optionally mention what the function returns, but output schema likely handles that, so it is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 17 parameters have descriptions in the schema (100% coverage), so baseline is 3. The description adds value by explaining span fields 'stitch trace hierarchies together' and the failure condition for request_provider, which goes beyond the schema's generic descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Insert log records for requests that bypassed the gateway' – a specific verb and resource with context. It distinguishes from sibling tools by focusing on bypassed requests, which is unique among the many create and log tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description specifies when to use (bypassed gateway requests), includes a failure condition ('call will fail if request_provider does not match a configured integration'), and provides guidance on trace stitching. However, it does not explicitly state when not to use or compare to alternative logging methods.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

invite_userA

Invite a new org user and optionally provision workspace access and an API key in one call. Workspace assignments apply only after acceptance; use add_workspace_member or update_workspace_member later for follow-up changes.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesOrganization-level role: 'admin' for full access, 'member' for limited access
emailYesEmail address of the user to invite
last_nameNoUser's last name
first_nameNoUser's first name
workspacesYesList of workspaces and corresponding roles to grant to the user
workspace_api_key_detailsNoOptional API key to be created for the user

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate non-read-only, non-destructive, non-idempotent, and open-world. The description adds behavioral context by noting that workspace assignments are effective only after acceptance, which is not captured in annotations. This is valuable transparency beyond the structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences that cover purpose, optional features, and important behavioral nuance. Every word adds value, with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (6 parameters, nested objects, output schema exists), the description covers the main purpose, optional capabilities, and key behavioral point about delayed workspace assignment. It is sufficiently complete for an AI agent to understand the tool's role, though it could briefly mention that an invitation is sent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already describes all parameters. The description mentions optional provisioning of workspace access and API key, which aligns with the 'workspaces' and 'workspace_api_key_details' parameters but adds little extra meaning. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool invites a new org user and optionally provisions workspace access and an API key. It distinguishes from sibling tools like add_workspace_member by noting that workspace assignments apply only after acceptance, prompting use of separate tools for follow-ups.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states the tool's use case (inviting a new org user) and clarifies when to use alternative tools for workspace changes after acceptance. This provides clear guidance on when and when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_all_usersA
Read-onlyIdempotent

List accepted org users with id, name, email, role, and timestamps. Use this to find a user_id before get_user, update_user, delete_user, or add_workspace_member; use list_user_invites for pending invitations. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive. Description adds key behavioral context: 'Enterprise-gated. Returns 403 on non-Enterprise Portkey plans', which is beyond annotations and critical for agent awareness.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences. First front-loads purpose and resource, then provides usage guidance and constraints. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, return fields, usage guidance, alternatives, and enterprise constraint. Output schema exists, so return format is already detailed. Complete for a list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for both parameters. Description does not add extra parameter details, but baseline is 3 since schema already covers meaning adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'List accepted org users with id, name, email, role, and timestamps', specifying the verb and resource. Distinguishes from sibling tools like list_user_invites and mentions usage context for get_user, update_user, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells when to use: 'find a user_id before get_user, update_user, delete_user, or add_workspace_member' and points to alternative 'use list_user_invites for pending invitations'. Also notes enterprise gating and 403 error on non-Enterprise.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_api_keysA
Read-onlyIdempotent

List Portkey API keys for auditing access, scopes, defaults, limits, and expiration. Use this for API keys only; use list_virtual_keys for provider keys. Returns total plus id, type, status, workspace/user scope, limits, defaults, alert emails, and creation mode.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination
workspace_idNoFilter by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the basic safety is covered. The description adds value by detailing the kind of data returned (scopes, limits, expiration), which helps set expectations beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, and the second sentence provides differentiation and return summary without any wasted words. Every sentence serves a clear function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of a full output schema and annotations covering safety, the description succinctly covers the tool's purpose, differentiation, and the key aspects of its return data. No gaps remain for a competent agent to use this tool effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with all parameters documented. The description does not add new information about the parameters themselves, but it mentions the return data types (e.g., workspace/user scope) that indirectly relate to filtering via workspace_id. This meets the baseline expectation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists Portkey API keys for auditing, specifying the exact resource and purpose. It also distinguishes itself from the sibling tool list_virtual_keys, which is for provider keys.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool ('for API keys only') and provides a clear alternative ('use list_virtual_keys for provider keys'), giving the agent a direct comparison.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_audit_logsA
Read-onlyIdempotent

List audit log events for a Portkey workspace or organization. Returns paginated action-level records with actor, resource, metadata, and timestamps for compliance or incident review; use this instead of analytics when you need individual events, not aggregates. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionNoFilter by action type (e.g., 'create', 'update', 'delete', 'login')
actor_idNoFilter by the user ID who performed the action
end_timeNoEnd of time range filter (ISO 8601 format, e.g., '2024-01-31T23:59:59Z')
page_sizeNoNumber of results per page (max 100)
start_timeNoStart of time range filter (ISO 8601 format, e.g., '2024-01-01T00:00:00Z')
resource_idNoFilter by specific resource ID
current_pageNoPage number for pagination (starts at 1)
workspace_idNoFilter audit logs by workspace ID
resource_typeNoFilter by resource type (e.g., 'user', 'workspace', 'config', 'virtual_key')

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses pagination, return fields (actor, resource, metadata, timestamps), and enterprise restriction, adding value beyond annotations which already indicate read-only, idempotent, non-destructive behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with core purpose, followed by usage differentiation and limitation—every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given comprehensive parameters, annotations, and output schema, the description covers purpose, usage context, and limitations completely, leaving no critical gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with self-describing parameter names and types; description does not add parameter-level details but context about return fields aids understanding. Baseline score appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'List audit log events' with specific resource scope, and explicitly distinguishes from analytics tools by mentioning individual events vs aggregates.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use guidance ('use this instead of analytics when you need individual events, not aggregates') and notes enterprise-gating with 403 error for non-Enterprise plans.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_collectionsA
Read-onlyIdempotent

List prompt collections in the workspace, optionally filtering by name or workspace. Returns ids, names, slugs, and timestamps so you can choose a collection_id before create_prompt, get_collection, or list_prompts.

ParametersJSON Schema
NameRequiredDescriptionDefault
searchNoSearch collections by name
page_sizeNoResults per page (max 100)
current_pageNoPage number for pagination
workspace_idNoFilter by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnly, openWorld, idempotent, and non-destructive. The description adds value by specifying the return fields (ids, names, slugs, timestamps) and filtering options, which is useful additional context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise: two sentences that cover purpose, filtering, return fields, and usage context with no redundant words. Front-loaded with the main action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only list tool with 4 optional parameters and an output schema (not shown but mentioned), the description adequately conveys the return fields and intended use case, making it complete for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description mentions filtering by name or workspace, which aligns with the search and workspace_id parameters, but does not add new information beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'list', resource 'prompt collections', scope 'in the workspace', and optional filters. It also distinguishes from sibling tools by mentioning it returns collection_id for use with create_prompt, get_collection, or list_prompts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool: to obtain a collection_id before creating prompts or listing prompts. While it doesn't state when not to use it or name alternatives, the context is sufficient for an agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_configsA
Read-onlyIdempotent

List configs in the org with id, slug, name, status, workspace, and timestamps. Use this summary view to find a slug; use get_config for the full routing, cache, retry, and target settings before updating or deleting.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds that it returns a summary view with specific fields, which is useful context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first presents purpose and output, second provides usage guidance. No wasted words, front-loaded, and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given annotations, schema, and output schema, the description is complete. It covers the tool's functionality and workflow context without missing essential information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline 3 is appropriate. The description does not add meaning beyond the schema for the two optional pagination parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists configs with specific fields (id, slug, name, status, workspace, timestamps). It also distinguishes from get_config by noting that this is a summary view.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance: use this to find a slug, then use get_config for full settings before updating/deleting. This clearly indicates when to use this tool vs alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_config_versionsA
Read-onlyIdempotent

List every version of a config with version number, config payload, creator, and timestamp. Use this to audit history or compare revisions before update_config or delete_config.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesConfiguration slug to list versions for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the description carries less burden. It adds the fields returned but does not disclose potential behavioral details like pagination, ordering, or whether all versions are returned at once.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first sentence states what it does, second provides usage guidance. Every sentence earns its place, no fluff, front-loaded with the most important information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given one parameter, read-only operation, and the presence of an output schema (not shown but indicated), the description covers the key aspects: what is listed (versions with fields) and why to use it (audit, compare). It is sufficient for an agent to decide when to invoke this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage with parameter 'slug' described as 'Configuration slug to list versions for'. The tool description does not add additional parameter meaning beyond what is in the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool lists every version of a config with specific fields (version number, config payload, creator, timestamp). It distinguishes itself from siblings like get_config (single version) and list_configs (list configs) by focusing on listing versions of a single config.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides when to use: 'Use this to audit history or compare revisions before update_config or delete_config.' This gives clear guidance on context and alternative operations without being verbose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_guardrailsA
Read-onlyIdempotent

List guardrails in the org with id, slug, status, ownership, and optional workspace/org filters. Use this to find IDs and slugs before get_guardrail, update_guardrail, or delete_guardrail.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of items per page (1-100, default: 100)
current_pageNoPage number for pagination
workspace_idNoFilter guardrails by workspace ID
organisation_idNoFilter guardrails by organization ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint, idempotentHint, destructiveHint) already cover safety traits. The description adds no new behavioral details beyond stating it lists guardrails with optional filters, which is consistent but not additional disclosure. Moderate value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences, no wasted words. The first sentence states purpose and fields, the second provides usage guidance. Clearly front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given comprehensive annotations, full schema coverage, and presence of an output schema, the description is complete. It specifies output fields, filters, and usage pattern, leaving no essential gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with all parameter descriptions provided. The description mentions optional workspace/org filters but does not add meaning beyond what the schema already conveys. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List guardrails in the org' and mentions specific fields (id, slug, status, ownership) and optional filters. It distinguishes from siblings by advising use before get, update, or delete operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises 'Use this to find IDs and slugs before get_guardrail, update_guardrail, or delete_guardrail,' providing clear when-to-use guidance and implicit when-not-to-use (when you need a single guardrail, use get_guardrail).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_input_guardrail_workspace_exclusionsA
Read-onlyIdempotent

List workspaces excluded from organisation-wide input guardrails for one organisation. Use this to audit exceptions or establish the current state before the matching update tool; it reads input exclusions only and does not return the organisation's default guardrail list. Requires an organisation service API key with organisation_exclusions.list scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
organisation_idYesOrganisation ID whose workspace exclusions should be listed

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only/idempotent/destructive hints. The description adds valuable context about required API key scope, Enterprise-gated availability, and the 403 behavior on non-Enterprise plans, which goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, all directly informative, with front-loaded primary action. No redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only list tool with an output schema, the description covers purpose, usage, auth, enterprise gating, and returns meaningful constraints. It is fully sufficient for an agent to select and invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter, so the schema fully documents 'organisation_id'. The description adds no additional parameter-level detail, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists workspaces excluded from organisation-wide input guardrails for one organisation, with a specific verb and resource. It also distinguishes itself by noting it only reads input exclusions, not the default guardrail list, which differentiates it from sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: to audit exceptions or establish current state before the matching update tool. It also clarifies what it does not do (does not return default guardrail list), providing clear context versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_integration_modelsA
Read-onlyIdempotent

List models enabled on an integration. Use this to verify model availability before creating prompts or configs. Returns total plus model ids, display names, enabled state, and custom-model markers. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
page_sizeNoNumber of results per page
current_pageNoPage number for pagination

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint and idempotentHint. The description adds value by detailing the returned fields (total, model ids, display names, enabled state, custom-model markers) and the enterprise-gating restriction. It does not discuss pagination behavior (page_size/current_page parameters) but the schema covers pagination and output schema likely defines structure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is highly concise, consisting of five sentences each serving a purpose: stating the action, providing a use case, listing return fields, noting enterprise restriction, and specifying error behavior. No superfluous words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has a good annotation set, a full schema, and an output schema, the description adequately explains the purpose, usage context, return fields, and error case. It could mention pagination behavior or ordering, but the output schema likely covers the structure. Overall, it is sufficiently complete for an agent to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all three parameters. The description does not provide additional context beyond what the schema already offers for the slug, page_size, or current_page parameters. Therefore, it adds minimal semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb 'List' and resource 'models enabled on an integration', clearly distinguishing it from sibling tools like list_integrations (which lists integrations) and update_integration_models (which modifies models). It also provides a concrete use case: 'verify model availability before creating prompts or configs'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description specifies when to use the tool ('before creating prompts or configs') and mentions the enterprise limitation with a 403 error for non-Enterprise plans, indicating when it may not work. However, it does not explicitly compare to alternative tools like update_integration_models for modification, though this is implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_integrationsA
Read-onlyIdempotent

List org-level AI provider connections with optional workspace or type filters. Use this to find integration slugs before model or workspace updates. Returns total plus id, name, slug, provider, status, description, workspace counts, and config summary.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoFilter by integration type: 'workspace', 'organisation', or 'all' (default)
page_sizeNoNumber of results per page (default 100, max 100)
current_pageNoPage number for pagination
workspace_idNoFilter integrations accessible by a specific workspace

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readonly, idempotent, and non-destructive behavior. The description adds context on return fields (id, name, slug, provider, status, etc.) but doesn't elaborate on pagination details or authentication requirements, which are partially covered by the schema. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences that are front-loaded with purpose and filters, followed by a use case and return information. Every word adds value, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich annotations, complete schema, and presence of an output schema, the description effectively covers purpose, parameters, return fields, and usage context. Minor omission: no mention of pagination behavior beyond schema, but that is documented elsewhere.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description references 'optional workspace or type filters,' which maps to the 'workspace_id' and 'type' parameters, but adds no new semantic meaning beyond what the schema already provides. No compensation needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List org-level AI provider connections with optional workspace or type filters' – a specific verb and resource. It further differentiates from siblings like 'list_integration_models' and 'list_integration_workspaces' by indicating it returns integration details and slugs, which are prerequisites for updates.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises 'Use this to find integration slugs before model or workspace updates,' providing a clear use case. While it doesn't list alternatives or when not to use it, the context implies this is the correct tool for discovering integrations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_integration_workspacesA
Read-onlyIdempotent

List workspaces that can use an integration, with their limits. Use this to audit access or confirm per-workspace cost and rate settings. Returns total plus workspace ids, names, enabled state, usage limits, and rate limits. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
page_sizeNoNumber of results per page
current_pageNoPage number for pagination

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint false. Description adds that it returns total plus specific fields (ids, names, enabled state, usage limits, rate limits), and enterprise restriction with 403. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences covering purpose, use case, and return value plus constraints. No fluff, front-loaded with main action. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given output schema exists, description covers return fields, enterprise gating, and error condition. Adequate for a paginated list tool with read-only behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 3 parameters are described in the schema (slug, page_size, current_page) with 100% coverage. Description adds no extra semantic meaning beyond the schema. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'List workspaces that can use an integration, with their limits.' Verb+resource is specific. Differentiates from siblings like list_mcp_integration_workspaces by focusing on integrations (not MCP). Also adds audit use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use ('audit access or confirm per-workspace cost and rate settings'). Also notes Enterprise-gating and 403 error. Does not explicitly exclude alternatives, but clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_log_exportsA
Read-onlyIdempotent

List log export jobs in a workspace with status, filters, and timestamps. Use this to find an export_id before calling get_log_export, start_log_export, cancel_log_export, or download_log_export. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesWorkspace ID to list exports for (required)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate this is read-only and idempotent, so the description's additional disclosure of the Enterprise-gating and 403 error behavior adds value. However, it doesn't mention pagination or other behavioral nuances that might matter for listing jobs, though with output schema present this is less critical.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured: the first sentence defines the action, the second provides usage context, and the third states a critical limitation. No wasted words; every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with a simple single-parameter schema and an output schema, the description covers the core purpose, usage, and error condition. The only minor gap is not explicitly mentioning pagination or ordering, but the output schema likely covers return structure. Overall complete for its complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides 100% coverage for the single parameter, workspace_id, with a clear description. The tool description merely reiterates 'in a workspace' without adding further meaning beyond the schema, so it meets the baseline but doesn't exceed it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists log export jobs in a workspace with status, filters, and timestamps. It distinguishes itself from related tools by explicitly positioning it as the way to find an export_id before using other log export operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance: 'Use this to find an export_id before calling get_log_export, start_log_export, cancel_log_export, or download_log_export.' It also discloses a restriction (Enterprise-gated, returns 403 on non-Enterprise plans), which helps the agent decide if this tool is appropriate in context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_integration_capabilitiesA
Read-onlyIdempotent

List capabilities exposed by the external MCP server for an integration. Returns total plus enabled-state entries so you can decide what to toggle; use before update_mcp_integration_capabilities when you need to compare the current surface.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool as read-only (readOnlyHint: true), non-destructive, idempotent, and open-world. The description adds value by specifying that it returns 'total plus enabled-state entries' and positions it as a preparatory step for toggling. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with the primary purpose, and no redundant information. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple listing tool with one parameter and an assumed output schema, the description provides the purpose, return summary, and usage context (preparation for update). It is complete enough for an agent to understand when and how to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single parameter 'id', and its description is adequate. The tool description does not add extra parameter details, but since the schema already covers it, a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List', the resource 'capabilities exposed by the external MCP server for an integration', and distinguishes it from siblings like 'update_mcp_integration_capabilities' and 'list_mcp_integrations'. It also specifies the return content (total plus enabled-state).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to 'use before update_mcp_integration_capabilities when you need to compare the current surface', providing clear context and an alternative. This helps the agent decide when to invoke this tool versus the update sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_integrationsA
Read-onlyIdempotent

List MCP integrations in the organization. Returns paginated integration records plus total and has_more for discovering integration IDs; use get_mcp_integration for one integration's full Portkey-side config and list_mcp_servers for the servers under an integration.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination
workspace_idNoFilter by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is clear. The description adds behavioral details about paginated results (total, has_more) and the purpose of discovering IDs, which is valuable context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences. The first states the core function, the second provides return structure and sibling tool references. No extraneous words, well-organized.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of an output schema and full parameter descriptions, the description covers all necessary context: what the tool does, what it returns (paginated records with total/has_more), and how to get more detail from siblings. Complete for a list endpoint.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with all three parameters having descriptions. The description does not add significant new parameter information, as the schema already documents page_size, current_page, and workspace_id adequately. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists MCP integrations in the organization and distinguishes it from siblings like get_mcp_integration and list_mcp_servers, which are explicitly named for specific use cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool (to discover integration IDs via pagination) and when to use alternatives (get_mcp_integration for full config, list_mcp_servers for servers under an integration).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_integration_workspacesA
Read-onlyIdempotent

List which workspaces can access an MCP integration. Returns the global access mode plus per-workspace enablement for audit or permission review; use before update_mcp_integration_workspaces.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only and idempotent behavior. The description adds valuable context by stating that it returns global access mode and per-workspace enablement, and that it should be used before updates. This goes beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences that front-load the purpose and then add return details and usage guidance. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite the presence of an output schema, the description explains the return contents (global access mode and per-workspace enablement), the use case (audit/permission review), and ties to the sibling update tool. For a simple list tool with one parameter, this is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema fully describes the single parameter 'id' with a clear description. The tool description does not add new meaning beyond referencing the integration context. With 100% schema coverage, baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action ('List which workspaces can access an MCP integration') and specifies the return content (global access mode plus per-workspace enablement). It also distinguishes from the sibling update tool by explicitly recommending use before update_mcp_integration_workspaces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance: use for audit/permission review and before an update. However, it does not differentiate from the similar sibling tool 'list_integration_workspaces', which could cause confusion. The guidance is good but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_server_capabilitiesA
Read-onlyIdempotent

List capabilities exposed by an MCP server instance. Returns total plus the current tool, resource, and prompt surface; use this instead of the integration-level capability list when you need server-specific exposure.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint, idempotentHint, and destructiveHint false. The description adds value by detailing the returned content (total, tool, resource, prompt surface) but does not explicitly mention pagination behavior. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the primary action, and every word adds value. It is concise without any fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has an output schema and annotations cover safety, the description covers purpose, return content, and usage guidance well. Minor missing detail on pagination behavior prevents a perfect score, but overall it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with each parameter described. The description adds marginal value by hinting at pagination via 'total' and mentioning returned fields, but it does not elaborate on parameter syntax or constraints beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists capabilities exposed by an MCP server instance, specifying it returns total plus tool, resource, and prompt surface. It explicitly distinguishes itself from the sibling integration-level capability list, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'use this instead of the integration-level capability list when you need server-specific exposure.' This clarifies when to use this tool and names the alternative (list_mcp_integration_capabilities), offering clear usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_server_connectionsA
Read-onlyIdempotent

List active connection records for one Portkey-managed MCP server, including user, connected state, and connection timestamps. Use it to audit sessions or identify the user/workspace before disconnect_mcp_server_connection; service keys can list all users, while user keys default to their own user. Organisation admin keys must provide workspace_id. This reads live connection state and does not test upstream server reachability (use test_mcp_server for that).

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesMCP server ID or slug whose connections to list
user_idNoUser UUID to filter; user-scoped keys default to their own user when omitted
page_sizeNoConnections per page, from 1 through 500; defaults to 100
current_pageNoZero-based results page; defaults to 0
workspace_idNoWorkspace ID or slug; required with an organisation admin API key

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark it read-only, idempotent, and non-destructive, but the description adds useful behavioral details: it reads live connection state, lists fields returned, and explicitly excludes upstream reachability testing. It also explains permission scoping for different API key types, which goes beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core action, then covers use cases, access control, and limitations in just three sentences. Every sentence contributes unique value with no repetition of schema details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists and annotations convey safety, the description is complete: it covers what is returned, when to use it, who can use it with which key types, and what it does not do. There are no significant gaps for an agent to select and invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% parameter coverage with defaults and filtering semantics. The description adds meaning beyond the schema by explaining that service keys can list all users, user keys default to their own user, and org admin keys must provide workspace_id—this supplements the raw schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'List active connection records for one Portkey-managed MCP server,' including user, connected state, and timestamps. This distinguishes it from sibling tools like list_mcp_servers and test_mcp_server, which cover different server details or connectivity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states when to use the tool ('audit sessions or identify the user/workspace before disconnect_mcp_server_connection') and provides an alternative for a different need: 'does not test upstream server reachability (use test_mcp_server for that).' It also clarifies key-scope behavior (service keys vs. user keys, org admin workspace_id requirement).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_serversA
Read-onlyIdempotent

List MCP servers in the organization. Returns paginated server records plus total for discovering server IDs; use get_mcp_server for one server's details and list_mcp_integrations for the parent integration.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination
workspace_idNoFilter by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds value by disclosing pagination and total count behavior, enhancing transparency beyond structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with front-loaded purpose ('List MCP servers in the organization'), no redundant or extraneous content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given annotations and output schema coverage, the description fully covers key aspects: purpose, pagination, sibling differentiation, and usage context, leaving minimal gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline 3. Description does not add parameter-specific details beyond schema, but implies pagination and filtering context, maintaining adequacy.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'List MCP servers in the organization' with a specific verb and resource. It explicitly distinguishes from siblings by directing to get_mcp_server for details and list_mcp_integrations for parent integration, ensuring no confusion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Returns paginated server records plus total for discovering server IDs' and contrasts with alternatives for one server or parent integration, covering when and when not to use the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_server_user_accessA
Read-onlyIdempotent

List per-user access for an MCP server. Returns the default access mode, override flags, and connection status so you can audit who can use it; use before update_mcp_server_user_access.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, etc. The description adds value by specifying that the tool returns audit-relevant fields (default access mode, override flags, connection status) and its role as a precursor to updates.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences efficiently convey action, resource, return data, and usage guidance without superfluous words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's purpose, return content, and usage hint. Pagination is handled by parameters. With an output schema present, the description is adequately complete for a list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema fully describes parameters. The description does not add extra meaning beyond implying the 'id' is an MCP server identifier. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'List per-user access for an MCP server' with a specific verb and resource. It lists returned data (default access mode, override flags, connection status) and distinguishes from siblings by mentioning audit and the update tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly says 'use before update_mcp_server_user_access,' providing explicit when-to-use context. It does not list exclusions or alternatives, but the read-only nature of the tool implies appropriate usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_output_guardrail_workspace_exclusionsA
Read-onlyIdempotent

List workspaces excluded from organisation-wide output guardrails for one organisation. Use this to audit exceptions or establish the current state before the matching update tool; it reads output exclusions only and does not return the organisation's default guardrail list. Requires an organisation service API key with organisation_exclusions.list scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
organisation_idYesOrganisation ID whose workspace exclusions should be listed

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds valuable behavioral context: it 'reads output exclusions only', states the required API key scope, emphasizes enterprise gating, and documents the 403 error on non-Enterprise plans. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each carrying substantive information (purpose, usage context, permissions/error behavior). No redundant phrasing. Well-structured and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity, the description covers purpose, usage scenarios, exclusions vs. defaults, authentication scope, enterprise restriction, and error condition. An output schema exists, so return values need no further explanation. Description is complete for tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a clear description for organisation_id. The description reinforces the parameter by saying 'for one organisation', but does not add significant new semantic details beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List workspaces excluded'), the resource ('organisation-wide output guardrails'), and the scope ('for one organisation'). It distinguishes from the sibling tool list_input_guardrail_workspace_exclusions by explicitly specifying 'output guardrails' and from the matching update tool by using 'list' and 'reads'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Use this to audit exceptions or establish the current state before the matching update tool'. It also clarifies what the tool does not return ('does not return the organisation's default guardrail list'), differentiating it from get_organisation_defaults, and notes enterprise gating and 403 behavior.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_partial_versionsA
Read-onlyIdempotent

List all versions for one partial, including version numbers, descriptions, status, and timestamps. Use this when you need history or want to choose a version_id before publish_partial.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_partial_idYesPrompt partial ID or slug to list versions for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, non-destructive, idempotent, and openWorld. Description adds that the tool returns specific fields (version numbers, descriptions, status, timestamps), giving extra clarity on output content beyond the output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose and output, second specifies usage scenario. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With one simple parameter, full schema coverage, output schema present, annotations covering safety, and a clear link to a sibling tool (publish_partial), the description is complete and sufficient for an agent to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% coverage with clear parameter description. Description does not add significant extra meaning beyond the schema, but the mention of 'version_id' in context provides a slight connection.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it lists all versions for one partial, including specific fields (version numbers, descriptions, status, timestamps). It implicitly differentiates from sibling tools like list_prompt_versions by focusing on partials.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use this when you need history or want to choose a version_id before publish_partial,' providing clear when-to-use context and linking to a related action.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_prompt_labelsA
Read-onlyIdempotent

List labels across the workspace or organisation, with optional search and scope filters. Returns ids, names, colors, status, and timestamps so you can choose a label_id before get_prompt_label or update_prompt_version.

ParametersJSON Schema
NameRequiredDescriptionDefault
searchNoSearch labels by name
page_sizeNoResults per page (max 100)
current_pageNoPage number for pagination
workspace_idNoFilter by workspace ID
organisation_idNoFilter by organisation ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, destructiveHint=false, fully disclosing the safe, read-only, idempotent nature. The description adds the specific return fields (ids, names, colors, status, timestamps) and the purpose for downstream usage, but does not reveal behavioral traits beyond what annotations already convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and scope, second lists return fields and links to related tools. No extraneous words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Description covers purpose, use case (choosing label_id), return fields, and optional filters. With comprehensive annotations (readOnly etc.) and a fully described schema, the description is complete and informative for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with all 5 parameters already described in the input schema. The description only adds 'optional search and scope filters' as a summary, which does not materially enhance understanding of individual parameters. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the verb 'List' and resource 'labels', specifies scope (workspace/organisation) and optional filters. It explicitly distinguishes from sibling tools like get_prompt_label and update_prompt_version by explaining the return fields and downstream use.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description provides clear context: it can be used to obtain a label_id for subsequent operations (get_prompt_label, update_prompt_version). It mentions optional search and scope filters but does not explicitly exclude alternatives or provide when-not-to-use scenarios, which is acceptable for a list tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_prompt_partialsA
Read-onlyIdempotent

List partials across collections, with optional collection filtering. Returns ids, slugs, names, collections, and status so you can choose a prompt_partial_id before get_prompt_partial, update_prompt_partial, delete_prompt_partial, or publish_partial.

ParametersJSON Schema
NameRequiredDescriptionDefault
collection_idNoFilter by collection ID. Optional — omit to list all partials across collections

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds value by specifying returned fields and purpose, but does not disclose additional behavioral traits beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (two sentences) and front-loaded with the action. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (1 optional param, annotations present, output schema exists), the description covers the return fields and purpose adequately for agent invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the parameter description is already clear. The description reiterates the optional parameter usage but does not add new semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it lists partials across collections with optional filtering, and mentions return fields. It distinguishes from sibling tools like get_prompt_partial and update_prompt_partial by indicating its role as a list operation for selection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description explains when to use (to list partials and choose an ID for further actions) and mentions optional collection filtering. It doesn't explicitly state when not to use, but provides clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_promptsA
Read-onlyIdempotent

List prompts across the workspace, with optional collection, workspace, or search filters. Returns a paginated summary with id, name, slug, model, and status so you can choose a prompt_id before get_prompt, update_prompt, or render_prompt.

ParametersJSON Schema
NameRequiredDescriptionDefault
searchNoSearch prompts by name
page_sizeNoResults per page (max 100)
current_pageNoPage number for pagination
workspace_idNoFilter by workspace ID
collection_idNoFilter by collection ID (recommended for app-specific prompts)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate safe, read-only, idempotent behavior. The description adds value by specifying the return fields (pagination summary with id, name, slug, model, status) and the intended use case, without contradicting annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences that are front-loaded with purpose and filters, then return format and use. Every sentence adds value; no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (5 optional params, output schema present), the description covers scope, filters, return fields, and use case. It does not mention sorting or default pagination, but output schema and API conventions likely cover these.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with good parameter descriptions. The description groups filters ('optional collection, workspace, or search filters') to provide context, though it doesn't mention page_size or current_page. This adds understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly specifies the verb 'list' and resource 'prompts' across the workspace, with optional filters. It distinguishes from sibling tools like get_prompt, update_prompt, and render_prompt by stating the purpose: to choose a prompt_id for subsequent operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool (listing prompts with optional filters) and implies that for a single prompt by ID, one would use get_prompt. However, it lacks explicit exclusions or alternative recommendations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_prompt_versionsA
Read-onlyIdempotent

List all versions of one prompt, including version number, description, status, label, and a short template preview. Use this for history or to choose a version_id before publish_prompt or update_prompt_version.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug to list versions for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint, destructiveHint, idempotentHint). Description adds that it returns specific fields but does not mention ordering or limits. Adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and output, second gives usage guidance. No unnecessary words, well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With output schema present, description appropriately lists return fields. Usage guidance is complete for a simple list tool. No gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% coverage with description for prompt_id. Description does not add extra semantics beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists all versions of one prompt, specifying the fields returned (version number, description, status, label, template preview). It distinguishes from siblings like list_prompts and get_prompt_version.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly suggests use for history or choosing a version_id before publish_prompt or update_prompt_version. Provides clear context but does not explicitly exclude alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_providersA
Read-onlyIdempotent

List workspace-scoped provider instances and their limits or status. Use this to find provider slugs for workspace-level updates; use list_integrations for the org-level source connection. Returns total plus provider name, slug, integration, status, limits, expiration, and reset flags.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100, default 50)
current_pageNoPage number for pagination
workspace_idNoWorkspace ID - required when using organization admin keys, optional with workspace API keys

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the behavioral profile is clear. The description adds value by listing the return fields (total, provider name, slug, integration, status, limits, expiration, reset flags), which provides context beyond annotations without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no fluff: first sentence states the action and scope, second gives usage guidance and return field enumeration. Information is front-loaded and every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (3 parameters, annotations present, output schema exists), the description covers purpose, usage guidance, and return fields completely. Pagination details are already in the schema, so no further elaboration needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with each parameter described (page_size, current_page, workspace_id). The description does not add additional parameter-specific guidance beyond what the schema provides, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'List workspace-scoped provider instances and their limits or status,' which is a specific verb+resource+scope. It also distinguishes from the sibling tool list_integrations by noting the different scopes (workspace vs org-level).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool ('to find provider slugs for workspace-level updates') and when to use an alternative ('use list_integrations for the org-level source connection'). This provides clear context and exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_rate_limitsA
Read-onlyIdempotent

List rate limits in the org with id, type, unit, value, status, scope, conditions, and grouping. Use this to find an existing policy before get_rate_limit, update_rate_limit, or delete_rate_limit.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idNoFilter rate limits by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds value by listing returned fields and suggesting it's a discovery tool, though it could mention pagination or ordering if applicable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, first lists fields, second provides usage guidance. No wasted words, front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given one optional parameter, existing output schema, and thorough annotations, the description is complete enough: it explains what is listed and when to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers the single parameter workspace_id at 100%. Description does not add extra meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists rate limits with specific fields (id, type, unit, value, status, scope, conditions, grouping), differentiating it from sibling tools like get_rate_limit (single) or update_rate_limit (mutate).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to use this tool before get_rate_limit, update_rate_limit, or delete_rate_limit, providing a clear workflow and context for when to choose this over alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_scim_groupsA
Read-onlyIdempotent

Search and page through identity-provider groups synchronized to Portkey over SCIM. Use this to resolve a group ID or exact display name before create_scim_workspace_mapping; it reads directory groups only and does not show their workspace mappings or individual members. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoZero-based results page to retrieve; the first page is 0. Unlike current_page used by other list tools in this API (which is 1-based), do not pass 1 for the first page here.
searchNoCase-insensitive text to match against SCIM group names
page_sizeNoSCIM groups per page, from 1 through 100

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite annotations already declaring readOnlyHint, openWorldHint, idempotentHint, and destructiveHint, the description adds valuable context: the tool is Enterprise-gated and returns 403 on non-Enterprise plans. It also clarifies scope limitations (directory groups only, no mappings/members), which go beyond annotation data. No contradiction exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences: purpose, usage guidance, and behavioral/entitlement notes. Every sentence adds distinct value, and the most important information (what it does) is front-loaded. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need no explanation. The description covers purpose, usage context, scope limitations, and error behavior for a 3-parameter read-only tool, making it fully complete for an agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with each parameter (page, search, page_size) already having meaningful descriptions. The tool description does not add additional parameter semantics, but the schema carries the burden effectively, meeting the baseline for high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Search and page through identity-provider groups synchronized to Portkey over SCIM.' It clearly distinguishes itself from sibling tools by noting it 'reads directory groups only and does not show their workspace mappings or individual members,' which separates it from list_scim_workspace_mappings and related tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs when to use it: 'Use this to resolve a group ID or exact display name before create_scim_workspace_mapping.' It also states what it does not do (show workspace mappings or members), offering clear exclusionary guidance. However, it does not explicitly name an alternative tool for those excluded cases, so it falls just short of full alternative specification.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_scim_workspace_mappingsA
Read-onlyIdempotent

List identity-provider SCIM group mappings that automatically grant Portkey workspace roles. Use it to audit provisioned access or obtain mapping_id before delete_scim_workspace_mapping; filter by workspace, group, or role and page through large directories. This reads mappings only and does not query individual workspace members. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoZero-based results page to retrieve; the first page is 0. Unlike current_page used by other list tools in this API (which is 1-based), do not pass 1 for the first page here.
roleNoReturn only mappings that grant this workspace role
page_sizeNoMappings per page, from 1 through 100
workspace_idNoReturn only mappings for this Portkey workspace ID
scim_group_idNoReturn only mappings for this identity-provider SCIM group ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds meaningful behavior beyond annotations: clarifies the read scope ('reads mappings only'), discloses enterprise gating and the 403 error on non-Enterprise plans, and notes pagination for large directories. No contradiction with readOnlyHint or idempotentHint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise, information-dense sentences. Front-loaded with purpose, followed by usage guidance and behavioral caveats. No redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, usage triggers, exclusions, authentication requirements, error behavior, filtering, and pagination. With an output schema present, return values need no explanation. The tool is fully contextualized for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description mentions filtering by workspace, group, or role and pagination, but the schema already provides detailed parameter descriptions. No additional semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'List identity-provider SCIM group mappings that automatically grant Portkey workspace roles.' It clearly distinguishes from sibling tools like list_scim_groups by specifying 'mappings' and the role-granting behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: 'audit provisioned access or obtain mapping_id before delete_scim_workspace_mapping.' It also clarifies what it does not do ('does not query individual workspace members') and mentions enterprise gating, giving clear context versus alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_secret_referencesA
Read-onlyIdempotent

List Secret References without returning authentication configuration or resolved secret values. Filter by manager type, tags, or name and use a returned UUID or slug with get_secret_reference.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoReturn references matching these tags
searchNoSearch references by name
page_sizeNoReferences per page, from 1 to 100; defaults to 20
current_pageNoZero-based page number; defaults to 0
manager_typeNoFilter by external secret manager: aws_sm, azure_kv, or hashicorp_vault

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false; the description adds that it does not return authentication configuration or resolved secret values, which are important behavioral traits beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose and key constraint, no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations, output schema, and parameter descriptions, the description completes the picture by clarifying what is not returned and how to proceed for more detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with parameter descriptions. The description reinforces filtering by manager type, tags, or name but adds minimal extra meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the tool lists secret references without returning sensitive data (auth config or resolved values), and distinguishes itself from get_secret_reference by mentioning use of UUID/slug for detailed retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says what the tool does not return and suggests using get_secret_reference for full details. Provides filtering options (manager type, tags, name) but does not explicitly state when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_usage_limit_entitiesA
Read-onlyIdempotent

List the entities currently tracked against a usage limit, including current usage. Use this to see who is near or over budget before reset_usage_limit_entity or delete_usage_limit.

ParametersJSON Schema
NameRequiredDescriptionDefault
limit_idYesUsage limit policy ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds 'including current usage' as a brief output detail but does not elaborate on behavior beyond what annotations convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose, second gives usage guideline. No unnecessary words, well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose and usage context. With an output schema present, it is mostly complete, though could clarify what 'entities' refers to (e.g., users, workspaces) or mention pagination.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with one parameter (limit_id) described as 'Usage limit policy ID'. The description does not add additional semantics or examples beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists entities tracked against a usage limit, with current usage. It distinguishes from siblings like list_usage_limits (lists limits, not entities) and names related tools (reset_usage_limit_entity, delete_usage_limit).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use: 'Use this to see who is near or over budget before reset_usage_limit_entity or delete_usage_limit.' Provides clear context for usage and avoids ambiguity with sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_usage_limitsA
Read-onlyIdempotent

List usage limits in the org with id, type, credit_limit, status, reset schedule, scope, conditions, and grouping. Use this before get_usage_limit, update_usage_limit, or delete_usage_limit.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idNoFilter usage limits by workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds the list of returned fields but no extra behavioral context like pagination or error cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence plus a usage hint, no fluff, and front-loads the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists and the tool is simple (one optional param), the description fully covers necessary context for an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has only one optional parameter (workspace_id) with a clear description. The tool description adds no additional meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists usage limits with specific fields (id, type, credit_limit, etc.), distinguishing it from sibling tools like get_usage_limit or update_usage_limit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises using this before get_usage_limit, update_usage_limit, or delete_usage_limit. However, it does not mention the sibling list_usage_limit_entities tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_user_invitesA
Read-onlyIdempotent

List pending and sent invitations with id, email, role, status, and expiry. Use this to check invite state; use list_all_users for users who already accepted. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only, idempotent, non-destructive behavior. The description adds layering context (returns 403 on non-Enterprise plans) and scope (only pending/sent invites).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences. First sentence states purpose and output fields, second gives usage guidance, third notes access restriction. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return value documentation is unnecessary. The description covers purpose, usage context, and access restriction adequately for a list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for both parameters (page_size, current_page). The description adds no additional parameter details beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists pending and sent invitations with specific fields. It differentiates from the sibling tool 'list_all_users' by explicitly stating to use that for accepted users.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance: 'Use this to check invite state; use list_all_users for users who already accepted.' Also notes enterprise gating and 403 response.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_virtual_keysA
Read-onlyIdempotent

List provider API keys stored as virtual keys in your Portkey org. Use this to find slugs before wiring prompts/configs or auditing limits. Returns total plus name, slug, status, usage limits, rate limits, reset state, and model config.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of results per page (max 100)
current_pageNoPage number for pagination

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as a safe, read-only, non-destructive, idempotent operation. The description adds value by listing the return fields (total, name, slug, status, usage limits, etc.), but does not disclose additional behavioral traits like pagination behavior beyond the schema parameters or authentication requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first specifies the action and resource, the second provides usage guidance and return details. No unnecessary words, every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with complete schema coverage and an output schema, the description adequately covers purpose, usage context, and key return information. It is sufficient for an agent to understand when and how to use it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already fully documents the two parameters (page_size, current_page). The description does not add meaning beyond the schema's descriptions, qualifying for the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists provider API keys stored as virtual keys, specifies the context (Portkey org), and directly addresses a common use case (finding slugs). It effectively distinguishes itself from sibling tools like list_api_keys or get_virtual_key by naming the resource type and purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises when to use the tool ('to find slugs before wiring prompts/configs or auditing limits'). It provides clear context but does not explicitly state when not to use it or name alternative tools for different scenarios (e.g., retrieving a single key).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_workspace_membersA
Read-onlyIdempotent

List every member in a workspace with organization role, workspace role, status, and timestamps. Use this to find a user_id before get_workspace_member, update_workspace_member, or remove_workspace_member.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspace_idYesThe workspace ID to list members for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, establishing safe, read-only behavior. The description adds value by detailing the specific fields returned (organization role, workspace role, status, timestamps), which goes beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences, front-loading the purpose and providing immediate actionable guidance. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (one parameter, output schema exists), the description is largely complete. It mentions the key fields returned, though it could optionally address pagination or ordering. However, the absence of such details does not significantly hinder completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% coverage with one required parameter (workspace_id) and a clear description. The tool description does not add further semantic meaning beyond what the schema provides, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists every member in a workspace with specific fields (organization role, workspace role, status, timestamps) and explicitly distinguishes from sibling tools like get_workspace_member, update_workspace_member, and remove_workspace_member.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use this to find a user_id before get_workspace_member, update_workspace_member, or remove_workspace_member.' This tells the agent exactly when to use this tool and suggests alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_workspacesA
Read-onlyIdempotent

List workspaces with id, name, slug, default settings, and timestamps. Use this to find a workspace_id before get_workspace, update_workspace, add_workspace_member, or remove_workspace_member.

ParametersJSON Schema
NameRequiredDescriptionDefault
page_sizeNoNumber of workspaces to return per page (max 100)
current_pageNoPage number to retrieve when results are paginated

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, open-world, idempotent, and non-destructive hints. The description adds that it returns specific fields and implies pagination via page_size and current_page, which is consistent with annotations and adds useful context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, then usage guidance. No redundant phrasing; every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given two optional parameters, no required, and an output schema, the description adequately covers purpose, usage, and return fields. No gaps for a list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already describes both parameters (page_size, current_page). The description does not add additional meaning beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List workspaces' and specifies the returned fields (id, name, slug, default settings, timestamps), distinguishing it from sibling tools like get_workspace (returns a single workspace).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states to use this tool to find a workspace_id before several other tools (get_workspace, update_workspace, etc.), providing clear when-to-use context. However, it does not explicitly mention when not to use it, but the context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

migrate_promptA
Idempotent

Create or update a prompt in one idempotent step for CI/CD and prompt-as-code flows, unlike create_prompt which always makes a new prompt. Looks up the prompt by name within collection_id: if missing it creates the prompt, if found with different content it adds a new version (nothing is overwritten), and if the content already matches it no-ops; dry_run reports what would happen without changing anything. Stores app/env in template_metadata; get collection_id from list_collections. Returns the action taken (created, updated, or unchanged), dry_run flag, message, prompt id, slug, and version id.

ParametersJSON Schema
NameRequiredDescriptionDefault
appYesApp identifier (REQUIRED). Use your deployed app name, for example 'hourlink' or 'support-console'.
envYesEnvironment identifier (REQUIRED). Use your environment name, for example 'dev', 'staging', 'prod', or 'qa'.
nameYesPrompt name to create or find for update
modelNoModel identifier
toolsNoTool definitions
stringNoLegacy prompt template string with {{variable}} mustache syntax.
dry_runNoWhen true, only check what action would be taken without making changes
messagesNoStructured chat template alias for migrations. Serialized to the legacy string format before the prompt is created or updated.
functionsNoFunction definitions
parametersYesDefault values for template variables
tool_choiceNoTool choice strategy
virtual_keyYesVirtual key slug for model access
collection_idYesCollection ID to search in and create under
template_metadataNoAdditional custom metadata
version_descriptionNoDescription for this version

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already set `idempotentHint=true` and `destructiveHint=false`. The description goes beyond by explaining that 'nothing is overwritten' and that if content matches, it 'no-ops'. It also details the dry_run behavior ('reports what would happen without changing anything') and what the tool stores in template_metadata (app/env). This adds valuable behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, dense paragraph that wastes no words. Each sentence adds value: purpose, contrast with sibling, behavior, dry_run, metadata, return fields. It is front-loaded with the most critical information and remains brief given the tool's complexity (15 params).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's high parameter count and nested objects, the description covers the core idempotent behavior, dry_run, key parameters, and return fields. It mentions that `messages` are serialized to legacy string format but does not detail the exact serialization process. However, the output schema likely describes return values. Minor gap, but overall sufficient for agent selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaning by noting that `collection_id` is used for lookup and creation, `dry_run` controls dry-run mode, and `template_metadata` stores app/env. However, it omits explanation for some parameters like `model`, `tools`, `messages`, and `string`. Still, the provided details enhance understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Create or update a prompt in one idempotent step for CI/CD and prompt-as-code flows'. It immediately distinguishes from `create_prompt`, which always creates a new prompt. The verb 'migrate' combined with the explanation of idempotent upsert behavior makes the purpose precise and actionable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool versus the sibling `create_prompt` ('unlike create_prompt which always makes a new prompt'). It also explains the lookup logic by name within collection_id, the three possible outcomes (create, update, no-op), and the dry_run flag for safe testing. It directs the user to `list_collections` for obtaining collection_id.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

promote_promptA

Copy a prompt from one environment to another and create or update the target automatically. Use this for staged releases when you want the target prompt synchronized without manual edits, and it returns both source and target version ids.

ParametersJSON Schema
NameRequiredDescriptionDefault
target_envYesEnvironment identifier (REQUIRED). Use your environment name, for example 'dev', 'staging', 'prod', or 'qa'.
target_nameNoTarget prompt name (defaults to source name with env suffix replaced)
virtual_keyNoVirtual key ID to use (defaults to source prompt's virtual_key)
source_prompt_idYesSource prompt ID or slug (e.g., staging prompt)
target_collection_idYesTarget collection ID for the promoted prompt

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate mutation (readOnlyHint false) and non-destructiveness, and the description adds that it creates or updates the target and returns version IDs. However, given openWorldHint true, the description does not disclose potential side effects or required permissions, leaving some behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise—just two sentences—with no extraneous information. It front-loads the primary action and then provides usage guidance, making it efficient and easy to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's 5 parameters and mutation nature, the description covers core purpose, usage context, and return values. It does not mention error conditions or prerequisites, but the presence of a (presumed) output schema reduces the need for full detail. Overall, it is fairly complete for the use case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 5 parameters have descriptions in the schema (100% coverage), so the description adds minimal extra parameter-level detail. It mentions return values (version IDs) but not parameter specifics, warranting the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: copy a prompt between environments and auto-create/update the target. It explicitly distinguishes itself by mentioning staged releases and synchronization without manual edits, which sets it apart from siblings like create_prompt, update_prompt, and migrate_prompt.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear when-to-use context: for staged releases when automated synchronization is desired. However, it does not explicitly state when not to use this tool or compare it to alternatives like migrate_prompt, which is present among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

publish_partialA

Publish a specific partial version as the default, unlike update_prompt_partial which creates a new draft without activating it. Use after list_partial_versions to pick a version_id; this immediately changes what {{> partial_name}} resolves to for all prompts and replaces the previously active version without a rollback path.

ParametersJSON Schema
NameRequiredDescriptionDefault
versionYesVersion number to publish as default
prompt_partial_idYesPrompt partial ID or slug

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that it immediately changes the default for all prompts, replaces the active version, and has no rollback path. This adds value beyond annotations (which indicate non-read-only behavior).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose and differentiation, second gives usage guidance and consequences. No redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given simple two-param schema, annotations, and presence of an output schema, the description covers purpose, usage, effect, and caution. Sufficient for correct tool invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameters with descriptions. The description provides usage context (e.g., 'use after list_partial_versions') but adds no new meaning to the parameters themselves.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool publishes a specific partial version as default and distinguishes it from update_prompt_partial, which creates a draft. It specifies the effect on partial resolution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to use after list_partial_versions to pick a version_id, describes immediate changes and no rollback path. Lacks explicit 'when not to use', but context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

publish_promptA

Publish a specific version of a prompt as the active default, unlike promote_prompt which copies across environments or update_prompt which creates a new draft. This immediately routes all callers using the slug to that version and there is no rollback, so use list_prompt_versions to pick the version and update_prompt first if you need to create new content before promoting it.

ParametersJSON Schema
NameRequiredDescriptionDefault
versionYesVersion number to publish as the default
prompt_idYesPrompt ID or slug to publish

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only indicate non-read-only and non-destructive. The description adds crucial behavioral context: immediate routing of all callers and no rollback possibility. This is essential for understanding the irreversible impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise: three sentences that front-load the core action, contrast with siblings, explain impact, and give a recommended workflow. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description explains the effect, irreversibility, and workflow. It complements the existing schema and annotations well. Could optionally mention error conditions or prerequisites, but given the output schema exists and annotations are adequate, it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameter descriptions. The description does not add further detail about the parameters themselves but provides contextual usage advice ('use list_prompt_versions to pick the version'). Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Publish a specific version of a prompt as the active default'. It also distinguishes itself from sibling tools 'promote_prompt' and 'update_prompt', making the action unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use guidance by contrasting with 'promote_prompt' and 'update_prompt'. It also advises a workflow: use 'list_prompt_versions' to pick the version and 'update_prompt' if new content is needed before publishing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

remove_workspace_memberA
Destructive

Remove a user from a workspace and revoke workspace access. This does not delete the user from the organization; use delete_user for full removal.

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesThe user ID to remove
workspace_idYesThe workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructiveHint=true and readOnlyHint=false. The description adds valuable context by clarifying that workspace access is revoked but the user remains in the organization, which helps the agent understand the exact scope of destruction. It does not cover reversibility or permissions, but the essential behavioral traits are disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the primary action, and contains no fluff. The second sentence adds important clarification about what the tool does not do, which is essential for disambiguation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is complete for the tool's complexity: there is an output schema, the parameters are fully described in the schema, annotations cover safety traits, and the description provides the needed scope and alternative guidance. No critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both required parameters (user_id and workspace_id) already documented in the schema. The description does not add further parameter-level detail, but the schema carries the full semantic burden, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action: 'Remove a user from a workspace and revoke workspace access.' It clearly distinguishes from the sibling delete_user by noting this action does not delete the user from the organization, making the tool's purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly provides usage guidance: use this tool to remove a user from a workspace, and use delete_user for full removal from the organization. This directly addresses when to use this tool versus an alternative, which is strong guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

render_promptA
Read-onlyIdempotent

Render a prompt by substituting variables and returning the final messages without calling the model. Use this to verify template output before a completion; run_prompt_completion is the tool that actually invokes the model.

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesPrompt ID or slug to render
variablesYesVariable values to substitute into the template
hyperparametersNoOverride default hyperparameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and non-destructive. Description adds important behavioral context: it substitutes variables, returns final messages, and does not call the model. This clarifies the exact operation beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first explains what the tool does, second provides usage guidance and distinguishes from a sibling. No unnecessary words, front-loaded with essential info.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with nested parameters and output schema, the description covers the core purpose, usage context, and comparison to sibling. It notes return of final messages, which suffices given output schema exists. Complete for informed selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. Description does not repeat parameter schemas but implicitly connects the 'variables' parameter to substitution. No extra detail beyond schema, but given full coverage, this is adequate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it renders a prompt by substituting variables and returning final messages without calling the model. It explicitly distinguishes from the sibling tool run_prompt_completion, which invokes the model. The verb and resource are specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool: 'Use this to verify template output before a completion'. Names the alternative tool run_prompt_completion for when model invocation is needed. Provides clear context that this is for verification only.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

resend_user_inviteA

Resend the email for a pending invite that has not been accepted, unlike invite_user which creates a new invite. This sends a fresh email without modifying the invite record, expiry, or role; use get_user_invite first if you are unsure whether the invite still exists and list_user_invites to discover invite_ids.

ParametersJSON Schema
NameRequiredDescriptionDefault
invite_idYesThe invite ID to resend

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description clarifies that the tool 'sends a fresh email without modifying the invite record, expiry, or role', which complements the annotations (readOnlyHint=false, destructiveHint=false) by specifying the non-destructive nature despite being a mutation. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences that efficiently deliver purpose, behavior, usage differentiation, and prerequisite tool recommendations without any redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple parameter set and comprehensive annotations, the description provides complete context including behavior, side effects (none), and usage hints. No gaps remain for an agent to safely invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (single parameter invite_id with clear description). The tool description does not add further semantic meaning beyond what the schema already provides for the parameter, thus baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Resend the email for a pending invite that has not been accepted') and contrasts with sibling tool 'invite_user' which creates a new invite, making the purpose and distinction unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool (for resending pending invites) and when not (use invite_user to create new invites). Additionally, recommends calling get_user_invite first to verify invite existence and list_user_invites to discover IDs, providing excellent usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reset_usage_limit_entityA
Destructive

Reset tracked usage for one entity under a usage limit. This changes accumulated usage for that entity only; use list_usage_limit_entities to confirm the target first.

ParametersJSON Schema
NameRequiredDescriptionDefault
limit_idYesUsage limit policy ID
entity_idYesEntity ID to reset usage for

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate destructive nature (destructiveHint=true). The description adds that it changes accumulated usage for that entity only, which is useful beyond annotations. It could mention reversibility but is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, no wasted words. Highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the annotations and output schema existence, the description adequately covers the action and usage hint. It does not explain the exact reset value, but the context of a mutation tool with destructiveHint makes it sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% description coverage for both parameters, so the description does not need to add new meaning. The description reinforces context but does not enhance parameter semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Reset tracked usage for one entity under a usage limit' with specific verb and resource. It distinguishes from sibling tools like 'list_usage_limit_entities' by mentioning it changes accumulated usage for that entity only.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises using 'list_usage_limit_entities' to confirm the target first, providing clear context and a prerequisite step. It does not mention when not to use, but the guidance is sufficient for a destructive tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rotate_api_keyA
Destructive

Rotate an API key without changing its identity or scopes. The new secret is returned once and exposed to this MCP transcript, while the previous secret remains valid until key_transition_expires_at. Store the new key securely, update callers during the transition window, and never log either secret.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesAPI key UUID obtained from list_api_keys
key_transition_period_msNoOverlap in milliseconds while the previous key remains valid; minimum 1,800,000 (30 minutes), for example 3,600,000 for 1 hour

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations (destructiveHint=true), the description reveals that the new secret is exposed in the transcript, the previous secret remains valid during a transition period, and both secrets must never be logged. This adds critical behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences: purpose, behavioral detail, and actionable guidance. No fluff; front-loaded with the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers essential aspects (purpose, transition window, security). Lacks explicit error conditions or prerequisites, but given the output schema and full parameter descriptions, it is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds context about the transition period's effect (key_transition_expires_at) but does not significantly enhance parameter meaning beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Rotate an API key without changing its identity or scopes,' specifying the action and resource. It implicitly distinguishes from sibling tools like update_api_key by emphasizing identity and scope preservation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides post-rotation guidance (store securely, update callers, never log) and hints at the transition window, but does not explicitly state when to use rotate versus alternatives like create or update.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_prompt_completionA

Execute a prompt against the configured model and return the completion. This makes a billable model call, so use render_prompt first when you want to check the template and validate_completion_metadata when billing fields are uncertain.

ParametersJSON Schema
NameRequiredDescriptionDefault
metadataYesBilling metadata - client_id, app, env are REQUIRED for cost attribution
prompt_idYesPrompt ID or slug to execute
variablesYesVariable values to substitute into the template
hyperparametersNoOverride default hyperparameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that the tool makes a billable model call, adding context beyond annotations. Annotations already indicate readOnlyHint=false and destructiveHint=false. The description adds value but could mention other side effects like logging.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first defines purpose, second provides usage guidelines. No wasted words; front-loaded with essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With output schema present and extensive parameter descriptions in schema, the description is largely complete. It covers purpose and billing context but could mention error handling or rate limits for a more complete picture.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds minimal parameter info beyond the schema, mostly restating that billing metadata is required. No significant enhancement.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states 'Execute a prompt against the configured model and return the completion.' This provides a specific verb and resource, and distinguishes from siblings by mentioning render_prompt and validate_completion_metadata.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises to use render_prompt for template checking and validate_completion_metadata when billing fields are uncertain, given the billable nature of the call. Provides clear usage context and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_log_exportA

Start processing a previously created log export job. This is asynchronous, only queues the export, and does not return rows or a download file; use get_log_export to poll progress and download_log_export after the job completes. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export to start

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses key behavioral traits beyond annotations: asynchronous queueing, no immediate rows/download return, dependency on prior job creation, enterprise-only gating with 403 on non-Enterprise plans. This adds significant context not present in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences, each adding essential information: action, async behavior, alternative tools, and enterprise restriction. No redundant phrases or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter schema, existing output schema, and rich annotations, the description covers all critical aspects: what it does, how it behaves asynchronously, what it does not return, next steps, and error condition. Fully sufficient for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter export_id is already well-described in the schema. The description reinforces that it must be a previously created job, but this adds minimal new semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action: 'Start processing a previously created log export job.' It distinguishes itself from siblings like create_log_export and get_log_export by emphasizing that this is the start step, not creation or polling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides usage context: it operates on a previously created job, is asynchronous, and directs the agent to use get_log_export for polling and download_log_export after completion. Enterprise gating is also noted, covering when it is and isn't applicable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

test_mcp_serverA

Test connectivity to an MCP server. Sends a live check and returns success, response time, HTTP status, and any error; use this before changing configuration or when diagnosing reachability.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug to test

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that it sends a live check and returns success, response time, HTTP status, and any error. Annotations indicate idempotentHint false and openWorldHint true, which the description supports by mentioning external interaction. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no wasted words. The description is front-loaded with the purpose, followed by usage guidance. Highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple diagnostic tool with one parameter and an output schema, the description covers essential aspects: purpose, when to use, and what to expect in return. Complete given the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the description does not add extra meaning beyond what the schema provides for the 'id' parameter. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Test connectivity to an MCP server' with a specific verb and resource, and the context of diagnosing reachability distinguishes it from sibling tools like get_mcp_server.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises use before configuration changes or when diagnosing reachability, providing clear context. However, it does not explicitly name alternative tools for other use cases, though these can be inferred from sibling names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_api_keyA

Update an API key's name, description, scopes, defaults, or limits, unlike delete_api_key which revokes it or create_api_key which issues a new one. Changes take effect immediately for downstream callers, type and sub-type stay fixed after creation, and the call returns success without rotating the secret.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe UUID of the API key to update
nameNoNew display name for the key
scopesNoNew permission scopes for the key
expires_atNoNew expiration date in ISO 8601 format, or null to remove expiration
descriptionNoNew description for the key
alert_emailsNoNew email addresses for alerts
credit_limitNoNew credit limit for usage
rate_limit_rpmNoNew rate limit in requests per minute
alert_thresholdNoNew alert threshold percentage (0-100)
default_metadataNoNew default metadata key-value pairs
default_config_idNoNew default configuration ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide readOnlyHint=false and destructiveHint=false but no further details. The description adds key behavioral traits: changes take effect immediately, type/sub-type stay fixed, and the call returns success without rotating the secret. This goes beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with purpose and sibling differentiation, followed by behavioral details. Every sentence adds unique value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 11 parameters and existence of an output schema, the description is fairly complete. It covers key effects, constraints, and return behavior. However, it could mention that the 'id' is required and that other parameters are optional, but the schema already indicates this. No major gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter has a description. The description only lists general categories (name, description, scopes, defaults, limits) without adding detail beyond what's in the schema. Baseline 3 is appropriate since schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update an API key's name, description, scopes, defaults, or limits', specifying the verb and resource. It also explicitly distinguishes from sibling tools delete_api_key and create_api_key, which is essential for correct selection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description tells when to use this tool vs alternatives (unlike delete/create), but does not mention scenarios where it should not be used (e.g., changing type/sub-type is impossible). The immediate effect and fixed fields provide good context, but some exclusion guidance is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_collectionA

Update a collection's name or description only. This does not move prompts or change membership, so use it for metadata changes rather than reorganization.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the collection
descriptionNoNew description for the collection
collection_idYesCollection ID to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds valuable behavioral context beyond annotations, specifying that only name/description can be updated and that it does not affect membership or prompts. Annotations already indicate non-destructive and non-read-only, so the description complements without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no wasted words. The first sentence states the action and scope; the second provides exclusions and usage guidance. Front-loaded and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple update tool with 3 parameters and an output schema, the description is complete. It covers what is updated, what is not, and when to use the tool. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds meaning by stating 'name or description only', confirming the intended parameters. While the schema lists them, the description reinforces the limitation, adding some value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update a collection's name or description only', specifying the verb (update), resource (collection), and scope (name or description only). It effectively distinguishes from sibling tools like update_prompt or update_workspace by highlighting its limited scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'use it for metadata changes rather than reorganization.' It also clarifies what the tool does not do ('does not move prompts or change membership'), helping the agent avoid misuse.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_configA

Update a config by slug; every call creates a new config version rather than overwriting, so earlier versions stay recoverable via list_config_versions. Only provided fields change: name, status, and all routing/cache/retry settings (cache_mode, cache_max_age, retry_attempts, retry_on_status_codes, strategy_mode, targets) are editable, while the slug stays fixed. Changes apply immediately to every API key and prompt referencing the config; get the slug from list_configs and review current settings with get_config before editing. Returns the config id, slug, and updated config payload.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the configuration
slugYesConfiguration slug to update
statusNoConfiguration status
targetsNoArray of target providers
cache_modeNoCache mode: 'simple' or 'semantic'
cache_max_ageNoCache max age in seconds
strategy_modeNoRouting strategy
retry_attemptsNoNumber of retry attempts (1-5)
retry_on_status_codesNoHTTP status codes to retry on

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

It discloses important non-obvious behavior: each call creates a new version instead of overwriting, changes apply immediately to all API keys and prompts, only provided fields change, and the slug is immutable. With only false annotations, this description carries the full burden of safety and semantics, and it does so comprehensively.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place: versioning behavior, editable fields and slug constraint, immediate effects and prerequisite workflow, and return value. It avoids fluff while covering all critical operational details for a complex update tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (versioning, partial updates, global side effects) and the availability of an output schema, the description provides all necessary context: how to prepare, what changes, what remains fixed, how to recover, and what is returned. No significant gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds valuable semantics beyond the schema by clarifying partial-update behavior ('Only provided fields change') and the immutability of the slug. It also enumerates the editable settings, reinforcing which parameters are meaningful for an update call.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Update a config by slug', which is a specific verb+resource statement. It distinguishes this from create_config by noting slug stays fixed and every call creates a new version, and from delete_config by the recoverability of old versions. The editable field groups are enumerated, making the scope clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context by instructing to get the slug from list_configs and review current settings with get_config before editing. It also mentions list_config_versions for recovery of earlier versions. However, it does not explicitly name create_config as the alternative for creating a new slug, so the when-not-to-use guidance is only implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_feedbackA

Update an existing feedback record by ID. Returns the updated status and feedback IDs, changes only value, weight, and metadata, and leaves the trace linkage immutable; use create_feedback only for a new record.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe unique identifier of the feedback to update
valueNoNew feedback value/rating. Common patterns: 1 for positive, 0 for negative.
weightNoNew weighting factor for the feedback
metadataNoNew or updated custom metadata for the feedback

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate non-readonly, non-destructive, non-idempotent, open-world. The description adds that only value, weight, and metadata are changed, trace linkage is immutable, and it returns updated status and IDs. This provides useful context beyond annotations, though it could mention potential side effects (e.g., impact on analytics).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the action and return, then constraints and alternative. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (4 parameters, output schema exists) and the presence of sibling alternatives, the description covers the essential aspects: purpose, modifiable fields, immutability of trace linkage, and usage guidance. No critical information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all four parameters. The description summarizes which parameters are modifiable but does not add new semantic details beyond the schema. The mention of return values is about output, not input parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update an existing feedback record by ID' with a specific verb and resource. It details what fields are changed (value, weight, metadata) and distinguishes from the sibling tool create_feedback, which is for new records.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use this tool (to update existing feedback) and when not (use create_feedback for a new record). It also clarifies that trace linkage is immutable, guiding parameter selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_guardrailA

Update a guardrail's name, checks, or actions, unlike create_guardrail which registers a new one or delete_guardrail which removes it. This creates a new version that takes effect immediately for dependent configs, so review list_guardrails first; returns the updated id, slug, and version_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the guardrail
checksNoUpdated array of checks to apply
actionsNoUpdated actions configuration
guardrail_idYesThe guardrail UUID or slug to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that updating creates a new version that takes effect immediately for dependent configs, and returns specific identifiers. This adds value beyond annotations (which are minimal). However, it does not cover potential side effects like cascading impacts or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (two sentences) with front-loaded purpose and sibling distinction. Every sentence adds value: first defines the action, second explains versioning and returns. No redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (4 parameters, nested objects), the description covers the main effect (new version, immediate impact) and suggests reviewing list_guardrails. Output schema exists (implicitly), so return values are adequately stated. Missing minor details like validation or permission mentions, but overall sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the description adds little semantic meaning beyond what the schema already provides for parameters. It mentions updating name, checks, or actions, but the schema already details each parameter. The description's focus on return values does not enhance parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates a guardrail's name, checks, or actions, and distinguishes itself from create_guardrail and delete_guardrail. It also specifies the return values (id, slug, version_id), leaving no ambiguity about the tool's purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool (updating guardrails) and advises reviewing list_guardrails first. It mentions creating a new version and immediate effect, but does not explicitly exclude scenarios where update should not be used (e.g., if no changes needed).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_input_guardrail_workspace_exclusionsA
DestructiveIdempotent

Set workspace exclusions from organisation-wide input guardrails. Each entry excludes or restores one workspace; override_existing replaces prior states while the default merge behavior preserves unmentioned workspaces. Review the matching list tool first because enforcement changes immediately. Repeating the same states is safe. Requires an organisation service API key with organisation_exclusions.update scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspacesYesWorkspace exclusion states to apply
organisation_idYesOrganisation ID whose workspace exclusions should be updated
override_existingNoReplace existing exclusion states instead of merging changes

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses merge vs. override behavior, immediate enforcement changes, idempotency of repeating states, authentication requirements, and 403 errors on non-Enterprise plans. This significantly enriches the annotations (readOnlyHint=false, destructiveHint=true, idempotentHint=true), which only hint at these behaviors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and information-dense. Six short sentences each contribute unique value: purpose, entry semantics, merge behavior, guidance, auth, and error condition. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderately complex mutation tool, the description covers all essential context: auth scope, Enterprise gating, immediate enforcement impact, merge vs. override semantics, and idempotency. With annotations and an output schema present, this description is fully sufficient for correct tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by explaining that each entry excludes or restores one workspace and that default merge preserves unmentioned workspaces, which clarifies the workspaces array and override_existing parameter beyond schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool sets workspace exclusions for organisation-wide input guardrails, with a specific verb and resource. It distinguishes itself from the sibling update_output_guardrail_workspace_exclusions by specifying 'input guardrails' and explains the core entry semantics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises reviewing the matching list tool first because enforcement changes immediately, and specifies required API key scope and Enterprise plan gating. While it doesn't name alternatives, the context strongly implies the input/output distinction and provides clear preconditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_integrationA
DestructiveIdempotent

Update an integration's name, description, API key, provider config, Secret Reference mappings, or pricing adjustments by slug. Only provided fields change; key, secret mapping, and config changes take effect immediately and can disrupt dependent providers or live requests, while pricing multipliers change cost analytics and budget accounting. Review get_integration first. Model availability and workspace access remain separate in update_integration_models and update_integration_workspaces.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyNoNew API key for the provider
nameNoNew human-readable name for the integration
slugYesThe slug of the integration to update
aws_regionNoNew AWS region (for AWS Bedrock)
api_versionNoNew API version (for Azure OpenAI)
custom_hostNoNew custom base URL for the provider
descriptionNoNew description for the integration
resource_nameNoNew resource name (for Azure OpenAI)
vertex_regionNoNew GCP region (for Vertex AI)
deployment_nameNoNew deployment name (for Azure OpenAI)
secret_mappingsNoReplacement runtime Secret Reference mappings; each target_field must be unique
aws_access_key_idNoNew AWS access key ID (for AWS Bedrock)
vertex_project_idNoNew GCP project ID (for Vertex AI)
pricing_adjustmentsNoReplacement cost multiplier configuration, or null to clear adjustments
aws_secret_access_keyNoNew AWS secret access key (for AWS Bedrock)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

While annotations already flag destructiveHint and readOnlyHint, the description adds vital behavioral context: partial updates are honored, changes to key/secret/config are immediately effective and can disrupt live traffic, pricing adjustments modify cost analytics, and related concerns are deliberately isolated. This materially enriches what the bare annotations convey and aligns with them (no contradiction).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences deliver purpose, side effects, usage safety, and sibling differentiation without any filler. The most important behavioral caveat (disruption risk) is front-loaded, and every clause contributes information an agent needs before invoking the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's 15 parameters, rich schema, and existing output schema, the description covers purpose, mutation semantics, immediate effects, risk profile, and separation from closely related tools. It leaves no major gap: an agent knows what to call, what to expect, and how to avoid unintended consequences. The explicit pointer to get_integration for pre-review further rounds out operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already covers all 15 parameters with detailed per-field descriptions, so the baseline is a 3 per the rubric. The description adds value by grouping fields into behavioral categories (e.g., 'key, secret mapping, and config' vs. 'pricing adjustments') and clarifying partial update semantics ('Only provided fields change'), which helps the agent understand how parameter combinations behave even though individual properties are schema-documented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Update') and an explicit resource ('an integration'), then enumerates the exact updatable fields: name, description, API key, provider config, Secret Reference mappings, and pricing adjustments. It further distinguishes this tool from siblings by naming update_integration_models and update_integration_workspaces as separate concerns, making its scope unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete guidance: only provided fields change, key/secret/config changes take effect immediately and can disrupt providers or live requests, and pricing multiplier changes affect cost analytics. It also instructs the agent to review get_integration first and explicitly redirects model/workspace access changes to separate tools, providing both usage context and exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_integration_modelsA
DestructiveIdempotent

Bulk enable or disable integration models, register custom or fine-tuned models, set per-model hosts and headers, and attach static token pricing. allow_all_models controls whether future provider models start enabled. These changes affect every workspace using the integration; inspect list_integration_models first and use get_model_pricing when deriving custom rates. Returns success and the number of models updated.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
modelsYesArray of model configurations to update
allow_all_modelsNoWhether newly available provider models are enabled by default

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true. The description adds valuable context beyond annotations: 'These changes affect every workspace using the integration' and 'Returns success and the number of models updated.' This improves transparency, though it doesn't cover all nuances like atomicity or rollback behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is four sentences, each earning its place: first lists actions, second clarifies a parameter, third warns about scope and gives guidance, fourth notes the return. It is front-loaded and concise without waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 3 parameters and a nested models array, the description covers purpose, safety, scope, sequencing, alternatives, and return value. An output schema exists, so detailed return fields do not need to be explained. The description is complete and well-rounded.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description summarizes parameter effects (e.g., 'allow_all_models controls whether future provider models start enabled') but does not add significant new meaning beyond the schema definitions. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb and resource: 'Bulk enable or disable integration models, register custom or fine-tuned models, set per-model hosts and headers, and attach static token pricing.' It enumerates distinct actions, which distinguishes it from sibling tools like list_integration_models or delete_integration_model.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'inspect list_integration_models first' and 'use get_model_pricing when deriving custom rates.' It names alternative tools and gives sequencing context, fulfilling the when-to-use vs alternatives criterion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_integration_workspacesA
DestructiveIdempotent

Control per-workspace and global access to an integration, including cost/rate limits, usage resets, and automatic default-provider creation. global_workspace_access_enabled affects current and future workspaces; override_existing_workspace_access determines whether it replaces explicit workspace settings. Per-workspace default-provider fields override top-level values. Review list_integration_workspaces first because access and limits change downstream usage immediately.

ParametersJSON Schema
NameRequiredDescriptionDefault
slugYesThe slug of the integration
workspacesYesArray of workspace configurations to update
global_credit_limitNoGlobal cost credit limit applied with global workspace access
default_provider_slugNoDefault slug for providers auto-created in workspaces
global_rate_limit_rpmNoGlobal requests-per-minute limit for workspace access
global_alert_thresholdNoGlobal cost alert threshold applied to workspace access
create_default_providerNoAuto-create providers when granting workspace access
global_workspace_access_enabledNoEnable or disable access for all current and future workspaces
override_existing_workspace_accessNoApply global settings over existing per-workspace access

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag destructiveHint, idempotentHint, and openWorldHint. The description adds valuable behavioral context beyond these: global_workspace_access_enabled affects current and future workspaces, override_existing_workspace_access determines replacement behavior, and per-workspace fields override top-level values. It also notes immediate downstream impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each earning its place: first states purpose and scope, second explains key toggles, third clarifies precedence, fourth gives operational warning. Slightly dense but well structured and front-loaded with the main purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 parameters, global vs per-workspace settings), the description covers the critical nuances and includes a caution to review first. An output schema exists, so return values need not be explained. It is complete enough for a knowledgeable agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaningful interactions between parameters (e.g., precedence rules, override semantics) that are not apparent from individual property descriptions. This elevates the score to 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool controls per-workspace and global access to an integration, enumerating specific features (cost/rate limits, usage resets, provider creation). This distinguishes it from read-only siblings like list_integration_workspaces and the broader update_integration tool, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance to 'Review list_integration_workspaces first' and warns that access and limits change immediately, giving clear context on when and how to use the tool. It stops short of naming alternative tools for exclusions, but the prerequisite sequence is helpful.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_log_exportA

Update an existing log export configuration before or between export runs. Only workspace_id, time_of_generation_max, and requested_fields can change after creation, so use get_log_export to review the current job and start_log_export after the definition is ready. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
export_idYesThe unique ID of the log export to update
workspace_idNoWorkspace ID for the export
requested_fieldsNoFields to include in export: id, trace_id, created_at, request, response, is_success, ai_org, ai_model, req_units, res_units, total_units, request_url, cost, cost_currency, response_time, response_status_code, mode, config, prompt_slug, metadata
time_of_generation_maxNoMaximum time filter in date format (e.g., '2024-07-25' or ISO 8601)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful behavioral details beyond annotations: only specific fields can be changed after creation, the operation is suitable only before or between runs, and it returns 403 on non-Enterprise plans. This goes beyond the basic readOnly/destructive flags and helps the agent anticipate side effects and limitations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences pack all essential information: purpose, timing, mutable fields, alternative tools, and a critical error condition. No filler or redundant repetition of schema details. Front-loaded with the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the moderate complexity, complete parameter schema, and presence of an output schema, the description covers all necessary context: what can be changed, when to call it, what to do before/after, and a failure mode. An agent has enough to decide when and how to invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by explicitly identifying which parameters are mutable (workspace_id, time_of_generation_max, requested_fields) and implying export_id is the required target. This contextualizes parameter usage beyond the raw schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates an existing log export configuration, with a specific verb ('Update') and resource ('log export configuration'). It also distinguishes itself from sibling tools by specifying timing ('before or between export runs') and pointing to alternatives like get_log_export and start_log_export.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit usage guidance: which fields are mutable, when to use it (before or between export runs), and exactly when to use other tools ('use get_log_export to review the current job and start_log_export after the definition is ready'). It also notes Enterprise-gating and 403 behavior, giving clear context for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_integrationA
DestructiveIdempotent

Update an MCP integration's name, description, URL, auth, transport, headers, or runtime Secret Reference mappings. Only supplied fields change; URL, auth, header, and secret changes apply immediately and can break active clients, so inspect get_mcp_integration first. Use update_mcp_server when changing only a Portkey server instance's display metadata.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug to update
urlNoNew URL endpoint
nameNoNew display name
auth_typeNoNew authentication type
transportNoNew transport protocol
descriptionNoNew description
custom_headersNoNew custom headers for authentication. Sent via configurations.custom_headers
secret_mappingsNoReplacement runtime Secret Reference mappings; each target_field must be unique

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses critical behavioral traits beyond annotations: partial update semantics ('Only supplied fields change'), immediate effect and potential to break active clients for URL/auth/header/secret changes, and the need to inspect first. This adds significant value beyond the destructiveHint annotation and does not contradict it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, each with a distinct purpose: what the tool does, key behavioral warnings, and alternative tool guidance. It is front-loaded with the core action and fields, and contains no extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (8 params, nested objects) and the existing output schema, the description covers all essential aspects: purpose, partial update behavior, immediate impact warnings, prerequisite inspection, and alternative tooling. It is fully sufficient for an agent to use the tool safely and correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so all parameters are documented, but the description adds important semantic context about partial updates ('Only supplied fields change') and groups fields by impact, which helps the agent understand how parameters combine. This elevates it above the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Update') and resource ('MCP integration'), lists the specific updatable fields, and distinguishes itself from the sibling tool update_mcp_server by explicitly naming the alternative for server display metadata changes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides explicit guidance on when to use this tool versus the alternative ('Use update_mcp_server when changing only a Portkey server instance's display metadata') and advises inspecting get_mcp_integration first, giving clear contextual usage instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_integration_capabilitiesA
Idempotent

Bulk enable or disable capabilities (tools, prompts, resources) on an MCP integration. A reversible toggle, not a deletion: only the capabilities named in the array change state, the change hides or exposes them immediately for connected users, and re-running with enabled flipped restores them. Source the integration id from list_mcp_integrations and current capability names, types, and states from list_mcp_integration_capabilities. Returns a success confirmation message.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug
capabilitiesYesArray of capability updates

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations show readOnlyHint=false, destructiveHint=false, idempotentHint=true. The description adds meaningful context: reversible toggle, immediate exposure, and that only named capabilities are affected. No contradictions. It does not cover rate limits or auth, but annotations already indicate non-destructive and idempotent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, then key behavioral points. Every sentence is purposeful and concise. No fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (bulk update with multiple capability types), the description is self-sufficient. It explains purpose, usage, side effects, and prerequisite data sources. The output schema existence is not critical as it mentions a confirmation message.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with clear descriptions. The description adds value by explaining the effect of the array (state toggle), reversibility, and the immediate visibility. This enriches the agent's understanding beyond raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with 'Bulk enable or disable capabilities (tools, prompts, resources) on an MCP integration,' clearly specifying the verb (enable/disable), resource (capabilities on an MCP integration), and distinguishing it from sibling tools like update_mcp_integration and list_mcp_integration_capabilities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides explicit context: reversible toggle, not deletion; only named capabilities change state; immediate for connected users; re-running with enabled flipped restores them. It also instructs to source the integration id from list_mcp_integrations and current states from list_mcp_integration_capabilities. No direct when-not-to-use, but implied through reversibility.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_integration_workspacesA
Idempotent

Grant or revoke workspace access to an MCP integration in bulk. Reversible: only the workspaces listed change, access applies or is removed immediately for all users in those workspaces, and re-running with enabled flipped undoes a change. Source the integration id from list_mcp_integrations, workspace ids from list_workspaces, and the current access state from list_mcp_integration_workspaces. Returns a success confirmation message.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP integration ID or slug
workspacesYesArray of workspace access updates

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description explains idempotency ('re-running with enabled flipped undoes a change') and immediate application for all users, adding value beyond annotations (idempotentHint: true). It does not mention auth or rate limits but is sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the purpose, and includes all essential information without filler. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema (though not shown), so return values are covered. The description covers purpose, parameters, behavior, and prerequisites. It could mention error handling but is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds context on sourcing IDs but does not significantly expand on the schema's descriptions of 'id' and 'workspaces'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Grant or revoke workspace access to an MCP integration in bulk,' specifying the verb and resource. It distinguishes from sibling tools like list_mcp_integration_workspaces by detailing how to source IDs and the bulk nature.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit context for when to use this tool (bulk granting/revoking) and gives sourcing guidance from related tools (list_mcp_integrations, list_workspaces, list_mcp_integration_workspaces). It lacks explicit when-not-to-use or alternatives but is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_serverA

Update an MCP server's name or description. Changes apply immediately, but URL and auth live on the parent integration, so use update_mcp_integration for those fields.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug to update
nameNoNew display name
descriptionNoNew description

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description notes that changes apply immediately, adding behavioral context beyond annotations. No contradictions with annotations (readOnlyHint=false, destructiveHint=false). Adequate for a simple update tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose, second provides usage guidance. No redundant information, front-loaded with key action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, an output schema exists, and the description covers what fields are updatable, immediate effect, and sibling tool delegation. Complete for this context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for each parameter. The description adds no additional meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Update') and the resource ('MCP server') and specifies the fields affected (name or description). It distinguishes from the sibling tool 'update_mcp_integration' by noting that URL and auth are handled separately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent when to use this tool versus the sibling ('use update_mcp_integration for those fields'), providing clear guidance on tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_server_capabilitiesA

Enable or disable capabilities on an MCP server. Changes take effect immediately and override the integration-level settings for this server; use list_mcp_server_capabilities first to inspect the current surface.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug
capabilitiesYesArray of capability updates

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate readOnlyHint=false and destructiveHint=false, so the description adds context about immediacy and overriding integration settings. This provides behavioral insight beyond annotations, but it doesn't detail potential side effects, authorization needs, or reversibility. Given annotations cover some aspects, a score of 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loaded with the action, and every sentence provides essential information. No unnecessary words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with a complex schema and good annotations, the description covers key context: action, effect (immediate override), and prerequisite inspection step. It doesn't document output, but an output schema exists. It might be slightly lacking on preconditions (e.g., server existence) but overall is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the description doesn't need to elaborate much on parameters. It mentions 'enable or disable capabilities' which aligns with the 'enabled' field, but adds no new meaning beyond the schema. The description does not explain the structure of the capabilities array or enum values, but the schema already documents these.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool enables or disables capabilities on an MCP server, uses a specific verb ('update_mcp_server_capabilities' inherently suggests modification), and distinguishes from siblings like 'update_mcp_integration_capabilities' by noting it overrides integration-level settings. It also advises to use 'list_mcp_server_capabilities' first, further clarifying scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly recommends using 'list_mcp_server_capabilities' first to inspect current capabilities, providing clear when-to-use guidance. It states changes take effect immediately and override integration settings, helping differentiate from other update tools. However, it does not explicitly mention when not to use this tool or list all alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_mcp_server_user_accessA

Grant or revoke individual user access to an MCP server. Changes take effect immediately and override the default access setting for the selected users; use list_mcp_server_user_access first if you need the current state.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe MCP server ID or slug
usersYesArray of user access updates

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate a write operation with no destructive or idempotent guarantees. The description adds that changes 'take effect immediately' and 'override the default access setting', providing useful behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the core action, the second adds a crucial usage hint. No wasted words, front-loaded with purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers the action, immediate effect, override behavior, and a recommended precursor call. With an output schema available (as noted in context signals), the description is sufficient for a simple mutation tool, though edge cases like non-existent users are not addressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with clear parameter descriptions. The description does not add significant new meaning to the parameters beyond reiterating their purpose, so baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('grant or revoke individual user access') and the resource ('MCP server'). It distinguishes itself from sibling tools like list_mcp_server_user_access and update_mcp_server by focusing on per-user access overrides.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance to use list_mcp_server_user_access first for current state, and explains that changes override default access. While it doesn't cover when not to use it, the alternative recommendation is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_organisation_defaultsA
Idempotent

Replace the organisation-wide default input and/or output guardrail lists inherited by workspaces. Only supplied directions change, but enforcement updates immediately across non-excluded workspaces; inspect get_organisation_defaults and the directional workspace exclusions first. Repeating the same lists is safe. Requires an organisation service API key with organisation_settings.update scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
input_guardrailsNoOrdered guardrail IDs or slugs to enforce on model input
output_guardrailsNoOrdered guardrail IDs or slugs to enforce on model output

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (idempotentHint, readOnlyHint=false), the description discloses immediate enforcement propagation, partial-update behavior, required API key scope, Enterprise gating, and the 403 error on non-Enterprise plans. This adds substantial operational context that annotations alone do not convey. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact—four sentences—with each sentence earning its place: function, behavior, safety, permissions, and error handling. It is front-loaded with the core purpose and avoids redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a moderately complex mutation tool, the description covers all essential aspects: what it does, how it behaves (immediate propagation), prerequisites (inspect read tools), idempotency, authentication, and error condition. The existence of an output schema also covers return values. No significant gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics by stating "Only supplied directions change," clarifying that parameters are optional and act as partial updates. This is extra value beyond the schema descriptions, warranting a slightly higher score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb "Replace" and clearly identifies the resource: "organisation-wide default input and/or output guardrail lists inherited by workspaces." It distinguishes itself from sibling tools like update_input_guardrail_workspace_exclusions and update_output_guardrail_workspace_exclusions by focusing on the defaults rather than exclusions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description instructs the agent to "inspect get_organisation_defaults and the directional workspace exclusions first," providing clear prerequisite steps. It also notes that only supplied directions change and that repetition is safe, giving practical usage context. It stops short of explicitly naming alternatives or when-not-to-use scenarios, but the guidance is solid.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_output_guardrail_workspace_exclusionsA
DestructiveIdempotent

Set workspace exclusions from organisation-wide output guardrails. Each entry excludes or restores one workspace; override_existing replaces prior states while the default merge behavior preserves unmentioned workspaces. Review the matching list tool first because enforcement changes immediately. Repeating the same states is safe. Requires an organisation service API key with organisation_exclusions.update scope. Enterprise-gated. Returns 403 on non-Enterprise Portkey plans.

ParametersJSON Schema
NameRequiredDescriptionDefault
workspacesYesWorkspace exclusion states to apply
organisation_idYesOrganisation ID whose workspace exclusions should be updated
override_existingNoReplace existing exclusion states instead of merging changes

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations (which already mark destructiveHint=true and idempotentHint=true) by disclosing concrete behavioral traits: enforcement changes immediately, override_existing replaces prior states while default merge preserves unmentioned workspaces, requires an organisation service API key with organisation_exclusions.update scope, is Enterprise-gated, and returns 403 on non-Enterprise plans. It also clarifies that repeating the same states is safe. No contradiction with annotations is present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but every sentence earns its place. It covers purpose, usage prerequisite, behavioral side effects, idempotency, authentication scope, enterprise gating, and expected error response—all in five concise sentences. The structure is front-loaded with the core action, followed by key behavioral nuance, then supporting operational details. No redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a potentially destructive mutation tool with an output schema present, the description is comprehensive. It covers the tool's side effects (immediate enforcement), merge/override semantics, idempotency, required auth scope, enterprise gating, and a specific error condition (403). The presence of an output schema means return-value details need not be in the description. The combination of schema coverage, annotations, and description provides a complete picture for an agent to invoke this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all parameters, which would normally set a baseline of 3. The description adds meaningful semantic detail beyond the schema: it explains that 'override_existing replaces prior states while the default merge behavior preserves unmentioned workspaces', which clarifies the interaction between the override_existing boolean and the workspaces array. This goes beyond the schema's per-parameter descriptions, so a 4 is warranted.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Set workspace exclusions from organisation-wide output guardrails', which is a specific verb+resource statement. The tool name includes 'output_guardrail_workspace_exclusions' and there is a sibling 'update_input_guardrail_workspace_exclusions', so the explicit 'output guardrails' wording clearly distinguishes it from the input-guardrail alternative. The purpose is unambiguous and actionable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly instructs to 'Review the matching list tool first because enforcement changes immediately', providing a clear prerequisite and warning about side effects. It also notes that 'Repeating the same states is safe', which guides usage around idempotency. It does not explicitly state 'use this instead of the input guardrail tool', but the purpose clarity and context make the usage context well-defined.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_promptA

Update an existing prompt and create a new archived version. Only provided fields change, and publish_prompt is what makes the new version active. For multi-message chat prompts pass messages (preferred) or a JSON-encoded array as string.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew display name for the prompt
modelNoNew model identifier
toolsNoNew tool definitions
stringNoLegacy prompt template string. Use plain text for single-message prompts, or a JSON-encoded messages array string for multi-message chat prompts.
dry_runNoWhen true, validate without updating
messagesNoStructured chat template alias for updates. Serialized to the legacy string format before the prompt is updated.
functionsNoNew function definitions
prompt_idYesPrompt ID or slug to update
parametersNoNew default values for template variables
tool_choiceNoNew tool choice strategy
virtual_keyNoNew virtual key slug
collection_idNoMove to a different collection
template_metadataNoNew metadata
version_descriptionNoDescription for this version

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds context about version creation and dry_run validation beyond the annotations. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with essential information, no redundant text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 14 parameters and existing output schema, the description covers core behavior, versioning, and validation with dry_run. It is complete enough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds value by clarifying partial updates and the preferred messages parameter for chat prompts.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates an existing prompt and creates a new archived version, distinguishing it from create_prompt and publish_prompt.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains partial updates, the role of publish_prompt, and provides guidance for multi-message prompts. It lacks explicit when-not-to-use scenarios but is generally clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_prompt_labelA

Update a prompt label's name, description, or color only, unlike update_prompt_version which changes which label a version carries. This takes effect immediately for all versions already tagged with the label, but does not reassign labels or touch history; use list_prompt_labels to find the label_id first.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the label
label_idYesLabel ID to update
color_codeNoNew hex color code (e.g., '#FF5733')
descriptionNoNew description

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are basic (non-readonly, non-destructive). The description adds that changes take effect immediately for all tagged versions and do not reassign labels or touch history, offering valuable behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, followed by behavioral notes and prerequisite. No redundant information; every phrase adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 4 parameters (1 required) and an output schema, the description covers immediate effects, side effects, and prerequisite steps. No gaps given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions. The description reinforces that only name, description, or color can be updated, adding operational context. While not adding new parameter details, it clarifies mutation scope.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update a prompt label's name, description, or color only', specifying the verb and resource. It distinguishes from sibling update_prompt_version, which changes label assignments, making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly contrasts with update_prompt_version and advises using list_prompt_labels to find the label_id first, providing clear when-to-use and prerequisite instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_prompt_partialA

Create a new version of a partial by updating its content or metadata. Only provided fields change, and the new version stays inactive until publish_partial makes it current.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew display name for the partial
statusNoNew status for the partial
stringNoNew content for the partial
descriptionNoDescription for this version
prompt_partial_idYesPrompt partial ID or slug to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are present and consistent (readOnlyHint=false, destructiveHint=false, idempotentHint=false, openWorldHint=true). The description adds that only provided fields change and that the new version is inactive until published, which are important behaviors not covered by annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, each adding value. The purpose is front-loaded, and there is no redundant or unnecessary text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description does not need to explain return values. It covers the key behavioral context (partial update, new inactive version) and is complete for this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all parameters. The description adds value by stating 'Only provided fields change', clarifying partial update semantics, which goes beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb (create) and resource (a new version of a partial), and implies the resource is a partial, distinguishing it from sibling tools like publish_partial that make a version current.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains that only provided fields change and the new version stays inactive until publish_partial is used, providing clear context for when to use this tool and hinting at the alternative publish tool. However, it does not explicitly say 'use publish_partial to make it current'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_prompt_versionA

Update a specific prompt version's label assignment. This only assigns or removes a label, and null clears the label after you look up ids with list_prompt_labels.

ParametersJSON Schema
NameRequiredDescriptionDefault
label_idYesLabel ID to assign to this version, or null to remove the label
prompt_idYesPrompt ID or slug
version_idYesVersion UUID to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate mutation (readOnlyHint=false) but no destruction. The description adds that the tool only affects label assignment and clarifies null behavior ('null clears the label'). This adds meaningful context beyond the basic annotation flags.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences. The first states the core purpose, the second elaborates on null behavior and prerequisite workflow. No unnecessary words; all content earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with three required parameters and an output schema, the description covers the essential: what it does, how null works, and where to get label IDs. No gaps for this complexity level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, providing baseline 3. The description adds value by explaining the label_id parameter's role in assigning/removing labels and guiding the agent to use 'list_prompt_labels' for ID lookup. This goes beyond schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it updates a prompt version's label assignment, with specific verbs ('assigns', 'removes', 'clears'). It distinguishes itself from siblings like 'update_prompt' (which updates the prompt itself) and 'list_prompt_labels' (for looking up label IDs).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context: it only assigns or removes a label, and recommends looking up IDs with 'list_prompt_labels'. It does not explicitly state when not to use it or list alternative tools, but the scope is well-defined.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_providerA

Update a provider's metadata, limits, or expiration. reset_usage clears accumulated usage counters immediately, so use it only when you intend to reset quota tracking. Returns the updated provider id and slug.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew display name for the provider
noteNoNew note or description for the provider
slugYesThe slug of the provider to update
expires_atNoNew expiration date in ISO 8601 format
reset_usageNoSet to true to reset accumulated usage metrics
credit_limitNoNew credit limit for usage
workspace_idNoWorkspace ID - required when using organization admin keys
periodic_resetNoPeriod for resetting usage limits: 'monthly' or 'weekly'. Defaults to 'monthly'.
alert_thresholdNoNew alert threshold percentage (0-100)
rate_limit_unitNoRate limit unit: 'rpm' (requests per minute), 'rph' (requests per hour), or 'rpd' (requests per day)
rate_limit_valueNoNew rate limit value
usage_limit_typeNoType of usage limit: 'cost' (monetary) or 'tokens' (token count). Defaults to 'cost'.

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that reset_usage clears counters immediately, adding behavioral context beyond the annotations (which only indicate non-destructive, non-idempotent). It also mentions the return value. This is good but could be more detailed about other side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the main purpose, followed by a critical usage note and return value. Every word is necessary and no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 12 parameters and an existing output schema, the description covers the main categories and return value. It lacks details on authorization or behavior for omitted fields, but the output schema and schema descriptions compensate adequately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage, the baseline is 3. The description only generally mentions 'metadata, limits, or expiration' and specifically calls out reset_usage, but does not add significant meaning beyond what the schema's parameter descriptions already provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates a provider's metadata, limits, or expiration, with a specific verb and resource. While it differentiates from sibling update tools by targeting 'provider', it does not explicitly distinguish from other update tools, but the resource is clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance on when to use the reset_usage parameter ('only when you intend to reset quota tracking'), but does not offer broader when-to-use or when-not-to-use guidance relative to other tools, though the context is sufficient given the tool's specificity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_rate_limitA

Update a rate limit's name, unit, or value by id. Conditions and group_by are immutable after creation; use get_rate_limit first if you need the full policy.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe unique identifier of the rate limit
nameNoNew name for the rate limit
unitNoNew time unit: 'rpm' (per minute), 'rph' (per hour), or 'rpd' (per day)
valueNoNew maximum allowed value per unit

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate mutation (readOnlyHint=false) and non-destructive nature. Description adds key behavioral context: immutability of conditions/group_by and the need to fetch the full policy via get_rate_limit. This goes beyond annotations but could mention if updates trigger immediate effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no fluff. Important information is front-loaded and every sentence contributes value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (partial update with immutable fields) and the presence of an output schema, the description provides sufficient context: what can be updated, what is immutable, and a suggested prerequisite step.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% description coverage for all 4 parameters. The description adds minimal additional meaning beyond 'by id' and which fields are updatable, so a baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it updates a rate limit's name, unit, or value by id. Distinguishes from siblings by noting that conditions and group_by are immutable and referencing get_rate_limit for the full policy.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly mentions when to use get_rate_limit first if the full policy is needed, and clarifies that conditions and group_by cannot be updated, guiding the agent to alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_secret_referenceA
DestructiveIdempotent

Update at least one selected field on a Secret Reference. auth_config is merged into the existing manager-specific configuration. allowed_workspaces replaces all workspace mappings and sets allow_all_workspaces=false; allow_all_workspaces=true removes workspace-specific mappings. Credential values are sent to Portkey and exposed to the MCP transcript.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesSecret Reference UUID or slug
nameNoNew human-readable name
tagsNoReplacement string key-value tags, or null to clear them
secret_keyNoNew key within a structured secret, or null to use the whole value
auth_configNoProvide at least one changed authentication field to merge into the existing configuration; choose one AWS, Azure, or HashiCorp field family and do not mix families
descriptionNoNew description, or null to clear it
secret_pathNoNew path of the secret in the external secret manager
allowed_workspacesNoReplacement list of workspace UUIDs or slugs allowed to use this reference; supplying it sets allow_all_workspaces=false
allow_all_workspacesNoWhether every workspace may use this reference; cannot be true with allowed_workspaces

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds value beyond annotations by detailing that auth_config is merged, allowed_workspaces replaces mappings, and credential values are exposed to transcript. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three efficient sentences: purpose, key behavioral details, security note. Front-loaded with action, no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers critical merge/replace behaviors and credential exposure. With output schema present, return value explanation is unnecessary. Minor gap: no prerequisites or error conditions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, baseline 3. Description adds meaning by specifying merge behavior for auth_config and replacement logic for allowed_workspaces, enhancing schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Update at least one selected field on a Secret Reference', specifying verb and resource. Differentiates from sibling create/delete/get/list by emphasizing partial update.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description explains merging vs replacing behavior for auth_config and allowed_workspaces, providing clear context for use. Lacks explicit when-not to use, but siblings are distinct.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_usage_limitA

Update a usage limit's name, credit_limit, alert_threshold, reset schedule, or reset target by id, unlike update_rate_limit which tunes request throttling. New values apply immediately to tracked usage, conditions and group_by are immutable after creation, and the call returns the updated id without clearing accumulated usage (use reset_usage_limit_entity for that).

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesThe unique identifier of the usage limit
nameNoNew name for the usage limit
credit_limitNoNew maximum allowed usage value
periodic_resetNoNew periodic reset schedule
alert_thresholdNoNew alert threshold percentage (0-100)
reset_usage_for_valueNoReset usage counters for a specific group_by value

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds behavioral context beyond annotations: 'New values apply immediately to tracked usage', 'conditions and group_by are immutable after creation', and 'returns the updated id without clearing accumulated usage'. It aligns with annotations (readOnlyHint=false, destructiveHint=false) and provides useful caveats.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that packs purpose, sibling distinction, usage notes, and behavioral caveats without redundancy. It is efficient, though breaking into multiple sentences could improve readability slightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the tool's purpose, parameters, sibling differentiation, and key behavioral traits (immediate application, immutability, no usage reset). With an output schema present and good annotations, it provides a solid understanding for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

While the schema has 100% coverage on parameter descriptions, the description adds value by grouping the updatable fields and noting immutability of certain fields. This helps the agent understand which parameters can be modified and which cannot, enhancing the schema info.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Update' and the specific resource 'usage limit' along with the fields that can be updated. It also contrasts with the sibling tool 'update_rate_limit' by noting it tunes request throttling, making the purpose distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: it says to update 'by id' and contrasts with 'update_rate_limit'. It also notes that 'conditions and group_by are immutable after creation' and directs to 'reset_usage_limit_entity' for resetting usage, indicating when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_userA

Update a user's first name, last name, or organization role by id. Email and workspace roles are not editable here; use update_workspace_member for workspace membership changes.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleNoNew organization-level role
user_idYesThe user ID to update
last_nameNoNew last name
first_nameNoNew first name

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate it's a mutation (readOnlyHint: false). The description confirms it modifies user fields and adds the important constraint that email and workspace roles are off-limits. It does not describe side effects, error conditions, or permissions, but given annotations already signal mutability, the description adds sufficient behavioral context for a simple update.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Delivers maximum information in two sentences: first sentence states the action and scope, second sentence specifies exclusions and suggests an alternative. No wasted words; front-loaded with essential details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with an output schema, the description covers what is updatable, what is not, and directs to an alternative. It lacks mention of prerequisites (e.g., user existence, authentication) or error behavior. However, given the tool's low complexity and the presence of openWorldHint, it is fairly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for each parameter. The overall description adds value by clarifying which parameters are and aren't editable, providing context beyond individual parameter descriptions. This compensates for the baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb 'Update', resource 'user', and lists specific fields (first name, last name, organization role). Explicitly distinguishes from update_workspace_member by stating what is not editable. This leaves no ambiguity about the tool's purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use and when-not-to-use guidance: 'Email and workspace roles are not editable here; use update_workspace_member for workspace membership changes.' Directly contrasts with a sibling tool, helping an agent select the correct tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_virtual_keyA

Update a virtual key's name, secret, note, or limits. Rotating the key takes effect immediately, and limit changes apply to downstream prompts and configs using this slug. Returns the updated name, slug, and status.

ParametersJSON Schema
NameRequiredDescriptionDefault
keyNoNew provider API key value
nameNoNew display name for the key
noteNoNew note or description
slugYesThe slug of the virtual key to update
credit_limitNoNew credit limit for usage
rate_limit_rpmNoNew rate limit in requests per minute
alert_thresholdNoNew alert threshold percentage (0-100)

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations indicate it's not read-only or destructive. The description adds that rotating the key is immediate and limit changes propagate to downstream prompts/configs, providing useful behavioral context beyond what annotations offer. It builds on the 'openWorldHint' by explaining downstream effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loading the purpose and then adding key effects and return value. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the description appropriately avoids repeating return details but mentions updated name, slug, and status. It covers the tool's effect on downstream resources, making it fairly complete for an update operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 100% of parameters with descriptions, so the description's mention of 'name, secret, note, or limits' adds minimal new information. It does not clarify edge cases or constraints beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Update a virtual key's name, secret, note, or limits,' using a specific verb ('Update') and resource ('virtual key'). It distinguishes from siblings like create_virtual_key and delete_virtual_key by focusing on modification.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context by noting that key rotation takes effect immediately and limit changes affect downstream, but it does not explicitly state when to use this tool versus alternatives or when not to use it. No exclusions or comparisons are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_workspaceA

Update a workspace's name, slug, description, default flag, or metadata by id, unlike update_workspace_member which changes role assignments within a workspace. Only provided fields change and updates take effect immediately; changing the slug can break URLs, API key references, and other external links, so confirm no dependencies first.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the workspace
slugNoNew slug for the workspace
metadataNoNew metadata key-value pairs
is_defaultNoSet as default workspace (1 = yes, 0 = no)
descriptionNoNew description
workspace_idYesThe workspace ID to update

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds behavioral context beyond annotations by noting that updates take effect immediately and that changing the slug can break URLs, API key references, etc. Annotations already indicate mutation (readOnlyHint=false) but the description enriches this with practical side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: the first clearly states purpose and differentiates from a sibling, the second provides critical behavioral notes. No unnecessary words; front-loaded with essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (6 params, nested metadata) and the presence of annotations and output schema, the description covers the main use case, side effects, and differentiation. It does not detail return values but the output schema likely handles that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already describes each parameter. The description merely lists the field names without adding new semantic detail. This meets the baseline but does not add significant value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states it updates a workspace's properties by ID, listing specific fields (name, slug, description, default flag, metadata) and contrasts with the sibling tool update_workspace_member, making the purpose distinct and clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a direct comparison with update_workspace_member and warns about slug changes breaking external links. However, it does not explicitly state when to use this tool versus other alternatives like get_workspace for reading, leaving some implicit context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_workspace_memberA

Update a workspace member's role by workspace_id and user_id. Only the role changes here; use list_workspace_members or get_workspace_member to confirm the current assignment first.

ParametersJSON Schema
NameRequiredDescriptionDefault
roleYesNew role in the workspace
user_idYesThe user ID to update (must be a valid UUID from list_all_users, not an email address)
workspace_idYesThe workspace ID

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, idempotentHint=false, destructiveHint=false, and openWorldHint=true. The description adds the key behavioral constraint that only the role field changes and recommends a confirmation step before updating, which is useful context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two focused sentences: the first states the action and required keys, the second states the scope and a pre-check workflow. There is no redundant or filler content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the schema covers all parameters, annotations cover the safety profile, and an output schema exists, the description is sufficient for selection and invocation. It includes the purpose, the specific keys, the role enum context, and a confirmation step, though it could briefly hint at alternative member-management tools for non-role updates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers all three parameters thoroughly, including the role enum and user_id UUID constraints. The description reinforces that role is the only mutable field, adding semantic clarity beyond the schema fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Update') and names the exact resource ('workspace member's role') plus the identifying keys ('by workspace_id and user_id'). It distinguishes itself from sibling member tools by explicitly limiting the scope to role changes only.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly states the tool is for role updates only and tells the agent to use list_workspace_members or get_workspace_member to confirm the current assignment first. It does not explicitly name alternative mutation tools (e.g., remove_workspace_member), but the scope exclusion ('Only the role changes here') provides clear when-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_completion_metadataA
Read-onlyIdempotent

Preflight billing metadata before run_prompt_completion. Validates required fields and values without making changes, so you can catch attribution errors before paying for the call.

ParametersJSON Schema
NameRequiredDescriptionDefault
appNoApp identifier (REQUIRED). Use your deployed app name, for example 'hourlink' or 'support-console'.
envNoEnvironment identifier (REQUIRED). Use your environment name, for example 'dev', 'staging', 'prod', or 'qa'.
featureNoFeature name for tracking
client_idNoClient ID for billing attribution
project_idNoProject ID for granular billing

Output Schema

ParametersJSON Schema
NameRequiredDescription
okYesWhether the tool call succeeded and returned structured data
dataNoStructured success payload when ok is true
errorNoStructured error payload when ok is false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint=true and idempotentHint=true, and the description reinforces 'without making changes'. It adds specific context about billing and attribution error checking, which goes beyond the annotations. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the key action, and every word adds value. No unnecessary information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The explanation covers the purpose, usage context, and behavioral safety. Since the tool has an output schema, the description does not need to explain return values. It is sufficiently complete for a validation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and each parameter has a detailed description. The tool description does not add extra meaning beyond the schema; it only mentions 'validates required fields and values' without specifying which fields. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as a preflight validator for billing metadata before run_prompt_completion. It uses a specific verb ('validate') and resource ('billing metadata'), and distinguishes itself from the sibling 'run_prompt_completion' by being a read-only check.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states when to use the tool ('before run_prompt_completion') and why ('catch attribution errors before paying for the call'). It implies not to use it when not about to call run_prompt_completion, but does not explicitly name alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 5 tool updatesv1.0.5
    • Changedget_workspace_member3 fields changed
      • changedInput schema / properties / user_id / description
        Previous value: -"The user ID to retrieve"New value: +"The user ID to retrieve (must be a valid UUID from list_all_users, not an email address)"
      • addedInput schema / properties / user_id / format
        Added value: +"uuid"
      • addedInput schema / properties / user_id / pattern
        Added value: +"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$"
    • Changedlist_scim_groups1 field changed
      • changedInput schema / properties / page / description
        Previous value: -"Zero-based results page to retrieve; the first page is 0"New value: +"Zero-based results page to retrieve; the first page is 0. Unlike current_page used by other list tools in this API (which is 1-based), do not pass 1 for the first page here."
    • Changedlist_scim_workspace_mappings1 field changed
      • changedInput schema / properties / page / description
        Previous value: -"Zero-based results page to retrieve; the first page is 0"New value: +"Zero-based results page to retrieve; the first page is 0. Unlike current_page used by other list tools in this API (which is 1-based), do not pass 1 for the first page here."
    • Changedremove_workspace_member2 fields changed
      • addedInput schema / properties / user_id / format
        Added value: +"uuid"
      • addedInput schema / properties / user_id / pattern
        Added value: +"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$"
    • Changedupdate_workspace_member3 fields changed
      • changedInput schema / properties / user_id / description
        Previous value: -"The user ID to update"New value: +"The user ID to update (must be a valid UUID from list_all_users, not an email address)"
      • addedInput schema / properties / user_id / format
        Added value: +"uuid"
      • addedInput schema / properties / user_id / pattern
        Added value: +"^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$"
  2. 23 tool updates
    • Changedcreate_config2 fields changed
      • addedInput schema / properties / targets / items / properties / provider / description
        Added value: +"Provider slug to route requests to"
      • addedInput schema / properties / targets / items / properties / virtual_key / description
        Added value: +"Virtual key slug to use for this routing target"
    • Changedcreate_integration4 fields changed
      • addedInput schema / properties / create_default_provider
        Added value: +{
        +  "description": "Create a provider automatically for a workspace-scoped integration; defaults to true",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / default_provider_slug
        Added value: +{
        +  "description": "Custom slug for the automatically created workspace provider",
        +  "maxLength": 255,
        +  "pattern": "^[a-zA-Z0-9_-]+$",
        +  "type": "string"
        +}
      • addedInput schema / properties / pricing_adjustments
        Added value: +{
        +  "anyOf": [
        +    {
        +      "properties": {
        +        "multiplier": {
        +          "description": "Per-unit multipliers applied over Portkey catalog pricing",
        +          "properties": {
        +            "additional_units": {
        +              "anyOf": [
        +                {
        +                  "additionalProperties": {
        +                    "anyOf": [
        +                      {
        +                        "minimum": 0,
        +                        "type": "number"
        +                      },
        +                      {
        +                        "type": "null"
        +                      }
        +                    ]
        +                  },
        +                  "propertyNames": {
        +                    "type": "string"
        +                  },
        +                  "type": "object"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Multipliers keyed by Portkey pricing unit, such as web_search or file_search"
        +            },
        +            "cache_read_audio_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read audio input token price multiplier"
        +            },
        +            "cache_read_image_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read image input token price multiplier"
        +            },
        +            "cache_read_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read input token price multiplier"
        +            },
        +            "cache_read_text_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read text input token price multiplier"
        +            },
        +            "cache_write_image_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write image input token price multiplier"
        +            },
        +            "cache_write_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write input token price multiplier"
        +            },
        +            "cache_write_text_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write text input token price multiplier"
        +            },
        +            "default": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Fallback multiplier for pricing units; 1 is unchanged, 0.8 is a 20% discount"
        +            },
        +            "image": {
        +              "anyOf": [
        +                {
        +                  "properties": {
        +                    "default": {
        +                      "anyOf": [
        +                        {
        +                          "minimum": 0,
        +                          "type": "number"
        +                        },
        +                        {
        +                          "type": "null"
        +                        }
        +                      ],
        +                      "description": "Default image-generation price multiplier"
        +                    }
        +                  },
        +                  "type": "object"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image-generation pricing multipliers"
        +            },
        +            "prediction_accepted_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Accepted predicted-output token price multiplier"
        +            },
        +            "prediction_rejected_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Rejected predicted-output token price multiplier"
        +            },
        +            "reasoning_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Reasoning token price multiplier"
        +            },
        +            "request_audio_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Audio input token price multiplier"
        +            },
        +            "request_image_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image input token price multiplier"
        +            },
        +            "request_text_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Text input token price multiplier"
        +            },
        +            "request_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Input token price multiplier"
        +            },
        +            "response_audio_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Audio output token price multiplier"
        +            },
        +            "response_image_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image output token price multiplier"
        +            },
        +            "response_text_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Text output token price multiplier"
        +            },
        +            "response_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Output token price multiplier"
        +            }
        +          },
        +          "type": "object"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Negotiated discount or markup multipliers for cost accounting"
        +}
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Unique runtime Secret Reference mappings; mapping key allows the key field to be omitted",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key to select from a multi-value secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference UUID or slug accessible to the workspace",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Integration field populated at runtime: key or configurations.<field>",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "string",
        +              "json"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Treat the resolved value as a plain string or parsed JSON"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Changedcreate_mcp_integration1 field changed
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Runtime Secret Reference mappings; every configurations.<field> target must be unique",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key to select from a multi-value secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference UUID or slug accessible to this integration",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Configuration field resolved at runtime, such as configurations.oauth_metadata",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "string",
        +              "json"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Use json when the target configuration field expects a structured object"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Addedcreate_scim_workspace_mapping
    • Addeddelete_scim_workspace_mapping
    • Addeddisconnect_mcp_server_connection
    • Addedget_log
    • Addedget_log_export_field_restrictions
    • Addedget_model_pricing
    • Addedget_organisation_defaults
    • Addedlist_input_guardrail_workspace_exclusions
    • Addedlist_mcp_server_connections
    • Addedlist_output_guardrail_workspace_exclusions
    • Addedlist_scim_groups
    • Addedlist_scim_workspace_mappings
    • Changedupdate_config2 fields changed
      • addedInput schema / properties / targets / items / properties / provider / description
        Added value: +"Provider slug to route requests to"
      • addedInput schema / properties / targets / items / properties / virtual_key / description
        Added value: +"Virtual key slug to use for this routing target"
    • Addedupdate_input_guardrail_workspace_exclusions
    • Changedupdate_integration2 fields changed
      • addedInput schema / properties / pricing_adjustments
        Added value: +{
        +  "anyOf": [
        +    {
        +      "properties": {
        +        "multiplier": {
        +          "description": "Per-unit multipliers applied over Portkey catalog pricing",
        +          "properties": {
        +            "additional_units": {
        +              "anyOf": [
        +                {
        +                  "additionalProperties": {
        +                    "anyOf": [
        +                      {
        +                        "minimum": 0,
        +                        "type": "number"
        +                      },
        +                      {
        +                        "type": "null"
        +                      }
        +                    ]
        +                  },
        +                  "propertyNames": {
        +                    "type": "string"
        +                  },
        +                  "type": "object"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Multipliers keyed by Portkey pricing unit, such as web_search or file_search"
        +            },
        +            "cache_read_audio_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read audio input token price multiplier"
        +            },
        +            "cache_read_image_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read image input token price multiplier"
        +            },
        +            "cache_read_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read input token price multiplier"
        +            },
        +            "cache_read_text_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-read text input token price multiplier"
        +            },
        +            "cache_write_image_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write image input token price multiplier"
        +            },
        +            "cache_write_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write input token price multiplier"
        +            },
        +            "cache_write_text_input_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Cache-write text input token price multiplier"
        +            },
        +            "default": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Fallback multiplier for pricing units; 1 is unchanged, 0.8 is a 20% discount"
        +            },
        +            "image": {
        +              "anyOf": [
        +                {
        +                  "properties": {
        +                    "default": {
        +                      "anyOf": [
        +                        {
        +                          "minimum": 0,
        +                          "type": "number"
        +                        },
        +                        {
        +                          "type": "null"
        +                        }
        +                      ],
        +                      "description": "Default image-generation price multiplier"
        +                    }
        +                  },
        +                  "type": "object"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image-generation pricing multipliers"
        +            },
        +            "prediction_accepted_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Accepted predicted-output token price multiplier"
        +            },
        +            "prediction_rejected_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Rejected predicted-output token price multiplier"
        +            },
        +            "reasoning_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Reasoning token price multiplier"
        +            },
        +            "request_audio_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Audio input token price multiplier"
        +            },
        +            "request_image_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image input token price multiplier"
        +            },
        +            "request_text_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Text input token price multiplier"
        +            },
        +            "request_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Input token price multiplier"
        +            },
        +            "response_audio_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Audio output token price multiplier"
        +            },
        +            "response_image_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Image output token price multiplier"
        +            },
        +            "response_text_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Text output token price multiplier"
        +            },
        +            "response_token": {
        +              "anyOf": [
        +                {
        +                  "minimum": 0,
        +                  "type": "number"
        +                },
        +                {
        +                  "type": "null"
        +                }
        +              ],
        +              "description": "Output token price multiplier"
        +            }
        +          },
        +          "type": "object"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    {
        +      "type": "null"
        +    }
        +  ],
        +  "description": "Replacement cost multiplier configuration, or null to clear adjustments"
        +}
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Replacement runtime Secret Reference mappings; each target_field must be unique",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key to select from a multi-value secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference UUID or slug accessible to the workspace",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Integration field populated at runtime: key or configurations.<field>",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "string",
        +              "json"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Treat the resolved value as a plain string or parsed JSON"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Changedupdate_integration_models5 fields changed
      • addedInput schema / properties / allow_all_models
        Added value: +{
        +  "description": "Whether newly available provider models are enabled by default",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / models / items / properties / base_model_slug
        Added value: +{
        +  "description": "Base model slug for a fine-tuned model",
        +  "type": "string"
        +}
      • addedInput schema / properties / models / items / properties / configurations
        Added value: +{
        +  "description": "Per-model custom upstream host and headers",
        +  "properties": {
        +    "custom_headers": {
        +      "additionalProperties": {
        +        "type": "string"
        +      },
        +      "description": "Headers sent only for this model; values may contain secrets",
        +      "propertyNames": {
        +        "type": "string"
        +      },
        +      "type": "object"
        +    },
        +    "custom_host": {
        +      "description": "Custom upstream URL used only for this model",
        +      "format": "uri",
        +      "type": "string"
        +    }
        +  },
        +  "type": "object"
        +}
      • addedInput schema / properties / models / items / properties / is_finetune
        Added value: +{
        +  "description": "Whether this entry is a fine-tuned model",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / models / items / properties / pricing_config
        Added value: +{
        +  "description": "Static custom pricing for this integration model",
        +  "properties": {
        +    "pay_as_you_go": {
        +      "description": "Pay-as-you-go rates for this integration model",
        +      "properties": {
        +        "request_token": {
        +          "description": "Input token pricing",
        +          "properties": {
        +            "price": {
        +              "description": "Static price per token in the integration pricing unit",
        +              "minimum": 0,
        +              "type": "number"
        +            }
        +          },
        +          "required": [
        +            "price"
        +          ],
        +          "type": "object"
        +        },
        +        "response_token": {
        +          "description": "Output token pricing",
        +          "properties": {
        +            "price": {
        +              "description": "Static price per token in the integration pricing unit",
        +              "minimum": 0,
        +              "type": "number"
        +            }
        +          },
        +          "required": [
        +            "price"
        +          ],
        +          "type": "object"
        +        }
        +      },
        +      "type": "object"
        +    },
        +    "type": {
        +      "const": "static",
        +      "description": "Static per-token pricing configuration",
        +      "type": "string"
        +    }
        +  },
        +  "required": [
        +    "type"
        +  ],
        +  "type": "object"
        +}
    • Changedupdate_integration_workspaces10 fields changed
      • addedInput schema / properties / create_default_provider
        Added value: +{
        +  "description": "Auto-create providers when granting workspace access",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / default_provider_slug
        Added value: +{
        +  "description": "Default slug for providers auto-created in workspaces",
        +  "type": "string"
        +}
      • addedInput schema / properties / global_alert_threshold
        Added value: +{
        +  "description": "Global cost alert threshold applied to workspace access",
        +  "maximum": 100,
        +  "minimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / global_credit_limit
        Added value: +{
        +  "description": "Global cost credit limit applied with global workspace access",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / global_rate_limit_rpm
        Added value: +{
        +  "description": "Global requests-per-minute limit for workspace access",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / global_workspace_access_enabled
        Added value: +{
        +  "description": "Enable or disable access for all current and future workspaces",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / override_existing_workspace_access
        Added value: +{
        +  "description": "Apply global settings over existing per-workspace access",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / workspaces / items / properties / create_default_provider
        Added value: +{
        +  "description": "Override whether a provider is auto-created for this workspace",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / workspaces / items / properties / default_provider_slug
        Added value: +{
        +  "description": "Override the auto-created provider slug for this workspace",
        +  "type": "string"
        +}
      • addedInput schema / properties / workspaces / items / properties / reset_usage
        Added value: +{
        +  "description": "Reset exhausted usage status to active",
        +  "type": "boolean"
        +}
    • Changedupdate_mcp_integration1 field changed
      • addedInput schema / properties / secret_mappings
        Added value: +{
        +  "description": "Replacement runtime Secret Reference mappings; each target_field must be unique",
        +  "items": {
        +    "properties": {
        +      "secret_key": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Optional key to select from a multi-value secret"
        +      },
        +      "secret_reference_id": {
        +        "description": "Secret Reference UUID or slug accessible to this integration",
        +        "type": "string"
        +      },
        +      "target_field": {
        +        "description": "Configuration field resolved at runtime, such as configurations.oauth_metadata",
        +        "type": "string"
        +      },
        +      "value_format": {
        +        "anyOf": [
        +          {
        +            "enum": [
        +              "string",
        +              "json"
        +            ],
        +            "type": "string"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ],
        +        "description": "Use json when the target configuration field expects a structured object"
        +      }
        +    },
        +    "required": [
        +      "target_field",
        +      "secret_reference_id"
        +    ],
        +    "type": "object"
        +  },
        +  "type": "array"
        +}
    • Addedupdate_organisation_defaults
    • Addedupdate_output_guardrail_workspace_exclusions
  3. 29 tool updatesv0.5.0
    • Addedcreate_secret_reference
    • Addeddelete_secret_reference
    • Changedget_analytics_group_metadata2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedget_analytics_group_models2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedget_analytics_group_users2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Addedget_secret_reference
    • Changedget_user_stats1 field changed
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_all_users3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_api_keys3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_audit_logs3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_collections3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_configs3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_guardrails3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_integration_models2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_integration_workspaces2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_integrations2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_mcp_integrations2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_mcp_server_capabilities3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_mcp_server_user_access3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_mcp_servers2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_prompt_labels3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_prompts3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_providers2 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
    • Addedlist_secret_references
    • Changedlist_user_invites3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_virtual_keys3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Changedlist_workspaces3 fields changed
      • addedInput schema / properties / current_page / maximum
        Added value: +9007199254740991
      • changedInput schema / properties / current_page / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / page_size / type
        Previous value: -"number"New value: +"integer"
    • Addedrotate_api_key
    • Addedupdate_secret_reference
  4. 23 tool updatesv1.0.2
    • Changedget_analytics_group_metadata1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids",
        -  "metadata_key"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max",
        +  "metadata_key"
        +]
    • Changedget_analytics_group_models1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_analytics_group_users1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_cache_hit_latency1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_cache_hit_rate1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_cost_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_error_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_error_rate_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_error_stacks_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_error_status_codes_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_feedback_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_feedback_models_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_feedback_scores_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_feedback_weighted_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_latency_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_request_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_rescued_requests_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_token_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_user_requests_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedget_user_stats2 fields changed
      • changedInput schema / properties / page_size / description
        Previous value: -"Number of results per page (for pagination)"New value: +"Number of results per page (max 100)"
      • addedInput schema / properties / page_size / maximum
        Added value: +100
    • Changedget_users_analytics1 field changed
      • changedInput schema / required
        Previous value: -[
        -  "time_of_generation_min",
        -  "time_of_generation_max",
        -  "api_key_ids"
        -]New value: +[
        +  "time_of_generation_min",
        +  "time_of_generation_max"
        +]
    • Changedlist_guardrails2 fields changed
      • changedInput schema / properties / page_size / description
        Previous value: -"Number of items per page (1-1000, default: 100)"New value: +"Number of items per page (1-100, default: 100)"
      • changedInput schema / properties / page_size / maximum
        Previous value: -1000New value: +100
    • Changedlist_workspaces2 fields changed
      • changedInput schema / properties / page_size / description
        Previous value: -"Number of workspaces to return per page (default varies by endpoint)"New value: +"Number of workspaces to return per page (max 100)"
      • addedInput schema / properties / page_size / maximum
        Added value: +100
  5. 7 tool updatesv0.3.7
    • Changedget_request_analytics1 field changed
      • changedInput schema / properties / api_key_ids / description
        Previous value: -"API key UUIDs. Accepts the legacy comma-separated string or a structured array; normalized to the legacy Portkey query param before the request is sent."New value: +"Legacy Portkey query param for API key UUIDs. Comma-separated string; request_analytics also accepts an array and normalizes it to this form."
    • Changedlist_all_users2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_configs2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_mcp_server_capabilities2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_mcp_server_user_access2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_user_invites2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
    • Changedlist_virtual_keys2 fields changed
      • addedInput schema / properties / current_page
        Added value: +{
        +  "description": "Page number for pagination",
        +  "exclusiveMinimum": 0,
        +  "type": "number"
        +}
      • addedInput schema / properties / page_size
        Added value: +{
        +  "description": "Number of results per page (max 100)",
        +  "exclusiveMinimum": 0,
        +  "maximum": 100,
        +  "type": "number"
        +}
  6. 150 tool updatesv1.0.1
    • Addedadd_workspace_member
    • Addedcancel_log_export
    • Addedcreate_api_key
    • Addedcreate_collection
    • Addedcreate_config
    • Addedcreate_feedback
    • Addedcreate_guardrail
    • Addedcreate_integration
    • Addedcreate_log_export
    • Addedcreate_mcp_integration
    • Addedcreate_mcp_server
    • Addedcreate_prompt
    • Addedcreate_prompt_label
    • Addedcreate_prompt_partial
    • Addedcreate_provider
    • Addedcreate_rate_limit
    • Addedcreate_usage_limit
    • Addedcreate_virtual_key
    • Addedcreate_workspace
    • Addeddelete_api_key
    • Addeddelete_collection
    • Addeddelete_config
    • Addeddelete_guardrail
    • Addeddelete_integration
    • Addeddelete_integration_model
    • Addeddelete_mcp_integration
    • Addeddelete_mcp_server
    • Addeddelete_prompt
    • Addeddelete_prompt_label
    • Addeddelete_prompt_partial
    • Addeddelete_provider
    • Addeddelete_rate_limit
    • Addeddelete_usage_limit
    • Addeddelete_user
    • Addeddelete_user_invite
    • Addeddelete_virtual_key
    • Addeddelete_workspace
    • Addeddownload_log_export
    • Addedget_analytics_group_metadata
    • Addedget_analytics_group_models
    • Addedget_analytics_group_users
    • Addedget_api_key
    • Addedget_cache_hit_latency
    • Addedget_cache_hit_rate
    • Addedget_collection
    • Addedget_config
    • Addedget_cost_analytics
    • Addedget_error_analytics
    • Addedget_error_rate_analytics
    • Addedget_error_stacks_analytics
    • Addedget_error_status_codes_analytics
    • Addedget_feedback_analytics
    • Addedget_feedback_models_analytics
    • Addedget_feedback_scores_analytics
    • Addedget_feedback_weighted_analytics
    • Addedget_guardrail
    • Addedget_integration
    • Addedget_latency_analytics
    • Addedget_log_export
    • Addedget_mcp_integration
    • Addedget_mcp_integration_metadata
    • Addedget_mcp_server
    • Addedget_prompt
    • Addedget_prompt_label
    • Addedget_prompt_partial
    • Addedget_prompt_version
    • Addedget_provider
    • Addedget_rate_limit
    • Addedget_request_analytics
    • Addedget_rescued_requests_analytics
    • Addedget_token_analytics
    • Addedget_usage_limit
    • Addedget_user
    • Addedget_user_invite
    • Addedget_user_requests_analytics
    • Addedget_user_stats
    • Addedget_users_analytics
    • Addedget_virtual_key
    • Addedget_workspace
    • Addedget_workspace_member
    • Addedinsert_log
    • Addedinvite_user
    • Addedlist_all_users
    • Addedlist_api_keys
    • Addedlist_audit_logs
    • Addedlist_collections
    • Addedlist_config_versions
    • Addedlist_configs
    • Addedlist_guardrails
    • Addedlist_integration_models
    • Addedlist_integration_workspaces
    • Addedlist_integrations
    • Addedlist_log_exports
    • Addedlist_mcp_integration_capabilities
    • Addedlist_mcp_integration_workspaces
    • Addedlist_mcp_integrations
    • Addedlist_mcp_server_capabilities
    • Addedlist_mcp_server_user_access
    • Addedlist_mcp_servers
    • Addedlist_partial_versions
    • Addedlist_prompt_labels
    • Addedlist_prompt_partials
    • Addedlist_prompt_versions
    • Addedlist_prompts
    • Addedlist_providers
    • Addedlist_rate_limits
    • Addedlist_usage_limit_entities
    • Addedlist_usage_limits
    • Addedlist_user_invites
    • Addedlist_virtual_keys
    • Addedlist_workspace_members
    • Addedlist_workspaces
    • Addedmigrate_prompt
    • Addedpromote_prompt
    • Addedpublish_partial
    • Addedpublish_prompt
    • Addedremove_workspace_member
    • Addedrender_prompt
    • Addedresend_user_invite
    • Addedreset_usage_limit_entity
    • Addedrun_prompt_completion
    • Addedstart_log_export
    • Addedtest_mcp_server
    • Addedupdate_api_key
    • Addedupdate_collection
    • Addedupdate_config
    • Addedupdate_feedback
    • Addedupdate_guardrail
    • Addedupdate_integration
    • Addedupdate_integration_models
    • Addedupdate_integration_workspaces
    • Addedupdate_log_export
    • Addedupdate_mcp_integration
    • Addedupdate_mcp_integration_capabilities
    • Addedupdate_mcp_integration_workspaces
    • Addedupdate_mcp_server
    • Addedupdate_mcp_server_capabilities
    • Addedupdate_mcp_server_user_access
    • Addedupdate_prompt
    • Addedupdate_prompt_label
    • Addedupdate_prompt_partial
    • Addedupdate_prompt_version
    • Addedupdate_provider
    • Addedupdate_rate_limit
    • Addedupdate_usage_limit
    • Addedupdate_user
    • Addedupdate_virtual_key
    • Addedupdate_workspace
    • Addedupdate_workspace_member
    • Addedvalidate_completion_metadata
  7. 117 tool updates
    • Removedadd_workspace_member
    • Removedcancel_log_export
    • Removedcreate_api_key
    • Removedcreate_collection
    • Removedcreate_config
    • Removedcreate_feedback
    • Removedcreate_guardrail
    • Removedcreate_integration
    • Removedcreate_log_export
    • Removedcreate_prompt
    • Removedcreate_prompt_label
    • Removedcreate_prompt_partial
    • Removedcreate_provider
    • Removedcreate_rate_limit
    • Removedcreate_usage_limit
    • Removedcreate_virtual_key
    • Removedcreate_workspace
    • Removeddelete_api_key
    • Removeddelete_collection
    • Removeddelete_config
    • Removeddelete_guardrail
    • Removeddelete_integration
    • Removeddelete_integration_model
    • Removeddelete_prompt
    • Removeddelete_prompt_label
    • Removeddelete_prompt_partial
    • Removeddelete_provider
    • Removeddelete_rate_limit
    • Removeddelete_usage_limit
    • Removeddelete_user
    • Removeddelete_user_invite
    • Removeddelete_virtual_key
    • Removeddelete_workspace
    • Removeddownload_log_export
    • Removedget_api_key
    • Removedget_cache_hit_latency
    • Removedget_cache_hit_rate
    • Removedget_collection
    • Removedget_config
    • Removedget_cost_analytics
    • Removedget_error_analytics
    • Removedget_error_rate_analytics
    • Removedget_guardrail
    • Removedget_integration
    • Removedget_latency_analytics
    • Removedget_log_export
    • Removedget_prompt
    • Removedget_prompt_label
    • Removedget_prompt_partial
    • Removedget_provider
    • Removedget_rate_limit
    • Removedget_request_analytics
    • Removedget_token_analytics
    • Removedget_trace
    • Removedget_usage_limit
    • Removedget_user
    • Removedget_user_invite
    • Removedget_user_stats
    • Removedget_users_analytics
    • Removedget_virtual_key
    • Removedget_workspace
    • Removedget_workspace_member
    • Removedinsert_log
    • Removedinvite_user
    • Removedlist_all_users
    • Removedlist_api_keys
    • Removedlist_audit_logs
    • Removedlist_collections
    • Removedlist_config_versions
    • Removedlist_configs
    • Removedlist_guardrails
    • Removedlist_integration_models
    • Removedlist_integration_workspaces
    • Removedlist_integrations
    • Removedlist_log_exports
    • Removedlist_partial_versions
    • Removedlist_prompt_labels
    • Removedlist_prompt_partials
    • Removedlist_prompt_versions
    • Removedlist_prompts
    • Removedlist_providers
    • Removedlist_rate_limits
    • Removedlist_traces
    • Removedlist_usage_limits
    • Removedlist_user_invites
    • Removedlist_virtual_keys
    • Removedlist_workspace_members
    • Removedlist_workspaces
    • Removedmigrate_prompt
    • Removedpromote_prompt
    • Removedpublish_partial
    • Removedpublish_prompt
    • Removedremove_workspace_member
    • Removedrender_prompt
    • Removedresend_user_invite
    • Removedrun_prompt_completion
    • Removedstart_log_export
    • Removedupdate_api_key
    • Removedupdate_collection
    • Removedupdate_config
    • Removedupdate_feedback
    • Removedupdate_guardrail
    • Removedupdate_integration
    • Removedupdate_integration_models
    • Removedupdate_integration_workspaces
    • Removedupdate_log_export
    • Removedupdate_prompt
    • Removedupdate_prompt_label
    • Removedupdate_prompt_partial
    • Removedupdate_provider
    • Removedupdate_rate_limit
    • Removedupdate_usage_limit
    • Removedupdate_user
    • Removedupdate_virtual_key
    • Removedupdate_workspace
    • Removedupdate_workspace_member
    • Removedvalidate_completion_metadata
  8. 117 tool updates
    • First observedadd_workspace_member
    • First observedcancel_log_export
    • First observedcreate_api_key
    • First observedcreate_collection
    • First observedcreate_config
    • First observedcreate_feedback
    • First observedcreate_guardrail
    • First observedcreate_integration
    • First observedcreate_log_export
    • First observedcreate_prompt
    • First observedcreate_prompt_label
    • First observedcreate_prompt_partial
    • First observedcreate_provider
    • First observedcreate_rate_limit
    • First observedcreate_usage_limit
    • First observedcreate_virtual_key
    • First observedcreate_workspace
    • First observeddelete_api_key
    • First observeddelete_collection
    • First observeddelete_config
    • First observeddelete_guardrail
    • First observeddelete_integration
    • First observeddelete_integration_model
    • First observeddelete_prompt
    • First observeddelete_prompt_label
    • First observeddelete_prompt_partial
    • First observeddelete_provider
    • First observeddelete_rate_limit
    • First observeddelete_usage_limit
    • First observeddelete_user
    • First observeddelete_user_invite
    • First observeddelete_virtual_key
    • First observeddelete_workspace
    • First observeddownload_log_export
    • First observedget_api_key
    • First observedget_cache_hit_latency
    • First observedget_cache_hit_rate
    • First observedget_collection
    • First observedget_config
    • First observedget_cost_analytics
    • First observedget_error_analytics
    • First observedget_error_rate_analytics
    • First observedget_guardrail
    • First observedget_integration
    • First observedget_latency_analytics
    • First observedget_log_export
    • First observedget_prompt
    • First observedget_prompt_label
    • First observedget_prompt_partial
    • First observedget_provider
    • First observedget_rate_limit
    • First observedget_request_analytics
    • First observedget_token_analytics
    • First observedget_trace
    • First observedget_usage_limit
    • First observedget_user
    • First observedget_user_invite
    • First observedget_user_stats
    • First observedget_users_analytics
    • First observedget_virtual_key
    • First observedget_workspace
    • First observedget_workspace_member
    • First observedinsert_log
    • First observedinvite_user
    • First observedlist_all_users
    • First observedlist_api_keys
    • First observedlist_audit_logs
    • First observedlist_collections
    • First observedlist_config_versions
    • First observedlist_configs
    • First observedlist_guardrails
    • First observedlist_integration_models
    • First observedlist_integration_workspaces
    • First observedlist_integrations
    • First observedlist_log_exports
    • First observedlist_partial_versions
    • First observedlist_prompt_labels
    • First observedlist_prompt_partials
    • First observedlist_prompt_versions
    • First observedlist_prompts
    • First observedlist_providers
    • First observedlist_rate_limits
    • First observedlist_traces
    • First observedlist_usage_limits
    • First observedlist_user_invites
    • First observedlist_virtual_keys
    • First observedlist_workspace_members
    • First observedlist_workspaces
    • First observedmigrate_prompt
    • First observedpromote_prompt
    • First observedpublish_partial
    • First observedpublish_prompt
    • First observedremove_workspace_member
    • First observedrender_prompt
    • First observedresend_user_invite
    • First observedrun_prompt_completion
    • First observedstart_log_export
    • First observedupdate_api_key
    • First observedupdate_collection
    • First observedupdate_config
    • First observedupdate_feedback
    • First observedupdate_guardrail
    • First observedupdate_integration
    • First observedupdate_integration_models
    • First observedupdate_integration_workspaces
    • First observedupdate_log_export
    • First observedupdate_prompt
    • First observedupdate_prompt_label
    • First observedupdate_prompt_partial
    • First observedupdate_provider
    • First observedupdate_rate_limit
    • First observedupdate_usage_limit
    • First observedupdate_user
    • First observedupdate_virtual_key
    • First observedupdate_workspace
    • First observedupdate_workspace_member
    • First observedvalidate_completion_metadata

TDQS

A4/5.0
Disambiguation4/5

Most tools have distinct purposes due to consistent verb_noun naming and clear descriptions. However, the large number of analytics tools (e.g., get_error_analytics, get_error_rate_analytics, get_error_stacks_analytics) and similar names for get_user_stats, get_users_analytics, get_analytics_group_users cause some potential confusion, though descriptions help differentiate.

Naming Consistency5/5

Nearly all tools follow a consistent verb_noun pattern (e.g., create_api_key, list_workspace_members, update_prompt_version). There are minor exceptions like validate_completion_metadata, but the overall pattern is very regular.

Tool Count2/5

With 150 tools, the server is extremely large. While it covers a broad admin domain, this number exceeds typical coherence thresholds; many tools could be consolidated or the server scope narrowed.

Completeness5/5

The tool surface is remarkably comprehensive, covering full CRUD for users, workspaces, keys, integrations, promts, configs, analytics, MCP, and more. Obvious lifecycle gaps are minimal; the set supports end-to-end admin workflows.

Maintenance

ActivityActive
ResponsivenessWithin a week

Related MCP Connectors

Related MCP Servers

  • F
    license
    A
    quality
    F
    maintenance
    MCP server that exposes 300+ AI agents as tools via a single API key. Supports listing agents, invoking any agent with chat-completion style messages, checking agent health, and retrieving platform statistics.
    5
    3
    -
  • F
    license
    Not graded
    quality
    D
    maintenance
    Exposes LiteLLM Proxy admin APIs as MCP tools for managing internal users, virtual keys, and spend logs via streamable-http, enabling agents to administer LiteLLM without custom HTTP glue.
    1
    -
  • A
    license
    Not graded
    quality
    D
    maintenance
    MCP server that provides tools to interact with the LiteLLM proxy API, enabling LLM completions, embeddings, image generation, and admin operations.
    18
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/CodesWhat/portkey-admin-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server