Skip to main content
Glama

FleetQ — Open-Source AI Agent Orchestration Platform

Self-hosted mission control for AI agents. Build, run, and monitor autonomous multi-agent systems with a visual DAG builder, human-in-the-loop approvals, MCP server integration, and full audit trail. Works with Claude, GPT-4o, Gemini, Ollama, Codex, Claude Code, and any OpenAI-compatible LLM.

CI License: AGPL v3 PHP Laravel MCP Server

Keywords: AI agents · agent orchestration · MCP server · Model Context Protocol · LangGraph alternative · CrewAI alternative · n8n for AI · Claude agents · LLM workflow · autonomous agents · agent framework · AI automation · self-hosted

☁️ Prefer managed? Try FleetQ Cloud — zero setup, free tier. ⭐ Like the project? Give it a star on GitHub — it helps others find FleetQ.


Table of Contents


Related MCP server: LangChain Agent MCP Server

Why FleetQ?

Most agent frameworks give you a Python notebook. FleetQ gives you a production platform.

  • 🧩 675+ MCP tools across 45 domains — every feature is exposed via Model Context Protocol, so any LLM (Claude Desktop, Cursor, ChatGPT, local agents) can drive the platform programmatically. New in 1.27: web UIs for previously headless capabilities (agent sessions, release signing keys, drift & eval monitors, broadcasts, test suites, CSV import); eight outbound chat channels as first-class drivers; the Agentic AI Flywheel (self-growing eval set + drift/production monitors); policy-governed autonomy (versioned per-agent policies + replay); cost-aware orchestration and Return on Cognitive Spend (ROCS) metrics.

  • 🔁 Visual DAG workflows with 8 node types (agent, conditional, human-task, switch, dynamic-fork, do-while, compensation, sub-workflow) — no Python glue code.

  • 👥 Multi-agent crews with coordinator/worker/reviewer roles, weighted QA scoring, and cross-validation.

  • 🛡️ Real-World Action governance — assistant tool calls, integration writes, and git pushes route through a per-tier risk policy (auto / ask / reject for low / medium / high). Approvals auto-execute. Audit trail attached.

  • 💰 Budget controls with a real credit ledger, pessimistic locking, and auto-pause on overspend — not just token counters.

  • 🧠 Agent evolution — LLM analyzes execution history and proposes config changes you approve with one click.

  • ⚙️ BYOK + Local LLMs — Anthropic, OpenAI, Google, plus Ollama, LM Studio, vLLM, Codex, Claude Code. Zero vendor lock-in.

  • 🔒 Production-grade — tenant isolation, encrypted credential vault, HMAC webhooks, SSRF guards, circuit breakers, audit trail.

  • 📊 OpenTelemetry observability — structured error codes (gRPC-canonical), deadline propagation, distributed tracing. Jaeger UI one-command away. Per-team OTLP collector endpoints for BYO observability.

  • 📈 Live team graph — Cytoscape.js force-directed visualization of agents, humans, and crews. Real-time updates via Laravel Reverb WebSockets.

  • 🏠 Self-host or cloud — MIT-friendly AGPLv3 license, runs on Docker Compose, or use FleetQ Cloud.

Key Concepts

Concept

What it is

When to use

Agent

A configured AI personality with role, goal, backstory, skills, and tool access

The basic unit — one agent per specialized task

Skill

A reusable LLM prompt, rule, connector, or GPU compute call

When multiple agents need the same capability

Experiment

A stateful run through a 20-stage pipeline (scoring → planning → building → executing → evaluating)

Any non-trivial agent task with lifecycle

Crew

A team of agents working on one goal (sequential, parallel, hierarchical, adversarial, fanout, chat-room)

Multi-perspective tasks or when you need review/QA

Workflow

A visual DAG template (reusable across experiments) with branching, loops, human-tasks

Recurring processes — CI/CD, content pipelines, QA flows

Project

A continuous (cron-scheduled) or one-shot container for experiments, with budget + milestones

Long-running initiatives, scheduled agent work

Signal

An inbound event (webhook, RSS, email, bug report, GitHub issue) that can trigger agents

Event-driven automation

MCP Tool

A programmatic action any LLM can call to query or mutate the platform

Expose FleetQ to external agents (Claude, Cursor, etc.)

Screenshots

Dashboard KPI overview with active experiments, success rate, budget spend, and pending approvals.

Agent Template Gallery Browse 14 pre-built agent templates across 5 categories. Search, filter by category, and deploy with one click.

Agent LLM Configuration Per-agent provider and model selection with fallback chains. Supports Anthropic, OpenAI, Google, and local agents.

Agent Evolution AI-driven agent self-improvement. Analyze execution history, propose personality and config changes, and apply with one click.

Crew Execution Live progress tracking during multi-agent crew execution. Each task shows its assigned skill, provider, and elapsed time.

Task Output Expand any completed task to inspect the AI-generated output, including structured JSON responses.

Visual Workflow Builder DAG-based workflow editor with conditional branching, human tasks, switch nodes, and dynamic forks.

Tool Management Manage MCP servers, built-in tools, and external integrations with risk classification and per-agent assignment.

AI Assistant Sidebar Context-aware AI chat embedded in every page with 28 built-in tools for querying and managing the platform.

Experiment Detail Full experiment lifecycle view with timeline, tasks, transitions, artifacts, metrics, and outbound delivery.

Settings & Webhooks Global platform settings, AI provider keys (BYOK), outbound connectors, and webhook configuration.

Error Handling Failed tasks display detailed error information including provider, error type, and request IDs for debugging.

Features

Agents, crews, and workflows

  • AI Agents — role, goal, backstory, personality traits, skill assignments, per-agent provider/model fallback chains

  • Agent Templates — 14 pre-built templates across 5 categories (engineering, content, business, design, research)

  • Agent Evolution — LLM analyzes execution history, proposes config changes, one-click approval

  • Agent Crews — Multi-agent teams with coordinator/QA/worker roles, 7 process types (sequential, parallel, hierarchical, self-claim, adversarial, fanout, chat-room), weighted QA scoring

  • Pre-Execution Scout Phase — cheap LLM pre-call identifies what knowledge the agent needs → targeted semantic search instead of generic recall

  • Step Budget Awareness — agent system prompt targets 80% of allowed steps for core work, reserves the rest for synthesis

  • Experiment Pipeline — 20-state machine with automatic stage progression (scoring → planning → building → approval → executing → metrics → evaluating)

  • Visual Workflow DAG — 8 node types (agent, conditional, human-task, switch, dynamic-fork, do-while, compensation, sub-workflow). Pre-built Web Dev Cycle template. NL → workflow generator.

  • Projects — one-shot and continuous projects with cron scheduling, budget caps, milestones, overlap policies

LLMs and compute

  • BYOK — bring your own keys for Anthropic (Claude), OpenAI (GPT-4o), Google (Gemini)

  • Local LLMs — Ollama, LM Studio, vLLM, llama.cpp via OpenAI-compatible endpoints; 17 preset Ollama models; SSRF protection

  • Local Agents — Codex and Claude Code as execution backends (auto-detected, zero cost)

  • Portkey Gateway — optional drop-in that unlocks 250+ LLM providers with semantic caching and fallbacks

  • RunPod GPU Integration — invoke RunPod serverless endpoints or manage full GPU pod lifecycles as skills; BYOK API key; spot pricing

  • Pluggable Compute Providersgpu_compute skills backed by RunPod, Replicate, Fal.ai, Vast.ai

  • AI Gateway — provider-agnostic via PrismPHP with 6-layer middleware (rate-limit, budget, idempotency, semantic-cache, schema-validation, usage-tracking), circuit breakers, fallback chains

  • Semantic Cache — pgvector-backed cosine similarity (threshold 0.92) cross-team cache — cuts LLM spend on repeat prompts

Signals, triggers, outbound

  • Signal connectors — 20+ drivers: webhook, RSS, IMAP, Slack, Discord, WhatsApp, GitHub, Linear, Jira, PagerDuty, Sentry, Datadog, ClearCue, Telegram, Matrix, Notion, Confluence, Screenpipe, Searxng, more

  • Bug Report signals — lightweight QA pipeline with public JS widget, screenshot + console + network + action log capture, threaded comments (reporter + agent + support), agent delegation, SLA escalation

  • Trigger rules — event-driven automation with condition evaluator, dry-run testing

  • Multi-Channel Outbound — Email (SMTP), Webhook, ntfy plus eight chat channels as first-class drivers (Telegram, Slack, Discord, Microsoft Teams, Google Chat, Matrix, Signal, Supabase Realtime), each with a config page, rate limiting and blacklist

  • Webhooks — inbound (HMAC-SHA256) + outbound (retry, event filtering)

Human-in-the-loop, budgets, security

  • Approvals — inbox with SLA enforcement + escalation

  • Human Tasks — embedded form schemas on workflow nodes

  • Credit Ledger — per-experiment and per-project with pessimistic locking and auto-pause on overspend

  • Credential Vault — encrypted external service credentials with rotation, OAuth2, expiry tracking, per-project injection

  • SSH tools — TOFU (Trust On First Use) fingerprint verification, per-tool allowed-commands whitelist, multi-layer command security policy

  • Audit Trail — full activity log (spatie/activitylog), searchable + filterable

  • Tenant Isolation — multi-layer TeamScope + BelongsToTeam + withoutGlobalScopes() discipline

Integrations & web dev pipeline

  • Integrations — GitHub, Slack, Notion, Airtable, Linear, Stripe, Vercel, Netlify, generic webhook/polling with OAuth 2.0

  • Autonomous Web Dev Pipeline — agents can open PRs, merge, dispatch CI workflows, create releases, trigger Vercel/Netlify/SSH deploys through MCP tools

  • Website Builder — AI-generated static sites with 8 widget types, Vercel + ZIP deployment drivers, form submissions, blog/navigation/contact widgets

  • Founder Mode pack — marketplace bundle of 6 persona agents (Strategist, Product Lead, Growth Hacker, Finance Advisor, Ops Manager, Risk Officer), 20 framework skills (RICE, SPIN, BANT, MEDDIC, OKRs, Shape Up, Unit Economics, Kano, TAM-SAM-SOM, K-Factor, NPV-IRR, RACI, A/B Testing, OWASP), 5 pre-built workflows

  • Marketplace — browse, publish, install shared skills, agents, workflows, and bundles with AI risk scanning

API & MCP surface

  • REST API — 175+ endpoints under /api/v1/ with Sanctum auth, cursor pagination, auto-generated OpenAPI 3.1 at /docs/api

  • MCP Server675+ Model Context Protocol tools across 45 domains (62 tool groups) (stdio + HTTP/SSE + OAuth2/PKCE)

  • Real-World Action governanceActionProposal flow gates assistant tool calls, integration writes, and git pushes through a per-tier risk policy with auto-execute on approval

  • Public discovery endpointGET /.well-known/fleetq returns a config-gated capability manifest so external AI tools can auto-configure

  • Live team graph/team-graph page with real-time updates via Laravel Reverb WebSockets

  • Structured MCP errors — canonical gRPC-style error codes (UNAVAILABLE, PERMISSION_DENIED, RESOURCE_EXHAUSTED, DEADLINE_EXCEEDED, INVALID_ARGUMENT, FAILED_PRECONDITION, NOT_FOUND, INTERNAL) with retryable hints — agents know when to retry vs. fail fast

  • Per-tool deadlines — optional deadline_ms parameter on every MCP tool; agents can bound wall-clock time per call

  • OpenTelemetry tracing — OTLP HTTP exporter, Jaeger all-in-one via docker compose --profile observability up, spans for MCP tool → AI gateway → LLM provider

  • Tool Management — MCP servers (stdio/HTTP), built-in tools (bash/filesystem/browser), risk classification, per-agent assignment

  • MCP client compatibility — Claude Desktop, Claude.ai, ChatGPT Apps, Cursor, Codex, Claude Code, Gemini CLI, any OAuth2 client

Infrastructure

  • Queue Management — Laravel Horizon with 6 priority queues and auto-scaling

  • Testing — regression test suites for agent outputs with automated evaluation

  • Per-Call Working Directory — local/bridge agents can operate in a configured working directory per-agent, isolated project contexts

Use Cases

FleetQ is built for teams running AI agents in production, not toy demos.

  • Autonomous dev pipelines — agent opens PR → CI runs → reviewer agent approves → merge → deploy. Human approves only on risk signals.

  • Customer support triage — bug report widget → agent extracts reproduction steps from console/network log → experiment runs → notifies reporter with fix or agent-generated workaround.

  • Multi-agent research — crew of Strategist + Researcher + Writer with QA reviewer. Each step weighted by domain rubric.

  • Scheduled content ops — continuous project runs daily, each run executes a DAG: draft → review → SEO-check → publish → schedule social.

  • Incident response — PagerDuty/Sentry signal → trigger rule → diagnosis agent → human approval on runbook action → Slack notify.

  • GPU workloads — agent calls gpu_compute skill on RunPod serverless (Whisper, FLUX, Bark) as part of a larger workflow, with cost accounting.

  • Local-first agent dev — Ollama + Codex + Claude Code auto-detected, zero API cost for prototyping; switch to cloud providers for production.

  • Bring FleetQ into Claude — expose your internal data + tools as MCP server, Claude Desktop/ChatGPT/Cursor can drive the platform programmatically.

How FleetQ compares

FleetQ

n8n

CrewAI

LangGraph

Make.com

Open source

✅ AGPLv3

✅ Sustainable Use

✅ MIT

✅ MIT

❌ Proprietary

Visual DAG builder

✅ 8 node types

✅ (not AI-first)

Multi-agent crews

✅ 7 process types

✅ (build-your-own)

MCP server (native)

✅ 675+ tools

Human-in-the-loop

✅ native

⚠️ workaround

⚠️ code

⚠️ code

⚠️ approve-node

Budget ledger + locks

✅ pessimistic

Audit trail

✅ every action

BYOK + local LLMs

✅ both

⚠️ BYOK only

⚠️ depends

⚠️ BYOK

Self-hosted

✅ Docker Compose

n/a (library)

n/a (library)

Agent evolution (self-improve)

OpenTelemetry tracing

✅ native

⚠️ partial

Credit/usage metering

✅ per-team/project

per-workspace

TL;DR — if you're building production agent systems with LLMs and want visual workflows + MCP + human oversight, FleetQ is the only platform that bundles all of it.

Quick Start (Docker)

git clone https://github.com/escapeboy/agent-fleet-o.git
cd agent-fleet
make install

This will:

  1. Copy .env.example to .env

  2. Build and start all Docker services

  3. Run the interactive setup wizard (database, admin account, LLM provider)

Visit http://localhost:8080 when complete.

Quick Start (Manual — Web Setup)

Requirements: PHP 8.4+, PostgreSQL 17+, Redis 7+, Node.js 20+, Composer

git clone https://github.com/escapeboy/agent-fleet-o.git
cd agent-fleet
composer install
npm install && npm run build
cp .env.example .env
# Edit .env — set DB_HOST, DB_DATABASE, DB_USERNAME, DB_PASSWORD, REDIS_HOST
php artisan key:generate
php artisan migrate
php artisan horizon &
php artisan serve

Then open http://localhost:8000 in your browser. The setup page will guide you through creating your admin account.

Alternative: Run php artisan app:install for an interactive CLI setup wizard that also seeds default agents and skills.

Authentication

  • No email verification — the self-hosted edition skips email verification entirely. Accounts are active immediately on registration.

  • Single user — all registered users join the default workspace automatically.

No-Password Mode (local installs)

If you're running FleetQ locally on your own machine and don't want to enter a password on every visit, set APP_AUTH_BYPASS=true in .env:

APP_AUTH_BYPASS=true   # Auto-login as first user
APP_ENV=local          # Required — bypass is disabled in production

With bypass enabled, the app logs you in automatically on every request. A logout link is still shown but you'll be logged back in on the next page load — this is intentional.

Warning: Never set APP_AUTH_BYPASS=true on a server accessible from the internet.

Configuration

All configuration is in .env. Key variables:

# Database (PostgreSQL required)
DB_CONNECTION=pgsql
DB_HOST=postgres
DB_DATABASE=agent_fleet

# Redis (queues, cache, sessions, locks)
REDIS_HOST=redis
REDIS_DB=0          # Queues
REDIS_CACHE_DB=1    # Cache
REDIS_LOCK_DB=2     # Locks

# LLM Providers -- at least one required for AI features
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GOOGLE_AI_API_KEY=

# Auth bypass -- local no-password mode (never use in production)
APP_AUTH_BYPASS=false

Additional LLM keys can be configured in Settings > AI Provider Keys after login.

To use local models (Ollama, LM Studio, vLLM):

LOCAL_LLM_ENABLED=true
LOCAL_LLM_SSRF_PROTECTION=false  # set false if Ollama is on a LAN IP (192.168.x.x)
LOCAL_LLM_TIMEOUT=180

Then configure endpoints in Settings > Local LLM Endpoints.

SSH Host Access

Agents can execute commands on the host machine (or any remote server) via SSH using the built-in SSH tool type. This is useful for running local scripts, interacting with the filesystem, or orchestrating host-level processes from an agent.

How it works

  1. The platform stores SSH private keys encrypted in the Credential vault.

  2. An SSH Tool is configured with host, port, username, credential_id, and an optional allowed_commands whitelist.

  3. On the first connection to a host, the server's public key fingerprint is stored via TOFU (Trust On First Use). Subsequent connections verify the fingerprint — a mismatch raises an error to prevent MITM attacks.

  4. Manage trusted fingerprints via Settings > SSH Fingerprints or the tool_ssh_fingerprints MCP tool.

Setup (Docker — connecting container to host)

The containers reach the host machine via host.docker.internal, which is pre-configured in docker-compose.yml via extra_hosts: host.docker.internal:host-gateway.

Step 1 — Enable SSH on the host

OS

Command

macOS

System Settings → General → Sharing → Remote Login → On

Ubuntu/Debian

sudo apt install openssh-server && sudo systemctl enable --now ssh

Fedora/RHEL

sudo dnf install openssh-server && sudo systemctl enable --now sshd

Windows

Settings → System → Optional Features → OpenSSH Server, then Start-Service sshd

Step 2 — Generate an SSH key pair

ssh-keygen -t ed25519 -C "fleetq-agent@local" -f ~/.ssh/fleetq_agent_key -N ""

Step 3 — Authorize the key on the host

cat ~/.ssh/fleetq_agent_key.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Step 4 — Create a Credential in FleetQ

Navigate to Credentials → New Credential:

  • Type: SSH Key

  • Paste the contents of ~/.ssh/fleetq_agent_key (private key)

Or via API:

curl -X POST http://localhost:8080/api/v1/credentials \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Host SSH Key",
    "credential_type": "ssh_key",
    "secret_data": {"private_key": "<contents of fleetq_agent_key>"}
  }'

Step 5 — Create an SSH Tool

Navigate to Tools → New Tool → Built-in → SSH Remote, or via API:

curl -X POST http://localhost:8080/api/v1/tools \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Host SSH",
    "type": "built_in",
    "risk_level": "destructive",
    "transport_config": {
      "kind": "ssh",
      "host": "host.docker.internal",
      "port": 22,
      "username": "your-username",
      "credential_id": "<credential-id>",
      "allowed_commands": ["ls", "pwd", "whoami", "uname", "date", "df"]
    },
    "settings": {"timeout": 30}
  }'

Step 6 — Assign the tool to an agent

In the Agent detail page, go to Tools and assign the SSH tool. The agent will now have an ssh_execute function available during execution.

Command security policy

The platform enforces a multi-layer security hierarchy for bash and SSH commands:

  1. Platform-level — always blocked: rm -rf /, mkfs, shutdown, reboot, pipe-to-shell patterns

  2. Organization-level — configure in Settings → Security Policy or via the tool_bash_policy MCP tool

  3. Tool-levelallowed_commands whitelist in the tool's transport config

  4. Project-level — additional restrictions in project settings

  5. Agent-level — per-agent overrides on the tool pivot

More restrictive layers always win. A command blocked at the platform level cannot be unblocked by any other layer.

SSH fingerprint management

Trusted host fingerprints are viewable and removable via:

  • API: GET /api/v1/ssh-fingerprints / DELETE /api/v1/ssh-fingerprints/{id}

  • MCP: tool_ssh_fingerprints with list or delete action

Remove a fingerprint when a host's SSH key is legitimately rotated — the next connection will re-verify via TOFU.

Architecture

flowchart LR
    subgraph Clients["Operators & external agents"]
        UI["Admin UI — Livewire 4 + Alpine"]
        MCPCLI["MCP clients (Claude Desktop, Cursor, Codex, Claude Code)"]
        APIC["REST clients — /api/v1/* (Sanctum)"]
        SIG["Inbound signals (webhook / RSS / IMAP / Slack / Telegram)"]
    end

    UI --> WEB
    APIC --> API
    MCPCLI -->|HTTP/SSE or stdio| MCP
    SIG --> INGEST

    subgraph App["FleetQ app (Laravel 13 / PHP 8.4)"]
        WEB["Web routes (auth:web)"] --> DOM
        API["/api/v1/* — Sanctum tokens"] --> DOM
        MCP["AgentFleetServer — 675+ MCP tools / 62 tool groups"] --> DOM
        INGEST["SignalWebhookController / IngestSignalAction"] --> TRIG["TriggerRule evaluator"]
        TRIG --> DOM

        DOM["Domain layer — Agent / Crew / Experiment / Workflow / Project / Approval / Budget / Tool / Credential / Skill / Outbound"]
        DOM --> SM["ExperimentStateMachine (20 states)"]
        SM --> EVT(("ExperimentTransitioned event"))
        EVT --> STAGE["BaseStageJob + PlaybookExecutor"]
        STAGE --> GATEWAY["AI Gateway (PrismPHP) — 6-layer middleware + circuit breakers"]
        GATEWAY --> LLM["Providers: Anthropic / OpenAI / Google / Ollama / vLLM / Codex / Claude Code"]
        STAGE --> TOOLS["ToolTranslator — MCP stdio/HTTP, bash, filesystem, browser, SSH (TOFU)"]
        STAGE --> APPR["ApprovalRequest / HumanTask (auth:web inbox)"]
        STAGE --> OUT["Outbound connectors — Email / Telegram / Slack / Webhook / ntfy"]
        STAGE --> ARTI[("Artifact + ArtifactVersion")]

        DOM --> DB[("Postgres 17 + pgvector — semantic cache, UUIDv7, JSONB+GIN")]
        STAGE --> QUEUE[("Redis 7 — 6 Horizon queues, cache, locks")]
        APPR --> DB
        ARTI --> DB
    end

    subgraph Optional["Optional Docker profiles"]
        REVERB["Reverb — WebSocket live team graph"]
        BROWSER["browserless (Chromium)"]
        SEARX["searxng"]
        VOICE["voice-worker (LiveKit / Deepgram)"]
        SANDBOX["bash_sidecar (sandboxed shell)"]
        RELAY["fleetq-bridge relay"]
        JAEGER["Jaeger — OTLP traces (--profile observability)"]
    end

    App -.OTLP spans.-> JAEGER
    UI <-->|WebSocket| REVERB
    TOOLS -.->|browser tools| BROWSER
    TOOLS -.->|web search skill| SEARX
    TOOLS -.->|bash skill| SANDBOX
    App <-->|relay| RELAY
    DOM <--> VOICE

The platform is a single Laravel 13 monolith that exposes three coequal control surfaces over the same domain layer: the Livewire admin UI, a Sanctum-authenticated REST API at /api/v1/* (~175 endpoints), and AgentFleetServer — an MCP server with 675+ tools across 62 tool groups served over both HTTP/SSE and local stdio. Inbound signals (webhook, RSS, IMAP, Slack, Telegram, and the rest of the 20+ connectors) flow through IngestSignalAction and the TriggerRule evaluator into the domain layer, where the ExperimentStateMachine walks a 20-state pipeline by emitting ExperimentTransitioned events whose listeners dispatch the next BaseStageJob onto Horizon-managed Redis queues. Stage jobs talk to LLMs through the PrismPHP-backed AI Gateway (rate-limit, budget, idempotency, semantic-cache, schema-validation, usage-tracking middleware + circuit breakers + provider fallbacks), invoke Tool instances translated to PrismPHP tool calls (MCP stdio/HTTP, built-in bash/filesystem/browser, SSH with TOFU fingerprints), park ApprovalRequest/HumanTask records for the human-in-the-loop inbox, and persist Artifact versions plus deliver outbound messages over Email/Telegram/Slack/Webhook/ntfy. State and tenant data live in Postgres 17 with pgvector (semantic cache, UUIDv7 primary keys, JSONB+GIN indexes); Redis 7 carries the six Horizon queues, application cache, and pessimistic budget locks. Optional Docker Compose profiles add Reverb for the live team-graph WebSocket, browserless for browser tools, searxng for web search, a voice worker (LiveKit/Deepgram), a sandboxed bash sidecar, the fleetq-bridge relay, and Jaeger for OpenTelemetry tracing via --profile observability.

Built with Laravel 13, Livewire 4, and Tailwind CSS. Domain-driven design with 45 bounded contexts — table below shows the 17 primary domains:

Domain

Purpose

Agent

AI agent configs, execution, personality, evolution

Crew

Multi-agent teams with lead/member roles

Experiment

Pipeline, state machine, playbooks

Signal

Inbound data ingestion

Outbound

Multi-channel delivery

Approval

Human-in-the-loop reviews and human tasks

Budget

Credit ledger, cost enforcement

Metrics

Measurement, revenue attribution

Audit

Activity logging

Skill

Reusable AI skill definitions

Tool

MCP servers, built-in tools, risk classification

Credential

Encrypted external service credentials

Workflow

Visual DAG builder, graph executor

Project

Continuous/one-shot projects, scheduling

Assistant

Context-aware AI chat with 28 tools

Marketplace

Skill/agent/workflow sharing

Integration

External service connectors (GitHub, Slack, Notion, Airtable, Linear, Stripe, Generic)

Docker Services

Service

Purpose

Port

app

PHP 8.4-fpm

--

nginx

Web server

8080

postgres

PostgreSQL 17

5432

redis

Cache/Queue/Sessions

6379

horizon

Queue workers

--

scheduler

Cron jobs

--

vite

Frontend dev server

5173

Common Commands

make start          # Start services
make stop           # Stop services
make logs           # Tail logs
make update         # Pull latest + migrate
make test           # Run tests
make shell          # Open app container shell

Or with Docker Compose directly:

docker compose exec app php artisan tinker       # REPL
docker compose exec app php artisan test          # Run tests
docker compose exec app php artisan migrate       # Run migrations

Upgrading

make update

This pulls the latest code, rebuilds containers, runs migrations, and clears caches.

Tech Stack

  • Framework: Laravel 13 (PHP 8.4)

  • Database: PostgreSQL 17

  • Cache/Queue: Redis 7

  • Frontend: Livewire 4 + Tailwind CSS 4 + Alpine.js

  • AI Gateway: PrismPHP

  • Queue: Laravel Horizon

  • Auth: Laravel Fortify (2FA) + Sanctum (API tokens)

  • Audit: spatie/laravel-activitylog

  • API Docs: dedoc/scramble (OpenAPI 3.1)

  • MCP: laravel/mcp (Model Context Protocol)

Contributing

Contributions are welcome. Please open an issue first to discuss proposed changes.

  1. Fork the repository

  2. Create a feature branch (git checkout -b feat/my-feature)

  3. Make your changes and add tests

  4. Run php artisan test to verify

  5. Submit a pull request

See CONTRIBUTING.md for coding conventions, commit style, and PR checklist.

Community & Support

Star History

If FleetQ saves you time, a ⭐ helps others find it. GitHub ranks repos by star velocity.

License

FleetQ Community Edition is open-source software licensed under the GNU Affero General Public License v3.0.

TL;DR of AGPLv3: You can self-host, modify, and run FleetQ for free — including commercial use. If you offer FleetQ as a hosted service to others, you must open-source your modifications. Questions? See our AGPLv3 FAQ.

Available Tools

34 tools
admin_manageCloud Admin Manage ToolA

Platform super-admin operations — suspend tenants, adjust billing, force-rotate user sessions, browse cross-team security signals. Restricted to users with is_super_admin=true (HTTP 403 for all other callers regardless of team role). Every write is audit-logged with the calling super-admin's id, target id, and reason.

When to use: incident response (suspend a compromised team, revoke leaked sessions), customer-success billing adjustments, or platform-wide security review. Do NOT use for routine tenant operations — those have team-scoped tools.

Actions (pass alongside action):

  • team_suspend (DESTRUCTIVE — disables tenant logins) — team_id (UUID), reason (string). Existing API tokens for the team are invalidated; users see a 403 on all API calls.

  • team_billing (read) — team_id. Returns current invoice + plan summary, including next-cycle estimate.

  • billing_credit (write — moves money) — team_id, amount (positive integer cents), reason (string). Adds to the team's CreditLedger; reflects in the next invoice.

  • billing_refund (DESTRUCTIVE — issues a real Stripe refund) — team_id, amount (cents). Refunds against the most recent successful charge; partial refunds supported.

  • security_overview (read) — recent suspicious activity across all teams: failed logins, token leaks, abuse signals.

  • user_revoke_sessions (DESTRUCTIVE) — user_id (UUID). Invalidates ALL sessions and Sanctum/Passport tokens for the user; they must re-authenticate everywhere.

  • user_send_password_reset (write) — user_id. Emails a password-reset link with a 60-minute one-time token.

Cloud note: feedback management (feedback_list/feedback_update) lives under platform support tooling, not exposed via MCP. Errors: 401, 403 (not super-admin), 404, 422, 429, 503 (Stripe down for billing actions).

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: team_suspend, team_billing, billing_credit, billing_refund, security_overview, user_revoke_sessions, user_send_password_reset
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
team_idYesUUID of the team to suspend or reactivate
reasonNoReason for suspension (required when suspending)
amount_centsYesAmount in cents (negative to add credit, e.g. -1000 = €10 credit)
descriptionNoDescription for the balance transactionAdmin credit
payment_intent_idYesStripe payment_intent ID to refund (e.g. pi_...)
user_idYesUUID of the user whose tokens to revoke

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It details audit logging, destructive side effects (token invalidation, Stripe refunds, session revocation), and error codes. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured with sections, but slightly verbose. Front-loaded purpose and actions list. Earns its length but could be trimmed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, and description only specifies return for team_billing. Other actions lack output description. Covers actions but misses return values for most, leaving a gap for the agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% but description adds behavioral context for each action beyond schema descriptions (e.g., token invalidation for team_suspend). However, some parameters like description and payment_intent_id are not elaborated.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it's for super-admin operations with specific verbs like suspend, adjust, force-rotate, and browse. It distinguishes from sibling tools by noting that routine tenant operations use team-scoped tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly provides when to use (incident response, billing adjustments, security review) and when not (routine tenant ops). Also mentions restriction to super-admin role with HTTP 403 for others.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agent_advancedAgent Advanced ToolA
Destructive

Auxiliary agent operations beyond core CRUD: configuration history, rollback, live runtime inspection, skill/tool wiring, user feedback. For create/update/delete/toggle use agent_manage. Every action requires agent_id; remaining params depend on action.

Actions:

  • config_history (read) — agent_id. Past config snapshots with timestamps.

  • rollback (DESTRUCTIVE) — agent_id, version. Overwrites the current config with the named snapshot — current state is lost unless already snapshotted.

  • runtime_state (read) — agent_id. Last execution status, queue depth, error counters.

  • skill_sync (write) — agent_id, skill_ids[]. Replaces attached skills (full set semantics).

  • tool_sync (write) — agent_id, tool_ids[]. Replaces attached tools (full set semantics).

  • feedback_submit (write) — agent_id, rating (1-5), comment.

  • feedback_list (read) — agent_id. Recent feedback entries.

  • feedback_stats (read) — agent_id. Aggregate score + sentiment.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: config_history, rollback, runtime_state, skill_sync, tool_sync, feedback_submit, feedback_list, feedback_stats
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
agent_idYesThe agent UUID
limitNoMax revisions to return (default 20, max 50)
revision_idYesThe revision UUID to roll back to (restores the before_config of that revision)
skill_idsYesArray of skill UUIDs to attach/detach/sync
modeNoOperation mode: sync (replace all), attach (add), detach (remove). Default: sync
tool_idsYesArray of tool UUIDs to attach/detach/sync
execution_idYesThe AgentExecution UUID to rate
scoreYes1 = positive, -1 = negative, 0 = neutral
commentNoOptional comment explaining the rating
correctionNoOptional correct output (for negative feedback)
labelNoOptional failure category label
daysNoLookback period in days (default 30)

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide destructiveHint: true, but the description adds significant context: 'rollback (DESTRUCTIVE) — ... current state is lost unless already snapshotted.' It also marks other actions as read/write. This goes well beyond the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with bullet points for each action, front-loaded with an overall summary. Every sentence is informative and necessary, with no extraneous text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema, the description hints at return values for each action (e.g., 'Past config snapshots with timestamps,' 'Last execution status, queue depth, error counters'). It covers all actions adequately for an agent to select and use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by explaining action-dependent parameters: 'Every action requires agent_id; remaining params depend on action.' It also elaborates on parameters like mode for skill_sync/tool_sync, which is not obvious from the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a clear summary: 'Auxiliary agent operations beyond core CRUD.' It explicitly lists all actions and distinguishes from the sibling 'agent_manage' tool by stating 'For create/update/delete/toggle use agent_manage.' This provides unambiguous differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool vs. alternatives: 'For create/update/delete/toggle use agent_manage.' It also categorizes each action with (read), (DESTRUCTIVE), or (write), guiding appropriate usage. No additional exclusions needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agent_manageAgent Manage ToolA
Destructive

Core CRUD for AI agents in the caller's team. For runtime inspection, rollback, skill/tool wiring or feedback use agent_advanced. provider and model are validated against team-configured BYOK and local-LLM credentials at create/update time.

Actions:

  • list (read) — optional: status, limit (default 50), cursor.

  • get (read) — agent_id.

  • create (write) — name, role, goal; optional: backstory, provider, model, skill_ids[], tool_ids[].

  • update (write) — agent_id + any creatable field. Partial updates allowed.

  • delete (DESTRUCTIVE) — agent_id, confirm=true. Soft-deletes; recoverable for 30 days.

  • toggle_status (write) — agent_id. Flips active ↔ disabled.

  • templates (read) — pre-built agent templates from the platform catalog.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, update, delete, toggle_status, templates
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: active, disabled
scopeNoFilter by scope: team, personal
limitNoMax results to return (default 10, max 100)
agent_idYesThe agent UUID
nameYesAgent name
roleNoAgent role description
goalNoAgent goal
backstoryNoAgent backstory
providerNoLLM provider key (e.g. anthropic, openai, google, claude-code). Defaults to platform default.
modelNoLLM model name. Defaults to platform default.
personalityNoAgent personality traits: {tone, communication_style, traits[], behavioral_rules[], response_format_preference}
data_classificationNoData classification level: public, internal, confidential, restricted. Confidential and restricted agents are routed to local-only providers.
tool_profileNoTool profile restricting tool access. Options: researcher, executor, communicator, analyst, admin, minimal
environmentNoEnvironment preset that auto-attaches a tool bundle. Options: minimal, coding, browsing, restricted.
reasoning_effortNoExtended thinking effort (Anthropic). Options: none, low, medium, high, auto. "auto" lets the platform pick by task complexity.
use_tool_searchNoEnable semantic tool auto-discovery — at run time, up to tool_search_top_k matching tools from the team pool are auto-attached based on the user prompt.
tool_search_top_kNoMaximum tools tool_search will surface per run (1–20, default 5). Only applies when use_tool_search=true.
sandbox_profileNoJSON string defining Docker sandbox profile for per-execution process isolation (enterprise only). Example: {"image":"python:3.12-alpine","memory":"512m","cpus":"1.0","network":"none","timeout":300}
knowledge_base_idNoUUID of a knowledge base to link to this agent for RAG-powered context
evaluation_enabledNoEnable A/B evaluation for this agent
evaluation_sample_rateNoFraction of requests to include in evaluation (0.0 to 1.0). Only used when evaluation_enabled is true.
heartbeat_definitionNoAgent health check config: {enabled: bool, cron: string, prompt: string}
budget_cap_creditsNoPer-agent budget cap in credits. Set to 0 to remove cap.
thinking_budgetNoAnthropic extended thinking budget in tokens (e.g. 1024, 4096, 8192). Only applies when agent provider is "anthropic". Set to 0 to disable. Enables chain-of-thought reasoning visible in experiment steps.
confirmYesMust be true to confirm deletion. This is a destructive action.
categoryNoFilter by category: engineering, content, business, design, research

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only provide `destructiveHint=true`. The description goes well beyond by detailing that delete is destructive and performs a soft-delete recoverable for 30 days, mentions validation of provider/model at creation/update, and notes that partial updates are allowed. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and sibling differentiation. It then uses a concise bullet list to detail each action, with no extraneous information. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having 28 parameters, no output schema, and complex nested objects, the description covers all key behaviors: CRUD actions, validation, soft-delete, and defaults. It gives sufficient context for an AI agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, but the description adds value by grouping parameters into actions, explaining defaults (limit default 10, max 100), and clarifying that `confirm=true` is required for deletion. It also states validation of provider/model at create/update time, which is not in the schema. This goes beyond baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Core CRUD for AI agents in the caller's team' and enumerates all seven actions (list, get, create, update, delete, toggle_status, templates). It also distinguishes itself from the sibling tool `agent_advanced` by specifying what that sibling is for, making purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool versus `agent_advanced` ('For runtime inspection, rollback, skill/tool wiring or feedback use `agent_advanced`'). It also gives a brief overview of each action, guiding the agent on when to invoke which action.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

approval_manageApproval Manage ToolA
Destructive

Human-in-the-loop approvals and workflow human-task completion. Use this to unblock workflow steps gated on reviewer decisions or to submit form data for human_task DAG nodes. Each decision is audit-logged and emits a domain event the workflow runtime listens for.

Actions:

  • list (read) — optional: status (pending/approved/rejected/expired), assignee_id, limit.

  • approve (write) — approval_id, optional comment. Unblocks the dependent step.

  • reject (write) — approval_id, reason. Terminates the dependent step (workflow may branch on rejection).

  • complete_human_task (write) — approval_id, form_data (JSON matching the node's form_schema). Validates against the schema before commit.

  • webhook_config (write) — approval_id, webhook_url. Configures external notification when status changes.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, approve, reject, complete_human_task, webhook_config
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: pending, approved, rejected, expired (default: pending)pending
limitNoMax results to return (default 10, max 100)
approval_idYesThe approval request UUID
notesNoOptional reviewer notes
reasonYesReason for rejection
form_responseYesThe form response data as key-value pairs matching the form_schema fields
callback_urlNoHTTPS URL to POST the decision payload to (set to null to clear)
callback_secretNoSecret used to sign the payload via HMAC-SHA256 (X-Signature-SHA256 header)

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds detail beyond destructiveHint annotation: audit-logging, domain events, and action consequences (approve unblocks, reject terminates). No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Efficient structure: opening sentence defines purpose, then bullet points for each action with parameter details. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite 10 parameters, no output schema, and multiple actions, the description covers all necessary context: when to use, per-action parameters, outcomes, and side effects. Agent can correctly select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so baseline 3, but description adds meaning by associating parameters with specific actions and clarifying usage (e.g., comment optional for approve, reason required for reject, form_data must match schema). Overcomes schema's over-required fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly defines tool as managing approvals and human-task completion. Explicitly lists actions (list, approve, reject, complete_human_task, webhook_config) and their effects, distinguishing from sibling manage tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States when to use: unblock workflow steps or submit form data for human_task DAG nodes. Describes effects of each action but does not explicitly mention alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

artifact_manageArtifact Manage ToolA
Destructive

Read-only access to artifacts produced by experiments, crew executions, and project runs (code, documents, data, media). Each artifact has 1..N ArtifactVersion entries; version defaults to latest when omitted. download returns a signed URL valid for ~5 minutes; content returns the body inline (base64 for binary MIMEs).

Actions:

  • list (read) — exactly one of: experiment_id, crew_execution_id, project_run_id; optional limit.

  • get (read) — artifact_id. Metadata only (size, mime, version count, created_by AiRun id).

  • content (read) — artifact_id; optional version. Returns raw or base64 body.

  • download (read) — artifact_id; optional version. Returns short-lived signed URL.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, content, download
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
experiment_idNoFilter by experiment UUID
crew_execution_idNoFilter by crew execution UUID
project_run_idNoFilter by project run UUID
limitNoMax results (default 20, max 100)
artifact_idYesThe artifact UUID
versionNoSpecific version number (default: latest)
include_contentNoInclude full content in response (default true). Set false for metadata only.

TDQS

A3.8/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructiveHint: true, but the description asserts 'Read-only access.' This is a direct contradiction: read-only tools should not have destructiveHint. The description fails to disclose why it is marked destructive, misleading the agent about potential side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Description is front-loaded with purpose, followed by a bulleted list of actions with precise details. Every sentence is informative; no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite strong parameter documentation, the glaring contradiction between 'read-only' description and destructiveHint annotation undermines completeness. The agent lacks understanding of whether this tool can cause harm. Otherwise, the description would be complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% coverage, but the description adds significant value: explains default version behavior, signed URL expiry (~5 min), content encoding for binary types, and clarifies each action's output. This goes well beyond the schema definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Read-only access to artifacts' and enumerates four specific read actions (list, get, content, download). It effectively communicates the tool's purpose and distinguishes it from other management tools that likely support write operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage scenarios: retrieving artifacts and their versions. It provides context for when to use this tool (read-only operations) but does not explicitly state when not to use it or mention alternatives among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

assistant_manageAssistant Manage ToolA
Destructive

FleetQ AI assistant conversations — the in-app chat panel that can call MCP tools on the user's behalf with role-gated authorization (read for all, write for Member+, destructive for Admin/Owner). Conversations bind to a context object (experiment, project, agent, crew, workflow) on first message.

Actions:

  • conversation_list (read) — optional: limit, context_type filter.

  • conversation_get (read) — conversation_id. Full history including tool_calls / tool_results.

  • send_message (write) — message; optional: conversation_id (omit to start new), context_type, context_id, attachments[]. Triggers a synchronous tool-loop LLM call; consumes team credits.

  • conversation_clear (DESTRUCTIVE) — conversation_id. Erases all messages, retains the conversation shell.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: conversation_list, conversation_get, send_message, conversation_clear
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
limitNoMax results (default 20, max 50)
conversation_idYesConversation UUID
messageYesThe message to send to the assistant
context_typeNoContext binding: experiment | project | agent | crew | workflow
context_idNoUUID of the bound context entity

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description details authorization levels per action, notes that send_message triggers a synchronous tool-loop LLM call consuming team credits, and explains that conversation_clear erases all messages. This adds valuable behavioral context not captured in annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured with bullet points for each action. It front-loads the purpose and efficiently covers authorization, parameters, and side effects without unnecessary verbosity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is thorough for a tool with multiple actions, covering purpose, authorization, parameter behaviors, and destructive effects. However, it lacks details on return values for actions like list/get, which would improve completeness for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% description coverage, but the description adds usage context (e.g., omit conversation_id to start new, optional limit and context_type filter for conversation_list). This provides meaningful guidance beyond the schema itself.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool manages FleetQ AI assistant conversations, listing specific actions (conversation_list, conversation_get, send_message, conversation_clear). It distinguishes itself from sibling tools by focusing on the in-app chat panel that can call MCP tools, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use each action based on authorization roles (read for all, write for Member+, destructive for Admin/Owner). However, it does not explicitly exclude alternatives among sibling tools or specify when not to use this tool, missing a small opportunity for clearer guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

boruna_manageBoruna Manage ToolA
Destructive

Boruna deterministic capability-safe .ax script runtime (v1.x LTS) — execute, validate, and audit Boruna scripts. Side effects depend entirely on the script's declared capabilities. Requires an active mcp_stdio Tool record pointing to a working Boruna binary; otherwise every action returns dependency_unavailable.

Actions:

  • run (write — side effects per script capabilities) — inline .ax source OR skill_id of a saved boruna_script skill; optional: timeout_ms, mem_limit_mb.

  • validate (read) — .ax source. Syntax + semantic check, no execution.

  • policy_validate (read; v0.4.0+) — policy JSON. Strict schema validation.

  • evidence (read) — run_id. Capability/effect evidence record for an executed run.

  • capability_list (read) — registered capabilities and capability_set_hash.

  • skill_manage (write) — sub-actions list/get/create/update/delete on boruna_script-typed skills.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: run, validate, policy_validate, evidence, capability_list, skill_manage
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
modeYesExecution mode: inline (run script directly) or skill (run a saved boruna_script skill)
scriptNo(inline mode) The .ax script source code to execute
policyNo(inline mode) Legacy capability policy shorthand: "allow-all" or "deny-all" (default: deny-all). For Boruna v0.2.0+ fine-grained gating, use policy_structured instead.
policy_structuredNo(inline mode, Boruna v0.2.0+) Structured Capability Policy object with required default_allow (bool), optional rules (per-capability {allow, budget}), and optional net_policy (allowed_domains, allowed_methods, max_response_bytes, timeout_ms, allow_redirects). Capability keys: net.fetch, fs.read, fs.write, db.query, ui.render, time.now, random, llm.call, actor.spawn, actor.send. When set, takes precedence over the legacy policy parameter. See https://github.com/escapeboy/boruna/blob/v0.2.0/docs/reference/policy-schema.md.
boruna_tool_idNo(inline mode) UUID of the mcp_stdio Tool pointing to the Boruna binary. If omitted, auto-detects.
inputNoOptional input data passed to the script as JSON
limitsNo(inline mode, v1.0+) Optional resource limits: max_wall_ms (int) and/or max_output_bytes (int)
skill_idNo(skill mode) UUID of the boruna_script Skill to execute
policy_jsonYesJSON-encoded Boruna policy object to validate. Must have a top-level default_allow boolean. Example: {"default_allow":false,"rules":{"net.fetch":{"allow":true}}}
execution_idYesUUID of the SkillExecution record from a boruna_script skill run
nameNo(create) Skill name
descriptionNo(create) Skill description
limitNo(list | executions) Max results (default 20)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that side effects depend on script capabilities, notes the 'dependency_unavailable' error, and categorizes actions as 'write' (run) or 'read' (others). This aligns with the destructiveHint: true annotation and adds context beyond it, though detailed effects per script are left to the script's capabilities.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is lengthy but well-structured with bullet points for each action and clear parameter groupings. It is not overly verbose given the complexity of the tool, and every sentence adds useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 15 parameters, multiple actions, no output schema, and complex dependencies, the description covers actions, error conditions, prerequisite setup, and parameter usage. It references external documentation for further details. Minor gaps: no description of return values for each action.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage, but the description adds meaning by grouping parameters per action (e.g., inline vs skill mode), explaining the relationship between 'policy' and 'policy_structured', and specifying which parameters are relevant for each action. This goes beyond the schema's property descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool executes, validates, and audits Boruna scripts, and lists specific actions (run, validate, etc.) with their purposes. It distinguishes from sibling tools (e.g., admin_manage, agent_manage) by focusing on Boruna-specific functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use each sub-action (e.g., 'run for execution, validate for syntax check') and mentions a prerequisite (mcp_stdio Tool record). However, it does not explicitly compare this tool to alternatives or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bridge_manageBridge Manage ToolA
Destructive

FleetQ Bridge — a WebSocket relay that lets cloud experiments reach private endpoints on a self-hosted runner (laptop, on-prem VPS): local Ollama instances, internal MCP servers, on-prem APIs. All actions operate on the team's currently-registered bridge.

Actions:

  • status (read) — connection state, last heartbeat, registered endpoint count.

  • endpoint_list (read) — local LLM agents + MCP servers announced by the bridge.

  • endpoint_toggle (write) — endpoint_id, enabled (bool). Flips visibility to cloud experiments without redeploying the bridge.

  • disconnect (DESTRUCTIVE) — terminates the active bridge session; the runner must re-register before agents can reach private endpoints again.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: status, endpoint_list, endpoint_toggle, disconnect
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
typeNoFilter by endpoint type. Defaults to all.
endpoint_idYesThe ID of the endpoint to toggle (from bridge_endpoint_list).
enabledNoWhether to enable (true) or disable (false) the endpoint.
connection_idNoUUID of a specific bridge connection to disconnect. Omit to disconnect all.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description labels actions as read/write/destructive and explains that disconnect terminates the bridge session requiring re-registration. This adds useful behavioral context about side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a brief overview followed by bullet-pointed actions. It is concise but informative, with few wasted words. Could be slightly more terse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers main actions but lacks details on return values for read actions, error handling, and does not clarify that endpoint_id is required in schema but only relevant for specific actions. This leaves gaps for an agent to fully utilize the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description does not add new semantics for parameters beyond listing actions; it mentions endpoint_id in context of toggle but schema already covers it. No extra value added.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose as a WebSocket relay for cloud experiments to reach private endpoints, and lists specific actions (status, endpoint_list, endpoint_toggle, disconnect) with precise verbs and resources. It distinguishes itself from siblings by its unique domain of bridge/private endpoint management.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for managing the bridge but does not explicitly state when to use this tool vs alternatives, nor does it provide when-not-to-use guidance. Given the many sibling tools, explicit differentiation would enhance this dimension.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

budget_manageBudget Manage ToolB
Destructive

Team-wide credit budget overview, pre-flight cost guards, and spend forecasting. Read-only — does not move money. 1 credit ≈ $0.001 USD; balances and reservations track LLM + outbound + compute spend with pessimistic locking. Use check before dispatching any expensive job to avoid mid-run pause-on-budget-exhausted events.

Actions:

  • summary (read) — current balance, pending reservations, MTD spend by category (llm/outbound/compute).

  • check (read) — estimated_cost (credits). Returns pass/fail without reserving funds.

  • forecast (read) — period (week|month|quarter), granularity (day|week). Projects spend by extrapolating recent ledger entries.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: summary, check, forecast
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
amountNoAmount of credits to check availability for. If not provided, returns general availability.

TDQS

B3.4/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description claims the tool is read-only ('does not move money'), but the annotation destructiveHint: true indicates the tool may be destructive. This contradicts the description, severely undermining transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with a clear overview and bullet list of actions. It is slightly verbose but earns each sentence. Front-loads the key purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema is provided, so the description must explain return values. It partially does (check returns pass/fail, forecast projects spend), but lacks detail on output structure for summary. Adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds some context (e.g., check returns pass/fail), but most parameter meaning is already in schema descriptions. No significant added value beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it provides budget overview, pre-flight cost guards, and spend forecasting. It lists specific actions (summary, check, forecast) and explicitly says it is read-only, distinguishing it from other 'manage' sibling tools that likely perform mutations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises using the 'check' action before expensive jobs to avoid budget exhaustion events, providing clear usage context. However, it does not mention when not to use the tool or alternatives among siblings, slightly reducing clarity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

chatbot_manageChatbot Manage ToolA
Destructive

Embeddable chatbots backed by an existing FleetQ agent. Each instance issues per-widget tokens for embedding, tracks sessions, and exposes analytics + learning entries. Conversations consume team credits via the bound agent's provider.

Actions:

  • list / get (read) — list all or fetch one (chatbot_id).

  • create (write) — name, agent_id, config (theme, greeting, allowed_origins[]).

  • update (write) — chatbot_id + any creatable field.

  • delete (DESTRUCTIVE) — chatbot_id. Cascades — also revokes all widget tokens.

  • toggle_status (write) — chatbot_id. Flips active ↔ disabled.

  • token_create (write) — chatbot_id. Returns an embeddable widget token (display once).

  • token_revoke (DESTRUCTIVE) — chatbot_id, token_id. Invalidates a single widget instance.

  • session_list / analytics / learning_entries (read) — chatbot_id; optional date range.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, update, delete, toggle_status, token_create, token_revoke, session_list, analytics, learning_entries
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: active, inactive, draft, suspended
limitNoMax results (default 20, max 100)
idYesChatbot UUID or slug
nameYesChatbot display name
typeYesChatbot type
system_promptYesSystem prompt for the backing agent
providerNoLLM provider (default: anthropic)anthropic
modelNoLLM model (default: claude-haiku-4-5)
descriptionNoOptional description
welcome_messageNoWelcome message shown on first open
confidence_thresholdNoConfidence threshold for escalation (0.0-1.0, default 0.7)
human_escalation_enabledNoEnable human escalation for low-confidence responses
workflow_idNoOptional workflow UUID to delegate message processing
approval_timeout_hoursNoHours before escalated approval expires (default 48)
fallback_messageNoFallback message for escalated responses
widget_configNoWidget config: {position, theme_color, title}
chatbot_idYesThe chatbot UUID
rotate_existingNoIf true, existing active tokens get a 48-hour expiry grace period before being invalidated. Default: false
token_idYesThe chatbot token UUID (returned by chatbot_token_create)
channelNoFilter by channel: web_widget, api, telegram, slack
daysNoNumber of days to look back (default 30, max 90)

TDQS

A3.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes far beyond the destructiveHint annotation by explicitly labeling actions as read, write, or destructive, detailing cascading effects (e.g., delete revokes all widget tokens), and noting credit consumption. This provides rich behavioral context that annotations alone do not cover.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a concise overview, a bulleted action list, and clear categorization. It is slightly long due to the number of actions, but each sentence serves a purpose. Minor redundancy could be trimmed without losing clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 23 parameters and 11 actions, the description covers the core aspects: action types, destructive behavior, credit usage, and optional filters. It does not describe output format or pagination details, but these are partially covered by the input schema. Overall, it provides sufficient context for safe invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the parameter semantics are already well-documented in the input schema. The tool description adds no additional parameter-specific details beyond listing actions, so it meets the baseline but does not surpass it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool manages 'Embeddable chatbots backed by an existing FleetQ agent' and lists all actions. While it differentiates from other tool management siblings by focusing on chatbots, it does not explicitly contrast with sibling tools like agent_manage or assistant_manage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus other similar manage tools. The description lacks any 'when to use' or 'when not to use' context, leaving the agent to infer from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

credential_manageCredential Manage ToolA
Destructive

Encrypted credential vault for external services (API keys, OAuth2 tokens, basic auth, bearer tokens). Secrets are encrypted at rest with the team's per-tenant key; secret_data is never returned by get — only metadata (name, type, expires_at, last_rotated_at).

Actions:

  • list (read) — optional: type, status filter.

  • get (read) — credential_id. Metadata only, secrets redacted.

  • create (write) — name, type (api_key/oauth2/basic_auth/bearer_token/custom), secret_data (object).

  • update (write) — credential_id + any creatable field.

  • delete (DESTRUCTIVE) — credential_id. Hard delete; not recoverable.

  • rotate (write) — credential_id, new_secret_data. Bumps last_rotated_at and re-encrypts.

  • oauth_initiate (write) — provider, scopes[]. Returns authorization URL.

  • oauth_finalize (write) — provider, code (from OAuth callback). Stores tokens, returns credential_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, update, delete, rotate, oauth_initiate, oauth_finalize
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: active, disabled, pending_review
creator_sourceNoFilter by creator source: human, agent, system
limitNoMax results to return (default 10, max 100)
credential_idYesThe credential UUID
nameYesCredential name
typeYesCredential type: api_token, oauth2, basic_auth, ssh_key, custom_kv
secret_dataYesSecret data object (e.g. {"token": "..."} or {"username": "...", "password": "..."})
descriptionNoCredential description
expires_atNoExpiration date in ISO 8601 format (e.g. 2025-12-31T23:59:59Z)
agent_idNoUUID of the agent creating this credential. Sets creator_source=agent and status=pending_review until a human approves it.
service_nameYesHuman-readable name of the service being authorized (e.g. "GitHub", "Slack")
scopesNoList of OAuth scopes to request (e.g. ["read:user", "repo"])
credential_nameNoName to give the stored credential once OAuth completes
correlation_idYesThe correlation_id returned by credential_oauth_initiate

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description details encryption at rest, that secret_data is never returned by get, hard delete irrecoverability, and OAuth workflow. This adds substantial behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is relatively long with multiple paragraphs. While it front-loads the main purpose and uses bullet points for actions, some redundancy exists (e.g., repeating action types also in schema). Could be more concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers actions and key behaviors but lacks return value descriptions for most actions (e.g., list, create). Since no output schema exists, this omission reduces completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed parameter descriptions. The description lists actions with associated parameters but does not add new meaning beyond the schema, so baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool as an 'Encrypted credential vault for external services' and lists all actions. This distinguishes it from sibling management tools which handle different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for credential management but does not explicitly exclude alternative tools or provide when-not-to-use guidance. Since no other credential tools exist, the context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

crew_manageCrew Manage ToolA
Destructive

Multi-agent crews — coordinated teams of agents that decompose a goal across roles (sequential, parallel, or hierarchical process). execute is async: it queues ExecuteCrewJob and returns immediately with an execution_id; poll execution_status for progress. Each crew member must be a real agent_id in the same team.

Actions:

  • list / get (read) — list all or fetch one (crew_id).

  • create (write) — name, process_type (sequential|parallel|hierarchical), agents[] (array of {agent_id, role}).

  • update (write) — crew_id + any creatable field.

  • delete (DESTRUCTIVE) — crew_id. Soft-deletes the crew.

  • execute (write — long-running) — crew_id, goal. Reserves budget, returns execution_id.

  • execution_status (read) — crew_id, execution_id. Status, current task, partial results.

  • executions_list (read) — crew_id; optional limit.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, update, execute, execution_status, executions_list
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: draft, active, archived
limitNoMax results to return (default 10, max 100)
crew_idYesThe crew UUID
nameYesCrew name
coordinator_agent_idYesUUID of the coordinator agent
qa_agent_idNoUUID of the QA agent. Optional — when omitted, the coordinator reviews their own work (solo-mode crew).
descriptionNoCrew description
process_typeNoProcess type: sequential, parallel, hierarchical (default: hierarchical)hierarchical
convergence_modeNoHow to determine when the crew is done: any_validated (default), all_validated, threshold_ratio, quality_gate
min_validated_ratioNoFraction of tasks that must be validated when using threshold_ratio mode (e.g. 0.8 = 80%). Default: 1.0
goalYesThe goal/task for the crew to accomplish
execution_idYesThe crew execution UUID
include_full_outputNoInclude full final_output instead of 500-char preview (default false)

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description discloses behavioral traits: execute is async, returns execution_id, polls via execution_status, includes soft-delete, and reserves budget. This adds significant behavioral context beyond minimal annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear top-level explanation and bulleted actions. It is concise given the tool's complexity, though some redundancy exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the 15 parameters, no output schema, and multiple actions, the description provides a fairly complete picture. It covers all actions, async nature, and constraints. Missing details on return values and some parameters, but overall adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds context about async behavior and soft-delete for some actions. However, it mentions an 'agents[]' array in the create action that does not correspond to a schema parameter (instead using coordinator_agent_id and qa_agent_id), which may confuse.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool manages multi-agent crews with actions like CRUD and execution. It distinguishes the tool by focusing on crew coordination, but does not explicitly differentiate from sibling tools like team_manage or agent_manage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides guidance on async execution and polling, and mentions that agents must be real IDs. However, it lacks explicit when-to-use or when-not-to-use compared to alternatives. It implies usage but does not give clear exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

email_manageEmail Manage ToolA
Destructive

Email themes (visual styling) and templates (transactional + marketing copy). Templates can be hand-written MJML/HTML or AI-generated via template_generate (consumes credits). Deleting a theme used by templates is blocked unless those templates are deleted first.

Theme actions:

  • theme_list / theme_get (read).

  • theme_create (write) — name, styles (object: colors, fonts, spacing).

  • theme_update (write) — theme_id + any creatable field.

  • theme_delete (DESTRUCTIVE) — theme_id. Fails if any template references it.

Template actions:

  • template_list / template_get (read).

  • template_create (write) — name, subject, body (MJML/HTML); optional theme_id.

  • template_update (write) — template_id + any creatable field.

  • template_delete (DESTRUCTIVE) — template_id.

  • template_generate (write — costs credits) — prompt. Calls the team's default LLM to produce a template, returns draft for review.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: theme_list, theme_get, theme_create, theme_update, theme_delete, template_list, template_get, template_create, template_update, template_delete, template_generate
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: draft, active, archived
limitNoMax results (default 10, max 100)
idYesEmail theme UUID
nameYesTheme name
primary_colorNoPrimary/CTA color as hex (e.g. #2563eb). Default: #2563eb
background_colorNoEmail background color as hex. Default: #f4f4f4
canvas_colorNoContent area background color as hex. Default: #ffffff
text_colorNoBody text color as hex. Default: #1f2937
heading_colorNoHeading text color as hex. Default: #111827
font_nameNoFont display name (e.g. Inter, Georgia). Default: Inter
font_urlNoGoogle Fonts or web font URL for @import
font_familyNoFull CSS font-family stack. Default: Inter, Arial, sans-serif
logo_urlNoAbsolute URL to the team logo image
email_widthNoMaximum email width in pixels. Default: 600
company_nameNoCompany name shown in email footer
company_addressNoCompany address shown in email footer
footer_textNoFooter text or HTML (e.g. unsubscribe line)
muted_colorNoMuted/secondary text color as hex
divider_colorNoDivider/border color as hex
heading_font_sizeNoHeading font size in pixels
body_font_sizeNoBody font size in pixels
logo_widthNoLogo width in pixels
content_paddingNoContent area horizontal padding in pixels
visibilityNoFilter by visibility: private, public
subjectNoEmail subject line
preview_textNoShort preview text shown in email clients
email_theme_idNoOptional email theme UUID to associate
html_bodyNoRaw HTML content. Stored directly as the template HTML.
mjml_bodyNoComplete MJML document starting with <mjml>. Compiled server-side to cross-client HTML. Preferred over html_body.
descriptionYesNatural language description of the email to generate, e.g. "Monthly newsletter with hero image, 3 product highlights, and a CTA button"
theme_idNoOptional email theme UUID. Brand colors, fonts, company name, and logo from the theme will be injected into the generated MJML.
toneNoWriting tone: professional, friendly, or minimal (default: professional)

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes beyond the destructiveHint annotation by explicitly labeling theme_delete and template_delete as destructive and explaining the blockage when a theme is in use. It also discloses that template_generate consumes credits and compiles MJML server-side. This level of detail exceeds the annotation's bare hint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is moderately long but well-structured into paragraphs for general info, theme actions, and template actions. Each sentence carries relevant information (e.g., credit consumption, MJML compilation). Minor redundancy could be trimmed, but overall it's efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 34 parameters and no output schema, the description covers key workflows: dependencies (theme deletion blocked), credit costs for generation, and the difference between MJML and HTML storage. It lacks details on error responses or pagination for list actions, but it adequately addresses the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage, the baseline is 3. The description adds value by grouping parameters into actions (e.g., theme_create uses name and styles object) and explaining parameter relationships (e.g., mjml_body preferred over html_body, theme_id injects brand colors into generated MJML). It also notes default values for several fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it manages email themes (visual styling) and templates (transactional + marketing copy). It lists specific actions for themes and templates, including AI generation via template_generate. This distinguishes it from sibling tools, which have different domains (e.g., admin_manage, credential_manage).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context for when to use each action, such as distinguishing theme operations from template operations and noting that template_generate consumes credits. It also explains a dependency: deleting a theme fails if templates reference it. While it doesn't explicitly compare to alternatives (e.g., outbound_manage for sending), the guidance is clear for the tool's scope.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

evolution_manageEvolution Manage ToolA
Destructive

AI-generated improvement proposals — the platform analyzes recent agent runs and suggests prompt tweaks, model swaps, skill additions. Proposals must be reviewed (analyze), then either apply (mutates the target agent/skill) or reject. apply is irreversible without a manual rollback through agent_advanced.rollback.

Actions:

  • list (read) — optional: status (pending/applied/rejected), target_type, limit.

  • analyze (read) — proposal_id. Returns LLM-generated rationale, confidence score, diff preview.

  • approve (write) — proposal_id. Marks as approved without applying (queue for batch apply).

  • apply (DESTRUCTIVE) — proposal_id. Mutates the target entity in place; rollback only via config_history snapshot.

  • reject (write) — proposal_id, reason. Closes the proposal.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, analyze, approve, apply, reject
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
agent_idYesFilter by agent ID (required)
statusNoFilter by status: pending, approved, applied, rejected
limitNoMax results (default 10, max 50)
proposal_idYesThe evolution proposal UUID to approve
reasonNoOptional reason for rejection

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description explicitly warns that apply is destructive and irreversible without manual rollback, going beyond the destructiveHint annotation. It also describes read vs. write actions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, well-structured with bullet points for actions, and front-loads the purpose. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the workflow, actions, and rollback details. No output schema exists, but it mentions what analyze returns. Sufficient for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with descriptions for all parameters. The description adds context for the action parameter's values, but the schema already handles semantics well.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it manages AI-generated improvement proposals and lists specific actions (list, analyze, approve, apply, reject). It differentiates from siblings by its focus on evolution proposals.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains when to use each action (e.g., analyze returns rationale, apply is destructive) and mentions rollback via another tool. It lacks explicit comparisons to alternatives but provides sufficient context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

experiment_manageExperiment Manage ToolA
Destructive

Experiments — the platform's core unit of work. Each experiment runs a workflow (DAG) through a 20-state machine: Draft → Scoring → Planning → Building → AwaitingApproval → Approved → Executing → CollectingMetrics → Evaluating → (Iterating | Completed). Lifecycle transitions are validated by ExperimentTransitionMap; use valid_transitions to discover what's currently allowed.

Actions:

  • list (read) — optional: status, workflow_id, limit.

  • get (read) — experiment_id.

  • create (write) — name, hypothesis; optional workflow_id (else uses default workflow).

  • start (write) — experiment_id. Transitions Draft → Scoring; reserves budget.

  • pause / resume (write) — experiment_id. Pause holds at the current stage.

  • retry (write) — experiment_id. Re-runs the failed stage.

  • retry_from_step (write) — experiment_id, step_id. Graph-aware BFS reset of step + downstream.

  • kill (DESTRUCTIVE) — experiment_id. Terminal; cannot resume.

  • valid_transitions (read) — experiment_id. Allowed next states for current state.

  • cost / steps / share (read) — experiment_id. Cost breakdown / step list / public share token.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, start, pause, resume, retry, retry_from_step, kill, valid_transitions, cost, steps, share
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: draft, scoring, planning, building, executing, completed, killed, paused, etc.
limitNoMax results to return (default 10, max 100)
experiment_idYesThe experiment UUID
titleYesExperiment title
thesisNoExperiment thesis/hypothesis
trackNoExperiment track: growth, retention, revenue, engagement (default: growth)growth
budget_cap_creditsNoBudget cap in credits (default: 10000)
step_idYesThe playbook step UUID to retry from
reasonNoReason for killing the experiment
show_costsNoWhether to show cost data in the public view (for update action)
show_stagesNoWhether to show pipeline stages in the public view (for update action)
show_outputsNoWhether to show stage outputs in the public view (for update action)
expires_atNoISO8601 expiry datetime after which the share link is invalid. Pass null to remove expiry. (for update action)

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds significant behavioral context beyond the annotations: it details the state machine, lifecycle transitions, and notes that `kill` is destructive and terminal. It also mentions budget reservation on `start` and graph-aware BFS reset for `retry_from_step`. While annotations already indicate `destructiveHint: true`, the description enriches the behavioral model.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear opening statement about experiments and a bullet list of actions. It is informative without being overly verbose, though the state machine detail could be considered slightly heavy for a function description. Overall, it is concise and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (15 parameters, 4 required, no output schema), the description fails to explain return values or error states for actions. While it covers actions and states, it does not describe expected outputs (e.g., what `list` returns, cost breakdown format). This is a notable gap for a tool with no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the schema already documents all parameters. The description provides a high-level mapping of actions to required parameters (e.g., `list` uses optional `status`, `workflow_id`, `limit`), but does not add deeper semantic meaning beyond what the schema offers. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states that the tool manages experiments, which are the core unit of work. It lists all actions (list, get, create, start, etc.) and explains the 20-state machine, making the tool's purpose very clear and distinct from sibling tools focused on other domains.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage by listing actions and their effects, but it does not explicitly state when to use this tool versus alternatives (e.g., other manage tools). There is guidance on using `valid_transitions` to check allowed state transitions, but no comparative guidance or exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

git_manageGit Manage ToolA
Destructive

Connect, browse, and modify external git repositories (GitHub, GitLab, Bitbucket, generic SSH). Repo connections store encrypted credentials and use phpseclib for SSH. File and commit ops happen through the platform's AtomicCommittingGitClient so concurrent agent edits don't corrupt working trees.

Repo actions:

  • repo_list / repo_get (read).

  • repo_create (write) — name, url, credentials (object).

  • repo_update (write) — repo_id + any creatable field.

  • repo_delete (DESTRUCTIVE) — repo_id. Drops connection; does not delete the remote repo.

  • repo_test (read) — repo_id. Verifies connectivity + auth.

File / branch / commit actions:

  • file_read (read) — repo_id, path, optional branch.

  • file_write (write — pushes to remote) — repo_id, path, content, branch, message.

  • file_list (read) — repo_id, optional path, branch.

  • branch_create (write) — repo_id, name, source branch.

  • commit (write — pushes to remote) — repo_id, message, files[].

  • pr_create (write) — repo_id, title, body, source branch, target branch. Returns PR URL.

  • pr_list (read) — repo_id; optional state filter.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: repo_list, repo_get, repo_create, repo_update, repo_delete, repo_test, file_read, file_write, file_list, branch_create, commit, pr_create, pr_list
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
providerNoFilter by provider: github, gitlab, bitbucket, gitea, generic
modeNoFilter by mode: api_only, sandbox, bridge
statusNoFilter by status: active, disabled, error
limitNoMax results (default 15, max 100)
idYesRepository UUID
nameYesFriendly name for the repository
urlYesRepository URL (HTTPS or SSH)
default_branchNoDefault branch name (default: main)main
credential_idNoUUID of a Credential containing the PAT/SSH key for authentication
configNoMode-specific config. sandbox: {provider, instance_type, run_tests, test_command}. bridge: {repo_name, working_directory}. pr: {require_approval}
repository_idYesRepository UUID
pathYesFile path relative to repo root (e.g. "src/app.php")
refNoGit ref: branch name, tag, or commit SHA (default: HEAD)HEAD
contentYesFull file content to write
messageYesCommit message
branchNoBranch to commit to (defaults to repository default_branch)
fromNoSource branch or commit SHA to branch from (defaults to repository default_branch)
changesYesArray of file changes: [{path, content}] or [{path, deleted: true}]
titleYesPull request title
bodyNoPull request description/body
headYesSource branch (the branch with changes)
baseNoTarget branch (defaults to repository default_branch)
stateNoPR state filter: open, closed, all (default: open)open

TDQS

A4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description explicitly marks repo_delete as DESTRUCTIVE, notes that file_write and commit push to remote, details concurrency safety via AtomicCommittingGitClient, and explains encrypted credential storage. This adds rich behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-organized with bullet points grouping repo actions and file/branch/commit actions, and front-loads the main purpose. However, it is slightly verbose and could be more concise without losing clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (25 parameters, many actions), the description covers main categories and behaviors (destructiveness, concurrency, push). However, it lacks return value information (no output schema), error handling notes, and does not clarify conditional parameter requirements (e.g., required fields depend on action). The required array in the schema is misleading without further explanation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with each parameter having a description in the schema. The tool description adds some context (e.g., repo_create uses credentials object, file_write pushes) but does not significantly elaborate on individual parameters beyond what the schema already provides. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool connects, browses, and modifies external git repositories, listing specific actions (e.g., repo_list, file_read, commit) and supported providers (GitHub, GitLab, etc.). This specificity distinguishes it from sibling tools like integration_execute or workflow_manage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description enumerates all possible actions but does not provide explicit guidance on when to use this tool versus alternatives, nor does it mention when not to use it. Usage is implied through action descriptions, but no comparative or contextual advice is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

integration_executeIntegration Execute ToolA
Destructive

Execute an action on a connected integration, e.g. create_issue on GitHub, send_message on Slack.

ParametersJSON Schema
NameRequiredDescriptionDefault
integration_idYesIntegration UUID
integration_actionYesDriver action key, e.g. post_tweet, create_issue, send_message
paramsNoAction parameters (driver-specific)

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already signal destructiveHint:true. The description adds that it executes actions (implying mutation) and gives examples, but does not disclose additional behavioral traits such as required authentication, potential side effects, or rate limits. With annotations covering destructive nature, the description's extra behavioral context is minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with examples. Every part earns its place; no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 3 parameters and no output schema, the description is fairly complete. It explains the action concept and gives examples, but does not cover return values or error behavior, which would be beneficial.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with well-described parameters. The description's examples add context for the 'params' object but do not significantly extend beyond the schema. Baseline 3 is appropriate as the description provides no substantial additional meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Execute an action on a connected integration' with concrete examples (create_issue on GitHub, send_message on Slack), making the purpose unambiguous and distinguishing it from sibling tools like integration_manage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear examples of when to use, but does not explicitly mention when not to use or list alternatives. Given the sibling tools are mostly manage tools, the usage context is clear without exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

integration_manageIntegration Manage ToolB
Destructive

Third-party service integrations (Airtable, Notion, Linear, Stripe, Slack, GitHub via OAuth2 + driver interface). Each integration declares typed capabilities (action endpoints) discoverable at runtime. execute invokes one capability with params validated against the driver's schema; output is normalized to JSON.

Actions:

  • list (read) — optional: driver, status filter.

  • connect (write) — driver, name, credentials (object — driver-specific). Initiates OAuth or stores API keys.

  • disconnect (DESTRUCTIVE) — integration_id. Revokes tokens and deletes the connection.

  • ping (read) — integration_id. Health-check the upstream API.

  • execute (write — side effects on upstream) — integration_id, integration_action (capability name), params (object).

  • capabilities (read) — integration_id. Available actions + parameter schemas.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, connect, disconnect, ping, execute, capabilities
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
driverNoFilter by driver slug (e.g. github, slack, notion)
include_driversNoInclude list of available drivers (default false)
nameYesHuman-readable name for this integration instance
credentialsNoCredential key-value pairs, e.g. {"token": "ghp_..."}
configNoDriver-specific config, e.g. {"database_id": "..."} for Notion
integration_idYesIntegration UUID
integration_actionYesDriver action key, e.g. post_tweet, create_issue, send_message
paramsNoAction parameters (driver-specific)

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses behavioral traits like destructiveness for disconnect and side effects for execute, and the annotation already sets destructiveHint: true. However, it does not cover idempotency, rate limits, or auth requirements beyond mentioning OAuth for connect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a header and bullet points, and most sentences are necessary. However, it is somewhat verbose; could be more concise without losing clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (10 parameters, no output schema), the description fails to explain return values for most actions (only mentions 'output normalized to JSON' for execute). Error handling and detailed behavior for list, connect, etc., are missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds examples for credentials and config, but does not significantly enrich parameter understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool manages third-party integrations with six specific actions (list, connect, disconnect, ping, execute, capabilities) and describes each action's purpose. However, it does not differentiate from the sibling tool 'integration_execute', which could cause confusion about when to use each.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides some usage hints (e.g., 'optional: driver, status filter' for list, and 'DESTRUCTIVE' for disconnect), but lacks explicit guidance on when to use this tool versus alternatives (e.g., sibling integration_execute). No prerequisites or when-not-to-use scenarios are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

knowledge_manageKnowledge Manage ToolA
Destructive

Per-team knowledge bases — vector-indexed document collections agents can search at runtime. Ingestion runs an embedding job (consumes credits via the team's embedding provider). search is hybrid: cosine similarity over pgvector + keyword fallback.

Actions:

  • list (read) — all knowledge bases for the team.

  • create (write) — name, description.

  • ingest (write — costs embedding credits) — knowledge_base_id; one of: content (raw text), url (fetched + extracted), file_id.

  • search (read) — knowledge_base_id, query; optional limit (default 10), threshold.

  • delete (DESTRUCTIVE) — knowledge_base_id. Drops all ingested chunks and embeddings.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, create, ingest, search, delete
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
agent_idNoFilter by linked agent UUID
statusNoFilter by status: idle, ingesting, ready, error
limitNoMax results (default 20)
nameYesHuman-readable name for the knowledge base
descriptionNoOptional description
knowledge_base_idYesUUID of the knowledge base
contentYesRaw text content to ingest
source_nameNoDisplay name for this source (e.g. filename, URL)manual
source_typeNoSource type: text, file, or urltext
reindexNoIf true, delete existing chunks for this source before re-ingesting
queryYesNatural language query
top_kNoNumber of results to return (default 5, max 20)

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses critical behaviors: ingestion runs an embedding job consuming credits, search is hybrid cosine similarity with keyword fallback, and delete is destructive. This adds substantial context beyond the annotations, which only indicate destructiveHint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a concise summary, then breaks down each action in a structured bullet list. Every sentence adds value without redundancy, making it easy for an agent to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity and lack of output schema, the description covers key aspects: actions, credit cost, search method, and destructive delete. Minor gaps include no explicit mention of return formats for list/search, but overall it is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

While the input schema has 100% coverage with parameter descriptions, the tool description groups parameters by action (e.g., ingest uses knowledge_base_id plus content/url/file_id), adding semantic meaning beyond individual parameter definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool manages per-team knowledge bases with vector indexing. It lists five specific actions (list, create, ingest, search, delete), each with a clear verb and resource, distinguishing it from sibling 'manage' tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context for when to use each action, such as listing vs. creating, and notes that ingest consumes credits. It does not explicitly exclude alternative tools, but siblings are distinct categories, so usage context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

marketplace_manageCloud Marketplace Manage ToolA

Cross-team marketplace for shared skills, agents, and workflows. browse and categories are public (no auth scope); publish, install, review operate within the caller's team. Installing a listing copies the artifact into the team and increments the listing's install count.

Actions:

  • browse (read) — optional: query, category, sort. Public.

  • categories (read) — taxonomy of available categories. Public.

  • publish (write) — listing data: target_type (skill/agent/workflow), target_id, name, description, visibility (public/private/team).

  • install (write) — listing_slug. Copies the listed entity into your team.

  • review (write) — listing_slug, rating (1-5), comment. One review per user per listing.

  • analytics (read) — listing_slug. Install counts, ratings (publisher only).

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: browse, publish, install, review, categories, analytics
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
typeNoFilter by listing type (e.g. skill, agent, workflow)
limitNoMax results to return (default 10, max 100)
entity_typeYesType of entity to publish: skill, agent, workflow, or bundle
entity_idNoUUID of the entity to publish (not required for bundle)
bundle_itemsNoFor bundle type: array of {type, id} objects. e.g. [{"type":"skill","id":"uuid"},{"type":"agent","id":"uuid"}]
nameNoMarketplace listing name (required for bundle)
descriptionNoMarketplace listing description
visibilityNoListing visibility: public (all users), unlisted (direct link only), team (team members only). Default: publicpublic
listing_slugYesThe marketplace listing slug
ratingYesRating from 1 (poor) to 5 (excellent)
commentNoOptional review comment (max 1000 characters)
listing_idNo

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With empty annotations, the description fully discloses behavioral traits: public vs team-scoped actions, install side-effects (copy artifact, increment count), review constraints (one per user per listing), and analytics limited to publisher. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is moderately long but well-structured with bullet points for each action. Every sentence contributes useful information. Minor redundancy in action listings but overall efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, but the description does not specify return values for actions like browse or publish. For a tool with 14 parameters and 6 actions, more details on expected outputs would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is high (93%), and the description adds value by grouping parameters under actions and explaining constraints (e.g., bundle_items format, visibility defaults). However, it does not describe every parameter in detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool is a 'Cross-team marketplace for shared skills, agents, and workflows' and lists specific actions (browse, publish, install, etc.) with brief explanations. It distinguishes itself from sibling tools (e.g., admin_manage, agent_manage) by explicitly describing marketplace functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use different actions: 'browse and categories are public', others operate within the caller's team. It also notes that installing copies artifacts and increments count. However, it does not explicitly state when to avoid the tool or suggest alternatives for similar tasks.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

memory_manageCloud Memory Manage ToolA

Team-scoped semantic memory — short notes agents store and retrieve across conversations (preferences, constraints, prior decisions). Backed by pgvector with HNSW indexing; cross-team isolation enforced by TeamScope. Distinct from knowledge_manage: this stores unstructured short notes (1-2 paragraphs), knowledge_manage stores document corpora.

When to use: an agent needs to recall a stable fact about the user/team across runs, or wants to remember a decision so the next conversation starts with that context. Do NOT use for transient run-state — use the experiment artifact system instead.

Actions:

  • search (read; semantic) — query (string), optional limit (default 20, max 100). Returns ranked entries with similarity score 0..1; entries below threshold 0.6 are filtered.

  • list_recent (read) — optional limit (default 50). Reverse-chronological by created_at.

  • stats (read) — returns { total_entries, embedding_dim, last_write_at, storage_bytes }.

  • add (write — costs ~1 credit per ~1k tokens for embedding) — content (string, max 4000 chars), optional metadata (object).

  • delete (DESTRUCTIVE) — memory_id (UUID). Hard delete; not recoverable.

  • upload_knowledge (write — costs embedding credits, async) — one of: file content (string), url. Bulk-ingests into memory; returns job_id. Up to 50 MB per call.

Cloud note: supabase_provision not available in cloud (provisioning happens at platform level). Errors: 401, 403, 404 if memory_id missing, 413 on oversized payload, 422 validation, 429 rate limit, 503 if budget exhausted.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: search, list_recent, stats, add, delete, upload_knowledge
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
queryYesSearch keyword to match against memory content
agent_idNoFilter by agent UUID
limitNoMax results to return (default 10, max 100)
min_confidenceNoMinimum confidence score to filter results (0.0–1.0, default 0.0 to include all)
categoryNoFilter by memory category: preference, knowledge, context, behavior, goal
search_modeNoRetrieval mode: semantic=flat keyword search, local=1-hop graph traversal from matched entities, global=high-centrality entities, hybrid=semantic+local merged, mix=semantic+global mergedsemantic
tagsNoFilter by tags — only return memories containing ANY of these tags. E.g. ["barsy:client", "barsy:shared"]. Omit to return all memories regardless of tags.
topicNoNamespace pre-filter by topic slug, e.g. "auth_migration". Narrows the search to a named context before the vector scan for higher precision.
source_typeNoFilter by source type (e.g. execution, manual, signal)
contentYesThe memory text to store
project_idNoAssociate this memory with a specific project UUID (optional)
confidenceNoConfidence score 0.0–1.0. Default: 1.0 for manually added memories
metadataNoAdditional structured metadata (key-value pairs)
memory_idsYesArray of memory UUIDs to delete

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description details read/write actions, destructive delete, embedding costs, async upload, errors list (401,403,404,413,422,429,503), and cloud provision note. However, lacks explicit mention of authorization requirements and pagination behavior for list_recent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured: purpose first, then usage guidance, then action list. Each section serves a purpose, though the action list could be slightly more compact. No wasted sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers actions, errors, and cloud note. Lacks explicit return structure for most actions (only search and stats are described). No output schema, so description should explain all return values; missing for list_recent, add, delete, upload_knowledge.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds context for actions (e.g., search default limit 20, max 100) but does not significantly enhance parameter meanings beyond what the schema provides. No unique insights per parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly defines the tool as 'Team-scoped semantic memory' for short notes, and explicitly distinguishes it from sibling knowledge_manage by noting scope and content type (unstructured short notes vs document corpora).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States when to use: to recall stable facts across runs or remember decisions. Explicitly provides a negative case: 'Do NOT use for transient run-state — use the experiment artifact system instead.' Clearly identifies alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

outbound_manageCloud Outbound Manage ToolA

Outbound delivery connectors — channels through which agents send messages: email (SMTP), Slack, Telegram, generic webhook. One connector per channel per team (save uses upsert semantics). All sends pass through ChannelRateLimit and TargetRateLimit middleware and are recorded as OutboundActions.

Actions:

  • list / get (read).

  • save (write — upsert) — channel (email/slack/telegram/webhook), config (channel-specific). Replaces any existing connector for the channel.

  • delete (DESTRUCTIVE) — connector_id. Pending outbound actions on this connector are cancelled.

  • test (write — sends a real test payload) — connector_id, test_payload (object). Counts against rate limits and budgets.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, save, delete, test
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
channelNoFilter by channel: telegram, slack, discord, teams, google_chat, whatsapp, email, webhook
idNoConfig UUID
credentialsYesChannel-specific credentials object.
is_activeNoWhether the config is active

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully discloses behavioral traits: save uses upsert semantics, delete cancels pending outbound actions, test counts against rate limits and budgets. It mentions middleware (ChannelRateLimit, TargetRateLimit) and recording as OutboundAction, giving the agent a complete picture.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear introductory sentence followed by bulleted actions. It is slightly verbose but every sentence adds information. No fluff, but could be trimmed slightly for brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (6 parameters, multiple actions, no output schema), the description covers all necessary aspects: action semantics, rate limits, upsert behavior, and destructive effects. An agent can confidently select and invoke the tool based on this description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds value beyond schema by explaining save replaces existing connectors, delete is destructive, and test sends real payload. However, details on credentials and is_active remain schema-level only, so not a full 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as managing outbound delivery connectors for messaging channels. It lists specific actions (list, get, save, delete, test) with brief explanations, and distinguishes the tool from siblings by focusing on connector lifecycle and channel types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context for each action (e.g., 'save (write — upsert)', 'delete (DESTRUCTIVE)', 'test (write — sends a real test payload)'). It implicitly guides when to use which action but does not explicitly contrast with sibling tools like credential_manage or integration_manage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

profile_manageCloud Profile Manage ToolA

Caller's own profile and account settings (name, email, password, 2FA, OAuth links). Operates on the authenticated user — there is no user_id parameter; super-admins managing other users use admin_manage instead. All actions are rate-limited per user (60/min) and audit-logged.

When to use: agent or assistant needs to read/update the current user's profile, list linked OAuth providers, or unlink a social provider. Do NOT use to create accounts (that's a self-service signup flow only).

Actions:

  • get (read) — returns profile object: name, email, locale, timezone, current_team_id, last_login_at, email_verified_at.

  • update (write) — optional fields: name, email, locale, timezone. Email change re-issues the verification mail and clears email_verified_at until confirmed.

  • password_update (DESTRUCTIVE; rotates session) — current_password, password, password_confirmation. Verifies current password then bcrypts the new one; revokes all OTHER active sessions of this user.

  • 2fa_status (read) — returns { enabled: bool, recovery_codes_remaining: int }.

  • connected_accounts (read) — list of linked OAuth provider records (provider, provider_user_id, linked_at).

  • social_list (read) — duplicate of connected_accounts kept for backward client compatibility.

  • social_unlink (DESTRUCTIVE) — provider. Removes the OAuth link; rejected with 422 if it would leave the user with no remaining auth method.

Errors: 401 unauthenticated; 403 if Sanctum token lacks team:{id} ability; 422 for validation; 429 on rate limit.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: get, update, password_update, 2fa_status, connected_accounts, social_list, social_unlink
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
nameNo
emailNo
current_passwordNo
passwordNo
password_confirmationNo
providerNoThe social provider to disconnect.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are empty, so description bears full burden. Discloses side effects for each action: email update re-issues verification and clears email_verified_at, password_update is destructive and revokes sessions, social_unlink may be rejected if no remaining auth method. Also mentions rate limits and audit logging.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured: overview, usage guidance, then action list with behaviors. Every sentence adds value. No redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given complexity (multiple actions, side effects, no output schema), description covers all actions, their parameters, side effects, and error cases (401, 403, 422, 429). Missing explicit return format but described inline (e.g., profile object fields, 2fa_status returns {enabled, recovery_codes_remaining}). Highly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 38% (low), but description adds semantics for action-specific parameters (e.g., update fields are optional, password_update requires current_password/password/password_confirmation, provider enum values). Does not describe deadline_ms but it is optional. Compensates well for low coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states it operates on the caller's own profile/account settings, lists supported actions (get, update, password_update, etc.), and distinguishes from sibling tool admin_manage. Specific verbs and resource identified.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use instructions ('agent or assistant needs to read/update the current user's profile...'), excludes account creation (self-service signup), and references admin_manage as an alternative for managing other users.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

project_manageProject Manage ToolA
Destructive

Projects — recurring (continuous) or one-shot containers that schedule workflow runs against goals. Continuous projects spawn ProjectRuns on the configured cadence (cron-style or natural language); one-shot projects fire once on trigger_run. Lifecycle states: draft → active → (paused | archived). Each run reserves budget independently.

Actions:

  • list / get (read).

  • create (write) — name, type (one_shot | continuous), workflow_id.

  • update (write) — project_id + any creatable field.

  • activate / pause / resume / restart (write) — project_id. Lifecycle transitions.

  • trigger_run (write — costs credits) — project_id. Dispatches a single run immediately.

  • archive (write) — project_id. Stops scheduling and hides from default lists; reversible via restart.

  • schedule (write) — project_id, frequency or cron expression.

  • schedule_nlp (write) — project_id, natural language schedule (e.g. "every weekday at 9am"). Calls LLM, costs credits.

  • run_list (read) — project_id; optional limit, status filter.

  • run_get (read) — project_id, run_id.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, update, activate, pause, resume, restart, trigger_run, archive, schedule, schedule_nlp, run_list, run_get
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: draft, active, paused, completed, archived, failed
typeNoFilter by type: one_shot, continuous
limitNoMax results to return (default 10, max 100)
project_idYesThe project UUID
titleYesProject title
descriptionNoProject description
goalNoProject goal
execution_modeNoExecution mode: autonomous (full tool access) or watcher (read-only tools only). Default: autonomousautonomous
workflow_idNoUUID of an active workflow to run for each project execution. The workflow must be in active status.
crew_idNoUUID of a crew to run for each project execution. Alternative to workflow_id.
allowed_tool_idsNoRestrict which tools agents can use in this project. Pass an array of tool UUIDs. Empty = all team tools allowed.
allowed_credential_idsNoRestrict which credentials are available to agents in this project. Pass an array of credential UUIDs.
scheduleNoSchedule configuration. Required for continuous projects — omitting it creates a project that never runs. Use project_schedule_nlp to parse natural language schedules.
operationYesget: read schedule details | update: change schedule settings | enable: turn on the schedule | disable: turn off the schedule
run_idYesThe project run UUID

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, the description reveals that trigger_run costs credits, schedule_nlp calls an LLM, and archive is reversible. This adds useful behavioral context without contradicting the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear first sentence and bullet-pointed actions. It is appropriately sized for the complexity, though slightly verbose in places.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity and lack of output schema, the description covers lifecycle states, actions, and side effects fairly completely. It could briefly mention return types for read actions, but overall it provides sufficient context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage, the schema already describes parameters fully. The description adds minor value by grouping actions and mentioning 'project_id + any creatable field' for update, but does not significantly enhance parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as managing project containers for scheduling workflows, with a comprehensive list of actions. It distinguishes itself from siblings by focusing on project lifecycle and scheduling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for each action (e.g., 'write' vs 'read') and includes important notes like cost for trigger_run and LLM usage for schedule_nlp. However, it does not explicitly compare to sibling tools or state when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signal_connectorsCloud Signal Connectors ToolA

Inbound signal SOURCES (ticketing, alerts, Slack, HTTP monitors, ClearCue, Telegram bots) plus the team's knowledge graph (KG) read/write surface. Distinct from signal_manage: this tool wires up connector configurations; signal_manage operates on already-ingested signals.

When to use: bind a new external system to FleetQ so its events become signals (which trigger rules → projects → agents), or query/extend the team's knowledge graph that agents reason over. Do NOT use to send outbound messages — use outbound_manage for that.

Connector actions (each accepts a config object specific to that channel; idempotent — re-call to update):

  • ticket (write — upsert) — config (Jira/Linear/Zendesk fields). Polls every 5 min.

  • alert (write — upsert) — config (PagerDuty/Opsgenie webhook). Pushes signals on incident events.

  • slack (write — upsert) — config (workspace_id, channel_ids[], event filters).

  • http_monitor (write — upsert) — config: url, interval_seconds (min 60), expected_status. Emits a signal when status flips.

  • clearcue (write — upsert) — config (ClearCue API key, query).

  • inbound_connector (write) — sub-actions: create, update, delete on generic connectors.

  • subscription (write) — sub-actions: list, create, delete on per-connector subscriptions.

  • telegram (write — upsert) — config: bot_token, allowed_chat_ids[]. Webhook auto-registered.

Knowledge graph actions:

  • kg_search (read — hybrid semantic+symbolic) — query (string), optional types (array). Returns ranked entities with relevance score.

  • kg_facts (read) — entity_id (UUID). All facts attached, paginated.

  • kg_add_fact (write) — entity_id, fact: { predicate (string), object (string), source (string), confidence (0..1) }.

  • intent_score (read — costs ~1-2 LLM credits) — company (string), query (string). Returns intent classification + confidence.

Cloud note: supabase connector not exposed in cloud (TeamProviderCredential handles per-team Supabase access through integration_manage). Side effects: writes update KG and trigger downstream rule evaluation (may dispatch agent runs and consume budget). Errors: 401, 403, 404, 422 (config validation), 429, 503 (upstream connector down).

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: ticket, alert, slack, http_monitor, clearcue, intent_score, kg_search, kg_facts, kg_add_fact, inbound_connector, subscription, telegram
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
driverNoConnector driver for setup instructions: github | github_issues | jira | linear
urlNoURL to monitor (required for add)
monitor_typeNoMonitor type: availability | content_change | both (default: availability)availability
nameNoHuman-readable name for the monitor (optional, defaults to hostname)
connector_idNoConnector UUID (required for remove)
expected_statusNoExpected HTTP status codes (default: [200])
ssl_checkNoAlert when SSL certificate expires within 14 days (default: true)
entity_keyNoStable identifier for the entity: LinkedIn URL, company domain, or website URL. Required for get_score and get_signal_history.
entity_typeNoEntity type: company | person
thresholdNoMinimum threshold for list_hot_leads: hot (80+) | warm (50+) | lukewarm (20+)
limitNoMaximum number of results for list_hot_leads (default 20, max 100)
queryYesNatural language search query, e.g. "CEO of Acme Corp" or "latest price of Competitor X"
relation_typeNoFilter by relation type (snake_case), e.g. works_at, has_price, has_status, acquired_by
include_historyNoInclude invalidated historical facts (default: false — only current facts)
entity_nameYesName of the entity to look up, e.g. "Alice Chen" or "Acme Corp"
source_entityYesName of the source entity, e.g. "Alice Chen" or "Acme Corp"
source_typeYesType of the source entity: person | company | organization | location | date | product | technology | event | concept | process | topic
target_entityYesName of the target entity, e.g. "Beta Corp" or "$79/month"
target_typeYesType of the target entity: person | company | organization | location | date | product | technology | event | concept | process | topic
factYesHuman-readable fact statement, e.g. "Alice Chen is VP Engineering at Beta Corp"
valid_atNoISO 8601 datetime when this fact became true (defaults to now if omitted)
subscription_idNoSubscription UUID (required for get/toggle/delete)
integration_idNoIntegration UUID to bind this subscription to (required for create)
filter_configNoDriver-specific filter config. GitHub: {repo, filter_branches, event_types}. Linear: {team_id, resource_types, filter_actions}. Jira: {project_key, webhook_events}.
bot_tokenNoBot token from BotFather (required for register)
routing_modeNoRouting mode: assistant | project | trigger_rules (default: assistant)assistant

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full burden. It discloses side effects ('writes update KG and trigger downstream rule evaluation'), error codes (401, 403, 404, 422, 429, 503), idempotency for connector actions, polling intervals (5 min), and cloud-specific restrictions. While not exhaustive (e.g., no mention of retry behavior), it is quite thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections (purpose, when to use, action list, cloud note, side effects, errors). It is front-loaded with essential information. While it is lengthy (due to many actions), it is organized and every sentence adds value, making it efficient rather than verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (28 parameters, no output schema, nested objects), the description covers all major aspects: actions with config details, knowledge graph operations, side effects, error codes, and cloud-specific note. It does not describe return values, but since no output schema exists, this is acceptable. The description provides sufficient context for an agent to understand the tool's capabilities and boundaries.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, but the description adds significant value by grouping actions and explaining the behavior of each connector action (e.g., 'Polls every 5 min' for ticket, 'Pushes signals on incident events' for alert). It provides context for config objects and sub-actions (e.g., subscription: 'list, create, delete'). This goes beyond the schema's field-level descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool handles inbound signal sources and knowledge graph read/write, with a specific verb ('bind' and 'query/extend'). It distinguishes from sibling `signal_manage` by explaining that this tool wires up connector configurations while the sibling operates on already-ingested signals, providing clear differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidelines: 'When to use: bind a new external system to FleetQ so its events become signals...or query/extend the team's knowledge graph.' Also explicitly states when NOT to use: 'Do NOT use to send outbound messages — use `outbound_manage` for that.' This provides clear context and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

signal_manageSignal Manage ToolA
Destructive

Inbound signals — events from connectors (webhooks, RSS, email, Slack, ticketing) the platform processes through trigger rules into agent actions. Operates on already-ingested signals; for connector setup use signal_connectors.

Actions:

  • list (read) — optional: status, source, channel, limit.

  • get (read) — signal_id. Full payload + processing trail.

  • ingest (write) — source, payload (object). Manually emits a signal as if from a connector; runs trigger evaluation.

  • assign (write) — signal_id, assignee_user_id, reason.

  • connector_binding (write) — connector_id, channel_id. Links a connector to a logical channel.

  • connector_binding_delete (DESTRUCTIVE) — binding_id. Severs the link; future signals from that connector go unrouted.

  • contact (write) — sub-actions on Contact (action, contact data).

  • imap (write) — mailbox config object. Sets/updates IMAP poller settings.

  • email_reply (write — sends email) — signal_id, body. Replies to the originating email signal via the team's outbound email connector.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, ingest, assign, connector_binding, connector_binding_delete, contact, imap, email_reply
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
limitNoMax results to return (default 10, max 100)
signal_idYesThe signal UUID
sourceYesSignal source identifier (e.g. "mcp", "manual", "api")
payloadYesSignal payload data
assignee_user_idNoUser UUID to assign to, or null to unassign
reasonNoOptional note added as an internal comment
binding_idNoBinding UUID (required for approve/block/get)
status_filterNoFilter by status: pending | approved | blocked
channel_filterNoFilter by channel: telegram | whatsapp | discord | signal_protocol | matrix
contact_idNoContact identity UUID (required for get/merge/unlink_channel)
source_contact_idNoSource contact UUID to merge INTO contact_id (required for merge — source is deleted after merge)
channel_idNoChannel UUID to unlink (required for unlink_channel)
searchNoSearch term (name, email, phone, sender ID)
connector_idYesIMAP connector UUID. Use inbound_connector_manage(list_connectors) to discover configured accounts.
folderNoMailbox folder to operate on (default: INBOX)
fromNoFilter by sender email address (search only)
subjectNoFilter by subject keyword (search only)
sinceNoISO 8601 date — return emails received since this date, e.g. 2026-03-01 (search only)
unseen_onlyNoReturn only unread/unseen emails (search only)
uidNoEmail UID to fetch (read only)
bodyYesReply body (plain text or HTML)
auto_sendNoIf true, send immediately. If false (default), creates an approved OutboundProposal for review.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide destructiveHint: true, and the description amplifies this by labeling connector_binding_delete as 'DESTRUCTIVE' and explaining consequences ('future signals from that connector go unrouted'). It also notes that ingest 'runs trigger evaluation' and email_reply 'sends email via the team's outbound email connector', adding behavioral context beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with bullet points for actions, making it easy to scan. It covers all major aspects without excessive verbosity. The introductory line about signals could be slightly trimmed, but overall it's efficient for the complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 24 parameters, no output schema, and multiple actions, the description covers the core behaviors but lacks return value details for list/get actions and error handling. The deadline_ms parameter is not mentioned in the description. While it provides sufficient context for most uses, some gaps remain for a tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with detailed parameter descriptions, so the tool description adds little new information about parameters. It lists which action each parameter relates to implicitly, but the schema already provides restrictions (e.g., 'search only'). Baseline of 3 is appropriate as the schema carries the load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it manages inbound signals from connectors, lists all supported actions with specific verbs (list, get, ingest, etc.), and explicitly distinguishes from sibling tool `signal_connectors`. This provides a specific verb+resource scope and differentiates from alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description directly states 'Operates on already-ingested signals; for connector setup use `signal_connectors`', giving explicit when-to-use and when-not-to guidance. Each action is described with its effect (read/write/destructive), making it clear when each action is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

skill_manageCloud Skill Manage ToolA

AI skills — reusable units agents invoke (LLM prompt templates, connector calls, rules, hybrid pipelines, guardrails). Skill executions are versioned (SkillVersion) and metered against the team budget. Direct execution actions (guardrail, multi_model, code_exec, browser) bypass the agent layer for ad-hoc invocation by humans/assistants.

When to use: build reusable prompt or workflow primitives that multiple agents share, or run a one-off skill (e.g. code_exec to evaluate Python in a sandbox). Do NOT use for raw LLM calls — use assistant_manage instead.

CRUD actions:

  • list (read) — optional: type (llm|connector|rule|hybrid|guardrail), status, limit (default 50).

  • get (read) — skill_id. Returns full config + latest version.

  • create (write) — name, type, config (type-specific JSON). Returns skill_id.

  • update (write) — skill_id + any creatable field. Bumps version; old version retained.

  • delete (DESTRUCTIVE) — skill_id. Soft-deletes; existing version history retained for 30 days.

  • versions (read) — skill_id. Version log with diffs and created_by AiRun id.

Direct execution (each costs credits; rate-limited per skill):

  • guardrail (read — costs ~1-3 LLM credits) — input (string), rules (array). Returns { passed: bool, violations: [...] }.

  • multi_model (write — costs ~N × LLM credits where N=models[].length) — prompt, models[]. Runs same prompt across providers and returns consensus.

  • code_exec (DESTRUCTIVE — runs in sandboxed Docker, costs sandbox-minute credits) — code, language (python|node|bash). Network-disabled by default; max 60s wall-clock; max 256 MB memory.

  • browser (write — costs browser-minute credits) — url, actions (array of {type, selector, value}). Headless Chromium with SSRF guards.

Cloud note: supabase_edge_function not exposed in cloud (use integration_execute with a Supabase integration instead). Errors: 401, 403, 404, 422, 429, 503 (sandbox/browser worker pool exhausted).

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, update, delete, versions, guardrail, multi_model, code_exec, browser
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
typeNoFilter by type: llm, connector, rule, hybrid
frameworkNoFilter by framework key (e.g. rice, spin, bant, okrs, unit_economics). See framework_list for full enum.
limitNoMax results to return (default 10, max 100)
skill_idYesThe skill UUID
nameYesSkill name
descriptionNoSkill description
prompt_templateNoSystem prompt template for LLM-backed skills
data_classificationNoData classification level: public, internal, confidential, restricted.
step_idNoFor get_result. The playbook step UUID.
workflow_node_idNoFor set_node_guardrail / remove_node_guardrail. The workflow node UUID.
guardrail_skill_idNoFor set_node_guardrail. The guardrail skill UUID to attach.
execution_idNoFor get_execution: the SkillExecution UUID.
worktree_execution_idNoFor get_execution / get_diff: the WorktreeExecution UUID.
statusNoFor list_executions: filter by status (pending_approval, completed, failed, approved, rejected).

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully shoulders the burden. It details the effects of each action (read/write/destructive), versioning behavior, soft-delete with 30-day retention, costs and rate limits for direct execution, sandbox restrictions (network-disabled, max 60s, 256 MB), and lists possible error codes. This is comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is lengthy (over 500 words) but well-organized with sections for CRUD, direct execution, cloud notes, and errors. Given the tool's complexity (multiple actions, 16 parameters, no output schema), the detail is necessary. A minor reduction could improve conciseness, but it remains clear and scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description fully covers the tool's purpose, all actions, parameter usage, error handling, cost implications, and cloud-specific limitations. Despite lacking an output schema, the description provides enough context for an agent to select and invoke the tool correctly across all scenarios.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema description coverage is 100%, the description adds significant context beyond parameter names and types. It groups parameters under actions (e.g., 'create (write) — name, type, config (type-specific JSON). Returns skill_id.'), explains the meaning of type, framework, and data_classification, and clarifies defaults and constraints for limit, deadline_ms, etc.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly defines the tool as managing AI skills (reusable units) and lists all CRUD and direct execution actions. It explicitly distinguishes from siblings, such as warning not to use for raw LLM calls (use assistant_manage) and noting that supabase_edge_function is not exposed in cloud (use integration_execute).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes a 'When to use' section that specifies appropriate use cases (building reusable primitives, one-off skill execution) and explicitly says what not to do ('Do NOT use for raw LLM calls'). It provides clear alternatives (assistant_manage, integration_execute) and context-specific guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

system_manageCloud System Manage ToolA

Platform system administration, monitoring, and infrastructure control — KPIs, health checks, audit log, semantic cache, GPU compute providers, browser-relay bridge. Mixed permission tiers: read actions (dashboard_kpis, health, version_check, audit_log, cache_stats) need any authenticated user; write actions marked PLATFORM ADMIN require the user's is_super_admin=true flag (HTTP 403 otherwise). Every write is audit-logged.

When to use: agent monitoring platform health, on-call investigating an incident via audit log, super-admin adjusting platform-wide settings or purging the semantic cache after a model swap. Do NOT use for tenant-level operations — those have dedicated tools (e.g. team_manage).

Read actions:

  • dashboard_kpis — { active_users, runs_today, spend_today_credits, ... }.

  • health — { status: ok|degraded|down, db, cache, queue, providers[] }.

  • version_check — { current, latest, update_available }.

  • audit_log — optional: actor_id, action, entity_type, since, limit (max 100). Cursor-paginated.

  • cache_stats — semantic cache hit/miss/savings since last purge.

Write actions:

  • global_settings (write — PLATFORM ADMIN) — settings object. Replaces platform-wide settings.

  • blacklist (write — PLATFORM ADMIN) — sub-actions: list, add, remove. Affects email/domain blocklists for sign-ups.

  • security_policy (write — PLATFORM ADMIN) — sub-actions: list, save, delete on security policies.

  • cache_purge (DESTRUCTIVE — PLATFORM ADMIN) — purges semantic cache CROSS-TEAM. All teams lose cached LLM responses; first calls after purge will be slower and more expensive.

  • compute (write — PLATFORM ADMIN) — sub-actions on compute provider records (Replicate, Fal.ai, Vast.ai, RunPod).

  • runpod (write — PLATFORM ADMIN, costs real $) — sub-actions on RunPod GPU pods. Spinning up a pod bills the platform Stripe account immediately at the pod's per-minute rate.

  • browser_relay (write — relays through local bridge) — sub-actions for browser automation through the team's connected FleetQ Bridge runner. Requires an active bridge.

Errors: 401, 403 (insufficient role for PLATFORM ADMIN action), 404, 422, 429, 503 (provider down).

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: dashboard_kpis, health, version_check, audit_log, global_settings, blacklist, security_policy, cache_stats, cache_purge, compute, runpod, browser_relay
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
forceNoBypass the cache and fetch the latest version from GitHub immediately.
subject_typeNoFilter by subject type (e.g. experiment, agent, approval)
ocsf_class_uidNoFilter by OCSF class UID (e.g. 3002 = API Activity, 3001 = Account Change, 3006 = Financial Activity, 1001 = Process Activity, 4002 = HTTP Activity)
limitNoMax results to return (default 20, max 100)
settingsYesKey-value pairs to update. Allowed keys: assistant_llm_provider, assistant_llm_model, default_llm_provider, default_llm_model, budget_cap_credits, rate_limit_rpm, outbound_rate_limit, experiment_timeout_seconds, weekly_digest_enabled, audit_retention_days
operationYeslist: get all blacklist entries | add: add a new entry | remove: remove an entry by id
typeNoRequired for add. Type of entry to block.
valueNoRequired for add/remove. The value to block (e.g. "spam@example.com") or the entry UUID to remove.
reasonNoOptional reason for add operations. Stored for audit purposes.
policyNoRequired for update. Policy fields to set.
providerNoFilter by provider (e.g. "anthropic"). Omit to purge all.
modelNoFilter by model (e.g. "claude-sonnet-4-5-20250929"). Omit to purge all.
expired_onlyNoWhen true, only purge entries past their expiry date.
api_keyNoAPI key for credential_save
endpoint_idNoProvider endpoint/model identifier (required for health_check and run)
inputNoInput payload for run
use_syncNoUse synchronous mode for run (default: true)
timeout_secondsNoMax wait time in seconds for run (default: 90)
job_idNoJob ID returned by endpoint_run async mode (required for endpoint_status)
pod_idNoPod ID for pod_status or pod_stop
pod_configNoPod configuration for pod_create (imageName, gpuTypeIds, gpuCount, env, etc.)
toolNoBrowserMCP tool name (required for call_tool)
paramsNoTool parameters for BrowserMCP (optional for call_tool)

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description fully carries the burden. It details permission tiers (any authenticated user for reads, is_super_admin=true for writes), audits logging on writes, and specific behavioral consequences like cache_purge causing cross-team cache loss with performance/cost impacts, and runpod incurring immediate billing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with headers for read/write actions, bullet points, and clear sections. It is longer due to high complexity but every section earns its place. Slight redundancy could be trimmed, but overall efficient for the scope.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (25 parameters, no output schema, nested objects), the description is remarkably complete: it covers all actions, permissions, error codes, and behavioral traits. The only minor omission is not explicitly stating output formats, but that is implied for a system tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, baseline 3. The description adds value by grouping actions into read/write and explaining the meaning of sub-actions beyond enum labels (e.g., 'blacklist' sub-actions list/add/remove). However, many parameter details are already covered in the schema, so the marginal gain is moderate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as 'Platform system administration, monitoring, and infrastructure control' and lists specific actions (KPIs, health checks, audit log, etc.). It explicitly distinguishes from sibling tools by stating 'Do NOT use for tenant-level operations — those have dedicated tools (e.g. team_manage).'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use examples: 'agent monitoring platform health, on-call investigating an incident via audit log, super-admin adjusting platform-wide settings or purging the semantic cache after a model swap.' It also gives exclusion guidance: 'Do NOT use for tenant-level operations' and points to a specific alternative tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

team_manageCloud Team Manage ToolA

Caller's team — settings, membership, BYOK provider credentials, API tokens, notifications, and KMS-managed encryption keys. Most write actions require team role owner or admin (HTTP 403 otherwise); viewer and member are limited to read + their own profile-level toggles (notifications). Every write is audit-logged.

When to use: agent or assistant managing the current team's configuration — invite a member, mint an API token, register a BYOK Anthropic key, etc. Do NOT use for cross-team operations — those require admin_manage (super-admin only).

Core team actions:

  • get (read) — returns team object (name, slug, plan, settings, owner_id, member_count).

  • update (write — admin/owner) — optional: name, settings (object). Plan changes happen via Stripe webhook only.

  • members (write — admin/owner) — sub-actions: list, invite (email, role), remove (user_id). Invitations expire after 7 days.

LLM provider config (admin/owner):

  • local_llm (read) — bridge-discovered local LLM agents (Ollama, LM Studio, Codex, Claude Code).

  • byok_credential (write) — sub-actions on BYOK keys (Anthropic, OpenAI, Google, Mistral, Perplexity). Keys encrypted at rest; never echoed back.

  • custom_endpoint (write — plan-enforced; pro/enterprise only) — sub-actions on custom OpenAI-compatible LLM endpoints (vLLM, LiteLLM, custom proxies).

Tokens & access (admin/owner):

  • api_token (write) — sub-actions: create (returns token once), list, revoke. Tokens are team-scoped and inherit the user's role; rotate on suspected leak.

Notifications & system:

  • notification (write — any role) — sub-actions: list, dismiss, dismiss_all on the user's notification inbox.

  • join_request (write — admin/owner) — sub-actions: list, approve, reject pending team join requests.

  • kms (DESTRUCTIVE — owner only) — sub-actions on KMS encryption keys: list, rotate (re-encrypts all team credentials with new key — cannot be undone), revoke.

Errors: 401, 403 (insufficient role), 404, 409 (cannot remove self if last owner), 422, 429.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: get, update, members, local_llm, byok_credential, custom_endpoint, api_token, notification, join_request, kms
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
nameNoNew team name
settingsNoTeam settings object (merged with existing settings)
providerNoProvider: ollama | openai_compatible (required for configure/remove/discover)
base_urlNoBase URL of the endpoint (e.g. http://localhost:11434)
api_keyNoOptional API key for authenticated endpoints
modelsNoComma-separated model IDs for openai_compatible endpoints
token_idNoToken ID to revoke (required for revoke action)
notification_idNoRequired for mark_read. The notification UUID to mark as read.
titleNoRequired for send. Notification title.
bodyNoRequired for send. Notification body text.
typeNoFor send. Notification type (e.g. agent_alert, budget_warning, info).
action_urlNoFor send. Optional URL the user can click to navigate.
user_idNoFor send. Target user ID. If omitted, notifies all team members.
preferencesNoFor update_preferences. Map of notification_type => array of channels (in_app, mail, push). E.g. {"experiment.stuck": ["in_app","mail","push"]}
request_idNoJoin request UUID (required for approve/deny)
credentialsNoProvider-specific credentials. Required for test and enable. AWS: {role_arn, key_arn, region}. GCP: {project_id, location, key_ring, key_id, service_account_json}. Azure: {tenant_id, client_id, client_secret, vault_url, key_name, key_version?}.
key_identifierNoThe key identifier (ARN for AWS, resource name for GCP, vault URL + key name for Azure). Required for enable.
forceNoForce removal even if KMS is unreachable. Only for remove action.

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Detailed disclosure of behavioral traits: role-based access (owner/admin for writes, viewer/member for read+notifications), audit logging, destructive nature of KMS rotate/revoke, token rotation advice, and error types. Annotations are absent, so description fully informs the agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured with headers and bullet points, front-loading purpose and role requirements. Though lengthy due to complexity, every sentence adds value without redundancy. Appropriate size for the tool's scope.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers almost all necessary context: roles, errors (401,403,404,409,422,429), action details, and return values for key actions like 'get'. Despite no output schema, the description provides sufficient information for correct usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds significant context beyond the input schema: explains action enum with sub-actions, special notes like 'never echoed back' for BYOK, 'plan-enforced' for custom endpoints, and token creation returning once. Schema coverage is 100%, but description enriches each parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the tool manages the caller's team with specific verbs (settings, membership, BYOK, tokens, notifications, KMS keys). It distinguishes from sibling 'admin_manage' for cross-team operations, ensuring clear purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use guidance (agent/assistant managing current team) and when-not (use admin_manage for cross-team). Also clarifies role requirements for different actions, aiding correct invocation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tool_manageCloud Tool Manage ToolA

LLM tool management — registers MCP servers (stdio, HTTP, bridge) and built-in tools (bash, filesystem, browser, SSH) that agents can call at inference time. Tool execution may have any side effect declared by the underlying tool; the platform constrains bash/filesystem/SSH only via the agent's bash_policy. Tool credentials are encrypted at rest with the team's per-tenant key.

When to use: connect a new external MCP server (e.g. GitHub, Slack, Notion) so agents can call its tools, toggle a tool's availability without deleting it, or test a remote MCP server's reachability before wiring it to an agent. Do NOT use to invoke tools — agents call them automatically based on attached configuration.

CRUD actions:

  • list (read) — optional: type (mcp_stdio|mcp_http|mcp_bridge|built_in), status, limit (default 50).

  • get (read) — tool_id (UUID). Metadata only; credentials redacted in response.

  • create (write) — name, type, config (type-specific JSON: command for stdio, url+headers for http, kind for built_in).

  • update (write) — tool_id + any creatable field. Re-encrypts credentials if provided.

  • delete (DESTRUCTIVE) — tool_id. Soft-deletes; agents lose access on next ResolveAgentTools call (cached up to 5 minutes).

  • activate / deactivate (write) — tool_id. Flips active flag without deletion; affects all agents that reference the tool.

Discovery (cloud restricts these to safe operations):

  • probe_remote (read — calls the remote URL, no registration) — url. Sends an MCP initialize request; returns server name, version, declared capabilities.

Cloud note: discover_mcp and import_mcp (which auto-register all tools from a remote server) are NOT available in cloud — they require explicit per-tool review under cloud's security policy. Use probe_remote then create per tool instead.

Errors: 401, 403, 404, 422 (config validation), 429, 503 (remote MCP server unreachable for probe_remote).

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, update, delete, activate, deactivate, probe_remote
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: active, disabled
platform_onlyNoIf true, return only platform-level tools (shared across all teams)
limitNoMax results to return (default 10, max 100)
tool_idYesThe tool UUID
nameYesTool name
descriptionNoTool description
typeNoTool type: mcp_stdio, mcp_http, mcp_bridge, built_in (default: mcp_stdio)mcp_stdio
transport_configNoTransport configuration (command, args, env for stdio; url, headers for http)
risk_levelNoRisk classification: safe, read, write, destructive
credential_idNoUUID of a linked Credential to use for this tool (optional; preferred over inline api_key)
network_policyNoJSON string defining egress rules for Docker sandbox (built_in bash only). Example: {"rules":[{"protocol":"tcp","host":"api.example.com","port":443}],"default_action":"deny"}
clear_credential_idNoSet true to remove the linked credential from this tool
credential_overridesNoKey-value pairs of environment variable overrides (e.g. API keys). Values are stored encrypted.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite no annotations, the description fully discloses behavioral traits: side effects depend on underlying tool, platform constraints via bash_policy, credential encryption, soft-delete with caching, and error codes. It also notes that credentials are redacted in responses, adding safety context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with sections, bullet points, and clear headings. It is detailed but each sentence earns its place. Minor redundancy could be trimmed, but it remains highly readable and informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (15 parameters, nested objects, no output schema), the description is remarkably complete: it covers all actions, side effects, error codes, caching behavior, and cloud-specific limitations. No gaps are evident for a sophisticated agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, but the description adds value by explaining parameter usage in context (e.g., probe_remote is read-only, delete is destructive with caching). It does not merely repeat schema but enhances understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: managing MCP servers and built-in tools for agents. It lists specific actions (CRUD, activation, probe) and distinguishes from sibling tools like agent_manage or credential_manage by focusing on tool registration and lifecycle.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'When to use' section explicitly guides when to use this tool (e.g., connect new MCP server, toggle availability) and includes a warning not to use it for invocation. It also clarifies cloud-specific restrictions (discover_mcp not available), providing clear context for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

trigger_manageTrigger Manage ToolA
Destructive

Event-driven trigger rules — when-this-then-that automations that fire on signals or domain events. Rules evaluate conditions (expression-based) against the event payload and execute actions (start_experiment, send_outbound, etc.). test is a dry-run that returns whether the rule would have matched without executing actions.

Actions:

  • list (read) — optional: event filter, status filter.

  • get (read) — trigger_id.

  • create (write) — name, event (e.g. "signal.ingested"), conditions (array of expressions), actions (array of action specs).

  • update (write) — trigger_id + any creatable field.

  • delete (DESTRUCTIVE) — trigger_id. Future events stop matching this rule.

  • test (read — costs no credits) — trigger_id, sample payload. Returns matched (bool), action_preview (what would have run).

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, create, update, delete, test
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: active | paused
limitNoMax results (default 20, max 100)
nameYesDescriptive name for the rule
source_typeNoSignal source type to match (* = any). E.g. sentry, imap, telegram, rss*
project_idNoUUID of the project to trigger
conditionsNoConditions to match on signal payload. Keys are dot-notation field paths, values are {operator: value} objects. Operators: eq, neq, gte, lte, contains, not_contains, exists
input_mappingNoMap signal fields to project input_data. Keys are target fields, values are dot-notation source paths
cooldown_secondsNoSeconds between triggers (0 = no cooldown)
max_concurrentNoMax active runs before skipping trigger (-1 = unlimited)
rule_idYesUUID of the trigger rule
payloadNoSignal payload to test against. Use dot-notation to nest: {"metadata": {"severity": "error"}}
executeNoIf true, actually trigger the project run (default: false — dry run only)

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations include destructiveHint=true, and the description adds context that delete prevents future events from matching. It also notes that test costs no credits. However, it does not disclose other behaviors like rate limits, required permissions, or side effects beyond delete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is moderately long but well-structured with bullet points for actions. The first sentence clearly states the purpose. Every sentence adds value, but it could be slightly more concise without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 14 parameters and no output schema. The description explains what each action does but does not describe the return values or response structure. For example, it says 'list (read) — optional filters' but not what the list response contains. Given the complexity, more details on outputs are needed for completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already describes each parameter. The description adds little extra meaning beyond the schema descriptions. For example, the action parameter is explained in the description, but the schema also has a description. The baseline is 3 because the description does not significantly enhance parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool manages event-driven trigger rules (when-this-then-that automations) and lists specific actions with brief explanations. It distinguishes itself from sibling tools by focusing on triggers and rules, not other management types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides some guidance on when to use each action (e.g., 'test is a dry-run') and notes that delete is destructive. However, it does not explicitly state when not to use the tool or compare it to alternatives like signal_manage or workflow_manage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

webhook_manageCloud Webhook Manage ToolA

Outbound webhook endpoints — URLs the platform POSTs to when subscribed events fire (experiment.completed, signal.ingested, approval.decided, etc.). Each delivery is HMAC-signed with the configured secret (header: X-FleetQ-Signature). Failed deliveries retry with exponential backoff up to 24h.

Actions:

  • list (read) — all webhooks for the team.

  • get (read) — webhook_id. Includes recent delivery stats.

  • create (write) — url, events[] (array of event names), secret (used for HMAC signing; show once).

  • update (write) — webhook_id + any creatable field. Updating secret invalidates the old one immediately.

  • delete (DESTRUCTIVE) — webhook_id. In-flight deliveries are cancelled.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, create, update, delete
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
active_onlyNoOnly return active endpoints
nameYesDisplay name for the webhook
urlYesURL to receive webhook payloads
eventsYesEvent types to subscribe to
secretNoHMAC secret (auto-generated if omitted)
webhook_idYesThe webhook endpoint UUID
is_activeNoEnable or disable the webhook

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are empty, so the description fully bears the burden. It clearly discloses HMAC signing, retry with exponential backoff up to 24h, secret shown only on create, immediate invalidation on secret update, and cancellation of in-flight deliveries on delete. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear paragraph explaining the core purpose followed by a bullet list of actions. It is front-loaded with key information and contains no unnecessary sentences, making it efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 parameters, multiple actions, no output schema), the description covers behavioral traits well (signatures, retries, secret handling). However, it lacks details on output format for list/get and does not mention default values or optional parameters like deadline_ms, which are covered in schema but not in description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, but the description adds value by associating parameters with actions (e.g., 'create — url, events[], secret'). It explains secret behavior beyond schema, such as 'show once' and 'immediately invalidated'. This enhances understanding beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it manages outbound webhook endpoints and lists all available actions (list, get, create, update, delete). It specifies the resource and actions, and the name 'webhook_manage' aligns with its purpose, distinguishing it from sibling tools that manage other resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists actions but does not explicitly state when to use this tool versus alternatives or provide prerequisites (e.g., required permissions). Usage context is implied through the actions, but no when-not-to-use or alternative tool references are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

workflow_graphWorkflow Graph ToolA
Destructive

Surgical edits to a workflow's DAG (nodes + edges). For metadata changes (name, description, status, validation, AI generation) use workflow_manage. Every edit re-validates the graph; an edit that would create a cycle, an unreachable node, or an invalid edge type is rejected before commit.

Actions:

  • save_graph (write — full replace) — workflow_id, nodes[], edges[]. Atomically replaces the entire graph; existing in-flight runs continue on the old graph.

  • node_add (write) — workflow_id, type (start|end|agent|conditional|human_task|switch|dynamic_fork|do_while), config (type-specific).

  • node_update (write) — workflow_id, node_id, config (partial).

  • node_delete (DESTRUCTIVE) — workflow_id, node_id. Cascade-deletes incident edges; rejected if it would orphan nodes.

  • edge_add (write) — workflow_id, source_id, target_id; optional condition / case_value for switch nodes.

  • edge_delete (DESTRUCTIVE) — workflow_id, edge_id. Rejected if it would disconnect the graph.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: save_graph, node_update, node_add, node_delete, edge_add, edge_delete
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
workflow_idYesThe workflow UUID
nodesYesArray of node objects. Each must have: type (start|end|agent|conditional|human_task|switch|dynamic_fork|do_while), label (string), and optionally agent_id, skill_id, config.
edgesNoArray of edge objects. Each must have: source_node_index (int), target_node_index (int), and optionally condition, label, is_default.
node_idYesThe workflow node UUID
labelNoNew label for this node
agent_idNoUUID of the agent to assign to this node. Pass empty string to detach.
skill_idNoUUID of the skill to assign to this node. Pass empty string to detach.
crew_idNoUUID of the crew to assign to this node. Pass empty string to detach.
configNoNode configuration object (e.g. timeout, retries, prompt_override)
expressionNoCondition expression for conditional/switch nodes (e.g. "output.score > 0.8")
position_xNoHorizontal position on the canvas
position_yNoVertical position on the canvas
typeYesNode type (see tool description for full list and config details)
source_node_idYesUUID of the source node (where the edge originates)
target_node_idYesUUID of the target node (where the edge points to)
conditionNoCondition object for conditional edges (e.g. {"field": "score", "op": "gt", "value": 0.8})
case_valueNoCase value for switch node routing — this edge is taken when the expression equals this value
is_defaultNoMark this as the default edge when no other condition matches. Default: false
source_channelNoOutput port of the source node, e.g. "on_success", "on_error", "on_timeout"
target_channelNoInput slot of the target node (for multi-input nodes)
edge_idYesThe workflow edge UUID to delete

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already set destructiveHint: true. Description adds important behavioral details: atomic replacement for save_graph with old graph retained for in-flight runs, cascade deletion for node_delete, and rejection conditions. No contradiction with annotations. Could include more on authorization or rate limits, but sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured with clear introduction and bullet-point actions. Front-loaded with purpose and sibling pointer. Each sentence adds value, though slightly verbose. Could be trimmed but effective.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 23 parameters, 6 actions, and no output schema, the description covers all actions with behavioral notes, rejection rules, and parameter roles. Missing return value description is a minor gap, but overall complete for complex tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (baseline 3). Description adds concrete meaning beyond schema property descriptions: e.g., for node_delete explains cascade deletion and orphan rejection; for edge_add mentions optional condition/case_value. This extra context justifies +1.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states tool purpose ('Surgical edits to a workflow's DAG (nodes + edges)') and explicitly distinguishes it from sibling 'workflow_manage' for metadata changes. The verb 'edits' combined with specific action list makes purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use (DAG edits) and when-not-to-use ('For metadata changes... use workflow_manage'). Also warns that edits are rejected if they would create cycles, unreachable nodes, or invalid edge types, giving clear usage constraints.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

workflow_manageWorkflow Manage ToolA
Destructive

Workflow templates — reusable DAGs that experiments and project runs execute. This tool covers metadata and lifecycle; for graph editing use workflow_graph. Lifecycle states: draft → active → archived. Activation is gated on validate passing.

Core actions:

  • list / get (read) — optional: status filter.

  • create (write) — name, description.

  • update (write) — workflow_id + any creatable field.

  • delete (DESTRUCTIVE) — workflow_id. Soft-deletes; running experiments continue on cached graph.

  • validate (read) — workflow_id. Returns errors[] (cycles, orphans, invalid types) and warnings[].

  • activate (write) — workflow_id. Requires validation to pass.

  • duplicate (write) — workflow_id. Creates a draft copy with the same graph.

AI / cost:

  • generate (write — costs LLM credits) — prompt. Decomposes natural language into a workflow graph and saves as draft.

  • estimate_cost (read) — workflow_id. Projected per-run credit cost.

  • suggestion (read — costs LLM credits) — context (object). Recommends improvements.

Advanced:

  • time_gate (write) — workflow_id, config (delay/window). Adds time-based gating around step execution.

  • execution_chain (write) — workflow_id, chain config. Configures sequential workflow chaining.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform: list, get, create, update, validate, activate, duplicate, generate, estimate_cost, suggestion, time_gate, execution_chain
deadline_msNoOptional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.
statusNoFilter by status: draft, active, archived
limitNoMax results to return (default 10, max 100)
workflow_idYesThe workflow UUID
nameYesWorkflow name
descriptionNoWorkflow description
checkpoint_modeNoCheckpoint durability mode: sync (safest, DB write per step), async (Redis buffer + background flush), exit (in-memory, flushed on completion). Default: sync
budget_cap_creditsNoMaximum credits this workflow may consume per execution. Propagated to each experiment created from this workflow. Omit for no cap.
observability_configNoObservability provider config. Schema: {"provider":"langfuse|langsmith|none","enabled":true,"config":{"public_key":"...","secret_key":"...","host":"https://cloud.langfuse.com"}}
titleNoName for the new workflow (defaults to "<original name> (copy)")
promptYesNatural language description of the workflow to create
experiment_idYesID of the completed or evaluating workflow experiment to analyze.
event_typeNoFilter by event type: started, completed, failed, waiting_time, waiting_human, skipped

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds significant behavioral context beyond annotations: delete is soft-delete, generate and suggestion incur LLM credits, activation requires validation, time_gate and execution_chain add advanced behavior. It does not contradict the destructiveHint annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but well-structured with sections for core actions, AI/cost, and advanced. It front-loads the purpose and uses bullet points, making it scannable despite length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (12 actions, 14 params, no output schema), the description covers all actions, lifecycle states, cost implications, and edge cases like soft-delete and validation gating. It is comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by grouping parameters with actions (e.g., 'create (write) — name, description'), which provides context beyond individual schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool covers workflow template metadata and lifecycle, and distinguishes from the sibling tool workflow_graph for graph editing. It lists specific actions with their purposes, making the scope well-defined.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit context for when to use certain actions (e.g., graph editing via workflow_graph, lifecycle states, activation gated on validation). It lacks explicit exclusions for other sibling tools but is otherwise clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 34 tool updates
    • Changedadmin_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedagent_advanced1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedagent_manage16 fields changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / environment
        Added value: +{
        +  "description": "Environment preset that auto-attaches a tool bundle. Options: minimal, coding, browsing, restricted.",
        +  "enum": [
        +    "minimal",
        +    "coding",
        +    "browsing",
        +    "restricted"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / evaluation_enabled
        Added value: +{
        +  "default": false,
        +  "description": "Enable A/B evaluation for this agent",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / evaluation_sample_rate
        Added value: +{
        +  "description": "Fraction of requests to include in evaluation (0.0 to 1.0). Only used when evaluation_enabled is true.",
        +  "type": "number"
        +}
      • addedInput schema / properties / heartbeat_definition
        Added value: +{
        +  "description": "Agent health check config: {enabled: bool, cron: string, prompt: string}",
        +  "type": "object"
        +}
      • addedInput schema / properties / knowledge_base_id
        Added value: +{
        +  "description": "UUID of a knowledge base to link to this agent for RAG-powered context",
        +  "type": "string"
        +}
      • changedInput schema / properties / model / description
        Previous value: -"LLM model name (default: claude-sonnet-4-5)"New value: +"LLM model name. Defaults to platform default."
      • removedInput schema / properties / provider / default
        Removed value: -"anthropic"
      • changedInput schema / properties / provider / description
        Previous value: -"LLM provider: anthropic, openai, google (default: anthropic)"New value: +"LLM provider key (e.g. anthropic, openai, google, claude-code). Defaults to platform default."
      • removedInput schema / properties / provider / enum
        Removed value: -[
        -  "anthropic",
        -  "openai",
        -  "google"
        -]
      • addedInput schema / properties / reasoning_effort
        Added value: +{
        +  "description": "Extended thinking effort (Anthropic). Options: none, low, medium, high, auto. \"auto\" lets the platform pick by task complexity.",
        +  "enum": [
        +    "none",
        +    "low",
        +    "medium",
        +    "high",
        +    "auto"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / scope
        Added value: +{
        +  "description": "Filter by scope: team, personal",
        +  "enum": [
        +    "team",
        +    "personal"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / thinking_budget
        Added value: +{
        +  "description": "Anthropic extended thinking budget in tokens (e.g. 1024, 4096, 8192). Only applies when agent provider is \"anthropic\". Set to 0 to disable. Enables chain-of-thought reasoning visible in experiment steps.",
        +  "type": "integer"
        +}
      • addedInput schema / properties / tool_profile
        Added value: +{
        +  "description": "Tool profile restricting tool access. Options: researcher, executor, communicator, analyst, admin, minimal",
        +  "type": "string"
        +}
      • addedInput schema / properties / tool_search_top_k
        Added value: +{
        +  "description": "Maximum tools tool_search will surface per run (1–20, default 5). Only applies when use_tool_search=true.",
        +  "type": "integer"
        +}
      • addedInput schema / properties / use_tool_search
        Added value: +{
        +  "description": "Enable semantic tool auto-discovery — at run time, up to tool_search_top_k matching tools from the team pool are auto-attached based on the user prompt.",
        +  "type": "boolean"
        +}
    • Changedapproval_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedartifact_manage2 fields changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / limit / default
        Added value: +20
    • Changedassistant_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedboruna_manage8 fields changed
      • changedInput schema / properties / action / description
        Previous value: -"Action to perform: run, validate, evidence, capability_list, skill_manage"New value: +"Action to perform: run, validate, policy_validate, evidence, capability_list, skill_manage"
      • changedInput schema / properties / action / enum
        Previous value: -[
        -  "run",
        -  "validate",
        -  "evidence",
        -  "capability_list",
        -  "skill_manage"
        -]New value: +[
        +  "run",
        +  "validate",
        +  "policy_validate",
        +  "evidence",
        +  "capability_list",
        +  "skill_manage"
        +]
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / limits
        Added value: +{
        +  "description": "(inline mode, v1.0+) Optional resource limits: max_wall_ms (int) and/or max_output_bytes (int)",
        +  "type": "object"
        +}
      • changedInput schema / properties / policy / description
        Previous value: -"(inline mode) Capability policy: allow-all or deny-all (default: deny-all)"New value: +"(inline mode) Legacy capability policy shorthand: \"allow-all\" or \"deny-all\" (default: deny-all). For Boruna v0.2.0+ fine-grained gating, use policy_structured instead."
      • addedInput schema / properties / policy_json
        Added value: +{
        +  "description": "JSON-encoded Boruna policy object to validate. Must have a top-level default_allow boolean. Example: {\"default_allow\":false,\"rules\":{\"net.fetch\":{\"allow\":true}}}",
        +  "type": "string"
        +}
      • addedInput schema / properties / policy_structured
        Added value: +{
        +  "description": "(inline mode, Boruna v0.2.0+) Structured Capability Policy object with required default_allow (bool), optional rules (per-capability {allow, budget}), and optional net_policy (allowed_domains, allowed_methods, max_response_bytes, timeout_ms, allow_redirects). Capability keys: net.fetch, fs.read, fs.write, db.query, ui.render, time.now, random, llm.call, actor.spawn, actor.send. When set, takes precedence over the legacy policy parameter. See https://github.com/escapeboy/boruna/blob/v0.2.0/docs/reference/policy-schema.md.",
        +  "type": "object"
        +}
      • changedInput schema / required
        Previous value: -[
        -  "action",
        -  "mode",
        -  "execution_id"
        -]New value: +[
        +  "action",
        +  "mode",
        +  "policy_json",
        +  "execution_id"
        +]
    • Changedbridge_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedbudget_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedchatbot_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedcredential_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedcrew_manage5 fields changed
      • addedInput schema / properties / convergence_mode
        Added value: +{
        +  "description": "How to determine when the crew is done: any_validated (default), all_validated, threshold_ratio, quality_gate",
        +  "enum": [
        +    "any_validated",
        +    "all_validated",
        +    "threshold_ratio",
        +    "quality_gate"
        +  ],
        +  "type": "string"
        +}
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / min_validated_ratio
        Added value: +{
        +  "description": "Fraction of tasks that must be validated when using threshold_ratio mode (e.g. 0.8 = 80%). Default: 1.0",
        +  "type": "number"
        +}
      • changedInput schema / properties / qa_agent_id / description
        Previous value: -"UUID of the QA agent"New value: +"UUID of the QA agent. Optional — when omitted, the coordinator reviews their own work (solo-mode crew)."
      • changedInput schema / required
        Previous value: -[
        -  "action",
        -  "crew_id",
        -  "name",
        -  "coordinator_agent_id",
        -  "qa_agent_id",
        -  "goal",
        -  "execution_id"
        -]New value: +[
        +  "action",
        +  "crew_id",
        +  "name",
        +  "coordinator_agent_id",
        +  "goal",
        +  "execution_id"
        +]
    • Changedemail_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedevolution_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedexperiment_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedgit_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Addedintegration_execute
    • Changedintegration_manage3 fields changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / integration_action
        Added value: +{
        +  "description": "Driver action key, e.g. post_tweet, create_issue, send_message",
        +  "type": "string"
        +}
      • changedInput schema / required
        Previous value: -[
        -  "action",
        -  "name",
        -  "integration_id"
        -]New value: +[
        +  "action",
        +  "name",
        +  "integration_id",
        +  "integration_action"
        +]
    • Changedknowledge_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedmarketplace_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedmemory_manage5 fields changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / search_mode
        Added value: +{
        +  "default": "semantic",
        +  "description": "Retrieval mode: semantic=flat keyword search, local=1-hop graph traversal from matched entities, global=high-centrality entities, hybrid=semantic+local merged, mix=semantic+global merged",
        +  "enum": [
        +    "semantic",
        +    "local",
        +    "global",
        +    "hybrid",
        +    "mix"
        +  ],
        +  "type": "string"
        +}
      • changedInput schema / properties / tags / description
        Previous value: -"Tags for grouping and filtering memories"New value: +"Filter by tags — only return memories containing ANY of these tags. E.g. [\"barsy:client\", \"barsy:shared\"]. Omit to return all memories regardless of tags."
      • removedInput schema / properties / tags / items
        Removed value: -{
        -  "type": "string"
        -}
      • addedInput schema / properties / topic
        Added value: +{
        +  "description": "Namespace pre-filter by topic slug, e.g. \"auth_migration\". Narrows the search to a named context before the vector scan for higher precision.",
        +  "type": "string"
        +}
    • Changedoutbound_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedprofile_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedproject_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedsignal_connectors5 fields changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • changedInput schema / properties / source_type / description
        Previous value: -"Type of the source entity: person | company | location | product | topic"New value: +"Type of the source entity: person | company | organization | location | date | product | technology | event | concept | process | topic"
      • changedInput schema / properties / source_type / enum
        Previous value: -[
        -  "person",
        -  "company",
        -  "location",
        -  "product",
        -  "topic"
        -]New value: +[
        +  "person",
        +  "company",
        +  "organization",
        +  "location",
        +  "date",
        +  "product",
        +  "technology",
        +  "event",
        +  "concept",
        +  "process",
        +  "topic"
        +]
      • changedInput schema / properties / target_type / description
        Previous value: -"Type of the target entity: person | company | location | product | topic"New value: +"Type of the target entity: person | company | organization | location | date | product | technology | event | concept | process | topic"
      • changedInput schema / properties / target_type / enum
        Previous value: -[
        -  "person",
        -  "company",
        -  "location",
        -  "product",
        -  "topic"
        -]New value: +[
        +  "person",
        +  "company",
        +  "organization",
        +  "location",
        +  "date",
        +  "product",
        +  "technology",
        +  "event",
        +  "concept",
        +  "process",
        +  "topic"
        +]
    • Changedsignal_manage5 fields changed
      • changedInput schema / properties / action / description
        Previous value: -"Action to perform: list, get, ingest, connector_binding, connector_binding_delete, contact, imap, email_reply"New value: +"Action to perform: list, get, ingest, assign, connector_binding, connector_binding_delete, contact, imap, email_reply"
      • changedInput schema / properties / action / enum
        Previous value: -[
        -  "list",
        -  "get",
        -  "ingest",
        -  "connector_binding",
        -  "connector_binding_delete",
        -  "contact",
        -  "imap",
        -  "email_reply"
        -]New value: +[
        +  "list",
        +  "get",
        +  "ingest",
        +  "assign",
        +  "connector_binding",
        +  "connector_binding_delete",
        +  "contact",
        +  "imap",
        +  "email_reply"
        +]
      • addedInput schema / properties / assignee_user_id
        Added value: +{
        +  "description": "User UUID to assign to, or null to unassign",
        +  "type": "string"
        +}
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / reason
        Added value: +{
        +  "description": "Optional note added as an internal comment",
        +  "type": "string"
        +}
    • Changedskill_manage2 fields changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / framework
        Added value: +{
        +  "description": "Filter by framework key (e.g. rice, spin, bant, okrs, unit_economics). See framework_list for full enum.",
        +  "type": "string"
        +}
    • Changedsystem_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedteam_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedtool_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedtrigger_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedwebhook_manage1 field changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
    • Changedworkflow_graph3 fields changed
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • changedInput schema / properties / type / description
        Previous value: -"Node type: agent (executes an agent), conditional (branches on expression), human_task (waits for human), switch (multi-way branch), dynamic_fork (parallel split), do_while (loop)"New value: +"Node type (see tool description for full list and config details)"
      • changedInput schema / properties / type / enum
        Previous value: -[
        -  "agent",
        -  "conditional",
        -  "human_task",
        -  "switch",
        -  "dynamic_fork",
        -  "do_while"
        -]New value: +[
        +  "agent",
        +  "conditional",
        +  "human_task",
        +  "switch",
        +  "dynamic_fork",
        +  "do_while",
        +  "llm",
        +  "http_request",
        +  "parameter_extractor",
        +  "variable_aggregator",
        +  "template_transform",
        +  "knowledge_retrieval",
        +  "annotation",
        +  "iteration",
        +  "workflow_ref"
        +]
    • Changedworkflow_manage3 fields changed
      • addedInput schema / properties / budget_cap_credits
        Added value: +{
        +  "description": "Maximum credits this workflow may consume per execution. Propagated to each experiment created from this workflow. Omit for no cap.",
        +  "type": "integer"
        +}
      • addedInput schema / properties / deadline_ms
        Added value: +{
        +  "description": "Optional: max wall-clock time (ms) the tool may spend. If exceeded during the call, returns a DEADLINE_EXCEEDED error. Minimum 100 ms. Leave unset for no deadline.",
        +  "type": "number"
        +}
      • addedInput schema / properties / observability_config
        Added value: +{
        +  "description": "Observability provider config. Schema: {\"provider\":\"langfuse|langsmith|none\",\"enabled\":true,\"config\":{\"public_key\":\"...\",\"secret_key\":\"...\",\"host\":\"https://cloud.langfuse.com\"}}",
        +  "type": "object"
        +}
  2. 33 tool updatesv1.10.0
    • First observedadmin_manage
    • First observedagent_advanced
    • First observedagent_manage
    • First observedapproval_manage
    • First observedartifact_manage
    • First observedassistant_manage
    • First observedboruna_manage
    • First observedbridge_manage
    • First observedbudget_manage
    • First observedchatbot_manage
    • First observedcredential_manage
    • First observedcrew_manage
    • First observedemail_manage
    • First observedevolution_manage
    • First observedexperiment_manage
    • First observedgit_manage
    • First observedintegration_manage
    • First observedknowledge_manage
    • First observedmarketplace_manage
    • First observedmemory_manage
    • First observedoutbound_manage
    • First observedprofile_manage
    • First observedproject_manage
    • First observedsignal_connectors
    • First observedsignal_manage
    • First observedskill_manage
    • First observedsystem_manage
    • First observedteam_manage
    • First observedtool_manage
    • First observedtrigger_manage
    • First observedwebhook_manage
    • First observedworkflow_graph
    • First observedworkflow_manage

TDQS

A3.9/5.0
Disambiguation5/5

Each tool targets a distinct domain or entity (e.g., agents, approvals, credentials, experiments, signals). Even related tools like signal_connectors and signal_manage have clearly separated responsibilities (setup vs. operation). Descriptions and action lists further clarify boundaries, leaving no ambiguity.

Naming Consistency4/5

Most tools follow a consistent <noun>_manage pattern (e.g., agent_manage, credential_manage). Minor deviations exist: integration_execute, signal_connectors, and workflow_graph break the pattern but remain descriptive and predictable.

Tool Count2/5

With 34 tools, the server feels overloaded. While the platform is broad, many tools (e.g., boruna_manage, evolution_manage) are niche. The count exceeds the 'well-scoped' range and may overwhelm agents, though each tool serves a distinct purpose.

Completeness4/5

The tool surface covers the platform's major domains comprehensively: CRUD for core entities, lifecycle management, actions for special operations, and many auxiliary features. Minor gaps exist (e.g., no dedicated reporting tool), but overall the surface is robust.

Maintenance

ActivityActive
ResponsivenessWithin a week

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/escapeboy/agent-fleet-o'

If you have feedback or need assistance with the MCP directory API, please join our Discord server