Skip to main content
Glama

πŸ“‘ Table of Contents


AgentLens is a flight recorder for AI agents. It captures every LLM call, tool invocation, approval decision, and error β€” then presents it through a queryable API and real-time web dashboard.

Related MCP server: GoLogX (logx-mcp)

πŸ”’ Tamper-evident by design

What sets AgentLens apart from other observability tools: every event is SHA-256 hash-chained to the one before it, the same way git commits and blockchains are linked. The audit log is append-only and cryptographically verifiable β€” alter, delete, or reorder a single record after the fact and verification fails, pointing at the exact event that broke. Purpose-built for the record-keeping obligations of EU AI Act Article 12 and the emerging IETF Agent Audit Trail work.

See it for yourself in 30 seconds (needs Docker):

git clone https://github.com/agentkitai/agentlens && cd agentlens
./demo/aha.sh
1/5  Starting AgentLens (SQLite, zero-config)…   βœ“ up at http://localhost:3400
2/5  Ingesting a 5-event agent trace…            βœ“ 5 events ingested
3/5  Verifying the hash chain…                    βœ“ CHAIN VALID β€” no tampering detected
4/5  Tampering with one event in the database…   βœ“ altered llm_call (changed the logged model)
5/5  Re-verifying the hash chain…                 βœ— CHAIN BROKEN β€” tampering detected βœ…

The demo ingests a real trace, verifies the chain (passes), edits one record directly in the database behind the audit log's back, then re-verifies (fails). Auditors get a signed, verifiable JSON snapshot from GET /api/audit/verify/export.

Five ways to integrate β€” pick what fits your stack:

Integration

Language

Effort

Capture

πŸ”­ OpenTelemetry

Any

Point your OTLP exporter

Any gen_ai.*-instrumented agent β€” no AgentLens SDK

πŸ€– OpenClaw Plugin

OpenClaw

Copy & enable

Every Anthropic call β€” prompts, tokens, cost, tools β€” zero code

🐍 Python Auto-Instrumentation

Python

1 line

Every OpenAI / Anthropic / LangChain call β€” deterministic

πŸ”Œ MCP Server

Any (MCP)

Config block

Tool calls, sessions, events from Claude Desktop / Cursor

πŸ“¦ SDK

Python, TypeScript

Code

Full control β€” log events, query analytics, build integrations

πŸš€ Quick Start

One command β€” server + dashboard on SQLite, zero config:

docker run -p 3400:3400 -e AUTH_DISABLED=true -e JWT_SECRET=dev-secret ghcr.io/agentkitai/agentlens
# Open http://localhost:3400

Or without Docker:

npx @agentkitai/agentlens-server
# http://localhost:3400 with SQLite β€” zero config

AUTH_DISABLED=true is for a quick local trial (JWT_SECRET is still required by the hardened image). For anything shared, drop AUTH_DISABLED, set a real JWT_SECRET, and create an API key (below).

Full stack (Postgres + Redis, auth, TLS) β€” runs from source:

git clone https://github.com/agentkitai/agentlens && cd agentlens
cp .env.example .env
docker compose up
# production overlay (auth, restart policies):
docker compose -f docker-compose.yml -f docker-compose.prod.yml up

Create an API Key

curl -X POST http://localhost:3400/api/keys \
  -H "Content-Type: application/json" \
  -d '{"name": "my-agent"}'

Save the als_... key from the response β€” it's shown only once. Then head to the Integration Guides to instrument your agent.

πŸ“– Full setup guide β†’

πŸ—οΈ Architecture

graph TB
    subgraph Agents["Your AI Agents"]
        PY["Python App<br/>(OpenAI, Anthropic, LangChain)"]
        MCP_C["MCP Client<br/>(Claude Desktop, Cursor)"]
        TS["TypeScript App"]
        OC["OpenClaw Plugin"]
    end

    PY -->|"agentlensai.init()<br/>auto-instrumentation"| SERVER
    MCP_C -->|MCP Protocol| MCP_S["@agentkitai/agentlens-mcp"]
    MCP_S -->|HTTP| SERVER
    TS -->|"@agentkitai/agentlens-sdk"| SERVER
    OC -->|HTTP| SERVER

    subgraph Server["@agentkitai/agentlens-server"]
        direction TB
        INGEST[Ingest Engine]
        QUERY[Query Engine]
        ALERT[Alert Engine]
        LLM_A[LLM Analytics]
        HEALTH[Health Scoring]
        COST[Cost Optimizer]
        REPLAY[Session Replay]
        BENCH[Benchmark Engine]
        GUARD[Guardrails]
    end

    SERVER --> DB[(SQLite / Postgres)]
    SERVER --> DASH["Dashboard<br/>(React SPA)"]

    EXT["AgentGate / FormBridge"] -->|Webhook| SERVER

πŸ”§ Integration Guides

πŸ”­ OpenTelemetry (any GenAI agent β€” no SDK)

If your agent is already instrumented with the OpenTelemetry GenAI semantic conventions β€” via OpenLLMetry, OpenInference, or the official OTel instrumentations β€” just point its OTLP exporter at AgentLens. No AgentLens SDK required.

# Send standard OTLP/HTTP to AgentLens (JSON or protobuf, /v1/traces)
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:3400
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:3400/v1/traces

AgentLens maps gen_ai.* spans into its model and into the tamper-evident audit log:

OTel GenAI span (gen_ai.operation.name)

Becomes

chat / text_completion / generate_content

a paired llm_call + llm_response (model, provider, messages, usage.input_tokens/output_tokens, finish reason, latency, cost)

execute_tool

tool_call (gen_ai.tool.name, gen_ai.tool.call.id, arguments)

embeddings

embedding event with token usage

invoke_agent / create_agent

agent-invocation event

Each OTel trace maps to a session (or gen_ai.conversation.id if present), and every event is hash-chained like any other β€” so traces from any GenAI framework get the same verifiable audit trail. Set OTLP_AUTH_TOKEN to require a bearer token on the OTLP endpoints in production.

Cost with no SDK: OTel GenAI instrumentation reports tokens but rarely cost. AgentLens reconstructs costUsd from the model's per-1M-token pricing (fuzzy-matched on the model id), so OTel-only agents get the same cost analytics as SDK-instrumented ones β€” no per-call cost attribute required.

πŸ€– OpenClaw Plugin

If you're running OpenClaw, the AgentLens plugin captures every Anthropic API call automatically β€” prompts, completions, token usage, costs, latency, and tool calls.

cp -r packages/relay-plugin /usr/lib/node_modules/openclaw/extensions/agentlens-relay
openclaw config patch '{"plugins":{"entries":{"agentlens-relay":{"enabled":true}}}}'
openclaw gateway restart

Set AGENTLENS_URL if your AgentLens instance isn't on localhost:3400. See the plugin README for details.

🐍 Python Auto-Instrumentation

One line β€” every LLM call captured automatically across 9 providers (OpenAI, Anthropic, LiteLLM, AWS Bedrock, Google Vertex AI, Google Gemini, Mistral AI, Cohere, Ollama):

pip install agentlensai[all-providers]
import agentlensai

agentlensai.init(
    url="http://localhost:3400",
    api_key="als_your_key",
    agent_id="my-agent",
)
# Every LLM call is now captured automatically

Key guarantees: βœ… Deterministic Β· βœ… Fail-safe Β· βœ… Non-blocking Β· βœ… Privacy (init(redact=True))

πŸ“– Python SDK full docs β†’

πŸ”Œ MCP Integration

For Claude Desktop, Cursor, or any MCP client β€” add to your config:

{
  "mcpServers": {
    "agentlens": {
      "command": "npx",
      "args": ["@agentkitai/agentlens-mcp"],
      "env": {
        "AGENTLENS_API_URL": "http://localhost:3400",
        "AGENTLENS_API_KEY": "als_your_key_here"
      }
    }
  }
}

AgentLens ships 22 MCP tools β€” covering core observability, intelligence & analytics, and operations. Full MCP tool reference β†’

πŸ“– MCP setup guide β†’

πŸ“¦ Programmatic SDK

Python:

pip install agentlensai
from agentlensai import AgentLensClient
client = AgentLensClient("http://localhost:3400", api_key="als_your_key")
sessions = client.get_sessions()
analytics = client.get_llm_analytics()

TypeScript:

npm install @agentkitai/agentlens-sdk
import { AgentLensClient } from '@agentkitai/agentlens-sdk';
const client = new AgentLensClient({ baseUrl: 'http://localhost:3400', apiKey: 'als_your_key' });
const sessions = await client.getSessions();

πŸ“– SDK reference β†’

✨ Key Features

  • 🐍 Python Auto-Instrumentation β€” agentlensai.init() captures every LLM call across 9 providers automatically. Deterministic β€” no reliance on LLM behavior.

  • πŸ”Œ MCP-Native β€” Ships as an MCP server. Works with Claude Desktop, Cursor, and any MCP client.

  • πŸ”­ OpenTelemetry GenAI β€” Ingests gen_ai.* OTLP traces from any OTel-instrumented agent (OpenLLMetry, OpenInference, official OTel) β€” no AgentLens SDK required.

  • 🧠 LLM Call Tracking β€” Full prompt/completion visibility, token usage, cost aggregation, latency measurement, and privacy redaction.

  • πŸ“Š Real-Time Dashboard β€” Session timelines, event explorer, LLM analytics, cost tracking, and alerting.

  • πŸ”’ Tamper-Evident Audit Trail β€” Append-only event storage with SHA-256 hash chains per session.

  • πŸ’° Cost Tracking β€” Track token usage and estimated costs per session, per agent, per model. Alert on cost spikes.

  • 🚨 Alerting β€” Configurable rules for error rate, cost threshold, latency anomalies, and inactivity.

  • β€οΈβ€πŸ©Ή Health Scores β€” 5-dimension health scoring with trend tracking.

  • πŸ’‘ Cost Optimization β€” Complexity-aware model recommendation engine with projected savings.

  • πŸ“Ό Session Replay β€” Step-through any past session with full context reconstruction.

  • βš–οΈ A/B Benchmarking β€” Statistical comparison of agent variants using Welch's t-test and chi-squared analysis.

  • πŸ›‘οΈ Guardrails β€” Automated safety rules with dry-run mode for safe testing.

  • πŸ”Œ Framework Plugins β€” LangChain, CrewAI, AutoGen, Semantic Kernel β€” auto-detection, fail-safe, non-blocking.

  • πŸ”— AgentKit Ecosystem β€” Integrations with AgentGate, FormBridge, Lore, and AgentEval.

  • πŸ”’ Tenant Isolation β€” Multi-tenant support with per-tenant data scoping and API key binding.

  • 🏠 Self-Hosted β€” SQLite by default, no external dependencies. MIT licensed.

πŸ“Έ Dashboard

AgentLens ships with a real-time web dashboard for monitoring your agents.

Overview β€” At-a-Glance Metrics

Dashboard Overview

The overview page shows live metrics β€” sessions, events, errors, and active agents β€” with a 24-hour event timeline chart, recent sessions with status badges, and a recent errors feed.

Sessions β€” Track Every Agent Run

Sessions List

Every agent session with sortable columns: agent name, status, start time, duration, event count, error count, and total cost.

Session Detail β€” Timeline & Hash Chain

Session Detail

Full event timeline with tamper-evident hash chain verification. Filter by event type, view cost breakdown.

Events Explorer β€” Search & Filter Everything

Events Explorer

Searchable, filterable view of every event across all sessions.

🧠 LLM Analytics β€” Prompt & Cost Tracking

LLM Analytics

Total LLM calls, cost, latency, and token usage across all agents with model comparison.

🧠 Session Timeline β€” LLM Call Pairing

LLM Timeline

LLM calls in session timeline with model, tokens, cost, and latency.

πŸ’¬ Prompt Detail β€” Chat Bubble Viewer

LLM Call Detail

Full prompt and completion in a chat-bubble style viewer with metadata panel.

β€οΈβ€πŸ©Ή Health Overview β€” Agent Reliability

Health Overview

5-dimension health score for every agent with trend tracking.

πŸ’‘ Cost Optimization β€” Model Recommendations

Cost Optimization

Analyzes LLM call patterns and recommends cheaper model alternatives with confidence levels.

πŸ“Ό Session Replay β€” Step-Through Debugger

Session Replay

Step through any past session event by event with full context reconstruction.

βš–οΈ Benchmarks β€” A/B Testing for Agents

Benchmarks

Create and manage A/B experiments with statistical significance testing.

πŸ›‘οΈ Guardrails β€” Automated Safety Rules

Guardrails

Create and manage automated safety rules with trigger history and activity feed.

☁️ AgentLens Cloud

Don't want to self-host? AgentLens Cloud is a fully managed SaaS β€” same SDK, zero infrastructure:

import agentlensai
agentlensai.init(cloud=True, api_key="als_cloud_your_key_here", agent_id="my-agent")
  • Same SDK, one parameter change β€” switch url= to cloud=True

  • Managed Postgres β€” multi-tenant with row-level security

  • Team features β€” organizations, RBAC, audit logs

  • No server to run β€” dashboard at app.agentlens.ai

πŸ“– Cloud Setup Guide Β· Migration Guide Β· Troubleshooting

πŸ“¦ Packages

Python (PyPI)

Package

Description

PyPI

agentlensai

Python SDK + auto-instrumentation for 9 LLM providers

PyPI

TypeScript / Node.js (npm)

Package

Description

npm

@agentkitai/agentlens-server

Hono API server + dashboard serving

npm

@agentkitai/agentlens-mcp

MCP server for agent instrumentation

npm

@agentkitai/agentlens-sdk

Programmatic TypeScript client

npm

@agentkitai/agentlens-core

Shared types, schemas, hash chain utilities

npm

@agentkitai/agentlens-cli

Command-line interface

npm

@agentkitai/agentlens-dashboard

React web dashboard (bundled with server)

private

πŸ”Œ API Overview

Endpoint

Description

POST /api/events

Ingest events (batch)

GET /api/events

Query events with filters

GET /api/sessions

List sessions

GET /api/sessions/:id/timeline

Session timeline with hash chain verification

GET /api/analytics

Bucketed metrics over time

Full API Reference β†’

⌨️ CLI

npx @agentkitai/agentlens-cli health                          # Overview of all agents
npx @agentkitai/agentlens-cli health --agent my-agent          # Detailed health with dimensions
npx @agentkitai/agentlens-cli optimize                          # Cost optimization recommendations

Both commands support --format json for machine-readable output. See agentlens health --help for all options.

πŸ› οΈ Development

git clone https://github.com/agentkitai/agentlens.git
cd agentlens
pnpm install

pnpm typecheck && pnpm test && pnpm lint  # Run all checks
pnpm dev                                   # Start dev server

Requirements: Node.js β‰₯ 20.0.0 Β· pnpm β‰₯ 10.0.0

🀝 Contributing

We welcome contributions! See CONTRIBUTING.md for setup instructions, coding standards, and the PR process.

🧰 AgentKit Ecosystem

Project

Description

AgentLens

Observability & tamper-evident audit trail for AI agents

⬅️ you are here

AgentGate

Human-in-the-loop approval gateway + reactive guardrails

Lore

Cross-agent memory and lesson sharing

AgentEval

Testing & evaluation framework

FormBridge

Agent-human mixed-mode forms

πŸ“„ License

MIT Β© Amit Paz

Available Tools

22 tools
agentlens_agentsA

List, inspect, and manage AgentLens agents.

When to use: To see which agents are registered, check agent details and error rates, or unpause a paused agent.

Actions:

  • list: List all agents with error rates

  • detail: Get agent detail by ID

  • unpause: Clear paused state for an agent

Example: agentlens_agents({ action: "list" })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform
agentIdNoAgent ID (required for detail/unpause)
clearModelOverrideNoClear model override on unpause

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Each action is described (e.g., unpause clears paused state), and the tool's primary behaviors are disclosed. However, no annotations exist, and the description could further clarify that list/detail are read-only and unpause is a write operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with a clear structure: purpose, when-to-use, action list, and example. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description completely covers the tool's functionality given the 3 parameters and no output schema. It explains all actions and their outcomes adequately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description adds context by explaining what each action does (e.g., 'list: List all agents with error rates'). This supplements the schema's parameter descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List, inspect, and manage AgentLens agents' with specific actions (list, detail, unpause) differentiating it from sibling tools like agentlens_alerts or agentlens_analytics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly provides 'When to use' guidance covering listing, inspecting details, and unpausing agents, though it does not explicitly mention when not to use or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_alertsA

Manage alert rules and view alert history.

When to use: To create alerting rules for error rates, costs, or latency thresholds; manage existing rules; or review past alert triggers.

Actions:

  • list: List all alert rules

  • create: Create a new alert rule

  • update: Update an existing alert rule

  • delete: Delete an alert rule

  • history: View recent alert triggers

Example: agentlens_alerts({ action: "create", name: "High error rate", condition: "error_rate_above", threshold: 0.1, windowMinutes: 60 })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform
ruleIdNoRule ID (required for update/delete)
nameNoAlert rule name (required for create)
conditionNoCondition: error_rate_above, cost_above, latency_above (required for create)
thresholdNoThreshold value (required for create)
windowMinutesNoEvaluation window in minutes (required for create)
scopeNoScope: global or agentId
notifyChannelsNoNotification channels
enabledNoEnable/disable rule
limitNoMax history results

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It lists actions and gives an example, but doesn't disclose side effects, success/failure behavior, or idempotency. Adequate but not comprehensive for a multi-action tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured with a summary, usage section, actions list, and example. Each sentence adds value, no fluff. Front-loaded with key purpose and usage context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 10 parameters and no output schema, the description covers actions and provides an example. However, it lacks explicit details about return values for each action (e.g., what list returns). Still fairly complete for CRUD-like management.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description adds an example that maps params to actions, but doesn't provide significant additional meaning beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool manages alert rules and views history, with specific actions (list, create, update, delete, history). It distinguishes itself from sibling tools like agentlens_analytics by focusing on alerts. The example reinforces the purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states 'When to use' for creating alerting rules, managing existing rules, or reviewing triggers. Provides a concrete example. While it doesn't mention when not to use, the context is clear for typical alert management scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_analyticsA

Query operational analytics: metrics, costs, agent performance, and tool usage.

When to use: To understand system performance trends, cost breakdowns, agent activity, or tool usage patterns over time.

Actions:

  • metrics: Get bucketed metrics with optional range/date filters

  • costs: Get cost breakdown

  • agents: Get per-agent metrics

  • tools: Get tool usage statistics

Example: agentlens_analytics({ action: "metrics", range: "24h" })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform
rangeNoShorthand: 1h, 6h, 24h, 3d, 7d, 30d
fromNoStart date ISO
toNoEnd date ISO
granularityNoBucket granularity
agentIdNoFilter by agent ID

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It correctly indicates the tool is for querying (non-destructive) and lists actions. However, it does not disclose any additional behavioral traits such as pagination, rate limits, or authorization requirements. The absence of such details is acceptable for a simple query tool but could be improved.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured: a brief intro, when-to-use, actions list, and an example. Every sentence adds value, and it is front-loaded with the purpose. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 6 parameters (one required) and no output schema. The description explains the actions and provides an example, which is sufficient for a query tool. It does not explain return values, but since no output schema exists, the description could be slightly more complete regarding expected output format.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds value by explaining the 'action' parameter's options, the 'range' shorthand, and providing an example. This clarifies parameter usage beyond the schema definitions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool queries operational analytics covering metrics, costs, agent performance, and tool usage. The verb 'Query' and resource 'analytics' are specific. However, it does not explicitly differentiate from similar sibling tools like agentlens_stats, leaving some ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes a 'When to use' section that explicitly lists use cases like understanding system performance trends, cost breakdowns, etc. It provides context for when the tool is appropriate, but it does not mention when not to use it or provide alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_benchmarkA

Manage A/B benchmarks: create, list, check status, get results, and control lifecycle.

When to use: To set up controlled experiments comparing different agent configurations (models, prompts, parameters), track which variant performs better, and get statistical results.

Workflow:

  1. create β€” Define a benchmark with 2+ variants and metrics

  2. Tag sessions with variant tags during data collection

  3. start β€” Transition benchmark to running

  4. status β€” Check progress (session counts per variant)

  5. results β€” Get statistical comparison with p-values

  6. complete β€” Finalize the benchmark

Actions:

  • create: Set up a new benchmark (name, variants[], metrics[])

  • list: List benchmarks, optionally filter by status

  • status: Get benchmark detail with per-variant session counts

  • results: Get formatted comparison table with statistical analysis

  • start: Transition benchmark to running state

  • complete: Transition benchmark to completed state

Example: agentlens_benchmark({ action: "create", name: "GPT-4o vs Claude", variants: [{name: "gpt4o", tag: "v-gpt4o"}, {name: "claude", tag: "v-claude"}], metrics: ["cost", "latency", "success_rate"] })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform
nameNoBenchmark name (required for create)
descriptionNoBenchmark description
variantsNoVariants to compare (required for create, min 2)
metricsNoMetrics to track (e.g., ["cost", "latency", "success_rate"])
minSessionsNoMinimum sessions per variant before results are meaningful
agentIdNoAgent ID to scope the benchmark to
statusNoFilter by status (for list action)
benchmarkIdNoBenchmark ID (required for status/results/start/complete)

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are absent, so the description fully carries the burden. It describes lifecycle actions (create, start, complete) and what each does, but it omits details about side effects, data persistence, or required permissions. The behavioral profile is adequate but not deep.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with headers and sections, and it front-loads the main purpose. It is moderately concise; every sentence adds information, though it could be slightly trimmed without loss.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (9 parameters, 1 required, no output schema), the description covers the main actions, workflow, and key parameters. It lacks details on return values, but the example and action list provide good context. Annotations would have helped, but overall it's fairly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds value by explaining the workflow, listing required parameters per action (e.g., 'name required for create'), and providing an example. This enriches understanding beyond the raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Manage A/B benchmarks: create, list, check status, get results, and control lifecycle.' It uses specific verbs and a concrete resource (benchmarks), and it distinguishes itself from sibling tools by focusing on experiment lifecycle management.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes a 'When to use' section that explains the context ('set up controlled experiments comparing different agent configurations') and provides a workflow. While it doesn't explicitly exclude alternatives, the workflow and action list give clear guidance on typical use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_contextA

Retrieve cross-session context for a topic β€” related session summaries and lessons ranked by relevance.

When to use: At the start of a session to load relevant history, when building a system prompt with past experience, when starting work on a topic the agent has handled before, or to audit what happened with a specific topic.

What it returns: Related sessions (with summaries, key events, and relevance scores) and related lessons, all ranked by relevance to the topic. Includes an overall summary.

Example: agentlens_context({ topic: "database migrations", limit: 5 }) β†’ returns past sessions about DB migrations with key events, plus any lessons learned about migrations.

ParametersJSON Schema
NameRequiredDescriptionDefault
topicYesTopic to retrieve context for (natural language)
userIdNoFilter by user ID
agentIdNoFilter by agent ID
fromNoStart date filter (ISO 8601)
toNoEnd date filter (ISO 8601)
limitNoMaximum number of sessions to include (default: 5)

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations, so description carries full burden. Explains return structure (related sessions, lessons, relevance scores) and gives example. Does not mention read-only nature or potential caching, but is transparent for a retrieval tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One paragraph and an example, no wasted words. Front-loaded with purpose. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, but description fully explains return structure with sessions, lessons, summaries, relevance scores. Covers use cases, parameters, and example. Complete for a context retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, baseline 3. Description adds meaning by explaining tool's use of parameters (e.g., topic as natural language, limit for max sessions) and provides example. Adds moderate value beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it retrieves cross-session context for a topic, with specific verb and resource. It distinguishes from sibling tools like agentlens_sessions or agentlens_query_events by focusing on context retrieval.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly lists when to use: at start of session, building system prompt, handling familiar topic, audit. Does not state when not to use or name alternatives, but use cases are clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_cost_budgetsA

Manage cost budgets and anomaly detection.

When to use: To create/manage spending limits, check budget utilization, or configure cost anomaly detection.

Actions:

  • list: List all cost budgets

  • create: Create a new budget

  • update: Update an existing budget

  • delete: Delete a budget

  • status: Check spend vs limit for a budget

  • anomaly_config: Get anomaly detection configuration

  • anomaly_update: Update anomaly detection settings

Example: agentlens_cost_budgets({ action: "create", scope: "global", period: "daily", limitUsd: 10, onBreach: "alert" })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform
budgetIdNoBudget ID (required for update/delete/status)
scopeNoBudget scope
agentIdNoAgent ID (for agent-scoped budgets)
periodNoBudget period
limitUsdNoSpending limit in USD
onBreachNoAction on budget breach
downgradeTargetModelNoTarget model for downgrade action
enabledNoEnable/disable budget
zScoreThresholdNoZ-score threshold for anomaly detection
lookbackDaysNoLookback period in days for anomaly detection

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It lists actions including delete, but does not warn about irreversible deletion or other side effects. The example shows a create action but lacks details on error handling or performance impacts.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections (When to use, Actions, Example). It is concise, front-loaded with purpose, and every sentence adds value. No unnecessary text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 11 parameters and no output schema, the description could be more complete. It lacks details on return values for actions like status or list, and does not explain error scenarios or configuration nuances for anomaly detection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the parameters are documented. The description adds an example that maps parameters to values, but does not elaborate on parameter meaning beyond what the schema already provides. Thus, it adds marginal value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Manage cost budgets and anomaly detection.' It lists specific actions like create, update, delete, and status, making the tool's purpose precise and distinguishable from sibling tools that focus on agents, alerts, analytics, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes a 'When to use' section: 'To create/manage spending limits, check budget utilization, or configure cost anomaly detection.' This gives clear context, though it does not explicitly state when not to use or provide alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_delegateA

Delegate a task to another agent in the AgentLens network.

When to use: When you've discovered an agent capable of handling a specific task (via agentlens_discover) and want to delegate work to it.

Example: agentlens_delegate({ action: "delegate", targetAgentId: "anon-abc123", taskType: "translation", input: { text: "Hello", targetLang: "es" } })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesOperation to perform: delegate
targetAgentIdYesAnonymous agent ID (from discovery results)
taskTypeYesTask type to delegate
inputYesInput data for the delegated task
fallbackEnabledNoEnable fallback to alternative agents on failure (default: false)
maxRetriesNoMaximum retry attempts with alternative agents (default: 3, max: 10)
timeoutMsNoTimeout in milliseconds (default: 30000)

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears full responsibility. It explains the core action but does not disclose behavioral traits such as whether the call is synchronous, what happens on failure, or if the operation is reversible. The schema includes fallback and retry parameters but the description does not elaborate on their behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with four sentences front-loading the purpose, usage guidelines, and an example. No extraneous information is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 7 parameters and no output schema, the description effectively covers the usage context and prerequisite. However, it lacks information about return values and edge cases, which would be needed for full completeness. Nonetheless, it is nearly complete for a delegation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description adds no additional semantic value beyond the example usage. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'delegate' and the resource 'another agent', and distinguishes it from siblings by referencing agentlens_discover as a prerequisite. The purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states the condition for use: after discovering an agent via agentlens_discover. It also provides a concrete example, leaving no ambiguity about when to invoke this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_discoverA

Discover available agent capabilities in the network.

When to use: Before delegating a task, to find agents that can handle a specific task type. Returns ranked results with trust scores, estimated cost, and latency.

Example: agentlens_discover({ action: "discover", taskType: "code-review", minTrustScore: 70, limit: 5 })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesOperation to perform: discover
taskTypeYesTask type to search for (e.g., translation, summarization, code-review, data-extraction, classification, generation, analysis, transformation, custom)
minTrustScoreNoMinimum trust score percentile (0-100)
maxCostNoMaximum estimated cost in USD
maxLatencyNoMaximum estimated latency in milliseconds
limitNoMax results to return (default: 10, max: 20)

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It discloses that results are ranked with trust scores, estimated cost, and latency. It implies a read-only operation with no side effects, and the fixed action parameter reinforces this. While it could mention permissions or rate limits, for a discovery tool this is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences plus an example. It uses a header to highlight when to use, and the example is clear and labeled. No redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 6 parameters and no output schema, the description explains the purpose, usage context, and return type (ranked results with scores, cost, latency). It does not detail the output structure, but the mention of fields provides a sufficient mental model. For a tool with no annotations and no output schema, this is fairly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description does not add additional meaning beyond the schema; it only provides an example call. The example is helpful but not essential for understanding parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it discovers agent capabilities in the network, with a specific verb and resource. It distinguishes from sibling tools like agentlens_agents (list all) and agentlens_delegate (delegate tasks) by focusing on discovery based on task type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says when to use: before delegating, to find agents for a specific task type. It also lists what it returns (ranked results with trust scores, cost, latency). However, it does not explicitly mention when not to use or provide alternatives, which would elevate it to a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_guardrailsA

Check guardrail status for the current agent. Returns active guardrail rules, their current state, and recent trigger history.

When to use: To check what guardrails are protecting this agent, whether any have been triggered recently, and what conditions/actions are configured.

What it returns: A list of configured guardrail rules with their status (enabled/disabled, trigger count, last trigger time) and recent trigger history.

Example: agentlens_guardrails({}) β†’ returns all guardrail rules and their status.

ParametersJSON Schema
NameRequiredDescriptionDefault
agentIdNoAgent ID to check guardrails for (defaults to current agent)

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description carries the burden of behavioral disclosure. It describes the return value (list of guardrail rules with status and trigger history) but does not explicitly state that the operation is read-only or mention any side effects. However, it is clear enough for a safe read operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured with clear sections for when to use, what it returns, and an example. Every sentence adds value without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one optional parameter, no output schema), the description is complete. It explains purpose, usage, return format, and provides an example, leaving no ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description mentions the default behavior for agentId, which matches the schema. No additional semantics are added beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Check guardrail status for the current agent.' It identifies the specific verb and resource, and the focus on guardrails distinguishes it from sibling tools like agentlens_agents or agentlens_alerts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly provides a 'When to use' section, stating it is for checking guardrail protection, recent triggers, and configured conditions/actions. This gives clear context and implies when not to use it, such as for configuring guardrails.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_healthA

Check the health score of the current agent. Returns overall score (0-100), trend, and dimension breakdown.

When to use: To assess the current health and performance of the agent, to check if error rates or latency are degrading, or to get a quick overview of agent reliability metrics.

What it returns: An overall health score (0-100), a trend indicator (improving/stable/degrading), and a breakdown by five dimensions: error rate, cost efficiency, tool success, latency, and completion rate.

Example: agentlens_health({ window: 7 }) β†’ returns health score with dimension breakdown for the last 7 days.

ParametersJSON Schema
NameRequiredDescriptionDefault
windowNoRolling window in days (default: 7)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explains that the tool returns an overall score (0-100), trend indicator, and dimension breakdown. Though it does not explicitly state read-only behavior or side effects, the nature of 'checking health' implies safe, non-destructive operation, and the return structure is clearly defined.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured with clear sections (description, when to use, what it returns, example). Every sentence adds value without redundancy. Front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (one optional parameter), the description fully covers what the tool does, when to use it, and what data it returns. No output schema is provided, but the description details the return structure, making it complete for agent decision-making.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (one parameter 'window' with a description). The description includes an example but does not add additional semantic meaning beyond what the schema already provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool checks the health score of the current agent, with a specific verb ('check') and resource ('health score'). It distinguishes itself from sibling tools like agentlens_stats and agentlens_agents by focusing solely on health assessment.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use the tool: 'To assess the current health and performance of the agent, to check if error rates or latency are degrading, or to get a quick overview of agent reliability metrics.' It lacks explicit when-not-to-use or alternative tool references but still offers clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_log_eventB

Log an event to an active AgentLens session.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesSession ID from agentlens_session_start
eventTypeYesEvent type (e.g., tool_call, tool_response, custom)
payloadYesEvent payload β€” structure depends on eventType
severityNoSeverity level (default: info)
metadataNoArbitrary metadata (tags, labels, correlation IDs)

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description is too brief to disclose behavioral traits such as whether logging is synchronous, what happens if the session is inactive, or if there are rate limits. The description adds no value beyond the input schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words. However, it could include more useful information without becoming verbose, so it is good but not perfect.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 5 parameters and no output schema, the description lacks necessary context about return values, side effects, and how logged events relate to the AgentLens system. It does not mention that events can be queried later or the consequences of incorrect usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description does not add any extra meaning beyond the schema definitions, warranting the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Log' and the resource 'event' targeted at 'active AgentLens session'. It distinguishes this tool from related siblings like agentlens_log_llm_call and agentlens_query_events, which have more specific purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives (e.g., log_llm_call for LLM-specific events). It does not mention prerequisites like having an active session or that events can be queried later with agentlens_query_events.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_log_llm_callB

Log a complete LLM call (request + response) to an active AgentLens session. Emits paired llm_call and llm_response events.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesSession ID from agentlens_session_start
providerYesLLM provider name (e.g., "anthropic", "openai", "google")
modelYesModel identifier (e.g., "claude-opus-4-6", "gpt-4o")
messagesYesThe prompt messages sent to the model
systemPromptNoSystem prompt (if separate from messages)
completionYesThe completion content returned by the model
toolCallsNoTool calls requested by the model
finishReasonYesStop reason (e.g., "stop", "length", "tool_use", "content_filter", "error")
usageYesToken usage counts
costUsdYesCost of this call in USD
latencyMsYesLatency in milliseconds
parametersNoModel parameters (temperature, maxTokens, etc.)
toolsNoTool/function definitions provided to the model

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavior. It mentions 'emits paired events' but does not describe side effects (e.g., mutating the session), required permissions, error states, or whether it overwrites or appends data. The mutation is implied but not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence of 18 words. Every word is necessary and no space is wasted. It is well-structured for quick understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (13 parameters, 9 required, nested objects, no output schema), the description is too brief. It fails to mention required parameters, usage patterns, or any caveats about the session state. A more detailed description is needed to guide correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add additional meaning beyond what the schema already provides. It does not explain relationships between parameters (e.g., messages vs systemPrompt) or provide examples. The description adds no extra semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly specifies the verb ('Log') and the resource ('complete LLM call'), and mentions the emitted events ('llm_call and llm_response'). It distinguishes itself from sibling tools like agentlens_log_event, which logs generic events, making it obvious when to use this tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide any guidance on when to use this tool versus alternatives, such as agentlens_log_event. It also fails to specify prerequisites (e.g., requiring an active session from agentlens_session_start) or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_optimizeA

Get cost optimization recommendations. Analyzes LLM call patterns and suggests cheaper model alternatives.

When to use: To identify cost-saving opportunities by switching expensive models to cheaper alternatives for tasks that don't require the most capable model. Analyzes call complexity (simple/moderate/complex) and success rates.

What it returns: A list of model switch recommendations with estimated monthly savings, confidence levels, and success rate comparisons. Sorted by potential savings.

Example: agentlens_optimize({ period: 7 }) β†’ returns recommendations like "Switch gpt-4o β†’ gpt-4o-mini for SIMPLE tasks, saving $89/month".

ParametersJSON Schema
NameRequiredDescriptionDefault
periodNoAnalysis period in days (default: 7, max: 90)
limitNoMax recommendations to return (default: 5, max: 50)

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It explains the tool analyzes patterns and returns recommendations, but doesn't explicitly state that no changes are made to the system, which could be inferred. It lacks details on authentication or rate limits, but these are less relevant for an analysis tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with clear sections: summary, when to use, what it returns, and an example. Every sentence adds value, and it is front-loaded with the main purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description adequately explains the return format (list of recommendations with savings, confidence, success rate comparisons). It is complete enough for a simple analysis tool, though it could explicitly mention that no actions are taken on the system.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters have descriptions in the schema (100% coverage), and the description adds an example call showing usage and expected return format, including savings, confidence levels, and success rate comparisons. This adds value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool provides cost optimization recommendations by analyzing LLM call patterns and suggesting cheaper model alternatives. This differentiates it from sibling tools like cost_budgets, which focus on budget management.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes a dedicated 'When to use' section that explains the tool is for identifying cost-saving opportunities by switching to cheaper models for tasks that don't require the most capable model. It provides clear context but doesn't explicitly list alternative tools or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_promptsC

Manage prompt templates and versions.

Actions:

  • list: List prompt templates (optional category, search filters)

  • get: Get a template with all versions by ID

  • create: Create a new prompt template with initial content

  • update: Create a new version of an existing template

  • analytics: Get per-version metrics for a template

  • fingerprints: List auto-discovered prompt fingerprints

Example: agentlens_prompts({ action: "list", category: "system" })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform
templateIdNoTemplate ID (for get, update, analytics)
nameNoTemplate name (for create)
contentNoPrompt content (for create, update)
descriptionNoTemplate description (for create)
categoryNoCategory filter or value
variablesNoJSON array of variable definitions (for create)
changelogNoChange description (for update)
searchNoName search filter (for list)
fromNoStart date ISO (for analytics)
toNoEnd date ISO (for analytics)
agentIdNoAgent ID filter (for fingerprints)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description lacks behavioral details beyond action names. No annotations are provided, so the description should disclose side effects, permissions, or behavior (e.g., what 'create' returns, whether updates are versioned). It only briefly mentions 'auto-discovered prompt fingerprints' without elaboration.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, well-structured with a header, bullet actions, and an example. It is front-loaded with purpose. However, the action list could be more compactly described.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 12 parameters and 6 actions, the description lacks detail for each action (e.g., return values, metric definitions). No output schema. Missing complete behavior for 'create', 'update', 'analytics', and 'fingerprints'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter has a description indicating which actions it applies to. The tool description echoes this with a list, but adds no additional semantics like constraints, defaults, or usage examples. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool manages prompt templates and versions, and lists specific actions. However, it does not differentiate from sibling tools like agentlens_agents or agentlens_context, lacking sibling distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. The description lists actions but does not provide context or prerequisites for choosing this tool over sibling prompt-related tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_query_eventsC

Query events from an AgentLens session.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesSession ID to query events from
limitNoMaximum number of events to return (default: 50)
eventTypeNoFilter by event type

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description implies a read-only operation ('query'), but with no annotations (e.g., readOnlyHint), the agent gets minimal behavioral clues. Absent details like pagination, sorting, or whether events are returned in chronological order, the description lacks transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely conciseβ€”just one sentenceβ€”with no superfluous words. However, it may be too terse, missing important context that could be added without much bloat.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (3 parameters, no output schema, no annotations), the description is incomplete. It fails to mention return format, potential event types, or behavior when limit is exceeded. The agent would need to guess or inspect schema only.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All three parameters have descriptions in the input schema (100% coverage), so the description doesn't add new meaning beyond what's already provided. A baseline score of 3 is appropriate since the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'query' and the resource 'events from an AgentLens session', making the tool's purpose straightforward. However, it does not differentiate from sibling tools like agentlens_log_event or agentlens_reflect, which might also involve querying session data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. Sibling tools include agentlens_log_event (which logs events) and agentlens_replay (which may replay events), but the description offers no distinctions or usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_reflectA

Analyze behavioral patterns from agent sessions β€” error patterns, tool sequences, cost analysis, and performance trends.

When to use: To identify recurring errors and their root causes (error_patterns), to understand cost drivers and optimize model usage (cost_analysis), to discover common tool usage chains and their success rates (tool_sequences), or to track performance over time (performance_trends).

What it returns: A list of structured insights with type, summary, data, and confidence score, plus metadata about how many sessions/events were analyzed. Each analysis type returns different data shapes.

Example: agentlens_reflect({ analysis: "error_patterns", agentId: "my-agent", from: "2026-01-01" }) β†’ returns recurring error patterns with counts, first/last seen, and affected sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault
analysisYesType of analysis to run: error_patterns (recurring errors), tool_sequences (common tool usage patterns), cost_analysis (cost breakdown and trends), performance_trends (success rate and duration trends)
agentIdNoFilter analysis to a specific agent
fromNoStart of time range (ISO 8601)
toNoEnd of time range (ISO 8601)
paramsNoAdditional parameters (e.g., { model: "gpt-4o" } for cost_analysis)
limitNoMaximum number of results to return (default: 20)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It describes the return value as a list of structured insights with type, summary, data, confidence score, and metadata, and gives an example. It does not discuss authorization, rate limits, or destructive actions, but the behavioral transparency is good for a non-destructive analysis tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections: overall purpose, when to use, what it returns, and an example. Every sentence adds value, and there is no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (6 parameters, nested objects, no output schema), the description covers the main aspects: purpose, usage guidelines, return value shape, and an example. It could mention error handling or pagination, but it is complete enough for the agent to understand and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaning by explaining the analysis enum values with context in the 'When to use' section and providing a concrete example that shows how parameters are used together. This additional context aids understanding beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool analyzes behavioral patterns from agent sessions, listing four specific analysis types (error_patterns, tool_sequences, cost_analysis, performance_trends). It distinguishes from sibling tools like agentlens_agents or agentlens_stats by focusing on reflection and patterns rather than listing agents or raw statistics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is an explicit 'When to use' section that details use cases for each analysis type, including identifying recurring errors, understanding cost drivers, and tracking performance. It does not explicitly state when not to use the tool or mention alternatives among siblings, but the guidance is clear and context-rich.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_replayA

Replay a past session as a structured, human-readable timeline.

When to use: To review what happened in a previous session β€” understand failures, decision patterns, timing, or cost accumulation. Great for debugging or post-mortem analysis.

What it returns: A session header (agent, status, duration, cost, event counts) followed by numbered, timestamped steps with event type icons and context annotations.

Parameters:

  • sessionId (required): The session to replay

  • fromStep/toStep: Replay a specific step range

  • eventTypes: Comma-separated filter (e.g., "llm_call,tool_call")

  • summaryOnly: Set true to get just the summary header (fast for large sessions)

Example: agentlens_replay({ sessionId: "ses_abc123", summaryOnly: true }) β†’ returns session summary without steps.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesSession ID to replay
fromStepNoStart step number (0-based)
toStepNoEnd step number (inclusive)
eventTypesNoComma-separated event types to filter (e.g., "llm_call,tool_call")
summaryOnlyNoReturn only the summary header (no steps). Default: false

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are given, so the description carries the full burden. It discloses the return format (session header with agent, status, duration, cost, event counts, followed by numbered steps) and special behavior like summaryOnly mode for large sessions. This provides sufficient behavioral context beyond the input schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with clear sections (general purpose, when to use, what it returns, parameters, example). It is informative but not excessively long. The information is front-loaded and each sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 5 parameters with 100% schema coverage and no output schema, the description adequately explains the return values (header plus steps) and parameter nuances. It is complete enough for the tool's complexity, though an explicit note on pagination or step limits would be a minor addition.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by explaining the purpose of fromStep/toStep (specific step range), eventTypes (comma-separated filter), and summaryOnly (fast for large sessions). It also provides a concrete example, making parameter usage clearer than the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool replays a past session as a structured timeline. It uses a specific verb (replay) and resource (session), and the context of debugging or post-mortem analysis distinguishes it from sibling tools like agentlens_sessions or agentlens_analytics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly provides 'When to use' with concrete scenarios (understand failures, decision patterns, timing, cost accumulation). It lacks explicit 'when not to use' or mention of alternatives, but the provided use cases are clear enough for an AI to decide.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_session_endB

End an active AgentLens monitoring session.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesSession ID to end
reasonYesReason for ending the session
summaryNoOptional summary of the session

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavioral traits. It only states 'End', omitting consequences such as whether data is saved, if the session can be resumed, or any cleanup actions. This is insufficient for an agent to understand side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no redundant words. Information is front-loaded and directly addresses the tool's core function. Highly concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (3 parameters, no output schema), the description provides minimal but adequate context for an agent to understand the basic function. However, missing behavioral details reduce completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description adds no extra meaning beyond the schema. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb (End) and resource (AgentLens monitoring session), distinguishing it from sibling tools like agentlens_session_start. However, it could be more specific about what 'end' entails (e.g., stop and archive).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidelines on when to use this tool versus alternatives. While the context implies it is the counterpart to agentlens_session_start, there is no mention of prerequisites or related tools, leaving the agent to infer usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_sessionsA

Browse and inspect AgentLens sessions.

When to use: To find past sessions, inspect session details, or view a timeline of events within a session. Useful for debugging, auditing, or reviewing agent activity.

Actions:

  • list: List sessions with optional filters (agentId, status, date range, tags)

  • detail: Get full session detail with aggregates

  • timeline: Get timestamped event list for a session

Example: agentlens_sessions({ action: "list", agentId: "my-agent", status: "completed", limit: 10 })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform
sessionIdNoSession ID (required for detail/timeline)
agentIdNoFilter by agent ID (list)
statusNoFilter by status: active, completed, error (list)
fromNoStart date ISO (list)
toNoEnd date ISO (list)
tagsNoFilter by tags (list)
limitNoMax results, default 20 (list)
offsetNoPagination offset (list)

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided. The description uses 'Browse and inspect' implying read-only, but does not explicitly confirm non-destructive behavior or provide other behavioral traits. The actions are query-based, adding some transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise: 3 sentences plus a bullet list of actions. It's well-structured with sections and an example, no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 9 params and no output schema, the description covers all actions and key parameters. Example shows typical usage. Lack of return value documentation is acceptable given no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description groups parameters by action (e.g., sessionId required for detail/timeline) and provides an example, adding meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Browse and inspect AgentLens sessions' and lists specific actions (list, detail, timeline). It distinguishes from sibling tools like agentlens_session_start by focusing on past session inspection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'When to use' section explicitly states the tool is for finding/inspecting past sessions, debugging, auditing, reviewing. It doesn't explicitly say when not to use, but the sibling context implies alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_session_startA

Start a new AgentLens monitoring session. Returns a sessionId to use for subsequent events.

ParametersJSON Schema
NameRequiredDescriptionDefault
agentIdYesUnique identifier for the agent
agentNameNoHuman-readable agent name
tagsNoTags for categorizing this session

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the basic behavior (starting a session, returning an ID) but omits details like session expiration, concurrency limits, or required permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero wasted words. The first sentence states the primary purpose, and the second adds the key return value. Highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple session creation tool with no output schema and absent annotations, the description covers the essential purpose and return value. It could mention uniqueness or limitations of session IDs, but overall adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description does not add any parameter-specific context beyond what the schema already provides. Baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action ('Start') and resource ('a new AgentLens monitoring session'), and specifies the return value ('Returns a sessionId'). It distinguishes itself from sibling tools like agentlens_session_end and agentlens_sessions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for initiating monitoring, but provides no explicit guidance on when to use vs. alternatives (e.g., agentlens_sessions for listing), nor any prerequisites or restrictions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_statsA

Get storage statistics and system overview metrics.

When to use: To check database/storage utilization or get a high-level system overview.

Actions:

  • storage: Get storage stats (database size, event counts, etc.)

  • overview: Get overview metrics (active sessions, agents, recent activity)

Example: agentlens_stats({ action: "storage" })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It describes the two actions and their outputs (storage stats, overview metrics) and includes an example. However, it does not disclose whether the tool has side effects, requires authentication, or has rate limits. Given the read-only nature implied by 'get', a 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with no wasted words. It uses a clear structure: one-line summary, when-to-use sentence, bulleted actions with descriptions, and a code example. Every sentence serves a purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that the tool has only one parameter with 100% schema coverage and no output schema, the description does enough. It explains both action values and gives an example. While it could be more precise about the exact fields in the output (e.g., specific metrics returned), it is not required since there is no output schema to complement.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a single parameter 'action' defined as an enum. The description adds meaning by explaining the enum values ('storage': Get storage stats, 'overview': Get overview metrics) beyond the schema's 'Action to perform'. This helps the agent understand the parameter's semantics clearly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves storage statistics and system overview metrics. It distinguishes itself from sibling tools by specifying 'storage statistics' and 'system overview' which are not covered by other agentlens_* tools like agentlens_agents or agentlens_alerts. However, it could be more precise about the exact scope of metrics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use guidance: 'To check database/storage utilization or get a high-level system overview.' It also lists the two actions with descriptions, which helps an agent decide which action to invoke. Does not explicitly state when not to use or list alternatives, but the sibling tools cover other domains, so the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

agentlens_trustA

Get trust scores for agents.

When to use: To check the trust/reliability score of an agent before delegating tasks or to monitor agent reputation.

Actions:

  • score: Get trust score for a specific agent

Example: agentlens_trust({ action: "score", agentId: "my-agent" })

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesAction to perform
agentIdNoAgent ID (required for score)

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden. It only states the function and gives an example, but fails to disclose the return format, whether it's read-only, any side effects, or error conditions. This is insufficient for a tool with no annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very concise, with a clear purpose statement, a 'When to use' section, and a code example. No wasted words, well-structured for an AI agent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool lacks an output schema, so the description should explain the return value. It does not describe what the trust score looks like (e.g., numeric range, confidence level). Also, no error handling info. This leaves the agent guessing about the response format.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the description adds value by explaining the action enum (only 'score') and clarifying that agentId is required for score. The example further illustrates usage, going beyond the schema's bare descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Get trust scores for agents', which is a specific verb and resource. It distinguishes from siblings like agentlens_agents or agentlens_health by focusing on trust scores.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit 'When to use' guidance: to check trust before delegating or monitor reputation. Though it doesn't mention when not to use or alternatives, the context is clear and helpful.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 22 tool updatesv1.0.0
    • First observedagentlens_agents
    • First observedagentlens_alerts
    • First observedagentlens_analytics
    • First observedagentlens_benchmark
    • First observedagentlens_context
    • First observedagentlens_cost_budgets
    • First observedagentlens_delegate
    • First observedagentlens_discover
    • First observedagentlens_guardrails
    • First observedagentlens_health
    • First observedagentlens_log_event
    • First observedagentlens_log_llm_call
    • First observedagentlens_optimize
    • First observedagentlens_prompts
    • First observedagentlens_query_events
    • First observedagentlens_reflect
    • First observedagentlens_replay
    • First observedagentlens_session_end
    • First observedagentlens_session_start
    • First observedagentlens_sessions
    • First observedagentlens_stats
    • First observedagentlens_trust

TDQS

A3.8/5.0
Disambiguation5/5

Each tool targets a distinct domain (agents, alerts, analytics, benchmarks, etc.) with clear boundaries. Even closely related tools like agentlens_sessions and agentlens_session_start/end are differentiated by lifecycle management vs. browsing. No significant overlap.

Naming Consistency5/5

All tools follow a consistent 'agentlens_' prefix with snake_case naming. The pattern is uniform across all 22 tools, with verb_noun style for actions (e.g., agentlens_session_start, agentlens_log_event) and noun style for collections (e.g., agentlens_agents, agentlens_alerts).

Tool Count4/5

22 tools is slightly above the typical 3-15 range but still appropriate for a comprehensive monitoring platform. Each tool serves a clear purpose, and the count reflects the breadth of functionality (monitoring, alerts, budgets, benchmarks, delegation, etc.) without being excessive.

Completeness5/5

The tool set provides full coverage for agent monitoring: session lifecycle, metrics, alerts, cost budgets, benchmarks, delegation, trust, guardrails, logging, prompt management, optimization, and reflection. It covers all common operational needs with no obvious gaps for the stated domain.

Maintenance

ActivitySlowing
ResponsivenessResponsive

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    B
    maintenance
    Universal MCP server that emits Context Passport records for AI agent decisions and actions. Drop into any MCP-compatible client to give your agent a commit/verify/replay/export toolset for verifiable, tamper-evident records.
    5
    293
    Apache 2.0
  • A
    license
    Not graded
    quality
    C
    maintenance
    Tamper-evident audit logging for AI agents. Append-only, hash-chained, optionally Ed25519-signed log. The MCP server lets an agent keep and verify a record of what it actually did.
    7
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    MCP server that provides cryptographic audit trails for AI agent actions, making every action tamper-evident via HMAC-SHA256 signed hash chains.
    Apache 2.0
  • A
    license
    Not graded
    quality
    A
    maintenance
    Local-first MCP server that lets AI agents query their own LLM call history as a branchable DAG and offload conversation context into immutable, AES-256-GCM-encrypted capsules β€” restorable in full or per segment, crypto-shreddable, with RAID-style replication. 12 tools, no API keys, no cloud.
    179
    3
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/agentkitai/agentlens'

If you have feedback or need assistance with the MCP directory API, please join our Discord server