This server provides LLM observability tools to query and analyze your Spanlens workspace data — giving AI assistants direct access to usage metrics, traces, anomalies, and cost insights.
get_stats: Retrieve workspace-level summaries of LLM cost, request count, latency, and error rates over a chosen timeframe (1h, 24h, 7d, 30d), with optional breakdown by model or provider.
query_requests: List and filter individual LLM calls with details like cost, latency, model, status, and error messages — filterable by model, provider, status, end-user ID, or time range.
list_traces: Browse agent/multi-step workflow traces with summary info (name, status, duration, span count, tokens, cost), optionally filtered by status, time, or search query.
get_trace: Fetch the complete span tree for a single agent trace, including every LLM, tool, and retrieval span with timing, token usage, and cost — ideal for debugging slow or failed agent runs.
get_anomalies: Surface unacknowledged statistical anomalies (cost spikes, latency deviations, error-rate surges) measured in standard deviations from your 7-day baseline, with configurable sensitivity.
get_savings: Retrieve model-swap recommendations projecting monthly cost savings, including whether a recommendation has already been adopted.
get_user_analytics: View per-end-user usage breakdowns (request counts, cost, latency, models used) — either as a top-N list or a detailed view for a specific user.
Provides integration for CrewAI agents, allowing observability of LLM calls within CrewAI workflows.
Provides a callback handler integration to trace LangChain chains and record LLM calls for observability.
Provides a callback handler integration to trace LangGraph workflows for observability.
Allows recording and monitoring of local Ollama LLM calls via a wrapped client, capturing trace data for observability.
Allows recording and monitoring of OpenAI LLM calls, including cost, latency, tokens, and full request/response bodies.
Provides integration with the Vercel AI SDK via a tracker to record and monitor LLM calls.
Open-source LLM observability you can turn on in one line. Point your OpenAI, Anthropic, or Gemini client at Spanlens and every call is logged with cost, tokens, latency, and full agent traces. No SDK rewrite, no platform migration. Eleven providers supported, plus native Vercel AI SDK, LangChain, and LlamaIndex integrations, and you can query it all from Cursor or Claude Desktop through the bundled MCP server. Self-hostable in one Docker command. MIT.
Why it exists. I shipped an LLM app on OpenAI and Gemini and hit a wall. The provider dashboards showed total spend and nothing else. I could not tell which feature burned the most tokens, which model was cheapest per task, or what each endpoint actually cost. Spanlens is the layer I wanted. It turns on in one line, stays off the critical path, and is open source so you can self-host the exact code we run.
⭐ If Spanlens is useful to you, please star the repo. It takes a second, and it is the single biggest thing that helps other developers find the project.
Hosted: spanlens.io · npm:
@spanlens/sdk· PyPI:spanlens· CLI:@spanlens/cli· MCP:@spanlens/mcp-server· Status: status.spanlens.io · Changelog: spanlens.io/changelog
Live demo (no signup): spanlens.io/demo/requests
Helicone was acquired and its roadmap is uncertain.
Langfuse is powerful but complex to set up and expensive to scale.
Spanlens ships the 20% of features that cover 80% of real production needs. You get request log, cost tracking, agent tracing, anomaly detection, PII scanning, and prompt versioning with a clean UI, a two-minute setup, and pricing that doesn't punish growth.
Spanlens | Langfuse Pro | Helicone | |
Open source | ✅ MIT | ✅ MIT | ✅ MIT |
Self-hostable | ✅ Docker one-liner | ✅ | ✅ |
Free tier | 50K req/mo | 50K events/mo | 10K req/mo |
Team plan (1M req/mo) | $149/mo | $271/mo | ~$200/mo |
Agent tracing | ✅ | ✅ | ⚠️ limited |
LLM-as-judge evals | ✅ | ✅ | ❌ |
PII + injection scan | ✅ | ❌ | ❌ |
Model recommendations | ✅ | ❌ | ❌ |
Prompt A/B experiments | ✅ | ✅ | ❌ |
Predictable bills, no quota cliff. Free hits a hard 429 at 50K requests so a runaway loop in dev can't cost you money. Paid plans use a soft limit with authorized overage (Pro: +$8 / 100K, Team: +$5 / 100K) up to a hard cap you control, so a traffic spike charges you fairly instead of dropping requests.
Seats: Free 1 · Pro 3 · Team 10 · Enterprise unlimited. Unlimited projects on every paid tier.
⭐ Like where this is going? A star helps more developers find a lightweight, open alternative in a space full of heavy, acquired tools.
Related MCP server: iris-eval/mcp-server
npx @spanlens/cli initThe wizard:
Installs @spanlens/sdk with your package manager (npm / pnpm / yarn / bun)
Writes SPANLENS_API_KEY to .env.local
Rewrites every new OpenAI({ apiKey, baseURL }) into createOpenAI()
Paste your Spanlens API key once, confirm two prompts, done. Your LLM calls are now flowing through the Spanlens proxy and visible in www.spanlens.io/requests.
import { createOpenAI } from '@spanlens/sdk/openai'
const openai = createOpenAI() // reads SPANLENS_API_KEY, uses Spanlens proxy baseURLpip install "spanlens[openai]"from spanlens.integrations.openai import create_openai
client = create_openai() # reads SPANLENS_API_KEY from env
res = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello"}],
)For agent tracing in Python (multi-step, async, tool calls) see the Python SDK README.
Already using an orchestration framework? Plug Spanlens in as a callback. No code rewrites.
Vercel AI SDK (Next.js / edge friendly)
import { SpanlensClient } from '@spanlens/sdk'
import { createSpanlensTracker } from '@spanlens/sdk/vercel-ai'
const tracker = createSpanlensTracker({
client: new SpanlensClient({ apiKey: process.env.SPANLENS_API_KEY! }),
modelName: 'gpt-4o',
})
await generateText({
model: openai('gpt-4o'),
messages,
onStepFinish: tracker.onStepFinish,
onFinish: tracker.onFinish,
})LangChain JS / LangGraph
import { createSpanlensCallbackHandler } from '@spanlens/sdk/langchain'
const handler = createSpanlensCallbackHandler({ client })
await chain.invoke({ input }, { callbacks: [handler] }) // LangChain
await graph.invoke({ input }, { callbacks: [handler] }) // LangGraphLlamaIndex TS
import { Settings } from 'llamaindex'
import { registerSpanlensCallbacks } from '@spanlens/sdk/llamaindex'
const unregister = registerSpanlensCallbacks(Settings, { client })
// ... run queries ... unregister() on shutdownPython: LangChain: from spanlens.integrations.langchain import SpanlensCallbackHandler. Same BaseCallbackHandler contract, works with chains, LCEL, and LangGraph.
More integrations: AWS Bedrock, CrewAI, Flowise, Instructor, LlamaIndex, OpenAI Assistants, MCP server. Full setup walkthroughs at spanlens.io/docs/integrations.
Ollama (local LLMs): Ollama runs on your machine, so it does not go through the hosted proxy. Get a ready client with createOllama() and wrap each call with observeOllama() so the span is logged and tagged as Ollama.
import { SpanlensClient } from '@spanlens/sdk'
import { createOllama, observeOllama } from '@spanlens/sdk/ollama'
const spanlens = new SpanlensClient({ apiKey: process.env.SPANLENS_API_KEY! })
const ollama = createOllama() // points at http://localhost:11434/v1
const trace = spanlens.startTrace({ name: 'chat' })
const res = await observeOllama(trace, 'chat', (headers) =>
ollama.chat.completions.create(
{ model: 'llama3.1', messages: [{ role: 'user', content: 'Hello' }] },
{ headers },
),
)
await trace.end({ status: 'completed' })
Every request logged with model, provider, latency, tokens, cost, and full prompt + response body. Filter, search, export. Streaming responses reconstructed automatically.
Feature | Description |
Request log | Every LLM call logged with model, tokens, cost, latency, and full request/response body (streaming reconstructed too) |
Agent tracing | Multi-step workflows as Gantt waterfall span trees with Critical Path highlighted (the longest dependency chain across a fan-out, not just the slowest single span), plus a node-and-edge graph topology view for LangChain / LangGraph callback traces |
Cost tracking | Per-request cost breakdown with daily rollups and budget alerts. Prompt-cache tokens ( |
Per-end-user analytics | Tag calls with |
Anomaly detection | 3σ deviations in latency, cost, or error rate vs. your 7-day baseline, with root-cause hints (token delta, HTTP status breakdown) |
Alerts | Threshold rules on budget, error rate, and p95 latency. Delivered via Email (Resend), Slack, or Discord webhooks. Evaluated on a 15-minute cron with at-least-once delivery |
PII + prompt-injection scan | Regex-based detection on request and response bodies; optional per-project blocking (422) for injections; instant alert emails to workspace owner |
Savings (model recommendations) | The |
Response caching | Opt in per request with |
Email digests & health alerts | A weekly workspace digest (requests, cost with week-over-week change, top models, anomalies) lands every Monday, and a data-silence alert emails admins when a workspace that was sending traffic suddenly goes quiet for 24 hours, so a broken key or dropped env var is caught before it becomes silent churn |
Prompt versioning + A/B | Register prompt templates, run traffic-split experiments, compare versions side by side on latency / cost / error rate, reported with Welch's t-test on latency and cost plus a z-test on error rate, so you get statistical significance rather than just averages |
Prompts Playground | Execute any prompt version with variable injection directly in the dashboard to see real cost and response before shipping |
Datasets | Reusable (input, expected_output) test sets you can rerun against any prompt version or model. Upload CSV / JSONL files directly from the dashboard or POST programmatically. Powers offline evals and regression checks |
Evals & Experiments | Build LLM-as-judge evaluators (judge with OpenAI, Anthropic, or Gemini, whichever is cheapest or best for the criterion) with rubric anchors and confidence intervals on pass rates. Supports pairwise A vs B mode for head-to-head prompt comparison, agent trajectory mode for scoring whole traces (not just final text), and judge-result caching keyed by |
OpenAPI 3.0 spec + Swagger UI | Machine-readable spec at |
Saved filters | Pin frequently used request-log queries (model, status, cost range, tags) and share them across the workspace |
Outbound webhooks | Subscribe to |
OpenTelemetry / OTLP ingest |
|
Provider-key security | Weekly digest emails for stale (unused 90d+) provider keys + daily GitGuardian leak scan against your active keys, with per-key scan history |
Privacy controls | Per-request |
Data export | CSV or JSON download for requests, traces, anomalies, and flagged security events ( |
Spanlens is multi-user out of the box. Invite teammates, hand out roles, and spin up a separate workspace per client.
Roles are admin (members + billing), editor (data + settings), and viewer (read-only). The last admin is protected against demotion / removal.
Email invitations have a 7-day expiry with sha256-hashed tokens. Sent via Resend when RESEND_API_KEY is set; falls back to console-logging the accept URL for local dev.
The pending-invitation banner surfaces unaccepted invites at the top of the dashboard, even if the recipient never opened the email. Accept joins and auto-switches the active workspace; Decline removes the row.
Multi-workspace lets you switch between workspaces from the sidebar (sb-ws cookie + hard reload so middleware re-resolves scope). Useful for consultants juggling multiple clients or one team running prod / staging as separate workspaces.
Two-step onboarding sends new signups to /onboarding: name your workspace, answer two optional survey questions, done. Invitees get a short-circuited variant where Accept skips workspace creation entirely.
The audit log (Settings → Audit log) records every membership / role / invitation event with actor + timestamp.
Spanlens/
├── apps/
│ ├── web/ Next.js 16 dashboard (www.spanlens.io)
│ └── server/ Hono LLM proxy + REST API (api.spanlens.io)
├── packages/
│ ├── sdk/ @spanlens/sdk: TypeScript / JavaScript SDK
│ ├── sdk-python/ spanlens (PyPI): Python SDK
│ ├── cli/ @spanlens/cli: npx wizard for 1-command setup
│ └── mcp-server/ @spanlens/mcp-server: MCP server for Cursor / Claude Desktop / Continue
└── supabase/
├── migrations/ Postgres schema (orgs, projects, keys, prompts, requests; RLS-gated)
└── seeds/ model_prices.sql etc.Everything is in Supabase Postgres. Organizations, projects, members, API and provider keys, prompts, datasets, alerts, billing, and the audit log are ordinary RLS-gated tables.
The requests table (one row per LLM call) is partitioned by month on created_at, so expiring old logs drops a partition instead of running a bulk DELETE. Reads go through apps/server/src/lib/requests-query.ts, which injects the organization_id filter and the per-plan retention window (free=14d / pro=90d / team=365d). If a log write fails, the row is parked as JSON in a requests_fallback queue and a cron replays it every 5 minutes. Replay is idempotent (ON CONFLICT (created_at, id) DO NOTHING), so a retry cannot double-count a request against your quota.
A workspace can hold multiple projects (e.g. dev / staging / prod, or one per app). Each project gets its own quota slice, provider keys, and prompt namespace.
Unified API keys give you one sl_live_* key per project that is provider-agnostic. Spanlens infers the provider from the request path (/proxy/openai/*, /proxy/anthropic/*, /proxy/gemini/*, /proxy/mistral/*, /proxy/openrouter/*, /proxy/groq/*, /proxy/deepseek/*, /proxy/xai/*, /proxy/cohere/*, /proxy/azure/*), so you only need one Spanlens key even if you call multiple model vendors.
X-Spanlens-* headers (set automatically by the SDK helpers withUser(), withSession(), withPromptVersion(), withLogBody()): tag a request with end-user / session IDs, link it to a prompt-version experiment, or limit how much body Spanlens stores. Full list in /docs/proxy.
Streaming safety ensures proxy responses are gracefully closed at 290s with a truncated=true flag in the log, so long streams never silently disappear.
Prerequisites: Node 20+, pnpm 10.33.0+, Docker (for local Supabase), Vercel CLI optional.
# 1. Clone + install
git clone https://github.com/spanlens/Spanlens.git
cd Spanlens
pnpm install
# 2. Start local Supabase (requires Docker)
supabase start
supabase db push # apply Postgres migrations
supabase gen types --lang typescript --local > supabase/types.ts
# 3. Env vars (see apps/server/.env.example)
cp apps/server/.env.example apps/server/.env
# 4. Run everything (web on :3000, server on :3001)
pnpm devpnpm typecheck # TS across all packages
pnpm lint # ESLint
pnpm test # Vitest: server + sdk + cli suites
pnpm build # production build smoke testSee CLAUDE.md for architecture rules and Known Gotchas (streaming, RLS, Paddle billing, Vercel Edge runtime, npm publish).
The easiest way to self-host is with the included docker-compose.yml. Two containers, the dashboard (web) and the proxy/API server, both pulled pre-built from GHCR. Your Supabase project is the only other piece, and it can be Supabase Cloud or your own self-hosted Supabase.
Open your Supabase project → SQL Editor → New query, paste the contents of supabase/init.sql, and click Run. That's it. No CLI needed.
Alternative (psql):
psql "postgresql://postgres:<password>@db.<ref>.supabase.co:5432/postgres" \ -f supabase/init.sql
.env file# Supabase (cloud or self-hosted)
NEXT_PUBLIC_SUPABASE_URL=https://xxxx.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJ...
SUPABASE_URL=https://xxxx.supabase.co
SUPABASE_ANON_KEY=eyJ...
SUPABASE_SERVICE_ROLE_KEY=eyJ...
# Encryption key (generate with: openssl rand -base64 32)
ENCRYPTION_KEY=<32-byte base64>
# Random secret for cron endpoint
CRON_SECRET=<random string>
# Pooled Postgres connection for the `requests` table (required).
# Supabase Dashboard: Connect > Direct > Transaction pooler, with
# "Use IPv4 connection" switched on. Port 6543, not 5432.
# The dedicated pooler shown by default is IPv6-only, so anything without
# outbound IPv6 (Vercel functions included) fails at DNS. Copy the host from
# that dialog rather than assembling it: the shared pooler hostname carries a
# numbered prefix that the region does not tell you.
# This is a full database credential, so keep it out of logs.
SUPABASE_DB_POOLER_URL=postgresql://postgres.<ref>:<password>@<pooler-host>:6543/postgres
# Optional (for invite emails)
# WEB_URL=https://your-domain.com
# RESEND_API_KEY=re_...
# RESEND_FROM=Spanlens <no-reply@your-domain.com>
# Optional (Paddle billing, only if you sell paid plans on your instance)
# PADDLE_API_KEY=...
# PADDLE_NOTIFICATION_SECRET=...
# PADDLE_ENVIRONMENT=sandbox # or productiondocker compose up -dDashboard: http://localhost:3000
API / proxy: http://localhost:3001
Health: GET /health (liveness) and GET /health/deep (database pool, fallback queue depth, cron freshness)
Upgrading from an older release? Request logs live in Postgres now. Re-apply
supabase/init.sqlso therequeststable exists, addSUPABASE_DB_POOLER_URLto your.env, then drop the ClickHouse container and its fourCLICKHOUSE_*variables. Rows already in ClickHouse are not copied across, so export anything you still need before tearing it down.
The web container passes NEXT_PUBLIC_* vars as build arguments (Next.js bakes them into the client bundle), so they must be present before docker compose build.
If you only need the proxy/API and run the dashboard separately:
docker pull ghcr.io/spanlens/spanlens-server:latest
docker run -p 3001:3001 \
-e SUPABASE_URL=... \
-e SUPABASE_ANON_KEY=... \
-e SUPABASE_SERVICE_ROLE_KEY=... \
-e SUPABASE_DB_POOLER_URL=... \
-e ENCRYPTION_KEY=... \
ghcr.io/spanlens/spanlens-server:latestconst openai = createOpenAI({
baseURL: 'https://your-spanlens.example.com/proxy/openai/v1',
})Your Spanlens instance talks to your Supabase. We never see your data.
The hosted instance ships with the following cron tasks (see apps/server/vercel.json). On self-host, point any scheduler at the same paths with the CRON_SECRET bearer:
Path | Schedule | Purpose |
| every 15m | Evaluate threshold + anomaly alerts, fire notifications |
| daily 01:00 | Materialize daily anomaly baselines |
| every 5m | Replay the |
| weekly Mon 09:00 | Email digest of idle provider keys |
| daily 04:00 | GitGuardian scan of active provider keys |
| daily 09:00 | Email model-swap savings opportunities |
| daily 10:00 | D-3 / D-1 warnings + auto-downgrade past-due subs |
| every 6h | Hard-delete accounts past their 30-day soft-delete grace |
| every 5m | Drain background-migration queue (data backfills) |
| hourly | Catch new model IDs in the request log that have no row in |
| hourly :31 | Spanlens dogfoods itself: heartbeat eval into the internal workspace |
| hourly :17 | Flag spans whose parent trace never arrived |
| daily 03:00 | TTL-evict stale |
| daily 03:15 | Reclaim expired |
| weekly Mon 09:00 | Email each workspace a weekly summary (requests, cost trend, top models, anomalies) |
| every 6h | Alert admins when a previously-active workspace stops sending data for 24h |
| every 5m | Lightweight ping that keeps the Vercel function warm (skip on always-on platforms like Fly.io / Railway) |
PRs and issues welcome. See CONTRIBUTING.md for the project layout, local-dev setup, coding conventions, and what we look for in a PR. Commit messages follow Conventional Commits and the PR template walks through the safety checklist.
Security issues: please email support@spanlens.io instead of opening a public issue. See SECURITY.md.
MIT. Use, fork, self-host, or build on top freely. The hosted service at spanlens.io is the recommended way to run Spanlens, but you can always pull the Docker image and run it yourself (see docs/self-host).
List unacknowledged cost / latency / error-rate anomalies the platform has detected. Each anomaly carries a deviations field (how many sigmas off baseline). Use when the user asks "anything weird going on?", "any spikes?", or wants a quick health check.
| Name | Required | Description | Default |
|---|---|---|---|
| sigma | No | Minimum deviations (in sigmas) to flag. Default 3. Lower = more sensitive. | |
| since | No | ISO 8601 timestamp. Sets the observation window: behaviour since this time is compared against the preceding baseline. Clamped to the last 15 minutes – 72 hours; default is the last hour. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses that anomalies are unacknowledged, includes deviations field, and explains parameter behavior (sigma default, since clamping). However, does not mention pagination, result limits, or authentication requirements, leaving some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with front-loaded purpose and immediate usage examples. No wasted words; every sentence contributes to understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and only two simple parameters, the description covers the core behavior and parameter semantics well. Lacks info on pagination or result limits, but for a straightforward list tool it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters. Description adds valuable context beyond schema: explains sigma default and sensitivity, since parameter range clamping and default. Agent gains practical insight for parameter tuning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (List) and resource (unacknowledged cost/latency/error-rate anomalies). The description distinguishes this tool from siblings like get_savings, get_stats, etc., which focus on different metrics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage guidance with example user queries ('anything weird going on?', 'any spikes?', quick health check). Lacks explicit when-not-to-use or alternative tool references, but context is clear given sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List model-swap recommendations the platform thinks would save money without losing quality. Each item carries projected monthly savings, prior-window cost, and an achieved flag if the swap has already been adopted.
| Name | Required | Description | Default |
|---|---|---|---|
| hours | No | Analysis window in hours. Default 168 (7 days). Longer = more confident, but slower. | |
| minSavings | No | Only return recommendations projecting at least this many USD/month in savings. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must compensate. It names fields in items (savings, cost, achieved flag) but omits side effects, rate limits, data freshness, or safety assurances typical for a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, each adding value. No redundant information, and it is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains item fields adequately. It could mention ordering or defaults but is complete for a simple list tool. Siblings are distinct, reducing confusion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters. The tool description adds no additional meaning beyond the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists model-swap recommendations for cost savings without quality loss. It distinguishes from siblings like get_anomalies, get_stats, etc., which serve different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for cost-saving recommendations but does not explicitly state when to use this tool versus alternatives or provide any when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get aggregate LLM cost, request count, latency, and error-rate stats for the workspace. Use when the user asks about spend, usage volume, or how things have been going.
| Name | Required | Description | Default |
|---|---|---|---|
| groupBy | No | When set, returns per-group breakdown from /stats/models instead of overview totals. | |
| timeframe | No | Time window. Default '7d'. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It does not state whether the operation is read-only, idempotent, or any side effects. For a tool that likely performs a query, failing to mention that it is safe to call repeatedly is a gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no redundancy. First sentence states functionality, second gives usage guidance. Efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema, so the description should hint at the output format. It lists the types of stats (cost, request count, latency, error-rate) but does not describe the structure (e.g., per time period, totals). With only 0 required parameters and 2 enums, the tool is simple, but the output remains vague.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters have schema descriptions (100% coverage). The description adds clarifying context for groupBy ('returns per-group breakdown from /stats/models instead of overview totals') and notes the default for timeframe, which improves understanding beyond the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it gets aggregate LLM cost, request count, latency, and error-rate stats. It also provides usage guidance. However, it does not explicitly differentiate from sibling tools like get_savings or get_anomalies, which slightly reduces clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use when the user asks about spend, usage volume, or how things have been going,' which provides clear context. It does not mention when not to use or alternatives, but the guidance is still helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch the full span tree for a single agent trace by id — every llm/tool/retrieval span with timing, tokens, and cost. Use when the user names a trace, asks why one was slow, or asks what an agent did step by step. Pair with list_traces if you need to discover the trace id first.
| Name | Required | Description | Default |
|---|---|---|---|
| traceId | Yes | UUID of the trace. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It describes the returned data comprehensively (span tree with timing, tokens, cost) but does not mention potential limits like trace size or pagination, though for a fetch-by-ID it is reasonably transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three concise sentences, front-loaded with purpose, then usage guidance, all without wasted words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately explains the return data and usage context. It could mention error handling or if the trace is not found, but overall it is complete for a simple fetch operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the traceId parameter described as 'UUID of the trace.' The description adds no additional parameter-specific meaning beyond that, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches the full span tree for a single agent trace, specifying the types of spans (llm/tool/retrieval) and data included (timing, tokens, cost). It distinguishes from siblings like list_traces and other analytics tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: when the user names a trace, asks why it was slow, or asks about step-by-step actions. Also provides pairing guidance with list_traces for discovering the trace ID.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get per-end-user usage breakdown — total requests, cost, latency, models used, recent calls. The 'user' here is the customer's end-user, identified by the x-spanlens-user header the SDK attaches.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Max rows when returning the top-N list. Default 20. | |
| userId | No | When set, returns the detail view for a single end-user. When omitted, returns the top-N usage list across all end-users. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full burden. It clearly states the tool returns usage breakdown data, implies read-only operation via 'get', and clarifies the user identifier. However, it omits details like pagination limits or safety guarantees.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, and a second sentence clarifying a critical context (the user definition). No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately lists the return fields (requests, cost, latency, models, recent calls) but does not describe the response structure or data format, leaving some ambiguity for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not elaborate on parameters beyond what the schema already provides (limit and userId meanings). It adds no parametric context beyond the user header clarification.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies the verb 'get', the resource 'per-end-user usage breakdown', and lists concrete metrics (requests, cost, latency, models, recent calls), clearly distinguishing it from siblings like get_anomalies or get_stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus siblings. The description implies it is for end-user analytics but does not mention prerequisites or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List agent traces with optional filters. Use to discover trace IDs to feed into get_trace, or to scan recent agent runs. Returns trace summaries (name, status, duration, span count, total tokens, total cost). Does NOT include individual span data — call get_trace with a trace ID for that.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Max traces to return. Default 20, max 100. | |
| query | No | Substring match on trace name or trace id. | |
| since | No | ISO 8601 lower bound on `started_at`. Only return traces that started at or after this time. | |
| status | No | Filter by trace status. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses return fields (name, status, duration, etc.) and explicitly states it excludes span data, but lacks mention of rate limits or authorization needs. However, for a read-only list tool, this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences that front-load purpose, then usage, then return info, and finally what is not included. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, return fields, and limitation to summaries. With 4 optional parameters and no output schema, the description provides essential context for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter documented. Description adds no additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'List agent traces with optional filters' and distinguishes from sibling get_trace by noting it returns summaries and not individual span data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises 'Use to discover trace IDs to feed into get_trace, or to scan recent agent runs' and clarifies what it does not do, guiding appropriate use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List individual LLM requests with cost, latency, model, status, and error message. Use when the user wants to see specific calls — recent ones, errors only, particular model, particular user, etc.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Max rows to return. Default 20, max 100. | |
| model | No | Filter to a specific model substring. | |
| since | No | ISO 8601 timestamp lower bound. Only return requests created at or after this time. | |
| status | No | Filter by overall status — success (2xx) or error (4xx/5xx). | |
| userId | No | Filter to a specific end-user (the value the customer attaches via x-spanlens-user). | |
| provider | No | Filter to a specific provider. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose behavioral traits beyond the basic listing function. It does not mention pagination behavior beyond 'limit', rate limits, auth requirements, or what happens if no results match. The description is adequate but lacks depth for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, minimal and well-structured. First sentence states purpose and output fields; second sentence gives usage guidance. No redundant or extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but the description mentions the fields returned. However, it lacks details on default ordering, sorting, or full response structure. Given the tool has 6 parameters and no output schema, the description could be more complete, but it covers essential purpose adequately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with each parameter already described. The tool description restates filter options (e.g., 'recent ones, errors only, particular model') but adds no new semantic meaning beyond what the schema already provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb ('List'), specific resource ('individual LLM requests'), and explicit fields returned (cost, latency, model, status, error message). Distinguishes from sibling tools like get_stats or list_traces by focusing on individual requests with detailed attributes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use: 'when the user wants to see specific calls' and provides concrete examples (recent ones, errors only, particular model, particular user). Does not explicitly state when not to use, but the context and sibling names imply this is for detailed request-level queries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v1.0.0get_anomaliesget_savingsget_statsget_traceget_user_analyticslist_tracesquery_requestsEach tool has a clearly distinct purpose: anomalies, savings, stats, trace details, user analytics, trace listing, and request queries. No overlap or ambiguity.
All tools follow a consistent verb_noun pattern (get_, list_, query_). No mixing of cases or styles.
Seven tools is well-scoped for an LLM monitoring server. Each tool earns its place without being excessive or insufficient.
Covers anomalies, savings, stats, traces, user analytics, and individual requests. The surface is complete for monitoring and analysis, with no obvious gaps.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
MCP observability. Query live traffic, errors, duration, and alerts from your AI agent.
Read-only analytics for Convex apps, queryable via MCP from Claude, Cursor, and other clients.
Open-source agent that observes and fixes your application. Query logs, traces, metrics, incidents.
Query your warehouse or a CSV with Claude/ChatGPT over MCP, governed by table-level ACL + audit.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/spanlens/Spanlens'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
