Skip to main content
Glama
JaviMaligno

langfuse-mcp-extended

by JaviMaligno

Langfuse MCP Server (Extended)

A comprehensive Model Context Protocol (MCP) server for Langfuse, providing AI assistants with full access to traces, observations, scores, datasets, and sessions.

Prompt Management: This server focuses on observability data. For prompt management, Langfuse provides a built-in MCP server (GitHub). We're working on integrating both - see Contributing.

Features

  • 24 tools for complete Langfuse observability access

  • Traces: List, get, and delete traces with filtering and pagination

  • Observations: Query generations, spans, and events with usage metrics

  • Scores: Full CRUD operations for evaluation scores (numeric, categorical, boolean)

  • Score Configs: Manage score configuration templates

  • Datasets: Complete dataset management including items and evaluation runs

  • Sessions: Access session data with associated traces

  • Cloud & Self-hosted: Works with Langfuse Cloud (US/EU) and self-hosted instances

Related MCP server: langfuse-mcp

Quick Start

One-liner Installation (Claude Code)

claude mcp add langfuse -e LANGFUSE_PUBLIC_KEY=pk-lf-xxx -e LANGFUSE_SECRET_KEY=sk-lf-xxx -- npx -y langfuse-mcp-extended

For EU Cloud or self-hosted, add the base URL:

claude mcp add langfuse -e LANGFUSE_PUBLIC_KEY=pk-lf-xxx -e LANGFUSE_SECRET_KEY=sk-lf-xxx -e LANGFUSE_BASE_URL=https://eu.cloud.langfuse.com -- npx -y langfuse-mcp-extended

Installation by Client

Add to your Claude Desktop configuration file:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

  • Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "langfuse": {
      "command": "npx",
      "args": ["-y", "langfuse-mcp-extended"],
      "env": {
        "LANGFUSE_PUBLIC_KEY": "pk-lf-...",
        "LANGFUSE_SECRET_KEY": "sk-lf-..."
      }
    }
  }
}

For EU Cloud or self-hosted, add LANGFUSE_BASE_URL:

{
  "mcpServers": {
    "langfuse": {
      "command": "npx",
      "args": ["-y", "langfuse-mcp-extended"],
      "env": {
        "LANGFUSE_PUBLIC_KEY": "pk-lf-...",
        "LANGFUSE_SECRET_KEY": "sk-lf-...",
        "LANGFUSE_BASE_URL": "https://eu.cloud.langfuse.com"
      }
    }
  }
}

Option 1: One-liner with environment variables

claude mcp add langfuse -e LANGFUSE_PUBLIC_KEY=pk-lf-xxx -e LANGFUSE_SECRET_KEY=sk-lf-xxx -- npx -y langfuse-mcp-extended

Option 2: Add to project's .mcp.json

{
  "mcpServers": {
    "langfuse": {
      "command": "npx",
      "args": ["-y", "langfuse-mcp-extended"],
      "env": {
        "LANGFUSE_PUBLIC_KEY": "pk-lf-...",
        "LANGFUSE_SECRET_KEY": "sk-lf-..."
      }
    }
  }
}

Option 3: Use shell environment variables

claude mcp add langfuse -- npx -y langfuse-mcp-extended

Then set in your shell profile (.bashrc, .zshrc, etc.):

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."

Add to your Cursor MCP configuration:

  • Project-specific: .cursor/mcp.json in your project directory

  • Global: ~/.cursor/mcp.json in your home directory

{
  "mcpServers": {
    "langfuse": {
      "command": "npx",
      "args": ["-y", "langfuse-mcp-extended"],
      "env": {
        "LANGFUSE_PUBLIC_KEY": "pk-lf-...",
        "LANGFUSE_SECRET_KEY": "sk-lf-..."
      }
    }
  }
}

Or add via UI: File → Preferences → Cursor Settings → MCP

Add to .vscode/mcp.json in your workspace:

{
  "mcpServers": {
    "langfuse": {
      "command": "npx",
      "args": ["-y", "langfuse-mcp-extended"],
      "env": {
        "LANGFUSE_PUBLIC_KEY": "pk-lf-...",
        "LANGFUSE_SECRET_KEY": "sk-lf-..."
      }
    }
  }
}
npm install -g langfuse-mcp-extended

Then use langfuse-mcp-extended as the command instead of npx -y langfuse-mcp-extended.

Configuration

Required Environment Variables

Variable

Description

LANGFUSE_PUBLIC_KEY

Your Langfuse public key (pk-lf-...)

LANGFUSE_SECRET_KEY

Your Langfuse secret key (sk-lf-...)

Optional Environment Variables

Variable

Default

Description

LANGFUSE_BASE_URL

https://cloud.langfuse.com

Langfuse instance URL

LOG_LEVEL

info

Logging level: debug, info, warn, error

Langfuse Instance URLs

Instance

URL

US Cloud

https://cloud.langfuse.com (default)

EU Cloud

https://eu.cloud.langfuse.com

Self-hosted

Your instance URL (e.g., http://localhost:3000)

API Reference

Pagination

All list operations use page-based pagination with a default limit of 10 items per page:

  • page (number, optional): Page number, 1-indexed. Default: 1

  • limit (number, optional): Items per page, max 100. Default: 10

Response Size Control

Get operations (getTrace, getSession, getObservation) support an includeIO parameter:

  • includeIO (boolean, optional): Include input/output fields. Default: false

When includeIO is false (default), large input and output fields are stripped from responses to prevent exceeding LLM context limits. Set to true when you need the full payload.


listTraces

List traces with filtering and pagination.

Inputs:

  • page (number, optional): Page number

  • limit (number, optional): Items per page

  • name (string, optional): Filter by trace name

  • userId (string, optional): Filter by user ID

  • sessionId (string, optional): Filter by session ID

  • tags (string[], optional): Filter by tags (AND logic)

  • fromTimestamp (string, optional): Start of time range (ISO 8601)

  • toTimestamp (string, optional): End of time range (ISO 8601)

  • environment (string, optional): Filter by environment

  • orderBy (string, optional): Sort by timestamp, latency, or totalCost

getTrace

Get a specific trace with full details.

Inputs:

  • traceId (string, required): The trace ID

  • includeIO (boolean, optional): Include input/output fields. Default: false

Returns: Trace with observations and scores.

deleteTrace

Delete a trace.

Inputs:

  • traceId (string, required): The trace ID to delete

listObservations

List observations (generations, spans, events) with cursor-based pagination.

Inputs:

  • cursor (string, optional): Pagination cursor

  • limit (number, optional): Items per page

  • traceId (string, optional): Filter by trace ID

  • name (string, optional): Filter by observation name

  • type (string, optional): Filter by type: GENERATION, SPAN, EVENT

  • parentObservationId (string, optional): Filter by parent

  • fromStartTime (string, optional): Start of time range (ISO 8601)

  • toStartTime (string, optional): End of time range (ISO 8601)

  • userId (string, optional): Filter by user ID

  • version (string, optional): Filter by version

getObservation

Get a specific observation with all details.

Inputs:

  • observationId (string, required): The observation ID

  • includeIO (boolean, optional): Include input/output fields. Default: false

Returns: Observation with usage, costs, and timing.

createScore

Create a score for a trace or observation.

Inputs:

  • traceId (string, required): ID of the trace to score

  • name (string, required): Score name (e.g., accuracy, relevance)

  • value (number | string, required): Score value

  • observationId (string, optional): ID of specific observation to score

  • dataType (string, optional): NUMERIC, CATEGORICAL, or BOOLEAN

  • comment (string, optional): Explanation of the score

  • configId (string, optional): Score config ID for validation

  • id (string, optional): Custom ID for idempotency

listScores

List scores with filtering.

Inputs:

  • page, limit (pagination)

  • traceId (string, optional): Filter by trace ID

  • observationId (string, optional): Filter by observation ID

  • name (string, optional): Filter by score name

  • source (string, optional): Filter by source: API, ANNOTATION, EVAL

  • dataType (string, optional): Filter by data type

  • configId (string, optional): Filter by config ID

  • fromTimestamp, toTimestamp (string, optional): Time range

  • userId (string, optional): Filter by user ID

  • operator (string, optional): Comparison operator: <, >, <=, >=, !=, =

  • value (number, optional): Value to compare (requires operator)

getScore

Get a specific score.

Inputs:

  • scoreId (string, required): The score ID

deleteScore

Delete a score.

Inputs:

  • scoreId (string, required): The score ID to delete

createScoreConfig

Create a score configuration template.

Inputs:

  • name (string, required): Unique config name

  • dataType (string, required): NUMERIC, CATEGORICAL, or BOOLEAN

  • minValue (number, optional): Minimum value (NUMERIC only)

  • maxValue (number, optional): Maximum value (NUMERIC only)

  • categories (array, optional): Category definitions (CATEGORICAL only)

  • description (string, optional): Description

listScoreConfigs

List all score configurations.

Inputs:

  • page, limit (pagination)

getScoreConfig

Get a specific score configuration.

Inputs:

  • configId (string, required): The config ID

createDataset

Create a new dataset.

Inputs:

  • name (string, required): Dataset name

  • description (string, optional): Description

  • metadata (object, optional): Additional metadata

listDatasets

List all datasets.

Inputs:

  • page, limit (pagination)

getDataset

Get a dataset by name.

Inputs:

  • datasetName (string, required): The dataset name

createDatasetItem

Create or update a dataset item.

Inputs:

  • datasetName (string, required): Target dataset name

  • input (any, required): Item input data

  • expectedOutput (any, optional): Expected output

  • metadata (object, optional): Item metadata

  • sourceTraceId (string, optional): Source trace ID

  • sourceObservationId (string, optional): Source observation ID

  • id (string, optional): Custom ID for upsert

listDatasetItems

List items in a dataset.

Inputs:

  • datasetName (string, required): Dataset name

  • page, limit (pagination)

  • sourceTraceId (string, optional): Filter by source trace

  • sourceObservationId (string, optional): Filter by source observation

getDatasetItem

Get a specific dataset item.

Inputs:

  • datasetItemId (string, required): The item ID

deleteDatasetItem

Delete a dataset item.

Inputs:

  • datasetItemId (string, required): The item ID to delete

createDatasetRunItem

Link a trace/observation to a dataset item for evaluation.

Inputs:

  • runName (string, required): Name of the dataset run

  • datasetItemId (string, required): Dataset item ID

  • traceId (string, required): Trace ID

  • runDescription (string, optional): Run description

  • observationId (string, optional): Observation ID

  • metadata (object, optional): Run item metadata

listDatasetRuns

List runs for a dataset.

Inputs:

  • datasetName (string, required): Dataset name

  • page, limit (pagination)

getDatasetRun

Get a specific dataset run.

Inputs:

  • datasetName (string, required): Dataset name

  • runName (string, required): Run name

listSessions

List all sessions.

Inputs:

  • page, limit (pagination)

  • fromTimestamp (string, optional): Start of time range (ISO 8601)

  • toTimestamp (string, optional): End of time range (ISO 8601)

getSession

Get a specific session with its traces.

Inputs:

  • sessionId (string, required): The session ID

  • includeIO (boolean, optional): Include input/output in traces. Default: false

Returns: Session with associated traces.


Using with Official Langfuse MCP (Prompts)

This server provides observability tools (traces, scores, datasets). For prompt management, Langfuse provides a built-in MCP server that requires no installation.

The Langfuse MCP server is built directly into Langfuse at /api/public/mcp. See the official documentation for setup instructions.

Using Both Servers Together

To use Langfuse observability (this server) alongside the official prompts MCP:

{
  "mcpServers": {
    "langfuse-observability": {
      "command": "npx",
      "args": ["-y", "langfuse-mcp-extended"],
      "env": {
        "LANGFUSE_PUBLIC_KEY": "pk-lf-...",
        "LANGFUSE_SECRET_KEY": "sk-lf-..."
      }
    }
  }
}

Then configure the built-in Langfuse prompts MCP following the official guide.

Note: We're actively working with the Langfuse team to potentially integrate both servers. See Contributing for details.

Development

# Install dependencies
npm install

# Build
npm run build

# Run in development mode
npm run dev

# Run tests
npm test                  # Unit tests (59 tests)
npm run test:integration  # Integration tests (18 tests)
npm run test:all          # All tests

# Quality checks
npm run lint
npm run typecheck

Contributing

Contributions are welcome! This project aims to be integrated into the official Langfuse ecosystem.

Active discussions:

How to contribute:

  1. Fork the repository

  2. Create your feature branch (git checkout -b feature/amazing-feature)

  3. Commit your changes (git commit -m 'Add amazing feature')

  4. Push to the branch (git push origin feature/amazing-feature)

  5. Open a Pull Request

Author

Built by Javier Aguilar - AI Agent Architect specializing in multi-agent orchestration and MCP development.

License

MIT License - see LICENSE for details.

Available Tools

24 tools
createDatasetB

Create a new dataset for evaluation. Datasets contain items with expected inputs/outputs for testing LLM applications.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesUnique name for the dataset
descriptionNoDescription of the dataset
metadataNoAdditional metadata as key-value pairs

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not disclose side effects, permissions, rate limits, or return value information. The behavioral impact (e.g., a write operation) is only implicit from the word 'create'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, with no wasted words. It front-loads the core purpose and follows with a brief explanatory note about datasets.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is insufficient. It does not mention the return value, idempotency, or any constraints beyond parameter types, leaving the agent guessing about the tool's behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description adds no additional meaning beyond what is in the schema for parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (create) and resource (dataset), and provides context that datasets are for evaluation. This differentiates it from sibling tools like createDatasetItem, which operate on dataset contents.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when a new evaluation dataset is needed, but does not provide explicit guidance on when not to use this tool or mention alternatives like getDatasets or listDatasets.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createDatasetItemB

Create or upsert a dataset item. Use id parameter for idempotent updates.

ParametersJSON Schema
NameRequiredDescriptionDefault
datasetNameYesName of the dataset to add the item to
inputNoThe input for this dataset item (any JSON value)
expectedOutputNoThe expected output for evaluation
metadataNoAdditional metadata
sourceTraceIdNoLink to source trace that generated this item
sourceObservationIdNoLink to source observation (requires sourceTraceId)
idNoCustom ID for idempotent upsert
statusNoItem status (defaults to ACTIVE)

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It mentions upsert behavior, but lacks details on side effects (e.g., conflict resolution), return values, or required permissions. The upsert indication is helpful but insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with two sentences: one stating purpose and one providing a usage tip. Every word adds value, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (8 parameters, no output schema, no annotations), the description is minimal. It lacks context about return values, error conditions, prerequisites (e.g., dataset must exist), and differentiation from similar sibling tools like createDatasetRunItem.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with each parameter described. The description adds value by explaining the purpose of the id parameter for idempotent updates, but does not enrich other parameter semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create or upsert a dataset item,' specifying the verb and resource. It distinguishes from sibling tools like createDataset (creates dataset itself) and createDatasetRunItem, though it could be more explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides guidance on using the id parameter for idempotent updates, but it does not specify when to use this tool versus alternatives or mention any prerequisites or conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createDatasetRunItemA

Create a dataset run item linking a trace/observation to a dataset item for evaluation.

ParametersJSON Schema
NameRequiredDescriptionDefault
runNameYesName of the dataset run
runDescriptionNoDescription of the run
datasetItemIdYesID of the dataset item being evaluated
traceIdYesID of the trace that processed this item
observationIdNoID of specific observation (if applicable)
metadataNoAdditional run item metadata

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description must convey behavioral traits. It mentions creation (mutation) but omits side effects, permissions, idempotency, or error conditions. Adequate for a simple creation but lacks depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single, focused sentence that concisely conveys the purpose. No wasted words, front-loading the action and resource.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, so description should hint at return value or confirmation. It explains input and linking but doesn't describe the created item's structure or possible errors. Adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. Description does not add value beyond the schema; it only restates the linking concept. No parameter-specific details like constraints or examples.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description uses specific verb 'create' and resource 'dataset run item' and clearly states the linking purpose (trace/observation to dataset item for evaluation). It effectively distinguishes from sibling tools like createDataset (creates dataset) or createDatasetItem (creates item).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use vs alternatives like createScore or other evaluation tools. The description implies usage for evaluation but doesn't mention exclusions or prerequisites, leaving the agent to infer context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createScoreA

Create a score for a trace or observation. Supports numeric, boolean, and categorical score types.

ParametersJSON Schema
NameRequiredDescriptionDefault
traceIdYesID of the trace to score
observationIdNoOptional: ID of specific observation to score
nameYesName of the score (e.g., 'accuracy', 'relevance')
valueYesScore value - numeric for NUMERIC type, string for CATEGORICAL/BOOLEAN
dataTypeNoType of score value
commentNoOptional comment explaining the score
configIdNoOptional score config ID for validation
idNoOptional custom ID for idempotency

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It adds value by noting supported score types but omits idempotency, validation behavior, and side effects. Does not contradict schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with key purpose, no unnecessary words. Efficient and clear.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Lacks details on return value, interaction between traceId and observationId, and validation with configId. For a creation tool with 8 parameters and no output schema, more context would help agent usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with each parameter described. Description reinforces dataType enum but adds minimal new semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states action ('Create'), resource ('score for trace or observation'), and supported types ('numeric, boolean, and categorical'). This distinguishes it from sibling tools like createScoreConfig.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description implies use when scoring a trace or observation but offers no explicit guidance on when to choose this over alternatives or conditions to avoid. Sibling tools like createScoreConfig and deleteScore exist but are not mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createScoreConfigC

Create a score configuration that defines validation rules for scores. Supports numeric ranges, categorical values, and boolean types.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesUnique name for the score configuration
dataTypeYesType of score values this config accepts
minValueNoMinimum allowed value (NUMERIC only)
maxValueNoMaximum allowed value (NUMERIC only)
categoriesNoCategory definitions (CATEGORICAL only)
descriptionNoDescription of what this score measures

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description should disclose behavioral traits. It mentions creation and supported types but omits side effects (e.g., idempotency, overwrite behavior), permissions, or error conditions. Minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences, front-loaded with purpose. No wasted words. Slightly low on detail, but still concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Lacks description of return values (no output schema), prerequisites, or error conditions. For a creation tool with 6 parameters, this is incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already explains parameters. The description adds context that the config defines validation rules, but this is implied by the tool's purpose. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a score configuration that defines validation rules for scores' and lists supported types. This provides a specific verb and resource, distinguishing it from other tools like createScore. However, it does not explicitly differentiate from siblings, so it scores 4.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives, such as createScore. The description lacks context on prerequisites or exclusions, leaving the agent with no decision support.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteDatasetItemA

Delete a specific dataset item. This action is irreversible.

ParametersJSON Schema
NameRequiredDescriptionDefault
datasetItemIdYesThe unique dataset item identifier to delete

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that the action is irreversible, a key behavioral trait. However, without annotations, it omits other important details like required permissions, side effects on related data, or whether deletion cascades. The irreversibility note is helpful but incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with only two sentences, front-loaded with the key action. Every word adds value, and no unnecessary information is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema, no annotations), the description is largely adequate. It states the purpose and irreversibility. Additional context, such as success/error feedback or confirmation, would improve completeness but is not critical for this low-complexity tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter, so the schema already defines its meaning. The description adds no extra semantics beyond the schema, resulting in a baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete') and the resource ('a specific dataset item'), with no ambiguity. It also differentiates from sibling delete tools by specifying 'dataset item', making the purpose unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, such as other delete tools (e.g., deleteScore, deleteTrace). It lacks context about prerequisites or when not to use it, leaving the agent to infer.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteScoreB

Delete a specific score. This action is irreversible.

ParametersJSON Schema
NameRequiredDescriptionDefault
scoreIdYesThe unique score identifier to delete

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It only mentions irreversibility but lacks details on authorization, side effects on associated data, or whether any confirmation is needed. Inadequate for a destructive operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Extremely concise and front-loaded: two sentences that convey purpose and a critical behavioral note. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one required parameter, no output schema), the description covers the core purpose and irreversibility. It could mention that only the score record is deleted (no cascading effects) for completeness, but it's adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (scoreId is well described). The description adds no extra meaning beyond the schema—both state essentially the same thing. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (delete), the resource (specific score), and emphasizes irreversibility. It unambiguously distinguishes from sibling tools like createScore or getScore.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. There is no mention of prerequisites, permissions, or scenarios where deletion is appropriate or inadvisable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteTraceA

Delete a specific trace. This action is irreversible.

ParametersJSON Schema
NameRequiredDescriptionDefault
traceIdYesThe unique trace identifier to delete

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

States irreversibility, which is important for a delete operation. Lacks details on side effects, permissions, or confirmation steps. No annotations to supplement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences with no filler. Front-loaded with verb and resource.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a simple delete with one parameter, but lacks behavioral context like cascade effects. No output schema needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

100% schema coverage for parameter traceId; description adds no extra meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the action (Delete) and resource (a specific trace). Distinguishes from sibling tools like getTrace, listTraces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidelines on when to use this tool vs alternatives. Does not mention prerequisites, conditions, or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getDatasetA

Get a dataset by name including its items and run names.

ParametersJSON Schema
NameRequiredDescriptionDefault
datasetNameYesThe name of the dataset to retrieve

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. The description does not disclose any behavioral traits, such as required permissions, effects, or return format beyond the basic retrieval.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that is concise and front-loaded with the key purpose, containing no unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple retrieval tool with one parameter and no output schema, the description is adequate, mentioning that items and run names are included. It covers the essential context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter datasetName has 100% schema description coverage, so the description adds no additional value beyond the schema. Baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Get a dataset by name' and specifies the scope 'including its items and run names'. It differentiates from sibling tools like listDatasets and deleteDatasetItem.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when you need a specific dataset by name, but lacks explicit when-not or alternatives such as listDatasets for multiple datasets.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getDatasetItemA

Get a specific dataset item by ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
datasetItemIdYesThe unique dataset item identifier

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only states the retrieval operation without disclosing any behavioral traits, side effects, or return structure. Minimal disclosure is insufficient for a tool with no annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words. It front-loads the action and resource.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple retrieval with one parameter, the description is nearly complete. It lacks output description, but no output schema exists. In context of siblings, it is self-explanatory.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds 'by ID' but the schema already describes the parameter as 'identifier'. No additional semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Get' and the resource 'dataset item', and specifies the method 'by ID'. It distinguishes from sibling tools like listDatasetItems (list multiple) and createDatasetItem (create).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. The purpose is implied by the name and description, but there is no mention of when not to use it or which sibling to prefer in different contexts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getDatasetRunA

Get a specific dataset run by name including its run items.

ParametersJSON Schema
NameRequiredDescriptionDefault
datasetNameYesName of the dataset
runNameYesName of the run to retrieve

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description carries the transparency burden. It discloses that the result includes run items, which adds value beyond the input schema. However, it does not mention any side effects, permissions, or error conditions, which is acceptable for a simple read operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence of 12 words. Every word is necessary and efficiently communicates the tool's purpose. No wasted space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (2 required parameters, no output schema), the description is fairly complete. It explains that run items are included, though it could optionally describe the return format. Overall, it adequately covers the essential context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with clear descriptions for both datasetName and runName. The tool description adds no additional meaning beyond what is already in the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Get', the resource 'a specific dataset run', and the scope 'by name including its run items'. This differentiates it from siblings like listDatasetRuns (which lists without items) and getDataset (which gets the dataset itself).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when or when not to use this tool versus alternatives is provided. The description only states what it does, leaving the agent to infer usage context from the name and siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getObservationA

Get a specific observation with all details including input/output, usage, costs, and timing.

ParametersJSON Schema
NameRequiredDescriptionDefault
observationIdYesThe unique observation identifier
includeIONoInclude input/output fields (can be very large, disabled by default)

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Despite no annotations, description warns about large payloads when includeIO is true ('can be very large, disabled by default'). Adds important behavioral context beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with action and key details. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with 2 params and no output schema, description provides adequate context about what is returned and the large I/O warning. Missing error handling info but still complete enough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. Description adds minimal extra meaning beyond schema descriptions, only summarizing overall details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Get a specific observation' with verb and resource, and specifies details like input/output, usage, costs, and timing. Distinguishes from sibling list tools like listObservations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use or not use this tool compared to alternatives like listObservations or getTrace. Lacks context for agent decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getScoreB

Get a specific score by ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
scoreIdYesThe unique score identifier

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must convey behavioral traits. It does not disclose whether the operation is read-only, requires permissions, or how errors are handled (e.g., score not found).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single simple sentence, which is concise but may be too brief for a tool that could benefit from a minimal expansion (e.g., mentioning that it returns the full score object).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a straightforward retrieval tool with one parameter and no output schema, the description is minimally adequate. However, it does not specify the return format or error conditions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage for the single parameter is 100% via its description. The tool description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'Get a specific score by ID,' which clearly identifies the action (retrieve), resource (score), and method (by identifier). This distinguishes it from sibling tools like createScore or listScores.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives (e.g., listScores). The description lacks context about prerequisites or scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getScoreConfigA

Get a specific score configuration by ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
configIdYesThe unique score config identifier

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavior. It only states the basic action without mentioning idempotency, return format, or any side effects, which is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, clear sentence with no wasted words. It is appropriately concise and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

While adequate for a simple get-by-ID tool, it lacks information on what is returned (e.g., the full configuration object) and how to obtain the config ID, which would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes the single parameter with 100% coverage. The description adds no additional meaning beyond 'by ID', so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves a specific score configuration by ID, using a specific verb and resource, and is distinct from sibling tools like listScoreConfigs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when you have a specific config ID, but does not explicitly state when to use this tool versus alternatives like listScoreConfigs or createScoreConfig.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getSessionA

Get a specific session by ID including its traces.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe unique session identifier
includeIONoInclude input/output fields in traces (can be very large, disabled by default)

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It mentions 'including its traces', which hints at the response content, but it does not disclose any behavioral traits such as data size implications, permissions, or idempotency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no waste, but it lacks structural elements like separate sections for purpose and behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema, the description should explain return values in more detail. 'Including its traces' is helpful but incomplete for a tool with necessary parameters and no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with thorough descriptions for both parameters. The description does not add any additional meaning beyond what the input schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Get' and the resource 'specific session by ID including its traces'. It effectively distinguishes from sibling tools like listSessions (which lists sessions) and getTrace (which gets a single trace).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when you have a specific session ID and need traces, but it does not provide explicit guidance on when to use this tool versus alternatives like listSessions or getObservation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTraceA

Get a specific trace with full details including all observations and scores.

ParametersJSON Schema
NameRequiredDescriptionDefault
traceIdYesThe unique trace identifier
includeIONoInclude input/output fields (can be very large, disabled by default)

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It mentions 'full details' and the includeIO parameter in schema but does not state whether the operation is read-only, safe, or any potential side effects, nor does it address performance implications of large payloads.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence of 14 words, front-loaded with the action and resource, with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description should explain the return format. Saying 'full details including all observations and scores' is vague and does not provide enough structure or relationships with sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the baseline is 3. The description does not add meaning beyond the schema; it only describes the output, not the parameters themselves.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'a specific trace', and adds 'with full details including all observations and scores', which distinguishes it from sibling tools like listTraces or getObservation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies that this tool should be used when needing a trace with all its observations and scores, subtly differentiating from getObservation or getScore, but lacks explicit guidance on when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listDatasetItemsC

List items in a dataset with optional filtering.

ParametersJSON Schema
NameRequiredDescriptionDefault
datasetNameYesName of the dataset to list items from
pageNoPage number (1-indexed)
limitNoItems per page
sourceTraceIdNoFilter by source trace ID
sourceObservationIdNoFilter by source observation ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It does not disclose that this is a read-only operation or any side effects. The word 'list' implies read-only, but no explicit safety or behavioral context is given.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no waste. It efficiently communicates the core purpose, though it could be slightly more detailed without losing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 5 parameters and no output schema, the description is insufficient. It does not explain that results are paginated, what a dataset item is, or what the return format looks like. The tool has several filters, but the description only mentions 'optional filtering' generically.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides for each parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List') and resource ('items in a dataset'). It distinguishes this tool from siblings like listDatasets and listDatasetRuns, though it does not explicitly differentiate from these similar tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives (e.g., listDatasets or listDatasetRuns). No when-not-to-use or context for filtering is mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listDatasetRunsB

List all runs for a specific dataset.

ParametersJSON Schema
NameRequiredDescriptionDefault
datasetNameYesName of the dataset to list runs from
pageNoPage number (1-indexed)
limitNoItems per page

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'List all runs' without specifying pagination behavior, whether deleted runs are included, or error handling (e.g., dataset not found). This leaves significant gaps for the agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence that efficiently conveys the core purpose. It is front-loaded but could benefit from a brief mention of pagination or scope without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description is insufficient. It does not explain pagination behavior, result structure, or error conditions, which are important for a listing tool with three parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptive names (datasetName, page, limit) and descriptions. The tool description adds no additional meaning beyond what the schema already provides, meeting the baseline expectation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List') and resource ('runs for a specific dataset'), distinguishing it from siblings like 'listDatasets' (which lists datasets) and 'getDatasetRun' (which retrieves a single run). The phrasing is direct and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when you need runs for a specific dataset, but it lacks explicit guidance on when to use this tool versus alternatives (e.g., if filtering or searching is needed, other tools might be more appropriate). No exclusions or when-not-to-use scenarios are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listDatasetsC

List all datasets in the project.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-indexed)
limitNoItems per page

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not disclose that results are paginated with a default limit. The schema implies pagination through page/limit parameters, but the description lacks this behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. However, it could be slightly more informative without losing conciseness, e.g., mentioning pagination.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity and the schema coverage, the description is reasonably complete. However, it lacks mention of pagination behavior or the fact that it returns a list, which are important for an agent to invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters with clear descriptions (page, limit). The description does not add meaning beyond the schema, but since schema coverage is high, a baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('list') and resource ('datasets'), and specifies scope ('all in the project'). It is distinct from sibling tools like getDataset (single) and listDatasetItems (items within a dataset), but does not explicitly mention pagination or that it returns a list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool vs. alternatives like getDataset or listDatasetItems. For a listing tool, it should indicate that it is for paginated listing of all datasets, not for detailed views or filtering.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listObservationsA

List observations (generations, spans, events) with cursor-based pagination. Includes usage metrics, costs, and latency.

ParametersJSON Schema
NameRequiredDescriptionDefault
cursorNoPagination cursor from previous response
limitNoItems per page
traceIdNoFilter by trace ID
nameNoFilter by observation name
typeNoFilter by observation type
parentObservationIdNoFilter by parent observation
fromStartTimeNoStart of time range (ISO 8601)
toStartTimeNoEnd of time range (ISO 8601)
userIdNoFilter by user ID
versionNoFilter by version

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses pagination method and that results include metrics, costs, and latency. However, it does not mention ordering, rate limits, or that the tool is read-only, leaving gaps in behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences long, front-loads the core action and resource, and contains no extraneous information. Every word serves a purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich input schema and absence of annotations and output schema, the description adequately covers purpose and pagination but omits details like default ordering or behavior of time-range filters. It is sufficient but not exhaustive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 10 parameters are fully described in the input schema (100% coverage). The description adds no new meaning about parameters beyond what schema already provides, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (list), resource (observations), and specifies the types included (generations, spans, events). It also mentions cursor-based pagination and included data (usage metrics, costs, latency), effectively distinguishing from sibling tools like getObservation or listTraces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for listing observations with pagination but provides no explicit guidance on when to use this tool versus alternatives (e.g., getObservation for a single item). There are no when-not-to-use notes or comparisons to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listScoreConfigsB

List all score configurations in the project.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-indexed)
limitNoItems per page

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits. It only states it lists configurations, implying a read operation, but fails to mention pagination behavior, ordering, or whether it returns all items at once.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no redundancy. However, it could be improved by including more information about usage and behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description is incomplete. It lacks details about return format, pagination handling, and any limitations, leaving gaps for the agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, as both 'page' and 'limit' have descriptions. The tool description adds no extra meaning beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'List' and the resource 'score configurations' with scope 'in the project'. It effectively distinguishes from siblings like 'listScores' and 'getScoreConfig'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No usage guidance is provided. The description does not indicate when to use this tool versus alternatives, nor does it specify any prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listScoresB

List scores with filtering. Supports both trace and observation scores.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-indexed)
limitNoItems per page
traceIdNoFilter by trace ID
observationIdNoFilter by observation ID
nameNoFilter by score name
sourceNoFilter by score source
dataTypeNoFilter by data type
configIdNoFilter by score config ID
fromTimestampNoStart of time range (ISO 8601)
toTimestampNoEnd of time range (ISO 8601)
userIdNoFilter by user ID who created the trace
operatorNoComparison operator for value filter
valueNoValue to compare against (requires operator)

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavioral traits. It only states 'List scores with filtering,' omitting details like pagination behavior, result ordering, what happens with no results, or potential performance implications of combining multiple filters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short and front-loaded, stating the core action in the first sentence. However, it could include more detail without becoming verbose, so it's slightly below a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should explain the response format or behavior (e.g., pagination, filtering logic). It does not. Additionally, for 13 parameters, it fails to clarify how filters combine (AND/OR) or whether all are optional.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides for each parameter—it only generalizes that filtering is supported.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists scores with filtering, and explicitly mentions it supports both trace and observation scores. This specificity distinguishes it from sibling tools like createScore, getScore, listScoreConfigs, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for listing and filtering scores but does not provide explicit guidance on when to use this tool versus alternatives like getScore for a single score or listObservations. No exclusions or when-not-to-use advice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listSessionsC

List all sessions. Sessions group multiple traces from the same user interaction.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-indexed)
limitNoItems per page
fromTimestampNoStart of time range (ISO 8601)
toTimestampNoEnd of time range (ISO 8601)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. Description does not disclose pagination behavior, auth needs, or other important traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with action, no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and description does not mention return format or pagination details, leaving the agent uninformed about what the tool returns.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema provides full descriptions for all parameters; description adds no extra meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the tool lists sessions and defines sessions briefly. Does not explicitly differentiate from siblings like listTraces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like getSession or listTraces.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listTracesA

List traces with filtering and pagination. Returns trace metadata including latency, cost, and observation/score counts.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-indexed)
limitNoItems per page
nameNoFilter by trace name
userIdNoFilter by user ID
sessionIdNoFilter by session ID
tagsNoFilter by tags (AND logic)
fromTimestampNoStart of time range (ISO 8601)
toTimestampNoEnd of time range (ISO 8601)
environmentNoFilter by environment
orderByNoSort field

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description partially covers behavior by mentioning pagination and return metadata. However, it does not disclose whether the operation is read-only, idempotent, or any potential side effects. More behavioral context would benefit an AI agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that efficiently conveys purpose and key features. It is front-loaded and contains no unnecessary words, though splitting into two sentences could improve readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema, the description partially compensates by listing return fields (latency, cost, counts). However, it lacks details on default sorting, response format, and behavior for empty results, making it moderately complete for a complex list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds only a vague note on return metadata but does not enhance parameter meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists traces with filtering and pagination, and specifies what metadata is returned (latency, cost, observation/score counts). It uses a specific verb-resource pair ('list traces') and distinguishes from sibling tools like createTrace or deleteTrace.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for retrieving traces with filters but does not explicitly offer guidance on when to use this tool versus siblings like listObservations or listSessions. No alternative tools or exclusions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 24 tool updatesv0.1.2
    • First observedcreateDataset
    • First observedcreateDatasetItem
    • First observedcreateDatasetRunItem
    • First observedcreateScore
    • First observedcreateScoreConfig
    • First observeddeleteDatasetItem
    • First observeddeleteScore
    • First observeddeleteTrace
    • First observedgetDataset
    • First observedgetDatasetItem
    • First observedgetDatasetRun
    • First observedgetObservation
    • First observedgetScore
    • First observedgetScoreConfig
    • First observedgetSession
    • First observedgetTrace
    • First observedlistDatasetItems
    • First observedlistDatasetRuns
    • First observedlistDatasets
    • First observedlistObservations
    • First observedlistScoreConfigs
    • First observedlistScores
    • First observedlistSessions
    • First observedlistTraces

TDQS

A3.6/5.0
Disambiguation5/5

Each tool targets a distinct resource-action pair (e.g., createDataset vs. createScore, listTraces vs. listScores). There is no ambiguity as resource types are clearly separated and operations are specific.

Naming Consistency5/5

All tools follow a strict verb+noun pattern (e.g., createDataset, getDatasetItem, listObservations) using snake_case consistently. No mixing of styles or vague verbs.

Tool Count5/5

24 tools cover the major resources (datasets, scores, traces, observations, sessions) with predictable CRUD operations. The count is well-scoped for a comprehensive observability server without being excessive.

Completeness3/5

The tool set lacks create endpoints for traces and observations, which are fundamental to the platform. While dataset items support upsert, the absence of trace and observation creation is a notable gap for a complete evaluation workflow.

Maintenance

ActivityMaintained
ResponsivenessUnresponsive

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/JaviMaligno/langfuse-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server