langfuse-mcp-extended
This server provides comprehensive access to Langfuse observability data, enabling AI assistants to manage and query traces, observations, scores, datasets, and sessions. It includes 24 tools for listing, retrieving, creating, updating, and deleting these entities, with filtering, sorting, and pagination. Key features include the ability to control large input/output payloads via the includeIO parameter, support for both Langfuse Cloud (US/EU) and self-hosted instances, and compatibility with the Langfuse prompts MCP server. You can analyze multi-turn sessions, manage evaluation datasets with run linking, create score configurations, and track costs and usage metrics across generations and spans.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@langfuse-mcp-extendedlist the last 10 traces from today"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Langfuse MCP Server (Extended)
A comprehensive Model Context Protocol (MCP) server for Langfuse, providing AI assistants with full access to traces, observations, scores, datasets, and sessions.
Prompt Management: This server focuses on observability data. For prompt management, Langfuse provides a built-in MCP server (GitHub). We're working on integrating both - see Contributing.
Features
24 tools for complete Langfuse observability access
Traces: List, get, and delete traces with filtering and pagination
Observations: Query generations, spans, and events with usage metrics
Scores: Full CRUD operations for evaluation scores (numeric, categorical, boolean)
Score Configs: Manage score configuration templates
Datasets: Complete dataset management including items and evaluation runs
Sessions: Access session data with associated traces
Cloud & Self-hosted: Works with Langfuse Cloud (US/EU) and self-hosted instances
Related MCP server: langfuse-mcp
Quick Start
One-liner Installation (Claude Code)
claude mcp add langfuse -e LANGFUSE_PUBLIC_KEY=pk-lf-xxx -e LANGFUSE_SECRET_KEY=sk-lf-xxx -- npx -y langfuse-mcp-extendedFor EU Cloud or self-hosted, add the base URL:
claude mcp add langfuse -e LANGFUSE_PUBLIC_KEY=pk-lf-xxx -e LANGFUSE_SECRET_KEY=sk-lf-xxx -e LANGFUSE_BASE_URL=https://eu.cloud.langfuse.com -- npx -y langfuse-mcp-extendedInstallation by Client
Add to your Claude Desktop configuration file:
macOS:
~/Library/Application Support/Claude/claude_desktop_config.jsonWindows:
%APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"langfuse": {
"command": "npx",
"args": ["-y", "langfuse-mcp-extended"],
"env": {
"LANGFUSE_PUBLIC_KEY": "pk-lf-...",
"LANGFUSE_SECRET_KEY": "sk-lf-..."
}
}
}
}For EU Cloud or self-hosted, add LANGFUSE_BASE_URL:
{
"mcpServers": {
"langfuse": {
"command": "npx",
"args": ["-y", "langfuse-mcp-extended"],
"env": {
"LANGFUSE_PUBLIC_KEY": "pk-lf-...",
"LANGFUSE_SECRET_KEY": "sk-lf-...",
"LANGFUSE_BASE_URL": "https://eu.cloud.langfuse.com"
}
}
}
}Option 1: One-liner with environment variables
claude mcp add langfuse -e LANGFUSE_PUBLIC_KEY=pk-lf-xxx -e LANGFUSE_SECRET_KEY=sk-lf-xxx -- npx -y langfuse-mcp-extendedOption 2: Add to project's .mcp.json
{
"mcpServers": {
"langfuse": {
"command": "npx",
"args": ["-y", "langfuse-mcp-extended"],
"env": {
"LANGFUSE_PUBLIC_KEY": "pk-lf-...",
"LANGFUSE_SECRET_KEY": "sk-lf-..."
}
}
}
}Option 3: Use shell environment variables
claude mcp add langfuse -- npx -y langfuse-mcp-extendedThen set in your shell profile (.bashrc, .zshrc, etc.):
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."Add to your Cursor MCP configuration:
Project-specific:
.cursor/mcp.jsonin your project directoryGlobal:
~/.cursor/mcp.jsonin your home directory
{
"mcpServers": {
"langfuse": {
"command": "npx",
"args": ["-y", "langfuse-mcp-extended"],
"env": {
"LANGFUSE_PUBLIC_KEY": "pk-lf-...",
"LANGFUSE_SECRET_KEY": "sk-lf-..."
}
}
}
}Or add via UI: File → Preferences → Cursor Settings → MCP
Add to .vscode/mcp.json in your workspace:
{
"mcpServers": {
"langfuse": {
"command": "npx",
"args": ["-y", "langfuse-mcp-extended"],
"env": {
"LANGFUSE_PUBLIC_KEY": "pk-lf-...",
"LANGFUSE_SECRET_KEY": "sk-lf-..."
}
}
}
}npm install -g langfuse-mcp-extendedThen use langfuse-mcp-extended as the command instead of npx -y langfuse-mcp-extended.
Configuration
Required Environment Variables
Variable | Description |
| Your Langfuse public key ( |
| Your Langfuse secret key ( |
Optional Environment Variables
Variable | Default | Description |
|
| Langfuse instance URL |
|
| Logging level: |
Langfuse Instance URLs
Instance | URL |
US Cloud |
|
EU Cloud |
|
Self-hosted | Your instance URL (e.g., |
API Reference
Pagination
All list operations use page-based pagination with a default limit of 10 items per page:
page(number, optional): Page number, 1-indexed. Default: 1limit(number, optional): Items per page, max 100. Default: 10
Response Size Control
Get operations (getTrace, getSession, getObservation) support an includeIO parameter:
includeIO(boolean, optional): Include input/output fields. Default:false
When includeIO is false (default), large input and output fields are stripped from responses to prevent exceeding LLM context limits. Set to true when you need the full payload.
listTraces
List traces with filtering and pagination.
Inputs:
page(number, optional): Page numberlimit(number, optional): Items per pagename(string, optional): Filter by trace nameuserId(string, optional): Filter by user IDsessionId(string, optional): Filter by session IDtags(string[], optional): Filter by tags (AND logic)fromTimestamp(string, optional): Start of time range (ISO 8601)toTimestamp(string, optional): End of time range (ISO 8601)environment(string, optional): Filter by environmentorderBy(string, optional): Sort bytimestamp,latency, ortotalCost
getTrace
Get a specific trace with full details.
Inputs:
traceId(string, required): The trace IDincludeIO(boolean, optional): Include input/output fields. Default:false
Returns: Trace with observations and scores.
deleteTrace
Delete a trace.
Inputs:
traceId(string, required): The trace ID to delete
listObservations
List observations (generations, spans, events) with cursor-based pagination.
Inputs:
cursor(string, optional): Pagination cursorlimit(number, optional): Items per pagetraceId(string, optional): Filter by trace IDname(string, optional): Filter by observation nametype(string, optional): Filter by type:GENERATION,SPAN,EVENTparentObservationId(string, optional): Filter by parentfromStartTime(string, optional): Start of time range (ISO 8601)toStartTime(string, optional): End of time range (ISO 8601)userId(string, optional): Filter by user IDversion(string, optional): Filter by version
getObservation
Get a specific observation with all details.
Inputs:
observationId(string, required): The observation IDincludeIO(boolean, optional): Include input/output fields. Default:false
Returns: Observation with usage, costs, and timing.
createScore
Create a score for a trace or observation.
Inputs:
traceId(string, required): ID of the trace to scorename(string, required): Score name (e.g.,accuracy,relevance)value(number | string, required): Score valueobservationId(string, optional): ID of specific observation to scoredataType(string, optional):NUMERIC,CATEGORICAL, orBOOLEANcomment(string, optional): Explanation of the scoreconfigId(string, optional): Score config ID for validationid(string, optional): Custom ID for idempotency
listScores
List scores with filtering.
Inputs:
page,limit(pagination)traceId(string, optional): Filter by trace IDobservationId(string, optional): Filter by observation IDname(string, optional): Filter by score namesource(string, optional): Filter by source:API,ANNOTATION,EVALdataType(string, optional): Filter by data typeconfigId(string, optional): Filter by config IDfromTimestamp,toTimestamp(string, optional): Time rangeuserId(string, optional): Filter by user IDoperator(string, optional): Comparison operator:<,>,<=,>=,!=,=value(number, optional): Value to compare (requires operator)
getScore
Get a specific score.
Inputs:
scoreId(string, required): The score ID
deleteScore
Delete a score.
Inputs:
scoreId(string, required): The score ID to delete
createScoreConfig
Create a score configuration template.
Inputs:
name(string, required): Unique config namedataType(string, required):NUMERIC,CATEGORICAL, orBOOLEANminValue(number, optional): Minimum value (NUMERIC only)maxValue(number, optional): Maximum value (NUMERIC only)categories(array, optional): Category definitions (CATEGORICAL only)description(string, optional): Description
listScoreConfigs
List all score configurations.
Inputs:
page,limit(pagination)
getScoreConfig
Get a specific score configuration.
Inputs:
configId(string, required): The config ID
createDataset
Create a new dataset.
Inputs:
name(string, required): Dataset namedescription(string, optional): Descriptionmetadata(object, optional): Additional metadata
listDatasets
List all datasets.
Inputs:
page,limit(pagination)
getDataset
Get a dataset by name.
Inputs:
datasetName(string, required): The dataset name
createDatasetItem
Create or update a dataset item.
Inputs:
datasetName(string, required): Target dataset nameinput(any, required): Item input dataexpectedOutput(any, optional): Expected outputmetadata(object, optional): Item metadatasourceTraceId(string, optional): Source trace IDsourceObservationId(string, optional): Source observation IDid(string, optional): Custom ID for upsert
listDatasetItems
List items in a dataset.
Inputs:
datasetName(string, required): Dataset namepage,limit(pagination)sourceTraceId(string, optional): Filter by source tracesourceObservationId(string, optional): Filter by source observation
getDatasetItem
Get a specific dataset item.
Inputs:
datasetItemId(string, required): The item ID
deleteDatasetItem
Delete a dataset item.
Inputs:
datasetItemId(string, required): The item ID to delete
createDatasetRunItem
Link a trace/observation to a dataset item for evaluation.
Inputs:
runName(string, required): Name of the dataset rundatasetItemId(string, required): Dataset item IDtraceId(string, required): Trace IDrunDescription(string, optional): Run descriptionobservationId(string, optional): Observation IDmetadata(object, optional): Run item metadata
listDatasetRuns
List runs for a dataset.
Inputs:
datasetName(string, required): Dataset namepage,limit(pagination)
getDatasetRun
Get a specific dataset run.
Inputs:
datasetName(string, required): Dataset namerunName(string, required): Run name
listSessions
List all sessions.
Inputs:
page,limit(pagination)fromTimestamp(string, optional): Start of time range (ISO 8601)toTimestamp(string, optional): End of time range (ISO 8601)
getSession
Get a specific session with its traces.
Inputs:
sessionId(string, required): The session IDincludeIO(boolean, optional): Include input/output in traces. Default:false
Returns: Session with associated traces.
Using with Official Langfuse MCP (Prompts)
This server provides observability tools (traces, scores, datasets). For prompt management, Langfuse provides a built-in MCP server that requires no installation.
Built-in Langfuse MCP (Recommended)
The Langfuse MCP server is built directly into Langfuse at /api/public/mcp. See the official documentation for setup instructions.
Using Both Servers Together
To use Langfuse observability (this server) alongside the official prompts MCP:
{
"mcpServers": {
"langfuse-observability": {
"command": "npx",
"args": ["-y", "langfuse-mcp-extended"],
"env": {
"LANGFUSE_PUBLIC_KEY": "pk-lf-...",
"LANGFUSE_SECRET_KEY": "sk-lf-..."
}
}
}
}Then configure the built-in Langfuse prompts MCP following the official guide.
Note: We're actively working with the Langfuse team to potentially integrate both servers. See Contributing for details.
Development
# Install dependencies
npm install
# Build
npm run build
# Run in development mode
npm run dev
# Run tests
npm test # Unit tests (59 tests)
npm run test:integration # Integration tests (18 tests)
npm run test:all # All tests
# Quality checks
npm run lint
npm run typecheckContributing
Contributions are welcome! This project aims to be integrated into the official Langfuse ecosystem.
Active discussions:
langfuse/langfuse#5646 - Integration discussion
langfuse/mcp-server-langfuse#14 - Feature request
How to contribute:
Fork the repository
Create your feature branch (
git checkout -b feature/amazing-feature)Commit your changes (
git commit -m 'Add amazing feature')Push to the branch (
git push origin feature/amazing-feature)Open a Pull Request
Author
Built by Javier Aguilar - AI Agent Architect specializing in multi-agent orchestration and MCP development.
License
MIT License - see LICENSE for details.
Related
Langfuse - Open source LLM observability
Langfuse MCP Server (Prompts) - Built-in prompt management MCP
Model Context Protocol - MCP specification
MCP Servers Registry - Official MCP servers
Available Tools
24 toolscreateDatasetB
Create a new dataset for evaluation. Datasets contain items with expected inputs/outputs for testing LLM applications.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Unique name for the dataset | |
| description | No | Description of the dataset | |
| metadata | No | Additional metadata as key-value pairs |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose side effects, permissions, rate limits, or return value information. The behavioral impact (e.g., a write operation) is only implicit from the word 'create'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, with no wasted words. It front-loads the core purpose and follows with a brief explanatory note about datasets.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is insufficient. It does not mention the return value, idempotency, or any constraints beyond parameter types, leaving the agent guessing about the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds no additional meaning beyond what is in the schema for parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create) and resource (dataset), and provides context that datasets are for evaluation. This differentiates it from sibling tools like createDatasetItem, which operate on dataset contents.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a new evaluation dataset is needed, but does not provide explicit guidance on when not to use this tool or mention alternatives like getDatasets or listDatasets.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
createDatasetItemB
Create or upsert a dataset item. Use id parameter for idempotent updates.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetName | Yes | Name of the dataset to add the item to | |
| input | No | The input for this dataset item (any JSON value) | |
| expectedOutput | No | The expected output for evaluation | |
| metadata | No | Additional metadata | |
| sourceTraceId | No | Link to source trace that generated this item | |
| sourceObservationId | No | Link to source observation (requires sourceTraceId) | |
| id | No | Custom ID for idempotent upsert | |
| status | No | Item status (defaults to ACTIVE) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It mentions upsert behavior, but lacks details on side effects (e.g., conflict resolution), return values, or required permissions. The upsert indication is helpful but insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences: one stating purpose and one providing a usage tip. Every word adds value, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, no output schema, no annotations), the description is minimal. It lacks context about return values, error conditions, prerequisites (e.g., dataset must exist), and differentiation from similar sibling tools like createDatasetRunItem.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with each parameter described. The description adds value by explaining the purpose of the id parameter for idempotent updates, but does not enrich other parameter semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create or upsert a dataset item,' specifying the verb and resource. It distinguishes from sibling tools like createDataset (creates dataset itself) and createDatasetRunItem, though it could be more explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides guidance on using the id parameter for idempotent updates, but it does not specify when to use this tool versus alternatives or mention any prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
createDatasetRunItemA
Create a dataset run item linking a trace/observation to a dataset item for evaluation.
| Name | Required | Description | Default |
|---|---|---|---|
| runName | Yes | Name of the dataset run | |
| runDescription | No | Description of the run | |
| datasetItemId | Yes | ID of the dataset item being evaluated | |
| traceId | Yes | ID of the trace that processed this item | |
| observationId | No | ID of specific observation (if applicable) | |
| metadata | No | Additional run item metadata |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must convey behavioral traits. It mentions creation (mutation) but omits side effects, permissions, idempotency, or error conditions. Adequate for a simple creation but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, focused sentence that concisely conveys the purpose. No wasted words, front-loading the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, so description should hint at return value or confirmation. It explains input and linking but doesn't describe the created item's structure or possible errors. Adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. Description does not add value beyond the schema; it only restates the linking concept. No parameter-specific details like constraints or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'create' and resource 'dataset run item' and clearly states the linking purpose (trace/observation to dataset item for evaluation). It effectively distinguishes from sibling tools like createDataset (creates dataset) or createDatasetItem (creates item).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs alternatives like createScore or other evaluation tools. The description implies usage for evaluation but doesn't mention exclusions or prerequisites, leaving the agent to infer context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
createScoreA
Create a score for a trace or observation. Supports numeric, boolean, and categorical score types.
| Name | Required | Description | Default |
|---|---|---|---|
| traceId | Yes | ID of the trace to score | |
| observationId | No | Optional: ID of specific observation to score | |
| name | Yes | Name of the score (e.g., 'accuracy', 'relevance') | |
| value | Yes | Score value - numeric for NUMERIC type, string for CATEGORICAL/BOOLEAN | |
| dataType | No | Type of score value | |
| comment | No | Optional comment explaining the score | |
| configId | No | Optional score config ID for validation | |
| id | No | Optional custom ID for idempotency |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It adds value by noting supported score types but omits idempotency, validation behavior, and side effects. Does not contradict schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with key purpose, no unnecessary words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks details on return value, interaction between traceId and observationId, and validation with configId. For a creation tool with 8 parameters and no output schema, more context would help agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter described. Description reinforces dataType enum but adds minimal new semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action ('Create'), resource ('score for trace or observation'), and supported types ('numeric, boolean, and categorical'). This distinguishes it from sibling tools like createScoreConfig.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies use when scoring a trace or observation but offers no explicit guidance on when to choose this over alternatives or conditions to avoid. Sibling tools like createScoreConfig and deleteScore exist but are not mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
createScoreConfigC
Create a score configuration that defines validation rules for scores. Supports numeric ranges, categorical values, and boolean types.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Unique name for the score configuration | |
| dataType | Yes | Type of score values this config accepts | |
| minValue | No | Minimum allowed value (NUMERIC only) | |
| maxValue | No | Maximum allowed value (NUMERIC only) | |
| categories | No | Category definitions (CATEGORICAL only) | |
| description | No | Description of what this score measures |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It mentions creation and supported types but omits side effects (e.g., idempotency, overwrite behavior), permissions, or error conditions. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences, front-loaded with purpose. No wasted words. Slightly low on detail, but still concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks description of return values (no output schema), prerequisites, or error conditions. For a creation tool with 6 parameters, this is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains parameters. The description adds context that the config defines validation rules, but this is implied by the tool's purpose. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a score configuration that defines validation rules for scores' and lists supported types. This provides a specific verb and resource, distinguishing it from other tools like createScore. However, it does not explicitly differentiate from siblings, so it scores 4.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as createScore. The description lacks context on prerequisites or exclusions, leaving the agent with no decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deleteDatasetItemA
Delete a specific dataset item. This action is irreversible.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetItemId | Yes | The unique dataset item identifier to delete |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the action is irreversible, a key behavioral trait. However, without annotations, it omits other important details like required permissions, side effects on related data, or whether deletion cascades. The irreversibility note is helpful but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with only two sentences, front-loaded with the key action. Every word adds value, and no unnecessary information is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema, no annotations), the description is largely adequate. It states the purpose and irreversibility. Additional context, such as success/error feedback or confirmation, would improve completeness but is not critical for this low-complexity tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter, so the schema already defines its meaning. The description adds no extra semantics beyond the schema, resulting in a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('a specific dataset item'), with no ambiguity. It also differentiates from sibling delete tools by specifying 'dataset item', making the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as other delete tools (e.g., deleteScore, deleteTrace). It lacks context about prerequisites or when not to use it, leaving the agent to infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deleteScoreB
Delete a specific score. This action is irreversible.
| Name | Required | Description | Default |
|---|---|---|---|
| scoreId | Yes | The unique score identifier to delete |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only mentions irreversibility but lacks details on authorization, side effects on associated data, or whether any confirmation is needed. Inadequate for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise and front-loaded: two sentences that convey purpose and a critical behavioral note. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required parameter, no output schema), the description covers the core purpose and irreversibility. It could mention that only the score record is deleted (no cascading effects) for completeness, but it's adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (scoreId is well described). The description adds no extra meaning beyond the schema—both state essentially the same thing. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete), the resource (specific score), and emphasizes irreversibility. It unambiguously distinguishes from sibling tools like createScore or getScore.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. There is no mention of prerequisites, permissions, or scenarios where deletion is appropriate or inadvisable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deleteTraceA
Delete a specific trace. This action is irreversible.
| Name | Required | Description | Default |
|---|---|---|---|
| traceId | Yes | The unique trace identifier to delete |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
States irreversibility, which is important for a delete operation. Lacks details on side effects, permissions, or confirmation steps. No annotations to supplement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences with no filler. Front-loaded with verb and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple delete with one parameter, but lacks behavioral context like cascade effects. No output schema needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
100% schema coverage for parameter traceId; description adds no extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (Delete) and resource (a specific trace). Distinguishes from sibling tools like getTrace, listTraces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidelines on when to use this tool vs alternatives. Does not mention prerequisites, conditions, or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getDatasetA
Get a dataset by name including its items and run names.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetName | Yes | The name of the dataset to retrieve |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description does not disclose any behavioral traits, such as required permissions, effects, or return format beyond the basic retrieval.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is concise and front-loaded with the key purpose, containing no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with one parameter and no output schema, the description is adequate, mentioning that items and run names are included. It covers the essential context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter datasetName has 100% schema description coverage, so the description adds no additional value beyond the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get a dataset by name' and specifies the scope 'including its items and run names'. It differentiates from sibling tools like listDatasets and deleteDatasetItem.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you need a specific dataset by name, but lacks explicit when-not or alternatives such as listDatasets for multiple datasets.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getDatasetItemA
Get a specific dataset item by ID.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetItemId | Yes | The unique dataset item identifier |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states the retrieval operation without disclosing any behavioral traits, side effects, or return structure. Minimal disclosure is insufficient for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It front-loads the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval with one parameter, the description is nearly complete. It lacks output description, but no output schema exists. In context of siblings, it is self-explanatory.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds 'by ID' but the schema already describes the parameter as 'identifier'. No additional semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the resource 'dataset item', and specifies the method 'by ID'. It distinguishes from sibling tools like listDatasetItems (list multiple) and createDatasetItem (create).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The purpose is implied by the name and description, but there is no mention of when not to use it or which sibling to prefer in different contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getDatasetRunA
Get a specific dataset run by name including its run items.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetName | Yes | Name of the dataset | |
| runName | Yes | Name of the run to retrieve |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the transparency burden. It discloses that the result includes run items, which adds value beyond the input schema. However, it does not mention any side effects, permissions, or error conditions, which is acceptable for a simple read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 12 words. Every word is necessary and efficiently communicates the tool's purpose. No wasted space.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (2 required parameters, no output schema), the description is fairly complete. It explains that run items are included, though it could optionally describe the return format. Overall, it adequately covers the essential context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with clear descriptions for both datasetName and runName. The tool description adds no additional meaning beyond what is already in the schema, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get', the resource 'a specific dataset run', and the scope 'by name including its run items'. This differentiates it from siblings like listDatasetRuns (which lists without items) and getDataset (which gets the dataset itself).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when or when not to use this tool versus alternatives is provided. The description only states what it does, leaving the agent to infer usage context from the name and siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getObservationA
Get a specific observation with all details including input/output, usage, costs, and timing.
| Name | Required | Description | Default |
|---|---|---|---|
| observationId | Yes | The unique observation identifier | |
| includeIO | No | Include input/output fields (can be very large, disabled by default) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, description warns about large payloads when includeIO is true ('can be very large, disabled by default'). Adds important behavioral context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and key details. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with 2 params and no output schema, description provides adequate context about what is returned and the large I/O warning. Missing error handling info but still complete enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds minimal extra meaning beyond schema descriptions, only summarizing overall details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get a specific observation' with verb and resource, and specifies details like input/output, usage, costs, and timing. Distinguishes from sibling list tools like listObservations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use or not use this tool compared to alternatives like listObservations or getTrace. Lacks context for agent decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getScoreB
Get a specific score by ID.
| Name | Required | Description | Default |
|---|---|---|---|
| scoreId | Yes | The unique score identifier |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must convey behavioral traits. It does not disclose whether the operation is read-only, requires permissions, or how errors are handled (e.g., score not found).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single simple sentence, which is concise but may be too brief for a tool that could benefit from a minimal expansion (e.g., mentioning that it returns the full score object).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a straightforward retrieval tool with one parameter and no output schema, the description is minimally adequate. However, it does not specify the return format or error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage for the single parameter is 100% via its description. The tool description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Get a specific score by ID,' which clearly identifies the action (retrieve), resource (score), and method (by identifier). This distinguishes it from sibling tools like createScore or listScores.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., listScores). The description lacks context about prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getScoreConfigA
Get a specific score configuration by ID.
| Name | Required | Description | Default |
|---|---|---|---|
| configId | Yes | The unique score config identifier |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It only states the basic action without mentioning idempotency, return format, or any side effects, which is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, clear sentence with no wasted words. It is appropriately concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While adequate for a simple get-by-ID tool, it lacks information on what is returned (e.g., the full configuration object) and how to obtain the config ID, which would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes the single parameter with 100% coverage. The description adds no additional meaning beyond 'by ID', so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a specific score configuration by ID, using a specific verb and resource, and is distinct from sibling tools like listScoreConfigs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you have a specific config ID, but does not explicitly state when to use this tool versus alternatives like listScoreConfigs or createScoreConfig.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getSessionA
Get a specific session by ID including its traces.
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The unique session identifier | |
| includeIO | No | Include input/output fields in traces (can be very large, disabled by default) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It mentions 'including its traces', which hints at the response content, but it does not disclose any behavioral traits such as data size implications, permissions, or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no waste, but it lacks structural elements like separate sections for purpose and behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description should explain return values in more detail. 'Including its traces' is helpful but incomplete for a tool with necessary parameters and no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with thorough descriptions for both parameters. The description does not add any additional meaning beyond what the input schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the resource 'specific session by ID including its traces'. It effectively distinguishes from sibling tools like listSessions (which lists sessions) and getTrace (which gets a single trace).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you have a specific session ID and need traces, but it does not provide explicit guidance on when to use this tool versus alternatives like listSessions or getObservation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getTraceA
Get a specific trace with full details including all observations and scores.
| Name | Required | Description | Default |
|---|---|---|---|
| traceId | Yes | The unique trace identifier | |
| includeIO | No | Include input/output fields (can be very large, disabled by default) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It mentions 'full details' and the includeIO parameter in schema but does not state whether the operation is read-only, safe, or any potential side effects, nor does it address performance implications of large payloads.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 14 words, front-loaded with the action and resource, with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description should explain the return format. Saying 'full details including all observations and scores' is vague and does not provide enough structure or relationships with sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description does not add meaning beyond the schema; it only describes the output, not the parameters themselves.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'a specific trace', and adds 'with full details including all observations and scores', which distinguishes it from sibling tools like listTraces or getObservation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies that this tool should be used when needing a trace with all its observations and scores, subtly differentiating from getObservation or getScore, but lacks explicit guidance on when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listDatasetItemsC
List items in a dataset with optional filtering.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetName | Yes | Name of the dataset to list items from | |
| page | No | Page number (1-indexed) | |
| limit | No | Items per page | |
| sourceTraceId | No | Filter by source trace ID | |
| sourceObservationId | No | Filter by source observation ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It does not disclose that this is a read-only operation or any side effects. The word 'list' implies read-only, but no explicit safety or behavioral context is given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no waste. It efficiently communicates the core purpose, though it could be slightly more detailed without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters and no output schema, the description is insufficient. It does not explain that results are paginated, what a dataset item is, or what the return format looks like. The tool has several filters, but the description only mentions 'optional filtering' generically.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides for each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('items in a dataset'). It distinguishes this tool from siblings like listDatasets and listDatasetRuns, though it does not explicitly differentiate from these similar tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., listDatasets or listDatasetRuns). No when-not-to-use or context for filtering is mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listDatasetRunsB
List all runs for a specific dataset.
| Name | Required | Description | Default |
|---|---|---|---|
| datasetName | Yes | Name of the dataset to list runs from | |
| page | No | Page number (1-indexed) | |
| limit | No | Items per page |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'List all runs' without specifying pagination behavior, whether deleted runs are included, or error handling (e.g., dataset not found). This leaves significant gaps for the agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence that efficiently conveys the core purpose. It is front-loaded but could benefit from a brief mention of pagination or scope without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description is insufficient. It does not explain pagination behavior, result structure, or error conditions, which are important for a listing tool with three parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptive names (datasetName, page, limit) and descriptions. The tool description adds no additional meaning beyond what the schema already provides, meeting the baseline expectation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('runs for a specific dataset'), distinguishing it from siblings like 'listDatasets' (which lists datasets) and 'getDatasetRun' (which retrieves a single run). The phrasing is direct and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when you need runs for a specific dataset, but it lacks explicit guidance on when to use this tool versus alternatives (e.g., if filtering or searching is needed, other tools might be more appropriate). No exclusions or when-not-to-use scenarios are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listDatasetsC
List all datasets in the project.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-indexed) | |
| limit | No | Items per page |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose that results are paginated with a default limit. The schema implies pagination through page/limit parameters, but the description lacks this behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. However, it could be slightly more informative without losing conciseness, e.g., mentioning pagination.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the schema coverage, the description is reasonably complete. However, it lacks mention of pagination behavior or the fact that it returns a list, which are important for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with clear descriptions (page, limit). The description does not add meaning beyond the schema, but since schema coverage is high, a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('list') and resource ('datasets'), and specifies scope ('all in the project'). It is distinct from sibling tools like getDataset (single) and listDatasetItems (items within a dataset), but does not explicitly mention pagination or that it returns a list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs. alternatives like getDataset or listDatasetItems. For a listing tool, it should indicate that it is for paginated listing of all datasets, not for detailed views or filtering.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listObservationsA
List observations (generations, spans, events) with cursor-based pagination. Includes usage metrics, costs, and latency.
| Name | Required | Description | Default |
|---|---|---|---|
| cursor | No | Pagination cursor from previous response | |
| limit | No | Items per page | |
| traceId | No | Filter by trace ID | |
| name | No | Filter by observation name | |
| type | No | Filter by observation type | |
| parentObservationId | No | Filter by parent observation | |
| fromStartTime | No | Start of time range (ISO 8601) | |
| toStartTime | No | End of time range (ISO 8601) | |
| userId | No | Filter by user ID | |
| version | No | Filter by version |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses pagination method and that results include metrics, costs, and latency. However, it does not mention ordering, rate limits, or that the tool is read-only, leaving gaps in behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loads the core action and resource, and contains no extraneous information. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the rich input schema and absence of annotations and output schema, the description adequately covers purpose and pagination but omits details like default ordering or behavior of time-range filters. It is sufficient but not exhaustive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 10 parameters are fully described in the input schema (100% coverage). The description adds no new meaning about parameters beyond what schema already provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list), resource (observations), and specifies the types included (generations, spans, events). It also mentions cursor-based pagination and included data (usage metrics, costs, latency), effectively distinguishing from sibling tools like getObservation or listTraces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing observations with pagination but provides no explicit guidance on when to use this tool versus alternatives (e.g., getObservation for a single item). There are no when-not-to-use notes or comparisons to siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listScoreConfigsB
List all score configurations in the project.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-indexed) | |
| limit | No | Items per page |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states it lists configurations, implying a read operation, but fails to mention pagination behavior, ordering, or whether it returns all items at once.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no redundancy. However, it could be improved by including more information about usage and behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description is incomplete. It lacks details about return format, pagination handling, and any limitations, leaving gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, as both 'page' and 'limit' have descriptions. The tool description adds no extra meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'score configurations' with scope 'in the project'. It effectively distinguishes from siblings like 'listScores' and 'getScoreConfig'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance is provided. The description does not indicate when to use this tool versus alternatives, nor does it specify any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listScoresB
List scores with filtering. Supports both trace and observation scores.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-indexed) | |
| limit | No | Items per page | |
| traceId | No | Filter by trace ID | |
| observationId | No | Filter by observation ID | |
| name | No | Filter by score name | |
| source | No | Filter by score source | |
| dataType | No | Filter by data type | |
| configId | No | Filter by score config ID | |
| fromTimestamp | No | Start of time range (ISO 8601) | |
| toTimestamp | No | End of time range (ISO 8601) | |
| userId | No | Filter by user ID who created the trace | |
| operator | No | Comparison operator for value filter | |
| value | No | Value to compare against (requires operator) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavioral traits. It only states 'List scores with filtering,' omitting details like pagination behavior, result ordering, what happens with no results, or potential performance implications of combining multiple filters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and front-loaded, stating the core action in the first sentence. However, it could include more detail without becoming verbose, so it's slightly below a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should explain the response format or behavior (e.g., pagination, filtering logic). It does not. Additionally, for 13 parameters, it fails to clarify how filters combine (AND/OR) or whether all are optional.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides for each parameter—it only generalizes that filtering is supported.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists scores with filtering, and explicitly mentions it supports both trace and observation scores. This specificity distinguishes it from sibling tools like createScore, getScore, listScoreConfigs, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing and filtering scores but does not provide explicit guidance on when to use this tool versus alternatives like getScore for a single score or listObservations. No exclusions or when-not-to-use advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listSessionsC
List all sessions. Sessions group multiple traces from the same user interaction.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-indexed) | |
| limit | No | Items per page | |
| fromTimestamp | No | Start of time range (ISO 8601) | |
| toTimestamp | No | End of time range (ISO 8601) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description does not disclose pagination behavior, auth needs, or other important traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with action, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema and description does not mention return format or pagination details, leaving the agent uninformed about what the tool returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema provides full descriptions for all parameters; description adds no extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool lists sessions and defines sessions briefly. Does not explicitly differentiate from siblings like listTraces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like getSession or listTraces.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listTracesA
List traces with filtering and pagination. Returns trace metadata including latency, cost, and observation/score counts.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-indexed) | |
| limit | No | Items per page | |
| name | No | Filter by trace name | |
| userId | No | Filter by user ID | |
| sessionId | No | Filter by session ID | |
| tags | No | Filter by tags (AND logic) | |
| fromTimestamp | No | Start of time range (ISO 8601) | |
| toTimestamp | No | End of time range (ISO 8601) | |
| environment | No | Filter by environment | |
| orderBy | No | Sort field |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description partially covers behavior by mentioning pagination and return metadata. However, it does not disclose whether the operation is read-only, idempotent, or any potential side effects. More behavioral context would benefit an AI agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that efficiently conveys purpose and key features. It is front-loaded and contains no unnecessary words, though splitting into two sentences could improve readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description partially compensates by listing return fields (latency, cost, counts). However, it lacks details on default sorting, response format, and behavior for empty results, making it moderately complete for a complex list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds only a vague note on return metadata but does not enhance parameter meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists traces with filtering and pagination, and specifies what metadata is returned (latency, cost, observation/score counts). It uses a specific verb-resource pair ('list traces') and distinguishes from sibling tools like createTrace or deleteTrace.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving traces with filters but does not explicitly offer guidance on when to use this tool versus siblings like listObservations or listSessions. No alternative tools or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
24 tool updates
v0.1.2- First observed
createDataset - First observed
createDatasetItem - First observed
createDatasetRunItem - First observed
createScore - First observed
createScoreConfig - First observed
deleteDatasetItem - First observed
deleteScore - First observed
deleteTrace - First observed
getDataset - First observed
getDatasetItem - First observed
getDatasetRun - First observed
getObservation - First observed
getScore - First observed
getScoreConfig - First observed
getSession - First observed
getTrace - First observed
listDatasetItems - First observed
listDatasetRuns - First observed
listDatasets - First observed
listObservations - First observed
listScoreConfigs - First observed
listScores - First observed
listSessions - First observed
listTraces
TDQS
Each tool targets a distinct resource-action pair (e.g., createDataset vs. createScore, listTraces vs. listScores). There is no ambiguity as resource types are clearly separated and operations are specific.
All tools follow a strict verb+noun pattern (e.g., createDataset, getDatasetItem, listObservations) using snake_case consistently. No mixing of styles or vague verbs.
24 tools cover the major resources (datasets, scores, traces, observations, sessions) with predictable CRUD operations. The count is well-scoped for a comprehensive observability server without being excessive.
The tool set lacks create endpoints for traces and observations, which are fundamental to the platform. While dataset items support upsert, the absence of trace and observation creation is a notable gap for a complete evaluation workflow.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
MCP server that lets AI assistants use all OneSchema features exposed via the public API.
Cloud hosted Okahu MCP server that helps you manage genAI trace data
Related MCP Servers
- AlicenseCqualityCmaintenanceAn MCP server implementation that integrates AI assistants with Langfuse workspaces, allowing models to query LLM metrics by time range.1153Apache 2.0
- AlicenseCqualityAmaintenanceMCP server for Langfuse observability. Query traces, debug exceptions, analyze sessions, and manage prompts and datasets for your LLM applications.48105MIT
- AlicenseNot gradedqualityDmaintenanceA Model Context Protocol (MCP) server for comprehensive monitoring and observability of systems using Langfuse.1MIT
- AlicenseNot gradedqualityAmaintenanceMCP server for Langfuse providing 50 tools for prompt management, observability, instance/organization management, and metrics tracking via natural language.18MIT
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/JaviMaligno/langfuse-mcp-server'
If you have feedback or need assistance with the MCP directory API, please join our Discord server