Skip to main content
Glama

Clerk Chat MCP Server

MCP server for Clerk Chat voice AI tools and skills.

Features

Prompt Improvement

Autonomous prompt improvement loop for voice AI agents:

  • Analyze call transcripts and feedback

  • Generate improved prompts

  • Create regression tests with LLM evaluation

  • Track improvement runs in database

  • Iterate until tests pass

Related MCP server: Prompt Learning MCP Server

Setup

1. Install Dependencies

npm install

2. Configure API Access

The server requires a Clerk Chat API key for full functionality. You have two options:

Option A: Using .env file (Development)

  1. Copy the example configuration:

    cp .env.example .env
  2. Edit .env and add your API key:

    CLERK_CHAT_API_KEY=your_api_key_here

Option B: Using Claude Desktop config (Production)

Add environment variables directly to your Claude Desktop config at: ~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "clerk-chat": {
      "command": "node",
      "args": ["/absolute/path/to/clerk-chat-mcp/src/index.js"],
      "env": {
        "CLERK_CHAT_API_KEY": "07fe6d3f658d65d7fd906068e21eef5f5182fd2438e66c78e6786b661e668b2e",
        "CLERK_CHAT_API_BASE_URL": "https://puxgxqdkizwdzqyuaitm.supabase.co/functions/v1"
      }
    }
  }
}

Note: The API key shown above is for Technical Life Care company (testing). Each company has its own API key hash.

3. Start the Server

npm start

The server will validate your API configuration on startup and report if the API is enabled or disabled.

Skills

Skills are exposed as MCP resources. Available skills:

prompt-improvement/

  • feedback-analysis - Structure user feedback into actionable specs

  • prompt-improvement - Generate improved prompts

  • test-generation - Create regression tests

  • test-analysis - Diagnose test failures

  • improvement-loop - Orchestrate the full improvement cycle

Tools

Tools Management

list_tools

Get all available tools for a company. Use this when generating tests to determine if tool calls should be tested.

Parameters:

  • company_name (string): Company name in kebab-case (e.g., 'technical-life-care', 'tetrix')

Returns: List of tool definitions with IDs, names, descriptions, and parameters.

Example:

list_tools(company_name: "tetrix")
// Returns:
[
  {
    "id": "tool-uuid-123",
    "name": "search_knowledge_base",
    "description": "Search the knowledge base",
    "parameters": [{"name": "query", "type": "string"}]
  }
]

get_tool

Get details of a specific tool by ID.

Parameters:

  • company_name (string): Company name in kebab-case

  • tool_id (string): UUID of the tool

Returns: Tool definition with full details.

Test Management

list_test_cases

Get all test cases from the database for a specific company.

Parameters:

  • company_name (string): Company name in kebab-case

Returns: List of test cases with IDs, names, conversations, and expected outputs.

create_test_case

Create a new test case in the database.

Parameters:

  • name (string): Test case name

  • conversation (array): Array of {role, content} messages

  • expected_output (string): Expected AI response

  • tool_mocks (array, optional): Tool call mocks

  • expected_tool_call (object, optional): Expected tool call

Returns: Created test case with UUID.

run_tests

Run actual LLM evaluation against test cases. Supports two modes: direct prompt text or saved prompt ID (faster).

Parameters (use one of the first two):

  • system_prompt (string, optional): Direct prompt text to test

  • prompt_id (string, optional): UUID of saved draft prompt (from save_draft_prompt) - Preferred for performance

  • test_case_ids (array of strings, optional): Test case UUIDs to run. If omitted, runs all tests.

  • test_model (string, optional): Model to use for testing

  • tools (array, optional): Tool definitions

Returns: Test results with summary (total, passed, failed) and individual results.

Example (with prompt_id - faster):

{
  "prompt_id": "550e8400-e29b-41d4-a716-446655440000",
  "test_case_ids": ["uuid-1", "uuid-2"]
}

Example (with direct text - slower):

{
  "system_prompt": "You are a helpful assistant...",
  "test_case_ids": ["uuid-1", "uuid-2"]
}

Prompt Management

save_draft_prompt

Save a draft prompt to the database for later use in testing. Returns a prompt_id that avoids streaming full prompts during iterations.

Parameters:

  • prompt (string): The full prompt text to save

  • label (string, optional): Label like "iteration-1" or "baseline"

Returns: Saved prompt with UUID and timestamp.

Benefits:

  • Faster performance (no streaming of full prompts)

  • Automatic version history

  • Can re-run tests against old versions

Example:

{
  "prompt": "You are a helpful assistant...",
  "label": "iteration-1"
}
// Returns: { id: "550e8400-...", created_at: "2026-02-02T14:30:00Z" }

get_draft_prompt

Retrieve a previously saved draft prompt by ID.

Parameters:

  • prompt_id (string): UUID of the draft prompt

Returns: Prompt text, label, and metadata.

list_draft_prompts

List all saved draft prompts with their IDs and labels.

Returns: Array of draft prompts.

Improvement Tracking

save_improvement_run

Store a complete improvement cycle with prompts, analysis, and test results.

Parameters:

  • company_name (string): Company name in kebab-case

  • original_prompt (string): Starting system prompt text

  • new_prompt (string): Final improved system prompt text

  • client_feedback (string): User's description of what went wrong

  • analysis (object): Structured feedback analysis

    • what_went_wrong (string): Specific behavior that failed

    • why_it_went_wrong (string): Root cause analysis

    • recommended_fix (string): What changes were made to fix it

  • model_used (string, optional): Model used for testing (e.g., 'google/gemini-2.5-flash')

  • test_results (array): Test execution results with full details

    • test_name (string): Test case name

    • passed (boolean): Whether the test passed

    • is_generated (boolean): true for new tests from feedback, false for existing tests

    • expected (string): What the response should be

    • response (string): What the AI actually responded

    • conversation (array): Full conversation for this test

  • metadata (object, optional): Additional context (iterations, timestamps, etc.)

Returns: Saved improvement run with UUID and timestamp.

Example:

save_improvement_run(
  company_name: "tetrix",
  original_prompt: "You are a helpful assistant...",
  new_prompt: "You are a helpful assistant. Always confirm existing data...",
  client_feedback: "AI keeps re-asking for customer email even when on file",
  analysis: {
    what_went_wrong: "AI re-requests known customer information",
    why_it_went_wrong: "System prompt didn't specify to confirm existing data",
    recommended_fix: "Added explicit instruction to confirm rather than re-request"
  },
  model_used: "google/gemini-2.5-flash",
  test_results: [
    {
      test_name: "Confirm existing email",
      passed: true,
      is_generated: true,
      expected: "AI should confirm existing email",
      response: "I have john@example.com on file — is that current?",
      conversation: [
        { role: "user", content: "Hi, I have a question" },
        { role: "assistant", content: "I have john@example.com on file — is that current?" }
      ]
    }
  ]
)

Skills

list_skills

List all available skills (filesystem-only, no API required).

Usage Flow

Basic Improvement Loop

  1. Provide Claude with: transcript + feedback + current prompt

  2. Claude reads relevant skills

  3. Claude runs the improvement loop:

    • Analyze feedback (feedback-analysis skill)

    • Improve prompt (prompt-improvement skill)

    • Generate tests (test-generation skill → create_test_case tool)

    • Save prompt version (save_draft_prompt tool → returns prompt_id)

    • Run tests (run_tests tool with prompt_id - fast, no streaming)

    • Analyze failures (test-analysis skill)

    • Iterate until pass or stop condition

    • Save run (save_improvement_run tool)

Example Workflow

User: "Here's a transcript where the AI was too verbose. Current prompt: [...]"

Claude:
1. Uses feedback-analysis skill to structure the feedback
2. Uses prompt-improvement skill to generate new prompt
3. Uses test-generation skill to create test cases
4. Calls create_test_case for each test
5. Calls save_draft_prompt(new_prompt, "iteration-1") → gets prompt_id
6. Calls run_tests(prompt_id, test_ids) → fast, no streaming
7. If failures: uses test-analysis skill, improves prompt, repeats from step 5
8. If success: calls save_improvement_run to persist results

Architecture

src/
├── index.js              # Main MCP server
├── config.js             # Configuration management
├── api/
│   ├── client.js         # HTTP client with auth
│   ├── test-cases.js     # Test CRUD operations
│   ├── test-runner.js    # Test execution API
│   ├── prompts.js        # Draft prompt management
│   ├── tools.js          # Tool definitions API
│   └── improvement-runs.js # Improvement tracking
└── tools/
    ├── test-tools.js     # Test management tools
    ├── improvement-tools.js # Improvement tracking tools
    ├── prompt-tools.js   # Draft prompt tools
    ├── tools-management.js # Tool fetching tools
    └── skill-tools.js    # Skill listing tools

skills/
└── prompt-improvement/   # Markdown skills for Claude
    ├── feedback-analysis.md
    ├── prompt-improvement.md
    ├── test-generation.md
    ├── test-analysis.md
    └── improvement-loop.md

Error Handling

The server fails gracefully with clear error messages:

  • Missing API key: "API authentication failed. Set CLERK_CHAT_API_KEY in .env or Claude Desktop config."

  • Network error: "Unable to reach API. Check internet connection."

  • 404 Not Found: "Test case 'abc123' not found. Use list_test_cases to see available tests."

  • 422 Validation: "Invalid test case: 'name' is required."

  • 500 Server Error: "API error. Try again or check API status."

The MCP server never crashes - all errors are returned as tool results to Claude.

Development

Running Without API

The server can run without API configuration for skill-only functionality:

  • Skills will still be available as resources

  • list_skills tool will work

  • API-dependent tools (test management, improvement tracking) will not be registered

Testing API Integration

  1. Configure API key in .env or Claude Desktop config

  2. Restart Claude Desktop (if using config option)

  3. Test each tool:

    list_test_cases → Should return test cases from database
    create_test_case → Should create test with UUID
    run_tests → Should execute with real LLM evaluation
    save_improvement_run → Should persist to database

API Endpoints

The server integrates with these Supabase Edge Functions:

Tool Definitions:

  • GET /api-tools - List all tools for authenticated company

  • GET /api-tools?id=uuid - Get specific tool by ID

Test Execution:

  • POST /api-run-tests - Execute tests with LLM evaluation (supports prompt_id or system_prompt)

Test Cases:

  • GET /api-test-cases - List all test cases

  • POST /api-test-cases - Create new test case

  • PUT /api-test-cases?id=uuid - Update test case

  • DELETE /api-test-cases?id=uuid - Delete test case

Draft Prompts:

  • POST /api-prompts - Save draft prompt (returns prompt_id)

  • GET /api-prompts/:id - Get specific draft prompt

  • GET /api-prompts - List all draft prompts

Improvement Tracking:

  • POST /api-improvement-runs - Save improvement run

  • GET /api-improvement-runs - List improvement runs

Security

  • API keys are never logged

  • .env is gitignored

  • All credentials use environment variables

  • Input validation with Zod schemas

  • Sanitized error messages

License

MIT

Available Tools

10 tools
create_test_caseB

Create a new test case in the database for a specific company

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesTest case name
tool_mocksNoOptional tool call mocks for testing
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care')
conversationYesFull conversation context for the test
expected_outputYesExpected AI response
expected_tool_callNoOptional expected tool call

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden. It states 'Create ... in the database' which implies persistence, but it does not disclose whether the operation is idempotent, whether the company must already exist, what happens on duplicates, or any error behavior. Minimal behavioral detail beyond the obvious create action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It efficiently communicates the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (6 parameters, nested objects, 4 required), the description is too sparse. It does not explain when to use this tool, what it returns, or any constraints beyond 'for a specific company'. The schema covers parameters, but the description fails to provide contextual completeness for a moderately complex tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema descriptions cover 100% of the parameters, so the baseline is 3. The description adds nothing beyond schema: 'for a specific company' merely echoes the existing company_name parameter's schema description. No additional semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Create') and identifies the resource ('test case') and context ('for a specific company'). It clearly distinguishes this creation tool from sibling tools like run_tests and list_test_cases, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no explicit guidance on when to use this tool versus alternatives. It merely states the action, leaving the agent to infer usage from the existence of sibling tools. There are no exclusions or alternative references.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_draft_promptA

Retrieve a previously saved draft prompt by ID for a specific company

ParametersJSON Schema
NameRequiredDescriptionDefault
prompt_idYesThe UUID of the draft prompt to retrieve
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care')

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the burden of disclosing behavioral traits. It correctly implies a read-only action ('retrieve'), but it does not mention what happens if the prompt is not found, whether it returns null or errors, or any other side-effect-free confirmation. The description adds minimal behavioral context beyond the obvious read operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that is front-loaded with the verb and precisely states the action and scope. Every word contributes to meaning with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple getter tool with two well-documented parameters. The description fully covers the purpose and scope, but it does not mention the return format or error behavior, which would normally be expected given the absence of an output schema. Still, for the tool's simplicity, the description is reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description reinforces that prompt_id is the identifier and company_name is the scoping company, but it does not add new details beyond the schema, such as format or validation rules. It adequately aligns with the schema but does not enhance it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('retrieve') and clearly identifies the resource ('previously saved draft prompt') and the scoping ('by ID for a specific company'). This strongly distinguishes it from sibling tools such as list_draft_prompts (listing) and save_draft_prompt (saving).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The context is clear: use this when you need a specific saved draft prompt by ID for a company. It does not explicitly name alternatives or say when not to use, but the retrieval semantics are unambiguous. A score of 4 reflects the lack of explicit alternative guidance while acknowledging the clarity of the intended use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_toolA

Get details of a specific tool by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
tool_idYesUUID of the tool to retrieve
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care', 'tetrix')

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It indicates a read-only operation ('Get') but does not disclose the role of company_name (likely scoping), error behavior, or return format, leaving notable gaps for a simple get tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no unnecessary words. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, but the description does not explain why company_name is required or what 'details' entails. With no output schema, the agent lacks clarity on return values. It is a minimum viable description but leaves some contextual gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions cover 100% of parameters, so the baseline is 3. The description only echoes 'by ID', aligning with tool_id, and provides no extra meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('details of a specific tool') with a specific identifier ('by ID'), effectively distinguishing it from list_tools which lists all tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when a specific tool ID is known, but it does not explicitly mention alternatives or when not to use this tool. It provides clear context but lacks explicit exclusions or comparisons to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_draft_promptsB

List all saved draft prompts for a specific company

ParametersJSON Schema
NameRequiredDescriptionDefault
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care')

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavior. It implies a read-only list operation but does not mention return format, pagination, or any limitations beyond company scope. The phrase 'all saved draft prompts' adds some context but largely just restates the tool's name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, eight-word sentence that immediately states the action and resource. There is no redundancy or extraneous detail, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one well-documented parameter, the description conveys the core purpose adequately. However, without an output schema or annotations, the agent is left without details on response structure or behavior, and the description does not address potential limitations or alternatives.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides a thorough description of company_name, including format and example, and schema coverage is 100%. The tool description's mention of 'a specific company' aligns with the parameter but adds no new semantic value, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (List), the resource (saved draft prompts), and the scope (for a specific company). This distinguishes it from sibling tools like list_test_cases and get_draft_prompt, which target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention that get_draft_prompt is for retrieving a single prompt or that save_draft_prompt is for creating them, leaving the agent without clear selection criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_skillsA

List all available skills

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It only states that it lists skills, implying a read-only operation, but does not mention side effects, permissions, return format, or limitations. This is minimal disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no unnecessary words. It efficiently conveys the tool's purpose without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless listing tool, the description covers the core functionality. However, given sibling tools like list_tools and the absence of an output schema or annotations, a brief note on how skills differ from tools or what the return data looks like would improve completeness. Still, the low complexity makes this largely sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the input schema fully covers parameter semantics (100% coverage). The description adds no parameter details, but with no parameters, this is not a gap. Baseline of 4 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List all available skills' uses a specific verb ('list') and resource ('skills'), clearly stating the action and scope. It distinguishes from sibling tools like list_tools and get_tool by targeting skills rather than tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage (when you need to enumerate skills) but does not explicitly state when to use this tool versus alternatives like list_tools, nor any exclusions. It provides no guidance on selecting between related tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_test_casesA

List all test cases in the database for a specific company

ParametersJSON Schema
NameRequiredDescriptionDefault
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care')

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the transparency burden. It indicates a read-only operation via 'List' but does not disclose return format, pagination, ordering, or permission requirements. It provides basic transparency but lacks deeper behavioral details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence of 12 words with no redundant information. It is front-loaded and every word contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one well-documented parameter and no output schema, the description sufficiently conveys the purpose and scope. It covers the essential context, though it could optionally mention the return structure or any limits, but this is not critical for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already fully describes the only parameter 'company_name' with a clear description and example. The tool description adds nothing beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List'), the target resource ('test cases'), and the scope ('for a specific company'). It distinguishes this from sibling tools like create_test_case (creating) and run_tests (running), making the purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is used to retrieve test cases for a particular company, but it does not explicitly state when to use it instead of other list tools (e.g., list_skills, list_draft_prompts) or mention any exclusions. Usage context is present but not fully explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_toolsA

List all available tools for a company. Use this when generating tests to know what tool calls are possible.

ParametersJSON Schema
NameRequiredDescriptionDefault
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care', 'tetrix')

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It says 'List all available tools' but gives no details about response format, security, pagination, or what 'available' means. This is minimal and not much beyond the tool name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is just two short sentences, front-loaded with the core purpose. No wasted words—every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With only one parameter, no annotations, and no output schema, the description covers the basic purpose and a use case. However, it does not specify what the response contains (e.g., tool names, schemas), leaving some ambiguity for the agent. It is adequate but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the sole parameter company_name, including a clear example. The description adds no extra parameter information, but the schema already handles it fully, so a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists all available tools for a company, providing a specific verb, resource, and scope. It distinguishes from sibling tools like list_test_cases and list_skills by focusing on the 'tools' resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear usage context: 'Use this when generating tests to know what tool calls are possible.' However, it does not mention alternatives or when not to use it, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_testsA

Run regression tests against a system prompt with real LLM evaluation for a specific company. Use prompt_id (from save_draft_prompt) for faster performance, or system_prompt for direct text.

ParametersJSON Schema
NameRequiredDescriptionDefault
toolsNoOptional tool definitions for the test
prompt_idNoUUID of a saved draft prompt (from save_draft_prompt). Preferred for performance. Use this OR system_prompt, not both.
test_modelNoOptional model to use for testing (e.g., 'gpt-4')
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care')
system_promptNoThe system prompt text to test. Use this OR prompt_id, not both.
test_case_idsNoOptional array of test case UUIDs to run. If omitted, runs all tests.

TDQS

A3.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'real LLM evaluation' implying cost/latency, but does not disclose side effects (e.g., whether results are saved, if a run record is created), rate limits, or the response/output format. For a tool that executes tests, this is a significant transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences. The first sentence front-loads the core purpose ('Run regression tests against a system prompt with real LLM evaluation'), and the second covers the crucial parameter decision. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is adequate for a 6-parameter tool with full schema descriptions, but lacks information about return values or output format (no output schema), and does not explain the role of other parameters like tools, test_model, or test_case_ids beyond the schema. Given the tool's potential complexity (running LLM evaluations), the description should mention what the caller receives or what side effects occur.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantic guidance by explaining the relationship between prompt_id and system_prompt, noting that prompt_id is 'from save_draft_prompt' and preferred for 'faster performance'. This goes beyond the schema's simple 'use one or the other' instruction.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description immediately states the specific action 'Run regression tests' against a 'system prompt' with 'real LLM evaluation' for a specific company. This clearly distinguishes it from sibling tools like create_test_case or list_test_cases, which create or list rather than execute tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance on parameter usage: 'Use prompt_id (from save_draft_prompt) for faster performance, or system_prompt for direct text.' This is practical guidance for when to use each option. However, it does not explicitly exclude alternatives or contrast with other tools, though no sibling tool performs the same test-running function.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

save_draft_promptA

Save a draft prompt to the database for a specific company for later use in testing. Returns a prompt_id that can be used with run_tests.

ParametersJSON Schema
NameRequiredDescriptionDefault
labelNoOptional label like 'iteration-1' or 'baseline'
promptYesThe full prompt text to save
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care')

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden. It states that the tool saves to a database and returns a prompt_id, but does not disclose potential side effects such as overwriting existing drafts, required permissions, or idempotency behavior. This is a moderate disclosure level for a write operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences deliver the purpose, target, return value, and downstream usage with no redundant words. The structure is front-loaded with the action and resource.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no output schema, so the description appropriately mentions the return value (prompt_id) and how it connects to run_tests. Given its simplicity and the schema's completeness, the description adequately covers the tool's role and expected outcome, though it does not elaborate on handling of optional labels or edge cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of the parameters with descriptions, so the description adds little beyond mentioning the return value. It does not describe parameter formats or constraints beyond what the schema already provides. This matches the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb ('Save') and identifies the exact resource ('draft prompt') plus the target context ('for a specific company'). It also distinguishes this tool from siblings such as get_draft_prompt, list_draft_prompts, and run_tests by explicitly stating its role in saving for later testing and returning a prompt_id for run_tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool: to save a draft prompt for later testing, and it connects the returned prompt_id to run_tests. It doesn't explicitly name alternatives or exclusions, but the intended use is unambiguous given the sibling list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

save_improvement_runA

Save a complete improvement cycle with prompts, analysis, and test results for a specific company

ParametersJSON Schema
NameRequiredDescriptionDefault
analysisYesStructured feedback analysis
metadataNoAdditional context (iterations, timestamps, etc.)
model_usedNoModel used for testing (e.g., 'google/gemini-2.5-flash')
new_promptYesFinal improved system prompt text
company_nameYesCompany name in kebab-case (e.g., 'technical-life-care')
test_resultsYesTest execution results with full details
client_feedbackYesUser's description of what went wrong
original_promptYesStarting system prompt text

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Save') but does not reveal whether it creates a new record or overwrites an existing one, whether the company must already exist, or any other side effects. This is a significant transparency gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that immediately states the action and the object. It contains no filler or redundant information, and every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has a complex input schema (nested objects, many required fields), but the schema itself is comprehensive and fully documents the structure. The description supplies the overall purpose while the schema covers the parameters. The main gap is the lack of any mention of return value or post-save behavior, but for a save operation the purpose is clear enough to be considered mostly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with detailed descriptions for every parameter, including examples (e.g., company_name in kebab-case). The description adds only a high-level grouping of parameters ('prompts, analysis, and test results') without providing syntax or format details beyond what the schema already specifies, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Save') and resource ('complete improvement cycle'), and clarifies the contents ('prompts, analysis, and test results') and scope ('for a specific company'). This clearly distinguishes it from sibling tools like save_draft_prompt, which only saves a single prompt.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The context is implied: it is used after an improvement cycle has been completed. However, there is no explicit guidance on when to prefer this over related tools (e.g., save_draft_prompt) or when not to use it, so it lacks the exclusionary detail that would merit a higher score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 10 tool updatesv0.1.0
    • First observedcreate_test_case
    • First observedget_draft_prompt
    • First observedget_tool
    • First observedlist_draft_prompts
    • First observedlist_skills
    • First observedlist_test_cases
    • First observedlist_tools
    • First observedrun_tests
    • First observedsave_draft_prompt
    • First observedsave_improvement_run

TDQS

A3.8/5.0
Disambiguation5/5

Each tool targets a distinct resource and action: test cases, draft prompts, skills, tools, and improvement runs. There is no overlap between create/list/get functions for different entities.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in snake_case (e.g., create_test_case, list_draft_prompts, get_tool). The naming convention is uniform and predictable.

Tool Count5/5

The server has 10 tools, which falls well within the ideal 3-15 range. Each tool serves a clear purpose in the test management and prompt workflow, making the set well-scoped.

Completeness3/5

The core workflow of creating test cases, running tests, and saving results is covered, but there are gaps: no get/update/delete for test cases, and no update/delete for draft prompts. This limits full lifecycle management.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/williamjonathanbowen/clerk-chat-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server