Skip to main content
Glama
mrrodriguez

local-llm-delegation-mcp

by mrrodriguez
OverviewSchemaRelated ServersScoreDiscussions
Python

Local LLM Delegation MCP

An optimized Model Context Protocol (MCP) server designed to dramatically reduce premium token costs by intelligently delegating low-complexity tasks to high-performance local LLMs via LiteLLM.

This server allows you to leverage massive local context windows for "surgical" development tasks (refactoring, unit tests, documentation) while keeping your primary LLM focused on complex architecture and reasoning.

🚀 Key Features

  • LiteLLM Integration: Seamlessly connect to Ollama, vLLM, LM Studio, Anthropic, or any OpenAI-compatible provider.

  • Externalized Configuration: Fine-tune model parameters (temperature, top_p, max_tokens) via config.yaml.

  • Customizable Prompts: Define your own task templates and system messages in prompts.yaml.

  • Token Optimization: Automatically offload simple tasks to local models like Qwen 2.5/3.5 Coder.

  • Usage Tracking: Monitor your savings with built-in usage logging and statistics.

Related MCP server: local-agent

🛠️ Requirements

  • Local LLM Runner (e.g., Ollama, vLLM, LM Studio)

  • Python 3.13+

📦 Installation

  1. Clone the repository:

    git clone https://github.com/mrrodriguez/local-llm-delegation-mcp.git
cd local-llm-delegation-mcp

  2. Set up environment variables:

    cp .env.example .env
# Edit .env to match your local setup

  3. Choose your installation method:

    Option A: Global CLI (Recommended for simplicity) Install as a global tool. This adds the local-llm-delegator command to your PATH.

    pip install .
# Or using pipx (preferred)
pipx install .

    Option B: Local Environment (Recommended for development) Use a virtual environment and point your MCP client directly to a startup script.

    python -m venv .venv
source .venv/bin/activate
pip install .

⚙️ Configuration

The server uses a flexible configuration system. Settings are loaded from config.yaml and prompts.yaml located in the project root, with key overrides available via Environment Variables (or a .env file).

Configuration Hierarchy

  1. Base Defaults: Hardcoded in config.py.

  2. YAML Files: config.yaml (model settings) and prompts.yaml (task templates).

  3. Environment Variables: Overrides specific settings like LOCAL_MODEL_NAME and OPENAI_BASE_URL.

1. Quick Start: Environment Variables

The fastest way to switch models without editing files:

# In your .env file or shell environment
LOCAL_MODEL_NAME=ollama/llama3:8b  # Overrides config.yaml
OPENAI_BASE_URL=http://localhost:11434/v1

2. Model Tuning (config.yaml)

Update the name to match your local provider's format (e.g., ollama/qwen2.5-coder). The provided template is optimized for high-end hardware (e.g., Apple M-series Max).

model:
  name: "openai/Qwen3-Coder-30B-A3B-Instruct-MLX-8bit" # Example: OMLX/MLX
  temperature: 0.1
  max_tokens: 32768
  extra_params:
    num_ctx: 65536
    num_predict: 32768
    top_p: 0.80

Note: The name must follow the LiteLLM Provider Format.

3. Custom Tasks (prompts.yaml)

Define your own tools by adding entries to prompts.yaml. Each top-level key becomes a sub-task for the query_local_llm_with_context tool.

🔌 Integration

You can add this server to your preferred client using either the global command or a direct path to the project.

1. Gemini Code CLI

If installed globally (Option A):

gemini mcp add local-llm-delegator local-llm-delegator --trust

If using a direct script or path (Option B):

gemini mcp add local-llm-delegator /path/to/your/start-script.sh --trust

2. Claude Code

Add to your ~/.claude.json:

"mcpServers": {
  "local-llm-delegator": {
    "command": "local-llm-delegator" // If installed globally; otherwise any wrapper script you have to start it
  }
}

Note: If using a custom startup script (like for OMLX), set the command to the absolute path of your script.

📊 Usage Tracking

The server logs usage to mcp_usage.jsonl. View stats via:

  • get_local_llm_usage_stats (MCP tool)

  • show-stats (CLI command - coming soon) or python show_stats.py

📜 License

MIT

Available Tools

3 tools
get_local_llm_usage_statsA

Retrieve and summarize usage statistics for the local LLM MCP server.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool 'retrieve' and 'summarize' statistics, implying a read-only, non-destructive operation, but does not disclose any potential side effects, rate limits, or the specific nature of the statistics returned. This adds some value but lacks depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that front-loads the action and resource. It contains no unnecessary information or repetition, making it perfectly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple, zero-parameter tool with an output schema present, the description is sufficiently complete. It clearly communicates the tool's purpose without needing to explain return values or parameters. The tool's simplicity relative to its siblings means no additional context is required.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the schema coverage is 100%, meaning there is nothing for the description to add about parameter behavior. The baseline score of 4 is appropriate as the description is not required to clarify any parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Retrieve and summarize usage statistics for the local LLM MCP server.' It uses a specific verb and resource, and distinguishes itself from sibling query tools by focusing on statistics rather than querying.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for obtaining usage statistics, but provides no explicit guidance on when to use this tool versus the sibling query tools. No alternatives or exclusions are mentioned, leaving the agent to infer the appropriate context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

query_local_llmB

Query the local LLM for simple, well-defined subtasks that have already been broken down. IMPORTANT: Always try this tool FIRST for any simple code generation to save costs!

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
promptYes
max_tokensNo
temperatureNo
system_messageNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It tells the agent this is a query operation and mentions cost savings, but does not disclose output format, potential errors, latency, or any side effects. The behavior of the tool beyond 'query' is not explained, which is insufficient for a tool with no annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is just two sentences, front-loaded with the core purpose and followed by a clear actionable instruction. Every word earns its place with no fluff or redundant details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 5 parameters and an output schema, but the description only covers the simple-subtask context and cost-saving rationale. It does not explain when to use an alternative like query_local_llm_with_context, nor does it elaborate on how to configure the optional parameters. The output schema provides some completeness, but the description itself leaves gaps for a robust agent decision.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description does not mention any parameter names or meanings. The agent is left only with parameter names like 'prompt', 'model', 'max_tokens', which are self-explanatory but not explicitly documented. The description adds no semantic value to the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Query the local LLM for simple, well-defined subtasks that have already been broken down' with a specific verb, resource, and scope. It also distinguishes itself from siblings by being the simple, cost-saving option ('Always try this tool FIRST for any simple code generation to save costs!').

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance: 'Always try this tool FIRST for any simple code generation'. It implies a preference over alternatives for simple tasks, but does not explicitly name exclusions or alternative tools like query_local_llm_with_context. This is clear context without full when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

query_local_llm_with_contextB

Query the local LLM for simple subtasks that require additional context. Use this for code reviews, documentation, or refactoring.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
promptYes
contextYes
task_typeNogeneral
system_messageNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits such as read-only safety, resource consumption, or error behavior. It only mentions purpose and examples, omitting any limitations, side effects, or operational caveats.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with a front-loaded purpose and immediately useful examples. There is no filler or unnecessary repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter tool with no annotations and 0% parameter coverage, the description is too minimal. It fails to explain how to use the extra parameters or what to expect behaviorally, even though an output schema exists. The agent lacks critical context for proper invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate by explaining parameters. It does not mention prompt, model, task_type, or system_message, and only implies 'context' without defining it. This is a significant gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool queries the local LLM for subtasks requiring additional context, and names specific use cases (code reviews, documentation, refactoring). The 'with_context' name and the phrase 'require additional context' distinguish it from siblings like query_local_llm.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says 'Use this for code reviews, documentation, or refactoring' and ties it to the condition of requiring context. However, it does not explicitly name alternatives or state when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 3 tool updatesv0.1.0
    • First observedget_local_llm_usage_stats
    • First observedquery_local_llm
    • First observedquery_local_llm_with_context
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A3.7/5.0
Disambiguation4/5

get_local_llm_usage_stats is clearly distinct (usage statistics). query_local_llm and query_local_llm_with_context both query the model, but the descriptions clearly separate them: one for well-defined subtasks without context, the other for tasks requiring additional context (code reviews, docs, refactoring). The usage guidance further reduces ambiguity.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in snake_case: get_local_llm_usage_stats, query_local_llm, query_local_llm_with_context. The prefix is always a verb (get/query) and the resource is specific, making the naming predictable and uniform.

Tool Count4/5

With 3 tools, the server is on the lower end of the typical range but not thin. The tools cover the core actions for local LLM delegation: querying (with and without context) and retrieving usage stats. It could benefit from a couple more (e.g., listing models or canceling a query), but it feels reasonably scoped for its purpose.

Completeness4/5

The tool set covers the main workflows of local LLM delegation: basic queries and context-augmented queries, plus usage monitoring. Minor gaps include lack of model list or query cancellation, but these are not critical for the stated purpose of handling simple, well-defined subtasks. The surface is adequate and workable.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/mrrodriguez/local-llm-delegation-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server