Skip to main content
Glama
enached134-ctrl

Agentic RAG MCP

by enached134-ctrl
OverviewSchemaRelated ServersScoreDiscussions
Python
Hybrid

Agentic RAG MCP

CI Eval dashboard License: MIT Python 3.10+ MCP

A multi-agent Retrieval-Augmented Generation system exposed as an MCP server. Ask a question and a LangGraph pipeline plans the retrieval, pulls evidence from a pgvector knowledge base, optionally augments it with live web research, drafts a cited answer, and then self-critiques it for grounding — revising until the answer is supported by the sources.

It plugs into any MCP client (Claude Code/Desktop, Cursor, Windsurf, …) as three tools: ingest, ask, and search.

Why this design? A bare RAG endpoint is easy to copy; a multi-agent system that verifies its own answers and ships as an MCP server is not. The architecture is the moat — "easy to buy, hard to replicate."

Architecture

flowchart LR
    Q([Question]) --> P[🧭 Planner<br/>plan + search queries]
    P --> R[📚 Retriever<br/>pgvector top-k]
    R --> W[🌐 Web Researcher<br/>Firecrawl • optional]
    W --> S[✍️ Synthesizer<br/>cited answer]
    S --> C{🔎 Critic<br/>grounded?}
    C -- needs revision --> S
    C -- grounded --> A([Answer + citations])

    subgraph Stores
      DB[(Supabase<br/>pgvector)]
    end
    R <-->|cosine search| DB

    classDef agent fill:#1e293b,stroke:#7C3AED,color:#e2e8f0;
    class P,R,W,S,C agent;

Agent

Model / tool

Responsibility

Planner

Claude (claude-opus-4-8, adaptive thinking)

Decompose the question into focused search queries

Retriever

Voyage embeddings + pgvector

Cosine top-k over the knowledge base

Web Researcher

Firecrawl (optional)

Augment with live web results when a key is set

Synthesizer

Claude

Draft an answer grounded in context, with [n] citations

Critic

Claude

Verify grounding; loop back for revision if unsupported

Related MCP server: mcp-langchain-agent

MCP tools

Tool

Arguments

Returns

ingest

url: str

Scrapes the URL, chunks + embeds it, stores it. { url, chunks_added }

ask

question: str

Runs the full pipeline. { answer, citations, plan, grounded }

search

query: str, k: int = 5

Retrieval only — top-k chunks with similarity scores

Quickstart

# 1. Install (Python 3.10+)
uv venv && uv pip install -e ".[dev]"     # or: pip install -e ".[dev]"

# 2. Configure
cp .env.example .env                       # fill in ANTHROPIC_API_KEY, VOYAGE_API_KEY, DATABASE_URL

# 3. Create the vector table (Supabase SQL editor or psql)
psql "$DATABASE_URL" -f sql/schema.sql

# 4. Run the MCP server (stdio by default)
agentic-rag-mcp

Connect it to Claude Code

claude mcp add agentic-rag -s user \
  --env ANTHROPIC_API_KEY=sk-ant-... \
  --env VOYAGE_API_KEY=pa-... \
  --env DATABASE_URL=postgresql://... \
  -- agentic-rag-mcp

Then, from the client: "ingest https://example.com/docs" → "ask: how do I configure X?".

How it works

  1. Plan — Claude turns the question into a short plan + 1–5 search queries.

  2. Retrieve — each query is embedded (Voyage voyage-3.5) and matched against pgvector by cosine distance; results are de-duplicated and ranked.

  3. Research — if FIRECRAWL_API_KEY is set, live web results are added to the context.

  4. Synthesize — Claude writes an answer grounded only in the numbered context, citing each claim as [n].

  5. Critique — a strict fact-checker pass decides whether the answer is fully supported. If not (and revisions remain), it loops back to the synthesizer with feedback.

Configurable via env: RAG_MODEL, RAG_TOP_K, RAG_MAX_REVISIONS, RAG_EMBED_MODEL.

Evaluation — the CI gate

📊 Live eval dashboard: enached134-ctrl.github.io/agentic-rag-mcp — the golden dataset and the latest green run (20/20 passing), in one screen.

Answer quality is measured with promptfoo on a golden dataset (evals/golden.yaml — 20 seed cases: answerable / refusal / adversarial, written against a committed corpus) and enforced in CI on every push: the evals job spins up a pgvector service container, seeds it with evals/corpus/, and runs every case through the real pipeline — planner, retriever, synthesizer, self-critique. Nothing is mocked. A regression fails the build before it can reach a user.

Scored dimensions: citation presence (deterministic) · groundedness (LLM-as-judge) · refusal correctness (LLM-as-judge) · latency (threshold).

python evals/seed.py --schema --reset   # seed the corpus into your vector store
make eval                               # run the suite locally

See evals/ for the corpus, the golden dataset, and the regression-capture rule: every real-world failure becomes a new golden case, so no bug gets fixed twice.

The CI gate activates when the ANTHROPIC_API_KEY and VOYAGE_API_KEY repository secrets are configured; without them (e.g. on forks) the job skips with a visible notice.

Observability

Opt-in OpenTelemetry tracing to Arize Phoenix:

pip install -e ".[trace]"
phoenix serve                          # local Phoenix UI on :6006
PHOENIX_ENABLED=1 agentic-rag-mcp

Every ask run appears as a full trace — LangGraph node spans (plan → retrieve → research → synthesize → critique) plus every Claude call with token usage and latency per span. Point PHOENIX_COLLECTOR_ENDPOINT at a hosted collector to ship traces off-box.

Deploy

Containerised and ready for Railway (HTTP transport):

railway up        # uses Dockerfile + railway.json; set RAG_TRANSPORT=http

Expose RAG_HTTP_PORT and connect over --transport http. A cloudflared tunnel works for local demos.

Kubernetes

Production-shaped manifests — readiness/liveness probes, resource limits, secret-driven env — live in deploy/k8s/, including a kind-based local smoke test walkthrough.

Project layout

src/agentic_rag_mcp/
  config.py      # env-driven settings
  llm.py         # Anthropic (Claude) helper — adaptive thinking, JSON parsing
  embeddings.py  # Voyage embeddings
  store.py       # pgvector store (psycopg)
  web.py         # Firecrawl web research (optional)
  ingest.py      # chunking + ingestion
  state.py       # LangGraph state
  nodes.py       # planner / retriever / researcher / synthesizer / critic
  graph.py       # graph assembly
  tracing.py     # opt-in OpenTelemetry → Arize Phoenix
  server.py      # FastMCP server (ingest / ask / search)
sql/schema.sql   # pgvector schema
evals/           # golden dataset + corpus + promptfoo suite (runs in CI)
deploy/k8s/      # Kubernetes manifests + kind smoke test

License

MIT — see LICENSE.

Available Tools

3 tools
askA

Answer a QUESTION with a written, source-cited answer (the full multi-agent RAG pipeline: plan → retrieve → synthesize → self-critique). Use this when the user wants an ANSWER. For raw matching documents instead of a written answer, use search.

ParametersJSON Schema
NameRequiredDescriptionDefault
questionYesA natural-language question to answer from the knowledge base.

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fully discloses behavior: it describes the multi-agent RAG pipeline (plan, retrieve, synthesize, self-critique) and notes source-citation. This goes beyond a simple claim of answering questions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core purpose, and contains no redundant or unnecessary information. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the one-parameter schema, output schema existence, and clear sibling differentiation, the description provides all necessary context. It covers behavior, usage constraints, and alternatives comprehensively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and already describes the parameter as 'A natural-language question to answer from the knowledge base.' The tool description adds minimal extra semantic meaning, so baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool answers a question with a written, source-cited answer, distinguishing it from the sibling tool 'search'. The specific verb 'answer' and resource 'question' are well defined.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance is given: use when the user wants an answer, and for raw matching documents use 'search'. This provides clear context and an alternative, satisfying high standards.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ingestA

Add a web page to the knowledge base: scrape the URL, chunk and embed its text, and store it so future ask/search calls can use it. Use this to teach the system new source material before querying it.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe web page URL to scrape, chunk, embed, and add to the knowledge base.

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It discloses the main side effects: will scrape, chunk, embed, and store. However, it does not mention potential failure modes, authentication requirements, or whether the operation is idempotent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action, no unnecessary detail. Every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter) and the presence of an output schema, the description fully covers the purpose, use case, and process. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a clear schema description for the sole parameter 'url'. The description adds context about the overall process but does not add parameter-specific semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the action ('Add a web page to the knowledge base') and the specific steps (scrape, chunk, embed, store). It distinguishes itself from siblings 'ask' and 'search' by mentioning that the added content enables future queries.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description explicitly says 'Use this to teach the system new source material before querying it,' providing clear when-to-use context. It does not explicitly mention when not to use or name alternatives, but the sibling tools are implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 3 tool updatesv1.0.0
    • Changedask1 field changed
      • addedInput schema / properties / question / description
        Added value: +"A natural-language question to answer from the knowledge base."
    • Changedingest1 field changed
      • addedInput schema / properties / url / description
        Added value: +"The web page URL to scrape, chunk, embed, and add to the knowledge base."
    • Changedsearch2 fields changed
      • addedInput schema / properties / k / description
        Added value: +"How many top matching chunks to return."
      • addedInput schema / properties / query / description
        Added value: +"The search query to match against stored document chunks."
  2. 3 tool updatesv0.1.0
    • First observedask
    • First observedingest
    • First observedsearch
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.6/5.0
Disambiguation5/5

Each tool has a distinct purpose: 'ask' provides synthesized answers, 'search' returns raw chunks, and 'ingest' adds content. There is no overlap or ambiguity between them.

Naming Consistency5/5

All tool names are single-word verbs in lowercase, following a consistent and predictable pattern. No mixing of conventions.

Tool Count5/5

Three tools is ideal for this RAG server: one for ingestion, one for answering, and one for raw retrieval. The scope is well-defined without unnecessary tools.

Completeness4/5

The tool set covers the core RAG workflow (ingest, ask, search). A potential minor gap is the lack of a delete or update tool, but for the stated purpose it is highly complete.

Maintenance

ActivityStale
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/enached134-ctrl/agentic-rag-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server