knowledge-rag
The knowledge-rag server is a 100% local hybrid search and document management system that integrates with Claude Code via MCP, enabling you to search, manage, and evaluate a personal knowledge base with zero external dependencies.
Search & Retrieval
search_knowledge— Hybrid search combining semantic embeddings (FastEmbed ONNX) and BM25 keyword matching via Reciprocal Rank Fusion, with cross-encoder reranking; tune the balance withhybrid_alpha(0.0 = keyword only, 1.0 = semantic only); filter by category; auto-expands 54 security-term synonyms (e.g., "sqli" → "sql injection")search_similar— Find documents semantically similar to a reference documentget_document— Retrieve full content and metadata for a specific documentResults are diversified via MMR to reduce redundancy; repeat queries benefit from an LRU cache (5-min TTL)
Document Management
add_document— Add a new document from raw text contentupdate_document— Replace and re-index an existing documentremove_document— Remove from index (optionally delete from disk)add_from_url— Fetch a URL, strip HTML, convert to Markdown, and index itSupports 9 formats: Markdown, PDF, DOCX, XLSX, PPTX, CSV, TXT, Python, JSON
Smart chunking: Markdown files split by section headers (##/###); SHA256 deduplication prevents duplicate chunks
Index Management
reindex_documents— Incremental (changed files only), forced smart reindex, or full nuclear rebuild (for model upgrades); auto-reindex via file watcher (5-second debounce)get_index_stats— View total documents, chunks, embedding model, cache hit rate, etc.Auto-detects embedding dimension mismatches and triggers rebuilds when upgrading versions
Organization & Evaluation
list_categories— List all document categories with counts (security, development, ctf, logscale, general, redteam, blueteam, aar, etc.)list_documents— List all indexed documents, optionally filtered by categoryevaluate_retrieval— Benchmark retrieval quality with custom test cases; returns MRR@5 and Recall@5 metrics
All processing is fully local — no API keys, no Ollama, no data leaves your machine.
knowledge-rag
The MCP-first local RAG server for Claude Code, Cursor, and every AI agent.
Hybrid search · Cross-encoder reranking · 35 file formats · 100% local · Zero cloud · Enterprise-grade plumbing built-in.
pip install knowledge-rag → restart Claude Code → search_knowledge("your query")Quick Start · Why knowledge-rag · Compare · Enterprise Features · Docs
⭐ Star History
Chart updated daily by GitHub Action
Related MCP server: nexus-mcp-ci
🎯 Why knowledge-rag
Most RAG frameworks fall into one of three traps: (1) they require you to ship your data to a cloud API, (2) they hand you 300 building blocks and 0 opinionated defaults, or (3) they bundle RAG as a 5% feature of a much bigger platform you didn't ask for.
knowledge-rag does one thing well: it is the MCP-native local RAG server that Claude Code, Cursor, Windsurf, VS Code, Cline, Gemini CLI and Zed can search out of the box — with enterprise plumbing (bearer auth, Prometheus metrics, rate limiting, health probes, structured JSON logging, zero-downtime reindex) that no other RAG-focused OSS ships built-in.
🔒 100% local, 0% cloud
Your files never leave the machine. No vendor lock-in, no data-residency headache, no forced cloud dependency. LGPD / GDPR / HIPAA compliant by architecture — because there is nothing to comply about when nothing leaves.
🚀 Zero-friction setup
pip install knowledge-rag → restart your MCP client → done. No Docker mandatory. No Ollama required. No separate embedding server. Everything runs in-process via FastEmbed ONNX. Works offline after the first model download.
🛡️ Production-grade OSS
7-pillar quality gate on every PR (35+ automated checks), 9-cell OS×Python CI matrix (Linux + Windows + macOS × 3.11/3.12/3.13), nightly chaos + 50K-iteration soak + mutation testing. 700+ tests. 0 known regressions.
💰 Zero ongoing cost
No token bills. No SaaS tier. No paid features hidden behind a wall. MIT license, forever. Runs on the laptop you already have — GPU optional, CPU works fine with FastEmbed ONNX.
📊 How knowledge-rag compares to other RAG frameworks
We audited 16 popular RAG frameworks and platforms (LlamaIndex, LangChain, ChromaDB, Weaviate, Qdrant, RAGFlow, LightRAG, DSPy, GraphRAG, Haystack, RAG-Anything, kotaemon, txtai, llmware, Dify, open-webui, FastGPT) so you can pick honestly.
Legend: ✅ built-in · 🟡 plugin / paid tier / partial · ❌ not available · ⚠️ license or default concern
Dimension | 🎯 knowledge-rag | LlamaIndex | LangChain | Haystack | RAGFlow | txtai | open-webui | Dify | Qdrant |
100% local, zero cloud | ✅ | 🟡 | ✅ | 🟡 | 🟡 | ✅ | ✅ | 🟡 | 🟡 |
MCP native (Claude/Cursor) | ✅ 13 tools | 🟡 pkg | 🟡 adapter | 🟡 wrapper | 🟡 add-on | ✅ | ✅ consumer | ✅ | ❌ |
Hybrid BM25 + semantic | ✅ 128× faster | 🟡 | 🟡 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Cross-encoder rerank | ✅ builtin | ❌ | 🟡 | ✅ | ✅ fused | ❌ | ✅ | 🟡 | 🟡 |
Bearer auth builtin | ✅ | ❌ | ❌ | ❌ core | ❌ | 🟡 | ✅ RBAC | ✅ OAuth2 | ✅ |
Prometheus | ✅ | ❌ | ❌ | ❌ core | ❌ | ❌ | ✅ OTel | ❌ | ✅ |
Rate limiting | ✅ sliding-window | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |
Health probes ( | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | 🟡 | 🟡 | ✅ |
Structured JSON logging | ✅ opt-in | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ OTel | 🟡 | ✅ |
Zero-downtime reindex | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
Async background reindex | ✅ + polling | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | 🟡 |
GPU CUDA optional | ✅ 12 auto | ❌ | 🟡 | ✅ | ✅ | ✅ | ✅ | 🟡 | 🟡 |
File formats builtin | ✅ 20 | 0 (LlamaParse=$) | 50+ plugins | ✅ 36+ | 8+ | ? | ? | ~10 | ❌ |
Setup < 5 min POC | ✅ pip 1-liner | ✅ | ✅ | ✅ | ❌ 16GB RAM | ✅ | ✅ docker | ✅ docker | ✅ |
Nightly chaos + soak + mutation | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
License | ✅ MIT | MIT | MIT | Apache-2.0 | Apache-2.0 | Apache-2.0 | ⚠️ preserving | ⚠️ restrictive | Apache-2.0 |
The 5 dimensions where knowledge-rag is unique: health probes + JSON logging + Prometheus + rate limit + bearer auth simultaneously built-in on an OSS RAG-focused MCP server. Zero-downtime reindex + async background reindex + nightly chaos/soak/mutation are documented on nobody else's README.
🚀 Quick Start (3 minutes, from zero to your first query)
Pick your integration path — knowledge-rag ships the same server through every channel.
Path 1 — Claude Code, Cursor, Windsurf, Cline, VS Code, Gemini CLI, Zed (MCP)
pip install knowledge-rag
knowledge-rag init # scaffolds config.yaml + documents/Drop your PDFs, markdown, code files into documents/. Restart your MCP client. Ask it:
search_knowledge("your query")That's it. First query loads the ONNX embedding model (~200MB, one-off download). Subsequent queries are cached and hit sub-second latency.
Path 2 — HTTP / SSE server (multi-user, air-gapped, load-balanced)
# config.yaml
server:
transport: "sse" # or "streamable-http"
host: "0.0.0.0"
port: 8179
auth:
bearer_token: "your-secret-token"
rate_limit:
enabled: true
requests_per_minute: 60
metrics:
enabled: true
port: 9179
logging:
format: "json" # ELK / Loki / Datadog / CloudWatch readyknowledge-rag --transport sseHealth probe:
curl http://your-host:8179/health→ 200 + JSON payloadPrometheus scrape:
http://your-host:9179/metricsMCP dispatcher: authenticated via
Authorization: Bearer your-secret-token
Path 3 — Docker (models pre-downloaded, air-gapped ready)
docker pull ghcr.io/lyonzin/knowledge-rag:latest
docker run -v $(pwd)/documents:/app/documents -p 8179:8179 ghcr.io/lyonzin/knowledge-rag:latestFull installation guide with all 5 methods, 8 MCP client configurations, and GPU setup: docs/INSTALLATION.md →
🤖 Ready-to-use skills for AI agents
Installing knowledge-rag gives your agent 13 MCP tools. It does not tell the agent when to use them. That is what the skills/ folder solves — drop-in behavioural skills for Claude Code, Cursor, Windsurf, Cline, Zed, VS Code Copilot that turn "AI with access to RAG" into "AI that actually uses RAG first".
10 skills, MIT licensed, organized by kind:
# | Skill | What it does |
1 | Search the corpus before answering any technical claim | |
2 | Every claim ships with | |
3 | First interaction of a session probes what is indexed | |
4 | 3-step drill: | |
5 | Only hit the web when local RAG comes back empty | |
6 | Bug / error → RAG first for prior fixes | |
7 | Review consults ADRs / patterns before commenting | |
8 | After a decision, index it back — close the feedback loop | |
9 | Security tasks: MITRE / CVE / runbook first | |
10 | Weekly checkup — MRR@5 · Recall@5 · Precision@5 |
Install — pick the shortest path for your machine:
# Option 1 — Via skills.sh (needs Node — one command, zero clone)
npx skills add lyonzin/knowledge-rag
# Option 2 — Via our install.sh (no Node needed; works on Linux/macOS/WSL/Git Bash)
curl -fsSL https://raw.githubusercontent.com/lyonzin/knowledge-rag/master/skills/install.sh | bashBoth restart-Claude-Code and you are done. Option 2 supports --project, --only rag-check-first,rag-cite-sources, --dry-run, --help.
For Cursor, Windsurf, Cline and full manual instructions → skills/README.md · Full catalog with skill chains → skills/CATALOG.md
🛠️ The 13 MCP tools your agent gets
Once installed, your AI agent gets these 13 tools automatically:
Tool | Purpose |
| Hybrid semantic + BM25 with cross-encoder rerank |
| Retrieve full content of one document |
| Find documents similar to a reference |
| Measure MRR@5 · Recall@5 · Precision@5 |
| Index a new document via MCP |
| Re-index a changed document |
| Drop a document + all its chunks |
| Fetch, sanitize, and index a URL |
| Enumerate indexed documents |
| Auto-tagged by folder path |
| Corpus size, cache hit rate, embedding dim |
| Smart incremental OR nuclear rebuild |
| Live progress polling (async reindex) |
Full API reference with parameter details, return schemas, examples: docs/API.md →
🏢 Enterprise Features (built-in, zero configuration)
Every RAG framework claims "production-ready." Here is what knowledge-rag ships in the OSS core, verified by regression tests, that competitors either paywall, plugin-ify, or simply don't have.
Security
Bearer token auth on SSE / HTTP transports — constant-time comparison (
hmac.compare_digest), RFC 6750 challenge, 401 fenced withWWW-AuthenticateheaderPath traversal + symlink escape defenses —
validate_path_withinguarding 6 CRUD tools (CWE-22, CWE-59)Prompt injection 3-layer defense — sentinel neutralization + provenance fence +
external_sourceflag (OWASP LLM01:2025)OpenSSF Best Practices badge verified · CodeQL weekly scan · Bandit + Semgrep + Gitleaks + pip-audit on every PR
PyPI Trusted Publishing via OIDC (zero long-lived tokens in CI)
Observability
Prometheus
/metricsendpoint — custom histogram buckets tuned for RAG (p95 ≤ 10ms fast-path targets), 7 canonical metrics via@instrumentdecorator on all 13 toolsRate limiting — thread-safe sliding-window counter, per-client RPM + burst, zero overhead when disabled
Health probes —
GET /healthand/healthzreturning{status, version, uptime_seconds, cache}in front of the auth middleware (probes always succeed)Structured JSON logging — opt-in via
server.logging.format: "json", one JSON object per record ready for ELK / Loki / Datadog / CloudWatchPublic benchmark dashboard on GitHub Pages
Scale & performance
SSE / streamable-http transport — 1 server serves N MCP clients, ChromaDB WAL mode enabled automatically, shared embedding model + query cache
BM25 inverted-index — 128× faster than linear scan (custom implementation, replaces
rank-bm25)FTS5 SQLite fast-path (opt-in, ADR-002/003/006/008) — <10ms cold, <2ms hot on lexical queries
Cross-encoder reranking — Xenova/ms-marco-MiniLM-L-6-v2, +1.88pp Recall@10 (p<0.001)
GPU CUDA 12 with auto DLL discovery + graceful CPU fallback
Query cache — LRU + 5-min TTL, cuts p95 latency ~40%
Zero-downtime reindex — staging populate + validation + atomic swap + durable metadata rollback
Async background reindex with
get_reindex_status()polling
Reliability
Nightly chaos injection — HuggingFace Hub offline · ONNX zero-byte replay · watchdog crash recovery (3 scenarios in
tests/chaos/)50 000-iteration soak test — proves no memory leak after 1h of continuous queries (
KNOWLEDGE_RAG_SOAK_ITERATIONS=50000)Mutation testing (mutmut) on
instance_lock+preflight— catches tests that are too weakDeterminism check — full test suite × 3, catches flakes
Backwards-compat frozen — 13 MCP tool parameter names guarded by
tests/test_backwards_compat.py+ legacy YAML fixtures (v3.6.0 / v3.7.0) still parseAPI surface AST diff —
check_api_surface.pyblocks any breaking change at PR time9-cell CI matrix — Linux + Windows + macOS × 3.11 + 3.12 + 3.13
💼 Use Cases (real corpora, real teams)
Security Teams — Red / Blue / CTF
Preset: cybersecurity.yaml · 8 categories · 200+ routing keywords · 69 query expansions
Ingest MITRE ATT&CK, threat reports, exploit writeups, incident reports. Search from Claude Code with search_knowledge("privilege escalation windows") and get instant recall across your entire corpus. Air-gapped — nothing leaves the laptop.
Development Teams — Design Docs, Runbooks, Code
Preset: developer.yaml · 9 categories · 150+ routing keywords · 50+ expansions
Replace Confluence hunting. Ingest architecture docs, ADRs, runbooks, code, API specs. Devs ask their AI agent "how do we authenticate the payment service" and get the exact ADR + implementation file citation.
Research Labs — Papers, Notebooks, Datasets
Preset: research.yaml · 9 categories · 100+ routing keywords · 40+ expansions
Index arXiv papers, lab notebooks, dataset documentation. Semantic search finds papers by intent, not just keywords — cross-encoder reranking surfaces the actually-relevant one instead of five that share a term.
Enterprise Knowledge Base — Air-gapped, Auditable
Preset: general.yaml · blank slate, pure semantic search
Deploy via SSE on a single VM. 40+ users authenticated via bearer token, rate-limited, Prometheus-monitored, /health probes wired to your load balancer, JSON logs shipped to Datadog. No cloud calls. Meets LGPD, GDPR, HIPAA data-locality requirements by design.
Verified at scale: production reproduction on a 5 889-doc / 75 016-chunk corpus with concurrent queries during a nuclear rebuild — zero downtime, zero errors (see CHANGELOG v4.8.3).
🏗️ Architecture at a glance
End-to-end view of how MCP clients, the retrieval pipeline, storage, and enterprise plumbing connect. Every arrow is a real code path — nothing pictured here is aspirational.
flowchart TB
subgraph CLIENTS["MCP Clients (any of these)"]
C1[Claude Code]
C2[Claude Desktop]
C3[Cursor]
C4[Windsurf]
C5[VS Code · Cline · Gemini CLI · Zed]
end
subgraph TRANSPORT["Transport Layer"]
T1[stdio<br/>1 process per client]
T2[SSE / streamable-http<br/>1 server serves N clients]
end
subgraph MIDDLEWARE["ASGI Middleware Chain (HTTP mode)"]
M1[HealthMiddleware<br/>/health · /healthz]
M2[BearerAuthMiddleware<br/>constant-time compare]
M3[Rate Limiter<br/>sliding window]
end
subgraph MCP["13 MCP Tools (frozen contract)"]
MT1[search_knowledge]
MT2[get_document · search_similar]
MT3[add_document · add_from_url · update · remove]
MT4[reindex_documents · get_reindex_status]
MT5[list_documents · list_categories · get_index_stats · evaluate_retrieval]
end
subgraph SEARCH["Retrieval Pipeline"]
R[Query Router<br/>lexical vs semantic]
F[FTS5 Fast-Path<br/>opt-in · lt 10ms]
BM[BM25 Inverted Index<br/>128x faster than baseline]
SE[Semantic Search<br/>FastEmbed ONNX lazy-loaded]
RRF[Reciprocal Rank Fusion]
CE[Cross-Encoder Rerank<br/>MiniLM-L-6-v2]
QC[Query Cache<br/>LRU + 5-min TTL]
end
subgraph STORAGE["Storage (100% local)"]
CH[ChromaDB<br/>vectors + metadata<br/>WAL mode]
FT[SQLite FTS5<br/>lexical index<br/>WAL + busy-timeout]
MD[index_metadata.json<br/>durable state]
end
subgraph INGEST["Document Ingestion"]
FS[documents/ folder]
WD[Watchdog<br/>10s debounce]
PA[35 Parsers<br/>MD · PDF · DOCX · code · IaC · IPYNB]
CK[Chunker<br/>markdown-aware · code-aware]
EM[FastEmbed ONNX<br/>384D bge-small-en-v1.5]
DD[SHA256 Dedup]
SW[Zero-downtime Staging Swap<br/>rollback on validation fail]
end
subgraph OBS["Enterprise Observability (opt-in)"]
PM[Prometheus /metrics<br/>7 canonical + histograms]
LG[Structured JSON logs<br/>ELK · Loki · Datadog · CloudWatch]
HC[Health payload<br/>version · uptime · cache stats]
end
subgraph CFG["Configuration"]
YM[config.yaml<br/>+ 5 domain presets]
end
C1 & C2 & C3 & C4 & C5 -->|MCP protocol| T1
C1 & C2 & C3 & C4 & C5 -.->|remote deploy| T2
T1 --> MCP
T2 --> M1 --> M2 --> M3 --> MCP
MT1 --> QC
QC -->|cache miss| R
R -->|lexical| F
R -->|semantic| SE
R -->|hybrid| BM
F --> CH
F --> FT
BM --> CH
SE --> CH
BM --> RRF
SE --> RRF
RRF --> CE
CE --> QC
MT2 --> CH
MT3 --> INGEST
MT4 --> SW
MT5 --> CH
FS --> WD --> PA
PA --> CK --> EM --> DD --> CH
SW -.->|atomic swap| CH
SW -.-> FT
CH -.-> MD
MCP -.->|instrumented| PM
MCP -.->|logs| LG
M1 --> HC
YM -.-> SEARCH
YM -.-> STORAGE
YM -.-> OBS
YM -.-> MIDDLEWARE
classDef client fill:#3776AB,stroke:#1e5a8a,color:#fff
classDef transport fill:#00A67E,stroke:#006e54,color:#fff
classDef middleware fill:#6b46c1,stroke:#4c1d95,color:#fff
classDef storage fill:#4b5563,stroke:#1f2937,color:#fff
classDef obs fill:#dc2626,stroke:#7f1d1d,color:#fff
classDef ingest fill:#f59e0b,stroke:#78350f,color:#fff
class C1,C2,C3,C4,C5 client
class T1,T2 transport
class M1,M2,M3 middleware
class CH,FT,MD storage
class PM,LG,HC obs
class FS,WD,PA,CK,EM,DD,SW ingestReading the diagram (top → bottom):
Any MCP client — Claude Code, Cursor, Windsurf, and 5 others — connects via the transport of your choice (stdio for personal use, SSE/streamable-http for teams).
HTTP mode chains 3 ASGI middlewares in order: health probes first (always answered), then bearer auth (fenced with
WWW-Authenticate), then rate limiter (sliding window).All 13 MCP tools are decorated with
@rate_limited+@instrument— Prometheus counts every call, rate limiter enforces RPM+burst, both zero-cost when disabled.search_knowledgechecks the query cache first; cache miss routes through the Query Router (regex classifier) to either the FTS5 fast-path (lexical) or the hybrid pipeline (BM25 + semantic + RRF + cross-encoder rerank).Storage is 100% local: ChromaDB (WAL mode) for vectors + metadata, SQLite FTS5 (WAL + busy-timeout) for lexical fast-path,
index_metadata.jsonfor durable state.Document ingestion runs continuously: watchdog observes
documents/, 35 parsers handle each format, chunker respects language boundaries, FastEmbed ONNX generates embeddings, SHA256 deduplicates, and a staging swap performs zero-downtime rebuilds with rollback-on-failure.Enterprise observability (opt-in) — Prometheus
/metrics, structured JSON logs,/healthpayload — attaches to the same instrumentation points, no code changes required.config.yaml(with 5 domain presets) controls every subsystem — no environment variable spaghetti, no hardcoded paths.
Complete architecture — 4 detailed Mermaid diagrams (System Overview · Query Flow · Document Ingestion · hybrid_alpha effect): docs/ARCHITECTURE.md
📄 35 File Formats — parsed natively, no plugins needed
Every parser is chunk-aware — Markdown splits at ## headers, code splits at function/class boundaries, notebooks skip base64 outputs, PDFs use PyMuPDF, spreadsheets extract sheet-by-sheet. 33 formats are enabled by default; the 2 MetaTrader formats are opt-in (add to documents.supported_formats in config.yaml).
# | Format | Extension | Parser | Default | Notes |
1 | Markdown |
| Section-aware (splits at | Yes | Headers preserved as chunk boundaries |
2 | Plain Text |
| Fixed-size chunking | Yes | 1000 chars + 200 overlap |
3 |
| PyMuPDF extraction | Yes | Text-based PDFs only (no OCR) | |
4 | Word |
| python-docx | Yes | Headings preserved as markdown |
5 | Excel |
| openpyxl | Yes | Sheet-by-sheet extraction |
6 | PowerPoint |
| python-pptx | Yes | Slide-by-slide extraction |
7 | Jupyter Notebook |
| Cell-aware parser | Yes | Markdown + code cells only; skips outputs/base64 |
8 | JSON |
| Structure-aware | Yes | Flattened key-value extraction |
9 | CSV |
| Row-based parser | Yes | Headers + rows as text |
10 | XML |
| XML parser | Yes | Root element + namespace metadata |
11 | Python |
| Code-aware parser | Yes | Functions/classes as chunks |
12 | C Source |
| Code-aware parser | Yes | Functions / structs / includes extracted |
13 | C/C++ Header |
| Code-aware parser | Yes | Function declarations + structs extracted |
14 | C++ Source |
| Code-aware parser | Yes | Classes / structs / includes extracted |
15 | JavaScript |
| Code-aware parser | Yes | Functions / classes / imports (ESM + CJS) |
16 | React JSX |
| Code-aware parser | Yes | Same as JS parser |
17 | TypeScript |
| Code-aware parser | Yes | Functions / classes / interfaces / enums / imports |
18 | React TSX |
| Code-aware parser | Yes | Same as TS parser |
19 | Go |
| Code-aware parser | Yes | Functions / structs / imports extracted |
20 | Rust |
| Code-aware parser | Yes | Functions / structs / enums / traits / |
21 | Kotlin |
| Code-aware parser | Yes | Functions (incl. class members) / classes extracted |
22 | YAML |
| YAML parser | Yes | Kubernetes kind / apiVersion / name extracted |
23 | YAML |
| YAML parser | Yes | Same as YAML parser |
24 | HuJSON |
| HuJSON parser | Yes | JSON with comments + trailing commas (e.g. Tailscale ACLs) |
25 | CUE |
| Code-aware parser | Yes | Imports / package extracted |
26 | Protocol Buffers |
| Proto parser | Yes | Services / messages / RPCs extracted |
27 | Rego |
| Code-aware parser | Yes | OPA policies — imports / package extracted |
28 | SQL |
| SQL parser | Yes | Table names + statement types extracted |
29 | Shell |
| Shell parser | Yes | Function names extracted |
30 | jq |
| Shell parser | Yes | Indexed as shell-style script |
31 | Dockerfile |
| Text parser | Yes | Matched by exact filename (no extension) |
32 | Makefile |
| Text parser | Yes | Matched by exact filename (no extension) |
33 | Tiltfile |
| Code-aware parser | Yes | Starlark — |
34 | MQL4 Source |
| Code parser | No | MetaTrader — opt-in via |
35 | MQL4 Header |
| Code parser | No | MetaTrader — opt-in via |
Enable an opt-in format — add the extension to
documents.supported_formatsin yourconfig.yaml:documents: supported_formats: [".md", ".pdf", ".mq4", ".mqh"]
Full parser reference with per-format notes: docs/CONFIGURATION.md
🔌 Choose your MCP integration
Claude Code
~/.claude.json
Claude Desktop
claude_desktop_config.json
Cursor
~/.cursor/mcp.json
Windsurf
~/.codeium/windsurf/mcp_config.json
VS Code
Copilot Chat mcp.json
Cline · Gemini CLI · Zed Native MCP
Complete client configuration guide with JSON schemas per client: docs/INSTALLATION.md#use-with-other-mcp-clients →
⚙️ Configuration in 30 seconds
# config.yaml — everything is optional; defaults just work
paths:
documents_dir: "./documents"
data_dir: "./data"
models:
embedding:
profile: "compact" # "compact" | "quality" | "multilingual" | "custom"
gpu: "auto" # "auto" | "true" | "false"
reranker:
enabled: true # cross-encoder rerank
search:
default_results: 5
max_results: 100
server: # optional — SSE / HTTP mode
transport: "stdio" # or "sse" / "streamable-http"
auth:
bearer_token: "" # set a secret to enable auth
rate_limit:
enabled: false
metrics:
enabled: false
logging:
format: "text" # or "json"Pre-built presets: cybersecurity.yaml · developer.yaml · research.yaml · general.yaml · multilingual.yaml
Complete configuration reference — every field, every default, tuning guide: docs/CONFIGURATION.md →
🔒 Security & Compliance
knowledge-rag is designed for teams that cannot let their documents leave the perimeter.
Requirement | How knowledge-rag delivers |
Data locality (LGPD / GDPR / HIPAA) | 100% on-premise, zero egress network calls after initial model download |
Air-gapped deployment | ONNX models pre-cached; set |
CVE monitoring | Dependabot (weekly) + pip-audit + Socket + CodeQL |
Supply chain security | PyPI Trusted Publishing via OIDC (no long-lived tokens) |
Vulnerability disclosure | Private security advisory via SECURITY.md |
Signed release attestations | GitHub release attestations on every published version |
Reproducible builds | Locked |
Authenticated access | Bearer token middleware on SSE / HTTP transports (constant-time compare, RFC 6750) |
Rate limiting | Sliding-window per-client RPM + burst (opt-in, zero-cost when disabled) |
Audit-ready logging | Opt-in structured JSON logs → ship to your SIEM |
Path traversal defenses | CWE-22 / CWE-59 guards on 6 CRUD tools |
Prompt injection defense | 3-layer sanitization on |
OpenSSF Best Practices badge: passing · project ID #13864
📈 Numbers that matter
26 000+ total downloads on PyPI · 250+ GitHub stars · 70+ enterprise teams (private + community)
700+ tests collected · 1.33:1 test-to-code ratio · codecov trend gate ±0.5pp
35+ status checks on every PR (9-cell OS×Python matrix · 7 quality pillars)
35 file formats parsed natively · 13 MCP tools frozen · 5 domain presets (cyber · dev · research · multilingual · general)
BM25 128× faster than baseline · cross-encoder +1.88pp Recall@10 (p<0.001) · cache −40% p95 latency
1 800+ files / 39 K chunks indexed in < 3 min on a modern laptop (typical developer corpus)
Verified in production on 5 889-doc / 75 016-chunk corpora
Public benchmark dashboard: https://lyonzin.github.io/knowledge-rag/
📚 Documentation
Doc | What's inside |
5 install methods · 8 MCP client integrations · GPU setup | |
Complete reference for all 13 MCP tools | |
Every | |
4 Mermaid diagrams: System Overview · Query Flow · Ingestion · hybrid_alpha | |
11 common issues + solutions | |
Opt-in lexical fast-path — when and how | |
Zero-downtime rebuild · resume · checkpoint | |
CUDA 12 installation + troubleshooting | |
Embedding profile · multilingual · zero-downtime | |
Threat model · disclosure channel | |
Development · testing · PR process | |
All release notes since v1.0.0 |
🤝 Community & Support
Report a bug → Open an issue
Ask a question → GitHub Discussions
Report a vulnerability → Security advisory (private)
Contribute → CONTRIBUTING.md
Response SLA (best-effort, community project):
Security reports: within 48 h
Bug reports with reproduction: within 5 business days
Feature requests: triaged on next release cycle
🗺️ Recent releases
v4.8.5 (2026-08-13) — Enterprise observability:
/healthendpoint + opt-in JSON structured loggingv4.8.4 (2026-08-13) — Patch: security + durability + defensive fixes
v4.8.3 (2026-08-10) — Critical hotfix: nuclear-rebuild + smart-reindex hardening on 50k+ chunk corpora
v4.8.2 (2026-08-10) — FTS5 lexical fast-path opt-in release
v4.8.0 (2026-08-06) — Multilingual foundation + zero-downtime reindex
Full history: CHANGELOG.md →
📜 License
MIT License — LICENSE. Forever. No cloud upsell, no dual-licensing, no restrictive clauses. Fork it, sell derivatives, embed it in commercial products — the license does not care.
🙏 Acknowledgments
Built on the shoulders of amazing open-source projects:
Anthropic MCP — Model Context Protocol spec + Python SDK
ChromaDB — vector database that just works
FastEmbed — ONNX embeddings, no PyTorch bloat
HuggingFace — model hosting +
Xenova/ms-marco-MiniLM-L-6-v2cross-encoderBAAI — the
bge-small-en-v1.5embedding model
Community contributors: @Hohlas · @eeshsaxena · Sergey Khokhlov · and everyone who filed issues or PRs.
Built by Ailton Rocha (Lyon.) · Star ⭐ if this saves you time · Report an issue · Contribute
knowledge-rag — the MCP-first local RAG server for Claude Code, Cursor, Windsurf, and every AI agent.
Available Tools
13 toolsadd_documentA
Add a new document to the knowledge base from raw text content.
Mutating — writes a file to disk and indexes it immediately. No auth required.
Args: content: Full text content of the document (markdown supported) filepath: Relative path within documents directory (e.g., "security/new-technique.md"). The subdirectory should match the category. category: Document category — one of: security, ctf, logscale, development, general, redteam, blueteam (default: general)
Returns: JSON string with indexing results (filepath, chunks created, status).
Usage: Use to add new documents from text content. Use add_from_url() instead when the source is a web page. Use update_document() to replace content of an existing file. The document is immediately searchable after this call — no manual reindex needed.
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | ||
| category | No | general | |
| filepath | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavioral traits: mutating, writes to disk, indexes immediately, no auth, and returns JSON with specified fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise and well-structured: purpose statement, behavioral note, parameter descriptions, return value, and usage guidelines. Every sentence adds value without verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and 0% schema descriptions, the description covers all necessary aspects: purpose, parameters, behavior, return format, and usage context with sibling differentiation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds detailed semantics for all three parameters: markdown support for content, path conventions for filepath, and category enumeration with default.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds a document from raw text content, distinguishes from siblings like add_from_url and update_document, and mentions immediate searchability, making purpose very clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides when to use this tool (text content) and when to use alternatives (add_from_url for web pages, update_document for replacing files), plus notes no auth required.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_from_urlA
Fetch content from a URL, convert to markdown, and add to the knowledge base.
Mutating — makes an outbound HTTP request (requires internet access), strips HTML, converts to markdown, saves to disk, and indexes immediately.
Args: url: Full URL to fetch (https:// required). The page must be publicly accessible. category: Document category — one of: security, ctf, logscale, development, general, redteam, blueteam (default: general) title: Optional document title. Auto-detected from the page's tag if omitted.
Returns: JSON string with indexing results (detected title, filepath, chunks created, status).
Usage: Use to ingest web content (writeups, blog posts, documentation pages) directly by URL. Use add_document() instead when you already have the text content. The document is immediately searchable after this call — no manual reindex needed.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| title | No | ||
| category | No | general |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses all behavioral aspects: mutating, outbound HTTP, internet access required, HTML stripping, markdown conversion, disk save, immediate indexing. This is thorough and compensates for the lack of annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with intro, behavior, Args, Returns, and Usage sections. It is slightly verbose but efficiently conveys all necessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete: covers behavior, parameters, return value (JSON string with details), and usage context. It also mentions immediate searchability, leaving no ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully explains each parameter: url requires HTTPS and public accessibility, category lists possible values, title is optional and auto-detected. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches content from a URL, converts to markdown, and adds to the knowledge base. It distinguishes itself from the sibling tool add_document by specifying when to use each.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit usage guidance is provided: 'Use to ingest web content... Use add_document() instead when you already have text content.' This clearly tells when and when not to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
evaluate_retrievalA
Evaluate search quality by testing whether search_knowledge() retrieves expected documents.
Read-only. Runs multiple search queries internally. No side effects on the index.
Args: test_cases: JSON string array of test cases. Each item requires "query" (search string) and "expected_filepath" (path of the document that should appear in top-5 results). Example: [{"query": "suid exploit", "expected_filepath": "security/suid.md"}]
Returns: JSON string with MRR@5 (Mean Reciprocal Rank), Recall@5, and per-query hit/miss breakdown. MRR@5 above 0.7 indicates good retrieval quality.
Usage: Use to audit search quality after bulk document ingestion or after tuning hybrid_alpha. Use get_index_stats() for system health checks instead. Use search_knowledge() for actual document retrieval — this tool is for quality measurement only.
| Name | Required | Description | Default |
|---|---|---|---|
| test_cases | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It says 'Read-only. Runs multiple search queries internally. No side effects on the index.' It also explains the return value structure (MRR@5, Recall@5, etc.) and expected threshold. However, it does not mention potential performance impact or rate limiting, but for a quality evaluation tool, this is reasonably transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: first sentence states purpose, then line about side effects, then parameter explanation with example, then return value description, then usage with alternatives. It is front-loaded with key info and not overly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only one parameter and an output schema exists (context says 'Has output schema: true'), the description is complete. It explains parameter format, return metrics, and usage context. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% because input schema only has 'test_cases' as a string with no description inside. The description provides a detailed explanation: 'JSON string array of test cases. Each item requires "query" (search string) and "expected_filepath"...' with an example. This fully compensates for the lack of schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Evaluate search quality by testing whether search_knowledge() retrieves expected documents.' It specifies the verb 'evaluate' and the resource 'search quality/retrieval'. It distinguishes from siblings like 'search_knowledge' and 'get_index_stats' by mentioning them in the usage section.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: 'to audit search quality after bulk document ingestion or after tuning hybrid_alpha.' It also provides alternatives: 'Use get_index_stats() for system health checks instead. Use search_knowledge() for actual document retrieval — this tool is for quality measurement only.' This is excellent guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_documentA
Get the full content of a specific document by filepath.
Read-only. No side effects.
Args: filepath: Relative path to the document within the documents directory (e.g., "security/technique.md"). Must be an indexed file — use list_documents() to browse available paths, or search_knowledge() to find the filepath by topic first.
Returns: JSON string with full document content and metadata (filepath, category, size).
Usage: Use when you need the complete text of a known file — search_knowledge() returns chunks, not full docs. Use search_knowledge() first to find the filepath if unknown. Use list_documents() to browse all available files by category.
| Name | Required | Description | Default |
|---|---|---|---|
| filepath | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Declares 'Read-only. No side effects.' and describes return value format. With no annotations, the description carries full burden; it could mention error behavior for missing files, but overall adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured: purpose, behavior, args, returns, usage. Each sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool and presence of output schema, the description covers all needed aspects: usage, parameter, return value, and context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no description for filepath (0% coverage), but the description adds significant meaning: example path, relative path constraint, and pointers to find valid paths.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Get the full content of a specific document by filepath.' It distinguishes from siblings by noting that search_knowledge returns chunks and list_documents browses files.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use ('when you need the complete text of a known file') and provides clear guidance on alternatives (search_knowledge first if unknown, list_documents to browse).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_index_statsA
Get statistics and health metrics for the knowledge base index.
Read-only. No side effects.
Returns: JSON string with system metrics: total documents, total chunks, embedding model name, BM25 status, query cache hit rate, and file watcher status.
Usage: Use for system health checks — verifying the embedding model loaded, checking index population, or monitoring cache efficiency. Use list_categories() for per-category document counts instead. Use evaluate_retrieval() to measure actual search quality with test queries.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Declares read-only and no side effects. Lists return fields in detail: total documents, chunks, embedding model name, BM25 status, cache hit rate, file watcher status. With no annotations provided, the description fully carries the burden of transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise and well-structured. Opens with purpose, then read-only note, then lists return values, then usage guidance with alternatives. Every sentence is informative with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and an output schema, the description explains what the tool returns and its safe, read-only nature. It is completely adequate for an AI agent to understand and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema coverage is 100%. The description adds no parameter info because none are needed. Baseline of 4 is appropriate for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets statistics and health metrics for the knowledge base index. It uses specific verb+resource and distinguishes from sibling tools like list_categories and evaluate_retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit usage scenarios: system health checks, verifying embedding model, checking index population, monitoring cache efficiency. Also tells when not to use by naming alternatives (list_categories for per-category counts, evaluate_retrieval for search quality).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_reindex_statusA
Get the current status of a background reindex operation.
Lightweight — does not compute full index statistics. Use this to poll progress after calling reindex_documents().
Returns: JSON string with reindex status. When active: operation name, progress (processed/total), percent complete, indexed/skipped/errors counts, and start time. When inactive: active=false, plus last_result or last_error from the most recent completed reindex.
Usage: Call repeatedly after reindex_documents() to monitor progress. When reindex.active becomes false, the operation is complete. Use get_index_stats() for full index health metrics.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description discloses that the tool is lightweight and does not compute full index statistics. It details the return structure for both active and inactive states, providing full behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured. The first sentence gives the core purpose, followed by important details in a logical order. Returns are listed compactly, and usage instructions are clear. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and an output schema, the description thoroughly explains the return value and usage context. It also references sibling tools for completeness. The tool is simple, and the description covers all needed information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so the input schema fully covers parameter semantics. The description adds value by explaining the return format, which is beyond what the schema provides. Baseline 4 for no parameters is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets the current status of a background reindex operation. It uses a specific verb ('Get') and resource ('reindex status'), and distinguishes from sibling tools like 'get_index_stats' and 'reindex_documents'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use: poll after calling reindex_documents. It also provides an alternative: use get_index_stats for full index health. This gives clear guidance on tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_categoriesA
List all document categories with their document counts.
Read-only. No side effects. Reflects the live index state.
Returns: JSON string with category names, document counts per category, and total document count.
Usage: Use before filtering search_knowledge() or list_documents() by category to see which categories exist and how many documents each contains. Use get_index_stats() instead for broader system health metrics (model name, cache hit rate, BM25 status).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states the tool is read-only with no side effects and reflects the live index state. Since no annotations are provided, this fully covers behavioral traits such as safety and data freshness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences, each adding value: purpose, behavior, and usage guidance. It is efficiently front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is parameterless and has an output schema, the description covers everything needed: what it does, what it returns (category names, counts, total), and when to use it. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters and schema description coverage is 100% (trivially). There are no parameters to explain, so the description adds no parameter semantics, but this is acceptable for a parameterless tool. Baseline 4 for 0 params.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all document categories with their document counts, using a specific verb ('List') and resource ('document categories'). It is distinct from siblings like 'list_documents' which lists documents, not categories.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises using this tool before filtering search_knowledge or list_documents by category, and suggests get_index_stats as an alternative for broader system health metrics. This provides clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_documentsA
List all indexed documents, optionally filtered by category.
Read-only. No side effects.
Args: category: Optional category filter. Must be a valid category name — call list_categories() to see available options (e.g., security, ctf, logscale, development, general, redteam, blueteam).
Returns: JSON string with list of document filepaths, categories, and metadata for each indexed file.
Usage: Use to browse what's in the index or verify a specific file is indexed. Use list_categories() first to see valid category names. Use search_knowledge() when you want to find documents by topic rather than browsing the full list. Use get_document() to read a specific file once you have its filepath.
| Name | Required | Description | Default |
|---|---|---|---|
| category | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
States 'Read-only. No side effects' which is important for behavior. Discloses return format and parameter constraints. Lacks info on error handling or rate limits, but sufficient for this simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with summary, behavior, params, returns, and usage. Front-loaded with purpose. Slightly verbose but each sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers all needed aspects: purpose, parameters, return format, usage context, and alternatives. Output schema exists, so return details are sufficient. Annotations missing but description fills gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'category' has no schema description (0% coverage). The description compensates fully by explaining it's optional, must be a valid category name, and suggests calling list_categories for options, with examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list) and resource (indexed documents) with an optional category filter. It distinguishes from siblings by mentioning when to use alternatives like search_knowledge and get_document.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance on when to use this tool (browsing index, verifying file) and when not (use list_categories first, use search_knowledge for topics, use get_document for reading). Includes specific alternative tool names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
reindex_documentsA
Index or reindex all documents in the knowledge base (runs in background).
force — smart reindex (detect changed files + rebuild BM25). Use after
filesystem edits outside add_document/update_document.
full_rebuild — nuclear rebuild (delete + re-embed). Use only after
embedding-model change or index corruption. Mutually exclusive with resume.
resume — pick up an interrupted smart reindex from
data/reindex_checkpoint.json. Falls back to a fresh smart run silently
if the checkpoint is missing/corrupt/drifted (v4.8.0 Fase 4).
Returns a JSON envelope. Poll get_reindex_status() until
reindex.active becomes false. Add/update/URL tools already auto-index —
use these flags only for the recovery/rebuild scenarios above.
| Name | Required | Description | Default |
|---|---|---|---|
| force | No | ||
| resume | No | ||
| full_rebuild | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully bears the transparency burden. It discloses background execution, status polling via get_reindex_status(), silent fallback on checkpoint issues, and destructive deletion in full_rebuild ("nuclear rebuild (delete + re-embed)"). This goes well beyond basic expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured: the purpose is front-loaded, each parameter is explained on its own line, and the closing note ties everything together. Every sentence provides necessary context—no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a tool with three nuanced flags: it covers purpose, usage scenarios, background behavior, status polling, and the relationship to sibling tools. The output schema exists, but the description additionally clarifies the polling mechanism, making it self-sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions (0% coverage), but the description fully explains each parameter: force as smart reindex, full_rebuild as destructive rebuild with mutual exclusivity with resume, and resume with checkpoint path and fallback behavior. This adds essential meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with "Index or reindex all documents in the knowledge base (runs in background)", which clearly states the verb, resource, and scope. It distinguishes itself from siblings by explicitly noting that add/update/URL tools already auto-index, so this tool is for recovery/rebuild scenarios.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit usage guidelines are provided for each flag: "Use after filesystem edits outside add_document/update_document" for force, "Use only after embedding-model change or index corruption" for full_rebuild, and a closing note that add/update/URL tools auto-index, so these flags are only for recovery/rebuild. This clearly states when to use the tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
remove_documentA
Remove a document from the knowledge base index.
Mutating — removes index entries. If delete_file=True, also permanently deletes the file from disk (irreversible, cannot be undone).
Args: filepath: Path to the document file. Must be an indexed document — use list_documents() to find valid paths. delete_file: If True, permanently deletes the file from disk in addition to removing from the index (default: False).
Returns: JSON string with removal results (filepath, status).
Usage: Use to unindex a document while keeping the file on disk (default). Set delete_file=True only for permanent removal. Use update_document() to replace content instead of removing. Use reindex_documents(force=True) if you deleted the file manually on disk outside of this tool.
| Name | Required | Description | Default |
|---|---|---|---|
| filepath | Yes | ||
| delete_file | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses mutating nature, removal of index entries, and irreversible deletion with delete_file=True. Since no annotations exist, description fully covers behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with summary, args, returns, usage. Every sentence adds value; no redundancy. Front-loaded with core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 2 parameters, no annotations, but with output schema referenced, description thoroughly covers behavior, return format, and alternatives. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds crucial meaning beyond schema: filepath must be an indexed document and hints to use list_documents(); delete_file explains permanence and default. With 0% schema coverage, description compensates fully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Remove' and the resource 'document from the knowledge base index'. It distinguishes from siblings like update_document and reindex_documents, providing specific context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly guides when to use (unindex a document) and when to set delete_file=True. Provides alternatives: update_document for replacement, reindex_documents for manual deletions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_knowledgeA
Hybrid search combining semantic search + BM25 keyword search with cross-encoder reranking.
Read-only. No side effects.
Args:
query: Search query text (1–3 keywords recommended; phrase queries also work)
max_results: Maximum number of results (default: 5, max: 20)
category: Optional category filter — one of: security, ctf, logscale, development, general,
redteam, blueteam. Call list_categories() first to see available categories and counts.
hybrid_alpha: Balance between semantic and keyword search. 0.0 = keyword-only (best for exact
technical terms like CVE IDs or tool names), 0.3 = balanced default, 1.0 = semantic-only
(best for conceptual or natural-language queries).
min_score: Minimum normalized relevance score (0.0–1.0) to include a result. Results scoring
below this threshold are discarded. Default 0.0 returns all results. Use 0.2–0.4 to cut
low-relevance noise.
snippet_mode: When true (default), truncates content to ~500 characters at a natural break
point and adds a content_length field with the original size. Use get_document() to
fetch full content when needed. Set to false to return full chunk content.
search_method: Dispatch selector (v4.8.2+). One of "auto" (router picks FTS5 fast-path
for lexical queries when enabled, hybrid otherwise), "hybrid" (force hybrid path —
kill switch for suspected router misclassification), or "fts5" (force FTS5 fast-path
— debug/testing; errors out when the feature is disabled or the index is not ready).
Default "auto" preserves pre-v4.8.2 behavior byte-for-byte when the fast-path is
disabled in config.
Returns: JSON string with results including content chunks, source filepath, relevance score, and search method used. Returns chunks, not full document content.
Usage: Primary search tool — use for any topic or keyword lookup. Prefer search_similar() when you already have a reference document and want more like it. Prefer get_document() when you already know the exact filepath and need the full content.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| category | No | ||
| min_score | No | ||
| max_results | No | ||
| hybrid_alpha | No | ||
| snippet_mode | No | ||
| search_method | No | auto |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden and does so thoroughly. It discloses read-only nature ('Read-only. No side effects.'), snippet truncation behavior, min_score filtering, search_method dispatch behavior including compatibility notes, and return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
While long, the description is well-structured with sections (overview, read-only, Args, Returns, Usage) and each sentence earns its place. It is front-loaded with the primary purpose, and the parameter details are necessary for correct invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the return format (JSON string with content chunks, source filepath, relevance score, search method), parameter semantics, alternatives, and edge cases like FTS5 fast-path errors. Given the tool has 7 parameters and an output schema, this description is complete without relying on external documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, and it excels. Every parameter is explained with practical guidance: query recommendations (1-3 keywords), hybrid_alpha semantics (keyword vs semantic), min_score tuning (0.2-0.4 to cut noise), and search_method behavior including a kill switch and debug use case.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Hybrid search combining semantic search + BM25 keyword search with cross-encoder reranking,' clearly stating the tool's specific action (search) and resource (knowledge). It also differentiates from siblings by noting it 'Returns chunks, not full document content' and referencing search_similar and get_document.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The Usage section explicitly states this is the 'Primary search tool' and names direct alternatives: 'Prefer search_similar() when you already have a reference document...' and 'Prefer get_document() when you already know the exact filepath...' This provides explicit when-to-use vs when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_similarA
Find documents semantically similar to a given reference document.
Read-only. No side effects. Uses the document's embedding for similarity comparison.
Args: filepath: Path to the reference document (must already be indexed — use list_documents() to verify). E.g., "security/technique.md" max_results: Number of similar documents to return (default: 5, max: 20)
Returns: JSON string with list of similar document filepaths and similarity scores (0.0–1.0).
Usage: Use when you have a specific document and want to discover thematically related ones. Use search_knowledge() instead when you have a text query rather than a reference document. The reference document must be indexed — call list_documents() to confirm it exists before calling this tool.
| Name | Required | Description | Default |
|---|---|---|---|
| filepath | Yes | ||
| max_results | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description fully carries the burden. It states 'Read-only. No side effects.' and explains the use of embeddings for similarity. Return format is described in detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise and well-structured: brief summary, behavioral note, args list, returns, usage. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and presence of output schema, description covers all needed aspects: purpose, usage, parameters, behavior, and return format. No gaps identified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description compensates by fully explaining both parameters: filepath (path, must be indexed, example) and max_results (default and max). This adds significant meaning beyond the bare schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action: 'Find documents semantically similar to a given reference document.' It distinguishes from sibling tool 'search_knowledge' by specifying it works with a reference document, not a text query.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: when you have a specific document, and when not: use search_knowledge for text queries. Also includes prerequisite that document must be indexed and suggests verifying with list_documents().
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_documentA
Update the content of an existing document in the knowledge base.
Mutating — overwrites the file on disk and re-indexes immediately. Old chunks are removed and replaced with new ones. Full content replacement, not a patch.
Args: filepath: Full or relative path to the document file. Must be an already-indexed file — use list_documents() to find valid paths. content: New full-text content to replace the existing content entirely
Returns: JSON string with update results (old chunk count, new chunk count, status).
Usage: Use to replace a document's content completely. Use add_document() to create a new file instead. Use remove_document() to delete without replacing. Changes are immediately searchable — no manual reindex needed.
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | ||
| filepath | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description fully discloses behavior: 'Mutating — overwrites the file on disk and re-indexes immediately. Old chunks are removed and replaced with new ones. Full content replacement, not a patch.' Also states changes are immediately searchable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with separate sections for behavior, args, returns, and usage. Every sentence is informative, though slightly verbose. Front-loaded with purpose and behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with reindexing and full replacement, description covers prerequisites, effects, and output. References sibling tools for alternative actions. No gaps given the complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description explains both parameters: filepath must be an already-indexed file and reference to list_documents(), content is full-text replacement. Also describes return value format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Update the content of an existing document' with specific verb and resource. Distinguishes from siblings add_document and remove_document directly in the usage section.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use (replace content completely), when not (use add_document for new files, remove_document for deletion), and prerequisite (file must already be indexed, use list_documents to find paths).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
1 tool update
v4.8.3- Changed
search_knowledge1 field changed- added
Input schema / properties / search_methodAdded value: +{ + "default": "auto", + "title": "Search Method", + "type": "string" +}
1 tool update
v4.8.1- Changed
reindex_documents1 field changed- added
Input schema / properties / resumeAdded value: +{ + "default": false, + "title": "Resume", + "type": "boolean" +}
2 tool updates
v4.3.1- Added
get_reindex_status - Changed
search_knowledge2 fields changed- added
Input schema / properties / min_scoreAdded value: +{ + "default": 0, + "title": "Min Score", + "type": "number" +} - added
Input schema / properties / snippet_modeAdded value: +{ + "default": true, + "title": "Snippet Mode", + "type": "boolean" +}
12 tool updates
v3.0.0- First observed
add_document - First observed
add_from_url - First observed
evaluate_retrieval - First observed
get_document - First observed
get_index_stats - First observed
list_categories - First observed
list_documents - First observed
reindex_documents - First observed
remove_document - First observed
search_knowledge - First observed
search_similar - First observed
update_document
TDQS
Each tool has a clearly distinct purpose. add_document and add_from_url differ by input source; search_knowledge and search_similar differ by query type; all other tools are uniquely defined with no overlap.
All tool names follow a consistent verb_noun pattern using snake_case, e.g., add_document, list_documents, search_knowledge. The one minor variation 'add_from_url' still starts with a verb and is clear.
12 tools is well-scoped for a knowledge base server. Each tool corresponds to a distinct operation (CRUD, search, listing, evaluation, reindex), and none seem extraneous.
The tool surface covers all typical operations: document creation (text/URL), retrieval, update, delete, listing, category browsing, search (hybrid and similarity), search evaluation, reindexing, and health stats. No obvious gaps.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Connectors
Persistent, outcome-grounded episodic memory for Claude. 14ms CPU retrieval, no GPU, no vector DB.
Self-hosted AI-native knowledge workspace with hybrid search, GraphRAG, and MCP.
Private persistent memory for Claude, ChatGPT & Gemini via MCP - semantic search, zero-code setup.
No-data MCP handoff for local Claude Code to Codex harness moves. $49 lifetime.
Related MCP Servers
- AlicenseAqualityBmaintenancePersistent memory for Claude Code. Automatically indexes every conversation and provides production-grade hybrid search (BM25 + vectors + reranker) via MCP tools. 100% local, zero config, zero API keys, zero invoice.16577MIT
- AlicenseAqualityAmaintenanceUnified MCP server combining hybrid search (vector + BM25 + code graph), structural code analysis, and persistent semantic memory. 15 tools, 25+ languages, <350MB RAM, fully local.10MIT
- AlicenseAqualityCmaintenanceSemantic vector DB as an MCP server for Claude Code — SQLite + FTS5 + local ONNX embeddings. Gives Claude persistent, searchable project knowledge across conversations.12MIT
- FlicenseNot gradedqualityBmaintenanceA lightweight RAG (Retrieval-Augmented Generation) system for Claude Code that enables semantic search over local markdown documents via MCP tools, saving tokens and time.63-
Appeared in Searches
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/lyonzin/knowledge-rag'
If you have feedback or need assistance with the MCP directory API, please join our Discord server