Personal Knowledge-Base MCP Server

A Personal Knowledge-Base MCP Server that provides semantic search over student-owned documents using the Model Context Protocol (MCP), Google Gemini embeddings, and Qdrant.

The project combines a reusable MCP server with a lightweight authenticated web backend for document upload and source management.

Project Overview

This project exposes a personal knowledge base as callable MCP tools.

Instead of relying on keyword matching, documents are:

1. Extracted page-by-page from PDF files.

2. Split into smaller semantic chunks.

3. Converted into vector embeddings using Google Gemini.

4. Stored in Qdrant.

5. Retrieved using semantic similarity search.

The MCP server exposes the knowledge base through reusable tools that can be called by an MCP-compatible client such as MCP Inspector.

Related MCP server: genai-lab

Architecture

                         ┌──────────────────────┐
                         │      User / Client    │
                         └──────────┬───────────┘
                                    │
                     ┌──────────────┴──────────────┐
                     │                             │
                     ▼                             ▼
             ┌───────────────┐             ┌───────────────┐
             │ Web Frontend  │             │  MCP Client   │
             │ Upload/Search │             │ MCP Inspector │
             └───────┬───────┘             └───────┬───────┘
                     │                             │
                     ▼                             ▼
             ┌───────────────┐             ┌───────────────┐
             │ FastAPI       │             │ FastMCP       │
             │ Backend       │             │ MCP Server    │
             │ Authentication│             │               │
             │ Uploads       │             │ 3 MCP Tools   │
             └───────┬───────┘             └───────┬───────┘
                     │                             │
                     └──────────────┬──────────────┘
                                    │
                                    ▼
                         ┌──────────────────────┐
                         │ Document Processing  │
                         │                      │
                         │ PDF Extraction       │
                         │ Chunking             │
                         │ Gemini Embeddings    │
                         └──────────┬───────────┘
                                    │
                                    ▼
                         ┌──────────────────────┐
                         │ Qdrant Vector Store  │
                         │                      │
                         │ personal_knowledge   │
                         └──────────┬───────────┘
                                    │
                                    ▼
                         ┌──────────────────────┐
                         │ Ranked Search Results│
                         │                      │
                         │ Score                │
                         │ Source               │
                         │ Page                 │
                         │ Text                 │
                         └──────────────────────┘

Features

• PDF document ingestion

• Page-by-page PDF text extraction

• Recursive text chunking

• Google Gemini gemini-embedding-001 embeddings

• Qdrant vector storage

• Semantic similarity search

• Source and page citations

• Confidence filtering for low-relevance queries

• Full-document retrieval

• Indexed-source listing

• Authenticated backend API

• PDF upload support

• User document storage

• MCP Inspector support

• Retrieval evaluation

• 100% Hit@3 on the current evaluation dataset

MCP Tools

The MCP server exposes three tools.

1. search_notes

Searches the indexed knowledge base using semantic similarity.

Arguments

query
top_k

Returns

Each result contains:

similarity score
source filename
page number
relevant text chunk

Example

Query:
What is a complex variable?

Result:
Score: 0.7239
Source: Complex_Variables_Project_Report.pdf
Page: 2
Text: ...

2. get_document

Returns the complete text of an indexed document.

Argument

doc_id

Example

Complex_Variables_Project_Report.pdf

This allows an MCP client to retrieve the complete source document after identifying a relevant result through semantic search.

3. list_sources

Lists the documents currently indexed in the knowledge base.

Example output

1. Complex_Variables_Project_Report.pdf

This provides a simple way for an MCP client to discover which source documents are available.

Backend API

The project also includes a FastAPI backend used for document management and authentication.

The backend is located in:

backend/

The API can be started with:

uvicorn backend.main:app --reload

The development server runs at:

http://127.0.0.1:8000

Interactive API documentation is available through FastAPI:

http://127.0.0.1:8000/docs

Backend Health Check

The backend provides a health endpoint:

GET /health

A successful response confirms that the FastAPI application is running.

Example:

{
  "status": "ok"
}

Authentication

Protected backend endpoints require authentication using a Bearer token.

For example, attempting to access a protected endpoint without authentication returns:

401 Unauthorized

with:

{
  "detail": "Not authenticated"
}

This confirms that authentication protection is active.

Document Upload

Documents can be uploaded through the backend.

Uploaded user documents are stored under:

documents/users/

The ingestion pipeline processes an uploaded PDF through the following stages:

PDF Upload
    ↓
PDF Text Extraction
    ↓
Page Metadata
    ↓
Recursive Chunking
    ↓
Gemini Embeddings
    ↓
Qdrant
    ↓
Semantic Search

Document Ingestion

The project includes an ingestion script:

ingest.py

It can be executed with:

python ingest.py

The ingestion pipeline performs:

PDF
 ↓
Page extraction
 ↓
Chunking
 ↓
Gemini embeddings
 ↓
Qdrant storage

Each indexed chunk contains metadata including:

text
page
source

This metadata allows search results to provide source citations and page numbers.

Vector Database

The project uses Qdrant as its vector database.

The current collection is:

personal_knowledge

Qdrant stores the generated document embeddings together with their metadata.

For local development, Qdrant can be run at:

http://localhost:6333

Example Docker command:

docker run -d --name qdrant -p 6333:6333 -p 6334:6334 qdrant/qdrant

Embeddings

The project uses Google Gemini embeddings.

The configured embedding model is:

gemini-embedding-001

The Gemini API key is configured through an environment variable:

GEMINI_API_KEY=your_api_key_here

The .env file must never be committed to Git.

Semantic Search

The system performs semantic retrieval rather than simple keyword matching.

For example, a query such as:

How are complex numbers used in engineering?

can retrieve content discussing:

AC circuits
control systems
signal processing
complex exponentials

even when the exact wording of the query does not appear in the document.

Search results are ranked by vector similarity score.

Confidence Filtering

The search system uses a similarity confidence threshold to reduce irrelevant results.

The current threshold is approximately:

0.60

Relevant queries can produce scores such as:

0.72
0.76
0.79

Low-confidence results below the configured threshold are filtered.

When no sufficiently relevant result is found, the system can return:

No confident match found.

This prevents unrelated document content from being presented as a confident answer.

Retrieval Evaluation

A five-query evaluation dataset was used to measure retrieval quality.

The evaluation checks whether at least one expected relevant page appears within the top three retrieved results.

The current evaluation result is:

====================
Hit@3: 5/5
Hit@3 score: 100.00%
====================

Evaluation Queries

Test 1

What is a complex variable?

Expected page:

[2]

Result:

Retrieved pages: [2, 2, 2]
Hit@3: YES

Test 2

What are the Cauchy-Riemann equations?

Expected pages:

[2, 3]

Result:

Retrieved pages: [2, 3, 3]
Hit@3: YES

Test 3

How does the Laplace transform help engineering systems?

Expected page:

[4]

Result:

Retrieved pages: [4, 4, 4]
Hit@3: YES

Test 4

What is the difference between Laplace and Fourier transforms?

Expected pages:

[5, 7]

Result:

Retrieved pages: [7, 4, 5]
Hit@3: YES

Test 5

How is FFT used for audio noise reduction?

Expected page:

[6]

Result:

Retrieved pages: [6, 5, 6]
Hit@3: YES

Final Evaluation

Tests: 5
Successful hits: 5
Hit@3: 100%

The evaluation script can be run with:

python evaluation.py

MCP Server

The main MCP server is:

server.py

The server uses FastMCP and exposes:

search_notes
get_document
list_sources

The registered tools have been verified programmatically.

Example verification:

python -c "from server import mcp; print(list(mcp._tool_manager._tools.keys()))"

Expected output:

['search_notes', 'get_document', 'list_sources']

Running MCP Inspector

The MCP server can be tested using MCP Inspector:

mcp dev server.py

The MCP Inspector can then be used to:

• Discover the available tools

• Test search_notes

• Test get_document

• Test list_sources

• Inspect tool arguments

• Inspect returned results

Project Structure

The current project structure includes the MCP server, backend API, frontend, document processing services, and evaluation pipeline.

Personal-Knowledge-MCP/
│
├── backend/
│   ├── main.py
│   ├── auth.py
│   ├── models.py
│   └── database.py
│
├── frontend/
│   └── ...
│
├── services/
│   ├── chunking.py
│   ├── embedding.py
│   ├── pdf_reader.py
│   └── qdrant_service.py
│
├── documents/
│   └── users/
│       └── ...
│
├── .env
├── .gitignore
├── evaluation.py
├── ingest.py
├── requirements.txt
├── server.py
└── README.md

Installation

1. Clone the project

git clone <repository-url>
cd Personal-Knowledge-MCP

2. Create a virtual environment

python -m venv .venv

Activate it:

.venv\Scripts\Activate.ps1

3. Install dependencies

python -m pip install -r requirements.txt

Main dependencies include:

mcp
qdrant-client
google-genai
PyMuPDF
langchain-text-splitters
fastapi
uvicorn
python-multipart
python-jose
sqlalchemy
python-dotenv
email-validator

Environment Configuration

Create a .env file in the project root.

Example:

GEMINI_API_KEY=your_api_key_here

Additional backend/database configuration can be stored in environment variables as required by the application.

Never commit secret API keys to Git.

Running the Backend

Activate the virtual environment:

.venv\Scripts\Activate.ps1

Start FastAPI:

uvicorn backend.main:app --reload

Verify the API:

http://127.0.0.1:8000/health

Open interactive API documentation:

http://127.0.0.1:8000/docs

Running the MCP Server

For MCP development and testing:

mcp dev server.py

Available MCP tools:

search_notes
get_document
list_sources

Testing

The project has been syntax-checked across the major components.

Examples:

python -m py_compile server.py
python -m py_compile ingest.py
python -m py_compile evaluation.py
python -m py_compile backend\main.py
python -m py_compile backend\auth.py
python -m py_compile backend\models.py
python -m py_compile backend\database.py
python -m py_compile services\embedding.py
python -m py_compile services\pdf_reader.py
python -m py_compile services\qdrant_service.py

The retrieval evaluation also passes:

Hit@3: 5/5
Hit@3 score: 100.00%

Current Demonstration Corpus

The current demonstration document is:

Complex_Variables_Project_Report.pdf

The document contains:

7 pages

and the indexed collection contains approximately:

30 chunks

The document covers topics including:

• Complex variables

• Complex numbers

• Cauchy-Riemann equations

• Laplace transforms

• Fourier transforms

• FFT

• Engineering applications

• Audio signal processing

Security

The project includes authentication for protected backend endpoints.

Security practices include:

• API keys stored in .env

• .env excluded through .gitignore

• Bearer-token authentication for protected API routes

• User documents stored separately under documents/users/

• Secrets are not intended to be committed to Git

A request to a protected endpoint without authentication correctly returns:

401 Unauthorized

with:

{
  "detail": "Not authenticated"
}

Technologies

• Python

• FastMCP

• Model Context Protocol (MCP)

• FastAPI

• Google Gemini

• Gemini gemini-embedding-001

• Qdrant

• PyMuPDF

• LangChain Text Splitters

• SQLAlchemy

• JWT/Bearer Authentication

• Docker

• MCP Inspector

Project Outcome

The project demonstrates a complete semantic knowledge-base pipeline:

User Document
     ↓
PDF Extraction
     ↓
Chunking
     ↓
Gemini Embeddings
     ↓
Qdrant Vector Database
     ↓
Semantic Retrieval
     ↓
MCP Tools
     ↓
MCP Client / Inspector

The system successfully retrieves relevant document content using semantic similarity and provides source/page citations.

The current retrieval evaluation achieves:

Hit@3 = 100%

with all five evaluation queries successfully retrieving an expected relevant page within the top three results.

Future Improvements

Possible future improvements include:

• Support Markdown and TXT documents

• Improve duplicate-chunk handling

• Add persistent document IDs

• Expand the evaluation dataset

• Add additional retrieval metrics

• Support multiple document collections

• Add Qdrant Cloud deployment

• Add richer frontend search and document-management features

• Add document deletion and re-indexing controls

Submission Summary

This project fulfills the core Personal Knowledge-Base MCP Server requirements by providing:

• A real student-owned document corpus

• PDF ingestion

• Sensible document chunking

• Gemini-based embeddings

• Qdrant vector storage

• Semantic search

• Source and page citations

• Full document retrieval

• Indexed source listing

• FastMCP MCP server

• MCP Inspector compatibility

• Retrieval evaluation

• 100% Hit@3 evaluation score

• Authenticated backend for document management

The core MCP functionality is implemented and verified through the available tools and evaluation pipeline.