Substack Vault

English · 中文

Collect Substack newsletters from Zoho Mail, archive them locally, and analyze via MCP Streamable HTTP with cross-author search.

v0.1.0 — Initial release: Zoho sync, Web UI, article library, and MCP tools.

Features

Related MCP server: MailMesh MCP

Requirements

• Python 3.11+

• Zoho Mail account (accounts.zoho.com)

• A Self Client app in Zoho API Console

• (Optional) DeepSeek API key for in-app AI analysis

Quick start

git clone https://github.com/flap4fun/substack-vault.git
cd substack-vault

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
# Optional: OCR for image-only Substack emails (e.g. weekly image posts)
pip install -e ".[ocr]"

cp .env.example .env               # Windows: copy .env.example .env

Optional: image OCR (Tesseract)

Some paid newsletters put the body in images. Vault OCRs those from Zoho email HTML (not the public Substack site, which may be paywalled).

1. Install Tesseract with language packs chi_tra and eng

2. pip install -e ".[ocr]"

3. Re-sync articles to re-fetch email HTML when needed and OCR images

Optional: historical archive (session cookie)

Email sync only sees recent inbox mail. To pull older posts from publications you already pay for:

1. Sign in to Substack in your browser

2. Copy connect.sid / substack.sid (DevTools → Application → Cookies) or export a JSON cookie array

3. In Settings, paste the cookie and save (Vault probes profile/self and encrypts it locally)

4. In Subscriptions, use Import from Substack then Pull history

Do not commit cookies. This version has no password / OTP auto-login. Use only your own subscribed content.

Start the server:

substack-vault

Open http://127.0.0.1:8765 in your browser. Use this URL directly — do not serve web/ with Live Server or another static file server.

Docker (optional)

Run the whole app (including Tesseract OCR) in a container — no Python, venv, or Tesseract install on the host:

cd substack-vault
cp .env.example .env        # compose reads secrets from this file
docker compose up -d --build

Open http://127.0.0.1:8765. The SQLite database and encrypted keys live in the Docker named volume substack-vault_vault-data, so data survives image rebuilds.

Useful commands:

docker compose logs -f        # follow logs
docker compose restart         # restart the app
docker compose down            # stop (data kept)
docker compose down -v         # stop AND delete all data

Notes:

• The host port is bound to 127.0.0.1 only. Keep it that way — the MCP endpoint has no authentication.

• Fill in VAULT_ZOHO_* (and optionally DeepSeek credentials in Settings) before connecting; compose passes the VAULT_ZOHO_* values from your .env into the container.

• OCR (Tesseract chi_tra+eng) is already inside the image, so image-only posts work out of the box.

• Substack outbound access uses VAULT_PROXY from .env (for example, http://host.docker.internal:7890); it does not depend on the Windows system-proxy toggle and tries direct access when the proxy is unavailable.

• Linux users who prefer direct file access can bind-mount instead of the named volume: replace the volumes: block in docker-compose.yml with - ~/.substack-vault:/data, after sudo chown -R 1000:1000 ~/.substack-vault.

Connect Zoho Mail

Self Client apps do not support browser OAuth redirects. You must generate a Grant Code manually.

1. Create a Self Client in Zoho API Console

• Type: Self Client

• Redirect URI: http://localhost:8765/oauth/zoho/callback (must match .env)

• Scopes (paste when generating code):

ZohoMail.messages.READ,
ZohoMail.accounts.READ,
ZohoMail.folders.READ,
offline_access

offline_access is required for refresh tokens; without it you must re-authorize when tokens expire.

2. Connect in the Web UI

1. Open 邮箱账号 (Mail accounts) → 连接 Zoho Mail

2. Select your data center (default: Global .com)

3. Enter Client ID, Client Secret, and Grant Code

4. After connecting, use 手动同步 (Sync now) in the top bar

Credentials can also go in .env (see Configuration). UI-saved credentials are stored locally as well.

3. Subscriptions and sync

1. 订阅管理 (Subscriptions) → 扫描发现 (Scan inbox) to discover Substack senders

2. Enable subscriptions you want to sync

3. 手动同步 or wait for the background job (default: every 30 minutes)

Web UI

AI analysis (DeepSeek)

1. Open Settings → AI / DeepSeek, set API Key (default base URL https://api.deepseek.com)

2. Open AI analysis, pick or edit a scheme (seeded: “跨作者观点对比”)

3. Choose date range, 1–N authors, 1–N keywords (e.g. tickers), then Run

4. Or from Articles, set filters and click Send to AI analysis

Model is fixed to deepseek-v4-flash. The API key is Fernet-encrypted locally (same as Zoho secrets).

Display timezone defaults to Asia/Shanghai (UTC+8); the database stores UTC.

Cursor MCP

With substack-vault running, add to Cursor MCP settings:

{
  "mcpServers": {
    "substack-vault": {
      "url": "http://127.0.0.1:8765/mcp"
    }
  }
}

Tools

Example: ask an agent to compare how different authors cover a topic, or summarize the past week’s posts.

Configuration

.env example (VAULT_ prefix):

# Zoho OAuth
VAULT_ZOHO_CLIENT_ID=
VAULT_ZOHO_CLIENT_SECRET=
VAULT_ZOHO_REDIRECT_URI=http://localhost:8765/oauth/zoho/callback

# Server
VAULT_HOST=127.0.0.1
VAULT_PORT=8765

# Optional Substack proxy fallback; direct Substack access is also built in
VAULT_PROXY=

# Sync (optional)
VAULT_SYNC_INTERVAL_MINUTES=30
VAULT_SYNC_LOOKBACK_DAYS=90
VAULT_SYNC_SUBSCRIBED_ONLY=true

Data directory

Contains vault.db (SQLite), encryption key, and OAuth credentials. Do not commit this directory.

Development

pip install -e ".[dev]"
pytest

Layout:

src/substack_vault/
  api/          FastAPI REST + static Web
  connectors/   Zoho Mail connector
  parsers/      Substack email parser
  sync/         Sync engine
  storage/      SQLAlchemy models & DB
  mcp/          MCP Streamable HTTP
web/            Frontend static assets
tests/          Unit & API tests
openspec/       Specs & change history (OpenSpec)

Roadmap

• Outlook and other mail providers

• RSS body enrichment

• Article tags and advanced filters

• Analysis run history / streaming UI / more LLM providers

• Multi-user / remote deployment

Limitations

• Zoho Mail only (international and regional data centers)

• Single-machine local deployment; no multi-tenant or cloud sync

• Substack HTML parsing depends on email templates; unusual layouts may lose formatting

• Image-body posts need local Tesseract (chi_tra+eng); OCR is best-effort

• Paid full text: recent mail via Zoho; older / paywalled posts via your own session cookie in Settings. Never commit cookies

• In-app AI supports DeepSeek official only; model fixed to deepseek-v4-flash

• MCP endpoint has no authentication — do not expose to the public internet

License

MIT © 2026 flap4fun