nautilus-compass

Open-source memory & reliability layer for AI agents. Long-term memory that now beats mem0 on all three LongMemEval-S metrics while staying fully local & 14× cheaper — plus drift detection and cross-agent contracts that no other memory layer ships.

Plugin for Claude Code / Desktop · Cline · Cursor · Continue.dev · Zed · any MCP client.

Built by Nautilus Platform · open agent ecosystem · join as agent →

🇬🇧 English (this file) · 🇨🇳 中文

What this is (2026-08 state)

Three pillars, one plugin:

1 · Black-box long-term memory — now with SOTA retrieval. Raw text embedded locally with BGE-m3. No extraction LLM at ingest, no graph, no data leaving your machine. In Aug 2026 we added utterance-routed chunk retrieval: single-session and knowledge-update questions route to turn-window chunks (the answer usually lives in ONE user turn; whole-session embedding dilutes it), everything else uses session-level hybrid (BM25 + dense RRF). Result on LongMemEval-S full 500 questions, same-question head-to-head vs mem0 2.0.19 (both sides infer=False, each on its own default embedder — bge-m3 vs vertexai text-embedding-005 — our reproduction):

One-command reproduction (retrieval-only, no LLM calls, CPU works / GPU recommended for the full 500):

bash scripts/reproduce_lmes_retrieval.sh            # full 500
SUBSET=12 bash scripts/reproduce_lmes_retrieval.sh  # smoke

The same utterance ammo overtakes mem0 on its own home benchmark (LOCOMO-10, n=1986: 0.644 / 0.890 vs 0.592 / 0.802) and fixes the single-session collapse on LongMemEval-M (0.20 → 1.00). Full evidence chain with per-type breakdowns and every config flag: docs/evidence/headhead_mem0_full500_20260826.json — including the experiments that failed (cross-encoder reranking hurts on this corpus; candidate-pool K is a no-op; Qwen3-0.6B swap is a wash).

2 · Drift detection — the half nobody else solves. Memory recalled doesn't stop the AI from breaking the rule this time. compass scores every prompt against an anchor set of real failure patterns (25 positive + 35 negative) before the agent acts. AUC 0.83 held-out, p95 latency <50 ms, fire rate 0.5% in production traffic. White-box layers abstract prompts into facts before drift becomes checkable — structurally out of their reach.

3 · Cross-agent contracts + governance. When you run multiple agents (or multiple Claude dialogs) on shared files, compass derives implicit contracts from handoff files, tracks closure, and audits for fake-closure / red drift. A 4-dialog 28-hour field study lives in docs/case_study_4dialog_compass.md.

The trade that flipped: earlier versions traded −30 points on LongMemEval-S for local deployment and cost. As of 2026-08 there is no trade — full sweep at 1/14 the reproduction cost (~$3.50 per 500 questions vs $50+ for GPT-4o-judged stacks). Full argument: paper/BLACKBOX_VS_WHITEBOX.md.

Related MCP server: Recall

Quickstart

30 seconds (Claude Code / Desktop · local daemon)

git clone https://github.com/chunxiaoxx/nautilus-compass ~/.claude/plugins/nautilus-compass
bash ~/.claude/plugins/nautilus-compass/install.sh

# start the BGE-m3 daemon (one-time per boot)
bash ~/.claude/plugins/nautilus-compass/daemon_start.sh

Deploy notes (field-verified pitfalls, 2026-08-28):

• COMPASS_USE_INOTIFY=0 disables new-file discovery — recalls won't see fresh writes, silently. Only set it if you know why; the daemon logs a WARNING when it's off.

• drift fails loudly now: if anchors.json is missing from the plugin dir, responses carry drift.anchors_error + should_alert: true (it used to silently return "no risk" — that was a security hole).

• Token changes to tokens.json hot-reload via mtime check (no systemd restart needed since v3.1.0).

• First recall after daemon idle may take up to 90 s (model cold-load); the MCP client auto-retries once with the extended timeout.

The installer wires three hooks into ~/.claude/settings.json:

• UserPromptSubmit → time-bucketed memory recall + drift check

• PostToolUse → mid-session writer

• Stop → end-of-session summary (writes a session battle-report to ~/.claude/.cache/compass-last-session.txt)

Slash commands: /compass-verify · /compass-drift · /compass-recall · /compass-search · /compass-status.

Any other MCP client

python ~/.claude/plugins/nautilus-compass/scripts/install_to_agent.py

Auto-detects Claude Desktop, Cursor, Cline, Continue.dev, Zed and patches their MCP config. Per-agent copy-paste configs: docs/AGENT_ONBOARDING.md · raw protocol: docs/mcp-usage.md.

Cloud-hosted (open beta · self-serve)

The hosted gateway — MCP over HTTPS with scoped tokens and per-user memory isolation — is open for self-serve registration: sign up at https://compass.nautilus.social/signup, then create a token in the web console (or POST /tokens). Scopes are server-bound to your own space (read+write); cross-user access is denied and verified by automated probes. Design: docs/plans/2026-08-30-multi-tenant-memory-design.md.

MCP endpoint: https://compass.nautilus.social/mcp/ (Bearer token · streamable-http). A2A discovery: curl https://compass.nautilus.social/.well-known/agent.json

Nautilus platform agents (cloud ssh quickstart)

For agents on machines with ssh access to your Nautilus cloud box — generates a scoped token, wires the cloud MCP bridge, writes .mcp.json, and runs an end-to-end self-check. Add --hud to install the fused status line (live recall hit-counter 🧠, drift state, 5-min traffic).

bash ~/.claude/plugins/nautilus-compass/ops/agent_quickstart.sh my-agent

Headline numbers

We deliberately report Run 1 (44.4%) as the EverMemBench headline to avoid cherry-picking; cross-run mean 45.84% clears MemOS by +3.3pt. Dual-run + Gemini cross-judge sensitivity analysis: paper/sections/paper2_06_5_evermembench.tex.

Try it without installing: live drift-detection + Merkle-integrity demo at huggingface.co/spaces/chunxiaox/nautilus-compass (CPU only · metadata-mode jaccard fallback · no signup).

Reproduce the numbers — eval dataset (behavioral anchors + labeled traces + LongMemEval-S / EverMemBench scoring) on the Hub: huggingface.co/datasets/chunxiaox/nautilus-compass-test-data

from datasets import load_dataset
ds = load_dataset("chunxiaox/nautilus-compass-test-data")

Benchmark entrypoint: bash ops/bench_all.sh l0 (fast layer, no GPU) · bash ops/bench_all.sh l1 30 (LongMemEval subset). Retrieval levers are env-switched in tests/eval_longmemeval_accuracy.py (ZMM_UTTERANCE_RETRIEVE / ZMM_UTTERANCE_TYPES / ZMM_HYBRID / ZMM_RETRIEVE_K / ZMM_DATE_ANCHOR / ZMM_EMBED_CACHE).

Glossary

Terms this project coined and uses precisely. Other teams are welcome to use them for their own systems — that's what terms are for.

Judge hygiene — the discipline of keeping an LLM judge trustworthy: preregistered criteria, function-level smoke tests, silent-failure detection, dual accounting, confidence intervals. If your benchmark uses an LLM judge without these, the leaderboard is fiction. See the judging protocol · paper2.

Write-time wager — compressing or summarizing memory at write time is a bet on the future query distribution, which is structurally unknowable. That's why the compass write path makes zero LLM calls; all intelligence lives at read time (e2e 42.6% → 75.4% on identical memories and questions).

Dual accounting — every headline score is reported twice: full set, and judge-outage-excluded set (75.4% / 81.6%). A single number hides judge failures; two numbers disclose them.

How it works

            User prompt: "Fix bug X for me"
                         │
                         ▼
       ┌─────────────────────────────────────┐
       │  UserPromptSubmit Hook (this plugin)│
       └─────────────────────────────────────┘
                         │
            ┌────────────┼────────────┐
            ▼            ▼            ▼
       ┌────────┐  ┌─────────┐  ┌──────────┐
       │ recall │  │  drift  │  │ profile  │
       │ memory │  │  check  │  │ aggregate│
       └────────┘  └─────────┘  └──────────┘
                         │
                         ▼
       Hooks inject results into Claude's system prompt:
       - Time-bucketed past memory (BGE-m3 semantic + keyword hybrid)
       - Drift score + nearest negative anchor (if score < threshold)
       - Profile facts ("you have 3 unfinished tasks in this repo")
                         │
                         ▼
            Claude answers — with full context loaded

Drift detector: each prompt vs anchor set (real failure transcripts), BGE-m3 cosine. AUC 0.83 held-out.

What's exposed (MCP tools)

17 tools — core seven:

Latencies are local-daemon figures. Over the public HTTPS MCP endpoint (https://compass.nautilus.social/mcp/) add TLS + WAN round-trip: measured p50 ≈ 0.9–1.7 s per call (2026-08-28 field test).

Plus: thread_recall · proof_of_impact · long_task · platform bridge (submit_platform_task / ingest_platform_task_result) · governance (governance_dispatch / governance_audit / governance_lock_check · governance_plan) · add_worker. JSON-RPC 2.0 over stdio / TCP / TLS / mTLS; notifications/*, logging/setLevel, resources/* spec-complete. Full guide: docs/mcp-usage.md.

Token scopes (v2.3.1)

Tokens are scoped, not global. ops/compass_token_admin.py grant <agent> --scopes read:<project>,write:<project> issues a least-privilege token; read:* (all-project recall, incl. scope=user) requires an explicit --yes-i-want-star. The HTTP server enforces scopes per call (fail-closed); legacy list-format tokens map to full access for backward compatibility. The quickstart script signs read-only, current-project tokens by default.

Comparison

2026 newcomers not yet same-machine reproduced by us: Hindsight, Supermemory (self-reports LongMemEval SOTA), Cognee, LangMem, Membase — rows pending; their published numbers use their own harnesses and are not directly comparable to our head-to-head protocol. | Self-host + hosted both | ✅ | ☁ only | ✅ | ☁ only | ✅ | OSS only | OSS only | | License | Mod. MIT | Apache | Apache | proprietary | MIT | Apache | MIT |

n/r = not reported in their published evaluations. Smriti is a team conversation archive — different scope, listed for completeness.

Case study · 4-dialog OSS multi-agent reliability

28 hours, four Claude Code dialogs on shared filesystem protocols: drift fired 314×/7d (act-on rate instrumented), contract cnt_compass_soul_sub_a1 closed in 17.92h vs 6d21h budget, 13 plan-dup audits saved ~40-50h, first cross-dialog L4 fire settled 50 NAU. Field log

• 7 generalizable patterns: docs/case_study_4dialog_compass.md.

Advanced (opt-in surface)

Every fired alert gets a stable alert_id in .cache/drift_mitigation_log.jsonl. Acknowledge via feedback.py log <alert_id> fp|tp; audit_kpi.py reports act_on_rate(window_hours) (target ≥0.70; <0.30 = cry-wolf → raise threshold or retrain anchors).

from audit_kpi import act_on_rate
m = act_on_rate(window_hours=168)
assert m["rate"] >= 0.70

With no opt-in env set, daemon behavior is byte-equal to v2.0.1 — gated by tests/test_llm_opt_in.py on every PR.

Deterministic v3 surface (always on): typed knowledge graph layer (NO-OP until built), confidence scoring + contradiction hook, MEMORY_REPORT.md auto-gen, implementation_notes frontmatter. Registry: llm_opt_in.py.

OSS↔platform bridge without a new HTTP server: submit_platform_task (compass → platform queue, file-based or HTTP when COMPASS_PLATFORM_QUEUE_URL is set) · ingest_platform_task_result (platform → compass, searchable via recall). Round-trip demo: python examples/platform_flywheel_demo.py.

V7 governance (multi-executor deployments): governance_dispatch (decompose 1 task → N routed sub-tasks) · governance_audit (fake-closure / red-drift scan) · governance_lock_check (SHA256 lock on the L0 core). Demo: python examples/v7_governance_demo.py. Contract details: docs/PLATFORM_HANDSHAKE.md.

v3.0.0 · "from memory library to evolution engine" — same system closing the loop: memories feed a extract fuel → external verdict → distill cycle. Semantic-recall revival (Windows torch long-path fix), GOAL-SSOT ledger + hourly heartbeat, cloud capacity root-cause fixes (load 10-14 → 1.x), daemon atomic pkl + per-project locks, paired-control evidence (tribal-fact retrieval 0/3 → 3/3), fused HUD, 30-second quickstart.

v2.1.0 · drift v2 + line reconciliation — cry-wolf fix (fire rate 64.5% → 0.5% via rule-hit OR drift_score < −0.07), cross-agent contract scanner (L4 substrate), L3 tier promotion + PoI, daemon hardening (bounded pools, in-flight semaphore, BM25+vector RRF opt-in).

v2.0.0 · Opinionated EvoMap — deterministic lifecycle layer on the black-box base. No LLM at ingest / tier promotion / forgetting; no vendoring of GBrain/OpenViking; no graph rerank for closed haystacks (cost −6.2pt in v0.8 — paper/RESULTS_v0.8.md).

Full notes: CHANGELOG.md · release: v3.0.0

Documentation

• docs/AGENT_ONBOARDING.md — per-agent install configs (6 platforms + 3 frameworks)

• docs/mcp-usage.md — raw MCP protocol guide, TLS setup, RBAC

• docs/PLATFORM_HANDSHAKE.md — OSS↔SaaS coordination contract

• docs/evidence/ — raw benchmark evidence files (JSON, per-question rows)

• paper/ — two papers (drift detection + memory pipeline) and eval scripts

• ops/GPU_EVAL_RECIPE_4090.md — 12-minute rented-GPU benchmark recipe

• CHANGELOG.md · CONTRIBUTING.md

Citation

Paper 1 · drift detection:

@misc{nautiluscompass-drift-2026,
  title  = {Nautilus Compass: Black-box Persona Drift Detection
            for Production LLM Agents},
  author = {Chunxiao Wang},
  year   = {2026},
  note   = {Yiluo Technology Co., Ltd.},
  howpublished = {\url{https://github.com/chunxiaoxx/nautilus-compass}}
}

Paper 2 · memory pipeline + EverMemBench cross-bench:

@misc{nautiluscompass-memrecall-2026,
  title  = {Closing the Memory Recall Gap with Chinese LLMs:
            A Multi-Stage Retrieval Pipeline Achieving Zep-SOTA Performance
            on LongMemEval-S at 1/15 Cost},
  author = {Chunxiao Wang},
  year   = {2026},
  note   = {Yiluo Technology Co., Ltd.},
  howpublished = {\url{https://github.com/chunxiaoxx/nautilus-compass}}
}

Prior work we build on (cite as appropriate): BGE-m3 / BGE-Reranker (BAAI 2024) · Persona Vectors (Anthropic, arXiv:2507.21509, complementary white-box) · DPT-Agent (arXiv:2502.11882) · A-MEM (arXiv:2502.12110) · LongMemEval (Wu et al., NeurIPS 2024) · EverMemBench (Hu et al., 2026).

License

• Code, plugin, MCP wrapper, papers, scripts — Modified MIT License (MIT + trademark clause + hosted-service cap; self-hosting / internal deployment / personal use stay free forever — LICENSE)

• Behavioral anchor files (anchors*.json) — CC0 1.0 Universal (LICENSE-ANCHORS)

• Historical releases before this license's introduction were pure MIT.

Star history

Contributors

PRs welcome — see CONTRIBUTING.md.

Contact

• Author: Chunxiao Wang · Yiluo Technology Co., Ltd. · chunxiaoxx@gmail.com

• Issues: github.com/chunxiaoxx/nautilus-compass/issues

• Hosted gateway: compass.nautilus.social

• 中文文档: README.zh-CN.md