Rework

Why

When an AI agent works through a task list, "I finished X" is unverifiable by default. The agent could have written a tautological test (assert True), a test that never actually ran, or a test that passes regardless of whether the underlying logic is correct. A self-reported checkbox and a real fix look identical from the outside.

Rework closes that gap. It's an MCP server — plain enough that Claude Code, Claude Desktop, or any other MCP client can drive it — that forces every completion claim through one of two paths:

• Automated proof: a real FAIL_TO_PASS/PASS_TO_PASS test cycle, plus mandatory mutation-checking that deliberately breaks the target code to confirm the test would actually catch a real bug.

• An explicit, recorded judgment call — human or claude_review — for the cases automated proof genuinely can't reach.

Related MCP server: AI Workbench MCP

The 5 tools

project_list and project_get exist because of a real gap found in production use: project_status/project_check both require an exact project_id with no way to discover what IDs exist, and project_status only ever reports which bucket a criterion falls into — never the actual requirement text behind it. Any fresh session, or a second client with no memory of the original project_define() call, needs both to recover context.

The core loop

1. project_list()                                 discover what project_ids exist
2. project_define(project_id, goal, dod, ...)      interrogate a goal into real criteria
3. project_check(project_id, criterion_id, ...)    verify one criterion, repeated as work happens
4. project_get(project_id)                         recall a criterion's given/when/then
5. project_status(project_id)                      read the honest, unblended percentage

1 · project_define — interrogation before invention

Turns a vague goal into Given-When-Then acceptance criteria, but only after real interrogation:

• Every criterion must trace back to something the user actually said (grounding field) — nothing is invented from nothing.

• Every criterion is self-audited on two axes before submission: how underspecified it is, and how gameable it is. Score too high on either and it's rejected.

• If more than half the submitted criteria fail this audit, the whole batch is rejected — that signals the interrogation itself was too shallow, not that individual criteria need patching.

• Given-When-Then text is scanned for unquantified adjectives ("secure," "fast," "robust," ...) — rejected because they're not measurable, whole-word matched so it doesn't false-positive on things like "fastener."

• Duplicate criterion IDs, empty submissions, and a missing/non-string project_id are all rejected outright; malformed input produces a clean structured error, never a crash.

• reviewer_level picks which persona claude_review must embody for this project's whole lifetime.

2 · project_check — the load-bearing tool

Runs actual verification for one criterion, called once per criterion as real work happens:

• FAIL_TO_PASS / PASS_TO_PASS verification (SWE-bench style): a test must be shown to have failed before the fix and pass after — either via a real pre-change command re-run, or an explicit, trusted-not-verified assertion.

• Mutation checking is mandatory before a criterion's first real verification. Rework deliberately breaks the target code and confirms the test actually catches the break. A test that can't catch a broken implementation isn't real verification — it might be tautological. The engine tries, in order: comparators (==, <, ...), unary negation (!x), and — only as a fallback — binary arithmetic (+ - * /), so a pure-arithmetic function like return a + b is still mutation-checkable instead of always routing to review.

• Non-functional-risk gating: criteria flagged security/crypto/concurrency never pass on a plain green test alone — they require an explicit property test or human review.

• Editing the test file itself to make it pass triggers an automatic fail — the classic way agents cheat verification.

• When mutation-checking can't cleanly resolve, the tool doesn't guess — it routes to human_review or claude_review.

Language support

Reviewer levels

reviewer_level doesn't change any code path — it changes the bar the reviewing agent must hold itself to, and enforces a rising minimum length on claude_review_reasoning so a rubber-stamp verdict can't sneak through:

3 · project_status — the honest number

Read-only, safe to call anytime. percent_complete is calculated strictly from automated verified_complete criteria — never blending in verified_complete_reviewed (judgment calls) or fallback_complete (env errors, human-approved fallbacks). Criteria with corrupted/unrecognized status values are surfaced with a warning instead of silently vanishing from the count.

4 · project_list — discovery

Takes no arguments. Returns every tracked project's project_id, goal, reviewer_level, and a compact status summary. A corrupted project file is reported in its own errors[] list rather than crashing the whole call.

5 · project_get — recall

Returns a project's full stored definition — goal, dod, reviewer_level, and every criterion in full: feature, given_when_then, grounding, self_audit, non_functional_risk, status, and the complete verification record, including any claude_review reasoning.

Desktop dashboard

ui/ is an optional Tauri + React desktop app: a read-only status view over every tracked project, plus a Review button on each criterion that opens a real terminal running claude with a fully-briefed audit prompt — goal, given/when/then, grounding, current status, prior fallback/review history, and a pointer to AGENT.md — not a bare "check this" one-liner.

./setup.ps1 -InstallUI

This builds a real Windows installer and installs it silently (no admin prompt — currentUser mode). Rework shows up in Start Menu and Apps & Features like any normal installed program. The installed app finds your tracked projects via a small config file (%APPDATA%\Rework\repo_root.txt) that setup.ps1 writes, since an installed .exe no longer lives near the repo the way it does in dev mode.

Setup

Prerequisites

Backend — Windows, one command

./setup.ps1

This creates Rework's own venv, installs dependencies, registers the MCP server with Claude Code (claude mcp add, user-scoped) and Claude Desktop (auto-detects both legacy and packaged/MSIX-style install locations), installs AGENT.md as a Claude Code skill, and packages rework-skill.zip for Claude Desktop. It's idempotent — safe to re-run any time; existing rework registrations are replaced, not duplicated, and every other entry in either config is left untouched.

You'll be asked to pick a default claude_review evaluator persona (standard / skeptical / foreman, Enter for standard) — this is only a recommendation baked into the installed skill; each project still sets its own reviewer_level at project_define() time.

After it finishes: fully quit and reopen Claude Code and Claude Desktop (not just close the window) so both pick up the new MCP server.

Backend — manual / other platforms

python -m venv venv
venv/Scripts/activate   # or source venv/bin/activate on macOS/Linux
pip install -r requirements.txt
claude mcp add --scope user rework -- <path to venv python> <path to server.py>

For Claude Desktop, add the same command/args pair by hand to mcpServers in its config — %APPDATA%\Claude\claude_desktop_config.json on a traditional install, or %LOCALAPPDATA%\Packages\Claude_<hash>\LocalCache\Roaming\Claude\claude_desktop_config.json on a packaged/Store install.

Desktop dashboard install

The dashboard (ui/) is optional and off by default — building it is a multi-minute Rust release compile, so setup.ps1 skips it unless asked.

Install it as a real Windows app (recommended):

./setup.ps1 -InstallUI

This builds ui/ into a signed-locally NSIS installer and runs it silently (currentUser mode — no admin prompt). Rework then shows up in Start Menu and Apps & Features like any normal installed program, at %LOCALAPPDATA%\Rework. Safe to re-run — it reinstalls over the previous version.

Run it in dev mode instead (no installer, live-reloads on change):

cd ui
npm install
npm run tauri dev

Build the installer yourself without running it:

cd ui
npm install
npm run tauri build -- --bundles nsis
# installer lands in ui/src-tauri/target/release/bundle/nsis/*-setup.exe

Running tests

venv/Scripts/python -m pytest tests/

130 tests, all passing.

Seen in action

Because Rework is a plain MCP server, any MCP client can drive it — not just Claude Code. Here's Claude Desktop given one prompt ("pick a random tracked project, show its full status and every criterion's given/when/then, then write an action plan"), using project_list → project_get → project_status end to end with no prior context about the project:

It correctly explains why two criteria sit at verified_complete_reviewed instead of the automated verified_complete (their functions have no comparator for mutation-testing to flip — regex matching and file I/O), reports the real given_when_then for every criterion, and — without being asked to look for it — flags that none of the project's Definition-of-Done items are actually tracked by any individual criterion:

Proven on real work

Not just designed on paper — hardened against bugs found by actually running it:

• Tracked its own build, a bug-fix patch set, and a from-scratch PDF-to-Markdown converter built specifically to exercise the full loop end-to-end, including a real case where claude_review correctly flagged a test gap as not_trustworthy before it was fixed and re-verified honestly.

• Stress-tested with concurrent writes, 500-criterion payloads, 100KB strings, malformed/adversarial input, and repeated mutation cycles — a missing/non-string project_id used to silently write a garbage file to disk instead of being rejected, and a non-bool non_functional_risk used to be silently accepted despite gating on it as a plain truthy check. Both fixed.

• Found and fixed a real subprocess bug in production use: the MCP server's own long-lived stdin pipe was being inherited by every test subprocess it spawned, so a child that touched stdin for any reason could hang forever and get reported as a false failed — identical in symptom to a command that ran instantly outside the server. Fixed with stdin=DEVNULL and full process-tree termination on timeout, shared by all three mutation checkers.

Project structure

server.py         MCP entrypoint (FastMCP), registers the 5 tools
core/             criteria schema/validation, evaluator personas, mutation checkers, storage
tools/            project_define / project_check / project_status / project_list / project_get implementations
tests/            pytest suite (130 tests)
ui/               optional Tauri + React desktop dashboard
setup.ps1         one-shot Windows install/registration script
AGENT.md          operating manual for an agent driving Rework
REF SHEET- WHEN UNSURE/   original design spec this implements