You can perform comprehensive engineering audits of code repositories. The server lets you list and retrieve rule domains; start and monitor audit runs with metadata; configure audits interactively via a local web page or headlessly via environment variables; record per-rule verdicts, findings, and coverage with mandatory completeness; view run progress; preview and (with explicit confirmation) file GitHub issues; submit feedback to the tool author; and render final self-contained HTML and JSON reports. All operations guard against accidental overwrites and require user consent for outward-facing actions.
Integrates with OpenAI Codex CLI to run standalone audits against engineering rules and load rules inline via AGENTS.md fragments.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@engineering-audit Audit the current repo against the engineering rules and generate a report"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
TL;DR: engineering-audit turns the AI coding assistant you already use (Claude Code, Codex CLI, Gemini CLI) into an engineering-practice auditor. It has two audit modes:
Full repository audit: your assistant sweeps a whole repository against a pack of sourced engineering rules and produces a self-contained HTML report. Every finding says what is wrong, why it matters and how to fix it, with the citation behind the claim attached, and can be filed as a GitHub issue or copied out for pasting anywhere.
Inline decision-time checks: one-line triggers in your assistant's context make it load the relevant rules at the moment you are making a matching decision (designing a schema, cutting a branch, shaping an API). No report, just the right rules at the right moment.
Starting a new project? The optional Engineering Grill uses the same rules before you build. Your assistant interviews you, works out which domains matter, and records the decisions and checks the project will need, without expecting you to understand the framework first.
Three complete rule domains ship in this repository, ready to run: data modelling, testing strategy and presenting data, 54 rules with their full source citations, in examples/taster-rules/. You can run a real audit right now with no sign-up; the full sixteen-domain, 260-rule pack is available on request (Rules access).
Need a walk through? There is a free interactive walkthrough → that takes you from a bare Windows, Mac or Linux PC to a finished audit of your own repository. It rehearses every step below in a simulated terminal, with the wrong turns coached, before you do them for real. No terminal experience assumed.
Save me the chit chat: show me how to install it →
From nothing to a first audit in five steps. The taster rules ship in this repository, so none of this needs access to the full pack.
Never touched a terminal? Rehearse these five steps first in the free interactive walkthrough linked at the top: it takes a non-technical founder from a bare Windows, Mac or Linux PC to a finished audit, in a simulated terminal with the wrong turns coached.
The tool works through the assistant you already drive: Claude Code or OpenAI Codex CLI (both proven end to end; standalone audits recorded 2026-08-09 and 2026-08-10, with a further Codex run on Linux 2026-08-19), or Gemini CLI (documented, not yet exercised end to end; see the support matrix). GitHub Copilot is not supported.
Dependency | Why you need it | Check it |
runs the MCP server via |
| |
git | clones this repository for the taster rules, and any other rules pack |
|
your assistant's CLI | drives the audit and hosts the MCP registration |
|
GitHub CLI | only if the assistant should file findings as GitHub issues; filing from the report page instead needs only a PAT in your browser |
|
There is no pip install and no npm anywhere: uvx fetches and runs the tagged release
directly.
Clone this repository. The taster pack (the three complete domains named in the TL;DR) is
a working rules directory at examples/taster-rules/:
git clone https://github.com/rodlunt/engineering-audit
cd engineering-auditHave the full pack instead? Registration below is identical, just point --rules-dir at
that clone's domains/ directory. See Rules access for how to ask.
Every command below is pinned to the current release tag (@v0.15.0) rather than the
moving main branch: an unpinned git dependency resolves to whatever main holds at
install time and silently moves on later cache refreshes. Find the latest tag on the
Releases page. Updating to a newer
tag is a deliberate act with its own command, below, not something that happens on a
git pull: the pin lives in your assistant's MCP registration, not in this repository.
Paths beginning with /path/to/ are placeholders. Replace them with the real absolute path
on your computer; run pwd inside a folder when you need to see its absolute path.
Register the server (swap in the taster path from Step 3, or your full-pack path):
claude mcp add engineering-audit --scope user -- uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 \
engineering-audit-mcp --rules-dir /path/to/engineering-audit/examples/taster-rules--scope user registers it for every repository. Without it claude mcp add defaults to
local scope, which registers the server for the current directory alone and leaves it
unavailable everywhere else. That failure is silent and lands later (issue #245): a skill
run in any other project reports list_domains unavailable with nothing pointing at scope
as the cause. If that happens, run claude mcp list in the project where it failed; if
engineering-audit is missing there but present in the directory you installed from, it
was registered without --scope user. Fix it with claude mcp remove engineering-audit
followed by the add command above.
To update to a newer tag, remove first and re-add. claude mcp add refuses to
overwrite an existing name (MCP server engineering-audit already exists in user config),
so changing the tag on its own is not enough:
claude mcp remove engineering-audit
claude mcp add engineering-audit --scope user -- uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 \
engineering-audit-mcp --rules-dir /path/to/engineering-audit/examples/taster-rulesThe change only takes effect in a new session. The one you are in keeps the server it started with, so verify after restarting rather than before:
claude mcp list | grep engineering-audit # must show the tag you just setbegin_run's response states tool_version too. If it names the old version, the session
predates the re-registration and the run is not exercising the build you think it is.
Install the audit skill (gives you a natural-language entry point: "audit this repo"):
cd /path/to/engineering-audit
scripts/install-skills.sh auditIt copies rather than symlinks, so the installed skill does not change under you when this
checkout switches branch. scripts/install-skills.sh --check reports when a copy is stale.
Full details: integrations/claude-code/.
Register the server (verified against codex-cli 0.114.0):
codex mcp add engineering-audit \
--env ENGINEERING_AUDIT_RULES_DIR=/path/to/engineering-audit/examples/taster-rules \
-- uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 engineering-audit-mcpInline mode: generate the trigger fragment and append it to your repo's AGENTS.md (or
~/.codex/AGENTS.md for all repos):
uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 engineering-audit-fragments \
--rules-dir /path/to/engineering-audit/examples/taster-rules --out-dir .
cat AGENTS-fragment.md >> AGENTS.mdStandalone audit: in a codex session, ask it to read AUDIT.md from this repository and
run the audit. Headless notes and caveats: integrations/codex/.
There is no packaged extension. Gemini CLI resolves an extension only from a repository
root, which would put a manifest, a GEMINI.md and a commands/ directory in the root of a
tool that also serves Claude Code and Codex, and none of it was ever exercised against a real
Gemini CLI. Registering the server by hand is one paste and has no such cost.
Add the server to ~/.gemini/settings.json (or a project-level .gemini/settings.json),
swapping in the taster path from Step 3:
{
"mcpServers": {
"engineering-audit": {
"command": "uvx",
"args": [
"--from",
"git+https://github.com/rodlunt/engineering-audit@v0.15.0",
"engineering-audit-mcp"
],
"env": {
"ENGINEERING_AUDIT_RULES_DIR": "/path/to/engineering-audit/examples/taster-rules"
}
}
}
}Then start gemini in the repository you want audited and paste:
Audit this repository against the engineering rules. Read AUDIT.md from the
engineering-audit repository and follow it, driving the engineering-audit MCP tools
through to a rendered report.Inline triggers work too, and need no extension: generate the fragment and merge it into
whichever GEMINI.md tier you want it to apply to.
uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 engineering-audit-fragments \
--rules-dir /path/to/engineering-audit/examples/taster-rules --out-dir .
cat GEMINI-fragment.md >> GEMINI.mdGemini support is documented, untested: Gemini CLI was not available to exercise any of
it. Check gemini --help before an unattended run. Details and caveats:
integrations/gemini/.
Install Engineering Grill after registering the MCP server if you want a guided, plain-English planning conversation before code is written. It reads the domains from the connected rules pack, so the taster pack gives a three-domain grill and the full rules pack gives the complete framework.
Skip the interactive configuration page by pointing ENGINEERING_AUDIT_CONFIG at a saved
configuration JSON (shape documented in AUDIT.md); get_config then returns
immediately. Example driver, Claude Code:
claude -p "Read AUDIT.md at <path> and audit this repository via the engineering-audit \
MCP tools." --mcp-config mcp.json --allowedTools "mcp__engineering-audit__*,Read,Glob,Grep"Every run also checks this repository's tags for a newer release, on by default (see
Security for what that discloses). On an air-gapped machine, or anywhere the
network cost or the call itself is unwanted, pass --no-update-check to engineering-audit-mcp
or set the ENGINEERING_AUDIT_NO_UPDATE_CHECK environment variable.
Two checks tell you whether what you are running is current: one for the tool, one for the rules
pack. Neither ever guesses. A check that could not run reports could-not-check, which is a
distinct state from current, so a stale build is never reported as fine. What that honesty does
not do is tell you when a check has gone blind, and both go blind on install shapes that look
perfectly ordinary.
Check | Attached when | Blind when | What you lose |
Tool build | installed from a git URL, which is what every command in Step 4 does | installed from a downloaded archive or a plain wheel, or run from a local or editable checkout | nothing warns you that a pin left on an old tag, or a stale |
Rules pack | the rules directory is a git clone with an | the rules arrived as a downloaded zip, were vendored into another repository, have no remote, or have uncommitted changes | nothing warns you that the run is judging your repository against superseded rules |
Following Step 3 and Step 4 as written keeps both attached: git clone for the rules, and
uvx --from git+...@<tag> for the tool. Any other shape still runs and still audits correctly.
It just cannot tell you it is out of date.
The provenance rows in the report header carry the answer for the run in front of you. Both
reading could-not-check at once is the combination to watch for, because that is a build of
unknown age judging your repository against rules of unknown age, with nothing able to detect
either.
Ask your assistant to audit the repository you have open ("audit this repo against the
engineering rules"), tick the domains on the configuration page that opens, wait for the
sweep, then open audit-output/report.html. What to expect while it runs is in
What a run looks like; what it costs in tokens is in
What a full run costs.
Related MCP server: code-review-mcp
Status: beta, and the label is meant literally. The interview shipped in v0.13.0 as a separate
interrogate skill and was folded into Engineering Grill by issue #239: one pre-build skill, not
two. Nobody has yet completed a full interactive session with it. Expect it to change.
An audit sweeps the rules over a repository that already exists. Engineering Grill runs them the other way, as questions about work that has not started. It classifies every domain the pack returns, derives the full question set from the rules of those whose triggers genuinely fire, then puts the highest-consequence questions first, one at a time, in the Hot Seat. The rest are held, not discarded: it names the total, offers a deep dive through everything remaining, and records what was answered, what was deferred and what was never asked.
Triage is global on purpose. One domain's third-best question routinely matters more than another's first, and a per-domain ranking cannot see that. Questions carry a reversibility grade and a blast radius so they can be compared across domains at all.
The rule that shapes the whole design is that no full domain document enters the conversation with the user. That is stated as an invariant rather than an architecture: sub-agents satisfy it where the host has them, serial read-and-discard satisfies it where the host does not, which is what keeps one skill shippable on more than one assistant.
Ten of the sixteen domains are design-time by their own Load this when: statements, so most of
the pack was already pointed at the moment before the code, with nothing to deliver it there.
It never starts a run. It calls list_domains and get_domain only, both of which work with no
run in progress, so it works on a directory that is not a repository yet or on nothing but a
description. Setup is in integrations/engineering-grill/, and
the optional hook that offers it when plan mode starts is in
integrations/claude-code/README.md.
What has been exercised: question derivation on three domains (d02, d01, d15) against one brief, the no-run guarantee, and the hook's failure paths.
What has never been run even once: the cross-domain triage. It is the newest part and the part everything else now depends on, so treat its output with more suspicion than the rest.
What has not: the other thirteen domains, and the interactive loop itself with a real person answering. Nobody has yet finished a full interrogation. Until they have, treat the shape of the session as unproven and the question quality as sampled rather than measured.
This is a beta of a Claude Code integration specifically. There is no Codex or Gemini equivalent,
and unlike audit there is no assistant-neutral protocol document behind it: the skill file is
the whole specification. That is deliberate for now and is the first thing to change if it earns
its place.
This documentation states functional behaviour (what the tool does) and the security and
privacy properties documented inline (how the access token is held, what telemetry you opt
into, what leaves your machine and when). The two human-facing surfaces, the configuration page
and the report page, target WCAG 2.2 Level AA. Applying the
same proven versus documented, untested distinction used below for the
assistant integrations: one criterion is proven, the rest are documented, untested. Contrast is
machine-checked on every test run, tests/test_report_stylesheet.py computes contrast ratios
from the CSS custom properties and asserts 4.5:1 in both the light and dark palettes. No
keyboard-only pass and no screen reader pass has been recorded against either surface, and
neither template currently sets an explicit minimum target size on its interactive controls, so
those criteria (and WCAG 2.2's target size, success criterion 2.5.8, specifically) have not been
checked by anyone. The target is real; only the contrast ratio has a recorded check behind it.
Performance is explicitly out of scope for now: no throughput or latency target is stated or
tested anywhere in this repository.
The tool is a local MCP server (Python, stdio). It points at a local directory of rule documents and serves them to the agent driving the audit. The agent supplies the judgement; the server supplies everything that must not depend on an LLM's memory: schema-validated finding capture (a rule the agent did not check can never be recorded as a pass), the configuration page, deterministic report rendering, source citations attached from the rules pack itself, and GitHub issue filing with an explicit confirmation step.
Two audit modes, plus an optional project-start skill:
Engineering Grill: before code is written, the assistant sorts the loaded domains by
relevance, asks framework-backed questions and records the resulting plan. It uses only the
read-only list_domains and get_domain tools and does not start an audit run.
Standalone audit: tick the domains to audit on a local configuration page (or supply
a saved config for headless runs), the agent sweeps the repository, and you get
report.html plus optional GitHub issues.
Inline: one-line triggers merged into your assistant's instruction context tell it to
call get_domain(...) at decision moments (designing a schema, cutting a branch, shaping
an API), so the rules arrive exactly when they are useful.
Assistant | Inline mode | Standalone audit |
Claude Code | proven (in daily use via skills) | proven (recorded run 2026-08-09) |
OpenAI Codex CLI | documented, untested | proven (recorded run 2026-08-10) |
Gemini CLI | documented, untested | documented, untested |
GitHub Copilot | unsupported | unsupported |
"Proven" means a recorded end-to-end run exists. "Documented, untested" means the integration follows the assistant's official documentation, with individually verified pieces labelled in the integration README, but no full audit has been exercised on it yet. Copilot is deliberately unsupported rather than silently absent.
A standalone audit is a conversation plus one browser page. From your seat:
Ask for the audit ("audit this repo against the engineering rules"). The assistant gathers run metadata and starts the run.
A configuration page opens in your browser (http://127.0.0.1:<port>/). Opening it
is best-effort: in a remote or display-less session no tab can appear, so the assistant
also prints the URL; open it yourself if nothing popped up. Tick the domains to audit,
choose report-only or GitHub issue filing, and submit. Nothing proceeds until you
submit: the tool never falls back to a domain selection you did not make.
The assistant sweeps the repository domain by domain. This is the slow part: minutes for a small repository and a few domains, longer for a big selection. You can ask for progress; it can report which domains are recorded and which remain.
Everything lands in audit-output/ inside the audited repository: report.html
(the deliverable, openable in any browser; the assistant offers to open it when the
run finishes) and run-state.json (the raw machine-readable results, which can
re-render the same report later via engineering-audit-render). If you chose GitHub
filing, the assistant previews the issues and asks before filing anything.
audit-output/ belongs to the audited repository, not to this tool. Commit it, ignore it
or delete it as that repository's own conventions dictate.
A full sweep is token-hungry, and the configuration page's domain tick boxes are the cost control: cost scales close to linearly with the domains you tick. Budget from these recorded runs rather than guessing, one row per host that has completed one:
Host | Tool version | Scope | Active time | Findings | Tokens |
Claude Code, Fable 5 orchestrating Sonnet subagents (this repository, 2026-08-09) | 0.4.0 (established from tag history, not stamped into a retained report; see docs/example-audit-cost.md) | all 16 domains, 260 rules | 47 minutes end to end, sweeps running four at a time | 33 (every one filed as a GitHub issue) | 2,010,691 subagent tokens, roughly 100k to 170k per domain (excludes the orchestrator) |
Codex CLI 0.147.0, gpt-5.6-sol at high reasoning effort (external React SPA, roughly 344 files, 2026-08-10) | 0.5.1 | all 16 domains of the standard pack | 19 minutes 21 seconds | 32 (122 rules could not be evaluated) | 6,172,397 input plus output, of which 96% is cached input; 269,293 non-cached input plus output |
Both rows predate v0.9.0 and v0.9.1, which added required per-finding and per-domain
output that neither run had to produce; see the comparability note in
docs/example-audit-cost.md before budgeting a current run from
either figure.
The two token columns are not the same measurement and must not be subtracted or averaged. Codex does not fan out to one subagent per domain the way the Claude Code skill does, and that single difference drives everything: fanning out gives each subagent a small fresh context and bills mostly uncached input, while staying in one long context re-reads a large accumulated context every turn. Hence Codex's 6.17M being 96 per cent cache reads. The Claude Code figure counts per-subagent totals and excludes the orchestrator conversation entirely. Use each row within itself, for the shape of run it describes. Full per-host detail, including what each run did and did not measure, is in docs/example-audit-cost.md.
Self-contained HTML, generated locally; nothing leaves your machine unless you choose to send or file it. It is a written report meant to be read top to bottom, in this order:
A computed headline, first thing on the page: one sentence naming what needs attention first, built from the run's own counts, with a second line saying what was set aside or could not be evaluated so a partial sweep cannot read as a clean bill of health.
Findings, sorted worst first rather than in the order they were recorded, each in three parts (the issue and location, why it matters, suggested fix) with the rule's citation appended automatically from the rules pack. The tool refuses to publish a finding whose rule carries no citation. Each finding also carries its domain's self-assessed confidence and whether that domain's rule text was fetched this run, so a finding from a shaky domain does not look identical to one from a solid one. The confidence never appears on its own: it ships with how many of that domain's rules could not be evaluated, out of how many, so a domain claiming high confidence over half a domain it could not check says so in the same breath. The four severity levels are defined on the page, and stated as assigned by the assistant named in the header rather than measured.
Issues: tick boxes to select findings, then file them to GitHub directly from the report (fine-grained PAT, used in memory only, sent only to api.github.com), or copy the selected set for pasting into an LLM or editor, or copy them one at a time. Only critical and high findings are ticked on load; nothing is hidden, and an already-filed finding shows unticked with a link to it.
A tool performance summary about the audit run itself, below the findings rather than above them: one table with a row per domain carrying rule verdicts, findings by severity, files inspected and skipped, confidence and fetch status, with the could-not-evaluate and not-applicable reasons in full, and an evidence boundary block naming what each domain did not read. Every number ships with its base, a domain set aside in full cannot read as a domain swept clean, an unchecked rule is never presented as a pass, and a value the tool was told rather than measured says so. Longer sections sit behind summaries that each carry their own numbers, so the detail is one click away rather than occupying the top of the page.
Feedback to the author: freeform text, two optional questions asking what you concluded from the report and what you would fix first, and tick-box consent over which run statistics accompany any of it. Everything here is off by default and never prefilled. Finding text never leaves your machine through this channel.
It prints sensibly too: the issue-filing section is dropped from print rather than clipped, with a line saying where to find it, and collapsed sections are expanded on paper.
A live example: docs/demo/report.html (download and open locally;
GitHub does not render raw HTML in the browser). Generated from the invented demo rules pack
in tests/fixture_pack, not a real audit against a real repository.
The author's rules pack covers sixteen decision domains, 260 rules in all, each rule carrying a cited source, a volatility tier and a verification date, and each domain proven against a real system before it is trusted:
# | Domain | Rules | Fires when you are... |
d01 | Designing a Data Model | 15 | modelling entities, choosing keys, constraints, normalising, writing DDL or migrations |
d02 | Eliciting and Specifying Requirements | 16 | deciding what to build, writing requirements or user stories, checking the right problem is being solved |
d03 | Modelling Structure and Behaviour Before Building | 15 | deciding what to diagram before coding, drawing or reviewing FMC/UML/SysML models |
d04 | Structuring Code and Applying Design Patterns | 14 | designing classes or modules, weighing a design pattern, choosing data structures or error handling |
d05 | Choosing What to Test and How Much | 18 | choosing test levels and coverage, weighing testing against risk, planning load or soak tests and CI gates |
d06 | Structuring a Repo, Branches and CI/CD | 15 | structuring a repository, writing CI/CD workflows, handling automation credentials, cutting releases |
d07 | Handling Untrusted Input and Secure Coding | 16 | writing code untrusted input can reach: forms, auth flows, credentials, sessions |
d08 | Threat Modelling and Security Risk Decision-Making | 15 | running a risk assessment, threat-modelling a system, prioritising vulnerabilities, justifying a control |
d09 | Responding When Something Breaks in Production | 16 | writing incident response plans, defining recovery objectives, running post-incident reviews |
d10 | Designing APIs and Service Contracts | 14 | creating or extending an HTTP API, choosing verbs and status codes, versioning or deprecating an interface |
d11 | Choosing Architecture and Deployment Topology | 16 | picking an application architecture, deciding VM/container/serverless topology, planning scaling and rollout |
d12 | Making an Ethical or Professional Judgement Call | 17 | facing pressure to cut a corner, decisions affecting users or the public, handling personal data |
d13 | Estimating and Pricing Work | 16 | scoping work before quoting, choosing estimation methods, setting contingency, defending an estimate |
d14 | Fault Diagnosis of a Running System | 19 | investigating an outage, a slow or wrong-answering service, or an intermittent bug |
d15 | Interface Design and Prototyping | 17 | laying out a screen, designing a form, writing button and error copy, deciding confirmation vs undo |
d16 | Presenting Data for Decisions | 21 | putting a number, chart or table in front of somebody who has to decide something |
Three complete domains (d01, d05, d16, 54 rules with their full source citations) are published in examples/taster-rules/ as point-in-time exports from the maintained pack. They are a working rules directory, and they are what the How to use steps register by default: run a real audit before asking for anything.
The full pack lives in a private repository with access granted per user (the maintained originals, their revision history and proving records). Ask via the rules pack access request form.
Once granted, the pack arrives the same way the taster did: a repository you clone.
git clone https://github.com/rodlunt/engineering-frameworkThen swap the rules path in your Step 4 registration for the clone's domains/ directory
(claude mcp remove then re-add for Claude Code; codex mcp add overwrites in place).
Updating the rules afterwards is git pull in that clone. Keep it a real clone rather
than a downloaded archive: the clone keeps its origin remote, so the run's rules-pack
staleness check stays attached, exactly as described under "What keeps the staleness
checks working".
The
tooling works with any rules directory in the expected format (**Trigger:** header,
### N. Title rules, Rule id: footers with Source: fragments), so you can also write
your own pack.
uv sync
uv run pytest -qCI runs the same suite on every push and pull request. Tests use an invented fixture rules pack; no private rule content exists in this repository. The renderer and configuration page are deterministic and fully testable with no LLM involved.
This is a solo-maintainer repository, and its merge gate is deliberately CI-only: every change
lands via a pull request that must pass the check status; no human review requirement is
configured.
Up to now, the PR description has been the deliberate change record for this project: each PR
body explains what changed and why, and that has been treated as sufficient in place of a
separate issue-linked history. From now on, every PR that has a tracking issue links it with a
Closes #N line (or Fixes #N), so the issue tracker and the merge history stay in step
instead of relying on the PR description alone. Release PRs (chore(release): X.Y.Z) and
housekeeping PRs with no issue behind them are the standing exception: there is nothing for
either to close, so no keyword is expected on them. Nothing currently checks for the keyword on
a PR that does carry a tracking issue; a CI check failing a PR with no closing keyword and no
opt-out label is an option the maintainer can pick up separately, not something this policy
statement adds on its own.
evals/ holds a deterministic scorer for audit quality: a small fictional golden repository with
known planted findings and controls, and engineering-audit-eval to check a run-state.json
against them. The scorer is CI-safe and has its own tests; the audit run that feeds it calls a
real LLM and is run and checked by hand. See evals/README.md.
Thin CLI wrapper driving an agent CLI headlessly end to end (a manual, scripted version of this now lives in evals/README.md; a first-class wrapper command is still open).
Remotely served rules with revocable access.
Codex and Gemini support-matrix rows moving to proven once live runs are recorded.
The tooling in this repository (the MCP server, the deterministic report renderer, the configuration page, and every supporting script) is licensed under Apache-2.0. Rules packs are licensed separately and are not covered by this repository's licence; see Rules access.
Built by Rodney Lunt. If this saved you some time, you can buy me a coffee.
Start a fresh audit run and create its output directory, or resume an interrupted one.
assistant/model/repo_name/repo_commit/started are supplied by the
calling agent; tool_version defaults to the installed package version
if omitted. repo_dir is
the path to the repository being audited, on disk; it is optional,
but file_issues needs it to detect the GitHub repository to file
against, unless a repo is given explicitly on that call instead.
Calling this twice without finishing the first run (via
render_report) is an error, since it would silently discard whatever
domain results have already been recorded; pass replace=True to
explicitly discard the in-progress run and start over.
A run's progress is saved to a crash-recovery file in output_dir as it
goes, so a server that stops mid-run (host restart, dropped
connection, machine asleep) loses at most the domain in flight. When
this call finds such a file for an unfinished run in output_dir it
starts nothing and returns a description of it plus an instruction:
run_started is False, "meta" is absent, and "resumable" says whether
it can be continued at all. Call begin_run again with resume=True to
continue that run (its recorded domains are kept, and the response
lists which domains are still missing), or resume=False to discard it
and start fresh. resume=False is the only way to overwrite saved
results, and replace=True counts as the same explicit decision.
Resuming a run for a DIFFERENT repository is refused outright, as is
resuming one whose saved state cannot be read; either way, nothing is
started and nothing is deleted until told.
environment records the host facts the report header cannot carry, and
its keys are a closed set: 'os' (e.g. "macOS 15.2", "Ubuntu 24.04"),
'host_cli' (the CLI application driving this audit, e.g. "codex",
"claude-code") and 'host_cli_version' (that CLI's version string).
Collect them from the machine you are running on rather than guessing,
and omit any key you cannot determine: an omitted fact and a guessed
one are not the same thing. Any other key is refused outright, because
this metadata is included in feedback issues filed publicly on the
tool's own repository. Do not name the assistant, the model or the tool
version here; all three are already fixed rows in the report header.
The recorded metadata also stamps two provenance SHAs, best-effort:
tool_commit (the git commit the installed tool build was made from,
via its PEP 610 install record) and rules_pack_commit (the loaded
rules pack directory's git HEAD, '-dirty' suffixed if it has
uncommitted changes). Either is None when it could not be
determined, which the report renders as "unknown" rather than
guessing: a report must be traceable to the exact tool build and
rules version that produced it, not just a package version number
that can lag behind either.
started is the caller's own claim about when the run began, taken on
trust like everything else the calling agent asserts. This call also
stamps meta.server_started from the server's own clock at the moment
it runs, independent of that claim; render_report does the same for
meta.server_finished. Neither figure is treated as more authoritative
than the other in the rendered report: a resumed run genuinely spans a
wall-clock gap that is not audit work, so the server's elapsed time is
not automatically the truer duration, but an assistant-supplied
duration that was never checked against anything is worse. The report
states both and flags it when they diverge by more than expected,
rather than presenting an unmeasured number as fact.
The run also performs a best-effort tool update check, comparing
tool_commit against the tool's latest tagged release on GitHub, and
the rules pack against its own remote the same way. Each result
lands in the returned meta (update_check and pack_update_check
respectively), prefixed "current", "stale", "could-not-check" or
"not-checked" (see engineering_audit.update_check for the exact
strings). The calling agent MUST tell the user when either reports
stale or could-not-check, rather than silently proceeding as if the
installed build were confirmed current: this tool is installed via a
pinned uvx reference, and a stale pin or cache would otherwise serve
an old build forever with nothing to say so. This check runs
automatically and discloses only the caller's IP address and the
fact that this repository's tags were queried, no repository
content, findings or paths; it can be turned off with
--no-update-check or the ENGINEERING_AUDIT_NO_UPDATE_CHECK
environment variable, in which case both fields read "not-checked",
which is not something to warn the user about, since turning it off
was their own choice.
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | ||
| resume | No | ||
| replace | No | ||
| started | Yes | ||
| repo_dir | No | ||
| assistant | Yes | ||
| repo_name | Yes | ||
| output_dir | Yes | ||
| environment | No | ||
| repo_commit | Yes | ||
| tool_version | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden and does so thoroughly: it discloses that double-calling without finishing is an error, that replace=True discards results, that progress is crash-recoverable, that resuming a different repo is refused, and the update-check behavior including user warning obligations. This is exemplary behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long and dense, covering many edge cases and implementation details. While well-paragraphed and front-loaded with the core purpose, it could be trimmed; sentences like those detailing provenance SHA mechanics and elapsed-time philosophy, though valuable, add length that a more concise version might compress.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (11 params, resume/replace flow, crash recovery, provenance, update checks), the description covers all critical aspects: what the call does, what it returns (run_started, meta absence, resumable), side effects, and error conditions. The presence of an output schema is noted, but the description still explains return semantics thoroughly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must explain all parameters. It does: assistant/model/repo_name/repo_commit/started are caller-supplied, tool_version defaults, repo_dir is optional but needed for file_issues, environment has a closed key set, resume/replace semantics are defined, and started is a trust claim distinct from server-stamped times. Every parameter's meaning is addressed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific verb+resource: 'Start a fresh audit run and create its output directory, or resume an interrupted one.' This distinguishes it from sibling tools like run_status or render_report, which query or finalize rather than initialize or resume.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance, including the requirement to call begin_run before file_issues when repo_dir is needed, the error on calling twice without finishing, and clear conditions for resume=True vs resume=False. It also states prohibitions like refusing to resume for a different repository, leaving no ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Preview or file GitHub issues for every recorded finding, via the
user's own gh CLI.
Requires config.issue_mode == "github": if the user chose in-report
delivery instead, this raises rather than filing issues nobody asked
for. Requires at least one recorded domain result.
confirm=False (the default) NEVER files anything and never invokes
gh at all: it returns a preview {repo, count, titles, instruction}
so the calling agent can show the user exactly what is about to be
filed on their repository, and get explicit agreement, before a
single issue goes out. Filing on someone's repo is outward-facing;
this confirmation step is mandatory, not decorative.
confirm=True files one issue per finding that has not already been
filed, so retrying after a partial failure does not double-file the
ones that succeeded. Filed issues are tracked, and returned, per
finding under a key of the form "<rule id>#<n>" (n counting that
rule's findings in recording order), not per rule id: a domain result
may carry two findings for the same rule, and both of their issue
urls have to survive. The target repository is `repo` if given,
otherwise detected from the audited repository directory recorded
by begin_run's repo_dir. If any issue fails to file, filing stops
immediately and the error lists exactly which findings were filed
(with their URLs) and which were not, so a retry knows where to
resume.
Each filed issue carries the "engineering-audit" label. The label is
checked once per call and created on the target repository if it is
missing; the response's label field reports which of present,
created or unavailable happened. Unavailable (creation failed) files
the issues unlabelled and says so once, in warnings, rather than
once per issue.
| Name | Required | Description | Default |
|---|---|---|---|
| repo | No | ||
| confirm | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that confirm=False never invokes gh, that confirm=True is idempotent against partial failures, how issue keys are structured, how the target repo is resolved, what happens on failure (stops, lists filed vs unfiled), and the label creation fallback behavior. This is exemplary transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long, but each paragraph covers a unique and necessary aspect: purpose, prereqs, confirmation, idempotency, repo resolution, failure handling, and label behavior. It is well-structured, though slightly verbose compared to a tighter alternative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers all operational contexts: prerequisites, default preview behavior, retry semantics, error reporting, label handling, and repo selection. With an output schema present, return values are already defined, and the description still adds valuable context. It is fully complete for a safe-use tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides only names and types with no descriptions (0% coverage). The description fully compensates: it explains confirm=False's preview behavior and that repo defaults to begin_run's repo_dir when absent. Both parameters are semantically defined.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Preview or file GitHub issues for every recorded finding, via the user's own gh CLI,' which specifies a concrete action, target, and scope. This clearly distinguishes it from sibling workflow tools like get_domain or render_report.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states the required config mode ('Requires config.issue_mode == "github"') and warns that in-report delivery raises instead. It also explains the two usage modes (confirm=False preview vs confirm=True filing) and stresses that confirmation is mandatory before outward-facing actions, giving clear when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch the resolved audit configuration, or report that the user has not submitted the configuration page yet.
Requires start_config to have been called first. Every response carries
a "status" field, and it is the only field worth branching on:
- "configured": the configuration is resolved and is in the response's
"config" and "selected_domain_ids". Stop calling this tool.
- "waiting": the interactive page is up and nobody has submitted it
yet. This is NOT a failure and NOT a configuration. Tell the user the
audit is waiting on them at the "url" in the response, then CALL THIS
TOOL AGAIN. Keep calling it while the status says "waiting".
- a raised error: the run's overall deadline (timeout_s) elapsed with
no submission. Tell the user the audit is not proceeding. Never fall
back to a domain selection nobody chose.
In preset mode the configuration is already known and comes back as
"configured" on the first call.
This tool deliberately blocks for at most a short interval per call
(about 25 seconds) and then returns "waiting", rather than holding one
call open for the whole of timeout_s. Hosts impose their own per-tool
timeouts, independent of timeout_s (Codex has
mcp_servers.<name>.tool_timeout_sec), and a call held open past one of
those is cancelled by the host, which can take the whole MCP process
and this run's configuration page down with it (issue #85). timeout_s
remains
the run's overall waiting budget and is enforced here, cumulatively,
across however many calls it takes: it is measured from the moment the
page opened, so polling more often does not buy the user more time, and
polling less often does not cost them any. To keep waiting past the
deadline, call again with a larger timeout_s; that is an explicit
decision to extend, not a silent one.
| Name | Required | Description | Default |
|---|---|---|---|
| timeout_s | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Even without annotations, the description fully discloses the tool's blocking behavior (~25s per call), the cumulative timeout semantics, and the risk of host-level timeouts that can cancel the process (referencing issue #85). It also warns about not falling back to domain selections nobody chose, which is a significant behavioral constraint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Though lengthy, every paragraph serves a purpose: the first sentence states the purpose, the status list gives actionable branching logic, and the final paragraph justifies the polling interval and timeout behavior. The structure with clear bullet points and paragraph separation makes it navigable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers prerequisites, all response statuses, the response fields worth branching on, error behavior, and host timeout interactions. With a 1-parameter schema and no annotations, this description leaves no operational ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema only provides a title and default for timeout_s, but the description explains its true meaning: the overall waiting budget measured from when the page opened, cumulative across calls, and that extending it is an explicit decision. This is exactly the kind of semantic enrichment the schema lacks.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Fetch the resolved audit configuration, or report that the user has not submitted the configuration page yet', which clearly identifies the operation and resource. It also distinguishes this from sibling tools like list_domains or run_status by framing it as a polling endpoint for configuration submission.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states 'Requires start_config to have been called first' and provides a decision tree for the three response statuses, instructing to call again while waiting and to stop once configured. It also clarifies behavior in preset mode, making it clear when this tool should be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return the full document text for one domain, given its id (e.g. 'd01').
This tool serves the full rule text: it is meant for the local agent
driving the audit, which needs the rules to apply them. Nothing else
in this package returns rule body text.
Because of that, this call is recorded against the run in progress: it
is the one observable event that could have supplied the rules a
verdict is meant to rest on. record_domain_result says so when verdicts
arrive for a domain this was never called for, and the report names
that domain. The claim either way is only ever that the text was
fetched, never that it was read.
A fetch made when no run is in progress belongs to no run and is not
recorded: call begin_run first, then fetch each domain as you come to
it.
| Name | Required | Description | Default |
|---|---|---|---|
| domain_id | Yes |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility and excels. It discloses that the call is recorded against the run, that it is the sole observable event supplying rules for a verdict, and that recording only happens when a run is active. It even clarifies that a fetch is merely 'fetched, never read,' adding nuanced context beyond the obvious read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with a clear purpose sentence. The following paragraphs are detailed and each adds valuable context about audit recording, but they are somewhat verbose. Still, no sentence is wasted, and the structure is logical.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter, an output schema, and rich behavioral nuances, the description is complete. It covers purpose, intended user, prerequisites, side effects, and the distinction from siblings. The existence of an output schema means return values need no elaboration.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, domain_id, is clarified with an example ('e.g. 'd01''), which adds meaning beyond the bare schema (0% coverage). It could go further by explaining how to obtain a valid domain_id (e.g., from list_domains), but the example and context are sufficient for basic invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Return the full document text for one domain, given its id.' It immediately distinguishes itself from siblings by stating 'Nothing else in this package returns rule body text,' making the tool's unique purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states it is meant for the local agent driving the audit and provides a prerequisite: 'call begin_run first, then fetch each domain as you come to it.' It also clarifies when the call is recorded (run in progress) and that no other tool returns rule body text, serving as an exclusion criterion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List every domain loaded from the rules pack, and report any files in the pack directory that were skipped because they had no Trigger line.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden. It transparently discloses both the main listing action and a notable edge case (skipped files with no Trigger line). While it doesn't explicitly state read-only status or permissions, 'List' strongly implies a non-mutating operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with two closely related clauses. Every word contributes meaning, with no fluff or repetition of schema information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list tool with an output schema present, the description covers the core purpose and a valuable edge-case behavior. The output schema likely handles return-value details, so the description is sufficiently complete for selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema already fully covers the input surface. The baseline of 4 applies, and the description appropriately adds no parameter-specific noise.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('List') and resource ('every domain loaded from the rules pack'), and adds a distinct secondary behavior about reporting skipped files. This clearly differentiates it from sibling tools like get_domain, which likely fetches a single domain.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The context is clear: this is for enumerating all domains in the rules pack, not for looking up one specific domain (which get_domain likely handles). However, it does not explicitly state when to use this tool versus alternatives or mention any exclusions, stopping short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Record the audit result for one domain.
The payload itself is pydantic-validated by DomainResult (finding and
verdict consistency, could-not-run reason, could-not-evaluate and
not-applicable notes, both of which are the verdict's stated reason,
every finding's precondition, the completed domain's
uninspected_evidence, and that every consulted_sources entry has a
non-blank url, title and why). On top of that: the domain must be one of the domains
selected for this run, a completed result must carry a verdict for
every rule the domain defines, and every consulted_sources rule_id
must be one of this domain's own rules; a completed result missing a
verdict raises IncompleteResultError listing exactly which rule ids
are missing, and an unattributable consulted source raises
UnknownRuleIdError, so the agent can fix and resubmit rather than a
skipped rule silently passing or a citation silently pointing at
nothing. Re-recording an already-recorded domain requires
replace=True, to guard against an accidental overwrite.
Verdicts for a domain get_domain was never called for during this run
are recorded, not refused, and the response says "rules_fetched": false
and carries a warning naming what that means. The report names the
domain too. Recording rather than refusing is deliberate: refusing
would be trivially satisfied by fetching the text and ignoring it,
which destroys the signal, while the verdicts and the fact that they
were unsupported both survive this way. Tell the user when you see it.
Two fields are refused outright rather than recorded with a warning,
because unlike an unfetched domain there is no signal to preserve by
letting them through: a finding without a `precondition` (issue #178)
and a completed domain without `uninspected_evidence` (issue #179).
Both are one sentence the auditor already knows the answer to, and in
both cases being unable to write it is the finding. A finding whose
precondition cannot be named belongs at not-applicable, and a domain
that cannot say what it did not read has not established what its
absence claims are worth. See AUDIT.md step 3 and step 4.
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes | ||
| replace | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and delivers: it details validation layers, error types (IncompleteResultError, UnknownRuleIdError), the deliberate recording of unfetched domains with a warning, refusal of two specific fields, and the rationale for these behaviors. This is exemplary disclosure of edge-case handling and error semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long and dense, but every sentence contributes substantive guidance—validation rules, errors, edge cases, and rationales. It is front-loaded with the core purpose and organized into coherent paragraphs. It would be slightly more scannable with bullets, but the length is justified by the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of DomainResult validation and the absence of annotations, the description is remarkably complete: it covers success criteria, failure modes, exceptions, overwrite protection, unfetched domains, and field refusals. The output schema covers return values, and the description fills the behavioral gap thoroughly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The top-level schema has 0% description coverage, so the description must compensate. It adds significant meaning: constraints on DomainResult (every rule verdict required, consulted_sources rule_id association), replace semantics, and rejection criteria. It does not restate the JSON shape, but the nested schema $defs cover structure. Slightly more explicit field-level guidance would raise it further.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Record the audit result for one domain.' It further specifies scope (domain must be selected for this run, replace=True for re-recording), clearly distinguishing it from sibling tools like get_domain (fetching) and list_domains (listing). The purpose is unambiguous and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The tool explains when to use it (after auditing a domain, with validation of completeness), when to use replace=True (re-recording), and when results are refused (missing precondition or uninspected_evidence). It references AUDIT.md steps 3 and 4 for deeper context. It does not explicitly name alternatives, but the operational context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Finish the run and render its report.
Requires a resolved configuration. Sets meta.finished to the given
ISO timestamp, renders the deterministic HTML report (which itself
refuses to render an incomplete run: a selected domain with no
recorded result, or a completed result missing a rule verdict, raises
rather than producing a report that looks clean over a gap), and
writes both report.html and run-state.json to the run's deliverables
directory: config.deliverables_dir if the configuration page (or a
preset AuditConfig) named one, otherwise the run's own output_dir,
unchanged from how every run before that choice existed behaved.
output_dir itself is never affected by this choice; it stays the
run's working directory for the crash-recovery progress file
regardless of where the finished deliverables land (issue #109).
Any issue URLs filed this run via file_issues, and any
feedback issue filed via submit_feedback, are carried on the
RunState itself, so the written run-state.json is self-sufficient:
it (and its schema_version) can be handed to
engineering-audit-render later to re-render the same report without
this server, this run tracker, or either URL, still in memory.
This call also stamps meta.server_finished from the server's own
clock, alongside the caller-supplied finished. See begin_run's
server_started for why the report keeps both this figure and the
caller's rather than trusting either one alone.
The finished run stays reachable for one last submit_feedback (the
order AUDIT.md documents), which rewrites both files to carry the
feedback issue's link. It stops being reachable at the next
begin_run.
Both files are written atomically, and the run's crash-recovery file
is removed once they are on disk: from here the run-state.json is the
record, and a later begin_run on this output directory starts clean
rather than offering to resume a run that is already finished.
The response also carries "rules_fetched": which domains had their
rule text fetched this run, which recorded verdicts without it, and
which were carried in from a saved run that never recorded it. Any
domain in the second list is named in the report and must be named to
the user as well: it says the verdicts for that domain were reached
without the rules they are verdicts on.
| Name | Required | Description | Default |
|---|---|---|---|
| finished | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and it excels: it discloses atomic writes, file locations (deliverables_dir vs output_dir), removal of crash-recovery file, changes in run reachability, failure modes (raises on incomplete runs), and the response's 'rules_fetched' field. This is comprehensive behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured: the core purpose is front-loaded, and each subsequent paragraph covers a distinct behavioral aspect (server_finished, feedback ordering, atomicity, response). Some historical detail like 'issue #109' could be trimmed, but the length is largely justified by the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity and the presence of an output schema, the description is remarkably complete. It covers prerequisites, side effects, ordering with sibling tools, failure modes, and response content. The output schema handles return structure details, so the description needs no further additions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, 'finished', has 0% schema description coverage, but the description adds meaning by calling it an 'ISO timestamp' and explaining it sets meta.finished. This goes beyond the bare schema type, though it does not specify the exact ISO 8601 variant or format constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Finish the run and render its report,' a specific verb+resource statement that distinguishes it from siblings like begin_run, record_domain_result, and submit_feedback. It further elaborates on deliverables and side effects, leaving no ambiguity about the tool's role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states a prerequisite ('Requires a resolved configuration') and explains interaction order with submit_feedback and begin_run. It references begin_run for a design rationale but does not explicitly say 'use this instead of X', so it lacks explicit exclusionary guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Report progress for the current run: which selected domains have recorded results, which are still missing, and the findings count so far. Read-only over the run itself; it also carries any queued crash-recovery warning that no earlier response has reported yet.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It explicitly discloses the tool is read-only (non-destructive) and mentions the queued warning behavior, which is a useful nuance. It doesn't go into details like rate limits or auth, but for a simple status check it provides adequate transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, front-loaded with the primary purpose ('Report progress for the current run'). Every clause adds information: what is reported, the read-only nature, and the additional warning behavior. No waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with an output schema, the description fully covers what the tool does and what the response will contain (progress details plus optional warning). It is complete given the simplicity of the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the schema coverage is 100% (vacuously). The description doesn't need to explain any parameters. Baseline for zero-parameter tools is 4, and the description adds no unnecessary parameter detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('Report') and resource ('progress for the current run'). It enumerates what it reports (which selected domains have recorded results, which are missing, findings count), distinguishing it from sibling tools like get_domain or record_domain_result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use this tool: it is read-only over the run itself and additionally surfaces queued crash-recovery warnings that haven't been reported yet. It doesn't explicitly name alternatives, but the sibling set contains no similar status tool, so usage is fairly clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Begin configuring the audit run.
If the ENGINEERING_AUDIT_CONFIG environment variable names a path to
a valid AuditConfig JSON file, it is loaded immediately (the
documented headless/CI path); an invalid or unreadable file is a
loud error, never a silently-applied default. Otherwise this starts
the interactive localhost configuration page, opens it in the user's
browser when one is available (best-effort; the response's
opened_in_browser field says whether a tab actually opened), and
returns its URL for the agent to show the user as the fallback.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states that an invalid config file produces a loud error rather than a silent default, that browser opening is best-effort with an opened_in_browser field in the response, and that the URL is returned as a fallback. This is comprehensive and goes well beyond a bare statement of purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two sentences with a clear front-loaded purpose statement. Every clause provides meaningful behavioral or contextual detail, and there is no wasted wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema, the description is complete. It covers both operating modes, error behavior, browser handling, and response contents, leaving no significant gap for an agent to infer.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the input schema is an empty object with 100% schema coverage. The description doesn't need to elaborate on parameter semantics; the baseline for zero-parameter tools is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Begin configuring the audit run,' clearly identifying the tool's action and resource. It further distinguishes the headless/CI path from the interactive path, making its purpose distinct from sibling tools like get_config and begin_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when the headless/CI path is triggered (when ENGINEERING_AUDIT_CONFIG is set) and when the interactive page is used, providing clear contextual guidance. It doesn't explicitly state alternatives or exclusions, but the context is sufficient for an agent to decide when to invoke this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Send optional run feedback to the tool author.
Requires a resolved configuration. There is nothing to send unless
config.feedback_text was set on the configuration page, or the
calling agent supplies extra_text; if neither is present this
raises rather than filing an empty, pointless issue.
The feedback body always carries the free text plus a run-metadata
section (tool version, rules pack, assistant, model, repository,
timestamps), and then each telemetry section the user consented to
on the configuration page (coverage totals, findings rollup by
severity/domain id, self-assessment, environment, consulted sources
by rule id/url/why, rule verdict distribution by domain and in
total, run duration and the divergence verdict between its two
measurements, which domains had their rule text fetched via
get_domain, and the reader's own conclusions after reading the
report); an unconsented section is left out entirely. Finding text
itself is never included, only counts.
report_conclusion and report_fix_first (issue #135) are the
reader's own answers, in their own words, to the two questions the
finished report's own feedback form asks: in one sentence, what did
this report tell them about their repository, and what would they
fix first. Pass these only if the human using this session actually
read the finished report and dictated an answer back; never guess
or paraphrase one on their behalf. Both are ignored unless the
reader_conclusions section was consented to on the configuration
page, same as every other telemetry section here.
Files a labelled issue on the tool author's feedback repository via
gh. If gh is unavailable or filing fails for any reason, the
feedback is never lost: this returns a mailto fallback instead,
with the same body, so the agent can offer to open the user's mail
client or hand over the text to paste in manually.
May be called either before or after render_report. Called after,
it sends feedback for the run just finished and rewrites that run's
report.html and run-state.json so both carry the feedback issue's
link; the response's report_updated field says whether that rewrite
succeeded, and a failed rewrite is reported as a warning rather than
an error, because the issue is already filed by then and raising
would invite a retry that double-files it.
| Name | Required | Description | Default |
|---|---|---|---|
| extra_text | No | ||
| report_fix_first | No | ||
| report_conclusion | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the transparency burden and delivers extensively. It discloses failure behavior (raises when nothing to send, mailto fallback on gh failure), consent-based telemetry inclusion, rewriting of report.html/run-state.json, and the deliberate warning rather than error after a failed rewrite to avoid double-filing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is information-dense and front-loaded with a clear first sentence, but the telemetry section list is an extremely long parenthetical that hurts readability. All content earns its place, yet tightening the structure would improve conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description covers all critical operational context: preconditions, content construction, parameter ethics, failure modes, timing relative to render_report, side effects, and idempotency concerns. This level of detail is necessary for correct invocation and retry behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description compensates thoroughly. It explains extra_text as agent-supplied free text, and provides extensive semantic and ethical guidance for report_conclusion and report_fix_first, including when they are ignored and that they must be the human's own words.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence, 'Send optional run feedback to the tool author,' identifies a specific verb, resource, and audience, clearly distinguishing this from sibling tools like file_issues and render_report. Additional details about filing a labeled issue on the author's feedback repository reinforce the tool's unique role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context: it requires a resolved configuration, only sends if config.feedback_text or extra_text is present, and may be called before or after render_report with different behaviors. However, it does not explicitly contrast with sibling tools like file_issues, leaving some inference to the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v0.10.0record_domain_result2 fields changedInput schema / $defs / DomainResult / properties / uninspected_evidenceAdded value: +{
+ "anyOf": [
+ {
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "description": "Required on a completed domain. The evidence stores this repository points at that you did not read while verdicting this domain, one entry each, naming the store and where the repository points at it (e.g. 'GitHub Issues: README.md:9 sends requirements here; not inspected'). An empty list is a claim in its own right: the repository points at nothing you did not read. Findings are not rejected because of what is listed here; the report shows it beside them so a reader can judge the scope the verdicts were reached in.",
+ "title": "Uninspected Evidence"
+}Input schema / $defs / Finding / properties / preconditionAdded value: +{
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "description": "Required. The precondition this rule presumes, and where it was observed to hold in this repository (e.g. 'the rule presumes a release pipeline, present at .github/workflows/release.yml'). If you cannot name where the precondition holds, the honest verdict is not-applicable, not finding.",
+ "title": "Precondition"
+}v0.8.0begin_run1 field changedInput schema / properties / resumeAdded value: +{
+ "anyOf": [
+ {
+ "type": "boolean"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Resume"
+}record_domain_result4 fields changedInput schema / $defs / ConsultedSourceAdded value: +{
+ "description": "A source consulted outside the rules pack while reaching a verdict:\ndocumentation, a standard, a paper, anything fetched or read that is not\nthe pack itself.\n\nThe MCP server has no way to observe the driving agent's own web or file\nactivity, so this list is schema-demanded self-reporting, not something\nthe server can verify happened. That is a real limit, not a hidden one:\na source the agent never records here is a source this tool can never\nknow about, the same way a rule the agent never verdicts can never be\nrecorded as a pass.",
+ "properties": {
+ "accessed": {
+ "title": "Accessed",
+ "type": "string"
+ },
+ "rule_id": {
+ "title": "Rule Id",
+ "type": "string"
+ },
+ "title": {
+ "title": "Title",
+ "type": "string"
+ },
+ "url": {
+ "title": "Url",
+ "type": "string"
+ },
+ "why": {
+ "title": "Why",
+ "type": "string"
+ }
+ },
+ "required": [
+ "rule_id",
+ "url",
+ "title",
+ "why",
+ "accessed"
+ ],
+ "title": "ConsultedSource",
+ "type": "object"
+}Input schema / $defs / DomainResult / properties / consulted_sourcesAdded value: +{
+ "description": "Sources consulted outside the rules pack while reaching this domain's verdicts. Optional and self-reported; see validate_consulted_sources for the one check applied against it (every rule_id must be one of this domain's own rules).",
+ "items": {
+ "$ref": "#/$defs/ConsultedSource"
+ },
+ "title": "Consulted Sources",
+ "type": "array"
+}Input schema / $defs / Finding / properties / location / descriptionPrevious value: -"'path:line' or 'path'"New value: +"'path:line', 'path:start-end' or 'path'"Input schema / $defs / RuleVerdict / properties / note / descriptionPrevious value: -"Free text. Required when verdict is could-not-evaluate: the reason."New value: +"Free text. Required when verdict is could-not-evaluate: the reason. Required when verdict is not-applicable: the precondition of the rule that does not hold in this repository."submit_feedback2 fields changedInput schema / properties / report_conclusionAdded value: +{
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Report Conclusion"
+}Input schema / properties / report_fix_firstAdded value: +{
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Report Fix First"
+}v0.1.0begin_runfile_issuesget_configget_domainlist_domainsrecord_domain_resultrender_reportrun_statusstart_configsubmit_feedbackEach tool occupies a clear, distinct role in the audit lifecycle: configuration, run status, domain listing, rule text retrieval, feedback, run management, result recording, issue filing, and report rendering. There is no overlapping purpose; even start_config and get_config are cleanly separated as initiating versus polling for configuration. The descriptions further disambiguate any apparent similarity.
All tool names follow a consistent verb_noun pattern using lowercase snake_case: get_config, list_domains, begin_run, render_report, etc. No mixed conventions or vague verbs. The naming makes the tool's action and object predictable at a glance.
Ten tools is well within the ideal range and each tool maps to a necessary step in the audit workflow. The count feels neither sparse nor bloated, and every tool earns its place by serving a distinct function in the overall process.
The toolkit covers the full audit lifecycle: configuration, run initialization and resumption, domain and rule access, result recording with validation, report rendering, issue filing with preview, and feedback submission. There are no obvious dead ends or missing operations; even re-recording and discarding runs are handled via parameters like replace=True.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
AI-native git hosting — repos, PRs, issues, CI gates, and AI code review over MCP (60 tools).
MCP server for secureFlows: token-free URL builders and integration-linting tools for AI agents.
The OpenZeppelin Solidity Contracts MCP server integrates OpenZeppelin's security and style rules into AI-driven development workflows, enabling AI assistants to generate safe, correct, and production-ready smart contracts. It automatically validates generated code against OpenZeppelin standards (including imports, modifiers, naming conventions, and security checks) and supports various contract types including ERC-20, ERC-721, ERC-1155, Stablecoins, RWA, Governor, and Account contracts through prompt-driven workflows.
AI governance MCP server for EU AI Act compliance and jurisdiction verification
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/rodlunt/engineering-audit'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
