FIRSTMATE-34: Parallel validation - bind no-mistakes responds to each worker's own run #51
Loading…
Reference in a new issue
No description provided.
Delete branch "fm/nm-concurrent-intent-crosstalk"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Story: FIRSTMATE-34 - Parallel validation
Lets more than one change in the same repository go through no-mistakes validation at once, by binding every gate answer to the worker's own run, and keeps validation load proofs bounded.
Investigation
Proven on 2026-10-20 with two concurrent real runs on two branches of a disposable scratch repository (ejected afterwards); the full commands and output are in
docs/verification/no-mistakes-branch-isolation.md./proc/<pid>/exehash matches the installed v1.84.0 binary, and every run since recordsno_mistakes_version=v1.84.0. The v1.53.0 premise is out of date, so no restart is needed.axi statusandaxi respondresolve from the caller's checked-out branch. Each worktree's status showed only its own run. A worktree with no run of its own, the main checkout, and a detached HEAD all goterror: no active run to respond towhile another branch's run sat at a gate.respondfrom the second run's worktree answered only that run, and the first run's gate stayed untouched.axi respondhas no--runselector, so the checked-out branch is the only binding.axi status --run <id>exists for deliberate reads.daemon restart,daemon stop, andupdaterefuse while any run is active unless--forceis passed. The run store holds 19 runs that failed withdaemon shutting downfrom forced stops on older daemons. Procedure: confirm no run is active in any repository, then runno-mistakes daemon restartwithout--force, then confirm the running binary by hash. A forced restart while runs are live is not safe.Changes
bin/fm-dod-lib.sh(the generated no-mistakes contract, for every forge): the worker issues everyaxicall from its own copy with its ship branch checked out, records the run id that its starting drive call returns, reads status with--run <id>, runs a bareaxi statusimmediately before every respond, and answers only when the id, the branch, and the awaiting step all match, passing--stepexplicitly. A gate or a firstmate decision whosenm-<run>-<step>key names another run is refused with ablocked:run-mismatch line.niced loops, never one per core, and measure CPU time instead of generating load wherever that can show the same thing. A test-step gate that shows host-saturating load gets--action fix, never approve..agents/skills/validation-supervision/SKILL.md: concurrent no-mistakes runs on different branches of one repository need no per-repository serialization on v1.84.0 and later. Firstmate's decision relays now also name the run id.docs/verification/no-mistakes-branch-isolation.md: a new maintainer-verification record, classified indocs/documentation-audiences.json.tests/fm-brief.test.sh: tests that the binding and the bounded-load rule render on every forge and stay out of pipeline-free briefs. A mutation that drops the refusal fails the test.Verification
tests/fm-brief.test.sh,tests/fm-task-delivery.test.sh,tests/fm-dod-lib.test.sh,tests/fm-ask-user-authority.test.sh,tests/fm-classify-corr-token.test.sh,tests/fm-spawn-dispatch-profile.test.sh,tests/fm-pr-check-security.test.sh,tests/fm-control-relaunch.test.sh,tests/fm-tangle-guard.test.sh, andtests/fm-subagent-pretool-check.test.shall pass.bin/fm-lint.shandbin/fm-doc-audience-check.share clean.Follow-up (out of scope)
data/learnings.md"One repo = one pipeline", and thenomistakes-pipeline-is-repo-scopedmemory). Firstmate should retire them once this lands..no-mistakes.yamltest.instructions, which a worker brief cannot reach. Adding the bounded-load sentence there (Flow first) would bound the pipeline agent directly.Review 1 fix: recycled branch
A recycled copy can still carry an earlier task's active run on the new task's ship branch. Re-probed on v1.84.0 in a scratch repo (evidence now in the verification record): a plain
axi run --intentthere silently reattaches to the earlier run and ignores the new intent, while a launch with a fresh--launch-noncecreates a new run and cancels the earlier one (superseded by new push).The contract now proves ownership. Before starting, the worker refuses (
blocked) when its branch has an active run it did not create. Every new run starts with--launch-nonce <task-id>-<epoch> --validation-generation <task-id>. The worker drives a run only when itslaunch_receiptshowsdisposition: created, its own ship branch, andsubmitted_head_shaequal to its HEAD, and it saves that receipt in its task data directory. Every reattach must return that run id.tests/fm-brief.test.shadds a recycled-branch regression on every forge, and a mutation that drops the refusal fails it.Review 2 fixes
runs_on_current_branch: 0, a terminal run (completed,failed,cancelled), or the run id in the worker's own saved receipt. A parked run (which still readsstatus: running) or any unreadable output counts as a foreign owner and blocks, which also replaces the copied status vocabulary.validation-supervisionnow names firstmate's recovery for "run was not started by this task". If it is the same task, point the worker at its saved receipt. If it belongs to another live task or holds an open decision, leave it alone. Only a confirmed orphan with no open decision getsno-mistakes axi abort --run <id>.Implemented with Claude Opus 5.5 (claude-opus-5-5).
Adversarial review 1 of 2 - REQUEST CHANGES
Reviewer: OpenAI gpt-6-luna (Codex, high effort)
Reviewed head:
a0c10b5440bin/fm-dod-lib.sh:356-358;docs/verification/no-mistakes-branch-isolation.md:109. A recycled worktree can leave an active run on the task's ship branch. The generated contract assumes the ID returned byno-mistakes axi runbelongs to this task, then accepts a current-branch status with that ID and branch. The installedaxi run --helpdocuments reattachment to an active run, whileaxi respond --helphas no run selector. A new task on the reused branch can therefore reattach to the previous task's run, pass the ID/branch check, and answer its gate. The check proves branch association, not task/run ownership. Detached HEAD and no-run branches are covered by the verification record, but a prior active run on the same branch is not. Require proof that the run was newly created for this task and fail closed on reattachment, or retain serialization untilrespondcan target the verified run ID; add a recycled-worktree regression or CLI-level proof.Validation on the live head:
bash tests/fm-dod-lib.test.shandbash tests/fm-brief.test.shpassed;shellcheck -x bin/fm-dod-lib.shpassed. No load generator or real no-mistakes run was started. Full evidence is in/home/john/code/firstmate/data/rev-fm51-a1/report.md.Adversarial review 2 of 2 - PASS
Reviewer: Qwen 3.8 via AIRouter
Reviewed head:
4cc7a4971dIndependent of review 1; verdict formed at the live head (which moved to this sha while this review was in flight).
bin/fm-dod-lib.sh:361-362. The pre-flight foreign-run refusal keys off the worker parsing a status word ("pending, running, fixing, or ci"), while the failure mode this rule exists for - a prior task's run PARKED at a gate - reports top-levelstatus: running(see tests/captures/no-mistakes-v1.70.1/parked.toon). One misread of that barestatusTOON (truncation, help-line noise) lets the fresh-nonce launch through, and per this PR's own verification record the launch then cancels the foreign parked gate ("superseded by new push"), possibly one holding open ask-user decisions. Thedisposition: created+submitted_head_sha == HEADreceipt check does not save it: a parked run at the same head passes both. Suggest refusing structurally onruns_on_current_branch > 0instead of a parsed status word.blocked: ... was not started by this taskstate. Neither validation-supervision nor the stuck-worker playbook names what firstmate does: confirm-orphan thenaxi abort --run <id>and relaunch, vs. the run belongs to a live task and the slot goes back to it. The first real hit of this guard (which the recycled-copy shape makes likely) becomes an ad-hoc 3 a.m. decision. One paragraph in the skill would close it.test_no_mistakes_dod_bounds_test_load(loopsnone forgejo; the sibling binding and recycled-run tests loop all three). The bounded-load paragraph is forge-independent today, so this is coverage symmetry, not a defect - one line to add.$FM_HOME/data/$id/, outside the worktree - same established carve-out asnm-intent.txt, and it survives teardown (good for the F2 recovery path), but it is a second instance of a writable-outside-worktree file the brief's rule 2 does not name. No action; noted so a later "fix" of one does not orphan the other.Verified:
bash tests/fm-brief.test.shandbash tests/fm-dod-lib.test.shpass (including the three new tests);bin/fm-lint.shfull set passes (ShellCheck 0.11.0, actionlint 1.7.12); the binding block and bounded-load rule render verbatim into a real scaffolded no-mistakes brief with correct$id/data-dir substitution and stay out of direct-PR/local-only briefs;axi respond --help(no run selector),axi run --help(launch receipt: run id, created/reused disposition, submitted head, intent digest), and the daemon lifecycle--forcehelp text all match the verification record; the live shared daemon is the v1.84.0 binary the record tested (sha256 of /proc//exe vs ~/.no-mistakes/bin/no-mistakes); the 19 "daemon shutting down" failed runs are exactly 19 in the local run store (11 NULL-version, 8 v1.53.0); detached-HEAD and run-less-branch refusals were re-probed live on the shared daemon in a throwaway scratch repo (rc=1,current_branch: unknown/runs_on_current_branch: 0) matching the record; no remaining firstmate text demands one run per repository (AGENTS.md 195-196 is the pre-existing general semantic-dependency rule). No load generator and no real pipeline run was started; two read-only CLI probes ran in a disposable scratch repository, ejected afterward. No high findings; the respond path itself is sound - the residual exposure is a single misread on the pre-flight check, not in respond. Full evidence in /home/john/code/firstmate/data/rev-fm51-a2/report.md.Adversarial re-review - PASS
Reviewer: OpenAI gpt-6-luna (Codex, high effort)
Head:
7bc041cb9ePR: #51
No high or medium findings remain. The live-head
fm-briefsuite passed;shellcheck -x bin/fm-dod-lib.shpassed; and no-mistakes v1.84.0 help confirms launch receipts, run-selected status, and branch-bound respond with no run selector. The generated contract now refuses an active run without this task's created receipt, the validation skill owns recovery for an orphaned run, and the bounded-load test covers Gerrit. The report records the evidence and the limit that I did not repeat a live scratch run because it would persist scratch-run state through the shared daemon outside the task's allowed write paths.