ci: run CI on the forge's Arch native runner #12

Merged
john merged 9 commits from fm/fm-ci-forgejo-arch into main 2026-08-19 16:59:32 +00:00
Owner

Firstmate's CI has never actually executed. The workflows were written for GitHub-hosted runners and GitHub-supplied JS actions, and the fleet's own forge runner is a native Arch host with neither. So every workflow file in this repo described a pipeline that could not start, and the test suite behind it had been drifting unchecked for as long as that was true.

This converts CI to run on the forge's own Arch native runner, and then makes the suite actually pass on it.

What changed

A real runner image. .github/ci/Dockerfile builds an Arch image carrying the suite's true dependencies. Several were discovered only by running it: diffutils, sqlite, and lsof — the last because fm-teardown.sh's leaked-process detection depends on it. The image runs as a non-root ci user, because the permission-semantics tests are meaningless as root, and sets git safe.directory so git does not reject the checkout as dubiously owned.

Rewritten workflows. ci.yml and no-mistakes-required.yml now use runs-on: native and build/run that image directly. No JS actions anywhere — the native runner has no node to execute them with.

Twelve pre-existing test bugs, fixed. These are not migration fallout; they are the accumulated cost of a suite whose CI never ran. Two root causes:

  • fm_git_worktree and fm_git_init_commit never set an origin remote. Under --mode no-mistakes that hard-fails a genuine delivery-target check — a check that was correct all along and had simply never been reached.
  • One test assumed no system-wide node exists, which was true on the old runner and is false on Arch, defeating its PATH-isolation simulation.

Left unfixed, each of these turns the first real run red for reasons unrelated to this change, which is the worst possible first impression for a gate nobody yet trusts.

Decisions worth understanding

The macOS Bash 3.2 job is dropped, deliberately. Compiling bash 3.2 requires a pre-GCC-14 toolchain Arch does not carry, and no faithful prebuilt image exists. This is a real reduction in coverage — the suite no longer proves portability to the oldest bash — and it is recorded as such in both CONTRIBUTING.md and ci.yml rather than quietly omitted.

Verification was done against a live container, not inferred. Lint, the coverage guard, both portable-parallel shards, and all four portable-serial shards were each confirmed green inside a real Arch container built from this Dockerfile.

Still open

The Herdr E2E lane has not been validated locally. It needs the pinned herdr 0.7.4 that the workflow installs; the local environment has 0.7.5, which gives non-representative results. That lane's correctness rests on the pinned install behaving as written, and the first forge run is what will actually prove it.

More generally: this branch is the first thing that will ever exercise this pipeline for real. A green run here is the acceptance criterion, and it has to be observed on the forge rather than concluded from the local gate.

Firstmate's CI has never actually executed. The workflows were written for GitHub-hosted runners and GitHub-supplied JS actions, and the fleet's own forge runner is a native Arch host with neither. So every workflow file in this repo described a pipeline that could not start, and the test suite behind it had been drifting unchecked for as long as that was true. This converts CI to run on the forge's own Arch native runner, and then makes the suite actually pass on it. ## What changed **A real runner image.** `.github/ci/Dockerfile` builds an Arch image carrying the suite's true dependencies. Several were discovered only by running it: `diffutils`, `sqlite`, and `lsof` — the last because `fm-teardown.sh`'s leaked-process detection depends on it. The image runs as a non-root `ci` user, because the permission-semantics tests are meaningless as root, and sets `git safe.directory` so git does not reject the checkout as dubiously owned. **Rewritten workflows.** `ci.yml` and `no-mistakes-required.yml` now use `runs-on: native` and build/run that image directly. No JS actions anywhere — the native runner has no node to execute them with. **Twelve pre-existing test bugs, fixed.** These are not migration fallout; they are the accumulated cost of a suite whose CI never ran. Two root causes: - `fm_git_worktree` and `fm_git_init_commit` never set an `origin` remote. Under `--mode no-mistakes` that hard-fails a genuine delivery-target check — a check that was correct all along and had simply never been reached. - One test assumed no system-wide `node` exists, which was true on the old runner and is false on Arch, defeating its PATH-isolation simulation. Left unfixed, each of these turns the first real run red for reasons unrelated to this change, which is the worst possible first impression for a gate nobody yet trusts. ## Decisions worth understanding **The macOS Bash 3.2 job is dropped, deliberately.** Compiling bash 3.2 requires a pre-GCC-14 toolchain Arch does not carry, and no faithful prebuilt image exists. This is a real reduction in coverage — the suite no longer proves portability to the oldest bash — and it is recorded as such in both `CONTRIBUTING.md` and `ci.yml` rather than quietly omitted. **Verification was done against a live container, not inferred.** Lint, the coverage guard, both portable-parallel shards, and all four portable-serial shards were each confirmed green inside a real Arch container built from this Dockerfile. ## Still open **The Herdr E2E lane has not been validated locally.** It needs the pinned herdr 0.7.4 that the workflow installs; the local environment has 0.7.5, which gives non-representative results. That lane's correctness rests on the pinned install behaving as written, and the first forge run is what will actually prove it. More generally: this branch is the first thing that will ever exercise this pipeline for real. A green run here is the acceptance criterion, and it has to be observed on the forge rather than concluded from the local gate.
The GitHub-inherited workflows asked for ubuntu-latest/macos-latest runners
that do not exist on git.pyr0.dev, so every CI job sat in "waiting" forever.
Rewrite .github/workflows/ci.yml and no-mistakes-required.yml to run on the
forge's `native` label, driving `docker build`/`docker run` against a new
Arch image (.github/ci/Dockerfile) since Arch is the only platform this
software runs on and the `docker` runner label is non-functional here.

Drop the stock-macOS-Bash-3.2 compatibility job rather than reproduce it:
this fork is Arch-only and not published upstream, so there is no macOS
user base to protect, and reproduction itself is impractical (Arch's
compiler toolchain rejects bash 3.2's K&R source, no faithful prebuilt
3.2.57 image exists). Recorded in AGENTS.md and CONTRIBUTING.md.

Also fix a pre-existing, environment-independent bug across 12 test files
that CI never caught because it never ran: several fm-spawn.sh test
fixtures build a project repo via fm_git_worktree/fm_git_init_commit
(no origin remote) and then spawn with --mode no-mistakes, which now hard
fails the delivery-target check that mode requires. Switch those spawns to
--mode local-only where the test does not care about no-mistakes-specific
behavior; wire a real origin + no-mistakes state.sqlite fixture where it
does. Also fix fm-bootstrap.test.sh's missing-node simulation, which
assumed the host has no system node on PATH - not true on Arch.
no-mistakes(document): docs: resync CI docs to forge Arch runner
Some checks failed
CI / Test coverage guard (pull_request) Successful in 1m12s
CI / Behavior portable serial 1 (pull_request) Failing after 5s
CI / Behavior portable serial 2 (pull_request) Failing after 5s
CI / Behavior portable serial 3 (pull_request) Failing after 5s
CI / Behavior portable serial 4 (pull_request) Failing after 5s
CI / Behavior portable parallel 2 (pull_request) Successful in 4m0s
CI / Repo invariants (pull_request) Successful in 4s
Require no-mistakes / PR must be raised via no-mistakes (pull_request) Failing after 2s
CI / Behavior portable parallel 1 (pull_request) Successful in 4m15s
CI / Lint shell scripts (pull_request) Successful in 5m22s
CI / Behavior tests (Herdr) (pull_request) Has been cancelled
843f1debf3
ci: fix serial shard lane string on the forge's act_runner
Some checks failed
CI / Test coverage guard (pull_request) Successful in 11s
CI / Behavior portable parallel 2 (pull_request) Successful in 2m51s
CI / Behavior portable parallel 1 (pull_request) Successful in 3m5s
CI / Lint shell scripts (pull_request) Successful in 4m21s
CI / Behavior portable serial 1 (pull_request) Successful in 12m14s
CI / Behavior portable serial 4 (pull_request) Successful in 10m6s
CI / Repo invariants (pull_request) Successful in 3s
Require no-mistakes / PR must be raised via no-mistakes (pull_request) Failing after 0s
CI / Behavior portable serial 3 (pull_request) Failing after 12m43s
CI / Behavior tests (Herdr) (pull_request) Has been cancelled
CI / Behavior portable serial 2 (pull_request) Has been cancelled
e9cc912dca
strategy.job-total is a GitHub Actions-only expression; this forge's
act_runner (v13) evaluates it empty, dropping the shard count from
FM_SERIAL_LANE ("portable-serial-1of" instead of "...1of4") and failing
every portable-serial shard with "unknown lane" on the very first real
forge CI run (PR #12). Use a literal instead - bin/fm-test-run.sh's own
refusal of a mismatched "ofN" (docs/fm-test-portable-shards.md) is the
actual protection against the matrix and the shard count drifting apart,
and it does not depend on this expression at all.
ci: drop the Require no-mistakes gate, it can never pass on this fleet
Some checks failed
CI / Test coverage guard (pull_request) Successful in 10s
CI / Behavior portable parallel 2 (pull_request) Successful in 2m48s
CI / Behavior portable parallel 1 (pull_request) Successful in 3m3s
CI / Lint shell scripts (pull_request) Successful in 4m23s
CI / Behavior portable serial 1 (pull_request) Successful in 12m13s
CI / Behavior portable serial 4 (pull_request) Successful in 10m7s
CI / Repo invariants (pull_request) Successful in 3s
CI / Behavior portable serial 3 (pull_request) Failing after 12m41s
CI / Behavior portable serial 2 (pull_request) Successful in 15m21s
CI / Behavior tests (Herdr) (pull_request) Successful in 7m34s
4842e010af
The check greps every PR body for the deterministic signature no-mistakes
writes when IT opens a PR. On this fleet, captain policy has firstmate
open every PR and the pipeline's own PR step is configured skipped, so
that signature is never written and never will be - verified against
PRs 4-11, all of which merged with none carrying it. The gate never fired
before because CI itself never ran on this forge; this migration would
have switched on a check that is structurally unable to pass, on every PR
including this one.

Not replaced with a marker firstmate writes into its own PR bodies - that
would be firstmate vouching for itself, a check that cannot fail, which
is a standing rule against here. What should actually gate a merge on
this fork is deliberately left open as separate follow-up work rather
than papered over with a check that only looks like it verifies something.

Updates CONTRIBUTING.md's description of the gate to match.
no-mistakes(test): fix racy exact-age assertion in stale-check digest test
All checks were successful
CI / Behavior portable parallel 1 (pull_request) Successful in 3m9s
CI / Lint shell scripts (pull_request) Successful in 4m22s
CI / Behavior portable serial 1 (pull_request) Successful in 12m20s
CI / Test coverage guard (pull_request) Successful in 11s
CI / Behavior portable parallel 2 (pull_request) Successful in 2m55s
CI / Behavior portable serial 4 (pull_request) Successful in 10m6s
CI / Repo invariants (pull_request) Successful in 3s
CI / Behavior portable serial 3 (pull_request) Successful in 13m46s
CI / Behavior portable serial 2 (pull_request) Successful in 15m22s
CI / Behavior tests (Herdr) (pull_request) Successful in 7m33s
1739bb01eb
test_check_liveness_digest_flags_stale_and_errored asserted the literal
string "stale-check: stale - last run 80s ago", but the digest computes
elapsed time against real wall-clock at the moment it runs, not at the
moment the fixture set ts=now-80. Real elapsed time between those two
points can push the reported age past 80s, especially on a loaded
runner - confirmed failing twice in a row on the forge's shared runner
(portable-serial-3, PR #12), the 13th pre-existing suite bug this
migration has surfaced (unrelated to Arch; would fail identically on the
old runner if it had ever actually run).

Fix parses the reported age out of the digest and asserts it in a band
only genuine staleness reporting can satisfy: never younger than the
fixture's own 80s age, never implausibly older than a same-run digest
could produce (>120s), and always a real non-negative integer with the
exact unit. Deliberately not a wider string/regex match, which would
also accept a broken reading - the assertion must still fail on an
absent value, an empty string, a zero, a missing unit, or an
arbitrarily large number, not just tolerate upward drift.

Verified both directions:
- PASS: 3 consecutive runs against real output, plus a run under
  artificial full-core CPU saturation (the load condition that produces
  the drift) - all green, no flake.
- FAIL: the assertion logic tested standalone against every specified
  broken case - absent line, empty age, zero, missing unit, an
  arbitrarily large number, and a reading younger than the fixture's
  own 80s age - every one correctly rejected with a specific reason;
  only genuine in-band readings (80s-120s) are accepted.
john merged commit 664e74b460 into main 2026-08-19 16:59:32 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
john/firstmate!12
No description provided.