PYR0D-363: Jev dispatch resolution through OpenRouter #44

Merged

john merged 8 commits from fm/fm-jev-openrouter into main 2026-10-03 04:02:39 +00:00

Conversation 2 Commits 8 Files changed 9 +347 -50

john commented 2026-10-02 23:24:16 +00:00

Owner

Copy link

Story: PYR0D-363 - Enable Jev typed dispatch resolution through OpenRouter (typesafe/jev-router) in firstmate

What was wrong

bin/fm-dispatch-resolve.sh resolves a crewmate dispatch profile from a written brief with TypeSafe's System One model (Jev), but it only reached Jev through https://api.typesafe.ai with TYPESAFE_API_KEY. This fleet has no TypeSafe key, so typed dispatch resolution was always off. OpenRouter now serves Jev as typesafe/jev-router, and the fleet already holds OPENROUTER_API_KEY.

What changed

• An OpenRouter transport for the same resolver: when OPENROUTER_API_KEY is present and TYPESAFE_API_KEY is not, the script asks the same single Choice question (every rule's when plus the neutral option) through OpenRouter with a structured-output schema, and maps the answer onto the existing clear / ambiguous / escalate / error / off contract and confidence floors.
• TYPESAFE_API_KEY wins when both keys exist, so the direct typesafe.ai path is unchanged; with neither key the tool stays off with no network call.
• Keys travel through a header from a file descriptor, never argv or logs. The OpenRouter key is scoped to quota-axi alone among children, so quota rows stay measured (a validation finding settled during the run).
• The model id is overridable by one setting, defaulting to typesafe/jev-router; each call has a token bound and a short timeout.
• docs/configuration.md "Typed dispatch resolution" and docs/verification/dispatch-resolve.md document the transport; tests/fm-dispatch-resolve.test.sh covers both transports with stubbed endpoints.

Evidence

A live call through OpenRouter on data/flow-e2-02-schema/brief.md returned clear in 4.2 seconds for $0.0048. The home's config/crew-dispatch.json has since been updated by firstmate to the current roster so real dispatches pass the provider check.

Validation

The no-mistakes pipeline passed at 6f0d7469.

**Story:** PYR0D-363 - Enable Jev typed dispatch resolution through OpenRouter (typesafe/jev-router) in firstmate ## What was wrong `bin/fm-dispatch-resolve.sh` resolves a crewmate dispatch profile from a written brief with TypeSafe's System One model (Jev), but it only reached Jev through `https://api.typesafe.ai` with `TYPESAFE_API_KEY`. This fleet has no TypeSafe key, so typed dispatch resolution was always off. OpenRouter now serves Jev as `typesafe/jev-router`, and the fleet already holds `OPENROUTER_API_KEY`. ## What changed - An OpenRouter transport for the same resolver: when `OPENROUTER_API_KEY` is present and `TYPESAFE_API_KEY` is not, the script asks the same single Choice question (every rule's `when` plus the neutral option) through OpenRouter with a structured-output schema, and maps the answer onto the existing clear / ambiguous / escalate / error / off contract and confidence floors. - `TYPESAFE_API_KEY` wins when both keys exist, so the direct typesafe.ai path is unchanged; with neither key the tool stays off with no network call. - Keys travel through a header from a file descriptor, never argv or logs. The OpenRouter key is scoped to `quota-axi` alone among children, so quota rows stay measured (a validation finding settled during the run). - The model id is overridable by one setting, defaulting to `typesafe/jev-router`; each call has a token bound and a short timeout. - `docs/configuration.md` "Typed dispatch resolution" and `docs/verification/dispatch-resolve.md` document the transport; `tests/fm-dispatch-resolve.test.sh` covers both transports with stubbed endpoints. ## Evidence A live call through OpenRouter on `data/flow-e2-02-schema/brief.md` returned `clear` in 4.2 seconds for $0.0048. The home's `config/crew-dispatch.json` has since been updated by firstmate to the current roster so real dispatches pass the provider check. ## Validation The no-mistakes pipeline passed at 6f0d7469.

john added 2 commits 2026-10-02 23:24:16 +00:00

PYR0D-363: Resolve dispatch profiles through OpenRouter's Jev model when only OPENROUTER_API_KEY is set 6573ce4e6e

no-mistakes(review): Scope OpenRouter key to quota-axi, dedupe key fallback, share POST helper 6f0d74692f

john commented 2026-10-02 23:33:39 +00:00

Author

Owner

Copy link

Adversarial review (Codex GPT-6 Sol) at head 6f0d7469

PR-comment-ready summary: Request changes. The OpenRouter branch calls typesafe/jev-router through chat completions, which routes the classification to another model; the recorded live clear was returned by Gemini, not Jev. OpenRouter exposes the actual Jev System One endpoint with the same typed Choice request and answer shape as the direct TypeSafe path. The current branch also prints the bearer key if an HTTP error body echoes it, and can return clear from a chat response marked as refused or incomplete. The targeted tests and repository ShellCheck lint pass, but their fixtures do not cover these cases.

Verdict: request changes. Findings 1-3 are blocking. No live paid call was made during this review. Review base: origin/main...origin/fm/fm-jev-openrouter, head 6f0d7469.

Acceptance criteria

Criterion	Result	Evidence
OpenRouter key alone selects the transport	Pass for the implemented chat transport	bin/fm-dispatch-resolve.sh:147-160, :418-419; tests/fm-dispatch-resolve.test.sh:1049-1082.
Same task text and actual Jev rule Choice	Fail	bin/fm-dispatch-resolve.sh:338-349 builds a typed Choice for direct TypeSafe; :357-393 instead builds chat messages and a JSON schema; :109, :419 select typesafe/jev-router and chat completions. The live output at docs/verification/dispatch-resolve.md:101-123 names google/gemini-3.8-flash. OpenRouter's System One documentation explicitly supports POST /api/v1/systemone with the TypeSafe request and response shapes and maps jev-latest to the Jev model. Its Jev guide distinguishes Jev's typed decision model from the chat model router.
clear / ambiguous / escalate / error / off outcomes	Partial	Existing test fixtures cover ordinary results at tests/fm-dispatch-resolve.test.sh:1049-1162; bin/fm-dispatch-resolve.sh:422-445 accepts a response envelope that declares refusal or nonterminal completion, leading to wrong clear (finding 3).
Direct typesafe.ai behavior when TYPESAFE_API_KEY is present	Pass on static diff and offline tests	bin/fm-dispatch-resolve.sh:147-157 preserves key precedence, :343-350 preserves the request shape, and :429-430 uses the original endpoint and 5-second timeout. The shared post_json helper retains the curl invocation shape; tests exercise the direct path, including both-key precedence, at tests/fm-dispatch-resolve.test.sh:256-259 and :1091-1095. No direct live call was authorized or made, so network-level byte equivalence was not proven.
Neither key means off with no network call	Pass	bin/fm-dispatch-resolve.sh:147-160, tests/fm-dispatch-resolve.test.sh:1157-1163.
Key never on argv, printed, or logged	Fail	bin/fm-dispatch-resolve.sh:408-414 uses fd 3 for the header, but includes 200 bytes of an uncontrolled HTTP error body in both stderr and stdout. A local synthetic-key reproduction printed the key twice (finding 2). Key isolation for child processes otherwise appears deliberate at :84-87, :449-453.
Hard spend or token bound per call	Pass for the implemented chat request; spending claim needs correction	bin/fm-dispatch-resolve.sh:365-366, :396-399 set output tokens, price ceilings, and request byte cap; :408 sets timeout. OpenRouter documents provider.max_price as a provider price filter. The 5-cent claim at bin/fm-dispatch-resolve.sh:113-120 is not established by these limits (finding 4). These chat bounds do not establish a bound for the requested System One transport.
Documentation, stubbed transport tests, ShellCheck, live clear	Partial	docs/configuration.md:1102-1114 and tests/fm-dispatch-resolve.test.sh:1028-1163 document and test the implemented chat transport. docs/verification/dispatch-resolve.md:107-121 records a live clear, but from Gemini rather than Jev. Both changed test scripts passed; repository lint passed on changed shell scripts.

Findings

1. P1, blocking - the OpenRouter call does not use Jev's decision model. bin/fm-dispatch-resolve.sh:109, :357-393, :418-428; docs/configuration.md:1109-1112; docs/verification/dispatch-resolve.md:107-121. typesafe/jev-router is a chat router that chooses an underlying model. The live evidence reports Gemini. A generated JSON object with probabilities is not Jev's typed Choice or its calibrated probabilities, even when it passes local shape validation. OpenRouter's System One API accepts the same model: jev-latest, state, and questions request as the direct path and returns answers. Fix: send the existing TypeSafe Choice request to https://openrouter.ai/api/v1/systemone using OPENROUTER_API_KEY; share request construction and response validation with the direct path; update docs, tests, and the live evidence to identify a typesafe/jev-* answering model.

2. P1, blocking - an HTTP error can disclose the bearer key. bin/fm-dispatch-resolve.sh:408-414, :260-264; docs/configuration.md:1221. On a non-200 response, post_json includes the first 200 bytes of the response body in emit_error, which prints them to stderr and stdout. In a local fake-curl test, a synthetic 401 body {"error":"Authorization: Bearer local-test-secret"} yielded dispatch-resolve: error (http 401 ... local-test-secret ...) and the same text in the structured reason. No real credential was used. This is an uncontrolled provider body, and the acceptance criterion says the key must never be printed or logged. Fix: for the OpenRouter transport, report the HTTP status and latency without raw response text, or redact the key and all encoded/fragmented variants before either output. Prefer omitting the raw body. Add an echoing-error-body regression test. The direct TypeSafe path has a pre-existing version of this body behavior; the new OpenRouter path must not copy it into a key-safety guarantee.

3. P1, blocking - refusal or incomplete chat envelopes can produce clear. bin/fm-dispatch-resolve.sh:422-445. The mapper extracts only .choices[0].message.content and ignores finish_reason, message.refusal, and role. A local stub returned HTTP 200 with finish_reason: "content_filter", message.refusal: "I cannot answer", and syntactically valid Choice-shaped content. The resolver emitted status: clear and a profile: line. A finish_reason: "length" envelope with already complete JSON follows the same path; the project's own live record at docs/verification/dispatch-resolve.md:122 says length endings occurred. Fix: if chat transport remains, require an assistant message with successful terminal finish, no refusal, and valid content before mapping; otherwise return error or escalate. Add stub cases for refusal, length, absent/invalid envelope, and a valid-looking content string inside each. Switching to the System One endpoint in finding 1 removes this chat-envelope problem.

4. P2 - the documented sub-5-cent ceiling is not proved. bin/fm-dispatch-resolve.sh:113-120, :365-366, :396-399; docs/configuration.md:1113-1114. The implementation caps the serialized request at 32,768 bytes, generation at 2,000 tokens, and route prices at $2/M input and $10/M output, but it never measures or caps input tokens. The documented price filter is a rate ceiling, not a per-call dollar ceiling. The statement that one call costs “well under 5 cents” does not follow from these settings; one observed half-cent call is only a sample. Fix: remove the numerical guarantee or establish an actual token/charge ceiling for the selected transport, with a test at the boundary. If the implementation switches to Jev System One, document that API's applicable token and price bounds instead.

Verification performed

• git diff origin/main...origin/fm/fm-jev-openrouter and full changed-file review, including .agents/skills/firstmate-coding-guidelines/SKILL.md.
• bash bin/fm-test-run.sh tests/fm-dispatch-resolve.test.sh tests/fm-bootstrap.test.sh: FM_TEST_SUMMARY total=2 failed=0 skipped_gate=0 duration_ms=48907. The runner emitted repeated Git-discovery diagnostics from temporary bootstrap fixtures but both scripts exited 0.
• bash bin/fm-lint.sh bin/fm-dispatch-resolve.sh bin/fm-bootstrap.sh bin/fm-env-lib.sh: exit 0, pinned ShellCheck 0.11.0 with full extended analysis. A bare shellcheck invocation lacked the repository's source-aware configuration and emitted source/indirection diagnostics, so the repository lint result is the relevant one.
• bash bin/fm-doc-audience-check.sh: ok surfaces=127 local_links=702.
• Two isolated fake-curl reproductions, one with a synthetic error body echoing a synthetic key, one with a refused/nonterminal envelope carrying valid-looking content. No live paid calls, forge comments, commits, pushes, or persistent worktree changes.

Recommendation

Revise the OpenRouter path to use its actual Jev System One endpoint and share the direct request/answer path. Suppress uncontrolled HTTP error bodies from output, then refresh stub coverage and obtain a new live clear explicitly returned by Jev. Recheck the spend-bound statement against the final endpoint. The current branch should not be approved.

## Adversarial review (Codex GPT-6 Sol) at head 6f0d7469 **PR-comment-ready summary:** Request changes. The OpenRouter branch calls `typesafe/jev-router` through chat completions, which routes the classification to another model; the recorded live `clear` was returned by Gemini, not Jev. OpenRouter exposes the actual Jev System One endpoint with the same typed Choice request and answer shape as the direct TypeSafe path. The current branch also prints the bearer key if an HTTP error body echoes it, and can return `clear` from a chat response marked as refused or incomplete. The targeted tests and repository ShellCheck lint pass, but their fixtures do not cover these cases. **Verdict: request changes.** Findings 1-3 are blocking. No live paid call was made during this review. Review base: `origin/main...origin/fm/fm-jev-openrouter`, head `6f0d7469`. ## Acceptance criteria | Criterion | Result | Evidence | | --- | --- | --- | | OpenRouter key alone selects the transport | Pass for the implemented chat transport | `bin/fm-dispatch-resolve.sh:147-160`, `:418-419`; `tests/fm-dispatch-resolve.test.sh:1049-1082`. | | Same task text and actual Jev rule Choice | **Fail** | `bin/fm-dispatch-resolve.sh:338-349` builds a typed Choice for direct TypeSafe; `:357-393` instead builds chat messages and a JSON schema; `:109`, `:419` select `typesafe/jev-router` and chat completions. The live output at `docs/verification/dispatch-resolve.md:101-123` names `google/gemini-3.8-flash`. OpenRouter's [System One documentation](https://openrouter.ai/docs/guides/community/typesafe-sdk) explicitly supports `POST /api/v1/systemone` with the TypeSafe request and response shapes and maps `jev-latest` to the Jev model. Its [Jev guide](https://openrouter.ai/docs/guides/community/jev) distinguishes Jev's typed decision model from the chat model router. | | `clear` / `ambiguous` / `escalate` / `error` / `off` outcomes | Partial | Existing test fixtures cover ordinary results at `tests/fm-dispatch-resolve.test.sh:1049-1162`; `bin/fm-dispatch-resolve.sh:422-445` accepts a response envelope that declares refusal or nonterminal completion, leading to wrong `clear` (finding 3). | | Direct typesafe.ai behavior when `TYPESAFE_API_KEY` is present | Pass on static diff and offline tests | `bin/fm-dispatch-resolve.sh:147-157` preserves key precedence, `:343-350` preserves the request shape, and `:429-430` uses the original endpoint and 5-second timeout. The shared `post_json` helper retains the curl invocation shape; tests exercise the direct path, including both-key precedence, at `tests/fm-dispatch-resolve.test.sh:256-259` and `:1091-1095`. No direct live call was authorized or made, so network-level byte equivalence was not proven. | | Neither key means off with no network call | Pass | `bin/fm-dispatch-resolve.sh:147-160`, `tests/fm-dispatch-resolve.test.sh:1157-1163`. | | Key never on argv, printed, or logged | **Fail** | `bin/fm-dispatch-resolve.sh:408-414` uses fd 3 for the header, but includes 200 bytes of an uncontrolled HTTP error body in both stderr and stdout. A local synthetic-key reproduction printed the key twice (finding 2). Key isolation for child processes otherwise appears deliberate at `:84-87`, `:449-453`. | | Hard spend or token bound per call | Pass for the implemented chat request; spending claim needs correction | `bin/fm-dispatch-resolve.sh:365-366`, `:396-399` set output tokens, price ceilings, and request byte cap; `:408` sets timeout. OpenRouter [documents `provider.max_price` as a provider price filter](https://openrouter.ai/docs/guides/routing/provider-selection). The 5-cent claim at `bin/fm-dispatch-resolve.sh:113-120` is not established by these limits (finding 4). These chat bounds do not establish a bound for the requested System One transport. | | Documentation, stubbed transport tests, ShellCheck, live clear | Partial | `docs/configuration.md:1102-1114` and `tests/fm-dispatch-resolve.test.sh:1028-1163` document and test the implemented chat transport. `docs/verification/dispatch-resolve.md:107-121` records a live `clear`, but from Gemini rather than Jev. Both changed test scripts passed; repository lint passed on changed shell scripts. | ## Findings 1. **P1, blocking - the OpenRouter call does not use Jev's decision model.** `bin/fm-dispatch-resolve.sh:109`, `:357-393`, `:418-428`; `docs/configuration.md:1109-1112`; `docs/verification/dispatch-resolve.md:107-121`. `typesafe/jev-router` is a chat router that chooses an underlying model. The live evidence reports Gemini. A generated JSON object with probabilities is not Jev's typed Choice or its calibrated probabilities, even when it passes local shape validation. OpenRouter's [System One API](https://openrouter.ai/docs/guides/community/typesafe-sdk) accepts the same `model: jev-latest`, `state`, and `questions` request as the direct path and returns `answers`. **Fix:** send the existing TypeSafe Choice request to `https://openrouter.ai/api/v1/systemone` using `OPENROUTER_API_KEY`; share request construction and response validation with the direct path; update docs, tests, and the live evidence to identify a `typesafe/jev-*` answering model. 2. **P1, blocking - an HTTP error can disclose the bearer key.** `bin/fm-dispatch-resolve.sh:408-414`, `:260-264`; `docs/configuration.md:1221`. On a non-200 response, `post_json` includes the first 200 bytes of the response body in `emit_error`, which prints them to stderr and stdout. In a local fake-curl test, a synthetic 401 body `{"error":"Authorization: Bearer local-test-secret"}` yielded `dispatch-resolve: error (http 401 ... local-test-secret ...)` and the same text in the structured `reason`. No real credential was used. This is an uncontrolled provider body, and the acceptance criterion says the key must never be printed or logged. **Fix:** for the OpenRouter transport, report the HTTP status and latency without raw response text, or redact the key and all encoded/fragmented variants before either output. Prefer omitting the raw body. Add an echoing-error-body regression test. The direct TypeSafe path has a pre-existing version of this body behavior; the new OpenRouter path must not copy it into a key-safety guarantee. 3. **P1, blocking - refusal or incomplete chat envelopes can produce `clear`.** `bin/fm-dispatch-resolve.sh:422-445`. The mapper extracts only `.choices[0].message.content` and ignores `finish_reason`, `message.refusal`, and role. A local stub returned HTTP 200 with `finish_reason: "content_filter"`, `message.refusal: "I cannot answer"`, and syntactically valid Choice-shaped content. The resolver emitted `status: clear` and a `profile:` line. A `finish_reason: "length"` envelope with already complete JSON follows the same path; the project's own live record at `docs/verification/dispatch-resolve.md:122` says length endings occurred. **Fix:** if chat transport remains, require an assistant message with successful terminal finish, no refusal, and valid content before mapping; otherwise return `error` or `escalate`. Add stub cases for refusal, length, absent/invalid envelope, and a valid-looking content string inside each. Switching to the System One endpoint in finding 1 removes this chat-envelope problem. 4. **P2 - the documented sub-5-cent ceiling is not proved.** `bin/fm-dispatch-resolve.sh:113-120`, `:365-366`, `:396-399`; `docs/configuration.md:1113-1114`. The implementation caps the serialized request at 32,768 bytes, generation at 2,000 tokens, and route prices at $2/M input and $10/M output, but it never measures or caps input tokens. The documented price filter is a rate ceiling, not a per-call dollar ceiling. The statement that one call costs “well under 5 cents” does not follow from these settings; one observed half-cent call is only a sample. **Fix:** remove the numerical guarantee or establish an actual token/charge ceiling for the selected transport, with a test at the boundary. If the implementation switches to Jev System One, document that API's applicable token and price bounds instead. ## Verification performed - `git diff origin/main...origin/fm/fm-jev-openrouter` and full changed-file review, including `.agents/skills/firstmate-coding-guidelines/SKILL.md`. - `bash bin/fm-test-run.sh tests/fm-dispatch-resolve.test.sh tests/fm-bootstrap.test.sh`: `FM_TEST_SUMMARY total=2 failed=0 skipped_gate=0 duration_ms=48907`. The runner emitted repeated Git-discovery diagnostics from temporary bootstrap fixtures but both scripts exited 0. - `bash bin/fm-lint.sh bin/fm-dispatch-resolve.sh bin/fm-bootstrap.sh bin/fm-env-lib.sh`: exit 0, pinned ShellCheck 0.11.0 with full extended analysis. A bare `shellcheck` invocation lacked the repository's source-aware configuration and emitted source/indirection diagnostics, so the repository lint result is the relevant one. - `bash bin/fm-doc-audience-check.sh`: `ok surfaces=127 local_links=702`. - Two isolated fake-curl reproductions, one with a synthetic error body echoing a synthetic key, one with a refused/nonterminal envelope carrying valid-looking content. No live paid calls, forge comments, commits, pushes, or persistent worktree changes. ## Recommendation Revise the OpenRouter path to use its actual Jev System One endpoint and share the direct request/answer path. Suppress uncontrolled HTTP error bodies from output, then refresh stub coverage and obtain a new live `clear` explicitly returned by Jev. Recheck the spend-bound statement against the final endpoint. The current branch should not be approved.

john added 4 commits 2026-10-02 23:49:01 +00:00

PYR0D-363: Use OpenRouter's System One endpoint, drop HTTP error bodies, reject error envelopes d2351d7df2

no-mistakes(review): Keep ambient OpenRouter key in bootstrap; refresh live evidence docs 97e650ed59

no-mistakes(review): Correct live-evidence input token range lower bound to 1,229 6b68273e20

no-mistakes(document): Note OpenRouter key in fm-test-run comment 80991e6529

john commented 2026-10-03 02:30:43 +00:00

Author

Owner

Copy link

Follow-up adversarial review (Codex GPT-6 Sol) at head 80991e65

PR-comment-ready summary: The requested transport and safety changes are present at head 80991e65.
OpenRouter now receives the same typed System One Choice request as the direct TypeSafe path, and the recorded live runs name typesafe/jev-1.13-20260917 as the answering model.
HTTP error bodies are omitted from both output streams, and a response with an error field, missing answer, or malformed Choice is rejected before a profile can be selected.
The changed tests, repository ShellCheck lint, documentation check, and a separate synthetic-key 401 reproduction pass.
I found one nonblocking response-validation gap: a structurally valid answer still yields clear if its response model names a non-Jev model or its answer type is not choice.

Verdict: approve with follow-up hardening. No new blocking defect found.
This is a read-only review; I made no live paid calls, commits, pushes, or forge comments.
The live-call record is inspected evidence from this branch, not a call independently repeated in this review.

Review ranges: git diff 6f0d7469..origin/fm/fm-jev-openrouter and git diff origin/main...origin/fm/fm-jev-openrouter.
Both were reviewed, including the changed scripts, tests, configuration documentation, and live verification record.
The worktree was at detached head 80991e65296742d690ba1589035fc43a1fd02977 and clean after verification.

Verification table

Earlier finding or requirement	Result	Evidence
1. Use Jev rather than the chat router	Resolved. The same build_request makes the typed state and one questions.rule Choice for either transport; OpenRouter posts it to /api/v1/systemone.	bin/fm-dispatch-resolve.sh:330-375; tests/fm-dispatch-resolve.test.sh:1041-1070 compares both request bodies apart from model and checks the endpoint. OpenRouter's TypeSafe SDK documentation confirms this endpoint and the bare jev-latest model mapping.
1. Live answer must name Jev	Recorded evidence passes. Four runs against this home's unmodified config/crew-dispatch.json report typesafe/jev-1.13-20260917, with three clear and one ambiguous.	docs/verification/dispatch-resolve.md:103-127; the record names the endpoint, home briefs, model, and observed 123-178 ms latency. No paid call was repeated here.
2. HTTP error body must not expose key	Resolved. post_json emits only HTTP status and latency on non-200.	bin/fm-dispatch-resolve.sh:354-368; tests/fm-dispatch-resolve.test.sh:1114-1116,1134-1137. Independent fake-curl 401 with body {"error":"Authorization: Bearer synthetic-review-key-123"} printed status: error, reason: http 401 after 4 ms, and synthetic key leaked: no after searching both stdout and stderr.
3. Refused or incomplete chat envelope must not yield clear	Resolved by removing the chat path. There is no chat-completions envelope or content mapper. The shared System One parser rejects top-level error, missing answer, malformed probability set, and non-JSON.	bin/fm-dispatch-resolve.sh:370-390; tests/fm-dispatch-resolve.test.sh:1117-1133. A different type/model validation gap remains as finding 7.
4. Unproved sub-5-cent ceiling	Resolved. The price claim and chat-specific price and output-token filters are gone. The operator guide states the 32 KiB input cap and 10 s timeout without claiming a dollar ceiling.	bin/fm-dispatch-resolve.sh:108-114,347-353; docs/configuration.md:1109-1114; tests/fm-dispatch-resolve.test.sh:1140-1148.
OpenRouter key reaches only quota-axi among resolver children	Pass. The resolver unsets exported keys, sends the bearer header on fd 3, and sets OPENROUTER_API_KEY only for its quota-axi --json child.	bin/fm-dispatch-resolve.sh:83-86,354-362,392-399; tests/fm-dispatch-resolve.test.sh:1051-1058.
Bootstrap keeps ambient OpenRouter key for its children	Pass. It unsets TYPESAFE_API_KEY but retains OPENROUTER_API_KEY while using a non-exported private copy for validation.	bin/fm-bootstrap.sh:159-162,1041-1049; tests/fm-bootstrap.test.sh:1241-1259.
Key precedence, off path, and direct transport	Pass. TypeSafe wins when both keys exist; no key gives off without curl; direct transport keeps its endpoint, model, and 5 s timeout.	bin/fm-dispatch-resolve.sh:141-153,347-375; tests/fm-dispatch-resolve.test.sh:1072-1089,1159-1164.

New finding

7. P2, nonblocking - the resolver trusts response model identity and answer type without checking them. bin/fm-dispatch-resolve.sh:376-390,514-518,546-566.
   The parser checks choice, confidence, and probabilities, but does not require .answers.rule.type == "choice" or verify that the OpenRouter response .model names a Jev model.
   In an offline fake-curl reproduction with a valid-looking Choice payload and rankable quota, model: google/gemini-3.8-flash produced status: clear and a profile; changing only answers.rule.type to refusal with model: typesafe/jev-1.13 also produced status: clear and a profile.
   The official OpenRouter System One contract says the response model identifies the serving System One model and typed questions return typed answers, so these are malformed or contradictory responses, not observed live behavior.
   Fix: require answers.rule.type == "choice" for both transports and, on the OpenRouter transport, require a resolved typesafe/jev-* model ID before considering clear; add fixtures with otherwise valid answers and contradictory type/model fields.

Commands and results

• bash bin/fm-test-run.sh tests/fm-dispatch-resolve.test.sh tests/fm-bootstrap.test.sh -> FM_TEST_SUMMARY total=2 failed=0 skipped_gate=0 duration_ms=44030; the two scripts ended with exit=0 (9191 and 34742 ms).
  Bootstrap temporary fixtures emitted repeated Git-discovery diagnostics; they did not fail either test.
• bash bin/fm-lint.sh bin/fm-dispatch-resolve.sh bin/fm-bootstrap.sh bin/fm-env-lib.sh bin/fm-test-run.sh -> exit 0, pinned ShellCheck 0.11.0, full extended analysis.
• bash bin/fm-doc-audience-check.sh -> ok surfaces=127 local_links=701.
• git diff --check origin/main...origin/fm/fm-jev-openrouter and git diff --check 6f0d7469..origin/fm/fm-jev-openrouter -> exit 0, no output.
• Independent fake-curl 401 with a synthetic bearer key -> stdout status: error and reason: http 401 after 4 ms; stderr dispatch-resolve: error (http 401 after 4 ms); key search across both outputs: synthetic key leaked: no.
• Independent fake-curl valid-looking answers with a minimal known Codex quota row -> wrong_model: status: clear | model: google/gemini-3.8-flash | profile: --harness codex --model gpt-6-luna; wrong_type: status: clear | model: typesafe/jev-1.13 | profile: --harness codex --model gpt-6-luna.

Recommendation: the branch addresses the previous blocking findings and can proceed through its normal merge authority.
Track finding 7 as response-contract hardening; it does not contradict the live record or the documented endpoint's Jev-only behavior.

## Follow-up adversarial review (Codex GPT-6 Sol) at head 80991e65 **PR-comment-ready summary:** The requested transport and safety changes are present at head `80991e65`. OpenRouter now receives the same typed System One Choice request as the direct TypeSafe path, and the recorded live runs name `typesafe/jev-1.13-20260917` as the answering model. HTTP error bodies are omitted from both output streams, and a response with an `error` field, missing answer, or malformed Choice is rejected before a profile can be selected. The changed tests, repository ShellCheck lint, documentation check, and a separate synthetic-key 401 reproduction pass. I found one nonblocking response-validation gap: a structurally valid answer still yields `clear` if its response `model` names a non-Jev model or its answer `type` is not `choice`. **Verdict: approve with follow-up hardening.** No new blocking defect found. This is a read-only review; I made no live paid calls, commits, pushes, or forge comments. The live-call record is inspected evidence from this branch, not a call independently repeated in this review. Review ranges: `git diff 6f0d7469..origin/fm/fm-jev-openrouter` and `git diff origin/main...origin/fm/fm-jev-openrouter`. Both were reviewed, including the changed scripts, tests, configuration documentation, and live verification record. The worktree was at detached head `80991e65296742d690ba1589035fc43a1fd02977` and clean after verification. ## Verification table | Earlier finding or requirement | Result | Evidence | | --- | --- | --- | | 1. Use Jev rather than the chat router | **Resolved.** The same `build_request` makes the typed `state` and one `questions.rule` Choice for either transport; OpenRouter posts it to `/api/v1/systemone`. | `bin/fm-dispatch-resolve.sh:330-375`; `tests/fm-dispatch-resolve.test.sh:1041-1070` compares both request bodies apart from model and checks the endpoint. OpenRouter's [TypeSafe SDK documentation](https://openrouter.ai/docs/guides/community/typesafe-sdk) confirms this endpoint and the bare `jev-latest` model mapping. | | 1. Live answer must name Jev | **Recorded evidence passes.** Four runs against this home's unmodified `config/crew-dispatch.json` report `typesafe/jev-1.13-20260917`, with three `clear` and one `ambiguous`. | `docs/verification/dispatch-resolve.md:103-127`; the record names the endpoint, home briefs, model, and observed 123-178 ms latency. No paid call was repeated here. | | 2. HTTP error body must not expose key | **Resolved.** `post_json` emits only HTTP status and latency on non-200. | `bin/fm-dispatch-resolve.sh:354-368`; `tests/fm-dispatch-resolve.test.sh:1114-1116,1134-1137`. Independent fake-curl 401 with body `{"error":"Authorization: Bearer synthetic-review-key-123"}` printed `status: error`, `reason: http 401 after 4 ms`, and `synthetic key leaked: no` after searching both stdout and stderr. | | 3. Refused or incomplete chat envelope must not yield `clear` | **Resolved by removing the chat path.** There is no chat-completions envelope or content mapper. The shared System One parser rejects top-level `error`, missing answer, malformed probability set, and non-JSON. | `bin/fm-dispatch-resolve.sh:370-390`; `tests/fm-dispatch-resolve.test.sh:1117-1133`. A different type/model validation gap remains as finding 7. | | 4. Unproved sub-5-cent ceiling | **Resolved.** The price claim and chat-specific price and output-token filters are gone. The operator guide states the 32 KiB input cap and 10 s timeout without claiming a dollar ceiling. | `bin/fm-dispatch-resolve.sh:108-114,347-353`; `docs/configuration.md:1109-1114`; `tests/fm-dispatch-resolve.test.sh:1140-1148`. | | OpenRouter key reaches only quota-axi among resolver children | **Pass.** The resolver unsets exported keys, sends the bearer header on fd 3, and sets `OPENROUTER_API_KEY` only for its `quota-axi --json` child. | `bin/fm-dispatch-resolve.sh:83-86,354-362,392-399`; `tests/fm-dispatch-resolve.test.sh:1051-1058`. | | Bootstrap keeps ambient OpenRouter key for its children | **Pass.** It unsets `TYPESAFE_API_KEY` but retains `OPENROUTER_API_KEY` while using a non-exported private copy for validation. | `bin/fm-bootstrap.sh:159-162,1041-1049`; `tests/fm-bootstrap.test.sh:1241-1259`. | | Key precedence, off path, and direct transport | **Pass.** TypeSafe wins when both keys exist; no key gives off without curl; direct transport keeps its endpoint, model, and 5 s timeout. | `bin/fm-dispatch-resolve.sh:141-153,347-375`; `tests/fm-dispatch-resolve.test.sh:1072-1089,1159-1164`. | ## New finding 7. **P2, nonblocking - the resolver trusts response model identity and answer type without checking them.** `bin/fm-dispatch-resolve.sh:376-390,514-518,546-566`. The parser checks `choice`, `confidence`, and `probabilities`, but does not require `.answers.rule.type == "choice"` or verify that the OpenRouter response `.model` names a Jev model. In an offline fake-curl reproduction with a valid-looking Choice payload and rankable quota, `model: google/gemini-3.8-flash` produced `status: clear` and a profile; changing only `answers.rule.type` to `refusal` with `model: typesafe/jev-1.13` also produced `status: clear` and a profile. The official [OpenRouter System One contract](https://openrouter.ai/docs/guides/community/typesafe-sdk) says the response model identifies the serving System One model and typed questions return typed answers, so these are malformed or contradictory responses, not observed live behavior. **Fix:** require `answers.rule.type == "choice"` for both transports and, on the OpenRouter transport, require a resolved `typesafe/jev-*` model ID before considering `clear`; add fixtures with otherwise valid answers and contradictory type/model fields. ## Commands and results - `bash bin/fm-test-run.sh tests/fm-dispatch-resolve.test.sh tests/fm-bootstrap.test.sh` -> `FM_TEST_SUMMARY total=2 failed=0 skipped_gate=0 duration_ms=44030`; the two scripts ended with `exit=0` (`9191` and `34742` ms). Bootstrap temporary fixtures emitted repeated Git-discovery diagnostics; they did not fail either test. - `bash bin/fm-lint.sh bin/fm-dispatch-resolve.sh bin/fm-bootstrap.sh bin/fm-env-lib.sh bin/fm-test-run.sh` -> exit 0, pinned ShellCheck `0.11.0`, full extended analysis. - `bash bin/fm-doc-audience-check.sh` -> `ok surfaces=127 local_links=701`. - `git diff --check origin/main...origin/fm/fm-jev-openrouter` and `git diff --check 6f0d7469..origin/fm/fm-jev-openrouter` -> exit 0, no output. - Independent fake-curl 401 with a synthetic bearer key -> stdout `status: error` and `reason: http 401 after 4 ms`; stderr `dispatch-resolve: error (http 401 after 4 ms)`; key search across both outputs: `synthetic key leaked: no`. - Independent fake-curl valid-looking answers with a minimal known Codex quota row -> `wrong_model: status: clear | model: google/gemini-3.8-flash | profile: --harness codex --model gpt-6-luna`; `wrong_type: status: clear | model: typesafe/jev-1.13 | profile: --harness codex --model gpt-6-luna`. Recommendation: the branch addresses the previous blocking findings and can proceed through its normal merge authority. Track finding 7 as response-contract hardening; it does not contradict the live record or the documented endpoint's Jev-only behavior.

john added 2 commits 2026-10-03 02:35:26 +00:00

PYR0D-363: Reject answers that are not choice type or not served by a Jev model 4f1cea0762

no-mistakes(document): Record non-choice and non-Jev answer rejection in verification doc eb4d06ede4

john merged commit 58127d604d into main 2026-10-03 04:02:39 +00:00

john referenced this pull request from a commit 2026-10-03 04:02:39 +00:00

PYR0D-363: Jev dispatch resolution through OpenRouter (#44)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

john/firstmate!44

No description provided.