Read UK staging contract v3: a real blocked run status, block details and failure classes

[juaristi22]

Summary

Implements #206, the dashboard and collector half of PolicyEngine/microcosm#1147. UK staging documents move to schema_version 3: a run whose gates refuse its candidate closes as blocked, with a block (gate phase, blocking failure count, blocking gate ids), and every failure carries a failure_class. Version 1 and 2 runs read exactly as before.

Contract (staging-contract.ts)

• Accepts v3 exactly: run status running | completed | blocked | failed, event status adds blocked, block required on run manifests and progress, five-key failure with failure_class (lowercase identifier or null).
• Rules: blocked requires block and failure: null; failed requires failure and block: null; any other status has neither. A block's gate ids may be empty; its count is at least 1.
• Every check that was pinned to schema_version === 2 now treats versions 2 and 3 as the structured contract: event sequence, the mixed-version check and the run-consistency checks. A run cannot mix versions.

Readers (staging-artifact.ts, build-runs.ts)

• v3 manifests list as structured runs with their own version. They never fall back to the v1 runs.json index.
• A v3 run's calibration diagnostics come from the declared artifact path, and their digest is checked.
• blocked is a final status, so blocked runs get the long cache time and are not fetched again on every request.

Monitor (build-monitor.ts, views)

• blocked maps straight to the blocked state. The failure line comes from the block, e.g. "The gates refused the candidate at preflight (2 blocking failures, no gate named)". The block is read from the run documents, with the terminal blocked event's details as a fallback (the collector's copy).
• Today's inference from blocking_failure_count stays for version 2 runs only.
• A v3 failure's code and class come from progress.failure, falling back to the terminal event.
• Stop statistics count the class of a recorded refusal (v3) but not an inferred one (v2). A refusal with no gate failure lines lists the gates its block names.
• Labels for refused, aborted, unrecorded_gate_block, build_failure, unexpected_process_exit and dry_run_refusal. Blocked runs get a warning tone on the staging page and a blocked pill in its event table. Lists of names, such as the blocking gate ids, show as detail chips there.

Collector (telemetry-service)

• TelemetryEvent.status accepts blocked.
• A run / blocked event sets status blocked, current_stage blocked and ended_at, and keeps the block in the materialized progress and manifest.
• Failed events keep error_code in failure.
• A blocked event without a valid block (a safe-identifier phase, a blocking_failure_count of at least 1, and blocking_gate_ids, which may be empty) gets a 422, so a block of nulls is never stored. gate_statuses stays optional.
• No migration: status is Text, and the block is rebuilt from the events on every read.
• The README now lists which responses a producer treats as final (403, 404, 409, 413, 422) and which it retries.

Fixtures

• fixtures/staging-contract/v3/ is copied byte for byte from Make UK build outcomes honest in staging, telemetry and the Logbook microcosm#1147 at head 0e80b0361, with SHA256SUMS digest e31a5e33…bf3def5 pinned. The v2 digest 0d54c630…455e5 is unchanged. If #1147's fixtures change before it merges, this copy and the digest must be updated with them.

Before merging

• Make UK build outcomes honest in staging, telemetry and the Logbook microcosm#1147's v3 fixtures are unchanged: git show <#1147 head>:packages/microcosm-build/tests/fixtures/staging/v3/SHA256SUMS | shasum -a 256 gives e31a5e337840e24d7a1203b83eb1a5039eeb5f35777a895ab6ab28163bf3def5 (last checked at b82df99ac). If it doesn't, copy the fixtures again and update the pinned digest.

Order of operations

This must be deployed, and a blocked event posted to the qualification collector must return 2xx, before PolicyEngine/microcosm#1147 merges.

Verification

• cd frontend && bun test: 581 pass, 0 fail. New cases: v3 blocked at terminal (fixture) and at preflight (no gate counts, no gate named); a v3 failed run's code and class in the stop statistics; a v3 completed run with gate counts stays passed; a collector run closed by a blocked event; v3 listing and the diagnostics digest check; rejection of a sixth failure key, a missing failure_class, a block on a failed run, a blocked run without a block, a bad block, and v3 fields on v2 documents. A mutation check (versions pinned back to 2) turns three v3 tests red.
• bun run lint (tsc --noEmit) clean; bun run build passes.
• cd telemetry-service && uv run pytest -q: 80 passed, 8 skipped (the PostgreSQL integration tests need TEST_DATABASE_URL; CI runs them). New cases include a 422 for eight malformed blocked events and a 202 for one naming no gate. ruff check and ruff format --check clean.
• Rendered locally with MICROCOSM_LOCAL_RUNS_DIR pointing at the v3 fixtures plus a preflight-blocked run. The Build progress tab shows "Blocked by gates · at Preflight gates" with "2 blocking failures, no gate named", the terminal block with uk_target_fit, and the failed run with class Error and BUILD_FAILED.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
calibration-diagnostics	Ready	Preview	Oct 9, 2026 3:35pm UTC

[vahid-ahmadi]

Automated review pass (Claude Code, high effort) — round 1 at fc3820e8

Verdict: approve. The reader accepts exactly what microcosm#1147 writes at 0e80b0361, v1 and v2 runs read as before, and the collector takes a blocked event with a 2xx. It can be deployed ahead of #1147. The points below are small.

What checks out

• Fixtures match byte for byte. All 15 files under fixtures/staging-contract/v3/ are identical to packages/microcosm-build/tests/fixtures/staging/v3/ at #1147's head (0e80b0361), with the same file set.
• The contract matches the producer field for field (#1147's staging_v2.py):
  • Run status: running | completed | blocked | failed.
  • Event status: adds blocked.
  • Failure: five keys, with failure_class either null or ^[a-z][a-z0-9_]*$.
  • Block: exactly phase / blocking_failure_count (≥ 1) / blocking_gate_ids (strings, 1–200 chars, may be empty).
  • Exclusivity rules: blocked ⇔ block, and failed ⇔ failure.
  • contract_version: stays 2 on delivery receipts, as #1147 writes it.
• Every class #1147 emits gets a label or a sensible fallback: error, refused (BUILD_REFUSED), aborted (RUNG_ABORTED), unrecorded_gate_block, out_of_memory, interrupted, terminated, build_failure, unexpected_process_exit. Error codes all match ^[A-Z0-9_]+$.
• The collector matches the emitter. The emitter's block() sends event_type: run, stage_id: blocked, status: blocked with phase, count, gate ids and gate statuses. That closes the run as blocked with ended_at. A restart clears the block, and fail()'s error_code now reaches failure. status is Text with no check constraint, so no migration is needed.
• The dashboard tells the outcomes apart. Blocked runs get their own state, a warning tone and a pill. They're final for caching on both the staging and collector paths. Refused and aborted runs are failed runs with their own class labels. v2 still infers a refusal from gate counts only when the run declares v2.
• Tests: frontend 581 pass and tsc is clean. Collector 71 pass, 8 skipped (Postgres). With the eight source files reverted to main and the new tests kept, 15 frontend tests and 5 collector tests fail. CI is green.

Should

1. Collector documents say version 2 but carry version 3 semantics. run_documents writes "schema_version": 2 (storage.py:273, :294, :333) on documents that can now hold status: blocked, a block, and failures with error_code / failure_class.
   • It works today, because the collector path skips staging-contract.ts.
   • But anything that validates by version would reject these as v2, and by v2's own rules a refusal closes as completed.
   • Fix: either write 3 when the run carries a block or a classed failure, or add a line to docs/build-monitor.md saying collector documents are their own shape and aren't validated against the contract.

Nits

2. The collector stores a block even when the event details are empty. A run/blocked event with no phase or count still gets stored as a block of nulls (storage.py:172), which the HF-side contract would refuse. Rejecting it in TelemetryEvent with a 422 would keep the two sides consistent. #1147 always sends both fields, so this only guards future producers.
3. A 404 now makes the run local-only for good. #1147's client treats every 4xx except 408 and 429 as permanent, and the events route answers 404 "Run is not registered" (app.py:421). An event that reaches the collector before its registration commits would therefore turn the run local-only permanently. The same note is on #1147; a sentence in the collector's README covering the 404 and 409 semantics would help whoever operates it.
4. The fixture copy will go stale silently. It's pinned by hand to #1147's head. If #1147's fixtures change before it merges, nothing here fails. A short note in the PR checklist, or a test comparing the copy's SHA256SUMS digest with microcosm's, would catch it.
5. One label has no producer. dry_run_refusal has a label but nothing in #1147 emits it. That's harmless if it's meant for a future producer.

[juaristi22]

Thanks. Addressed in d36ee7e (nits) and d229e02 (point 1).

1. Collector documents say version 2. We took the documentation route. docs/build-monitor.md now says the collector's documents declare schema_version: 2 so the monitor reads them with the version 2 lifecycle. It also says they are the collector's own shape: they can carry version 3's blocked status, block and failure classes, and they are not validated against staging-contract.ts. That contract applies only to staged run documents, from the Hugging Face staging repository or a local run folder. Writing 3 would have claimed a contract these documents don't meet in other ways (no delivery contract shape, no sample, extra fields).

2. Block of nulls. TelemetryEvent now rejects a blocked event unless its details carry a valid block: a safe-identifier phase, a blocking_failure_count of at least 1 (not a boolean), and blocking_gate_ids as a list of 1–200-character strings, which may be empty. Anything else gets a 422, so the collector never stores a block of nulls. gate_statuses stays optional. New tests cover eight malformed blocks (each 422, run still running) and a block that names no gate and carries no statuses (202).

3. 404 and permanent 4xx. The collector README has a new section, "Responses a producer acts on". It lists what #1147's client treats as final (403, 404, 409, 413, 422) and what it retries (401 re-exchanges the token; 408, 429, 5xx and the maintenance 503 are retried). On the 404: the delivery service posts events only after its registration has returned 201, and registration commits before it answers. So an events 404 can't come from a race. It means the run disappeared after it registered, for example a database reset while a build was running.

4. Stale fixture copy. The PR body now has a "Before merging" check: the digest of #1147's v3/SHA256SUMS must still be e31a5e33…bf3def5. #1147 has since moved to b82df99ac, a merge of main, and its fixtures are unchanged there.

5. dry_run_refusal. It has a producer on microcosm main: the US fiscal-refresh release builder emits it when a gates dry run is refused before its stop point (tools/build_us_fiscal_refresh_release.py). It just isn't part of #1147. The label now carries a comment naming that producer.

Frontend 581 pass and tsc is clean. Collector 80 passed, 8 skipped (Postgres, CI only).