Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
6b1fcf4
Add a versioned contract for household calculations on Modal
anth-volk Oct 21, 2026
03cda9b
Record each country's capabilities in v2 manifest schema version 2
anth-volk Oct 22, 2026
0485255
Port the Public API's household input preparation
anth-volk Oct 21, 2026
7aeb914
Resolve household SPM selections and receipts like the Public API
anth-volk Oct 21, 2026
bf9d779
Register observability stages for the v2 household functions
anth-volk Oct 22, 2026
d52d956
Calculate a household under several policies as the Public API does
anth-volk Oct 22, 2026
389c6cc
Add calculate_household_us and calculate_household_uk to the v2 Modal…
anth-volk Oct 22, 2026
60d778e
Validate the household functions at release and record them in the ma…
anth-volk Oct 22, 2026
00b3409
Build every v2 image in the pre-merge image check
anth-volk Oct 22, 2026
9063f10
Word uncertifiable SPM receipts exactly as the Public API does
anth-volk Oct 22, 2026
d9462a3
Check a household's SPM settings exactly as sent, with the Public API…
anth-volk Oct 22, 2026
db61091
Load the household SPM forecast without the main path's SPM module
anth-volk Oct 22, 2026
85afc4e
List every known difference from the Public API in one place
anth-volk Oct 22, 2026
5c700a1
Test publishing after a publisher released before schema version 2
anth-volk Oct 22, 2026
fd45e26
Resolve a comparison report's simulation callable as a typed text
anth-volk Oct 22, 2026
615955f
Test a chosen US SPM selection and the validation household with real…
anth-volk Oct 22, 2026
7455b24
Measure the whole country model load in its own stage
anth-volk Oct 22, 2026
b760140
Keep the shared SPM receipt model's wording and translate it for hous…
anth-volk Oct 22, 2026
08f1301
Test the manifest with released document shapes and document rollback…
anth-volk Oct 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .github/scripts/modal-image-smoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@
# Usage: ./modal-image-smoke.sh <modal-environment>
#
# Each smoke imports the true app entrypoints inside the real (or, for
# the executor, prefix-identical) Modal image — catching in-image
# dependency breakage (issue #602's class) before merge.
# the executor and the v2 US worker, prefix-identical) Modal image —
# catching in-image dependency breakage (issue #602's class) before merge.
# The v2 image smoke builds every image of the v2 application: the report
# coordinator, the country simulation workers and the household functions.

set -euo pipefail

Expand Down Expand Up @@ -47,4 +49,10 @@ echo "=== Stage 12 static runtime file smoke (env: $MODAL_ENV) ==="
src/modal/v2_static_runtime_files_smoke.py
)

echo "=== v2 image smoke: every v2 image and its entry points (env: $MODAL_ENV) ==="
(
cd "$REPO_ROOT/projects/policyengine-simulation-executor"
uv run --frozen modal run --env="$MODAL_ENV" src/modal/v2_image_smoke.py
)

echo "=== Image smokes passed ==="
7 changes: 4 additions & 3 deletions .github/workflows/pr-image-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,10 @@ jobs:
# Same-repository pull requests only, and in a stack only the lowest
# unmerged one (see docs/engineering/skills/github-prs.md).
if: ${{ github.event.pull_request.head.repo.full_name == github.repository && (!github.event.pull_request.stack || github.event.pull_request.stack.base.ref == github.event.pull_request.base.ref) }}
# Warm cache: ~2 min. After a relock the executor smoke pays the
# certified dataset download layer (~15-20 min).
timeout-minutes: 45
# Warm cache: a few minutes. After a relock the executor smoke and the
# v2 image smoke each pay their certified dataset download layers
# (~15-20 min each; Modal builds the v2 smoke's images together).
timeout-minutes: 90

steps:
- uses: actions/checkout@v6
Expand Down
92 changes: 77 additions & 15 deletions docs/migration/stage-12-modal-worker-foundation.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,29 +396,54 @@ and coordinator images do not contain these files. Deployment validation runs
one real 2026 region-group baseline and requires it to load the precomputed
file without recalculation.

Readers released before cache releases reject fields they do not know, and
Manifest schema version 2 records, for each country a worker application
serves, whether it is economy-capable (it has a single-simulation worker, so
its reports may be given to the report coordinator) and which function
calculates its households. The Simulation Entrypoint's comparison adapter
skips any country the manifest does not mark as economy-capable, with the
reason `unsupported_country`, so the coordinator receives only those
countries' reports.

Readers released before a schema change reject fields they do not know, and
the manifest is published before the Cloud Run revision that reads it is
promoted. The publisher therefore writes the whole manifest under the
`active-with-region-datasets` key, the same manifest without each country
bundle's `region_datasets` under the `active-with-cache` key, and without cache
releases either under the `active` key; readers released before each field keep
reading the key that lacks it. Current readers use the
`active-with-region-datasets` copy only while it, with its region datasets
removed, equals the manifest under `active-with-cache`; otherwise they read
`active-with-cache`, falling back to `active` before the first publish that
writes it. Code released before region datasets republishes only the two older
keys, so after a rollback the out-of-date region dataset copy stops matching and
is no longer read. Removing the older copies needs a later change, once no
deployed or rollback revision reads them.
promoted. The publisher therefore writes the whole schema version 2 manifest
under the `active-schema-2` key and, in schema version 1 form (without the
capability and household fields), the same manifest under the
`active-with-region-datasets` key, without each country bundle's
`region_datasets` under the `active-with-cache` key, and without cache releases
either under the `active` key; readers released before each field keep reading
the key that lacks it. A schema version 1 document marks every listed country
economy-capable, with no household function.

Current readers use the `active-schema-2` document while each older key holds
its schema version 1 form. Schema version 1 cannot describe a country that is
not economy-capable, so a manifest holding one is written only under
`active-schema-2` and the older keys keep the last manifest written there; such
a document is used unless the document under the older keys was validated
later. Otherwise readers use the `active-with-region-datasets` copy only while
it, with its region datasets removed, equals the manifest under
`active-with-cache`; otherwise they read `active-with-cache`, falling back to
`active` before the first publish that writes it. Code released before schema
version 2 or before region datasets republishes only the older keys it knows, so
after a rollback the out-of-date copies stop matching and are no longer read,
and the next publish starts from what the older code wrote. That document has
no household functions, so the manifest names none for versions published
before the rollback until each is published again, and it lacks whatever only
`active-schema-2` held, such as a version with a household-only country. Code
released before cache releases writes only `active`, which current readers do
not read while `active-with-cache` exists; a rollback that far needs the newer
keys deleted from the Modal Dict first. Removing the older copies needs a later
change, once no deployed or rollback revision reads them.

A rollback to code released before region datasets has one window: the older
worker is deployed before its publish job runs, and until that publish the
region dataset copy still matches, so updated entry services keep sending
regional provenance for state and district runs, which the older worker refuses.
Only Stage 12 comparison runs fail in that window, never production responses;
if the older deploy's validation fails, the window lasts until a later publish.
To avoid it, delete the `active-with-region-datasets` key from the
`simulation-api-v2-version-manifest` Modal Dict before deploying the older code.
To avoid it, delete the `active-schema-2` and `active-with-region-datasets` keys
from the `simulation-api-v2-version-manifest` Modal Dict before deploying the
older code.

`region_datasets` names the dataset each region type loads. The entry service
records that dataset as a state or district run's dataset provenance, and the
Expand All @@ -434,6 +459,43 @@ not satisfy the requested output plan is recalculated. Each region-group log
records whether it loaded a complete file, recalculated an incomplete file, or
computed because no file existed.

## v2 household functions

The versioned v2 application also defines a household function for each
country, `calculate_household_us` and `calculate_household_uk`. Each call
calculates one household, in the Public API's situation shape and with
optional axes, under one to ten policies, so a baseline and its reform are
calculated in one call. It takes a `HouseholdExecutionRequest` and returns a
`HouseholdExecutionResult` or a typed `HouseholdExecutionFailure` from
`policyengine_simulation_contract.household_execution`, as JSON. The
calculation follows the Public API's household calculation, including its
input errors, warnings and SPM receipt. A function refuses a request for
another country, and returns a `bundle_mismatch` failure for a request that
names another bundle. Nothing calls these functions until the Simulation
Entrypoint has household routes.

- Image: the CA store, the frozen `modal-simulation-image` dependency group
and the source packages, without the bundle's datasets or the Stage 12
cache. Building it needs no secret.
- Secrets: none. A household calculation reads no private data, Cloud
Storage or database. Telemetry authenticates with the identity token Modal
gives every container (`MODAL_IDENTITY_TOKEN`) and the workload identity
settings baked into the image at deployment.
- Limits: 2 CPUs, 4,096 MiB of reserved and 16,384 MiB of maximum memory, a
900-second timeout, no retries, and a Modal `max_containers` value of 20.
- Each container loads a country model the first time it calculates that
country and keeps it for later calls.

Deployment validation calls each household function with a one-adult,
current-law household and requires a successful calculation with the deployed
bundle, reported by that function, with a number for the household's net
income. The validation record keeps each invocation ID, and the manifest
publisher records a country's `household_callable` only when the record shows
its function was validated. A failed household validation fails v2 worker
validation like any other validation failure: the manifest is not published,
and, as described under "Release and deployment constraints", the deployment
does not promote the new Simulation Entrypoint revision.

## Temporary direct runner endpoint

> **Temporary Stage 12 interface:** The authenticated routes in this section
Expand Down
Loading
Loading