Repository navigation
Refresh the data to 2026Q2 and regenerate the committed artefacts - #24
vahid-ahmadi wants to merge 5 commits into
Conversation
…20 Oct) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the 2026Q2 data Run on GitHub Actions (throwaway branch tmp/data-2026q2-regen, run 42154335977) with each script's committed defaults. Pins in test_committed_artifacts move with the new origin (49 -> 50) and the ONS revisions; no tolerance changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s_boe_obr.md Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run_replication.py --draws 10000 on Actions run 42154335977: 744 -> 765 accepted, ESS 356.1 -> 368.0. Sample unchanged (1992Q1-2025Q1); the moves come from the Blue Book 2026 revisions to UK GDP history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
….py (01/Jan/2027) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Independent verification of #24 (checked at 422bd5b; CI green)Result: the refresh is correct. I found one stale doc URL and fixed it in d4ec3e7. Merging #24 changes the published replication figures (details below). 1. Scope. The only code change is two literals in 2. The 2026Q2 values, against my own fetches from the ONS generator and BoE IADB:
The energy shock is visible in the raw data: CPI motor fuels (D7EC) rose 17.8 log points in Q2, household energy (D7CH) fell 5.3, and Brent went from $77.8 to $97.0. Note: Bank Rate rose to 4.00 on 5 Nov 2026. That is after the data edge. The model's median for 2027Q3 is 3.92. 3. Replication 744 → 765 is expected. The 1992Q1–2025Q1 window is unchanged, but the data inside it were revised:
With the same seed, the posterior differs and so does the set of accepted draws. The figures the macro site quotes that move:
The qualitative readings hold: a match on GDP, roughly 8.5pp short on CPI, and no skill against drift. Lines in macro
Lines that only go stale once Modal is redeployed (they say "data through 2026Q1"):
Recommendation: the paper attributes these numbers to "the repository's production artifact". Either pin the paper to the pre-#24 sha or tag, or update every line above in one macro PR when this merges. Pre-existing, not caused by #24: 4. The estimation sample still ends in 2025Q1, as intended. The One judgement call: the period held back from estimation is now five quarters (2025Q2–2026Q2), while Outturn check. CPI for 2026Q3 is already published at 3.1% (D7G7, 21 Oct). That is inside the new one-quarter-ahead 68% band of 2.51–3.43 (median 3.00). Bank Rate for 2026Q3 was 3.75, against a median of 3.85. 🤖 Generated with Claude Code |
|
Merging this PR leaves the PolicyEngine Macro site's BoE SVAR replication figures stale: 744 accepted draws, ESS 356.1, FEVD 37.4 / 42.3, 49 origins, smallest q 0.36. I've prepared both options as draft PRs:
One item from (b) belongs in this PR: |
Merge prerequisite: #18 (base
vahid/yoy-simple-rates). Merge #18 first.Why
No new PolicyEngine Macro forecast round is possible while the hosted SVAR's data end in 2026Q1: the next round has to condition on 2026Q2, and the round machinery refuses a repeated data edge (PolicyEngine/macro#224). ONS has now published 2026Q2. This PR refreshes the packaged dataset to 2026Q2 from the same official sources
scripts/download_data.pyalready uses. It then regenerates every committed artefact.Stacked on #18 (base
vahid/yoy-simple-rates); merge #18 first. On main, the staleness hash (evaluation_code_version) covers code only, so a data-only refresh would leave the gate green without proving that anything was regenerated. #18 adds the dataset to the hash. That is what makes "the staleness gate passes" below a real check: every evaluation JSON here was regenerated on the new CSV, or the gate would fail.No methodology change. Estimation still ends at 1992Q1-2025Q1, with the same lags, priors, seeds, draw counts and Covid dummies. Only the conditioning edge (forecast origin T) moves, from 2026Q1 to 2026Q2. The rolling and coverage evaluations gain one origin (49 -> 50) because their last origin is T - 9, which is what those scripts do by design.
Data (vintage table now in
data/README.md)download_data.py:1990Q1:2026Q2; the IADBDatetomoves to 1 Jan 2027).data/boe_var_data.csvandsrc/boe_var/_data/boe_var_data.csvare byte-identical, at 138 rows, 1992Q1-2026Q2.uk_unemployment_mgsx.csvgains 2026Q2 = 4.9%. The 1971Q1-2026Q1 values are unchanged in the 20 Oct LMS release.uk_gdp: the largest change to a quarterly growth rate is 0.18 log points, and the level is 2.8-4.3 log points higher. FRED revisions movedworld_gdpby up to 0.14 log points in growth. Every other column moves by at most 0.003. So even with the coefficient window unchanged, the coefficients move slightly.Regeneration (GitHub Actions, not the laptop)
Each script ran with its committed defaults on a throwaway branch,
tmp/data-2026q2-regen(run 42154335977). That branch has been deleted. The workflow file is not in this PR, and it is not part of any hash.results/rolling_evaluation.jsonrun_rolling_evaluation.pyresults/coverage_evaluation.jsonrun_coverage_evaluation.pyresults/unemployment_satellite_validation.jsonrun_satellite_validation.pyresults/forecast_summary.md, fig1, fig7-9run_forecast_revision.py --draws 6000results/summary.md, fig2-6run_replication.py --draws 10000results/forecast_vs_boe_obr.md(new top section)run_external_comparison.pyReplication (
summary.md): same 1992Q1-2025Q1 sample, revised historyForecast, old (T = 2026Q1) -> new (T = 2026Q2), production sample
CPI inflation, four-quarter %, simple rate:
GDP growth, four-quarter %:
2026Q3 is now one quarter ahead, where before it was two, so its band is narrower by construction. The direction of every change follows from the 2026Q2 energy quarter now being in the conditioning data: the identified shock at T is a world energy shock, with posterior weight 0.98 on the inflationary sign (
forecast_summary.md). Near-term CPI and the energy contribution rise; GDP from 2027 is 0.2-0.3pp weaker. The composite-IRF peak CPI effect is +0.22 -> +0.66pp at h3, and the GDP effect +0.27pp at h0 -> -0.36pp at h6. The Bank comparison columns inforecast_vs_boe_obr.md(July MPR, March EFO) are unchanged. Parts 1-3 of that file are the 22 Sep analysis, and the new top section says which of its numbers are superseded.Published numbers that move (pins in
tests/test_committed_artifacts.py, no tolerance changed)Checks
tests/test_committed_artifacts.py: 31 passed locally (under 1 s), includingtest_*_is_not_stale(the Compare model YoY rates with BoE/OBR on the simple-rate basis; hash the data #18 data hash). CI on head 422bd5b: fast tests (hard gate, includes the staleness and artefact tests) and ruff pass (run 42155198987). The nightly slow job is skipped on PRs.What Vahid has to do for the macro round machinery to use this
S.Sin one commit:integration/modal_app.py:133BOE_REVISION = "S"integration/pyproject.toml:53boe_var @ git+https://github.com/PolicyEngine/boe-var-model@Smodal deploy integration/modal_app.py) so the hosted image rebuilds atS. Do this before the next due day (12 Nov per #224).forecasts/nowcast.py --edge-gatereadsBOE_REVISION, fetchesdata/boe_var_data.csvat that sha and takes its last quarter (2026Q2). The required quarter is the one after the newest archived edge (2026Q1 -> 2026Q2), so the gate passes by itself and the round runs.data_edgein the round comes from the model'sforecast_origin, so it will read 2026Q2.integration/src/policyengine_macro/capabilities.py:89-90(data_edge,data_vintage),core.py:4362(data_vintageof the committed-artefact path),core.py:1366(comment), and the site copy insvar/index.html,svar/methodology/index.html,svar/validation/index.htmlandeconomy/topics/{growth,inflation}/index.html("conditioning data through 2026Q1"). The hosted 2,000-draw accepted/ESS figures quoted onsvar/methodologywill also change after the redeploy.Not done here: no forecast round created, nothing in macro edited, nothing deployed.
🤖 Generated with Claude Code