Lanes: packages under ledger-source-ingestor; any profile extension under ledger-target-profile-author
Depends on: conventions from #132 / #133 (GSS geography IDs, curated-extract lane, Stat-Xplore api_response pattern)
Context
policyengine_ledger/target_profiles/uk_local_geography.json declares 25 targets across five families (8 ons_population age bands, 5 dwp_universal_credit, 5 ons_housing, 4 hmrc income, 3 ons_income) at constituency + local_authority levels — with zero backing source packages. Of Ledger's 70 committed packages, all 4 UK ones are national firm/VAT. Meanwhile policyengine-uk-data feeds its local calibration from per-area CSVs under datasets/local_areas/{constituencies,local_authorities}/targets/ (built by targets/sources/local_age.py, local_income.py, local_uc.py, local_la_extras.py — these emit no registry Target()s; they are loss-matrix inputs).
Sub-national precedent exists: irs_soi/historic_table_2_state_*, census/pep_2024_state_age_sex, census/acs_s0101_state_2024 (state level), and per-CD Census API extractions.
Packages
| backing |
package(s) |
source mechanics |
ons_population age bands |
ons/census21_age_by_pcon24, ons/census21_age_by_lad |
Census 2021 / mid-year estimates by PCON24 and LAD; uk-data's local_age.py reads two per-area CSVs |
hmrc employment / self-employment income |
hmrc/spi_by_pcon24, hmrc/spi_by_lad |
SPI geography tables (collated 3.12–3.15a) — amounts and taxpayer counts |
earnings (profile hmrc count/amount complements) |
nomis/ashe_earnings_by_pcon24 |
NOMIS ASHE workbook uk-data commits (nomis_earning_jobs_data.xlsx) |
dwp_universal_credit households (+ by children) |
dwp/uc_households_by_pcon24, dwp/uc_households_by_lad |
Stat-Xplore extracts — archive the API response as the artifact per the existing api_response pattern |
ons_income equivalised income BHC/AHC + housing costs |
ons/small_area_income_msoa |
ONS small-area income estimates (MSOA), plus the aggregation lookup the profile's levels need — only source-published aggregation levels enter Ledger; any MSOA→PCON re-aggregation PolicyEngine performs is downstream build work, not a fact |
ons_housing tenure + private rent |
ons/tenure_by_lad, ons/private_rents_by_lad |
uk-data local_la_extras.py sources (PRMS rents, tenure) |
| LA council tax stock |
voa/council_tax_stock_by_lad |
wave-2 VOA publication, LA cut |
Geography contract: GSS codes as geography_id (E14… PCON24, E06/E07/E08/E09… + devolved LAD codes), explicit geography_vintage (pcon_2024, lad_2023-style), geography_level: constituency / local_authority matching the profile's levels.
Row-count mechanics (verify on the first package)
650 PCON24 × band and 360 LAD × band tables must not become hand-enumerated per-row YAML (the VAT worked example enumerates 9 rows manually). Multiple committed extraction methods parse full tables ("delimited text full-row parse with selected source-row cells", "zip archive CSV member full-row parse", Census API unpivot) — confirm the row-spec form that pins geometry for full-table parses on the first constituency package, and document it in docs/agent-source-package-harness.md if the answer isn't already written down.
Profile work
uk_local_geography already declares selectors and bindings — the deliverable is that its selectors resolve against these new facts. Any selector/binding corrections discovered while backing it are a separate small PR in the profile lane (no values, sum-only).
Acceptance
- Every declared
uk_local_geography target resolves from a consumer artifact at both declared geography levels, every fact carrying full lineage (manifest sha256 + R2 + cell guards).
- Coverage parity with uk-data's local CSV inputs: each per-area family uk-data feeds its constituency/LA loss matrices either has a Ledger package or an enumerated, signed exclusion in this issue — no silent drops.
- Deterministic checks + judge lanes green.
Out of scope
- Populace's local solve (already Ledger-profile-aware via
populace.build.uk.local_targets, populace#158). uk-data's local calibration remains as is until populace consumption switches over.
Sizing
~8–9 packages ≈ 3 PRs (ONS batch, HMRC/NOMIS batch, DWP/VOA batch) + possibly 1 profile-correction PR.
Lanes: packages under
ledger-source-ingestor; any profile extension underledger-target-profile-authorDepends on: conventions from #132 / #133 (GSS geography IDs, curated-extract lane, Stat-Xplore
api_responsepattern)Context
policyengine_ledger/target_profiles/uk_local_geography.jsondeclares 25 targets across five families (8ons_populationage bands, 5dwp_universal_credit, 5ons_housing, 4hmrcincome, 3ons_income) atconstituency+local_authoritylevels — with zero backing source packages. Of Ledger's 70 committed packages, all 4 UK ones are national firm/VAT. Meanwhilepolicyengine-uk-datafeeds its local calibration from per-area CSVs underdatasets/local_areas/{constituencies,local_authorities}/targets/(built bytargets/sources/local_age.py,local_income.py,local_uc.py,local_la_extras.py— these emit no registryTarget()s; they are loss-matrix inputs).Sub-national precedent exists:
irs_soi/historic_table_2_state_*,census/pep_2024_state_age_sex,census/acs_s0101_state_2024(state level), and per-CD Census API extractions.Packages
ons_populationage bandsons/census21_age_by_pcon24,ons/census21_age_by_ladlocal_age.pyreads two per-area CSVshmrcemployment / self-employment incomehmrc/spi_by_pcon24,hmrc/spi_by_ladhmrccount/amount complements)nomis/ashe_earnings_by_pcon24nomis_earning_jobs_data.xlsx)dwp_universal_credithouseholds (+ by children)dwp/uc_households_by_pcon24,dwp/uc_households_by_ladapi_responsepatternons_incomeequivalised income BHC/AHC + housing costsons/small_area_income_msoaons_housingtenure + private rentons/tenure_by_lad,ons/private_rents_by_ladlocal_la_extras.pysources (PRMS rents, tenure)voa/council_tax_stock_by_ladGeography contract: GSS codes as
geography_id(E14…PCON24,E06/E07/E08/E09…+ devolved LAD codes), explicitgeography_vintage(pcon_2024,lad_2023-style),geography_level: constituency/local_authoritymatching the profile's levels.Row-count mechanics (verify on the first package)
650 PCON24 × band and 360 LAD × band tables must not become hand-enumerated per-row YAML (the VAT worked example enumerates 9 rows manually). Multiple committed extraction methods parse full tables ("delimited text full-row parse with selected source-row cells", "zip archive CSV member full-row parse", Census API unpivot) — confirm the row-spec form that pins geometry for full-table parses on the first constituency package, and document it in
docs/agent-source-package-harness.mdif the answer isn't already written down.Profile work
uk_local_geographyalready declares selectors and bindings — the deliverable is that its selectors resolve against these new facts. Any selector/binding corrections discovered while backing it are a separate small PR in the profile lane (no values, sum-only).Acceptance
uk_local_geographytarget resolves from a consumer artifact at both declared geography levels, every fact carrying full lineage (manifest sha256 + R2 + cell guards).Out of scope
populace.build.uk.local_targets, populace#158). uk-data's local calibration remains as is until populace consumption switches over.Sizing
~8–9 packages ≈ 3 PRs (ONS batch, HMRC/NOMIS batch, DWP/VOA batch) + possibly 1 profile-correction PR.