Make estimate-vs-projection resolution explicit: assertion_policy on target profiles

[juaristi22]

Grew out of the #137 review question: NPP 2024 projections and MYE estimates sit side by side in the bundle under calendar_year periods — is the boundary carried by provenance_class, or left to the consumer profile to know?

The boundary's typed home is assertion (observation vs source_projection), which every fact already carries through to consumer rows — provenance_class answers how the publisher measured (an NBS balance-sheet estimate is model_output + observation; an NPP projection year is model_output + source_projection). The gap was that profile resolution ignored the axis: under latest_not_after_build_base_period, a selector matching both estimates and projections would resolve a projection without anyone noticing.

Lane: ledger-target-profile-author — judges ledger-target-profile + ledger-boundary.

What

Profiles (and individual targets) now declare an assertion_policy:

policy	behavior
observed_only (default)	Projections invisible. A projection-only family fails loudly (only_projection_facts); an older observation at a later requested period still demands an explicit period alignment — nothing resolves silently.
prefer_observed	Per series (one geography/entity/dimension tuple): a series with any observed fact resolves only from observations; a series with none falls back to projections. No single series ever mixes bases across periods, and a projection-only series is never starved by a neighbouring series' observation.
allow_source_projection	Both compete under the period policy — for forecast families (OBR EFO receipts/expenditure, NPP forward years). An observation and a projection colliding within one series at the chosen period resolve to the observation with an ambiguous_assertion_at_period warning naming the series, never both.

Two escape valves keep intent explicit rather than clever: a target whose chronicle_selector names assertion bypasses the policy (the selector is already maximal intent — the existing cbo.receipts pattern keeps working unchanged), and every projection resolution emits a resolved_from_projection warning while ResolvedTarget exposes the winning fact's assertion — so even opted-in consumers see the basis they got.

Existing profiles need no changes (uk_local_geography, uk_firms carry no projection facts today); the future uk_national profile declares allow_source_projection on its OBR lines and inherits observed_only everywhere else.

Gates

102 profile/consumer tests pass (7 new: the three policies, the per-target override, the selector bypass, loud failure on projection-only families, and vocabulary rejection); ruff clean. Harness doc gains the two-axes section so package authors and profile authors read the same contract.

🤖 Generated with Claude Code

[vahid-ahmadi]

Read the whole resolution path against the harness doc. The two-axes framing is right and the escape valves (explicit assertion selector bypasses the policy; every projection resolution emits resolved_from_projection and ResolvedTarget exposes the winning assertion) are the parts that make this safe to build on. Three things before merge — one is a real semantics/doc gap, two are smaller.

1. prefer_observed is family-level, not period-level

Both the harness doc and the PR table describe it as "projections fill only periods with no observation". The implementation filters the whole candidate set before _choose_period:

elif assertion_policy == "prefer_observed" and observed:
    candidates = observed

So a projection is only ever reachable when the family has zero observed facts anywhere. Concretely, with MYE observed 2023 and an NPP projection at 2024, requesting 2024:

• doc semantics → 2024 has no observation, so the projection resolves with a resolved_from_projection warning
• actual → every projection is dropped, _choose_period falls back to 2023, and you get a period-contract violation

test_prefer_observed_takes_the_observation_over_a_newer_projection locks in the second behaviour, so this is a deliberate implementation with a doc that describes a different policy — which makes prefer_observed ≈ observed_only plus a whole-family fallback. That's a defensible policy, but it isn't the one a profile author will read for. Either restate it in the doc/table ("projections are used only when the family has no observed facts at all") or group by period before filtering. My preference is fixing the doc — the family-level rule is easier to reason about and never silently mixes bases within one series.

2. Nothing flags an observation and a projection colliding at the same period

The emit loop appends every candidate at chosen_period:

for row in candidates:
    if dict(row["period"]) != chosen_period:
        continue

Under allow_source_projection that means a series carrying both an observation and a publisher projection at the same period resolves twice, and a consumer summing resolved rows double-counts. The only thing standing between us and that is the profile author pinning dimensions or record_set_id tightly enough — _selector_matches does exact-set matching on dimensions, so it usually saves us, but nothing in the resolver notices when it doesn't.

This stops being hypothetical the moment #137 lands: mid-2024 MYE (observation) and NPP 2024 (source_projection) are both UK / person / calendar_year:2024, distinguished only by source_table and the extra sex dimension. Worth either preferring observation on an exact-period tie under allow_source_projection, or emitting an ambiguous_assertion_at_period issue so it can't pass unnoticed.

3. Nit: use DEFAULT_ASSERTION instead of the literal

consumer.py already imports DEFAULT_ASSERTION from chronicle.core, and assertion is in the required list of consumer_fact.v1.schema.json — line 445 reflects that with a bare row["assertion"]. The two new reads default it instead:

if row.get("assertion", "observation") == "observation"
...
if row.get("assertion", "observation") == "source_projection"

Same for ASSERTION_POLICIES in model.py, which restates the vocabulary that ALLOWED_ASSERTIONS already owns. Cosmetic, but the defaulting reads as "this field is optional" next to code that knows it isn't.

---

Tests are good — the per-target override, the selector bypass and the vocabulary rejection are exactly the cases I'd have asked for. CI green. Happy for this to go in once (1) is settled one way or the other; (2) can be a follow-up if you'd rather not widen the PR, but let's not land the uk_national OBR profile before it exists.

[juaristi22]

Thanks @vahid-ahmadi — all three addressed in 782ab24 (also rebased onto current main).

1. prefer_observed semantics — took your preference: the implementation stands, the docs now tell the truth. The harness doc, the policy comment (now in chronicle/core.py), and the PR table all state the family-level rule: a family with any observed fact resolves only from observations; projections are reachable only when the family has none — so one series never mixes bases across periods. Your MYE-2023/NPP-2024 example resolving to a period-contract violation is now the documented behaviour, not a surprise.

2. Same-period collision — did both halves of your either/or, since they compose: under allow_source_projection, an observation and a projection colliding at the chosen period resolve to the observation only (no double-count) and the report carries an ambiguous_assertion_at_period warning naming the period and pointing at assertion/dimensions/record_set_id as the explicit outs. The explicit-assertion-selector bypass is untouched — a selector pinning assertion: source_projection still reaches the projection with no ambiguity (the selector filtered the tie away). Two new tests cover exactly the MYE-observation-vs-NPP-projection shape: test_allow_source_projection_resolves_the_observation_on_a_period_tie and test_explicit_assertion_selector_reaches_the_projection_despite_a_tie. So the resolver is safe for #137's facts the day it lands, and the uk_national profile doesn't have to wait on a follow-up.

3. Vocabulary — the two new reads are bare row["assertion"] (it's schema-required, as line 445 already knew), and ASSERTION_POLICIES/DEFAULT_ASSERTION_POLICY moved to chronicle.core beside ALLOWED_ASSERTIONS/DEFAULT_ASSERTION — one home for the assertion axis and its resolution vocabulary; the profile model imports it.

Full suite: 624 passed, 1 skipped.

[vahid-ahmadi]

This is a good axis to make explicit. Keeping assertion orthogonal to provenance_class is the right call — "how the publisher measured" and "had the period happened yet" genuinely are independent, and collapsing them would have been the easy wrong move. observed_only as the default, failing loudly with only_projection_facts, is the correct bias. The docs paragraph in agent-source-package-harness.md explains it well and the test matrix covers each policy plus the override.

Three things.

1. Nothing validates the assertion value itself, so a typo silently drops facts.

Every read is a defaulted .get:

row.get("assertion", "observation") == "observation"

A fact carrying assertion: projection (or Source_projection, or forecast) is therefore not an observation — so it's filtered out under observed_only — and not equal to "source_projection", so the resolved_from_projection warning never fires for it either. It just quietly vanishes from the candidate set. Given the stated goal is that "downstream builds never discover the estimate/projection boundary by accident", that's the one hole left.

Suggest validating the enum at fact load (alongside provenance_class, which per the docs already fails load on unknown values), so the two axes are policed the same way.

2. An explicit assertion selector silently overrides an explicit assertion_policy.

if "assertion" in target.chronicle_selector:
    assertion_policy = "allow_source_projection"

If an author writes both a selector on assertion and assertion_policy: observed_only on the same target, the declared policy is discarded with no signal. The comment argues the selector is maximal intent, which I buy as a default — but the case where both are set is a contradiction the author would want to hear about, not have resolved for them. A ValueError at profile load (or at minimum a warning issue) would fit the "resolve explicitly rather than by convention" framing of the rest of the PR.

3. Minor

• ResolutionIssue(code="only_projection_facts", ...) relies on the default severity while the sibling resolved_from_projection passes severity="warning" explicitly. The test asserts issue.severity == "error", so it's correct — but passing it explicitly on both would make the asymmetry deliberate rather than incidental.
• projection_resolved emits one warning per target even when several rows resolve. That reads like the intent; worth a one-line comment saying so, since the flag is set inside the row loop.
• Defaulting a missing assertion to "observation" is the right back-compat choice for existing facts. Might be worth stating that in the docs paragraph explicitly — it's load-bearing for every pre-existing package.

[juaristi22]

Thanks for the second pass @vahid-ahmadi — all three addressed in 341ecd7. One timing note first: this review raced 782ab24 (the response to your first pass), which had already replaced the defaulted .get reads with bare row["assertion"] — but the substantive hole you found survived that change in a different form, so:

1. Typo'd assertions now fail loudly on every path. You're right that the axis was only policed at file load (_load_consumer_rows already defaulted + rejected unknown values, mirroring provenance_class). Rows handed to resolve_profile_targets directly — tests, future in-memory pipelines — skipped that. The resolver now normalizes at entry with the same two moves: missing assertion defaults to observation (back-compat parity), unknown values raise naming the row and the allowed vocabulary. assertion: projection can no longer silently vanish from every policy's candidate set. Tests: test_unknown_assertion_values_fail_resolution_loudly, test_missing_assertion_defaults_to_observation_at_resolve.

2. Selector + per-target policy is now a load-time ValueError. Took your stronger option: a target declaring both assertion_policy and a chronicle_selector on assertion is a contradiction the author hears about at profile load, not a silent resolution in the selector's favour. The consumer bypass therefore only ever fires against an inherited default, which is the case where "maximal intent" is actually true. Test: test_assertion_selector_and_target_policy_together_are_rejected.

3. Minors, all taken: only_projection_facts passes severity="error" explicitly (the asymmetry is now deliberate); the one-warning-per-target flag carries a comment saying exactly that; and the docs paragraph now states the back-compat rule — a fact predating the axis loads as observation, and both the loader and the resolver default-and-validate the field the same way.

Full suite: 627 passed, 1 skipped.

[vahid-ahmadi]

Thanks — 341ecd7 closes both of my points properly, and you took the stronger option on each. Confirming what I checked:

• _normalize_assertion_rows copies each row (dict(row)), defaults a missing assertion, and raises naming the row index and the allowed vocabulary. assertion: projection can no longer vanish. The back-compat default matches the loader's.
• The selector/policy contradiction is now a load-time ValueError in _target_from_mapping, so the consumer bypass can only ever override an inherited default. That reasoning holds and the bypass comment is now accurate.
• severity="error" explicit, the one-warning-per-target comment, and the docs back-compat sentence are all there.

And no argument on the timing — my pass was against the pre-782ab24 diff. Good that the hole survived the rewrite in a form worth fixing anyway.

One new thing, in the tie-break that arrived in 782ab24.

The assertion tie-break is per (target, period), but resolution is per row.

rows_at_period = [row for row in candidates if dict(row["period"]) == chosen_period]
if len({row["assertion"] for row in rows_at_period}) > 1:
    rows_at_period = [row for row in rows_at_period if row["assertion"] == "observation"]

_select_rows filters on row["geography"]["level"] == geography_level — the level, not a specific geography — so one target routinely matches many rows at a period, one per geography/dimension tuple. ResolvedTarget carries geography, dimensions and entity precisely because that set is expected to be plural.

So the filter is coarser than the thing it's filtering. If any row at the chosen period is an observation, every projection row is dropped — including projections for geographies that have no observation of their own and were never in a tie with anything.

Concretely, and this is the #137 shape you're aiming at: a region-level target at calendar_year:2027 where England, Scotland and Wales have MYE observations and Northern Ireland has only an NPP projection. The set has two distinct assertions, so the branch fires, NI's row is dropped, and NI silently loses its target. The warning names the period only — not the dimension that lost a row — so the report reads as "resolved the observation" (singular) for what was actually three resolutions and one deletion.

Both _tied_assertion_rows tests use a single-series fixture where the tie really is one duplicated series, so the plural case isn't covered either way.

The fix is to group before breaking the tie — key on whatever distinguishes co-resolving rows (geography + dimensions + entity, or semantic_fact_key if that's already the right identity) and apply the observation-wins rule within each group, emitting one warning per genuinely-tied group. That also makes the warning message able to name which series was ambiguous, which is the actionable part.

Same question one level up, for prefer_observed.

observed = [row for row in candidates if row["assertion"] == "observation"]
...
elif assertion_policy == "prefer_observed" and observed:
    candidates = observed

This is global over the target's whole candidate set too, across every geography and every period. So a UK-wide target where GB has observations and NI has only projections resolves nothing for NI, ever. The docs describe this as family-level — "a family with any observed fact resolves only from observations" — and I read that as deliberate, but I suspect most readers will take "family" to mean the series, not "all geographies and periods at once". If the grouping change above is right for the tie-break, the same key probably wants to apply here, and if it doesn't, the docs sentence is worth one more clause saying the scope is the entire selector match.

Everything else looks good to me.

[juaristi22]

Implemented in b6d2e54 — you were right on both levels, and the NI example was exactly the bug.

The grouping. New _series_key identifies a co-resolving series as geography + entity + aggregation + dimension_set_key + universe_constraint_set_key + unit. Two deliberate blindnesses: source identity is excluded so one publisher's estimate and another table's projection of the same series still collide (MYE-vs-NPP same-geography stays a tie), and period/assertion are excluded because they're the axes under adjudication. One note on your semantic_fact_key suggestion: it can't serve directly — it deliberately hashes assertion (and period) into the identity, so an observation and its projection twin get different semantic keys by design. Hence the explicit key.

Tie-break is now per series at the chosen period: observation-wins applies within each genuinely tied group, ambiguous_assertion_at_period fires once per tied series and names it (geography, plus dimensions when present), and a projection-only series is untouched — it resolves with the existing resolved_from_projection warning. Your region-level shape is the test: test_assertion_tie_break_is_scoped_to_the_series has one geography in a genuine tie and another carrying only a projection; the tie resolves, the projection survives, the warning names only the tied geography.

prefer_observed takes the same key: a series with any observed fact resolves only from observations; a projection-only series falls back. test_prefer_observed_lets_a_projection_only_series_resolve is the NI shape resolving both geographies. Within one series nothing changed — test_prefer_observed_still_never_mixes_bases_within_one_series pins the observed-2022-beats-projected-2023 behaviour, and all pre-existing fixtures pass unmodified. Docs/table/policy comment now say series-level rather than family-level, which is what a reader assumed "family" meant anyway.

One honest residual you'll recognize from your first pass: an NPP row carrying an extra sex dimension against an MYE total lands in different groups — different dimension_set_key — so no tie fires and both resolve. That's a loose-selector problem (two overlapping observation sources would behave identically) and no assertion-axis mechanism can see it; the _series_key docstring says so explicitly rather than implying the grouping closes it.

Full suite: 630 passed, 1 skipped.

[vahid-ahmadi]

b6d2e54 closes it. Checked the implementation rather than the description:

• _series_key groups on geography + entity + aggregation + dimension_set_key + universe_constraint_set_key + unit, and the two exclusions are the right ones — dropping source identity is what keeps MYE-vs-NPP a tie, and dropping period/assertion is necessary since those are the axes under adjudication.
• The tie-break is now per series, drops by aggregate_fact_key (which the bundle contract asserts unique), fires one warning per tied series, and names the geography. A projection-only series is genuinely untouched.
• prefer_observed takes the same key via observed_series, so a row survives if it's an observation or its series has no observation. That's the NI shape resolving.

Good catch on semantic_fact_key — you're right that it hashes assertion and period into the identity, so it can't group an observation with its projection twin. I'd missed that; the explicit key is the correct call.

And thanks for stating the extra-sex-dimension residual in the docstring rather than letting the grouping imply more than it delivers. Agreed it's a selector-hygiene problem that two overlapping observation sources would hit identically — the assertion axis isn't the place to solve it.

One small thing on the key, take it or leave it:

unit is in the identity, so the same series published in two units doesn't tie. If an MYE total arrives in persons and the NPP projection of the same geography in thousands, they land in different groups, no tie fires, and both resolve — a double-count of exactly the kind the tie-break exists to prevent. That's arguably a different failure (a unit-normalization problem, not an assertion one) and it may well be impossible in practice if the bundle normalizes units upstream. But unlike the sex-dimension case, the two rows here are the same series by every other axis, so it's worth one line in the docstring saying whether unit divergence within a series is possible at all — if it isn't, that's the reassuring answer and the key is right as-is.

Nothing blocking from me.