Skip to content

Add the provider documentation set and finish the decomposition - #1047

Open
jwrosewell wants to merge 138 commits into
IABTechLab:mainfrom
jwrosewell:split/5-response-hook-docs
Open

jwrosewell wants to merge 138 commits into
IABTechLab:mainfrom
jwrosewell:split/5-response-hook-docs

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Stacks on #1046, the client-set Edge Cookie value path, and depends on #1043 to #1046 together, because this pull request documents what they build. The stack has six pull requests (#1043, #1044, #1045, #1046, #1047, #1094), each targeting main, with this one fifth and the last of the five that decompose #838 as requested in the #986 review. Compare split/4-client-resolve with split/5-response-hook-docs to see only this pull request's change.

The design specs for the series are carried by #1084, including the two this pull request used to carry:

What this pull request does

  • The IntegrationResponseMutator response-header hook that an earlier version of this pull request added is not in the series. The hook had no consumer, and the spec set's own rule is against speculative surface. The hook returns with the first integration that needs one, and its spec is the starting bar for that design.
  • The configuration reference gains the [ec] provider selector and the [ec.providers.hmac], [device] and [geo] sections, including the assume_single_jurisdiction acknowledgment and the requires-signal floor on a failed geo lookup, and a section on the country and region rules in permissions.yaml. The Edge Cookie guide is rewritten around providers and the permission model, including the narrow withdrawal rules, the hardened resolve endpoint and the resolved-marker cookie. The setup guide, API reference, error reference, Fastly guide and key-rotation guide are updated to match.
  • The example configuration adds a commented [ec.providers.host-signals] block and rewrites the [device] and [geo] comments to say what each default does and how to override it.
  • CI runs the Axum and Cloudflare jobs in .github/workflows/test.yml on windows-latest as well as ubuntu-latest, and the Axum job also runs the core library's unit tests natively, because on the WebAssembly targets the test harness stops at the first failing test and reports the rest as never run.
  • Two small code changes come with the documentation. A test in crates/trusted-server-core/src/ec/mod.rs again checks that a provider receives the empty string as the client IP when the host cannot determine one, and crates/trusted-server-core/src/evidence.rs corrects its module documentation and moves the HostSignals trait to the top of the file.

What happens to #838

Once the five pull requests that decompose #838 merge, #838 is closed. It stays open as a draft reference for the review period only.

How it was verified

Every job main's CI runs was run locally against 5406acf51, on Windows and under WSL, and all of them passed. That covers cargo fmt --all --check, Clippy with warnings denied on the Axum, Cloudflare, Spin and Fastly adapters, on both wasm targets and on the four permission signal crates, the Axum, Cloudflare and Spin adapter suites, the cross-adapter parity suite, the benchmark smoke run, the release wasm builds for Spin and Fastly, and the CLI, OpenRTB codegen, format-docs and template cache harness jobs that only run on Linux. The core suite passes with 2,885 tests natively and 2,879 under Viceroy. The head then gained the merge 65f7005c9, which touches only tools/permissions-inspector/wasm, a crate that is deliberately its own workspace, so no workspace job's result changes. That crate builds clean for wasm32-unknown-unknown and is rustfmt clean.

CI on this head is green across all 22 checks, being Run Tests, Run Format, Integration Tests, Permissions Inspector and CodeQL Advanced. There are 22 rather than the 20 the earlier pull requests run because this one puts the Axum test job and the Cloudflare check job on a [ubuntu-latest, windows-latest] matrix, so both now run on Windows as well.

References #777 and #778. Decomposes #838. Spec baseline from #986.

@jwrosewell
jwrosewell force-pushed the split/5-response-hook-docs branch 3 times, most recently from 7ebce99 to 700c913 Compare August 25, 2026 10:51
@jwrosewell jwrosewell changed the title Add the integration response-header hook and the provider documentation set Add the provider documentation set and finish the decomposition Aug 25, 2026
@jwrosewell
jwrosewell force-pushed the split/5-response-hook-docs branch 4 times, most recently from c17a7ea to 5b63f48 Compare August 27, 2026 05:37
@jwrosewell
jwrosewell force-pushed the split/5-response-hook-docs branch from 5b63f48 to 0bab4c0 Compare August 27, 2026 15:10
jwrosewell added a commit to jwrosewell/trusted-server that referenced this pull request Aug 27, 2026
The five-PR series (IABTechLab#1043 to IABTechLab#1047) opens the identity, device and geo
seams. The nine vendor integrations already in core sit behind the
integration registry instead, which is a private table, so none of them
can move out until that table is opened.

This spec defines the one core change that opens it: public registration
builders with a second input on IntegrationRegistry, browser JavaScript
carried on the registration, startup validation as a hook, the same
treatment for auction providers and the bid renderer contract, and
neutral replacements for the two places where a vendor reaches into
core. It then sets out the migration of all nine existing integrations,
one PR each. The change is complete in itself: after it, no vendor move
needs a core change.

Written against the series' tree with the file and line references for
every claim about the current code. Documentation only.
@jwrosewell
jwrosewell force-pushed the split/5-response-hook-docs branch 2 times, most recently from 3cfe393 to 45acb97 Compare August 31, 2026 12:50
jwrosewell added a commit to jwrosewell/trusted-server that referenced this pull request Aug 31, 2026
The five-PR series (IABTechLab#1043 to IABTechLab#1047) opens the identity, device and geo
seams. The nine vendor integrations already in core sit behind the
integration registry instead, which is a private table, so none of them
can move out until that table is opened.

This spec defines the one core change that opens it: public registration
builders with a second input on IntegrationRegistry, browser JavaScript
carried on the registration, startup validation as a hook, the same
treatment for auction providers and the bid renderer contract, and
neutral replacements for the two places where a vendor reaches into
core. It then sets out the migration of all nine existing integrations,
one PR each. The change is complete in itself: after it, no vendor move
needs a core change.

Written against the series' tree with the file and line references for
every claim about the current code. Documentation only.
jwrosewell added a commit to jwrosewell/trusted-server that referenced this pull request Aug 31, 2026
The review of IABTechLab#1043 asked that spec changes land before the code that
implements them, so a divergence is a decision taken in review rather
than a ratification of something already merged. PRs IABTechLab#1043 to IABTechLab#1047
each carried the design document for their own step, and IABTechLab#1043 carried
a 607-line spec describing device providers, geo providers, the
permission model and the browser resolve endpoint, none of which is in
that PR.

Move all six series documents here, so this PR carries the complete
normative set and no code:

- 2026-07-30-pluggable-providers-design.md (from IABTechLab#1043)
- provider-code-registry.md (from IABTechLab#1043)
- 2026-07-30-permission-model-design.md (from IABTechLab#1045)
- 2026-07-30-client-cycle-ec-resolve-design.md (from IABTechLab#1046, later
  revised by IABTechLab#1047)
- 2026-07-30-integration-response-header-hook-design.md (from IABTechLab#1047)
- 2026-07-30-provider-migration-rollout-design.md (from IABTechLab#1047)

Each file is taken verbatim at the tip of the stack, so the later
revisions are preserved: the provider-switching continuity section, the
geo requires-signal floor, and the code-envelope paragraph IABTechLab#1047 added
to the client-cycle spec. The revision-record tables are unchanged. No
document's substance was edited.

The only edits are to this spec's own status line, which said the PR
adds one document and that the series specs land with IABTechLab#1047, and a
revision-record row recording the move.
@aram356

aram356 commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

The sequencing discussion for this series is on #1084. This PR is superseded rather than rejected. The design in §3.6 is accepted and most of the provider work carries over onto the reordered base. See #1084.

jwrosewell added a commit to jwrosewell/trusted-server that referenced this pull request Sep 1, 2026
The four series specs (client-cycle EC resolve, permission model,
pluggable providers, migration and rollout) each carried a Status line
saying they were implemented. The code they describe is only in PRs
IABTechLab#1043 to IABTechLab#1047 and none of those is merged, so the line read as shipped
behavior. Each now says Proposed, names the PR that carries the
implementation and states that it is not yet on main, keeping the
existing revision dates and notes.

The integration provider seam spec carried counts and line references
that do not hold on main at d516a9e. Corrected against that commit:

- Section 4 said migration_guards.rs embeds "the thirteen vendor
  files". The directory holds 23 .rs files (2 infrastructure, 6 in
  nextjs/, 2 in datadome/, 13 top-level integration modules), the guard
  embeds 20 of them and 9 of those 20 belong to the nine vendors, with
  osano.rs and the two datadome/ files absent. builders() registers 13
  integrations, which is a different 13 from the file count.
- Section 3.5 gave no counts for the prepare and finalize calls. There
  are nine production prepare_request call sites across the four
  adapters and a tenth in core, and the single production
  finalize_response call site is in core rather than in any adapter.
- Section 8 item 3 described a proxy resolving geo twice, which does
  not happen on main. The real double resolution is the adapter EC
  context build against handle_auction on POST /auction.
- Section 8 item 5 understated the Spin gap and misdescribed
  Cloudflare. Cloudflare covers every route it registers and has no
  health route, while Spin skips its first-party bindings as well as
  its inline admin stubs.
- Line references: settings.rs:166 to :215, auction/mod.rs:49 to the
  list at :51 to :53, publisher.rs:4361 to :4369.

Section 6 now requires the round trip to be proven on the Fastly
adapter, the primary deployment target, rather than on any adapter,
because Fastly has no library target and the round trip otherwise only
runs on the Axum dev server.
@jwrosewell
jwrosewell force-pushed the split/5-response-hook-docs branch 6 times, most recently from 98764db to d48c98d Compare September 1, 2026 23:04
Brings the documentation and assertion message fixes up from
split/4-client-resolve. The merge has no conflicts.
Brings main at 066ea3c into split/1-ec-provider. Twelve files
conflicted and one more needed a change without a textual conflict.
Most resolutions keep both sides. Five needed a decision, recorded here
so that no behavior changes silently.

Edge Cookie generation, in ec/mod.rs. Main's IABTechLab#885 creates a row only
when no row holds the key, retries on a collision and binds the request
snapshot to the new row. This branch creates identifiers through the
selected provider. Both are kept. Each attempt asks the provider for a
candidate through the new EcContext::candidate_id, which runs the
reserved-header and identifier-bound checks, then creates the row with
create_if_absent under the provider's canonical key and binds the
snapshot to that key.

Orphan recovery, in ec/finalize.rs. Main's IABTechLab#885 rotates an orphaned
cookie by generating an HMAC identifier directly, which would give a
vendor-provider deployment a built-in identifier. Recovery now asks the
selected provider through candidate_id, keeps main's proof of absence
and retry limit, and does nothing when no provider is selected. Any
headers the provider asks for while creating the replacement are
applied to the response.

Withdrawal, in ec/finalize.rs. Main's IABTechLab#1113 hardening is kept whole,
being finalize_unusable_consent, the existence check inside
write_withdrawal_tombstone, the snapshot write-back and
log_tombstone_outcome. The tombstones are keyed by this branch's
withdrawal_kv_keys, the canonical keys of the cookie and the active
identifier, in place of withdrawal_ec_ids, and the write-back compares
against the active identifier's canonical key. EID ingestion on the
returning and generated paths uses main's collect-then-upsert form
under the canonical key.

Secret references, in config.rs. Main's IABTechLab#1036 resolves secret settings
from the secret store by the paths listed in secret_fields. This branch
makes the legacy ec.passphrase optional and adds the
[ec.providers.hmac] passphrase, which was not listed, so it would have
been read as a literal value. Both are now listed as optional paths and
checked as key references when the configuration is pushed, and the
integration fixture and the example configuration name the secret key
rather than a value.

Spin start-up, in the Spin adapter. Main's IABTechLab#1036 loads settings from
the config store with secret resolution, which fixes the same start-up
failure this branch's Spin commit fixed. Main's loader is taken and
SpinPlatformConfigStore, which nothing else uses, is removed.

The four adapters keep this branch's composition-root provider check,
and Axum, Cloudflare and Spin keep its error propagation from
build_ec_context, alongside main's compiled auction plan from IABTechLab#1016.

Tests. Main's collision-retry tests and its orphan-rotation tests built
their context with no provider selected, which main's code did not
need. They now select the built-in HMAC provider, as an HMAC deployment
does, because with no provider there is nothing to create or rotate. No
other test reaches identifier creation or orphan recovery without a
provider selected. Tests from each side were moved onto the other
side's interfaces, being this branch's six-parameter context helpers,
main's mutable context in ec_finalize_response, and the
[ec.providers.hmac] passphrase in main's two secret validation tests.
Main's IABTechLab#1016 builds the orchestrator and the integration registry from a
compiled auction plan, and limits IntegrationRegistry::new to core's own
tests. The merge of main moved the Axum adapter's state building onto the
plan and took main's imports, but missed the test helper
state_with_uninjected_provider, which still called build_orchestrator and
IntegrationRegistry::new, so the adapter's library tests stopped compiling.
The helper now builds both from the plan, the same way
build_state_with_settings does.
Brings upd/split/1-ec-provider at cb28ad3, which carries main at
066ea3c, into split/2-device-geo. Six files conflicted, all where this
branch's host signals meet main's compiled auction plan or main's
imports. Each resolution keeps both sides' changes, so nothing either
side does is dropped.

The Axum, Cloudflare and Spin adapters keep this branch's provider
calls, which pass the host signals (None in all three) as well as the
injected provider, with this branch's comments saying so. They then
build the orchestrator and the integration registry from main's
compiled auction plan, the way split/1 does.

The Fastly imports keep what each side still uses. app.rs keeps main's
StoreName alongside this branch's build_geo_provider and
FastlyHostSignals. main.rs drops the config_store_name import, because
main replaced its only use with the runtime store names, and keeps this
branch's device provider import. platform.rs keeps main's
BackendNamingPolicy and drops GeoInfo and PlatformGeo, which this branch
moved into the trusted-server-geo-fastly crate.
Main's IABTechLab#1036 resolves secret settings from the secret store by the
paths TrustedServerAppConfig::secret_fields lists, and push validation
skips the validators on those paths because they hold key names.
Merging main registered the [ec.providers.hmac] passphrase, but not this
branch's [ec.providers.host-signals] passphrase. A key name there failed
the passphrase length check when the configuration was pushed, and a
key name long enough to pass would have been used unresolved as the
HMAC key.

The path is now listed as an optional secret and checked as a key name.
EdgeZero matches secret paths against validation error keys verbatim,
and the derived validation keyed the block's errors by the Rust field
name host_signals, so EcProviders now validates each built-in block
under the key the configuration uses. New tests cover pushing a
host-signals key name, resolving one from the mapped store, and still
rejecting a short resolved value.
Brings upd/split/2-device-geo at 0832f00, which carries main at
066ea3c through split/1, into split/3-permissions. Eleven files
conflicted, and CLAUDE.md changed type. Most resolutions keep both
sides. Four needed a decision, recorded here so that no behavior changes
silently.

Withdrawal, in ec/finalize.rs. Main's IABTechLab#1113 moved the path for a request
whose Edge Cookie is not permitted into finalize_unusable_consent, and
tells that path whether the request withdraws with ec_consent_withdrawn,
which looks for an explicit withdrawal signal in the consent context.
This branch decides withdrawal in the permission model, where the signal
providers answer it when the permissions are assembled, and reads the
answer with storage_withdrawn. Main's hardened path is kept whole and is
given storage_withdrawn as that answer. Of the shipped schemes only a
TCF record refusing storage withdraws, and only where the storage
baseline is not granted. On main, has_explicit_ec_withdrawal also treats
a Global Privacy Control, GPP or US Privacy sale opt-out in a US state as
a withdrawal. On this branch those opt-outs suppress without
withdrawing, as the permission model spec records, so the Edge Cookie
headers are stripped and the cookie is kept.

The identifier sent to auction partners, in auction/endpoints.rs and
publisher.rs. Main's IABTechLab#885 makes the identifier an owned value so the
identity-graph snapshot can be written back to the context, and on the
navigation path keeps the active identifier unfiltered for the snapshot
preload and finalization. This branch forwards the identifier only when
sharing is permitted, being storage plus personalized-ad selection, the
same pair that gates EIDs. Both are kept. The snapshot preload still
reads the active identifier, and only the forwarded copy is gated by
sharing.

Settings validation, in settings.rs. Main's IABTechLab#1117 removed the warning
about disabled creative rewriting, because request-time settings loading
logged it on every request. The warning stays removed. This branch's log
of the permission baseline sits in the same function and would repeat
for the same reason, so it now logs at debug level rather than info.

The template cache harness. Main rewrote the configuration edits with a
replace_once helper that stops on a missing target. This branch's
switch to the platform geo provider, which gives the harness a US state
jurisdiction now that default_country is retired, is written the same
way. The harness fastly.toml gets main's local backend and secret stores
as well as this branch's Viceroy geolocation answer.

CLAUDE.md is a symlink to AGENTS.md on main (IABTechLab#923). The 80 lines this
branch added to CLAUDE.md are applied to AGENTS.md instead, and
CLAUDE.md stays the symlink.

The four adapters build main's compiled auction plan (IABTechLab#1016) and then
this branch's permission signal providers. The example configuration
keeps this branch's [permission_signal] notes and main's asset proxy
example.

Tests. Main's endpoint tests from IABTechLab#885 and IABTechLab#1016 called the test helper
with a jurisdiction, where this branch's helper takes the permission
gate. They now build a non-regulated context with the gate open, which
is what their jurisdiction gave them on main. Main's withdrawal tests
from IABTechLab#885 and IABTechLab#1113 used a Global Privacy Control opt-out as the
withdrawal. They now use a TCF record refusing storage and state the
withdrawal answer, as this branch's withdrawal tests do because core
links no provider, and what they check, the tombstone and cookie
handling, is unchanged. This branch's own opt-out test, which now passes
main's mutable context, keeps the case where an opt-out leaves the
cookie in place. Main's asset proxy test from IABTechLab#742 sets this branch's
permissions_script field to None, and its settings acknowledge
single-jurisdiction operation, which this branch requires of an Edge
Cookie provider with no geo provider.
The Windows notes said CI runs the adapter tests on both ubuntu-latest
and windows-latest. No job in this branch's test workflow runs on
Windows, so the sentence now says ubuntu-latest.

The notes also say that trusted-server-cli does not build on a Windows
host, because its dependency edgezero-adapter-fastly uses a standard
library feature that is unstable on Windows, so its tests and the
template cache harness, which builds it, run in WSL too.
Brings upd/split/3-permissions at 3c9f4b0, which carries main at
066ea3c through split/1, split/2 and split/3, into
split/4-client-resolve. Two files conflicted. Each resolution keeps both
sides' changes.

ec/finalize.rs takes main's imports for identity-graph writes and EID
collection, and keeps this branch's expire_ec_resolved_marker beside
them. This branch's two resolved-marker tests from ee07418 sit beside
main's orphan-recovery tests from IABTechLab#885, and now pass main's mutable
context to ec_finalize_response.

docs/guide/api-reference.md keeps this branch's section for the
POST /_ts/api/v1/ec/resolve endpoint and drops the POST /third-party/ad
section, which main's IABTechLab#1016 removed.
Brings upd/split/4-client-resolve at 69e7d57, which carries main at
066ea3c through split/1 to split/4, into split/5-response-hook-docs.
Five files conflicted, all in the documentation set and the example
configuration, and all for one reason. This branch documents split/1's
move of the Edge Cookie passphrase under [ec.providers.hmac] behind
provider = "hmac", still as a plaintext value, while main's IABTechLab#1036 made
the passphrase the name of a secret-store key. Every resolution keeps
this branch's layout with main's key-name model.

In configuration.md, ec-setup-guide.md, error-reference.md and
fastly.md, the TOML examples select provider = "hmac" with
passphrase = "ec_passphrase" under [ec.providers.hmac], and the
environment overrides use TRUSTED_SERVER__EC__PROVIDERS__HMAC__PASSPHRASE
with the key name. Three examples merged cleanly with the plaintext
placeholder still under [ec.providers.hmac] and take the key name too.
The [ec.providers.hmac] reference table gains main's note that the
passphrase is a key name, and its passphrase row says so. The security,
staging and placeholder bullets are main's, because this branch's
bullets described plaintext values in environment variables. The
validation bullets keep this branch's provider bullet and take main's
three, naming providers.hmac.passphrase.

trusted-server.example.toml keeps this branch's host-signal provider
block with a key-name placeholder, because that passphrase is also
resolved from the secret store (a75571d on split/2), and takes main's
partner comment.
The example configuration said the host-signal provider creates the
identifier from the host's TLS JA4 and HTTP/2 signals instead of the
client IP. HostSignalProvider::generate hashes the JA4 and HTTP/2
signals together with the client IP, as the provider's own
documentation says, so the comment now says so.
The configuration guide said a failed geo lookup at request time does
not fall back to default_country, a setting that no longer exists. The
sentence now says what a failed lookup does without naming it.
Christian Pavilonis's review thread on provider response effects asked
core to validate them against the managed ts- cookies, the x-ts-
namespace and framing headers. The check is in place, but several
comments said a rejected effect "fails the request", which is not what
happens. EcContext::generate_if_needed returns the error, and its only
two callers outside tests, the publisher fallback in the Fastly adapter
and IntegrationRegistry::handle_proxy, log it and serve the response
without an Edge Cookie. The reserved_response_effect and
apply_provider_response_headers docs, the comment in
EcContext::candidate_id and the reserved-surface test now say that.
The candidate_id comment no longer implies the rejection matches the
identifier-bounds check in every respect, because that check runs after
the provider's headers are captured.

A new test, a_rejected_provider_effect_never_reaches_the_finalized_response,
proves a rejected header never reaches the response. For each reserved
effect it lets generation fail, runs EC finalization on the same
context, and checks the response carries no forged ts-ec cookie, no
x-ts-ec header and no transfer-encoding. It failed when the header
capture in candidate_id was moved ahead of the reserved check, and
passes with the code as it is. The test helper now hands back the
context even when generation fails, so the test can finalize on it.

The review thread is
IABTechLab#1043 (comment)
Christian Pavilonis's review thread on partner paths asked for
validation and KV normalization to be dispatched by provider code.
Validation already was, but several paths still read or wrote identity
graph rows under the identifier as issued, while generation stores each
row under the owning provider's canonical form. For a provider whose
canonical form differs from the cookie value those paths found no row.

- Pull sync validated the identifier but kept only the raw value. It
  looked the request snapshot up under that value, while generation
  binds the snapshot to the canonical key and every read EC
  finalization makes uses that key, so it skipped every partner. Its
  revalidation read and write-back used the raw value too.
- The admin lookup answered 404 for a row that exists.
- The /auction, publisher navigation and /_ts/page-bids preloads loaded
  a miss, and resolve_auction_eids matched the snapshot by the raw
  identifier, so auctions carried no server-side EIDs.
- The navigation preload also replaced the snapshot generation had just
  bound to the canonical key with that miss, so a newly created
  identifier got no ts-ec cookie on a navigation with no EID cookies to
  ingest.

Pull sync now carries the canonical key and uses it for the request
snapshot lookup, the revalidation read and the write-back. Partners
still receive the identifier as issued, and the pull rate limit key
still hashes the issued identifier, which leaves rate limiting
unchanged for every provider. The admin lookup reads under the key,
reports the requested identifier as ec_id, and adds the key it read as
kv_key, which the API reference now describes. The three preloads load
under EcContext::ec_kv_key and resolve_auction_eids looks the entry up
under the key. EcContext::accepts_id lost its only caller and is
removed.

For the built-in HMAC provider the canonical key is the identifier
itself for every identifier read-back accepts, so these paths behave
as before for HMAC cookies. Apart from the new kv_key field, the one
change an HMAC deployment can see is that an admin lookup given an
identifier with an uppercase hash now finds the row stored under the
lowercase key instead of answering 404.

Each path has a test using CanonicalizingProvider, whose identifier
t0ca~MiXeD.CaseId is stored under t0ca~mixed.caseid, and all seven new
tests failed before the fix. The shared constants for that identifier
now live beside the provider, so the identify, finalization and new
tests use one definition. The known-gap note on EcContext::kv_key_for,
which cited commit 343ac3e from outside this branch, and the matching
note on AcceptedProviders now state which paths key rows through the
canonical form.

The review thread is
IABTechLab#1043 (comment)
Some comments added for the two review threads on provider response
effects and canonical keying claimed more than the code does, and one
changed comparison had no test that depended on it.

The AcceptedProviders doc said pull sync, batch sync and the admin
lookup all read and write rows, but the admin lookup only reads. The
comment in the test
a_rejected_provider_effect_never_reaches_the_finalized_response said a
rejected header kept anywhere on the context would reach the browser,
when EC finalization applies only the response headers the context
holds. The admin lookup docs and the API reference entry for kv_key
named the owning provider as the source of the row key, which does not
hold for a deployment with no provider selected, where the built-in
HMAC identifier format supplies the key. Three other comments now say
precisely what they mean. The EcContext::kv_key_for doc names the
function each reference points at, the EcContext::candidate_id comment
no longer credits an unnamed caller with serving the response, and the
dispatch_pull_sync doc names ec_hash as the input to the rate limit key.

Before replacing the snapshot that generation bound, the publisher
navigation preload compares that snapshot with its fresh read. The
comparison is keyed by the canonical key, yet the navigation tests for
a provider whose canonical key differs from the cookie value passed
with it keyed by the identifier as issued, because their stores
returned the row on the first read. The test for a newly created
identifier's cookie now also runs against a store whose first point
read misses the row generation just wrote. With the comparison keyed
by the identifier as issued, the preload replaced the snapshot with
that miss, EC finalization skipped the cookie and the test failed.
With the canonical key the test passes.
Main's configuration-driven OpenRTB auction providers (IABTechLab#1016) added
three startup tests each to the Cloudflare and Spin adapters, and their
settings set the deprecated [ec] passphrase with no geo provider. That
form migrates to provider = "hmac", and on this branch
GeoConfig::validate_jurisdiction_acknowledgment requires
[geo] assume_single_jurisdiction = true for an Edge Cookie provider
with no geo provider, so the settings failed to load and all six tests
failed. The four settings blocks now set it, as the other test settings
in both adapters that select an Edge Cookie provider already do.
A provider's own response headers, such as an evidence cookie, reached
the browser even when generation discarded the candidate they came
with. EcContext::candidate_id kept the headers before checking the
identifier against the cookie bounds, and generate_with_provider left
them on the context when the candidate collided with an existing row
or its row could not be written. EC finalization applies whatever
headers the context holds, and the publisher and integration proxies
log a generation error and still serve the response, so the cookie
went out with no identifier stored for it.

candidate_id now keeps the headers only once the identifier has passed
the bounds check, or when the provider produced no identifier at all,
and generate_with_provider drops them with a colliding or unpersisted
candidate. The reserved-surface check asked for in the review thread on
provider response effects already ran before any header was kept, so
the same rule now also holds when the identifier is rejected, when it
collides and when its row cannot be written.

A test covers each of those three cases, and all three failed before
the change with the provider's cookie on the finalized response.
HeaderSettingProvider now takes the identifier it returns, so a test
can pair a permitted header with an identifier outside the cookie-safe
alphabet. The finalization comment on applying provider headers now
names candidate_id as the place they are checked, where it named
generate_with_provider.

The review thread is
IABTechLab#1043 (comment)
The API reference said the explicit admin EC lookup route accepts an EC
ID in the bare {64 lowercase hex}.{6 alphanumeric} form. The route
accepts whatever AcceptedProviders::canonical_kv_key accepts, which is
an identifier created by the selected provider, such as the built-in
HMAC provider's hmac~ form, and the bare legacy form that provider
still reads, with both HMAC forms accepted when no provider is
selected. The hash may be given in either case, because
canonical_kv_key lowercases it before the check.

The note on retiring the legacy bare reader said a page view with
ts-eids or sharedId cookies runs ingest_eid_cookies in
ec_finalize_response and so restarts the row's one-year clock. Main's
change threading the EC KV read through the request (IABTechLab#885) moved
finalization to collect_eid_cookie_updates and
upsert_partner_ids_from_snapshot, which writes nothing unless a partner
ID is added or changed. The note now names that function and says only
such a view restarts the clock.
Brings upd/split/1-ec-provider at 1b88e42 into split/2-device-geo.
The five commits this branch lacked, on top of the cb28ad3 it
already had, answer two review threads on IABTechLab#1043 and correct what
reviewing those answers found.

- Pull sync, the admin lookup, /auction, the publisher navigation
  preload and /_ts/page-bids read and write identity-graph rows under
  the owning provider's canonical key (0e7f7eb and fb1bc28).
- The docs on a rejected provider effect say generation returns an
  error and the page is served without an Edge Cookie, with a test that
  a rejected header never reaches the finalized response (0980b73).
- A provider response's headers are kept only for a candidate that
  generation commits (cb6f717).
- The API reference lists the admin lookup's accepted EC ID forms, and
  the bare reader note names the function that writes EID updates
  (1b88e42).

The merge had no conflicts. The merged tree passes a native
all-targets check, 2,763 core tests, the Axum, Cloudflare and Spin
tests and the three wasm checks.
Brings upd/split/2-device-geo at 37eadae, which carries the split/1
review follow-ups (0980b73, 0e7f7eb, fb1bc28, cb6f717 and
1b88e42), into split/3-permissions. Two blocks conflicted.

- The build_pull_sync_context doc keeps this branch's statement that
  pull sync needs the sharing permission pair, and adds the follow-up's
  case where no provider this deployment reads owns the identifier.
- In handle_publisher_request the forwarded identifier keeps this
  branch's ec_sharing_allowed() filter, and the follow-up's
  active_kv_key line goes in front of it, so the navigation preload
  reads and compares the row under the canonical key.

One test needed a change without a textual conflict. The follow-up's
auction_endpoint_loads_the_row_under_the_canonical_key built its
context with make_ec_context(Jurisdiction::NonRegulated, ...), and on
this branch that helper takes the permission gate as a bool. The test
now uses this branch's make_non_regulated_ec_context, which opens the
gate in a non-regulated jurisdiction. The other new tests build their
context with EcContext::new_for_test, which on this branch grants
storage and personalized-ad selection, so they pass the sharing gate
unchanged.

Tests. The merged tree passes a native all-targets check, 2,851 core
tests, the Axum, Cloudflare and Spin adapter tests, the permission
signal crate tests, and the Fastly, Cloudflare and Spin wasm checks.
Brings upd/split/3-permissions at e3bb4b9 into split/4-client-resolve.
That carries the split/1 review follow-ups (0980b73, 0e7f7eb,
fb1bc28, cb6f717 and 1b88e42), their merge into split/3 under
that branch's permission gate, and split/3's be0ed2d, which lets the
Cloudflare and Spin startup tests from main's IABTechLab#1016 load under the
single-jurisdiction rule. This branch carries the same four settings
blocks, so it takes that fix here.

The merge had no conflicts. The resolve endpoint already applies a
provider's response headers only with the 200 that sets the cookie or
the 204 when the provider creates nothing, which is the rule cb6f717
brings to organic generation, so nothing in resolve.rs changes.

Tests. The merged tree passes a native all-targets check, 2,885 core
tests, the Axum, Cloudflare and Spin adapter tests, the permission
signal crate tests, and the Fastly, Cloudflare and Spin wasm checks.
Brings upd/split/4-client-resolve at 2c34be4 into
split/5-response-hook-docs. That carries the split/1 review follow-ups
(0980b73, 0e7f7eb, fb1bc28, cb6f717 and 1b88e42) through
split/2, split/3 and split/4, and split/3's be0ed2d fix for the
Cloudflare and Spin startup tests, whose four settings blocks this
branch carries too.

One block conflicted, the admin lookup sentence in
docs/guide/api-reference.md. This branch added the hmac~ prefix to
main's {64 lowercase hex}.{6 alphanumeric} sentence, and split/1's
1b88e42 rewrote the same sentence to list every form the route
accepts, being an identifier from the selected provider such as the
hmac~ form, the bare legacy form the HMAC provider still reads, and
both HMAC forms when no provider is selected. That sentence covers what
this branch added, so it is kept.

Tests. The merged tree passes a native all-targets check, 2,885 core
tests, the Axum, Cloudflare and Spin adapter tests, the permission
signal crate tests, and the Fastly, Cloudflare and Spin wasm checks,
and Prettier accepts the resolved API reference.
The inspector calls `assemble_permissions`, which this branch later gave two
more arguments, the request evidence and the signal providers. Nothing caught
it, because `tools/permissions-inspector/wasm` is deliberately its own
workspace, so the gates that build the main workspace never build it, and the
`Permissions Inspector` workflow that does is added by this branch too and so
first ran on this pull request. It failed with three E0061 errors.

The inspector now builds the same four providers every adapter offers, in the
same default order (`gpc`, `gpp-sale-opt-out`, `us-privacy`, `tcf`), and hands
them to `assemble_permissions`. Passing an empty slice would have compiled just
as well and silently stopped the page acting on any signal at all, which is the
one thing the page exists to show.

The evidence argument is an empty `OwnedRequestInfo`. The page carries no
request, only the consent signals its form collects, and each of the four
providers answers from the consent record rather than from request evidence, so
empty evidence changes none of their answers. The comment in `eval_json` says
so, and says that a provider reading a header or a cookie would need real
evidence there.

The lock file picks up the four crates, and with them the edgezero v0.0.7 to
v0.0.8 bump and `cssparser`, which this separate lock had not taken from the
`main` merge.

`build.rs` and one line of `src/lib.rs` are rustfmt output. The crate is outside
the workspace, so `cargo fmt --all` never reached it and `build.rs` had never
been formatted.

Tests. `./scripts/build-inspector-wasm.sh` builds clean for
wasm32-unknown-unknown, and `cargo fmt --check` on the crate is clean.
Brings upd/split/3-permissions at 88a7482 into split/4-client-resolve. That
carries the one commit which makes `tools/permissions-inspector/wasm` pass the
request evidence and the signal providers to `assemble_permissions`, and which
formats the crate. This branch carries the same inspector and the same
`Permissions Inspector` workflow, so it failed the same way.

The merge had no conflicts and touches only the four files under
`tools/permissions-inspector/wasm`.

Tests. `./scripts/build-inspector-wasm.sh` builds clean for
wasm32-unknown-unknown on this tree, and `cargo fmt --check` on the crate is
clean.
Brings upd/split/4-client-resolve at b7dd13f into
split/5-response-hook-docs. That carries split/3's 88a7482, which makes
`tools/permissions-inspector/wasm` pass the request evidence and the signal
providers to `assemble_permissions`, and which formats the crate. This branch
carries the same inspector and the same `Permissions Inspector` workflow, so it
failed the same way.

The merge had no conflicts and touches only the four files under
`tools/permissions-inspector/wasm`.

Tests. `./scripts/build-inspector-wasm.sh` builds clean for
wasm32-unknown-unknown on this tree, and `cargo fmt --check` on the crate is
clean.
Renames `config/permissions/vanilla.yaml` to `config/permissions/sample.yaml`
and says at the top of the file, in the guide and on the constant that the file
is for testing and evaluation only, is not a production policy and is not legal
advice. "Vanilla" described the flavour of the rules and said nothing about
whether anyone should run them, which is the thing a reader needs to know
first.

The display name becomes "Sample (testing and evaluation only)", so the
inspector's dropdown carries the warning wherever the page is opened. The
build script globs `config/permissions/*.yaml`, so its manifest picks the new
name up with no change.

`include_str!` in `permissions.rs` follows the rename. The doc comment above it
is rewrapped and loses a clause-joining semicolon. The guide gains a short
paragraph saying the same thing.

The word "vanilla" is left alone where it means plain JavaScript, in the
DataDome and Lockr script guards and in an older plan document.

Tests. `cargo fmt --all --check` is clean, the core suite passes 2,851 tests,
and `./scripts/build-inspector-wasm.sh` builds and writes a manifest reading
`{"file":"sample.yaml","name":"Sample (testing and evaluation only)"}`. The
docs Prettier configuration sets `proseWrap: preserve`, so the shorter path
cannot change the formatting.
Brings upd/split/3-permissions at a2a1dfc into split/4-client-resolve. That
renames `config/permissions/vanilla.yaml` to `config/permissions/sample.yaml`
and says in the file, in the guide and on the constant that the sample is for
testing and evaluation only, is not a production policy and is not legal
advice.

The merge had no conflicts. `trusted-server.example.toml` auto-merged, because
this branch changes other parts of the same file.

Tests. `cargo fmt --all --check` is clean and the core suite passes 2,885
tests.
configuration guide

Brings upd/split/4-client-resolve at e17090b into
split/5-response-hook-docs, which carries split/3's a2a1dfc renaming
`config/permissions/vanilla.yaml` to `config/permissions/sample.yaml`.

This branch adds the permissions paragraph in `docs/guide/configuration.md`,
which split/3 does not have, so the merge could not correct it. That sentence
now names the new path and says the sample is for testing and evaluation only
and is neither a production policy nor legal advice, matching the file header
and the permission model guide.

The merge had no conflicts. `trusted-server.example.toml` auto-merged.

Tests. `cargo fmt --all --check` is clean and the core suite passes 2,885
tests. The docs Prettier configuration sets `proseWrap: preserve`, so the
edited sentence cannot change the formatting of the file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants