feat(table): load spilled row lineage ahead of a commit

[BubbleCal]

Fourth of the §5.3 stack (#8931, #9250), on top of #9337. Updates on a table with spilled lineage keep every row's lineage, the way they do on an inline table.

Stack

1. feat(format): define hidden row lineage columns #9253 feat(format): define hidden row lineage columns
2. feat(dataset): read and write spilled row lineage columns #9336 feat(dataset): read and write spilled row lineage columns
3. feat(dataset): spill row lineage at compaction #9337 feat(dataset): spill row lineage at compaction
4. this PR feat(table): load spilled row lineage ahead of a commit
5. feat(dataset): spill row lineage when updating rows #9339 feat(dataset): spill row lineage when updating rows
6. feat(dataset): write compaction's spilled lineage into the fragment's data file #9347 feat(dataset): write compaction's spilled lineage into the fragment's data file
7. test(bench): measure row lineage placement inline versus spilled #9340 test(bench): row lineage spill benchmark

Problem

Building a manifest is synchronous and has no object store. Two commit-time paths need existing lineage: resolving which rows an update rewrote, to carry each row's created-at version over (resolve_update_version_metadata), and overlaying a partial column rewrite's patched offsets onto the fragment's last-updated-at sequence (refresh_row_latest_update_meta_for_partial_frag_rewrite_cols). Once a fragment's sequences are spilled, neither can read them, and the previous PRs made them refuse with NotSupported.

Change

The commit path in lance reads every spilled sequence of the current manifest ahead of each build attempt (load_spilled_row_lineage, served from the same caches the readers use) and hands them over in ManifestBuildConfig::spilled_row_lineage. The two paths consult that map for a spilled fragment and still refuse if a sequence they need is missing, so a caller of build_manifest that skips the read-ahead gets an error rather than defaulted lineage. Loaded per attempt, so a rebase onto a newer manifest sees that manifest's fragments; only Update and DataOverlay operations pay for it.

UpdateBuilder, merge_insert in both write modes, and externally assembled Operation::Updates therefore work on a spilled table unchanged. The lineage they produce for the rewritten rows is inline; a refreshed last-updated-at sequence goes back inline as well. An update-heavy table thus regrows its manifest between compactions and the next compaction spills it again.

Validation

• cargo test -p lance-table row_version (spilled source lineage resolves from the config; missing lineage still refuses)
• cargo test -p lance --lib rowids:: (update keeps the rewritten row's id and created-at; merge_insert keeps matched rows' lineage and stamps inserted ones; a partial column rewrite on a spilled fragment stamps only the patched rows)
• cargo test -p lance --lib -- rowid row_version stable_row update::tests merge_insert::tests dataset_transactions
• cargo clippy --all --tests --benches -- -D warnings, cargo fmt --all

Refs #8931, #9250

🤖 Generated with Claude Code