Skip to content

feat(encoding): add bounded block dictionary - #8043

Draft
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-4-offset-readerfrom
xuanwo/generic-block-stack-5-block-dictionary
Draft

feat(encoding): add bounded block dictionary#8043
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-4-offset-readerfrom
xuanwo/generic-block-stack-5-block-dictionary

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack 5 of the generic block compression series. Depends on #8042.

This PR adds the concrete block Dictionary codec without enabling production selection. Dictionary owns its u32 indices and typed item child codecs, exposes zero payload only when both children are metadata-only, and otherwise uses one bounded length-framed payload. Reader validation covers item count, child cardinality, frame boundaries, metadata-only child lengths, and index bounds.

Lance 2.0–2.2 selection and bytes remain unchanged. The encoder is exercised directly in tests and will be enabled by the following selector layer.

Validation:

  • cargo fmt --all
  • default and no-default cargo check for lance-encoding tests/benches
  • cargo test -p lance-encoding compression::block
  • cargo test -p lance-encoding compression::tests
  • cargo test -p lance-encoding encodings::physical
  • uv run mkdocs build from docs/
  • cargo clippy --all --tests --benches -- -D warnings

Stack navigation

@github-actions

Copy link
Copy Markdown
Contributor

Important

This PR touches the Lance format specification.

Substantive changes to the format specification — the .proto definitions
and the spec docs under docs/src/format/ — require a PMC vote before merge.
Minor edits such as typo fixes, wording, or formatting are excluded; use your
judgment.

If this is a meaningful format change:

  • Start a vote following the Lance community voting process.
    Format specification modifications need 3 binding +1 votes (excluding the
    proposer), held on GitHub Discussions, with a minimum voting period of 1 week.
  • Once the vote passes, link the completed vote in this PR. It should not be
    merged until the vote is linked.

@github-actions github-actions Bot added A-encoding Encoding, IO, file reader/writer A-format On-disk format: protos and format spec docs enhancement New feature or request labels Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer A-format On-disk format: protos and format spec docs enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant