fix(encoding): handle all-null dictionary arrays

[mmatczuk]

What

• retain dictionary key validity when the dictionary values array is empty
• add an encoder round-trip regression test for an all-null Dictionary<Int32, Utf8> array

Removing validity from an empty dictionary makes key 0 appear valid even though the values array has no entries. Arrow then rejects the rebuilt array as out of bounds. Retaining validity for this case lets the existing all-null encoding path handle the page safely.

Fixes #7906.

Testing

• cargo nextest run -p lance-encoding (512 passed)
• cargo clippy -p lance-encoding --all-targets -- -D warnings
• cargo fmt --all -- --check
• original dataset append reproducer against the local checkout; all dictionary and control cases succeeded

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: QUIET

Plan: Pro Plus

Run ID: 884f4fa1-2406-481b-9184-41e2e16bfaed

📥 Commits

Reviewing files that changed from the base of the PR and between 033daf8 and dda79e7.

📒 Files selected for processing (1)

• rust/lance-encoding/src/encodings/logical/primitive.rs

---

📝 Walkthrough

Walkthrough

PrimitiveStructuralEncoder::extract_validity_buf now preserves dictionary arrays with empty dictionaries, and a regression test verifies round-trip encoding and decoding of an all-null Dictionary(Int32, Utf8) array.

Changes

Dictionary validity handling

Layer / File(s)	Summary
Empty dictionary handling and round-trip coverage rust/lance-encoding/src/encodings/logical/primitive.rs	extract_validity_buf returns unchanged arrays when dictionary values are empty, and an async test covers all-null dictionary round trips.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: xuanwo, ali2arslan

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main fix: handling all-null dictionary arrays in encoding.
Description check	✅ Passed	The description directly matches the code change and regression test for empty all-null dictionaries.
Linked Issues check	✅ Passed	The change addresses #7906 by preserving validity for empty dictionary values and adding a round-trip regression test.
Out of Scope Changes check	✅ Passed	No unrelated code changes are evident beyond the targeted encoder fix and regression test.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}