Skip to content

[common] Fix CHAR min/max statistics collection - #4080

Open
beetle0915 wants to merge 4 commits into
apache:mainfrom
beetle0915:fix-4066-char-statistics
Open

[common] Fix CHAR min/max statistics collection#4080
beetle0915 wants to merge 4 commits into
apache:mainfrom
beetle0915:fix-4066-char-statistics

Conversation

@beetle0915

Copy link
Copy Markdown

Purpose

Linked issue: close #4066

DataTypeChecks.isSupportedStatisticsType() lists CHAR as a supported
statistics type, but LogRecordBatchStatisticsCollector does not handle
CHAR when collecting min/max values. As a result, the bounds remain null
and predicates on CHAR columns cannot prune record batches.

LogRecordBatchStatisticsWriter also does not handle non-null CHAR
min/max values, so adding collector support alone would cause statistics
serialization to fail.

Brief change log

  • Add CHAR min/max collection using InternalRow#getChar,
    BinaryString::compareTo, and BinaryString::copy.
  • Add CHAR statistics serialization using
    AlignedRowWriter#writeChar.
  • Add a serialization round-trip test covering min/max values, null count,
    and a value shorter than the declared CHAR length.
  • Add a predicate pruning test for a CHAR column.

Tests

  • LogRecordBatchStatisticsCollectorTest#testCharStatisticsSerializationRoundTrip:
    verifies that CHAR(3) values, including the shorter value "a", are
    collected and serialized with the expected min/max values and null count.
  • RecordBatchFilterTest#testCharStatisticsCanPruneRecordBatch:
    verifies that CHAR statistics can exclude a record batch that cannot
    satisfy the predicate.
  • Targeted tests: 24 tests, 0 failures.
  • mvn spotless:check -pl fluss-common,fluss-server passes.
  • Checkstyle and Apache RAT validation pass.

API and Format

No public API or statistics format change.

The implementation uses the existing CHAR internal representation and
statistics serialization format.

Documentation

No documentation change; behavior-only bug fix.

  • No generative AI tools used
  • Yes (please specify the tool below)

Assisted by: OpenAI Codex following the guidelines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CHAR columns are accepted for statistics but never collect min/max

1 participant