Skip to content

GH-1190: Reserve view slots for empty view vector values#1192

Open
goutamadwant wants to merge 1 commit into
apache:mainfrom
goutamadwant:fix/GH-1190-view-buffer-empty-values
Open

GH-1190: Reserve view slots for empty view vector values#1192
goutamadwant wants to merge 1 commit into
apache:mainfrom
goutamadwant:fix/GH-1190-view-buffer-empty-values

Conversation

@goutamadwant

Copy link
Copy Markdown

What's Changed

Fixes BaseVariableWidthViewVector.handleSafe so setSafe reserves a full 16-byte view slot for the target index even when the value length is zero.

Adds coverage for empty values written through the first view-buffer boundary for both ViewVarCharVector and ViewVarBinaryVector.

Tests

  • mvn -pl vector -am -Dtest=TestVariableWidthViewVector#testSetSafeEmptyValueAtViewBufferBoundary -Dsurefire.failIfNoSpecifiedTests=false test
  • mvn -pl vector -am -Dtest=TestVariableWidthViewVector -Dsurefire.failIfNoSpecifiedTests=false test

Closes #1190.

@github-actions

This comment has been minimized.

@mbutrovich mbutrovich left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @goutamadwant, this looks like what I was going to do! One coverage gap worth closing: handleSafe(index, 0) is also hit by setNull (1195) and fillEmpties via setValueCount (1022), so the bug was reachable without ever writing an empty string. The fix covers those too, but the test only exercises setSafe(i, new byte[0]). Suggest adding the two tests below. Non-blocking.

protected final void handleSafe(int index, int dataLength) {
final long targetCapacity = roundUpToMultipleOf16((long) index * ELEMENT_SIZE + dataLength);
// The view buffer stores one fixed-width view per value; payload bytes are allocated separately.
final long targetCapacity = roundUpToMultipleOf16(((long) index + 1) * ELEMENT_SIZE);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. Reserves the full slot regardless of payload; ((long) index + 1) avoids overflow. roundUpToMultipleOf16 is a no-op here but fine to keep.


@ParameterizedTest
@MethodSource({"vectorCreatorProvider"})
public void testSetSafeEmptyValueAtViewBufferBoundary(

@mbutrovich mbutrovich Jul 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only covers the empty-value path. setNull and fillEmpties/setValueCount reach the same handleSafe(index, 0). Suggest adding tests for those.

setValueCount trailing gap (silent trigger, no empty string written)

@ParameterizedTest
@MethodSource({"vectorCreatorProvider"})
public void testSetValueCountFillsEmptiesAtViewBufferBoundary(
    Function<BufferAllocator, BaseVariableWidthViewVector> vectorCreator) {
  try (final BaseVariableWidthViewVector vector = vectorCreator.apply(allocator)) {
    vector.allocateNew();
    final int valueCapacity = vector.getValueCapacity();
    vector.setSafe(valueCapacity - 1, "x".getBytes(StandardCharsets.UTF_8));
    // Leaves index valueCapacity unset: fillEmpties -> handleSafe(_, 0) on a full buffer.
    vector.setValueCount(valueCapacity + 1);
    assertTrue(vector.getValueCapacity() > valueCapacity);
    assertTrue(vector.isNull(valueCapacity));
  }
}

setNull at the boundary

@ParameterizedTest
@MethodSource({"vectorCreatorProvider"})
public void testSetNullAtViewBufferBoundary(
    Function<BufferAllocator, BaseVariableWidthViewVector> vectorCreator) {
  try (final BaseVariableWidthViewVector vector = vectorCreator.apply(allocator)) {
    vector.allocateNew();
    final int valueCapacity = vector.getValueCapacity();
    for (int i = 0; i <= valueCapacity; i++) {
      vector.setNull(i);
    }
    vector.setValueCount(valueCapacity + 1);
    assertTrue(vector.getValueCapacity() > valueCapacity);
    assertTrue(vector.isNull(valueCapacity));
  }
}

@lidavidm lidavidm added the bug-fix PRs that fix a big. label Jul 8, 2026
@github-actions github-actions Bot added this to the 20.0.0 milestone Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PRs that fix a big.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Java] BaseVariableWidthViewVector.handleSafe under-allocates the view buffer for empty (zero-length) values → IndexOutOfBoundsException

3 participants