GH-1190: Reserve view slots for empty view vector values#1192
GH-1190: Reserve view slots for empty view vector values#1192goutamadwant wants to merge 1 commit into
Conversation
This comment has been minimized.
This comment has been minimized.
mbutrovich
left a comment
There was a problem hiding this comment.
Thanks @goutamadwant, this looks like what I was going to do! One coverage gap worth closing: handleSafe(index, 0) is also hit by setNull (1195) and fillEmpties via setValueCount (1022), so the bug was reachable without ever writing an empty string. The fix covers those too, but the test only exercises setSafe(i, new byte[0]). Suggest adding the two tests below. Non-blocking.
| protected final void handleSafe(int index, int dataLength) { | ||
| final long targetCapacity = roundUpToMultipleOf16((long) index * ELEMENT_SIZE + dataLength); | ||
| // The view buffer stores one fixed-width view per value; payload bytes are allocated separately. | ||
| final long targetCapacity = roundUpToMultipleOf16(((long) index + 1) * ELEMENT_SIZE); |
There was a problem hiding this comment.
Correct. Reserves the full slot regardless of payload; ((long) index + 1) avoids overflow. roundUpToMultipleOf16 is a no-op here but fine to keep.
|
|
||
| @ParameterizedTest | ||
| @MethodSource({"vectorCreatorProvider"}) | ||
| public void testSetSafeEmptyValueAtViewBufferBoundary( |
There was a problem hiding this comment.
Only covers the empty-value path. setNull and fillEmpties/setValueCount reach the same handleSafe(index, 0). Suggest adding tests for those.
setValueCount trailing gap (silent trigger, no empty string written)
@ParameterizedTest
@MethodSource({"vectorCreatorProvider"})
public void testSetValueCountFillsEmptiesAtViewBufferBoundary(
Function<BufferAllocator, BaseVariableWidthViewVector> vectorCreator) {
try (final BaseVariableWidthViewVector vector = vectorCreator.apply(allocator)) {
vector.allocateNew();
final int valueCapacity = vector.getValueCapacity();
vector.setSafe(valueCapacity - 1, "x".getBytes(StandardCharsets.UTF_8));
// Leaves index valueCapacity unset: fillEmpties -> handleSafe(_, 0) on a full buffer.
vector.setValueCount(valueCapacity + 1);
assertTrue(vector.getValueCapacity() > valueCapacity);
assertTrue(vector.isNull(valueCapacity));
}
}setNull at the boundary
@ParameterizedTest
@MethodSource({"vectorCreatorProvider"})
public void testSetNullAtViewBufferBoundary(
Function<BufferAllocator, BaseVariableWidthViewVector> vectorCreator) {
try (final BaseVariableWidthViewVector vector = vectorCreator.apply(allocator)) {
vector.allocateNew();
final int valueCapacity = vector.getValueCapacity();
for (int i = 0; i <= valueCapacity; i++) {
vector.setNull(i);
}
vector.setValueCount(valueCapacity + 1);
assertTrue(vector.getValueCapacity() > valueCapacity);
assertTrue(vector.isNull(valueCapacity));
}
}
What's Changed
Fixes
BaseVariableWidthViewVector.handleSafesosetSafereserves a full 16-byte view slot for the target index even when the value length is zero.Adds coverage for empty values written through the first view-buffer boundary for both
ViewVarCharVectorandViewVarBinaryVector.Tests
mvn -pl vector -am -Dtest=TestVariableWidthViewVector#testSetSafeEmptyValueAtViewBufferBoundary -Dsurefire.failIfNoSpecifiedTests=false testmvn -pl vector -am -Dtest=TestVariableWidthViewVector -Dsurefire.failIfNoSpecifiedTests=false testCloses #1190.