Fix O_DIRECT expected to read number of bytes numpy reader

[JanuszL]

• Fixes calculation of the amount of data that numpy reader
  should read in O_DIRECT mode
• Adds test for O_DIRECT with 4096-byte aligned numpy file headers
  that this change should fix

Category:

Other (e.g. Documentation, Tests, Configuration)

Description:

• Fixes calculation of the amount of data that numpy reader
  should read in O_DIRECT mode
• Adds test for O_DIRECT with 4096-byte aligned numpy file headers
  that this change should fix
• Related to CPU numpy reader assert failed when numpy header size equals to O_DIRECT alignment length #6147

Additional information:

Affected modules and functionalities:

• numpy reader

Key points relevant for the review:

• NA

Tests:

• Existing tests apply
• New tests added
  • [s] Python tests
    • test_numpy.test_o_direct_alignment
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: N/A

[JanuszL]

!build

[greptile-apps]

Greptile Summary

Fixed critical bug in numpy reader's O_DIRECT mode where read_tail was calculated using absolute file offset (target->data_offset) instead of relative block offset (target_data_offset). This caused incorrect validation of read operations when numpy file headers were aligned to 4096-byte boundaries.

Key Changes:

• Changed read_tail calculation from alignment_offset(target->data_offset + target->nbytes, o_direct_chunk_size_) to alignment_offset(target_data_offset + target->nbytes, o_direct_chunk_size_) on line 299 of numpy_reader_op.cc
• Added test case test_o_direct_alignment that creates a numpy file with exactly 4096-byte aligned header to verify the fix works correctly

Confidence Score: 5/5

• This PR is safe to merge - it's a targeted one-line fix with comprehensive test coverage
• The change is a minimal, surgical fix to a clear bug. The incorrect variable was used in the calculation, causing wrong expectations for O_DIRECT read operations. The fix uses the correct relative offset variable that was already computed earlier in the code. The new test specifically validates this edge case with 4096-byte aligned headers.
• No files require special attention

Important Files Changed

Filename	Overview
dali/operators/reader/numpy_reader_op.cc	Fixes read_tail calculation to use relative offset instead of absolute file offset for O_DIRECT mode reads
dali/test/python/reader/test_numpy.py	Adds comprehensive test for O_DIRECT with 4096-byte aligned numpy headers that previously triggered the bug

Sequence Diagram

sequenceDiagram
    participant Client as Numpy Reader
    participant File as ODirectFileStream
    participant Memory as Aligned Memory Buffer
    
    Note over Client: Prefetch() called
    Client->>Client: Calculate block_start = align_down(data_offset, alignment)
    Client->>Client: Calculate block_end = align_up(data_offset + nbytes, alignment)
    Client->>Client: Calculate aligned_len
    Client->>Client: Calculate target_data_offset = alignment_offset(data_offset, alignment)
    
    Client->>Memory: Allocate aligned memory (aligned_len bytes)
    
    Note over Client: OLD BUG: read_tail = alignment_offset(data_offset + nbytes, chunk_size)
    Note over Client: FIX: read_tail = alignment_offset(target_data_offset + nbytes, chunk_size)
    
    loop For each chunk in aligned_len
        Client->>Client: Calculate read_size = min(chunk_size, remaining)
        Client->>Client: Calculate target_mem pointer
        Client->>Client: Calculate file_offset
        Client->>File: ReadAt(target_mem, read_size, file_offset)
        File-->>Client: Returns bytes read (ret)
        Client->>Client: Assert ret >= read_tail (validates sufficient bytes read)
    end
    
    Client->>Client: ShareData with target using target_data_offset

Loading

[dali-automaton]

CI MESSAGE: [40716397]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [40716397]: BUILD FAILED

[dali-automaton]

CI MESSAGE: [40716397]: BUILD PASSED