Repository navigation
[C++] Keep scanning after a fully filtered aligned page (#1038) - #1044
Merged
Merged
Conversation
The single-column aligned reader turned a successful decode that produced no rows into E_NO_MORE_DATA. The series scan iterator reads that as "chunk exhausted": it jumps to the next chunk, or ends the series when the current chunk is the last one, so matching pages after a fully filtered page were never read. A TimeIn query over a table with a missing measurement could therefore silently drop rows and still report E_OK. An empty result only means every row of the page was filtered out, so return E_OK and let the iterator's existing empty-block handling continue within the chunk. True end of stream is still signalled by has_more_data(). Add a table-level regression test that reproduces the reported layout (three devices, three chunks, page size 5, a never-written measurement and a missing column on one device) and covers empty pages with and without a successor chunk, plus all-matching and no-matching controls.
TsFile_Test is compiled with -fprofile-arcs/-ftest-coverage, but #1016 changed libtsfile's coverage runtime dependency to PRIVATE, so it no longer propagated to consumers. The Release LTO link then failed with undefined __gcov_init/__gcov_exit/__gcov_merge_add, which has kept the codecov job red since 2026-10-16. Link the coverage runtime into TsFile_Test itself.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1044 +/- ##
===========================================
+ Coverage 65.26% 66.46% +1.20%
===========================================
Files 761 773 +12
Lines 53610 55796 +2186
Branches 8547 8945 +398
===========================================
+ Hits 34986 37086 +2100
- Misses 16978 17055 +77
- Partials 1646 1655 +9 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fixes #1038.
A C++ aligned table query using
TimeIncould silently drop matching rows that follow a page whose rows were all rejected by the filter. The API still returnedE_OKand reported a normal end of results.Root cause
AlignedChunkReader::decode_tv_buf_into_tsblock_by_datatypeconverted a successful decode that produced no rows intoE_NO_MORE_DATA:That conflates "this page had no matching rows" with "the data stream is exhausted".
TsFileSeriesScanIterator::get_nextreacts toE_NO_MORE_DATAby advancing to the next chunk, or by ending the series when the current chunk is the last one — so the remaining pages of the chunk were never read. The iterator already has the correct handling for an empty successful block; the rewrite made it unreachable on this path.The multi-value aligned path and the offset-aware non-aligned path do not have this rewrite, which is why the bug only surfaced on the per-column aligned read path (e.g. when a requested measurement is never written and the query falls back to one SSI per column).
Fix
Return
E_OKwith zero rows for a fully filtered page and let the series iterator continue within the chunk. True end of stream is still signalled byhas_more_data().Regression test
TimeInContinuesAfterFullyFilteredAlignedPageincpp/test/reader/table_view/tsfile_reader_table_test.ccreproduces the reported layout: three devices, three chunks of seven rows with an explicit flush,page_writer_max_point_num_ = 5, aghostmeasurement that is never written,vmissing entirely on one device, sparse NULLs inv/w, serial and parallel read, and a 4096-byte internal TsBlock budget. It covers:TimeIn({0, 2, 3, 6, 7, 10, 13, 19, 20})query, which lost timestamps 19 and 20 per device before the fix;Verification
after scan ret=0 count=27 expected=27and exits 0.This is separate from the small-capacity hang tracked in #1035.