[fix](be) Propagate CCR delete bitmap calculation errors - #67143
Open
Ryan19929 wants to merge 2 commits into
Open
[fix](be) Propagate CCR delete bitmap calculation errors#67143Ryan19929 wants to merge 2 commits into
Ryan19929 wants to merge 2 commits into
Conversation
### What problem does this PR solve? Issue Number: None Related PR: None Problem Summary: Shared-Nothing IngestBinlog ignored the status returned by commit-phase delete bitmap task submission and by the asynchronous task token. A failed MOW delete bitmap calculation could therefore continue to commit the rowset without the required bitmap and expose duplicate unique keys. Propagate both statuses, wait for already-submitted tasks before aborting, and return the failure before commit. Capture downloaded files by reference so the existing failure cleanup removes them. Add a CCR MOW fault-injection regression case that verifies the failed version remains invisible and can be retried safely. ### Release note Fix CCR replay for MOW tables so delete bitmap calculation failures abort ingest instead of committing an incomplete delete bitmap. ### Check List (For Author) - Test: Regression test added; Groovy syntax parsing passed. The modified BE object compiled with toolchain 0.25, and clang-tidy passed. The full two-cluster CCR regression case was not run because no patched master source/target cluster was available. - Behavior changed: Yes. IngestBinlog now returns delete bitmap calculation failures and aborts the local tablet transaction. - Does this need documentation: No
### What problem does this PR solve? Issue Number: close apache#67120 Related PR: None Problem Summary: CCR IngestBinlog failure cleanup discarded the status returned by batch_delete and logged cleanup as completed even when file deletion failed. Check the cleanup status and report the secondary cleanup failure without replacing the original ingest error. ### Release note None ### Check List (For Author) - Test: Manual test - build-support/check-format.sh - Compiled backend_service.cpp with toolchain 0.25 - Changed-lines clang-tidy passed - Behavior changed: Yes. Failed cleanup is now reported instead of being logged as successful. - Does this need documentation: No
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
|
Reviewed the current head No blocking issue was found. LGTM from the static review perspective. Key conclusions:
Validation performed in this review:
|
16 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #67120
Related PR: None
Problem Summary:
In Shared-Nothing CCR incremental replication,
IngestBinlogdiscarded both the status returned byBaseTablet::commit_phase_update_delete_bitmap()and the asynchronous delete bitmap token'swait()status.For UNIQUE KEY merge-on-write tablets, a delete bitmap calculation failure could therefore be logged but not propagated. The downloaded rowset could still be committed and published with an incomplete delete bitmap, exposing historical rows and producing duplicate unique keys.
The failure cleanup also captured
download_success_filesby value before downloads occurred, so downloaded segment and index files were not removed. Additionally, the result ofbatch_delete()was discarded and cleanup could be logged as successful even when deletion failed.This PR:
commit_txn()so the prepared tablet transaction is aborted;batch_delete()failures without replacing the original ingest error;Release note
Fix CCR replay for MOW tables so delete bitmap calculation failures abort
IngestBinloginstead of publishing a rowset with an incomplete delete bitmap.Check List (For Author)
test_mow_ingest_binlog, including the new asynchronous delete bitmap worker failure case, passed.BaseTablet::calc_segment_delete_bitmap.inject_erron the target BE.IngestBinlogreturns an error from the delete bitmap failure branch before reaching the rowset commit step, the failed version remains invisible, no duplicate key becomes visible, and retrying the same binlog succeeds._ingest_binlogpaths passed.build-support/clang-format.shandbuild-support/check-format.shpassed with clang-format 16.Known test coverage limitations:
commit_phase_update_delete_bitmap().batch_delete()failure was not injected, so the cleanup failure logging branch is not covered by a runtime test.Known environment-only failures unrelated to this change:
Two
test_create_table_with_binlog_configcases conflict with the Docker image's forcedforce_enable_feature_binlogsetting.test_is_being_syncedrequires an unavailable external S3 bucket.clang-tidy 20 reports a pre-existing orphan
NOLINTENDattypes.h:576; no warning was reported on the lines changed by this PR.Behavior changed:
IngestBinlognow returns delete bitmap calculation errors and aborts before committing the rowset. Cleanup failures are also reported accurately.Does this need documentation?
Check List (For Reviewer who merge this PR)