Skip to content

Subscription: improve WAL replay gap observability - #18377

Open
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix/consensus-subscription-wal-gap-logging
Open

Subscription: improve WAL replay gap observability#18377
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix/consensus-subscription-wal-gap-logging

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Aggregate unreadable retained WAL logs

Downgrade per-file WAL open failures to DEBUG and emit one WARN summary when a replay scan is exhausted. The summary includes the WAL directory, skipped file count, first/last file, and the first concise error without repeating a stack trace for every file.

Report unavailable replay ranges

Detect discontinuities between the expected subscription search index and the next retained WAL entry. Record the skipped entry count only when the replay cursor advances, log the unavailable range and cumulative count, and include walGapSkippedEntries in periodic queue statistics.

This makes bounded WAL-retention loss visible even when the current lag later returns to zero.

Verification

  • ProgressWALIteratorTest: 8 tests passed
  • ConsensusPrefetchingQueueTest#testWalReplayCountsOnlyUnavailableSearchIndexes: passed
  • English DataNode main and test sources compiled during the focused test run
  • Chinese locale DataNode main sources compiled with -P with-zh-locale
  • Spotless and Checkstyle passed

This PR has:

  • been self-reviewed.
  • added unit tests or modified existing tests to cover new code paths.
  • added comments explaining intent where needed.

Key changed/added classes
  • ProgressWALIterator
  • ConsensusPrefetchingQueue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant