Implement manual retention sweep - #5845
Open
rbev wants to merge 10 commits into
Open
Conversation
rbev
commented
Sep 2, 2026
rbev
requested review from
abparticular,
johnsimons and
warwickschroeder
and removed request for
johnsimons
September 2, 2026 07:25
rbev
commented
Sep 3, 2026
rbev
marked this pull request as ready for review
September 3, 2026 01:40
johnsimons
approved these changes
Sep 4, 2026
rbev
force-pushed
the
allow-manual-retention-sweep
branch
from
September 7, 2026 02:27
599d097 to
6f54751
Compare
| DateTime? lastFinishedAt; | ||
| DateTime? lastErrorCutoff; | ||
| DateTime? lastEventsCutoff; | ||
| string? lastError; |
Contributor
There was a problem hiding this comment.
These seem to be shared between the manual purge and the hourly sweep. Is this going to cause a problem? Someone polling their manual, might get wrong values due to the hourly running? Or should the hourly be disabled while a manual is running, and vice versa?
Contributor
Author
There was a problem hiding this comment.
They are, but the two have slightly different paths -
- auto sweep blocks if another (manual) one is running
- manual run takes the lock and immediately and kicks off a background task to do the work
Both paths take the semaphore before mutating the variables and finish editing them before releasing it, however the manual run sets the start values before kicking off the background work.
rbev
force-pushed
the
allow-manual-retention-sweep
branch
from
September 8, 2026 09:17
35151d8 to
aa9f545
Compare
johnsimons
approved these changes
Sep 9, 2026
johnsimons
left a comment
johnsimons
left a comment
Member
There was a problem hiding this comment.
just a couple of comments to consider
rbev
force-pushed
the
allow-manual-retention-sweep
branch
from
September 9, 2026 02:38
c578068 to
9c92da7
Compare
warwickschroeder
approved these changes
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a manual data retention sweep API for ServiceControl, enabling users to trigger and monitor retention sweeps via new endpoints. It includes new API contracts, permission management, and enhancements to the EFCore persistence layer to support manual and scheduled sweeps, including proper status reporting and concurrency control. Additionally, it ensures that unsupported operations on RavenDB are handled gracefully.
Manual Retention Sweep API and EFCore Support:
RetentionSweepRequest,RetentionSweepResponse, andRetentionSweepStatusto enable manual triggering and status polling for data retention sweeps (src/ServiceControl.Api/Contracts/RetentionSweepRequest.cs,RetentionSweepResponse.cs,RetentionSweepStatus.cs). [1] [2] [3]IRetentionApiinterface defining methods for starting a sweep and retrieving sweep status, with support for indicating when the operation is not supported (e.g., on RavenDB) (src/ServiceControl.Api/IRetentionApi.cs).RetentionSweeperto support manual sweeps with caller-supplied cutoffs, concurrency control to prevent overlapping sweeps, and detailed status reporting for the new API (src/ServiceControl.Persistence.EFCore/Infrastructure/RetentionSweeper.cs,BasePersistence.cs). [1] [2] [3] [4] [5] [6] [7]API Permissions and Roles:
ErrorRetentionSweepand included it in the admin role permissions to control access to the manual retention sweep API (src/ServiceControl.Infrastructure/Auth/Permissions.cs,RolePermissions.cs). [1] [2]