Skip to content

Comments

Add leader election for FileStorage cleanup#26

Merged
mcollina merged 2 commits intomainfrom
file-storage-leader-election
Feb 18, 2026
Merged

Add leader election for FileStorage cleanup#26
mcollina merged 2 commits intomainfrom
file-storage-leader-election

Conversation

@mcollina
Copy link
Member

Summary

  • Implement acquireLeaderLock/renewLeaderLock/releaseLeaderLock on FileStorage using lock files (<basePath>/<lockKey>.lock) with JSON { ownerId, expiresAt }
  • When multiple FileStorage instances share the same basePath, only one instance (the leader) runs cleanup tasks (expired results, errors, workers)
  • Followers periodically try to acquire leadership; leaders periodically renew their lock
  • On disconnect, the leader releases the lock so another instance can take over immediately
  • Enables Reaper leader election to work with FileStorage (previously only RedisStorage supported it)

Test plan

  • Leader election methods: acquire/renew/release with same/different owners, expired locks
  • Cleanup leader behavior: only one of two instances runs cleanup; failover when leader disconnects
  • Full test suite passes (143 tests, 0 failures)
  • Lint and typecheck pass

🤖 Generated with Claude Code

mcollina and others added 2 commits February 18, 2026 15:59
When multiple FileStorage instances share the same basePath, only one
instance now runs cleanup (expired results/errors/workers). Implements
acquireLeaderLock/renewLeaderLock/releaseLeaderLock using lock files,
enabling Reaper leader election to work with FileStorage.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Verify that Reaper with leaderElection enabled works end-to-end with
FileStorage: leadership acquisition, stalled job recovery, multi-reaper
competition, failover on stop, lock expiry takeover, and lock renewal.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@mcollina mcollina force-pushed the file-storage-leader-election branch from e24120b to e6320f0 Compare February 18, 2026 15:10
@mcollina mcollina merged commit cdeccd2 into main Feb 18, 2026
1 check passed
@mcollina mcollina deleted the file-storage-leader-election branch February 18, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants