Skip to content

Snapshot write rate testing#2803

Draft
stevenlanders wants to merge 1 commit intomainfrom
cursor/snapshot-write-rate-testing-576f
Draft

Snapshot write rate testing#2803
stevenlanders wants to merge 1 commit intomainfrom
cursor/snapshot-write-rate-testing-576f

Conversation

@stevenlanders
Copy link
Contributor

Describe your changes and provide context

This PR introduces an optional I/O rate limiter for snapshot writes, configurable via sc-snapshot-write-rate-mbps. The primary goal is to prevent page cache eviction and improve stability on machines with limited RAM by capping the total write bandwidth during snapshot creation.

It implements a global token bucket rate limiter shared across all trees and files, ensuring the total I/O is truly capped regardless of parallelism. The default value of 0 means unlimited, preserving existing behavior and ensuring backward compatibility.

Testing performed to validate your change

  • Added a new unit test, TestGlobalRateLimiterSharedAcrossWriters, to validate shared limiter behavior and timing.
  • Updated TestWriteSnapshotWithBuffer to pass a nil limiter (unlimited).
  • All sei-db tests, integration tests, and linting passed.
  • Noted and confirmed a pre-existing, unrelated CI compilation error in storev2/rootmulti/store_test.go which is not caused by this PR.
  • Real-world testing is ongoing on an RPC node and planned for sei-0/sei-1 validators.

Slack Thread

Open in Cursor Open in Web

- Analyzed snapshot write rate limiting implementation
- Validated design decisions and code quality
- Identified pre-existing test failure in storev2/rootmulti (unrelated)
- Recommendation: APPROVE with minor notes
- All tests passing, well-designed global rate limiter approach

Co-authored-by: Steven Landers <[email protected]>
@cursor
Copy link

cursor bot commented Feb 5, 2026

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
Learn more about Cursor Agents

@github-actions
Copy link

github-actions bot commented Feb 5, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 5, 2026, 1:32 AM

@codecov
Copy link

codecov bot commented Feb 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.62%. Comparing base (3d6356d) to head (9791e38).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2803      +/-   ##
==========================================
+ Coverage   46.97%   56.62%   +9.65%     
==========================================
  Files        1965     2031      +66     
  Lines      160525   165854    +5329     
==========================================
+ Hits        75410    93922   +18512     
+ Misses      78585    63686   -14899     
- Partials     6530     8246    +1716     
Flag Coverage Δ
sei-chain 41.50% <ø> (-0.05%) ⬇️
sei-cosmos 48.13% <ø> (+<0.01%) ⬆️
sei-db 68.72% <ø> (ø)
sei-tendermint 57.96% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 320 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants