go bench read + write receipts/logs for parquet vs pebble#2794
Merged
go bench read + write receipts/logs for parquet vs pebble#2794
Conversation
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2794 +/- ##
===========================================
+ Coverage 58.14% 67.71% +9.57%
===========================================
Files 2106 45 -2061
Lines 173289 6953 -166336
===========================================
- Hits 100751 4708 -96043
+ Misses 63596 1694 -61902
+ Partials 8942 551 -8391
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
ebaad75 to
793dc6b
Compare
blindchaser
approved these changes
Feb 10, 2026
yzang2019
approved these changes
Feb 10, 2026
jewei1997
commented
Feb 23, 2026
| @@ -0,0 +1,126 @@ | |||
| package ethbloom | |||
Contributor
Author
There was a problem hiding this comment.
just a refactor, not new code added
cody-littley
approved these changes
Feb 23, 2026
Contributor
cody-littley
left a comment
There was a problem hiding this comment.
LGTM. Take my approval with a grain of salt, since I don't yet have a solid grasp of receipt store architecture.
My main comment is to make sure we do long running tests as well (not a blocker for this PR though).
yzang2019
pushed a commit
that referenced
this pull request
Feb 25, 2026
- Add benchmarks comparing pebble vs parquet receipt store read/write performance across realistic workloads (100 blocks with 10-100 receipts per block) - using 3000 receipts/block and 5k blocks for writes and 2k blocks for reads (block range is ~100-500 blocks) though. Writes - both pebble and parquet do sync writes to WAL + async writes to underlying store - receipt cache is not used in benchmark for more accurate comparison - command `go test -bench=BenchmarkReceiptWriteAsync -benchtime=1x -run=^$ ./sei-db/ledger_db/receipt/` Reads - pebble iterates over each receipt (still uses bloom filters) while parquet issues a SQL-like filter query - when hitting ledger cache both are fast as expected but outside of ledger cache, pebble super slow due to need to iterate over each receipt - tested over varying concurrency, block range, diff filter criteria (topics, address) - command `go test -run='^$' -bench 'BenchmarkReceiptReadStore100' -benchtime=1x -count=1 -timeout=30m ./sei-db/ledger_db/receipt/ -v` Parquet faster for writes and reads 👍 Writes: On detached pacific-1 RPC node (EC2 instance with EBS), getting 347,112.00 receipts/s writes using parquet, but only "only" 143,000 receipts/s using pebble Reads: Not even comparable. Parquet way faster. 20ms per log for pebble vs 0.05918 ms per log for parquet
yzang2019
added a commit
that referenced
this pull request
Feb 27, 2026
* main: (66 commits) feat(flatkv): include legacyDB in ApplyChangeSets, LtHash, and read path (#2978) Deflake mempool tests with Eventually-based block waits (#2983) Demote noisy gasless classification log to debug level (#2982) Harden `TestStateLock_NoPOL` against proposal/timeout race (#2980) added a config parameter to limit outbound p2p connections. (#2974) merged unconditional and persistent peers status (#2977) Fix race between file pruning and in-flight parquet queries (#2975) fix(giga): don't migrate balance on failed txs (#2961) Fix hanging upgrade tests by adding timeouts to wait_for_height (#2976) Add snapshot import for Giga Live State (#2970) Fix Rocksdb MVCC read timestamp lifetime for iterators (#2971) Reduce exposed tendermint RPC endpoint (#2968) Deflake `TestStateLock_NoPOL` by widening propose timeout in test (#2969) go bench read + write receipts/logs for parquet vs pebble (#2794) [giga] clear up cache after Write (#2827) fix: use correct EVM storage key prefix in benchmark key generation (#2966) Harden staking precompile test against CI flakiness (#2967) Don't sync flatKV DBs when committing (#2964) Fix flaky `TestStateLock_POLSafety1` (#2962) Add metrics for historical proof success/failure rate (#2958) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Writes
go test -bench=BenchmarkReceiptWriteAsync -benchtime=1x -run=^$ ./sei-db/ledger_db/receipt/Reads
go test -run='^$' -bench 'BenchmarkReceiptReadStore100' -benchtime=1x -count=1 -timeout=30m ./sei-db/ledger_db/receipt/ -vBenchmark Results
Parquet faster for writes and reads 👍
Writes:
On detached pacific-1 RPC node (EC2 instance with EBS), getting 347,112.00 receipts/s writes using parquet, but only "only" 143,000 receipts/s using pebble
Reads:
Not even comparable. Parquet way faster. 20ms per log for pebble vs 0.05918 ms per log for parquet