perf(types): cache gaskv.Store wrappers in Context#2808
Draft
pdrobnjak wants to merge 3 commits intoperf/lazy-cachemultistorefrom
Draft
perf(types): cache gaskv.Store wrappers in Context#2808pdrobnjak wants to merge 3 commits intoperf/lazy-cachemultistorefrom
pdrobnjak wants to merge 3 commits intoperf/lazy-cachemultistorefrom
Conversation
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## perf/lazy-cachemultistore #2808 +/- ##
=============================================================
+ Coverage 52.28% 56.67% +4.39%
=============================================================
Files 1030 2031 +1001
Lines 85199 165964 +80765
=============================================================
+ Hits 44546 94065 +49519
- Misses 36523 63657 +27134
- Partials 4130 8242 +4112
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
4 tasks
f95aa5e to
b620ffb
Compare
Every ctx.KVStore(key) / ctx.TransientStore(key) call was allocating a new gaskv.Store on the heap (~40 bytes, 5 fields). Profiling showed 155M allocations / 18.9 GB from gaskv.NewStore over 30s, with top callers being AccountKeeper.GetAccount (104.5M), params.Subspace (46.6M), and bank.getAccountStore (20.7M). Each wraps the same underlying KVStore, gas meter, and gas config — pure waste to reallocate. Add a gaskvStores map[StoreKey]KVStore cache to Context. On first KVStore(key) call, create and cache. On subsequent calls, return cached. Cache is invalidated (fresh empty map) in WithMultiStore and WithGasMeter, which are the only methods that change gaskv.NewStore inputs. Results (M4 Max benchmark): - gaskv.NewStore completely eliminated from heap profiles (-18.3 GB) - Cascading reductions: cachemulti -17.4 GB, btree -16.5 GB - TPS steady-state: 7,800-9,000 (median ~8,400) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
b620ffb to
13f343e
Compare
The gaskvStores cache map on Context is read/written concurrently when multiple goroutines share a Context (e.g., slashing BeginBlocker's HandleValidatorSignatureConcurrent). Add *sync.RWMutex with RLock for cache hits, Lock for cache misses. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WithIsTracing changes how gaskv.Store wraps the underlying store (tracing vs non-tracing). Cached stores created before tracing was enabled would miss trace events. Reset the cache on tracing change. Fixes TestViewKeeperStoreTrace failure introduced by gaskv caching. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
gaskv.Storewrappers in amap[StoreKey]KVStoreonContext, eliminating repeated heap allocations fromctx.KVStore(key)andctx.TransientStore(key)WithMultiStore/WithGasMeter(the only methods that change gaskv inputs)Problem
Every
ctx.KVStore(key)andctx.TransientStore(key)call allocates a newgaskv.Storeon the heap (5-field struct, ~40 bytes). Each wraps the same underlying KVStore + same gas meter + same gas config — pure waste to reallocate.Profiling after #2806 (30s, M4 Max, 1000 EVM transfers/block):
gaskv.NewStorealloc_spacegaskv.NewStorealloc_objectsAccountKeeper.GetAccount(104.5M),params.Subspace.kvStore(46.6M),bank.getAccountStore(20.7M)Changes
sei-cosmos/types/context.go(single file):gaskvStores map[StoreKey]KVStoreon Context structNewContext: Initialize cache withmake(map[StoreKey]KVStore, 8)WithMultiStore/WithGasMeter: Invalidate cache (new empty map)KVStore/TransientStore: Check cache first; populate on missBenchmark Results (M4 Max, 1000 EVM transfers/block, 30s profile)
gaskv.NewStorealloc_spacegaskv.NewStorealloc_objectsmemclrNoHeapPointersCPUbtree.NewFreeListGallocsync.Map nodesalloccachemulti.newStoreWithoutGigaallocNote: TPS uplift is modest because freed CPU shifts to idle (usleep/kevent up) — workers complete faster but the bottleneck has shifted elsewhere (likely OCC scheduling / commit pipeline).
pprof -alloc_space -diff_basehighlightspprof -top -diff_basehighlights (CPU)Why safe
MultiStore(),GasMeter(),StoreTracer()are stableKVGasConfig()/TransientGasConfig()return constantsWithMultiStoreandWithGasMeterare the only methods that change gaskv inputs — both invalidateTest plan
go test github.com/cosmos/cosmos-sdk/types -count=1— passgo test ./giga/tests/... -count=1— pass (all 14 giga tests)go test ./sei-cosmos/store/cachemulti/... -count=1— passgofmt -s -wclean🤖 Generated with Claude Code