Skip to content

perf(types): cache gaskv.Store wrappers in Context#2808

Draft
pdrobnjak wants to merge 3 commits intoperf/lazy-cachemultistorefrom
perf/cache-gaskv-context
Draft

perf(types): cache gaskv.Store wrappers in Context#2808
pdrobnjak wants to merge 3 commits intoperf/lazy-cachemultistorefrom
perf/cache-gaskv-context

Conversation

@pdrobnjak
Copy link
Contributor

@pdrobnjak pdrobnjak commented Feb 5, 2026

Summary

  • Cache gaskv.Store wrappers in a map[StoreKey]KVStore on Context, eliminating repeated heap allocations from ctx.KVStore(key) and ctx.TransientStore(key)
  • Cache invalidated on WithMultiStore / WithGasMeter (the only methods that change gaskv inputs)

Problem

Every ctx.KVStore(key) and ctx.TransientStore(key) call allocates a new gaskv.Store on the heap (5-field struct, ~40 bytes). Each wraps the same underlying KVStore + same gas meter + same gas config — pure waste to reallocate.

Profiling after #2806 (30s, M4 Max, 1000 EVM transfers/block):

Metric Value
gaskv.NewStore alloc_space 18,892 MB (#9 top allocator)
gaskv.NewStore alloc_objects ~143M objects
Top callers AccountKeeper.GetAccount (104.5M), params.Subspace.kvStore (46.6M), bank.getAccountStore (20.7M)

Changes

sei-cosmos/types/context.go (single file):

  1. New field: gaskvStores map[StoreKey]KVStore on Context struct
  2. NewContext: Initialize cache with make(map[StoreKey]KVStore, 8)
  3. WithMultiStore / WithGasMeter: Invalidate cache (new empty map)
  4. KVStore / TransientStore: Check cache first; populate on miss

Benchmark Results (M4 Max, 1000 EVM transfers/block, 30s profile)

Metric Before (after #2806) After Delta
TPS (steady-state range) 8,000–8,800 7,800–9,000 median ~8,400
gaskv.NewStore alloc_space 18,892 MB 0 MB -18,892 MB (eliminated)
gaskv.NewStore alloc_objects ~143M 0 eliminated from top 200
memclrNoHeapPointers CPU 6.96s 2.11s -4.85s
btree.NewFreeListG alloc 36,662 MB 20,943 MB -15,719 MB
sync.Map nodes alloc 33,216 MB 19,103 MB -14,113 MB
cachemulti.newStoreWithoutGiga alloc 39,081 MB 22,471 MB -16,610 MB
Total alloc_space (30s) 811,276 MB 457,143 MB -354,133 MB (-44%)

Note: TPS uplift is modest because freed CPU shifts to idle (usleep/kevent up) — workers complete faster but the bottleneck has shifted elsewhere (likely OCC scheduling / commit pipeline).

pprof -alloc_space -diff_base highlights

-18,298 MB  gaskv.NewStore               (direct savings from this PR)
-17,416 MB  cachemulti.newStoreWithoutGiga (cascading — fewer CMS recreations)
-16,483 MB  btree.NewFreeListG            (cascading — fewer cachekv stores)
-14,798 MB  sync.newIndirectNode          (cascading — fewer sync.Map allocs)
-11,050 MB  state.(*DBImpl).Snapshot      (cascading — less snapshot overhead)
-10,969 MB  sync.newEntryNode             (cascading — fewer sync.Map entries)
 -9,964 MB  vm.NewEVM                     (cascading — less GC pressure)
 -9,241 MB  multiversion.NewVersionIndexedStore

pprof -top -diff_base highlights (CPU)

 -4.85s  runtime.memclrNoHeapPointers (fewer allocs to zero)
 -2.37s  syscall.syscall              (less CGO overhead)
 -1.71s  syscall.syscall6
 -1.46s  runtime.heapBitsSmallForAddr (less heap bookkeeping)
 +3.19s  runtime.usleep               (workers idle more — finished faster)
 +1.61s  runtime.kevent               (same — less work to do)

Why safe

  • Within a single Context, MultiStore(), GasMeter(), StoreTracer() are stable
  • KVGasConfig() / TransientGasConfig() return constants
  • WithMultiStore and WithGasMeter are the only methods that change gaskv inputs — both invalidate
  • Map is reference type: copies share cache, but invalidation allocates a fresh map
  • Thread safety: each OCC worker has its own Context

Test plan

  • go test github.com/cosmos/cosmos-sdk/types -count=1 — pass
  • go test ./giga/tests/... -count=1 — pass (all 14 giga tests)
  • go test ./sei-cosmos/store/cachemulti/... -count=1 — pass
  • gofmt -s -w clean
  • Benchmark 2000+ blocks with heap profiling

🤖 Generated with Claude Code

@github-actions
Copy link

github-actions bot commented Feb 5, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 6, 2026, 6:42 PM

@codecov
Copy link

codecov bot commented Feb 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.67%. Comparing base (d52d4a1) to head (de7b59d).

Additional details and impacted files

Impacted file tree graph

@@                      Coverage Diff                      @@
##           perf/lazy-cachemultistore    #2808      +/-   ##
=============================================================
+ Coverage                      52.28%   56.67%   +4.39%     
=============================================================
  Files                           1030     2031    +1001     
  Lines                          85199   165964   +80765     
=============================================================
+ Hits                           44546    94065   +49519     
- Misses                         36523    63657   +27134     
- Partials                        4130     8242    +4112     
Flag Coverage Δ
sei-chain 41.53% <100.00%> (?)
sei-cosmos 48.15% <100.00%> (+0.02%) ⬆️
sei-db 68.72% <ø> (ø)
sei-tendermint 58.08% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-cosmos/types/context.go 94.48% <100.00%> (+19.19%) ⬆️

... and 1142 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Every ctx.KVStore(key) / ctx.TransientStore(key) call was allocating a
new gaskv.Store on the heap (~40 bytes, 5 fields). Profiling showed
155M allocations / 18.9 GB from gaskv.NewStore over 30s, with top
callers being AccountKeeper.GetAccount (104.5M), params.Subspace (46.6M),
and bank.getAccountStore (20.7M). Each wraps the same underlying KVStore,
gas meter, and gas config — pure waste to reallocate.

Add a gaskvStores map[StoreKey]KVStore cache to Context. On first
KVStore(key) call, create and cache. On subsequent calls, return cached.
Cache is invalidated (fresh empty map) in WithMultiStore and WithGasMeter,
which are the only methods that change gaskv.NewStore inputs.

Results (M4 Max benchmark):
- gaskv.NewStore completely eliminated from heap profiles (-18.3 GB)
- Cascading reductions: cachemulti -17.4 GB, btree -16.5 GB
- TPS steady-state: 7,800-9,000 (median ~8,400)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@pdrobnjak pdrobnjak force-pushed the perf/cache-gaskv-context branch from b620ffb to 13f343e Compare February 6, 2026 17:19
pdrobnjak and others added 2 commits February 6, 2026 18:43
The gaskvStores cache map on Context is read/written concurrently
when multiple goroutines share a Context (e.g., slashing BeginBlocker's
HandleValidatorSignatureConcurrent). Add *sync.RWMutex with RLock for
cache hits, Lock for cache misses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WithIsTracing changes how gaskv.Store wraps the underlying store
(tracing vs non-tracing). Cached stores created before tracing was
enabled would miss trace events. Reset the cache on tracing change.

Fixes TestViewKeeperStoreTrace failure introduced by gaskv caching.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant