Skip to content

perf: preallocate length-prefixed decision cache keys - #1774

Closed
kamalyes wants to merge 1 commit into
apache:masterfrom
kamalyes:perf/cache-key-preallocation
Closed

kamalyes wants to merge 1 commit into
apache:masterfrom
kamalyes:perf/cache-key-preallocation

Conversation

@kamalyes

Copy link
Copy Markdown

GetCacheKey repeatedly grows a zero-capacity builder while writing type- and length-prefixed fields. Long field lengths also allocate through strconv.Itoa. This change reserves the exact encoded size for all-string requests and appends lengths through a stack buffer.

The existing key encoding, type namespaces, and single invocation of each custom CacheableParam.GetCacheKey() are preserved. Mixed arguments keep the generic path. There are no public API or dependency changes.

Validation

  • go vet ./...
  • go test -race ./... with this patch applied independently
  • Added regressions for empty requests, UTF-8 byte lengths, delimiters, NUL bytes, long fields, unsupported arguments, and custom key namespaces/call counts.

Engine-only benchmarks on Apple M4, macOS arm64, Go 1.26.2, using identical workloads on base c92cc6a and the candidate. Five samples, -benchtime=500ms -count=5 -cpu=10; figures are medians:

Workload Before ns/op After ns/op Before B/op After B/op Before allocs/op After allocs/op
Three-string route key 64.69 55.42 56 32 3 1
Four-string domain key 85.83 59.31 120 48 4 1
Key with 160-byte path 128.50 57.57 560 192 5 1
Warm SyncedCachedEnforcer decision 86.26 69.81 104 80 4 2

The warm decision-cache latency change is not statistically significant in this sample (p=0.151); its allocation reduction is the supported result. These microbenchmarks do not measure end-to-end gateway latency.

Reproduce with the added benchmarks on both revisions:

go test -run='^$' -bench='BenchmarkReview(CacheKey|SyncedCacheHit)' -benchmem -benchtime=500ms -count=5 -cpu=10 .

Reserve the encoded size for string-only requests and format lengths with a stack buffer. Preserve custom key call counts and the collision-safe key encoding.

Add encoding regressions and allocation benchmarks. Verified with go vet ./... and go test -race ./... on this branch.
@hsluoyz

hsluoyz commented Nov 15, 2026

Copy link
Copy Markdown
Member

Thanks for the analysis and the benchmarks, the allocation numbers reproduced here as well.

This has been addressed in e9a3bad with a slightly different approach: GetCacheKey now resolves every part (including CacheableParam.GetCacheKey(), still called exactly once) in a single pass into a small stack array, sums the exact size, and then writes the key into a builder grown once. This keeps the key format unchanged and also covers mixed requests, most notably Enforce(EnforceContext{...}, ...), which the all-strings fast path in this PR left on the old path (4 → 2 allocs there; plain string requests go to 1 alloc as in this PR). The tests and benchmark were folded into the existing enforcer_cached_test.go and enforcer_cached_b_test.go.

Closing this PR in favor of that commit. Thanks again!

@hsluoyz hsluoyz closed this Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants