Skip to content

Experiment: test performance impact of snmalloc's new arena range#8085

Closed
achamayou wants to merge 6 commits into
mainfrom
achamayou-update-snmalloc-pr-855
Closed

Experiment: test performance impact of snmalloc's new arena range#8085
achamayou wants to merge 6 commits into
mainfrom
achamayou-update-snmalloc-pr-855

Conversation

@achamayou

@achamayou achamayou commented Jul 20, 2026

Copy link
Copy Markdown
Member

Summary

  • update vendored snmalloc to the head of New Arena Range snmalloc#855 (431834831cb3f100e663a7f15cfbed213a466f63)
  • update cgmanifest.json to record the exact source revision

Testing

  • Not run locally; CI will validate this update.

@achamayou
Update snmalloc to PR 855 head
e245345 
Vendor microsoft/snmalloc@4318348 and record the exact revision in cgmanifest.json.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 967c22cc-743f-47bf-826c-ca688823eec7
@achamayou achamayou changed the title Update snmalloc to PR 855 head Experiment: test performance impact of snmalloc's new arena range Jul 20, 2026
@github-actions

github-actions Bot commented Jul 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Description

Comparing 3 available runs from this branch (#8085) against the trend of the last 30 main runs.

Each chart plots every benchmark as an axis, with values normalized so 100 is the EWMA baseline of recent main runs, using a 7-run half-life. The 3 orange branch lines run from the oldest (faintest) to the latest (darkest and thickest); the darker blue band is the main baseline +/- 1 std dev and the lighter blue band around it is +/- 2 std dev.

Axis labels show the latest branch value and its difference from the main EWMA baseline, where 0% is on the baseline. They are coloured green where the latest run improves on the baseline, red where it regresses, and grey where the difference is within one std dev of the baseline (within noise). Higher is better for throughput and rate, lower for latency and memory.

Throughput (tx/s)

---
config:
  radar:
    width: 620
    height: 620
    marginTop: 90
    marginRight: 220
    marginBottom: 60
    marginLeft: 220
    axisLabelFactor: 1.12
    curveTension: 0.08
  theme: base
  themeCSS: |
    .radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
    .radarCurve-4{stroke-width:1.5px!important;stroke-opacity:0.40!important}
    .radarCurve-5{stroke-width:1.5px!important;stroke-opacity:0.50!important}
    .radarCurve-6{stroke-width:1.75px!important;stroke-opacity:1.00!important}
    .radarAxisLabel:nth-of-type(1){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(2){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(3){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(4){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(5){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(6){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(7){fill:#808A94!important}
  themeVariables:
    cScale0: "#62B5E5"
    cScale1: "#62B5E5"
    cScale2: "#62B5E5"
    cScale3: "#62B5E5"
    cScale4: "#F97316"
    cScale5: "#F97316"
    cScale6: "#F97316"
    radar:
      axisColor: "#9CA3AF"
      graticuleColor: "#E5E7EB"
      graticuleOpacity: 0
      axisStrokeWidth: 1
      curveOpacity: 0
---
radar-beta
  axis b0["Basic: 66,500 tx/s ▬ +2%"]
  axis b1["Basic Blocking: 1,008 tx/s ▬ 0%"]
  axis b2["Basic JS: 4,753 tx/s ▬ +4%"]
  axis b3["Basic Multi-Threaded: 86,467 tx/s ▬ +4%"]
  axis b4["Historical Queries: 207,312 tx/s ▬ +4%"]
  axis b5["Logging: 63,246 tx/s ▬ +5%"]
  axis b6["Logging JWT: 10,098 tx/s ▬ +2%"]
  curve stddev2_high["main EWMA + 2 std dev"]{116.52, 105.92, 116.07, 115.86, 123.77, 116.61, 116.82}
  curve stddev1_high["main EWMA + 1 std dev"]{108.26, 102.96, 108.03, 107.93, 111.89, 108.31, 108.41}
  curve stddev1_low["main EWMA - 1 std dev"]{91.74, 97.04, 91.97, 92.07, 88.11, 91.69, 91.59}
  curve stddev2_low["main EWMA - 2 std dev"]{83.48, 94.08, 83.93, 84.14, 76.23, 83.39, 83.18}
  curve branch_0["#8085 (2 runs earlier)"]{107.52, 99.70, 103.90, 103.04, 110.16, 101.63, 103.10}
  curve branch_1["#8085 (1 run earlier)"]{101.88, 99.13, 105.28, 104.09, 104.12, 103.79, 104.60}
  curve branch_2["#8085"]{102.26, 100.18, 103.71, 103.66, 103.59, 104.97, 102.22}
  graticule polygon
  max 141
  min 59
  ticks 0
  showLegend false
Loading

Latency (ms)

---
config:
  radar:
    width: 620
    height: 620
    marginTop: 90
    marginRight: 220
    marginBottom: 60
    marginLeft: 220
    axisLabelFactor: 1.12
    curveTension: 0.08
  theme: base
  themeCSS: |
    .radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
    .radarCurve-4{stroke-width:1.5px!important;stroke-opacity:0.40!important}
    .radarCurve-5{stroke-width:1.5px!important;stroke-opacity:0.50!important}
    .radarCurve-6{stroke-width:1.75px!important;stroke-opacity:1.00!important}
    .radarAxisLabel:nth-of-type(1){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(2){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(3){fill:#808A94!important}
  themeVariables:
    cScale0: "#62B5E5"
    cScale1: "#62B5E5"
    cScale2: "#62B5E5"
    cScale3: "#62B5E5"
    cScale4: "#F97316"
    cScale5: "#F97316"
    cScale6: "#F97316"
    radar:
      axisColor: "#9CA3AF"
      graticuleColor: "#E5E7EB"
      graticuleOpacity: 0
      axisStrokeWidth: 1
      curveOpacity: 0
---
radar-beta
  axis b0["Commit Latency 16ms: 4.53 ms ▬ -8%"]
  axis b1["Commit Latency 1ms: 1.81 ms ▬ +2%"]
  axis b2["Commit Latency 256ms: 207 ms ▬ 0%"]
  curve stddev2_high["main EWMA + 2 std dev"]{136.29, 108.32, 101.13}
  curve stddev1_high["main EWMA + 1 std dev"]{118.15, 104.16, 100.57}
  curve stddev1_low["main EWMA - 1 std dev"]{81.85, 95.84, 99.43}
  curve stddev2_low["main EWMA - 2 std dev"]{63.71, 91.68, 98.87}
  curve branch_0["#8085 (2 runs earlier)"]{106.51, 101.63, 99.94}
  curve branch_1["#8085 (1 run earlier)"]{118.25, 94.82, 99.60}
  curve branch_2["#8085"]{91.99, 101.68, 99.70}
  graticule polygon
  max 162
  min 38
  ticks 0
  showLegend false
Loading

Memory (bytes)

---
config:
  radar:
    width: 620
    height: 620
    marginTop: 90
    marginRight: 220
    marginBottom: 60
    marginLeft: 220
    axisLabelFactor: 1.12
    curveTension: 0.08
  theme: base
  themeCSS: |
    .radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
    .radarCurve-4{stroke-width:1.5px!important;stroke-opacity:0.40!important}
    .radarCurve-5{stroke-width:1.5px!important;stroke-opacity:0.50!important}
    .radarCurve-6{stroke-width:1.75px!important;stroke-opacity:1.00!important}
    .radarAxisLabel:nth-of-type(1){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(2){fill:#E5484D!important}
    .radarAxisLabel:nth-of-type(3){fill:#2DA44E!important}
    .radarAxisLabel:nth-of-type(4){fill:#2DA44E!important}
    .radarAxisLabel:nth-of-type(5){fill:#2DA44E!important}
    .radarAxisLabel:nth-of-type(6){fill:#808A94!important}
  themeVariables:
    cScale0: "#62B5E5"
    cScale1: "#62B5E5"
    cScale2: "#62B5E5"
    cScale3: "#62B5E5"
    cScale4: "#F97316"
    cScale5: "#F97316"
    cScale6: "#F97316"
    radar:
      axisColor: "#9CA3AF"
      graticuleColor: "#E5E7EB"
      graticuleOpacity: 0
      axisStrokeWidth: 1
      curveOpacity: 0
---
radar-beta
  axis b0["Basic: 84.7 MiB ▬ -3%"]
  axis b1["Basic Blocking: 71.7 MiB ▲ 1%"]
  axis b2["Basic JS: 69.6 MiB ▼ 2%"]
  axis b3["Basic Multi-Threaded: 86.8 MiB ▼ 3%"]
  axis b4["Logging: 74.4 MiB ▼ 2%"]
  axis b5["Logging JWT: 68.6 MiB ▬ 0%"]
  curve stddev2_high["main EWMA + 2 std dev"]{106.04, 100.92, 104.14, 101.99, 102.08, 101.60}
  curve stddev1_high["main EWMA + 1 std dev"]{103.02, 100.46, 102.07, 101.00, 101.04, 100.80}
  curve stddev1_low["main EWMA - 1 std dev"]{96.98, 99.54, 97.93, 99.00, 98.96, 99.20}
  curve stddev2_low["main EWMA - 2 std dev"]{93.96, 99.08, 95.86, 98.01, 97.92, 98.40}
  curve branch_0["#8085 (2 runs earlier)"]{105.09, 100.24, 99.62, 96.04, 98.66, 100.54}
  curve branch_1["#8085 (1 run earlier)"]{97.15, 99.95, 100.98, 99.47, 99.84, 99.86}
  curve branch_2["#8085"]{97.45, 100.56, 97.77, 96.97, 97.96, 100.14}
  graticule polygon
  max 111
  min 89
  ticks 0
  showLegend false
Loading

Rate (ops/s)

---
config:
  radar:
    width: 620
    height: 620
    marginTop: 90
    marginRight: 220
    marginBottom: 60
    marginLeft: 220
    axisLabelFactor: 1.12
    curveTension: 0.08
  theme: base
  themeCSS: |
    .radarCurve-0{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-1{fill:color-mix(in srgb, #62B5E5 40%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-2{fill:color-mix(in srgb, #62B5E5 13%, var(--color-canvas-default,var(--bgColor-default,#fff)))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarCurve-3{fill:var(--color-canvas-default,var(--bgColor-default,#fff))!important;fill-opacity:1!important;stroke:none!important;stroke-width:0!important}
    .radarAxisLabel,.radarTitle{fill:var(--color-fg-default,var(--fgColor-default,#111827))!important;color:var(--color-fg-default,var(--fgColor-default,#111827))!important}
    .radarCurve-4{stroke-width:1.5px!important;stroke-opacity:0.40!important}
    .radarCurve-5{stroke-width:1.5px!important;stroke-opacity:0.50!important}
    .radarCurve-6{stroke-width:1.75px!important;stroke-opacity:1.00!important}
    .radarAxisLabel:nth-of-type(1){fill:#2DA44E!important}
    .radarAxisLabel:nth-of-type(2){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(3){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(4){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(5){fill:#808A94!important}
    .radarAxisLabel:nth-of-type(6){fill:#808A94!important}
  themeVariables:
    cScale0: "#62B5E5"
    cScale1: "#62B5E5"
    cScale2: "#62B5E5"
    cScale3: "#62B5E5"
    cScale4: "#F97316"
    cScale5: "#F97316"
    cScale6: "#F97316"
    radar:
      axisColor: "#9CA3AF"
      graticuleColor: "#E5E7EB"
      graticuleOpacity: 0
      axisStrokeWidth: 1
      curveOpacity: 0
---
radar-beta
  axis b0["CHAMP get: 39,870,732 ops/s ▲ 8%"]
  axis b1["CHAMP put: 5,409,289 ops/s ▬ +1%"]
  axis b2["KV deserialisation: 1,575,051 ops/s ▬ +1%"]
  axis b3["KV serialisation: 1,440,922 ops/s ▬ +2%"]
  axis b4["KV snapshot deserialis...: 4,144 ops/s ▬ +2%"]
  axis b5["KV snapshot serialisation: 4,414 ops/s ▬ 0%"]
  curve stddev2_high["main EWMA + 2 std dev"]{115.12, 115.44, 115.21, 114.77, 114.82, 119.09}
  curve stddev1_high["main EWMA + 1 std dev"]{107.56, 107.72, 107.60, 107.39, 107.41, 109.54}
  curve stddev1_low["main EWMA - 1 std dev"]{92.44, 92.28, 92.40, 92.61, 92.59, 90.46}
  curve stddev2_low["main EWMA - 2 std dev"]{84.88, 84.56, 84.79, 85.23, 85.18, 80.91}
  curve branch_0["#8085 (2 runs earlier)"]{108.21, 101.65, 101.67, 102.53, 101.40, 105.62}
  curve branch_1["#8085 (1 run earlier)"]{106.75, 100.47, 102.49, 102.97, 100.47, 98.94}
  curve branch_2["#8085"]{107.94, 101.22, 100.87, 101.79, 101.90, 100.35}
  graticule polygon
  max 133
  min 67
  ticks 0
  showLegend false
Loading

@achamayou achamayou closed this Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

Assignees

No one assigned

Labels

bench-ab

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@achamayou @mjp41