perf: optimise and bench first_value, last_value aggregate#21383
perf: optimise and bench first_value, last_value aggregate#21383theirix wants to merge 6 commits intoapache:mainfrom
Conversation
theirix
force-pushed
the
optimise-first_last
branch
from
3bb5dcb to
1272caf
Compare
| // derived from `ordering_req`. | ||
| sort_options: Vec<SortOptions>, |
There was a problem hiding this comment.
Since these changes are for performance optimization, should we add some benchmarks for FirstValueAccumulator to quantify the improvement and prevent future regressions?
There was a problem hiding this comment.
I introduced a bench only for FirstLastGroupsAccumulator since it has a non-trivial buffer improvement in take_state.
Extracting the construction of sort_options out of the hot loop can only improve performance. We don't bench mutable update_batch and merge_batch (I have a patch though), which go through sort_options usage. Anyway, I added benches for FirstValueAccumulator and LastValueAccumulator for our future benefit.
| let accumulator = prepare_groups_accumulator(is_first); | ||
| black_box( | ||
| accumulator | ||
| .convert_to_state(std::slice::from_ref(&values), opt_filter) |
There was a problem hiding this comment.
Why are we benchmarking convert_to_state here? Since its implementation hasn't changed in this PR, I'm curious about the motivation.
There was a problem hiding this comment.
Mimicking other aggregate benches. Agree, it isn't directly affected by the changes.
| @@ -0,0 +1,262 @@ | |||
| // Licensed to the Apache Software Foundation (ASF) under one | |||
There was a problem hiding this comment.
If you make this new benchmark as a standalone PR it will be easier to use our automated benchmarking tools to verify the improvements
3 participants
Which issue does this PR close?
GroupsAccumulatorforfirst_valueaggregate (speed upfirst_valueandDISTINCT ONqueries) #17899Rationale for this change
A performance improvement of
first_last.rs(up to 36%), based on the original author's observations.UPD: rebased on yesterday's refactoring in #17899 - сс @UBarney @alamb
What changes are included in this PR?
take_state- no need to copy vectors and buffers- Rename structs and functions as recommended in TODOs - a majority of changes in this PRevaluatetest (to actually test taking state) andconvert_to_state(as in other benches)Are these changes tested?
Improvements: up to 36%
Raw bench result:
Details
first_value convert_to_state nulls=0%, filter=false time: [98.086 µs 99.014 µs 100.25 µs] change: [−15.400% −13.532% −11.559%] (p = 0.00 < 0.05) Performance has improved. Found 17 outliers among 100 measurements (17.00%) 4 (4.00%) low mild 3 (3.00%) high mild 10 (10.00%) high severeBenchmarking first_value evaluate_bench nulls=0%, filter=false, first(2): Collecting 100 samples in estimated 7.1575 s (10k ite
first_value evaluate_bench nulls=0%, filter=false, first(2)
time: [54.938 µs 55.482 µs 56.060 µs]
change: [−38.163% −36.295% −34.463%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
Benchmarking first_value evaluate_bench nulls=0%, filter=false, all: Collecting 100 samples in estimated 7.0741 s (10k iteratio
first_value evaluate_bench nulls=0%, filter=false, all
time: [50.624 µs 51.092 µs 51.612 µs]
change: [−17.955% −16.332% −14.593%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
first_value convert_to_state nulls=0%, filter=true
time: [2.0647 µs 2.0881 µs 2.1148 µs]
change: [−7.6314% −6.2174% −4.8530%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
Benchmarking first_value evaluate_bench nulls=0%, filter=true, first(2): Collecting 100 samples in estimated 9.6354 s (10k iter
first_value evaluate_bench nulls=0%, filter=true, first(2)
time: [54.708 µs 55.240 µs 55.805 µs]
change: [−27.249% −24.755% −22.195%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
Benchmarking first_value evaluate_bench nulls=0%, filter=true, all: Collecting 100 samples in estimated 9.6394 s (10k iteration
first_value evaluate_bench nulls=0%, filter=true, all
time: [50.540 µs 50.963 µs 51.424 µs]
change: [−6.8346% −3.9146% −0.8864%] (p = 0.01 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
Benchmarking first_value convert_to_state nulls=90%, filter=false: Collecting 100 samples in estimated 5.0440 s (50k iterations
first_value convert_to_state nulls=90%, filter=false
time: [98.054 µs 98.996 µs 100.10 µs]
change: [−3.4798% −2.2129% −1.0173%] (p = 0.00 < 0.05)
Performance has improved.
Found 19 outliers among 100 measurements (19.00%)
1 (1.00%) low severe
8 (8.00%) low mild
2 (2.00%) high mild
8 (8.00%) high severe
Benchmarking first_value evaluate_bench nulls=90%, filter=false, first(2): Collecting 100 samples in estimated 8.5385 s (10k it
first_value evaluate_bench nulls=90%, filter=false, first(2)
time: [53.780 µs 54.673 µs 55.639 µs]
change: [−17.702% −15.978% −14.173%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) low mild
5 (5.00%) high mild
2 (2.00%) high severe
Benchmarking first_value evaluate_bench nulls=90%, filter=false, all: Collecting 100 samples in estimated 8.2692 s (10k iterati
first_value evaluate_bench nulls=90%, filter=false, all
time: [49.851 µs 50.289 µs 50.755 µs]
change: [−4.8554% −3.1896% −1.3951%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) low severe
2 (2.00%) high mild
3 (3.00%) high severe
Benchmarking first_value convert_to_state nulls=90%, filter=true: Collecting 100 samples in estimated 5.0077 s (2.4M iterations
first_value convert_to_state nulls=90%, filter=true
time: [2.0339 µs 2.0465 µs 2.0603 µs]
change: [−1.8037% −0.7068% +0.3821%] (p = 0.22 > 0.05)
No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low mild
3 (3.00%) high mild
3 (3.00%) high severe
Benchmarking first_value evaluate_bench nulls=90%, filter=true, first(2): Collecting 100 samples in estimated 9.7925 s (10k ite
first_value evaluate_bench nulls=90%, filter=true, first(2)
time: [54.544 µs 55.119 µs 55.720 µs]
change: [−15.717% −13.982% −12.279%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high severe
Benchmarking first_value evaluate_bench nulls=90%, filter=true, all: Collecting 100 samples in estimated 9.7401 s (10k iteratio
first_value evaluate_bench nulls=90%, filter=true, all
time: [50.126 µs 50.886 µs 51.703 µs]
change: [+0.8379% +4.1713% +7.1024%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
6 (6.00%) high mild
3 (3.00%) high severe
last_value convert_to_state nulls=0%, filter=false
time: [97.957 µs 98.314 µs 98.692 µs]
change: [−2.8086% −2.0315% −1.2541%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
7 (7.00%) high mild
3 (3.00%) high severe
Benchmarking last_value evaluate_bench nulls=0%, filter=false, first(2): Collecting 100 samples in estimated 7.0582 s (10k iter
last_value evaluate_bench nulls=0%, filter=false, first(2)
time: [52.692 µs 53.414 µs 54.144 µs]
change: [−22.228% −20.636% −19.153%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
2 (2.00%) high mild
2 (2.00%) high severe
Benchmarking last_value evaluate_bench nulls=0%, filter=false, all: Collecting 100 samples in estimated 6.9411 s (10k iteration
last_value evaluate_bench nulls=0%, filter=false, all
time: [49.781 µs 50.226 µs 50.793 µs]
change: [−1.9658% −0.1634% +1.5825%] (p = 0.86 > 0.05)
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
1 (1.00%) low mild
6 (6.00%) high mild
4 (4.00%) high severe
last_value convert_to_state nulls=0%, filter=true
time: [2.0639 µs 2.0781 µs 2.0949 µs]
change: [−0.7535% +0.4491% +1.6752%] (p = 0.47 > 0.05)
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
3 (3.00%) low mild
4 (4.00%) high mild
4 (4.00%) high severe
Benchmarking last_value evaluate_bench nulls=0%, filter=true, first(2): Collecting 100 samples in estimated 9.8040 s (10k itera
last_value evaluate_bench nulls=0%, filter=true, first(2)
time: [53.779 µs 54.311 µs 54.868 µs]
change: [−15.863% −14.071% −12.391%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
Benchmarking last_value evaluate_bench nulls=0%, filter=true, all: Collecting 100 samples in estimated 9.6860 s (10k iterations
last_value evaluate_bench nulls=0%, filter=true, all
time: [50.276 µs 50.794 µs 51.429 µs]
change: [−1.6780% +0.0697% +1.9541%] (p = 0.94 > 0.05)
No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severe
last_value convert_to_state nulls=90%, filter=false
time: [97.508 µs 98.412 µs 99.486 µs]
change: [−1.2994% −0.3404% +0.8319%] (p = 0.52 > 0.05)
No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
2 (2.00%) low mild
3 (3.00%) high mild
4 (4.00%) high severe
Benchmarking last_value evaluate_bench nulls=90%, filter=false, first(2): Collecting 100 samples in estimated 8.9282 s (10k ite
last_value evaluate_bench nulls=90%, filter=false, first(2)
time: [54.064 µs 54.748 µs 55.433 µs]
change: [−15.790% −13.678% −11.790%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking last_value evaluate_bench nulls=90%, filter=false, all: Collecting 100 samples in estimated 9.2411 s (10k iteratio
last_value evaluate_bench nulls=90%, filter=false, all
time: [49.964 µs 50.731 µs 51.630 µs]
change: [−2.4530% −0.2470% +1.8407%] (p = 0.82 > 0.05)
No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
8 (8.00%) high mild
1 (1.00%) high severe
last_value convert_to_state nulls=90%, filter=true
time: [2.0660 µs 2.0874 µs 2.1139 µs]
change: [−3.2299% −1.8585% −0.5850%] (p = 0.01 < 0.05)
Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
4 (4.00%) high mild
6 (6.00%) high severe
Benchmarking last_value evaluate_bench nulls=90%, filter=true, first(2): Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.6s, enable flat sampling, or reduce sample count to 60.
Benchmarking last_value evaluate_bench nulls=90%, filter=true, first(2): Collecting 100 samples in estimated 5.6274 s (5050 ite
last_value evaluate_bench nulls=90%, filter=true, first(2)
time: [53.565 µs 54.399 µs 55.381 µs]
change: [−18.757% −16.680% −14.334%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
8 (8.00%) high mild
Benchmarking last_value evaluate_bench nulls=90%, filter=true, all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.3s, enable flat sampling, or reduce sample count to 60.
Benchmarking last_value evaluate_bench nulls=90%, filter=true, all: Collecting 100 samples in estimated 5.3402 s (5050 iteratio
last_value evaluate_bench nulls=90%, filter=true, all
time: [49.473 µs 50.055 µs 50.743 µs]
change: [−41.448% −32.836% −24.471%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
cargo bench --bench first_last -- --baseline main-first_last3 810.16s user 14.72s system 135% cpu 10:09.30 total
irix@tau ~/projects/third-party/datafusion [optimise-first_last (theirix/optimise-first_last)?] % time cargo bench --bench first_last -- --baseline main-first_last4
Compiling datafusion-functions-aggregate v53.0.0 (/Users/irix/projects/third-party/datafusion/datafusion/functions-aggregate)
Finished
benchprofile [optimized] target(s) in 5m 04sRunning benches/first_last.rs (target/release/deps/first_last-c2812404ab64de23)
Gnuplot not found, using plotters backend
first_value convert_to_state nulls=0%, filter=false
time: [100.29 µs 100.92 µs 101.62 µs]
change: [−1.3141% −0.4446% +0.4530%] (p = 0.34 > 0.05)
No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
7 (7.00%) high mild
2 (2.00%) high severe
Benchmarking first_value evaluate_bench nulls=0%, filter=false, first(2): Collecting 100 samples in estimated 7.6058 s (10k ite
first_value evaluate_bench nulls=0%, filter=false, first(2)
time: [57.325 µs 58.842 µs 60.567 µs]
change: [−14.545% −12.765% −10.817%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
Benchmarking first_value evaluate_bench nulls=0%, filter=false, all: Collecting 100 samples in estimated 7.1421 s (10k iteratio
first_value evaluate_bench nulls=0%, filter=false, all
time: [52.632 µs 53.182 µs 53.814 µs]
change: [−9.5351% −6.9990% −4.3937%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
6 (6.00%) high mild
3 (3.00%) high severe
first_value convert_to_state nulls=0%, filter=true
time: [2.2486 µs 2.3289 µs 2.4224 µs]
change: [−14.956% −12.528% −9.6570%] (p = 0.00 < 0.05)
Performance has improved.
Found 19 outliers among 100 measurements (19.00%)
2 (2.00%) high mild
17 (17.00%) high severe
Benchmarking first_value evaluate_bench nulls=0%, filter=true, first(2): Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.3s, enable flat sampling, or reduce sample count to 60.
Benchmarking first_value evaluate_bench nulls=0%, filter=true, first(2): Collecting 100 samples in estimated 5.3276 s (5050 ite
first_value evaluate_bench nulls=0%, filter=true, first(2)
time: [60.696 µs 62.295 µs 63.977 µs]
change: [−28.986% −27.090% −25.120%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) low mild
5 (5.00%) high mild
Benchmarking first_value evaluate_bench nulls=0%, filter=true, all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 60.
Benchmarking first_value evaluate_bench nulls=0%, filter=true, all: Collecting 100 samples in estimated 5.1224 s (5050 iteratio
first_value evaluate_bench nulls=0%, filter=true, all
time: [53.834 µs 54.827 µs 55.913 µs]
change: [−12.918% −8.8195% −4.0336%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) low mild
1 (1.00%) high mild
3 (3.00%) high severe
Benchmarking first_value convert_to_state nulls=90%, filter=false: Collecting 100 samples in estimated 5.1237 s (50k iterations
first_value convert_to_state nulls=90%, filter=false
time: [104.46 µs 106.26 µs 108.42 µs]
change: [−11.589% −8.9599% −6.3106%] (p = 0.00 < 0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high mild
Benchmarking first_value evaluate_bench nulls=90%, filter=false, first(2): Collecting 100 samples in estimated 8.1885 s (10k it
first_value evaluate_bench nulls=90%, filter=false, first(2)
time: [59.528 µs 60.651 µs 61.809 µs]
change: [−21.550% −19.288% −16.929%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high mild
Benchmarking first_value evaluate_bench nulls=90%, filter=false, all: Collecting 100 samples in estimated 8.2091 s (10k iterati
first_value evaluate_bench nulls=90%, filter=false, all
time: [53.384 µs 54.251 µs 55.248 µs]
change: [−9.1283% −6.6250% −4.4009%] (p = 0.00 < 0.05)
Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
7 (7.00%) high mild
6 (6.00%) high severe
Benchmarking first_value convert_to_state nulls=90%, filter=true: Collecting 100 samples in estimated 5.0021 s (2.4M iterations
first_value convert_to_state nulls=90%, filter=true
time: [2.1172 µs 2.1355 µs 2.1555 µs]
change: [−0.6685% +0.5813% +1.7649%] (p = 0.37 > 0.05)
No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
Benchmarking first_value evaluate_bench nulls=90%, filter=true, first(2): Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.2s, enable flat sampling, or reduce sample count to 60.
Benchmarking first_value evaluate_bench nulls=90%, filter=true, first(2): Collecting 100 samples in estimated 5.2407 s (5050 it
first_value evaluate_bench nulls=90%, filter=true, first(2)
time: [57.516 µs 58.757 µs 60.089 µs]
change: [−18.305% −15.732% −13.092%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high mild
Benchmarking first_value evaluate_bench nulls=90%, filter=true, all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.3s, enable flat sampling, or reduce sample count to 60.
Benchmarking first_value evaluate_bench nulls=90%, filter=true, all: Collecting 100 samples in estimated 5.2574 s (5050 iterati
first_value evaluate_bench nulls=90%, filter=true, all
time: [63.381 µs 65.433 µs 67.411 µs]
change: [+5.8137% +12.240% +19.268%] (p = 0.00 < 0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) high mild
9 (9.00%) high severe
last_value convert_to_state nulls=0%, filter=false
time: [100.59 µs 101.71 µs 103.06 µs]
change: [−0.8042% +0.1307% +1.0555%] (p = 0.79 > 0.05)
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
2 (2.00%) low mild
6 (6.00%) high mild
3 (3.00%) high severe
Benchmarking last_value evaluate_bench nulls=0%, filter=false, first(2): Collecting 100 samples in estimated 7.3301 s (10k iter
last_value evaluate_bench nulls=0%, filter=false, first(2)
time: [59.031 µs 60.067 µs 61.215 µs]
change: [−20.562% −15.398% −10.596%] (p = 0.00 < 0.05)
Performance has improved.
Benchmarking last_value evaluate_bench nulls=0%, filter=false, all: Collecting 100 samples in estimated 7.4307 s (10k iteration
last_value evaluate_bench nulls=0%, filter=false, all
time: [54.526 µs 55.568 µs 56.708 µs]
change: [−9.5126% −5.3244% −1.2387%] (p = 0.02 < 0.05)
Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
last_value convert_to_state nulls=0%, filter=true
time: [2.1428 µs 2.1838 µs 2.2289 µs]
change: [−5.7061% −3.1097% −0.6074%] (p = 0.02 < 0.05)
Change within noise threshold.
Benchmarking last_value evaluate_bench nulls=0%, filter=true, first(2): Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 70.
Benchmarking last_value evaluate_bench nulls=0%, filter=true, first(2): Collecting 100 samples in estimated 5.0694 s (5050 iter
last_value evaluate_bench nulls=0%, filter=true, first(2)
time: [63.666 µs 66.076 µs 68.373 µs]
change: [−17.910% −13.091% −8.0825%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
Benchmarking last_value evaluate_bench nulls=0%, filter=true, all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 60.
Benchmarking last_value evaluate_bench nulls=0%, filter=true, all: Collecting 100 samples in estimated 5.1148 s (5050 iteration
last_value evaluate_bench nulls=0%, filter=true, all
time: [51.817 µs 52.699 µs 53.714 µs]
change: [−15.171% −12.649% −10.079%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
3 (3.00%) high mild
3 (3.00%) high severe
last_value convert_to_state nulls=90%, filter=false
time: [107.51 µs 110.89 µs 114.78 µs]
change: [−0.7613% +1.8668% +4.7036%] (p = 0.17 > 0.05)
No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
Benchmarking last_value evaluate_bench nulls=90%, filter=false, first(2): Collecting 100 samples in estimated 9.6846 s (10k ite
last_value evaluate_bench nulls=90%, filter=false, first(2)
time: [58.180 µs 58.969 µs 59.915 µs]
change: [−19.674% −16.125% −12.269%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
Benchmarking last_value evaluate_bench nulls=90%, filter=false, all: Collecting 100 samples in estimated 9.2340 s (10k iteratio
last_value evaluate_bench nulls=90%, filter=false, all
time: [52.432 µs 53.242 µs 54.103 µs]
change: [−12.306% −9.3583% −6.3687%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
7 (7.00%) high mild
1 (1.00%) high severe
last_value convert_to_state nulls=90%, filter=true
time: [2.0731 µs 2.0901 µs 2.1137 µs]
change: [−10.926% −9.9495% −8.9933%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
3 (3.00%) low mild
2 (2.00%) high mild
3 (3.00%) high severe
Benchmarking last_value evaluate_bench nulls=90%, filter=true, first(2): Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.7s, enable flat sampling, or reduce sample count to 60.
Benchmarking last_value evaluate_bench nulls=90%, filter=true, first(2): Collecting 100 samples in estimated 5.7484 s (5050 ite
last_value evaluate_bench nulls=90%, filter=true, first(2)
time: [56.238 µs 57.148 µs 58.202 µs]
change: [−31.872% −29.982% −27.838%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
Benchmarking last_value evaluate_bench nulls=90%, filter=true, all: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.8s, enable flat sampling, or reduce sample count to 60.
Benchmarking last_value evaluate_bench nulls=90%, filter=true, all: Collecting 100 samples in estimated 5.7562 s (5050 iteratio
last_value evaluate_bench nulls=90%, filter=true, all
time: [51.057 µs 52.223 µs 53.542 µs]
change: [−21.819% −19.372% −16.998%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
4 (4.00%) high mild
2 (2.00%) high severe
cargo bench --bench first_last -- --baseline main-first_last4 786.76s user 16.16s system 135% cpu 9:50.80 total
Are there any user-facing changes?