Skip to content

bench: add ASOF join benchmarks - #23833

Draft
Xuanwo wants to merge 59 commits into
apache:mainfrom
Xuanwo:xuanwo/asof-benchmarks
Draft

bench: add ASOF join benchmarks#23833
Xuanwo wants to merge 59 commits into
apache:mainfrom
Xuanwo:xuanwo/asof-benchmarks

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 23, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Rationale for this change

This is the benchmark layer of the ASOF JOIN stack. It keeps performance
workloads separate from the execution, logical, and SQL implementation reviews.

GitHub shows the cumulative stack because fork branches cannot be selected as
upstream PR bases. The isolated diff for this layer is:
Xuanwo/datafusion@xuanwo/asof-sql...xuanwo/asof-benchmarks

What changes are included in this PR?

  • Add a dfbench asof command with six end-to-end workloads: ordered and
    grouped predecessor matching, wide payloads, descending successor matching,
    broadcast-side size asymmetry, and skewed equality keys.
  • Stream output batches during timing instead of retaining result buffers.
  • Assert that each workload plans to AsOfJoinExec.
  • Add a physical Criterion benchmark for Int64, wide UTF-8, and dictionary
    payloads.
  • Exercise both one and four left partitions while keeping the right input
    single-partitioned, covering broadcast build reuse and left-side parallelism.

Are these changes tested?

Yes:

  • cargo fmt --all
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo run -p datafusion-benchmarks --bin dfbench -- asof --iterations 1 --partitions 4
  • cargo bench -p datafusion-physical-plan --bench asof_join --features test_utils -- --test

The refreshed head completed all six end-to-end workloads with the expected
1,000,000, 1,000,000, 250,000, 500,000, 100,000, and 1,000,000 row counts. The
Criterion smoke run completed all six combinations of payload type and left
partition count. These debug/smoke runs validate the
harness and are not presented as performance claims.

Are there any user-facing changes?

This adds ASOF workloads to the repository benchmark tooling. It does not change
query semantics or runtime behavior.

This PR should be reviewed and merged after #23830. After its dependencies
squash-merge, this branch will be refreshed from main without rewriting its
published history.

@github-actions github-actions Bot added documentation Improvements or additions to documentation sql SQL Planner logical-expr Logical plan and expressions optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate proto Related to proto crate physical-plan Changes to the physical-plan crate labels Jul 23, 2026
@codecov-commenter

codecov-commenter commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.46326% with 1351 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.19%. Comparing base (a253a6a) to head (76ec959).
⚠️ Report is 81 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/physical-plan/src/joins/asof_join.rs 85.84% 118 Missing and 62 partials ⚠️
datafusion/expr/src/logical_plan/plan.rs 58.60% 62 Missing and 15 partials ⚠️
benchmarks/src/asof.rs 0.00% 62 Missing ⚠️
datafusion/physical-plan/src/execution_plan.rs 54.13% 58 Missing and 3 partials ⚠️
...sical-plan/src/aggregates/ordered_single_stream.rs 86.74% 49 Missing and 10 partials ⚠️
datafusion/core/src/physical_planner.rs 64.58% 48 Missing and 3 partials ⚠️
...n/src/joins/piecewise_merge_join/existence_join.rs 88.13% 11 Missing and 40 partials ⚠️
datafusion/datasource-parquet/src/file_format.rs 55.33% 37 Missing and 9 partials ⚠️
...afusion/core/src/datasource/file_format/options.rs 25.00% 42 Missing ⚠️
datafusion/physical-plan/src/aggregates/mod.rs 69.40% 27 Missing and 14 partials ⚠️
... and 67 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #23833      +/-   ##
==========================================
+ Coverage   81.03%   81.19%   +0.16%     
==========================================
  Files        1107     1114       +7     
  Lines      385064   392337    +7273     
  Branches   385064   392337    +7273     
==========================================
+ Hits       312040   318568    +6528     
- Misses      54616    55025     +409     
- Partials    18408    18744     +336     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details
     Cloning apache/main
    Building datafusion v54.1.0 (current)
       Built [  59.578s] (current)
     Parsing datafusion v54.1.0 (current)
      Parsed [   0.033s] (current)
    Building datafusion v54.1.0 (baseline)
       Built [  56.797s] (baseline)
     Parsing datafusion v54.1.0 (baseline)
      Parsed [   0.034s] (baseline)
    Checking datafusion v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.631s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [ 118.745s] datafusion
    Building datafusion-expr v54.1.0 (current)
       Built [  28.810s] (current)
     Parsing datafusion-expr v54.1.0 (current)
      Parsed [   0.075s] (current)
    Building datafusion-expr v54.1.0 (baseline)
       Built [  28.972s] (baseline)
     Parsing datafusion-expr v54.1.0 (baseline)
      Parsed [   0.075s] (baseline)
    Checking datafusion-expr v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   1.208s] 223 checks: 222 pass, 1 fail, 0 warn, 31 skip

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/enum_variant_added.ron

Failed in:
  variant LogicalPlan:AsOfJoin in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:301
  variant LogicalPlan:AsOfJoin in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:301

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  60.357s] datafusion-expr
    Building datafusion-optimizer v54.1.0 (current)
       Built [  26.947s] (current)
     Parsing datafusion-optimizer v54.1.0 (current)
      Parsed [   0.030s] (current)
    Building datafusion-optimizer v54.1.0 (baseline)
       Built [  26.590s] (baseline)
     Parsing datafusion-optimizer v54.1.0 (baseline)
      Parsed [   0.031s] (baseline)
    Checking datafusion-optimizer v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.160s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [  54.558s] datafusion-optimizer
    Building datafusion-physical-expr v54.1.0 (current)
       Built [  28.917s] (current)
     Parsing datafusion-physical-expr v54.1.0 (current)
      Parsed [   0.047s] (current)
    Building datafusion-physical-expr v54.1.0 (baseline)
       Built [  28.231s] (baseline)
     Parsing datafusion-physical-expr v54.1.0 (baseline)
      Parsed [   0.049s] (baseline)
    Checking datafusion-physical-expr v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.358s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [  58.432s] datafusion-physical-expr
    Building datafusion-physical-plan v54.1.0 (current)
       Built [  37.383s] (current)
     Parsing datafusion-physical-plan v54.1.0 (current)
      Parsed [   0.145s] (current)
    Building datafusion-physical-plan v54.1.0 (baseline)
       Built [  37.831s] (baseline)
     Parsing datafusion-physical-plan v54.1.0 (baseline)
      Parsed [   0.149s] (baseline)
    Checking datafusion-physical-plan v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.639s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [  77.406s] datafusion-physical-plan
    Building datafusion-proto v54.1.0 (current)
       Built [  52.580s] (current)
     Parsing datafusion-proto v54.1.0 (current)
      Parsed [   0.017s] (current)
    Building datafusion-proto v54.1.0 (baseline)
       Built [  52.527s] (baseline)
     Parsing datafusion-proto v54.1.0 (baseline)
      Parsed [   0.018s] (baseline)
    Checking datafusion-proto v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.125s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [ 106.602s] datafusion-proto
    Building datafusion-proto-models v54.1.0 (current)
       Built [  24.823s] (current)
     Parsing datafusion-proto-models v54.1.0 (current)
      Parsed [   0.127s] (current)
    Building datafusion-proto-models v54.1.0 (baseline)
       Built [  24.797s] (baseline)
     Parsing datafusion-proto-models v54.1.0 (baseline)
      Parsed [   0.127s] (baseline)
    Checking datafusion-proto-models v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   1.744s] 223 checks: 222 pass, 1 fail, 0 warn, 31 skip

--- failure enum_variant_added: enum variant added on exhaustive enum ---

Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/enum_variant_added.ron

Failed in:
  variant ExprType:NormalizeFloatZero in /home/runner/work/datafusion/datafusion/datafusion/proto-models/src/generated/prost.rs:1626
  variant ExprType:NormalizeFloatZero in /home/runner/work/datafusion/datafusion/datafusion/proto-models/src/generated/prost.rs:1626

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  53.017s] datafusion-proto-models
    Building datafusion-sql v54.1.0 (current)
       Built [  42.030s] (current)
     Parsing datafusion-sql v54.1.0 (current)
      Parsed [   0.029s] (current)
    Building datafusion-sql v54.1.0 (baseline)
       Built [  41.667s] (baseline)
     Parsing datafusion-sql v54.1.0 (baseline)
      Parsed [   0.030s] (baseline)
    Checking datafusion-sql v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.234s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [  85.025s] datafusion-sql
    Building datafusion-sqllogictest v54.1.0 (current)
       Built [  99.661s] (current)
     Parsing datafusion-sqllogictest v54.1.0 (current)
      Parsed [   0.022s] (current)
    Building datafusion-sqllogictest v54.1.0 (baseline)
       Built [  99.150s] (baseline)
     Parsing datafusion-sqllogictest v54.1.0 (baseline)
      Parsed [   0.021s] (baseline)
    Checking datafusion-sqllogictest v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.092s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [ 201.820s] datafusion-sqllogictest
    Building datafusion-substrait v54.1.0 (current)
       Built [ 308.090s] (current)
     Parsing datafusion-substrait v54.1.0 (current)
      Parsed [   0.016s] (current)
    Building datafusion-substrait v54.1.0 (baseline)
       Built [ 314.028s] (baseline)
     Parsing datafusion-substrait v54.1.0 (baseline)
      Parsed [   0.017s] (baseline)
    Checking datafusion-substrait v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.197s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [ 624.196s] datafusion-substrait

@github-actions github-actions Bot added the auto detected api change Auto detected API change label Jul 23, 2026
Xuanwo and others added 24 commits August 2, 2026 01:03
Co-authored-by: Yongting You <2010youy01@gmail.com>
# Conflicts:
#	datafusion/expr/src/logical_plan/tree_node.rs
#	datafusion/optimizer/src/analyzer/type_coercion.rs
@github-actions github-actions Bot added the physical-expr Changes to the physical-expr crates label Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto detected api change Auto detected API change core Core DataFusion crate documentation Improvements or additions to documentation logical-expr Logical plan and expressions optimizer Optimizer rules physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate proto Related to proto crate sql SQL Planner sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants