Skip to content

Avoid redundant bulk remote queue notifications - #2209

Merged
ericniebler merged 1 commit into
NVIDIA:mainfrom
fallintoplace:perf/static-thread-pool-bulk-notify
Aug 16, 2026
Merged

Avoid redundant bulk remote queue notifications#2209
ericniebler merged 1 commit into
NVIDIA:mainfrom
fallintoplace:perf/static-thread-pool-bulk-notify

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Check the result of push_front() in the static thread pool bulk enqueue path.
  • Notify a worker only when its submitter queue changes from empty to non-empty.
  • This is a follow-up to Avoid redundant single-task remote queue notifications #2178. The single-task paths were optimized there, while the bulk paths were left unchanged.

Benchmark

Focused sender-level schedule | bulk benchmark on Apple arm64 with AppleClang 21, -O3, 100,000 bulk operations per producer, and the median of 5 runs after 1 warmup. Each cell is end-to-end throughput in Mops/s: main -> patch (change).

Workers / agents 1 producer 2 producers 4 producers 8 producers
2 / 2 5.84 -> 6.59 (+13%) 5.78 -> 5.76 (0%) 4.64 -> 5.36 (+15%) 4.09 -> 4.47 (+9%)
4 / 4 3.39 -> 3.59 (+6%) 2.98 -> 3.41 (+15%) 2.77 -> 3.04 (+10%) 2.50 -> 2.76 (+10%)
8 / 8 1.84 -> 2.22 (+21%) 1.81 -> 1.99 (+10%) 1.67 -> 2.05 (+22%) 1.65 -> 1.85 (+12%)

I also checked smaller bulk operations with one agent per submission:

Workers / agents 2 producers 4 producers 8 producers
2 / 1 5.59 -> 6.56 (+17%) 5.71 -> 5.76 (+1%) 4.46 -> 4.82 (+8%)
4 / 1 4.79 -> 5.51 (+15%) 4.45 -> 4.92 (+10%) 4.33 -> 4.65 (+7%)
8 / 1 4.98 -> 5.20 (+4%) 3.98 -> 4.08 (+3%) 3.36 -> 3.34 (-1%)

The gains are largest when several producers share worker queues.

Testing

  • git diff --check
  • Static thread pool tests: 9 test cases, 18 assertions
  • Full test.exec: 336 test cases, 3,343 assertions

@copy-pr-bot

copy-pr-bot Bot commented Aug 16, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@fallintoplace
fallintoplace marked this pull request as ready for review August 16, 2026 11:10
@fallintoplace
fallintoplace marked this pull request as draft August 16, 2026 11:12
@fallintoplace
fallintoplace marked this pull request as ready for review August 16, 2026 11:13
@ericniebler

Copy link
Copy Markdown
Collaborator

/ok to test e1bd7c4

@ericniebler
ericniebler merged commit 98fbd12 into NVIDIA:main Aug 16, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants