Add API to prestart threads in threadpools #1032

Activity

1. eregon commented on Jan 17, 2024

   

   eregon

   on Jan 17, 2024

   Member

   More actions

   What is the advantage of doing so?
   The disadvantage is it likely causes extra resource consumption (CPU & memory).
2. catlee commented on Jan 17, 2024

   

   catlee

   on Jan 17, 2024

   Author

   More actions

   The advantage is that you get slightly improved latency on handling the first few items that are posted to the pool. On my system, I see about a 0.5ms improvement to handling the first few items when using prestart.
3. eregon commented on Jan 17, 2024

   

   eregon

   on Jan 17, 2024

   Member

   More actions

   I see. Could you share a repro for that? I'd like to try it locally.
4. catlee commented on Jan 17, 2024

   

   catlee

   on Jan 17, 2024

   Author

   More actions

   Here's how I'm trying to measure the impact:

   require "concurrent"
   
   def gettime
     Process.clock_gettime(Process::CLOCK_MONOTONIC)
   end
   
   def measure_latency(prestart)
     pool = Concurrent::FixedThreadPool.new(1, prestart: prestart)
     times = []
     start = gettime
     pool.post { times << (gettime - start) }
     pool.shutdown
     pool.wait_for_termination
     times.first
   end
   
   def percentiles(times, p)
     times.sort!
     times[(times.size * p).ceil - 1]
   end
   
   n = 1000
   no_prestart_times = n.times.map { measure_latency(false) }
   prestart_times = n.times.map { measure_latency(true) }
   
   puts "No prestart:"
   puts "  50th percentile: #{percentiles(no_prestart_times, 0.5)}"
   puts "  90th percentile: #{percentiles(no_prestart_times, 0.9)}"
   puts "  99th percentile: #{percentiles(no_prestart_times, 0.99)}"
   
   puts "Prestart:"
   puts "  50th percentile: #{percentiles(prestart_times, 0.5)}"
   puts "  90th percentile: #{percentiles(prestart_times, 0.9)}"
   puts "  99th percentile: #{percentiles(prestart_times, 0.99)}"
   
   puts "Delta:"
   puts "  50th percentile: #{percentiles(no_prestart_times, 0.5) - percentiles(prestart_times, 0.5)}"
   puts "  90th percentile: #{percentiles(no_prestart_times, 0.9) - percentiles(prestart_times, 0.9)}"
   puts "  99th percentile: #{percentiles(no_prestart_times, 0.99) - percentiles(prestart_times, 0.99)}"
5. bensheldon commented on Dec 11, 2025

   

   bensheldon

   on Dec 11, 2025

   Contributor

   More actions

   I think a small place to start would be to expose a Ruby equivalent of #prestartCoreThread: which creates a single thread unless max threads is reached.

   Then one could manage the lifecycle decision themselves of whether they want to do something like:

   executor = Concurrent::ThreadPoolExecutor.new(max_threads: 5)
   
   # immediately, or maybe later...
   executor.max_length.times { executor.add_worker }
   # or...
   executor.min_length.times { executor.add_worker }
   # or...
   arbitrary_number.times { executor.add_worker }