Skip to content

Conversation

@nadav-govari
Copy link
Collaborator

@nadav-govari nadav-govari commented Jan 30, 2026

Calling available_permits() didn't actually supply an accurate portrayal of capacity, as enough time may have elapsed that a refill is needed. So now a refill is automatically triggered on any call to available_permits() or acquire().

Additionally, the governor crate has a custom implementation of a clock that is faster than std::time. I benchmarked it as follows:

    while benchmark_start.elapsed() < Duration::from_seconds(20) {
        let acquire_start = Instant::now();
        let acquired = rate_limiter.acquire(100_000);
        let latency = acquire_start.elapsed();

        latencies.push(latency);
        iterations += 1;

        if acquired {
            successful_acquires += 1;
        } else {
            failed_acquires += 1;
        }
    }

With the following results:

Metric code in main with refill changes, Instant::now governor::DefaultClock governor::QuantaUpkeepClock updating every 1ms
Min 25ns 49ns 31ns 25ns
Avg 51ns 51ns 32ns 27ns
P50 51ns 50ns 32ns 27ns
P90 52ns 51ns 33ns 28ns
P99 52ns 52ns 34ns 28ns
P99.9 68ns 72ns 53ns 42ns
Max 45.04µs 147.62µs 94.50µs 26.45µs

Seeing as the benchmark itself used Instant::now, this is probably not scientific, but at these latencies this doesn't really matter.

I went with the default governor clock seeing as an OS thread updating every millisecond is overkill, and the speed of this is not going to contribute to any meaningful bottlenecks enough to optimize this further.

@nadav-govari nadav-govari enabled auto-merge (squash) January 30, 2026 19:19
@nadav-govari nadav-govari merged commit c48801e into main Jan 30, 2026
8 checks passed
@nadav-govari nadav-govari deleted the nadav/rate_limiter branch January 30, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants