Every public API eventually meets a client that sends too much, too fast. A rate limiter is the bouncer at the door. However, “allow 100 requests per minute” can mean quite different things, depending on the algorithm you choose.
In this article, we compare the two most popular choices, the token bucket and the sliding window. We explain how each works, simulate them under the same traffic, and see which one fits which job.
The Fixed Window Trap
Before comparing the two, it helps to see the naive approach. A fixed window counter simply resets every minute. As Cloudflare notes in its rate limiting write-up, this is not very accurate, because the reset lets regular traffic spikes slip through.
For instance, a client can send 100 requests at the very end of one minute and 100 more right after the reset. Consequently, 200 requests pass within a fraction of a second, even though the limit is 100. Both algorithms below exist largely to avoid this.
Token Bucket: Controlled Bursts
How it works
Imagine a bucket that holds tokens. Tokens drip in at a fixed rate, and the bucket has a maximum size. Each request takes one token, and if the bucket is empty, the request is rejected.
The bucket size sets the allowed burst, while the refill rate sets the long-term average. Furthermore, the state is tiny: just the current token count and the time of the last refill. Stripe’s API documentation even recommends a client-side token bucket as a common way to control your own request rate.
The catch
Because a full bucket can be emptied instantly, the most you can get in one window is the bucket size plus what refills meanwhile. That is by design. Still, it means a token bucket is not a strict “N per minute” counter.
Sliding Window: Counting the Recent Past
The exact version
The sliding window log stores a timestamp for every request and counts those inside the last window. It is accurate, but Cloudflare points out that it needs a lot of processing and memory, especially for long windows such as 5,000 requests per hour.
The practical version
The sliding window counter keeps only two numbers: the previous window’s count and the current one. It then weights the previous count by how much of that window still overlaps. In Cloudflare’s example, 42 requests last minute and 18 so far this minute, 15 seconds in, gives 42 × 0.75 + 18 = 49.5 requests.
The trick assumes the previous window was evenly spread. Even so, on 400 million real requests from 270,000 sources, Cloudflare reported that only 0.003% were wrongly allowed or limited. The average difference from the true rate was 6%, and no source below the threshold was blocked.
What the Numbers Say
To compare them directly, I wrote a small time-based simulation in Python. Every algorithm enforced 100 requests per minute. I tested three kinds of traffic: a 200-request burst straddling a window edge, a legitimate client that sends 30 requests every 20 seconds (90 per minute), and a sustained flood of 50 requests per second for 10 minutes.
| Algorithm | State kept | Burst at window edge (of 200) | Worst 60 s under flood | Legit bursty traffic accepted |
|---|---|---|---|---|
| Fixed window | 1 counter + window id | 200 | 200 | 100% |
| Sliding log | Up to 100 timestamps | 100 | 100 | 100% |
| Sliding counter | 2 counters + window id | 101 | 129 | 92.8% |
| Token bucket (size 20) | Tokens + last refill | 20 | 119 | 66.7% |
| Token bucket (size 50) | Tokens + last refill | 50 | 149 | 100% |
How to read the results
First, the fixed window failed exactly as predicted, letting 200 requests through. Next, the sliding log was perfect but stored up to 100 timestamps per client, which is costly at scale.
The sliding counter was cheap and close, yet not perfect. Under a sustained flood it allowed 129 in its worst minute, and it wrongly rejected 7.2% of the bursty legitimate client. Cloudflare’s far lower error comes from real traffic, so treat my flood as a stress test, not a prediction.
Finally, the token bucket shows a real trade-off. A bucket of 20 stops bursts early but rejected a third of the legitimate bursts. A bucket of 50 accepted them all, but it allowed 149 requests in the worst minute. Therefore, bucket size is a business decision, not just a setting.
Which One Should You Choose?
| You need | Better fit | Why |
|---|---|---|
| Allow short bursts, cap the average | Token bucket | Burst size and rate are separate knobs |
| A strict “N per window” promise | Sliding log | Exact count, at a memory cost |
| Cheap per-client state at scale | Sliding counter | Two counters, small error |
| Client-side throttling of your own calls | Token bucket | Smooth, simple, widely available |
Quick checklist
- Decide whether bursts are acceptable before picking an algorithm.
- Avoid plain fixed windows for anything abuse-sensitive.
- Size the token bucket from real client burst patterns.
- Accept the sliding counter’s small error when memory matters.
- Test with bursty, legitimate traffic, not only floods.
What We Have Learned
Fixed windows are cheap but let double traffic through at the window edge, so most systems choose something better. The token bucket separates burst size from average rate, which makes it ideal for tolerating short spikes. The sliding window counts the recent past, and its two-counter version gives near-exact limits with very little memory. In our simulation, the sliding log was exact, the sliding counter was cheap and close, and the token bucket’s behavior depended entirely on its size. In short, choose the token bucket when bursts are normal, and choose a sliding window when “N per window” is a promise you must keep.
