A rate limiter that was wrong for the first request only

Python kept appearing in job postings and I had nothing to show for it, so I built something small and real: Token Bucket, Leaky Bucket and Sliding Window, implemented as independent classes behind one interface, with a traffic simulator and a chart comparing how each behaves under load.

The bug

Each algorithm tracks when it last refilled or leaked. I initialised that lazily, on the first call, which is a reasonable instinct — you do not know when traffic will start, so you start the clock when it does.

Except I initialised it to the wall clock at that moment, not to the timestamp of the request being handled. Those are the same number when requests arrive live. They are different numbers when time is passed in, which is exactly what happens in a simulation and in a test.

# wrong: the clock starts when the code runs
if self._last is None:
    self._last = time.monotonic()

# right: the clock starts when the traffic starts
if self._last is None:
    self._last = now

The window for the very first request was measured from the wrong origin. Every request after it was correct, because by then _last had been set from a real timestamp.

Why the obvious tests missed it

A test that fires a burst and asserts the limit holds passes: request one is off by a fraction, requests two through forty are fine, the total is right. A test that checks refill timing passes, because refill only happens after the first request has already fixed the state.

It only showed up in the simulator, where I control time explicitly and start at zero. The first request behaved as though the window had begun at whatever the machine's clock said, which was not zero, so its allowance came out wrong. One dot out of place on a chart.

What caught it

Being able to inject time. Every algorithm takes now as a parameter rather than reading the clock itself. That was originally for testing, but it is what made the bug visible — with time as an input, first-request behaviour is something you can assert on, and the discrepancy between the injected timestamp and the internal state had somewhere to show up.

The suite is 16 tests covering limit enforcement, refill and leak timing, and window-expiry edges, plus live verification with curl against a running FastAPI server to confirm real 200s and 429s. But the test that would have caught this on day one is the boring one: assert that the very first request, at a known timestamp, behaves exactly like the second.

The lesson

Lazy initialisation quietly picks a value from context. When the context is a clock, it picks the wrong one the moment the caller has their own idea of what time it is. If your code takes time as a parameter, take it from the parameter — including the first time.