Background
Rate limiting at the edge is a solved problem if your traffic fits on one machine. It isn't once you have a fleet. The naive approach — in-process counters per node — breaks the moment a client routes across instances, which happens constantly behind a load balancer.
The alternative is pushing state to a shared store. Redis is the obvious candidate: single-threaded, low-latency, and expressive enough to encode token-bucket logic atomically. The implementation challenge is keeping that round-trip cost low enough to not become the bottleneck it's meant to control.
The token-bucket model
Token bucket gives you two levers: a capacity (burst size) and a refill rate (sustained throughput). Every request costs one token. Tokens refill continuously at the configured rate, capped at capacity.
The state per client is just two values:
tokens— the current countlast_refill— a timestamp used to compute elapsed time and apply the refill
This maps cleanly to a Redis hash. The atomic read-modify-write is the hard part.
Atomic execution via Lua
Redis executes Lua scripts atomically from the server's perspective — no other command runs between script start and finish. This lets us implement the full token-bucket cycle as a single network round-trip:
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2]) -- tokens per millisecond
local now = tonumber(ARGV[3]) -- unix ms from caller
local data = redis.call("HMGET", key, "tokens", "last_refill")
local tokens = tonumber(data[1]) or capacity
local last_refill = tonumber(data[2]) or now
-- Refill proportional to elapsed time
local elapsed = math.max(0, now - last_refill)
tokens = math.min(capacity, tokens + elapsed * rate)
if tokens < 1 then
return {0, tokens, last_refill}
end
tokens = tokens - 1
redis.call("HMSET", key, "tokens", tokens, "last_refill", now)
redis.call("PEXPIRE", key, math.ceil(capacity / rate) + 1000)
return {1, tokens, now}
The caller gets back an allowed/denied flag, the current token count (for headers), and the timestamp used — useful for debugging drift.
Caller-side integration
In Go, the limiter wraps a Redis client and handles script caching via EVALSHA. On first call it loads the script and stores the SHA; subsequent calls skip the upload:
type Limiter struct {
rdb *redis.Client
sha string
capacity int64
rate float64 // tokens/ms
}
func (l *Limiter) Allow(ctx context.Context, key string) (bool, error) {
now := time.Now().UnixMilli()
res, err := l.rdb.EvalSha(ctx, l.sha, []string{key},
l.capacity, l.rate, now,
).Int64Slice()
if err != nil {
return true, err // fail open under Redis errors
}
return res[0] == 1, nil
}
The fail open decision under Redis errors was deliberate: a limiter that blocks legitimate traffic on outage is worse than one that briefly lets excess through.
Performance
Measured at steady state on a three-node Redis cluster (3× r6g.large), with traffic generated by wrk across a 16-instance Go fleet:
| Scenario | P50 | P99 | P99.9 | |---|---|---|---| | Under limit | 0.4 ms | 1.1 ms | 2.8 ms | | At limit (rejecting) | 0.4 ms | 1.0 ms | 2.6 ms | | Redis failover in progress | 0.5 ms | 3.4 ms | 12 ms |
The overhead of the rate-limit check is dominated by the Redis round-trip — the Lua execution itself is sub-50µs. At 50k req/s across 16 callers, the Redis cluster CPU peaked at 18%.
What I learned
Clock drift is real. Early tests used time.Now() on the caller rather than TIME inside the Lua script. Servers drifting 10–20ms apart created subtle refill miscalculations. Passing the timestamp as an argument and accepting the caller's clock turned out to be fine in practice — the error is bounded to clock skew per request, not accumulated.
Fail open vs. fail closed is a product decision, not a systems one. I initially failed closed because it felt "safer." After one Redis blip dropped 30% of legitimate traffic for 90 seconds, the team overrode it. The tradeoff depends entirely on what you're protecting.
Pipeline the token count into response headers. Returning X-RateLimit-Remaining and X-RateLimit-Reset from the same Lua call costs nothing extra and makes client-side back-off trivial to implement correctly.