Design a distributed rate limiter that decides whether each request is within its client's configured quota, allows requests within the limit, and rejects excess requests before they reach application services.
Example 1
Input: Client user-42 has a limit of 3 requests per minute and has made 2 requests in the current window.
Output: Allow the next request and update the shared counter.
The request is still within the configured quota.
Example 2
Input: Client user-42 has already made 3 requests in the current window and sends another request.
Output: Reject it with HTTP 429 Too Many Requests.
Requests beyond the quota should not consume application or database capacity.
Example 3
Input: Two application servers receive requests from the same client at the same time.
Output: Both checks use one shared, atomic quota decision.
Per-process counters could disagree and allow the client to exceed its limit.
Constraints
- Quota decisions must remain consistent across multiple application servers.
- The limiter is on the request path, so its added latency should stay very low; the lesson uses 1 ms as a planning assumption.
- The design should account for a peak of 100,000 incoming requests per second.
- Define behavior for limiter storage failures, including whether the system fails open or fails closed.
THINK ABOUT IT
What do we ask before designing a rate limiter?