2. Design a Distributed Rate Limiter

The brief

Design a rate limiter used by API servers in two regions. Explain the client-visible quota contract, how simultaneous requests consume allowance, and what happens when shared state is unavailable. Choose and justify an algorithm without assuming all limits need identical consistency.

• Serve 100,000 checks/second across 10,000 tenants, with one tenant producing 20% of checks.
• Each tenant has a sustained request limit and a separately configured burst allowance.
• Clients retry after rejection. Regions can lose contact for five minutes while still receiving traffic.

Constraints

Check latency≤ 10 milliseconds

Declare p95 additional server-side latency during normal operation.

Quota contract

Specify the maximum overshoot allowed across simultaneous requests and regional partitions.

State outage policy

State which requests fail open or closed and how recovered allowance avoids a second full burst.

What to cover

1. 01

   Algorithm and contract

   Define tokens or counters, time semantics, rejection responses and retry guidance.

2. 02

   State and atomicity

   Show key ownership, atomic updates and tenant isolation across API servers.

3. 03

   Capacity and hot keys

   Estimate state size and load; handle the hot tenant and expired counters.

4. 04

   Partition walkthrough

   Trace a regional partition and recovery, comparing accuracy with availability.

Worked designs

Explore the architecture and decisions, then build on an example with Coach.

Official design

Design a Rate Limiter

Allow or reject a million requests a second in under two milliseconds, across every gateway, with limits that hold and rules that change safely: one-hour boards for junior, senior and staff, with the theory behind them.

Explore the design

Review rubric

AI feedback uses these criteria. Scores are practice feedback.