Design an API Gateway
Route API traffic with authentication, tenant limits and safe configuration rollout.
Infrastructure, platform and reliability engineers.
Your approach: Use diagrams or prose to explain responsibilities, state, capacity and failure behavior. Show the calculations requested by the question.
The problem
Design an API gateway for a multi-tenant product with independently deployed backend services. Follow a request through identity checks, route selection and failure handling. Explain how route and policy changes become effective while a gateway continues serving during configuration-service downtime.
- Route 200,000 requests/second at peak across 200 backend services in two regions.
- Tenants have different quotas, and some requests stream responses for ten minutes.
- Operators update routes and policies hourly; a bad change must be reversible without restarting the fleet.
Work within these constraints
Declare p95 overhead excluding backend processing and client network time.
Required target: ≤ 20 milliseconds
Do not trust caller-supplied identity headers; propagate authenticated context without leaking credentials.
Define last-known-good behavior and safe policy limits when updates cannot be fetched.
What to deliver
Request lifecycle
Show authentication, quotas, routing, timeouts, streaming and response handling.
Configuration lifecycle
Describe versioned route changes, validation, canaries and rollback.
Capacity and overload
Estimate gateway and connection capacity; isolate a slow or failing backend.
Failure and retry policy
Trace a bad route and a timed-out write; explain which retries are safe.