Design a Distributed Cache
Partition cached data, survive node loss and keep cache misses from overwhelming the source of truth.
Infrastructure, platform and reliability engineers.
Your approach: Use diagrams or prose to explain responsibilities, state, capacity and failure behavior. Show the calculations requested by the question.
The problem
Design a distributed cache for a read-heavy service with a separate durable database. Specify get, set and delete behavior, key ownership, expiry and eviction. Explain how clients find a key during membership changes and how the database is protected when popular entries disappear.
- Keep 100 million entries averaging 1 KB of value data; peak demand is 500,000 reads/second.
- Five percent of keys account for 80% of requests, and values may be stale for up to 30 seconds.
- Cache nodes can restart or be replaced during normal traffic. The database cannot sustain the full miss load.
Work within these constraints
Declare p95 cache-hit server latency in one region.
Required target: ≤ 5 milliseconds
Explain invalidation, expiry and the circumstances under which a stale value may be returned.
Bound concurrent fills and load on the source when nodes or hot keys disappear.
What to deliver
Contract and key placement
Define cache APIs, namespace isolation and client routing.
Memory and replication
Estimate capacity including overhead and any replicas; explain eviction.
Membership change
Trace node loss and replacement, including misses and any moved keys.
Hot-key recovery
Walk through mass expiry and compare duplicate fills, request coalescing and stale serving.