3. Design a Distributed Cache

The brief

Design a distributed cache for a read-heavy service with a separate durable database. Specify get, set and delete behavior, key ownership, expiry and eviction. Explain how clients find a key during membership changes and how the database is protected when popular entries disappear.

• Keep 100 million entries averaging 1 KB of value data; peak demand is 500,000 reads/second.
• Five percent of keys account for 80% of requests, and values may be stale for up to 30 seconds.
• Cache nodes can restart or be replaced during normal traffic. The database cannot sustain the full miss load.

Constraints

Cache-hit latency≤ 5 milliseconds

Declare p95 cache-hit server latency in one region.

Freshness contract

Explain invalidation, expiry and the circumstances under which a stale value may be returned.

Database protection

Bound concurrent fills and load on the source when nodes or hot keys disappear.

What to cover

1. 01

   Contract and key placement

   Define cache APIs, namespace isolation and client routing.

2. 02

   Memory and replication

   Estimate capacity including overhead and any replicas; explain eviction.

3. 03

   Membership change

   Trace node loss and replacement, including misses and any moved keys.

4. 04

   Hot-key recovery

   Walk through mass expiry and compare duplicate fills, request coalescing and stale serving.

Worked designs

Explore the architecture and decisions, then build on an example with Coach.

Official design

Design a Distributed Cache

A terabyte of key-value pairs in memory across fifty nodes, read in a millisecond, with LRU, TTLs, replication, hot keys and stampedes handled: one-hour boards for junior, senior and staff, with the theory behind them.

Explore the design

Review rubric

AI feedback uses these criteria. Scores are practice feedback.