Caching strategies in System Design Interviews: When to Use It and What to Say
When an interview asks you to design a high-traffic service, the first place you look for latency savings is the cache. A cache is a fast-access store that holds a copy of data that would otherwise require a slower round-trip to a database or another backend. It solves the classic "read-heavy, write-light" problem: the system can serve many requests without hammering the primary data store.
A concrete system: Photo-sharing feed
Imagine you are designing the home-feed for a photo-sharing app that shows the latest posts from a user’s friends. The feed is read millions of times per day, but new posts arrive only a few thousand times per hour. Without a cache, each feed request would trigger a database join across the user-friend table, the posts table, and a ranking engine – a costly operation that could take hundreds of milliseconds. By caching the pre-computed feed for each user, you can serve the request in single-digit milliseconds.
How caching works, step by step
- Client request – The front-end asks for
GET /feed?user_id=123. - Cache lookup – The application checks a fast key-value store (e.g., Redis) for a key like
feed:123. - Cache hit – If the key exists, the cached feed JSON is returned immediately.
- Cache miss – If the key is missing, the service queries the database, runs the ranking logic, and builds the feed.
- Populate cache – The freshly built feed is written back to the cache with an appropriate TTL (time-to-live).
- Return response – The feed is sent to the client. Subsequent requests hit the cache until the TTL expires or the cache is explicitly invalidated.
Worked mini-example
Suppose the feed is cached for 60 seconds. At 12:00:00 a request for user 123 arrives:
- Cache miss → DB query returns posts A, B, C.
- Service builds JSON and stores
feed:123 = [A,B,C]with TTL=60. - Client receives the feed.
At 12:00:30 another request for user 123 arrives:
- Cache hit → Redis returns
[A,B,C]in ~2 ms. - No DB call is made.
At 12:01:00 the TTL expires. A request at 12:01:05 triggers a miss, pulls any new posts (say D), and refreshes the cache.
Where caching shows up in classic interview questions
| Interview Question | Typical Cache Placement | Reason |
|---|---|---|
| Design Twitter timeline | Cache the timeline per user (timeline:user_id). | Timeline reads dominate; writes are infrequent (new tweets). |
| Design URL shortener | Cache the mapping short_code → long_url. | Redirects are read-only; cache avoids DB round-trip for every click. |
| Design a video streaming service | Edge CDN cache for video chunks. | Static video files are served to millions; edge cache reduces latency and bandwidth. |
In each case the cache sits right before the slowest component: the database for timelines, the key-value store for short URLs, and the edge network for video files.
Common caching strategies and their trade-offs
| Strategy | How it works | Consistency | Typical Use |
|---|---|---|---|
| Read-through (lazy) | Miss triggers backend fetch, then populates cache. | Stale data until TTL expires; possible cache stampede. | Hot reads with occasional writes (e.g., product catalog). |
| Write-through | Every write updates both backend and cache synchronously. | Stronger consistency; higher write latency. | Frequently updated data where freshness matters (e.g., user profile). |
| Write-behind (async) | Writes go to the cache first; the backend is updated later by an asynchronous flush. | Cache is current but the backend lags; risk of lost writes if the cache fails before the flush. | High-throughput write streams where occasional lag is acceptable. |
| Cache-aside | Application manually reads/writes cache; explicit invalidation. | Full control over staleness; extra code complexity. | Complex invalidation rules (e.g., permission changes). |
| Edge CDN | Distributed nodes cache static assets close to the user. | Content must be immutable or versioned; cache purge latency. | Large static files (images, JS, video). |
The alternative interviewers often ask about
When you mention a cache, interviewers will probe the eviction policy (LRU, LFU, TTL) and the cache invalidation mechanism. They may also ask how you would handle a cache miss burst (the “thundering herd” problem) – a typical answer is to use request coalescing or a per-key lock, so that only one request rebuilds the value.
A 30-second spoken pitch
"In a read-heavy service, I would add a read-through cache in front of the database. The request first checks a fast key-value store; on a hit we return the cached value in a few milliseconds. On a miss we fetch from the DB, build the response, write it back to the cache with a short TTL, and return it. This reduces latency, cuts DB load, and smooths traffic spikes. Trade-offs are staleness and eviction complexity, so I’d choose a TTL that matches the freshness requirement and add a background refresh to avoid stampedes."
Mistakes candidates often make
- Over-caching – Caching everything, including rapidly changing data, leads to stale responses and hard-to-debug bugs.
- Ignoring eviction – Assuming the cache is infinite; in reality you must pick an eviction policy and size the cache based on traffic patterns.
- Claiming zero latency – Even an in-memory cache adds a few milliseconds; exaggerating performance looks naive.
- Skipping invalidation – Forgetting to purge or update the cache after a write causes the classic "old data after update" bug.
- Not addressing cache stampede – When a key expires, many requests can hit the DB simultaneously; a proper solution (e.g., request coalescing) should be mentioned.
How to practice this
- Study a worked design – Open the System Design AI library and review the Cache-Aside (Lazy Loading) building block. Follow its walkthrough and click each component to read its notes.
- Sketch the flow – On the canvas, draw a request path that includes a cache lookup, a miss path, and the write-back step. Label each component with the talking points above.
- Run a mock interview – Explain the cache choice aloud, then answer follow-up questions about TTL, eviction, and cache-stampede mitigation. Repeat until the explanation fits comfortably in 30 seconds.
Questions people ask
When should I suggest a cache in a design interview?
Suggest a cache when the read volume is much higher than the write volume, latency is a key metric, and the data can tolerate a short period of staleness.
What is the difference between read-through and cache-aside?
Read-through automatically populates the cache on a miss, while cache-aside requires the application to manually fetch from the DB and write to the cache, giving more control over invalidation.
How do you avoid a cache stampede?
Use request coalescing (only one request fetches from the DB while others wait), add a small random jitter to TTL, or refresh hot keys in the background before they expire.
What eviction policy is safest for most interview scenarios?
Least Recently Used (LRU) is a common default because it evicts items that haven’t been accessed recently, which aligns with typical read-heavy workloads.