System Design AI

Caching strategies in System Design Interviews: When to Use It and What to Say

When an interview asks you to design a high-traffic service, the first place you look for latency savings is the cache. A cache is a fast-access store that holds a copy of data that would otherwise require a slower round-trip to a database or another backend. It solves the classic "read-heavy, write-light" problem: the system can serve many requests without hammering the primary data store.

A concrete system: Photo-sharing feed

Imagine you are designing the home-feed for a photo-sharing app that shows the latest posts from a user’s friends. The feed is read millions of times per day, but new posts arrive only a few thousand times per hour. Without a cache, each feed request would trigger a database join across the user-friend table, the posts table, and a ranking engine – a costly operation that could take hundreds of milliseconds. By caching the pre-computed feed for each user, you can serve the request in single-digit milliseconds.

How caching works, step by step

  1. Client request – The front-end asks for GET /feed?user_id=123.
  2. Cache lookup – The application checks a fast key-value store (e.g., Redis) for a key like feed:123.
  3. Cache hit – If the key exists, the cached feed JSON is returned immediately.
  4. Cache miss – If the key is missing, the service queries the database, runs the ranking logic, and builds the feed.
  5. Populate cache – The freshly built feed is written back to the cache with an appropriate TTL (time-to-live).
  6. Return response – The feed is sent to the client. Subsequent requests hit the cache until the TTL expires or the cache is explicitly invalidated.

Worked mini-example

Suppose the feed is cached for 60 seconds. At 12:00:00 a request for user 123 arrives:

  • Cache miss → DB query returns posts A, B, C.
  • Service builds JSON and stores feed:123 = [A,B,C] with TTL=60.
  • Client receives the feed.

At 12:00:30 another request for user 123 arrives:

  • Cache hit → Redis returns [A,B,C] in ~2 ms.
  • No DB call is made.

At 12:01:00 the TTL expires. A request at 12:01:05 triggers a miss, pulls any new posts (say D), and refreshes the cache.

Where caching shows up in classic interview questions

Interview QuestionTypical Cache PlacementReason
Design Twitter timelineCache the timeline per user (timeline:user_id).Timeline reads dominate; writes are infrequent (new tweets).
Design URL shortenerCache the mapping short_code → long_url.Redirects are read-only; cache avoids DB round-trip for every click.
Design a video streaming serviceEdge CDN cache for video chunks.Static video files are served to millions; edge cache reduces latency and bandwidth.

In each case the cache sits right before the slowest component: the database for timelines, the key-value store for short URLs, and the edge network for video files.

Common caching strategies and their trade-offs

StrategyHow it worksConsistencyTypical Use
Read-through (lazy)Miss triggers backend fetch, then populates cache.Stale data until TTL expires; possible cache stampede.Hot reads with occasional writes (e.g., product catalog).
Write-throughEvery write updates both backend and cache synchronously.Stronger consistency; higher write latency.Frequently updated data where freshness matters (e.g., user profile).
Write-behind (async)Writes go to the cache first; the backend is updated later by an asynchronous flush.Cache is current but the backend lags; risk of lost writes if the cache fails before the flush.High-throughput write streams where occasional lag is acceptable.
Cache-asideApplication manually reads/writes cache; explicit invalidation.Full control over staleness; extra code complexity.Complex invalidation rules (e.g., permission changes).
Edge CDNDistributed nodes cache static assets close to the user.Content must be immutable or versioned; cache purge latency.Large static files (images, JS, video).

The alternative interviewers often ask about

When you mention a cache, interviewers will probe the eviction policy (LRU, LFU, TTL) and the cache invalidation mechanism. They may also ask how you would handle a cache miss burst (the “thundering herd” problem) – a typical answer is to use request coalescing or a per-key lock, so that only one request rebuilds the value.

A 30-second spoken pitch

"In a read-heavy service, I would add a read-through cache in front of the database. The request first checks a fast key-value store; on a hit we return the cached value in a few milliseconds. On a miss we fetch from the DB, build the response, write it back to the cache with a short TTL, and return it. This reduces latency, cuts DB load, and smooths traffic spikes. Trade-offs are staleness and eviction complexity, so I’d choose a TTL that matches the freshness requirement and add a background refresh to avoid stampedes."

Mistakes candidates often make

  • Over-caching – Caching everything, including rapidly changing data, leads to stale responses and hard-to-debug bugs.
  • Ignoring eviction – Assuming the cache is infinite; in reality you must pick an eviction policy and size the cache based on traffic patterns.
  • Claiming zero latency – Even an in-memory cache adds a few milliseconds; exaggerating performance looks naive.
  • Skipping invalidation – Forgetting to purge or update the cache after a write causes the classic "old data after update" bug.
  • Not addressing cache stampede – When a key expires, many requests can hit the DB simultaneously; a proper solution (e.g., request coalescing) should be mentioned.

How to practice this

  1. Study a worked design – Open the System Design AI library and review the Cache-Aside (Lazy Loading) building block. Follow its walkthrough and click each component to read its notes.
  2. Sketch the flow – On the canvas, draw a request path that includes a cache lookup, a miss path, and the write-back step. Label each component with the talking points above.
  3. Run a mock interview – Explain the cache choice aloud, then answer follow-up questions about TTL, eviction, and cache-stampede mitigation. Repeat until the explanation fits comfortably in 30 seconds.

Questions people ask

When should I suggest a cache in a design interview?

Suggest a cache when the read volume is much higher than the write volume, latency is a key metric, and the data can tolerate a short period of staleness.

What is the difference between read-through and cache-aside?

Read-through automatically populates the cache on a miss, while cache-aside requires the application to manually fetch from the DB and write to the cache, giving more control over invalidation.

How do you avoid a cache stampede?

Use request coalescing (only one request fetches from the DB while others wait), add a small random jitter to TTL, or refresh hot keys in the background before they expire.

What eviction policy is safest for most interview scenarios?

Least Recently Used (LRU) is a common default because it evicts items that haven’t been accessed recently, which aligns with typical read-heavy workloads.