System Design AI

Caching Strategies: Follow-Up Questions Interviewers Ask (and How to Answer)

When you mention caching in a design interview, the interviewer immediately starts probing. Caching is a lever that can shave milliseconds off latency, reduce load on downstream services, and lower cloud spend – but it also introduces complexity around staleness, invalidation, and capacity planning. By asking follow-up questions, interviewers test whether you understand those trade-offs or are just naming a buzzword.

The goal of the probing is to see if you can reason about the whole system, not just the cache layer. They want to know whether you can pick the right cache tier, size it, handle failures, and keep the data model consistent under load. Below are the typical follow-up questions, ordered from the most common to the hardest, each answered as a candidate would respond.

Cache Placement and Scope

1. Where in the architecture should we place the cache?

I would start by placing a read-through cache close to the application tier, typically using an in-memory store like Redis or Memcached. For highly read-heavy endpoints, I’d also add a CDN edge cache for static assets. This layered approach isolates latency: the edge cache serves sub-second responses for global users, while the in-memory cache reduces round-trip time to the database from ~30 ms to under 1 ms for hot keys. The exact placement depends on the request path and where the bottleneck appears.

2. Should we use a client-side cache, an edge cache, or a server-side cache?

I’d combine all three where it makes sense. Client-side caches (e.g., browser HTTP cache) are great for immutable resources and can offload traffic entirely. Edge caches (CDNs) handle geographically dispersed traffic and reduce latency for static files. Server-side caches like Redis handle dynamic data that changes frequently but still benefits from fast lookup. For a product with 10 M daily reads, a CDN could serve roughly 80 % of the traffic, while Redis handles the remaining 20 % of personalized queries.

Cache Size and Eviction

3. How do you decide the cache size and eviction policy?

I begin by profiling the working set – the fraction of data accessed repeatedly. If the hot 5 % of keys account for 80 % of reads, I size the cache to hold that 5 % plus a safety margin, say 1.5×, to avoid churn. For eviction, I prefer LRU (least recently used) because it aligns with typical access patterns where recent items are likely to be accessed again. If a small set of keys stays popular over long periods, I might switch to LFU (least frequently used) so that a burst of one-off reads does not evict them.

4. What happens when the cache is warm versus cold?

When the cache is warm, hit rates can exceed 90 %, delivering sub-millisecond responses. In a cold start, the first request for each key incurs a full backend fetch, which can increase latency by an order of magnitude. To mitigate cold starts, I pre-warm the cache during deployment by loading the top-N keys, and I use a “cache-aside” pattern so that the application populates the cache on the first miss.

Consistency and Invalidation

5. How do you keep the cache consistent with the source of truth?

I use a write-through strategy where every update writes to the database and then to the cache in the same request. The two writes are not one atomic transaction, so I also keep a TTL as a safety net for the rare case where the cache write fails. For cases where write-through is too slow, I fall back to write-behind with a short TTL (e.g., 30 seconds) and a background invalidation job that publishes change events via a message bus. This hybrid approach balances latency with eventual consistency.

6. What invalidation mechanisms would you employ?

I’d start with TTL-based expiration for data that tolerates staleness, such as product listings that change a few times per hour. For strongly consistent data, I’d use explicit invalidation via a pub/sub channel: when a record updates, the service publishes an invalidation message, and all cache nodes evict the key immediately. This pattern scales to thousands of nodes because the message bus (e.g., Kafka) handles fan-out efficiently.

Failure Modes and Resilience

7. How do you handle cache failures?

If the cache becomes unavailable, the application should gracefully fall back to the database using a circuit-breaker pattern. I’d set a short timeout (e.g., 5 ms) for cache calls; on timeout, the request proceeds directly to the DB, and the failure is logged. The circuit-breaker trips after a configurable error rate, preventing cascading delays. This ensures the system remains functional, albeit with higher latency, rather than failing outright.

8. What about cache stampede on a miss?

To avoid a stampede, I’d implement request coalescing: the first request that misses acquires a lock (e.g., using Redis SETNX) and populates the cache, while subsequent requests wait or serve stale data. Additionally, I could use “probabilistic early expiration” where each request randomly expires a key slightly before its TTL, spreading load over time. This technique reduces the chance that many clients simultaneously trigger a heavy backend fetch.

Advanced Trade-offs

9. How do you decide between a distributed cache and a local in-process cache?

A distributed cache like Redis is essential when multiple application instances need a shared view of hot data, such as user session state. However, for ultra-low latency, a local in-process cache (e.g., Guava) can serve the hottest keys without network hop, achieving sub-microsecond lookups. The trade-off is consistency: local caches must be invalidated via a broadcast channel, which adds complexity. In practice, I keep a small local cache for the top-1 % of keys and rely on Redis for the rest, achieving a 2× speed-up for the most frequent reads while preserving consistency.

Senior vs. Staff Answers

A senior-level answer explains what to do and why it fits the product’s constraints. It references concrete numbers, shows awareness of trade-offs, and offers a clear implementation plan. A staff-level answer goes further: it evaluates system-wide impact, anticipates edge cases, and discusses how the caching choice interacts with monitoring, capacity planning, and cost. Staff candidates also articulate how to evolve the cache strategy as traffic patterns shift, and they may propose automated experiments to validate assumptions.

How to practice this

  1. Review the Cache-Aside (Lazy Loading) building block in the System Design AI library; click each component to read its notes.
  2. Sketch a full request flow on the canvas, placing caches at different tiers and writing brief notes on eviction, consistency, and failure handling.
  3. Run through the nine follow-up questions out loud, refining your answers until you can cite concrete numbers and trade-offs without hesitation.

Questions people ask

What is the difference between read-through and write-through caching?

Read-through loads data into the cache on a miss, while write-through writes to both the cache and the backend on every update. Read-through keeps writes cheap but can serve stale data until the entry expires or is invalidated; write-through keeps the cache fresh but adds latency on writes.

When should I use TTL versus explicit invalidation?

Use TTL for data that can tolerate bounded staleness, such as product listings that change hourly. Use explicit invalidation for strongly consistent data where any lag could cause errors, like user permissions.

How can I prevent a cache stampede?

Implement request coalescing with a lock on the first miss, and consider probabilistic early expiration so that keys expire slightly before their TTL, spreading load over time.

Is a local in-process cache worth the added complexity?

Yes, when a small set of hot keys accounts for a large share of total traffic and you need sub-microsecond latency. Keep the local cache small and use a broadcast channel to invalidate entries, accepting the added coordination cost.