System Design AI

Circuit Breakers in System Design Interviews: When to Use Them and What to Say

A circuit breaker is a pattern that stops calls to a downstream component once that component is repeatedly failing, and optionally provides a fallback while it recovers.

The problem it solves

When a service depends on another service, a failure cascade can bring down the whole system. Imagine an e-commerce checkout service that calls a payment gateway, an inventory service, and a shipping estimator. If the inventory service suddenly becomes slow or unresponsive, each request to checkout will block, tying up threads and inflating latency. The checkout service may time out, and the overload can spread to the load balancer and eventually the front-end servers. A circuit breaker prevents this by short-circuiting calls after a configurable error threshold, returning a fast error or fallback response instead of waiting for the downstream timeout.

How it works – step by step

  1. Closed state – All calls pass through. The breaker records success and failure counts over a sliding window.
  2. Threshold breach – If failures exceed a configurable percentage (e.g., 50 % of the last 20 requests) and the request volume is above a minimum, the breaker trips.
  3. Open state – New calls are immediately rejected (or routed to a fallback) without touching the downstream service. The response is typically a fast error like HTTP 503.
  4. Half-open state – After a cool-down period, the breaker allows a small number of trial calls. If they succeed, it closes again; if they fail, it returns to open.
  5. Fallback logic – The caller can provide a cached response, a default value, or a graceful degradation path.

Small worked example

Suppose a microservice "UserProfile" calls a third-party "Avatar" service to fetch a profile picture.

  • Window: last 10 requests, failure threshold: 40 %.
  • Requests 1-6 succeed, request 7 times out, 8 times out, 9 times out, 10 times out.
  • Failure rate = 4/10 = 40 % → breaker opens.
  • Requests 11-15 are immediately answered with a placeholder image (fallback) and never reach Avatar.
  • After 30 seconds, the breaker enters half-open; request 16 is allowed through. If Avatar returns a picture, the breaker closes; otherwise it stays open.

Where circuit breakers appear in classic interview questions

Design questionWhere the breaker is usedReason for inclusion
API Gateway for a SaaS platformAt the gateway before calling downstream microservices (e.g., billing, analytics)Shows you can protect the edge from a single flaky service.
Message-driven order processingAround the consumer that writes to an external payment providerDemonstrates safe handling of external APIs that may rate-limit or timeout.
Real-time chat serviceIn the WebSocket server when broadcasting to a presence serviceHighlights you can keep the chat latency low even if presence lookup fails.

Trade-offs and the alternative interviewers love to ask about

AspectCircuit breakerAlternative (e.g., retry + exponential back-off)
Latency impactImmediate failure in open state, low tail latency.Retries add extra latency; may worsen load on the failing service.
Resource usageSaves threads/CPU by not waiting on timeouts.Retries consume more resources while waiting.
ComplexityRequires state tracking, timers, and fallback logic.Simpler to implement; just wrap calls with retry logic.
False positivesShort windows or low thresholds can open on transient spikes.Retries may mask problems, leading to hidden overload.

Interviewers often follow up with questions like:

  • What if the fallback also fails? – you can chain another breaker or use a bulkhead.
  • How do you choose the window size and thresholds? – base it on observed latency histograms and error budgets.
  • What about state replication in a distributed system? – store breaker metrics locally per instance; for cross-region safety, use a shared store like DynamoDB or Redis.

30-second spoken version

"A circuit breaker monitors calls to a downstream service. When failures exceed a set threshold, it opens and instantly returns a fallback, preventing the caller from hanging. After a cool-down, it probes with a few trial calls; if they succeed, it closes again. This protects the system from cascading failures, reduces latency, and isolates problems, but you must tune thresholds to avoid false trips."

Common mistakes candidates make

  • Treating the breaker as a silver bullet – forgetting to also add retries, timeouts, and bulkheads.
  • Hard-coding thresholds – saying “5 % failure rate” without justification; interviewers expect a rationale based on traffic patterns.
  • Ignoring fallback design – simply returning an error instead of a graceful degradation path.
  • Placing the breaker at the wrong layer – putting it inside the downstream service instead of at the caller, which defeats the purpose.
  • Not discussing state sync – forgetting to mention how the breaker state is kept consistent across multiple instances.

How to practice this

  1. Pick a classic design question (e.g., API gateway) and draw the full component diagram on a canvas.
  2. Identify the call paths that could fail and insert a circuit breaker node with its state transitions.
  3. Use System Design AI’s practice brief to run through a mock interview: explain the breaker, justify thresholds, and answer follow-up trade-off questions.

Questions people ask

When should I add a circuit breaker instead of just increasing timeout values?

Add a breaker when a downstream service can become slow enough to tie up your threads or when you need a fast fallback. Timeouts only limit per-request latency; they don’t stop cascading failures.

How do I choose the failure threshold and window size?

Pick a window that covers a few minutes of traffic (e.g., 100 requests) and a failure percentage that aligns with your SLA error budget, often 5-10 %. Tune based on observed spike frequency and latency histograms.

Can a circuit breaker be used for database connections?

It can, but connection pools already limit concurrent attempts. A breaker is useful if the database repeatedly times out or refuses connections, providing a fast fallback instead of waiting on each request.

What is the difference between a circuit breaker and a bulkhead?

A bulkhead isolates resources per component, preventing one service from exhausting shared threads or connections. A circuit breaker stops calls after a failure threshold, focusing on latency and error propagation.