System Design AI

Observability: Follow-Up Questions Interviewers Ask (And How to Answer)

Observability is a cheap way to earn trust in a complex service. When a candidate brings it up, interviewers immediately dig deeper because the real value shows up in the details: what you measure, how you store it, and how you act on it. They want to know whether you treat observability as an after-thought or as a design pillar.

The follow-up questions also reveal how you balance cost, performance, and reliability. A senior candidate can list the right components; a staff-level candidate will also explain the why behind each choice and how the system evolves over time.

Why interviewers probe observability

Observability is the glue between architecture and operations. Without it, you cannot detect latencies, spot error spikes, or verify SLAs. Interviewers ask follow-ups to ensure you understand the end-to-end data flow, the granularity needed for debugging, and the operational processes that keep a service healthy.

They also look for signs that you think about scaling the observability stack itself. A design that logs every request to a single Elasticsearch cluster will break under load. The questions test whether you anticipate those limits and have a plan to evolve the stack.

What interviewers typically ask

1. Which core metrics would you collect for a typical request-response service?

I would collect latency, error rate, and request volume per endpoint. Latency gives us the user experience; error rate signals reliability problems; request volume helps with capacity planning. For example, I would instrument a histogram for latency with buckets at 1 ms, 10 ms, 100 ms, and 1 s, plus finer buckets around the target, and expose a 99th-percentile SLA of 200 ms.

2. How would you store and query these metrics at scale?

I would push the metrics to a time-series database like Amazon Timestream or Prometheus-compatible storage. They compress efficiently and support range queries. For a service handling roughly 5 M requests per day (≈58 req/s), a month of data would be on the order of a few gigabytes, which is well within those systems’ retention capabilities.

3. What alerts would you set up, and how would you avoid alert fatigue?

I would set alerts on latency > 95th-percentile exceeding 300 ms for more than five minutes, and error rate > 0.5% sustained for ten minutes. To avoid fatigue, I’d use multi-stage alerts: a warning threshold first, then a critical threshold, and finally a page only if the condition persists beyond the escalation window.

4. How do you ensure traceability across microservices?

I would propagate a unique request ID (e.g., UUID) via HTTP headers and enable distributed tracing with AWS X-Ray or OpenTelemetry. Each service would add a span to the trace, allowing us to reconstruct the full call graph. In a typical scenario with three hops, the end-to-end trace latency would be visible, helping pinpoint the slowest component.

5. What logs would you collect, and how would you balance verbosity with cost?

I would log structured events at INFO level for successful requests and ERROR level for failures, including request ID, endpoint, status code, and latency. For high-traffic paths, I’d sample logs at 1 % to keep storage costs low while still capturing enough data for anomaly detection. This approach reduces daily log volume from potentially terabytes to a manageable few hundred gigabytes.

6. How would you handle a sudden spike in error rate?

First, the alert would trigger an automated runbook that checks recent deployments and rolls back if a new version correlates with the spike. Simultaneously, I’d query the trace store for the affected endpoints to see where latency or error bursts originate. If the issue is downstream, I’d circuit-break the call and fallback to a cached response.

7. How do you verify that your observability stack itself is healthy?

I would instrument the observability components (metrics DB, log pipeline, tracing collector) with self-monitoring metrics: ingestion lag, query latency, and error rate. For example, if the ingestion lag for logs exceeds 30 seconds, an alert would fire. A health dashboard would show these metrics alongside the service’s own KPIs.

8. What trade-offs do you consider when choosing between push vs. pull metrics collection?

Push models (e.g., StatsD) reduce latency but can overload the collector during traffic bursts. Pull models (e.g., Prometheus scraping) give the collector control over load but add network overhead. I’d use push for high-frequency, low-overhead counters and pull for long-running services, where scrape load depends on the number of targets and the scrape interval rather than on traffic, so it stays flat even at a peak of roughly 1 k requests per second for a service handling 10 M daily requests.

9. How would you evolve the observability system as the product grows to billions of requests per day?

I would shard the time-series store by region and introduce a hierarchical aggregation layer that pre-computes daily and hourly rollups, reducing query load. I’d also adopt a sampling strategy for traces, keeping full traces for 0.1 % of requests but retaining all error traces. Finally, I’d automate capacity planning using the observed request volume trend, scaling the storage tier on a schedule that matches the growth curve.

Senior vs. staff answers

A senior-level answer lists the right metrics, tools, and basic alert thresholds. It shows competence in building a functional observability pipeline. A staff-level answer goes further: it explains the rationale behind each metric, discusses cost-benefit analysis, outlines self-monitoring of the observability stack, and presents a roadmap for scaling the system. The staff answer also ties observability back to business goals, such as meeting a 99.9 % SLA.

How to practice this

  1. Study the worked design on observability in the System Design AI library. Review the component notes and the diagram that builds the request path.
  2. Write your own answer on the canvas, covering metrics, storage, alerts, and scaling. Use the rubric to check completeness.
  3. Run the AI coach on your canvas. Incorporate the feedback, focusing on adding strategic trade-offs and evolution plans.

Questions people ask

What is the difference between metrics and logs?

Metrics are numeric aggregates like latency percentiles or request counts. They are cheap to store, fast to query, and ideal for dashboards and alerts. Logs are detailed, immutable records of each event, useful for deep debugging but more expensive to retain.

Why use distributed tracing instead of just logs?

Tracing links spans across services, showing the end-to-end latency and call graph for a request. It lets you pinpoint the slowest hop without manually correlating log timestamps, which speeds up root-cause analysis.

How often should I sample logs in a high-traffic service?

A typical pattern is to sample normal traffic at about 1 % and capture 100 % of error paths. This reduces daily log volume from potentially terabytes to a few hundred gigabytes while keeping enough data for anomaly detection.

Can I rely on a single observability tool for metrics, logs, and traces?

Integrated platforms exist, but separating metrics, logs, and tracing often yields better performance and flexibility. Dedicated time-series storage for metrics and a separate log pipeline prevent one workload from impacting the other.