System Design AI

Feature Flags: Follow-Up Questions Interviewers Ask, With Answers

Feature flags are a go-to tool for incremental rollouts, A/B testing, and quick rollbacks. In a design interview, mentioning them signals that you understand how to ship changes safely. Interviewers follow up because a flag is only as good as the surrounding processes – monitoring, consistency, and governance. They want to know whether you have thought through the operational impact, not just the code change.

The follow-up questions usually start simple – “how do you store flags?” – and quickly move to the hard problems of scaling, consistency across data centers, and ownership. Below is a typical progression of probes and concise candidate answers that demonstrate the right level of detail.

1. How do you store feature-flag configuration?

Answer

I store flag definitions in a strongly consistent key-value store such as DynamoDB with a primary key of service:flagName. The value contains a JSON payload describing the flag state, rollout percentages, and target audience. To keep read latency low for high-traffic services, I cache the flag data in an in-memory store like Redis with a TTL of 60 seconds, refreshing the cache on write-through. This setup gives sub-millisecond reads while ensuring that any change propagates within a minute.

2. How do you ensure low latency reads for flags?

Answer

I place a read-through cache at the edge of each service. When a request needs a flag, the service first checks its local in-process cache; if missing, it falls back to a Redis cluster that is co-located with the service’s VPC. The cache is refreshed via a publish/subscribe channel (e.g., SNS → SQS) that pushes updates immediately after a write. In practice, the hot path is an in-process memory read that takes microseconds, and only a local cache miss pays the sub-millisecond Redis round trip.

3. How do you roll out a flag gradually?

Answer

I implement percentage rollouts by storing a random bucket (0-99) for each user in a separate table. The flag payload includes a rollout percentage, say 30 %. When evaluating the flag, the service checks if the user’s bucket is less than 30. This deterministic approach guarantees that a given user sees a consistent experience across requests, while the overall traffic exposed to the flag grows smoothly as we increase the percentage.

4. What happens if a flag update fails mid-deployment?

Answer

All flag writes go through a transactional write to DynamoDB, ensuring atomicity. If the write fails, the client receives an error and retries with exponential backoff. Meanwhile, the cache still serves the previous version, so there is no partial state visible to users. I also instrument a health check that alerts if a flag’s version diverges between the DB and cache for longer than a minute.

5. How do you handle flag consistency across multiple data centers?

Answer

I replicate the flag store using DynamoDB global tables, which provide eventual consistency across regions within a few seconds. For stricter consistency, I use a write-ahead log (WAL) in Kafka that streams flag changes to all regions; each region’s consumer applies the change to its local store in order. In practice, this gives us <5 second window for a flag to be visible globally, which is acceptable for most feature-flag use cases.

Answer

I categorize flags by risk level. High-risk flags (e.g., those that affect authentication) are guarded by a circuit-breaker that defaults to the safe state if the flag service is unavailable. Additionally, I add a “kill-switch” that can be toggled via a separate admin endpoint, bypassing the cache and forcing the flag to a known safe value instantly. This dual-layer safety ensures that even a catastrophic bug can be mitigated in under a second.

7. How do you audit flag changes and enforce ownership?

Answer

Every flag change is required to go through a pull-request workflow with code owners approving the modification. The change is logged in CloudTrail, capturing who changed what and when. I also tag each flag with an ownerTeam attribute and enforce a policy that only the owning team can modify its flags via IAM role restrictions. This audit trail makes it easy to trace any incident back to the responsible team.

8. How do you scale flag evaluation to billions of requests per day?

Answer

For massive traffic, I move the flag check into a CDN edge function (e.g., CloudFront Lambda@Edge). The function reads a compact, pre-computed bitmap of enabled users for the flag from an S3 object that is refreshed every few minutes. This eliminates any round-trip to a backend store, delivering flag decisions in <2 ms at the edge. At roughly 1 billion daily requests, which is about 12 k requests per second, this keeps flag evaluation off the backend entirely.

9. How do you retire a flag and clean up associated data?

Answer

When a flag reaches 100 % rollout and is stable for a week, I mark it as deprecated and schedule a background job that removes the flag entry from DynamoDB and clears related cache entries. The job also deletes the user bucket table entries that were only needed for that flag. I keep the deletion window short – typically 48 hours – to avoid orphaned data while giving teams a buffer to verify the flag is truly unnecessary.

What separates a senior answer from a staff answer?

A senior candidate will demonstrate solid knowledge of the mechanics: storage choices, caching, rollout strategies, and safety nets. A staff-level answer goes further – it discusses cross-region consistency models, governance at scale, and architectural trade-offs such as moving flag evaluation to the edge. Staff candidates also talk about measuring impact (e.g., latency budgets) and designing a flag-service ownership model that scales with many teams.

How to practice this

  1. Sketch a full flag-driven architecture on a canvas, labeling the store, cache, and edge layer.
  2. Write out concise answers for each of the nine follow-up questions, inserting concrete numbers like latency targets or request rates.
  3. Use System Design AI’s practice brief to run through the scenario and get AI feedback on your explanations.

Questions people ask

What is the main purpose of a feature flag?

Feature flags let you enable or disable functionality at runtime, allowing safe rollouts, A/B tests, and instant rollbacks without redeploying code.

Can feature flags replace automated testing?

No. Flags control exposure, but you still need unit, integration, and end-to-end tests to verify correctness before a flag is turned on.

How do you avoid flag explosion?

Enforce a lifecycle: create, use, deprecate, and delete. Use tooling that flags stale entries after a defined period.

Is it safe to store flags in a relational database?

Yes, if you need strong consistency and transactional updates, but you should add a caching layer to meet low-latency requirements.