System Design AI

Design a Feature Flag Service: The Junior, Senior and Staff Answer

A feature flag service lets you turn code paths on or off for specific users, groups, or environments without redeploying. In an interview you are expected to turn a vague requirement into a concrete, production-ready system. The question probes three things: your ability to size a service, to choose appropriate data stores and caching layers, and to think about operations and failure modes at scale.

What the interviewer is testing

  • Scalability – can you handle millions of flags and high request rates?
  • Consistency model – do you understand the trade-off between eventual consistency for flag evaluation and strong consistency for management writes?
  • Operational awareness – can you reason about monitoring, rollout safety, and cost?
  • Communication – can you walk through the request path step by step, naming components and explaining why each choice matters?

Core requirements and driving numbers

RequirementTypical ballparkWhy it matters
Number of distinct flags~10 million (order of magnitude)Drives storage size and index design
Read QPS (flag evaluation)100 k req/s (≈0.01 req/s per flag on average)Determines cache and DB read capacity
Write QPS (flag updates, rollouts)1 k req/s (≈0.0001 req/s per flag on average)Affects write path latency and consistency
Availability99.9% (three nines)Sets replication and failover requirements
Latency budget for evaluation< 10 msUsers see the flag instantly, so cache hit rate must be high

These numbers are deliberately rough; they let you choose services that comfortably cover the load without over-provisioning.

Core design – request path, data model, key choice

1. Request path (evaluation)

  1. Client → API Gateway – expose a thin REST/GRPC endpoint. API Gateway provides throttling and TLS termination.
  2. API → Lambda (or container) – the evaluation logic runs here; it is cheap to scale and can be versioned.
  3. Lambda → ElastiCache (Redis) – first-level lookup for the flag value. Cache key is flag:{flag_id}:env:{env}.
  4. Cache miss → DynamoDB – the source of truth. Table schema:
  • PK = flag_id (string)
  • SK = env#target (e.g., prod#user:1234)
  • attributes: value, rollout_percentage, conditions.
  1. DynamoDB → Lambda – result returned, Lambda writes back to Redis with a short TTL (seconds) for hot flags.
  2. Lambda → API Gateway → Client – response delivered.

2. Management path (create, update, rollout)

  1. Client → API Gateway → Lambda – admin API.
  2. Lambda → DynamoDB (write) – strong write, using conditional updates to avoid race conditions.
  3. DynamoDB Stream → SQS → Worker – async propagation of changes to all cache nodes. Workers invalidate or update Redis entries.

3. Data model highlights

  • Flag granularity – each flag can have per-environment overrides and per-target (user, segment) rules.
  • Rollout percentages – stored as a list of hash ranges; evaluation code hashes the user ID and checks inclusion.
  • Versioning – each change creates a new revision attribute; old revisions stay for audit.

4. Key choice rationale

We pick a composite key (flag_id + env#target) over a simple hash because it lets us query all versions of a flag for a given environment without a full table scan. An alternative would be a single hash key with a JSON blob, but that makes partial updates expensive and hurts DynamoDB’s ability to scale reads.

What a complete junior answer covers

  • Lists basic functional requirements (create, read, update, delete flags; per-environment support).
  • Draws a simple two-tier architecture: API → DynamoDB → client.
  • Mentions a cache (Redis) to meet latency.
  • Shows a single request path diagram and explains each hop.
  • Discusses rough capacity: with a cache hit ratio around 90 %, about 10 k of the 100 k reads/s reach DynamoDB → roughly 10 k RCUs for strongly consistent reads of items up to 4 KB (half that with eventually consistent reads).
  • Identifies a few failure points: cache miss latency, DynamoDB throttling.
  • Ends with a brief note on monitoring (CloudWatch metrics) and cost awareness.

What a senior answer adds

  • Sharding strategy – partitions flags by hash of flag_id across multiple DynamoDB tables or partitions to avoid hot-spotting.
  • Consistency model – explains why eventual consistency is acceptable for flag evaluation but strong consistency is needed for admin writes; uses DynamoDB’s ConditionExpression (conditional writes) to enforce it.
  • Cache invalidation – uses DynamoDB Streams → SQS → Lambda workers to push invalidations, reducing stale reads.
  • Rollout patterns – describes percentage rollouts, canary releases, and targeting by segment (stored in a separate segments table, cached in ElastiCache).
  • Operational metrics – adds latency SLOs, error-rate alerts, and a simple dashboard (CloudWatch Alarms + SNS).
  • Security – IAM role separation for admin vs evaluation APIs, and API keys for client SDKs.
  • Cost estimation – rough monthly cost for DynamoDB reads/writes and Redis nodes, showing that the design stays within a few thousand dollars.

What a staff answer adds

  • Failure modes – models partial outage scenarios (e.g., Redis node loss, DynamoDB regional failover). Proposes multi-AZ Redis replication and DynamoDB global tables for cross-region resilience.
  • Operational runbooks – outlines steps for a rollout rollback, cache warm-up scripts, and a chaos-engineering plan (e.g., injecting latency via AWS Fault Injection Simulator).
  • Trade-off justification – defends the choice of DynamoDB over Aurora: DynamoDB offers predictable latency at scale and automatic sharding, whereas Aurora would add connection-pool complexity for high QPS.
  • Cost-benefit analysis – compares on-demand vs provisioned capacity, showing that provisioned read capacity costs less than on-demand when traffic is steady, as it is at 100 k reads/s.
  • Data durability – discusses TTL for old revisions, S3 backup of flag history, and periodic snapshots.
  • Observability depth – adds X-Ray tracing for request path, custom metrics for cache hit ratio, and a dashboard that correlates rollout percentage with error spikes.
  • Security hardening – uses AWS KMS to encrypt flag values at rest, and VPC endpoints for private connectivity.

Common mistakes that sink the question

MistakeWhy it hurts
Ignoring cache entirelyEvaluation latency jumps to > 100 ms, violating the < 10 ms budget.
Storing flags as a single large JSON blobUpdates become write-amplified; a small change forces a rewrite of the whole item, which uses more write capacity and can cause throttling.
Assuming strong consistency everywhereIncreases latency and costs; most evaluations can tolerate eventual consistency.
Not planning for hot flagsA few popular flags can dominate read traffic; without sharding they create a hotspot in DynamoDB.
Forgetting operational concernsInterviewers look for alerts, runbooks, and rollback safety; missing these suggests a design that won’t survive production.

How to practice this

  1. Sketch the request path on a blank canvas, labeling each AWS component and the data flow.
  2. Write a short rubric: list functional, non-functional, and operational criteria you would evaluate.
  3. Use System Design AI to draw your own answer and get AI feedback on missing pieces or unclear trade-offs.

Questions people ask

What size should the Redis cache be for 100 k reads/s?

Roughly 10 % of flags are hot at any moment. With 10 M flags, caching 1 M entries (each ~200 B) needs ~200 MB of memory, plus overhead – a single cache node of 2 GB is sufficient with room for growth.

Why not use a relational database like Aurora?

Aurora provides strong consistency but requires connection pooling for high QPS, and scaling reads adds read replicas. DynamoDB offers auto-sharding and predictable single-digit millisecond latency, which matches the evaluation latency target.

How do you handle a gradual rollout without affecting latency?

Store rollout percentages as hash ranges and evaluate them in the Lambda layer; this is a pure computation step and adds negligible latency. The cache still returns the pre-computed value for the user’s segment.

Can the service be multi-region?

Yes. Use DynamoDB global tables for replicated state and Redis replication across regions via Amazon ElastiCache Global Datastore. Route 53 latency-based routing can send each user to the API Gateway endpoint in the nearest region.