Design a Feature Flag Service: The Junior, Senior and Staff Answer
A feature flag service lets you turn code paths on or off for specific users, groups, or environments without redeploying. In an interview you are expected to turn a vague requirement into a concrete, production-ready system. The question probes three things: your ability to size a service, to choose appropriate data stores and caching layers, and to think about operations and failure modes at scale.
What the interviewer is testing
- Scalability – can you handle millions of flags and high request rates?
- Consistency model – do you understand the trade-off between eventual consistency for flag evaluation and strong consistency for management writes?
- Operational awareness – can you reason about monitoring, rollout safety, and cost?
- Communication – can you walk through the request path step by step, naming components and explaining why each choice matters?
Core requirements and driving numbers
| Requirement | Typical ballpark | Why it matters |
|---|---|---|
| Number of distinct flags | ~10 million (order of magnitude) | Drives storage size and index design |
| Read QPS (flag evaluation) | 100 k req/s (≈0.01 req/s per flag on average) | Determines cache and DB read capacity |
| Write QPS (flag updates, rollouts) | 1 k req/s (≈0.0001 req/s per flag on average) | Affects write path latency and consistency |
| Availability | 99.9% (three nines) | Sets replication and failover requirements |
| Latency budget for evaluation | < 10 ms | Users see the flag instantly, so cache hit rate must be high |
These numbers are deliberately rough; they let you choose services that comfortably cover the load without over-provisioning.
Core design – request path, data model, key choice
1. Request path (evaluation)
- Client → API Gateway – expose a thin REST/GRPC endpoint. API Gateway provides throttling and TLS termination.
- API → Lambda (or container) – the evaluation logic runs here; it is cheap to scale and can be versioned.
- Lambda → ElastiCache (Redis) – first-level lookup for the flag value. Cache key is
flag:{flag_id}:env:{env}. - Cache miss → DynamoDB – the source of truth. Table schema:
PK = flag_id(string)SK = env#target(e.g.,prod#user:1234)- attributes:
value,rollout_percentage,conditions.
- DynamoDB → Lambda – result returned, Lambda writes back to Redis with a short TTL (seconds) for hot flags.
- Lambda → API Gateway → Client – response delivered.
2. Management path (create, update, rollout)
- Client → API Gateway → Lambda – admin API.
- Lambda → DynamoDB (write) – strong write, using conditional updates to avoid race conditions.
- DynamoDB Stream → SQS → Worker – async propagation of changes to all cache nodes. Workers invalidate or update Redis entries.
3. Data model highlights
- Flag granularity – each flag can have per-environment overrides and per-target (user, segment) rules.
- Rollout percentages – stored as a list of hash ranges; evaluation code hashes the user ID and checks inclusion.
- Versioning – each change creates a new
revisionattribute; old revisions stay for audit.
4. Key choice rationale
We pick a composite key (flag_id + env#target) over a simple hash because it lets us query all versions of a flag for a given environment without a full table scan. An alternative would be a single hash key with a JSON blob, but that makes partial updates expensive and hurts DynamoDB’s ability to scale reads.
What a complete junior answer covers
- Lists basic functional requirements (create, read, update, delete flags; per-environment support).
- Draws a simple two-tier architecture: API → DynamoDB → client.
- Mentions a cache (Redis) to meet latency.
- Shows a single request path diagram and explains each hop.
- Discusses rough capacity: with a cache hit ratio around 90 %, about 10 k of the 100 k reads/s reach DynamoDB → roughly 10 k RCUs for strongly consistent reads of items up to 4 KB (half that with eventually consistent reads).
- Identifies a few failure points: cache miss latency, DynamoDB throttling.
- Ends with a brief note on monitoring (CloudWatch metrics) and cost awareness.
What a senior answer adds
- Sharding strategy – partitions flags by hash of
flag_idacross multiple DynamoDB tables or partitions to avoid hot-spotting. - Consistency model – explains why eventual consistency is acceptable for flag evaluation but strong consistency is needed for admin writes; uses DynamoDB’s
ConditionExpression(conditional writes) to enforce it. - Cache invalidation – uses DynamoDB Streams → SQS → Lambda workers to push invalidations, reducing stale reads.
- Rollout patterns – describes percentage rollouts, canary releases, and targeting by segment (stored in a separate
segmentstable, cached in ElastiCache). - Operational metrics – adds latency SLOs, error-rate alerts, and a simple dashboard (CloudWatch Alarms + SNS).
- Security – IAM role separation for admin vs evaluation APIs, and API keys for client SDKs.
- Cost estimation – rough monthly cost for DynamoDB reads/writes and Redis nodes, showing that the design stays within a few thousand dollars.
What a staff answer adds
- Failure modes – models partial outage scenarios (e.g., Redis node loss, DynamoDB regional failover). Proposes multi-AZ Redis replication and DynamoDB global tables for cross-region resilience.
- Operational runbooks – outlines steps for a rollout rollback, cache warm-up scripts, and a chaos-engineering plan (e.g., injecting latency via AWS Fault Injection Simulator).
- Trade-off justification – defends the choice of DynamoDB over Aurora: DynamoDB offers predictable latency at scale and automatic sharding, whereas Aurora would add connection-pool complexity for high QPS.
- Cost-benefit analysis – compares on-demand vs provisioned capacity, showing that provisioned read capacity costs less than on-demand when traffic is steady, as it is at 100 k reads/s.
- Data durability – discusses TTL for old revisions, S3 backup of flag history, and periodic snapshots.
- Observability depth – adds X-Ray tracing for request path, custom metrics for cache hit ratio, and a dashboard that correlates rollout percentage with error spikes.
- Security hardening – uses AWS KMS to encrypt flag values at rest, and VPC endpoints for private connectivity.
Common mistakes that sink the question
| Mistake | Why it hurts |
|---|---|
| Ignoring cache entirely | Evaluation latency jumps to > 100 ms, violating the < 10 ms budget. |
| Storing flags as a single large JSON blob | Updates become write-amplified; a small change forces a rewrite of the whole item, which uses more write capacity and can cause throttling. |
| Assuming strong consistency everywhere | Increases latency and costs; most evaluations can tolerate eventual consistency. |
| Not planning for hot flags | A few popular flags can dominate read traffic; without sharding they create a hotspot in DynamoDB. |
| Forgetting operational concerns | Interviewers look for alerts, runbooks, and rollback safety; missing these suggests a design that won’t survive production. |
How to practice this
- Sketch the request path on a blank canvas, labeling each AWS component and the data flow.
- Write a short rubric: list functional, non-functional, and operational criteria you would evaluate.
- Use System Design AI to draw your own answer and get AI feedback on missing pieces or unclear trade-offs.
Questions people ask
What size should the Redis cache be for 100 k reads/s?
Roughly 10 % of flags are hot at any moment. With 10 M flags, caching 1 M entries (each ~200 B) needs ~200 MB of memory, plus overhead – a single cache node of 2 GB is sufficient with room for growth.
Why not use a relational database like Aurora?
Aurora provides strong consistency but requires connection pooling for high QPS, and scaling reads adds read replicas. DynamoDB offers auto-sharding and predictable single-digit millisecond latency, which matches the evaluation latency target.
How do you handle a gradual rollout without affecting latency?
Store rollout percentages as hash ranges and evaluate them in the Lambda layer; this is a pure computation step and adds negligible latency. The cache still returns the pre-computed value for the user’s segment.
Can the service be multi-region?
Yes. Use DynamoDB global tables for replicated state and Redis replication across regions via Amazon ElastiCache Global Datastore. Route 53 latency-based routing can send each user to the API Gateway endpoint in the nearest region.