System Design AI

Design a Key-Value Store: Junior, Senior, and Staff Answers

When an interview asks you to design a key-value store you’re being evaluated on three things: how you translate vague requirements into concrete numbers, how you build a request path that scales, and how you think about operations and trade-offs. The question is simple on the surface, but the depth of your answer shows whether you understand distributed systems fundamentals.

What the Interviewer Is Testing

  • Scalability – Can you grow read/write capacity from a few hundred QPS to millions?
  • Consistency & durability – Do you know the difference between eventual and strong consistency and when each is acceptable?
  • Operational awareness – Do you consider monitoring, failure handling, and cost?
  • Communication – Can you walk through a request step-by-step and explain each component’s role?

Core Requirements and Driving Numbers

RequirementTypical BallparkWhy It Matters
Throughput10 K reads / sec, 5 K writes / sec (≈ 1.3 B ops per day)Drives number of partitions and cache size.
Latency< 10 ms read, < 20 ms write (95th percentile)Determines whether you need a fast cache layer.
Data sizeKeys ~ 64 B, values up to 1 MB, total stored data ≈ 10 TBAffects storage choice (SSD vs. HDD) and replication factor.
Availability99.9 % uptime (≈ 8 h downtime per year)Sets replication and failover strategy.

These numbers are not exact; they give you a sense of the scale you need to support. Use them to decide how many shards, what cache capacity, and which AWS services fit.

Core Design Walkthrough

1. Request Path – One Hop at a Time

  1. Client sends a GET or PUT request over HTTPS to an API Gateway.
  2. API Gateway forwards the request to a Lambda (or an EC2 service) that acts as the request router.
  3. The router hashes the key (e.g., Murmur3) to determine the target shard.
  4. The router looks up the shard location in a DynamoDB table that maps hash ranges to ElastiCache (Redis) clusters for hot keys.
  5. If the key is cached, the router reads/writes directly to Redis (single hop). If not, it forwards to the primary storage – a DynamoDB table or an Aurora Serverless cluster for larger values.
  6. Write-through or write-behind caching updates the Redis entry and persists to the backing store.
  7. The response travels back through the same path to the client.

2. Data Model

  • Key: opaque binary string, limited to 64 B. Chosen because most workloads use short identifiers (user IDs, session tokens). Longer keys would increase hash distribution overhead.
  • Value: binary blob up to 1 MB. Stored as a DynamoDB item attribute when small, or as an S3 object reference for larger values, because a DynamoDB item is capped at 400 KB. Using S3 for large blobs offloads DynamoDB throughput.

3. Key Choice and Alternatives

  • Hash-based partitioning vs. range-based: Hashing spreads load evenly, avoiding hotspot hotkeys that a range scheme could create.
  • Composite keys (e.g., userId#timestamp) are useful for versioned data but add complexity; a simple opaque key keeps the design clean for a baseline answer.

What a Complete Junior Answer Covers

  • A single DynamoDB table (or a single Aurora instance) with a basic read-through cache.
  • Simple request flow: client → API Gateway → Lambda → DynamoDB.
  • Mention of latency and throughput numbers, but no explicit sharding.
  • Basic durability: DynamoDB’s built-in replication.
  • Minimal operational concerns: CloudWatch metrics for request count and error rate.

What a Senior Answer Adds

  • Sharding: Split the key space into N shards by hashing the key. DynamoDB does this for you by hashing the partition key across its internal partitions; a self-managed store needs its own shard map and a rebalancing plan.
  • Caching layer: ElastiCache (Redis) for hot keys, reducing read latency to < 5 ms.
  • Write path: Write-behind caching with a background worker (SQS + Lambda) that persists to DynamoDB, handling bursts.
  • Consistency options: Offer strong reads from DynamoDB and eventual reads from Redis, explaining the trade-off.
  • Scaling strategy: Auto-scaling for DynamoDB read/write capacity units, and Redis cluster scaling based on CPU/memory metrics.
  • Operational metrics: Detailed CloudWatch dashboards for cache hit ratio, DynamoDB throttles, and latency.

What a Staff Answer Adds

  • Failure modes: Discuss node loss in Redis, DynamoDB partition outage, and network partitions. Show how a fallback to S3 or a secondary DynamoDB table maintains availability.
  • Operational tooling: Use AWS CloudTrail for audit, AWS Config for drift detection, and automated alarms (e.g., 95th-percentile latency > 15 ms).
  • Cost analysis: Rough estimate – DynamoDB provisioned capacity at a small fraction of a cent per WCU/RCU-hour, Redis cluster ≈ $0.10 per GB-hour, S3 storage ≈ $0.023 per GB-month. Show how scaling shards affects cost linearly.
  • Trade-off defense: Defend the choice of DynamoDB over Aurora for low-latency reads, citing its single-digit millisecond performance and built-in partitioning.
  • Operational procedures: Rolling upgrades of Redis nodes, backup strategy for DynamoDB (on-demand backups + point-in-time recovery), and disaster-recovery drills.
  • Security: IAM policies scoped to least privilege, VPC endpoints for private connectivity, and encryption at rest via KMS.

Common Mistakes That Sink the Question

MistakeWhy It Fails
Ignoring scaling numbersLeaves you unable to justify sharding or cache size.
Over-engineering the data modelAdds unnecessary complexity; interviewers expect a simple key-value schema.
Forgetting consistency trade-offsLeads to ambiguous answers about read/write guarantees.
Skipping failure handlingShows no operational mindset; staff interviews demand it.
Claiming “infinite” capacityUnrealistic; interviewers look for realistic limits and mitigation.

Junior / Senior / Staff Comparison

AspectJuniorSeniorStaff
ArchitectureSingle DynamoDB table, optional cacheDynamoDB + Redis cluster, write-behind workerFull multi-region setup, fallback to S3, robust HA
ConsistencyDynamoDB default (eventually consistent reads, strong on request)Configurable read-through cache (eventual)Detailed SLAs, quorum reads, fallback paths
ScalingManual provisioningAuto-scaling, partition planningCost-aware scaling, predictive capacity planning
OpsBasic CloudWatch alarmsDetailed dashboards, alert routingFull incident response runbooks, chaos testing
Trade-offsSimplicity vs. performanceLatency vs. cost (Redis vs. DynamoDB)Availability vs. operational complexity

How to Practice This

  1. Study the worked design in the System Design AI library. Observe how the request path is broken into hops and how each component is annotated.
  2. Draw your own canvas: start with the client, add API Gateway, Lambda router, Redis cache, DynamoDB shards, and optional S3 fallback. Label each hop with latency goals.
  3. Run a mock interview: explain the design aloud, then answer follow-up questions about failure modes, cost, and scaling. Use the rubric to check if you covered junior, senior, and staff expectations.

Questions people ask

What is the typical read/write latency for a key-value store on AWS?

Using DynamoDB with a hot-key cached in ElastiCache (Redis) you can achieve < 5 ms read latency and < 20 ms write latency for the 95th percentile.

How do you decide between DynamoDB and Aurora for the backing store?

DynamoDB offers single-digit millisecond latency and automatic partitioning, which is ideal for high-throughput key-value workloads. Aurora provides richer query capabilities but adds extra latency and operational overhead.

When should you store large values in S3 instead of DynamoDB?

If values exceed a few hundred kilobytes, storing them in S3 and keeping a reference in DynamoDB reduces DynamoDB write capacity consumption and keeps latency predictable.

What is a common pitfall when hashing keys for sharding?

Choosing a hash function with poor distribution can create hot shards. Use a well-tested function like Murmur3 or SHA-256 and monitor shard load to rebalance if needed.