Design a Typeahead for E-commerce Search: Junior, Senior, and Staff Answers
When an interviewer asks you to design a typeahead for an e-commerce site, they are probing three things: how you reason about massive read-heavy traffic, how you keep the user-perceived latency under a tight budget, and how you balance freshness of catalog data against cost. The problem looks simple—return a list of suggestions as the user types—but the design must survive millions of queries per day, handle spikes during sales, and keep the suggestions relevant as inventory changes.
Core Requirements and Driving Numbers
| Requirement | Detail |
|---|---|
| Latency | End-to-end response under 50 ms for 95th percentile. |
| Throughput | Roughly 100 k queries per second (≈8.6 billion per day). |
| Data size | Tens of millions of SKUs; the title, tags and popularity score that suggestions need come to tens of gigabytes. |
| Freshness | Updates to product titles or popularity should appear in suggestions within a few minutes. |
| Availability | 99.9 % uptime, graceful degradation under load. |
These numbers drive every architectural choice. A 100 k QPS load means each request must be served by a small number of highly optimized nodes, and the data model must support sub-millisecond lookups.
Request Path, One Hop at a Time
- Client → CDN: The browser sends an HTTP GET to a CloudFront edge location. The edge caches recent popular prefixes for a few seconds to shave off latency for hot queries.
- Edge → API Layer: If the prefix is not in the edge cache, CloudFront forwards the request to an Application Load Balancer (ALB) that terminates TLS and routes to a fleet of stateless API servers (ECS/Fargate). The request payload contains the typed prefix and optional user context.
- API → Cache: The API first checks an ElastiCache Redis cluster for the prefix. Keys are of the form
typeahead:{prefix}. A cache hit returns the pre-computed suggestion list immediately. - API → Store: On a miss, the API queries a DynamoDB table that stores a prefix index. Each item holds a prefix (up to 5 characters) and a sorted list of product IDs based on popularity. The query is an exact-match lookup on the partition key, using the typed prefix truncated to 5 characters.
- Store → Result Assembly: The API fetches product details (title, image URL, price) from a second DynamoDB table keyed by product ID. It assembles the top-N suggestions, applies any business rules (e.g., filter out out-of-stock items), and writes the result back to Redis with a short TTL (e.g., 5 seconds).
- Response → Client: The API returns a JSON array of suggestions. CloudFront caches the response for the TTL, so subsequent identical queries from nearby users hit the edge cache.
Data Model
- Prefix Index Table (DynamoDB): Partition key =
prefix(string, up to 5 chars). Sort key =popularityScore(number, descending). Each item stores a list of up to 50 product IDs. - Product Detail Table (DynamoDB): Partition key =
productId. Attributes =title,imageUrl,price,stockStatus. - Redis Cache: Key =
typeahead:{prefix}→ Value = serialized suggestion list.
Key Choice
Using DynamoDB for the prefix index gives predictable single-digit millisecond latency at any scale and automatic sharding by prefix. Alternatives like Elasticsearch provide richer full-text search but add operational complexity and higher cost for a use-case that only needs prefix matching. The design therefore chooses DynamoDB over Elasticsearch for simplicity, cost, and the ability to guarantee latency.
What a Junior Answer Covers
- Lists the basic flow (client → API → DB → response).
- Mentions using a relational store (e.g., MySQL/Aurora) for product data and a simple in-memory cache (Redis).
- Provides rough latency and QPS numbers.
- Draws a diagram with the components and notes what each does.
- Shows awareness of cache-aside pattern and TTL.
What a Senior Answer Adds
- Introduces a prefix index table to avoid full-text scans.
- Discusses sharding the DynamoDB table by prefix length to balance hot-spot traffic.
- Adds read-through cache with Redis and explains cache invalidation via a background worker that consumes product-update events from an SQS queue.
- Considers fallback to a slower search service (e.g., OpenSearch) for long prefixes or misspelled queries.
- Mentions monitoring (CloudWatch metrics for cache hit ratio, latency, and error rates) and auto-scaling policies for the API fleet.
What a Staff Answer Adds
- Failure Modes: Discusses CDN edge cache loss, Redis node failure, DynamoDB throttling, and network partitions. Proposes multi-AZ Redis replication, DynamoDB on-demand capacity, and circuit-breaker patterns in the API.
- Operations: Details deployment pipeline (CodePipeline → ECS rolling update), canary testing, and log aggregation (CloudWatch Logs). Describes a runbook for cache warm-up after a deployment.
- Cost Analysis: Rough estimate – DynamoDB read capacity for 100 k QPS with 5-item reads would cost tens of thousands of dollars per month if every request reached the table, so the estimate depends on the cache hit ratio; with about 99 % of requests served by CloudFront and Redis it falls to a few hundred dollars per month; Redis cluster of three nodes in a cache-optimized instance type adds similar cost. Uses cost-benefit argument to justify DynamoDB over a managed Elasticsearch cluster.
- Trade-off Defense: Defends the decision to limit prefix length to 5 characters to keep the index size manageable, acknowledging that longer prefixes would improve relevance but increase storage and write amplification. Shows how the design can be extended with a fallback to a full-text service for those cases.
- Data Freshness Pipeline: Describes a Kinesis Data Stream that captures product catalog changes, a Lambda consumer that updates the prefix index in DynamoDB, and a small delay (2-5 minutes) before the new suggestions appear.
Mistakes That Sink This Question
| Mistake | Why It Hurts |
|---|---|
| Ignoring latency budget | Leads to designs that require full database scans, which cannot meet 50 ms target at 100 k QPS. |
| Over-engineering with heavy NLP | Adds unnecessary complexity and cost; typeahead only needs prefix matching. |
| Forgetting misspellings | Users type quickly; not handling fuzzy matches results in poor UX and shows lack of product thinking. |
| Not accounting for hot prefixes | A few popular prefixes can dominate traffic; without sharding or cache, the system will hot-spot and throttle. |
| Skipping cache invalidation | Stale suggestions hurt conversion and demonstrate operational blind spots. |
How to Practice This
- Study the worked design in the System Design AI library to see a concrete board that walks through each component and its notes.
- Write your own answer on a blank canvas: start with the request path, then add the data model, caching strategy, and finally the staff-level extensions.
- Run a mock interview with a peer, or ask the AI coach to review your canvas, and iterate until the senior and staff sections feel natural.
Questions people ask
What latency should I aim for in a typeahead service?
Aim for sub-50 ms response for the 95th percentile. Users expect suggestions to appear instantly as they type.
Why use DynamoDB instead of Elasticsearch for prefix matching?
DynamoDB provides predictable single-digit millisecond reads, automatic sharding, and lower operational overhead. Elasticsearch offers richer text search but adds latency and cost that are unnecessary for strict prefix matches.
How do I handle hot prefixes like "sh" for shoes?
Partition the prefix index by the first two characters, replicate hot prefixes across multiple DynamoDB partitions, and keep a hot-key cache in Redis with a very short TTL.
What is a simple way to keep suggestions fresh without breaking latency?
Stream catalog updates through Kinesis, process them with Lambda, and update the DynamoDB prefix index asynchronously. The cache TTL ensures new data propagates within a few minutes.