Design a File Storage Service like Dropbox: Junior, Senior, and Staff Answers
When you’re asked to design a file storage service like Dropbox, the interviewer is looking for three things: your ability to break down a complex product into manageable components, your sense of the scalability and consistency trade-offs, and how you think about operations and cost at scale. Below is a step-by-step walk through the request path, the data model, and the key architectural choices, followed by what each interview level is expected to cover.
1. Core Requirements and Driving Numbers
| Requirement | Reason | Typical Target |
|---|---|---|
| Functional | Upload, download, versioning, sharing, folder hierarchy | All CRUD operations must finish within a few hundred ms |
| Non-functional | High availability, durability, low latency reads, eventual consistency for metadata | 99.99% uptime, 99.999999999% durability (S3 level) |
| Scale | Users, storage, request rate | ~100 M registered users, average 1 TB per user, peak ~10 k requests/second |
The numbers are only ball-park. 100 M users translates to roughly 1 k new uploads per second (assuming each user uploads about one file a day). 10 k requests/second covers a mix of uploads, downloads, and metadata queries during a busy hour.
2. Request Path – One Hop at a Time
- Client → API Gateway – The entry point validates TLS and routes to the appropriate microservice. Using Amazon API Gateway gives you throttling and quick scaling.
- Auth Service – Calls Amazon Cognito (or a custom JWT validator) to verify the user token.
- Metadata Service – A thin service that reads or writes file metadata (name, size, version, ACL) from DynamoDB. It also checks permissions.
- Storage Service – For uploads, the client receives a pre-signed URL to Amazon S3; for downloads, the service returns a signed URL that CloudFront can cache.
- CDN – CloudFront serves public or shared files, reducing latency for downloads.
- Async Processing – An SQS queue (or Kafka on MSK) receives events for virus scanning, thumbnail generation, and indexing. Workers pull from the queue and write back results to DynamoDB or S3.
Each hop is a separate microservice, allowing independent scaling and failure isolation.
3. Data Model and Key Choice
- FileMetadata Table (DynamoDB)
- Partition key:
userId– groups all a user’s files together, enabling efficient list operations. - Sort key:
filePath#version– encodes the hierarchical path and version number, allowing range queries for folder listings and version history. - Attributes:
size,contentHash,s3Key,acl,createdAt.
Why this over a relational model? A relational database would require joins for folder hierarchy and would struggle with the write-heavy pattern of uploads. DynamoDB’s single-digit millisecond latency and automatic sharding fit the access pattern where most operations are keyed by user.
4. What a Complete Junior Answer Covers
- High-level diagram showing the client, API gateway, auth, metadata service, S3, and CDN.
- Basic requirements (upload/download, durability, basic auth).
- Simple data model (one table for metadata, S3 for blobs).
- Scalability discussion limited to adding more API gateway instances and increasing S3 bucket throughput.
- Missing pieces: no mention of caching, async processing, or detailed failure handling.
5. What a Senior Answer Adds
- Sharding strategy for DynamoDB (e.g., adding a hash prefix to
userIdto avoid hot partitions). - Caching layer using ElastiCache (Redis) for frequently accessed metadata and directory listings.
- Event-driven pipeline with SQS or Kafka for post-upload tasks (virus scan, thumbnail, indexing). Explain at-least-once semantics and idempotent workers.
- Consistency model – eventual consistency for metadata reads, strong consistency for permission checks.
- Capacity planning – rough calculations showing that DynamoDB can handle 10 k reads/sec and 5 k writes/sec with on-demand capacity.
- Security – server-side encryption (SSE-S3) and bucket policies, plus signed URLs with limited TTL.
6. What a Staff Answer Adds
- Failure modes – region-wide outage, S3 latency spikes, DynamoDB throttling. Show how cross-region replication (DynamoDB global tables, S3 Cross-Region Replication), retries with backoff, and circuit-breaker patterns mitigate these.
- Operational metrics – latency histograms, error rates, storage growth, and cost dashboards. Mention CloudWatch alarms and automated remediation scripts.
- Cost analysis – rough estimate: S3 storage $0.023/GB/month, DynamoDB on-demand requests billed per million (writes cost several times more than reads), CloudFront data transfer $0.085/GB. Show how caching can cut read traffic by 30 %.
- Trade-off defense – choosing DynamoDB over Aurora for low-latency key-value access, accepting eventual consistency for directory listings to keep latency low.
- Operational readiness – blue-green deployments for the metadata service, canary releases, and automated rollbacks.
- Compliance – mention GDPR data-subject requests and how S3 Object Lock can enforce retention policies.
7. Common Mistakes That Sink This Question
- Over-engineering the file system – trying to model a full POSIX hierarchy in a relational DB leads to complex joins and poor performance.
- Ignoring upload/download asymmetry – uploads are write-heavy, downloads are read-heavy; a good design separates the two paths.
- No versioning strategy – forgetting to store
versionin the sort key makes it hard to support file rollback. - Missing security – using public S3 URLs without signed URLs exposes data.
- Leaving out async processing – virus scanning and thumbnail generation are essential for a production service; omitting them looks unrealistic.
8. Comparison Table
| Aspect | Junior Answer | Senior Answer | Staff Answer |
|---|---|---|---|
| Scope | High-level components only | Adds sharding, caching, async pipeline | Adds failure handling, ops, cost, compliance |
| Data Store | Single DynamoDB table | Sharded DynamoDB, Redis cache | DynamoDB global tables across regions, cost-aware sizing |
| Async Work | None or vague | SQS/Kafka for scans & thumbnails | Detailed idempotent workers, back-pressure handling |
| Reliability | Basic HA via multiple instances | Multi-AZ, circuit breakers | Full disaster recovery, blue-green deployments |
| Metrics | Mention latency loosely | Define latency buckets, cache hit rate | Full dashboard, alerts, cost monitoring |
How to practice this
- Study the worked design in the System Design AI library. Read the step-by-step walkthrough and note how each component is justified.
- Draw your own canvas on the practice board. Replicate the request path, add your own sharding and caching decisions, and label the AWS services you would use.
- Run through a mock interview: present the diagram, explain the data model, and answer the “what if” questions about failures, cost, and scaling.
Questions people ask
What core metrics should I track for a file storage service?
Track request latency (p95), error rate, storage growth (GB per month), cache hit ratio, and cost per GB of storage. CloudWatch can surface these automatically.
Why not use a relational database for metadata?
Relational databases need joins for hierarchical queries and can become a bottleneck under heavy write loads. DynamoDB gives single-digit ms latency and scales horizontally without complex schema changes.
How do I handle large file uploads without blocking the API?
Issue a pre-signed S3 URL to the client. The client uploads directly to S3, while the API records the metadata after receiving a callback or polling the upload status.
What is a good way to version files?
Store the version number as part of the sort key (filePath#version). Each new upload increments the version, and you can retrieve prior versions with a range query on the sort key.