Design a Multi-tenant SaaS Backend: Junior, Senior, and Staff Answers
When an interviewer asks you to design a multi-tenant SaaS backend, they are looking for three things: how you reason about scale, how you isolate customers, and how you think about operations. They want to see whether you can turn vague business goals into concrete architecture choices and whether you understand the trade-offs between simplicity, cost, and reliability.
Core Requirements and Driving Numbers
| Requirement | Typical Ballpark (hedged) |
|---|---|
| Number of tenants | 10,000 – 100,000 (order of 10⁴-10⁵) |
| Requests per second | 1,200 – 12,000 RPS (≈100 M – 1 B/day) |
| Data per tenant | 10 GB – 100 GB total, growing over time |
These three figures drive storage sizing, request routing capacity, and isolation strategy. If you assume 100 M daily requests, that is about 1,200 RPS (100 M ÷ 86,400 ≈ 1,158). A tenant count of 50 k means average per-tenant request rate of roughly 0.02 RPS, but hot tenants can spike to several hundred RPS, so you need a design that can handle both the aggregate load and per-tenant spikes.
Request-Path Walkthrough
- Client → API Gateway – The entry point is an Amazon API Gateway that terminates TLS, does basic throttling, and routes to a Lambda or ECS service based on the HTTP path.
- Auth → Cognito – The gateway forwards the
Authorizationheader to Amazon Cognito for token validation. The token contains the tenant ID (or a user-to-tenant mapping is looked up in a small DynamoDB table). - Routing → Service Layer – A thin Lambda (or a Fargate container) extracts the tenant ID and forwards the request to the appropriate service. For read-heavy paths, it checks an Amazon ElastiCache (Redis) entry keyed by
tenantId:resourceId. - Cache Miss → Data Store – If the cache misses, the service reads from Amazon Aurora (PostgreSQL) for relational data or DynamoDB for high-throughput key-value access. The query includes the tenant ID as a partition key, guaranteeing logical isolation.
- Write Path → Transaction Log – Writes go through the service layer, which persists to Aurora (using multi-tenant schemas) and publishes an event to an Amazon SQS queue. A background worker consumes the queue, updates secondary indexes, and pushes change events to Kafka on MSK for downstream analytics.
- Response → CDN – For static assets (reports, PDFs) the service writes to an S3 bucket. CloudFront distributes the objects globally, with cache-control headers that respect tenant-specific TTLs.
Data Model and Key Choice
- Tenant Isolation – Use a single-tenant identifier as the partition key in DynamoDB (
PK = tenantId). In Aurora, keep atenant_idcolumn on every table and enforce row-level security via PostgreSQL policies. This choice is preferred over separate databases per tenant because it simplifies provisioning and reduces operational overhead while still offering strong logical isolation. - Why Not Separate Schemas for Every Tenant? – Separate schemas become unwieldy beyond a few hundred tenants and make schema migrations painful. The single-table approach scales to millions of tenants with minimal admin effort.
What a Complete Junior Answer Looks Like
- Scope – Lists functional requirements (CRUD, authentication) and non-functional ones (availability ≈ 99.9%, latency < 200 ms).
- Diagram – Sketches a single API Gateway → Lambda → Aurora flow.
- Data Model – Shows a simple
userstable withtenant_idforeign key. - Isolation – Mentions using a tenant ID column but does not discuss trade-offs.
- Caching – May note a cache but leaves it as “optional”.
- Failure Handling – Briefly says “retry on error”.
- Operations – Touches on monitoring with CloudWatch but no deeper metrics.
What a Senior Answer Adds
- Tenant Isolation Strategy – Explains why a shared Aurora cluster with row-level security and a DynamoDB table keyed by tenant ID is chosen.
- Cache Layer – Adds ElastiCache with per-tenant eviction policies, reducing read latency.
- Async Processing – Introduces SQS for write-behind indexing and Kafka on MSK for event-driven analytics.
- Scalability – Discusses horizontal scaling of the Lambda layer via provisioned concurrency and autoscaling of Aurora read replicas.
- Security – Details IAM roles per service, VPC isolation, and encryption at rest (KMS) and in transit (TLS).
- Operational Metrics – Proposes dashboards for request latency, cache hit ratio, and queue depth.
- Cost Awareness – Names the main cost buckets: Aurora instance or ACU hours, DynamoDB read and write capacity, and ElastiCache node hours, and says which of them grow with tenant count.
What a Staff Answer Looks Like
- Failure Domains – Breaks the system into zones: API Gateway + Cognito (frontend), service layer (compute), data stores (Aurora/DynamoDB), and async pipelines (SQS/MSK). Shows how each zone can fail independently and how the design recovers (e.g., Lambda cold-start fallback, Aurora multi-AZ, DynamoDB global tables).
- Operational Playbooks – Defines runbooks for cache eviction storms, database replica lag, and Kafka broker failures. Mentions using AWS CloudWatch Alarms, AWS X-Ray for tracing, and automated canary deployments via CodeDeploy.
- Cost Optimization – Suggests Aurora Serverless v2 for unpredictable load, DynamoDB on-demand capacity (or provisioned capacity with auto-scaling for steady load), and spot instances for background workers.
- Trade-off Defense – Defends the choice of a shared Aurora cluster over per-tenant databases by citing operational simplicity, backup consistency, and the ability to use a single read replica pool to serve hot tenants.
- Security Depth – Adds tenant-level encryption keys stored in KMS, audit logging to CloudTrail, and a zero-trust network with VPC endpoints.
- Monitoring & Alerting – Presents a table of key SLOs (99.9% availability, 95th-percentile latency < 150 ms) and corresponding metrics.
- Future-Proofing – Discusses sharding the DynamoDB table by tenant ID hash prefix if tenant count grows beyond 100 k, and migrating hot tenants to dedicated Aurora instances.
Common Mistakes that Sink the Question
| Mistake | Why It Fails |
|---|---|
| Ignoring tenant isolation | Leads to data leakage and makes compliance impossible. |
| Over-engineering with a database per tenant | Adds operational burden and hurts scalability. |
| No caching strategy | Increases read latency, especially under hot-tenant spikes. |
| Forgetting async pipelines | Synchronous writes cannot sustain high write throughput. |
| Skipping cost discussion | Shows lack of real-world awareness; interviewers expect trade-off analysis. |
| No failure handling | System appears brittle; interviewers look for graceful degradation. |
Comparison of Junior, Senior, and Staff Answers
| Aspect | Junior | Senior | Staff |
|---|---|---|---|
| Isolation | Tenant ID column noted | Row-level security, DynamoDB partition key | Detailed security policies, KMS per-tenant keys |
| Caching | Optional mention | ElastiCache with per-tenant TTL | Cache eviction monitoring, fallback strategy |
| Async Processing | None or vague | SQS + Kafka for events | Full runbooks, canary deploys, spot workers |
| Scaling | Autoscale Lambda | Aurora read replicas, DynamoDB auto-scale | Sharding strategy, Aurora Serverless v2 |
| Ops & Monitoring | Basic CloudWatch metrics | Dashboards, alerts, tracing | Runbooks, X-Ray, automated remediation |
| Cost | Not addressed | Rough cost buckets | Detailed optimization, spot usage |
| Failure Modes | Simple retry | Zone-level redundancy | End-to-end failure domain analysis |
How to practice this
- Sketch the request path on a blank canvas. Start with API Gateway and add each hop, labeling where the tenant ID flows.
- Write a brief rubric: list the core requirements, the three driving numbers, and the extra senior/staff items you must cover.
- Use System Design AI to draw your own answer and get a review of missing pieces or weak trade-off arguments.
Questions people ask
What is the simplest way to achieve tenant isolation in a relational database?
Use a tenant_id column on every table and enforce row-level security policies in PostgreSQL. This keeps all tenants in one cluster while preventing cross-tenant reads.
When should I choose DynamoDB over Aurora for a SaaS backend?
DynamoDB shines for high-throughput key-value access and predictable latency, especially for session data or feature flags. Aurora is better for complex queries, joins, and transactional consistency.
How do I handle a sudden spike from a hot tenant without affecting others?
Place a per-tenant cache in ElastiCache, use read replicas for Aurora, and route writes through a dedicated SQS queue that can be scaled independently. Autoscaling policies should be tenant-aware.
What operational metrics matter most for a multi-tenant SaaS service?
Track request latency (p95), cache hit ratio, queue depth, replica lag, and error rates per tenant. Set alerts on thresholds that would breach your SLOs.