System Design AI

Design a Fraud Detection Pipeline: The Junior, Senior and Staff Answer

Designing a fraud detection pipeline is a classic system-design interview problem. The interviewer wants to see how you translate vague business goals into a concrete, scalable architecture. They care about three things: the ability to handle high request rates, the latency budget for flagging suspicious activity, and the flexibility to evolve detection logic without breaking the service.

What the interviewer is testing

  • Scalability – can you handle millions of transactions per day and grow to billions?
  • Latency – fraud must be caught before the user completes a purchase, typically under a second.
  • Data modeling – how you store historical behavior and real-time features.
  • Operational awareness – monitoring, alerting, and graceful degradation.
  • Trade-off reasoning – choosing a streaming system over batch, or a NoSQL store over a relational one.

Core requirements and driving numbers

RequirementTypical value (hedged)
Transaction volume~10 million per day (≈115 per second)
Detection latency≤ 1 second end-to-end
False-positive tolerance< 1 % of legitimate transactions
Data retention90 days of raw events, longer for aggregated scores

These numbers set the scale for every component. A system that can ingest 115 req/s easily scales to 10× that load by adding partitions, so we design for roughly 1 k req/s to give headroom.

The core design, step by step

1. Ingestion layer

A client (web, mobile, or POS) sends a transaction event to an HTTPS endpoint behind Amazon API Gateway. The gateway validates the payload and forwards it to Amazon Kinesis Data Streams (or MSK Kafka) for durable, ordered streaming. Kinesis gives us at-least-once delivery and automatic scaling; Kafka offers finer-grained control if the team prefers open-source tooling.

2. Real-time feature enrichment

A set of AWS Lambda functions subscribe to the stream. Each function enriches the event with:

  • User profile from DynamoDB (last login, device fingerprints)
  • Recent transaction history from ElastiCache (Redis) for fast look-ups
  • External risk scores via HTTP calls (e.g., third-party fraud APIs)

The enriched record is written back to a second Kinesis stream.

3. Scoring service

Another Lambda (or a containerized microservice on AWS Fargate) consumes the enriched stream. It loads a pre-trained model (e.g., XGBoost) from S3 and computes a fraud score. The score and a risk level are attached to the original event and pushed to an SQS queue for downstream processing.

4. Decision and action

A consumer reads from the SQS queue and decides:

  • Block: send a reject response back through the API Gateway (using a synchronous callback pattern) – this path must stay under the latency budget.
  • Review: write the event to Aurora PostgreSQL for analyst review and audit.
  • Learn: store raw event in S3 (partitioned by day) for offline model retraining.

5. Monitoring & alerting

All components emit CloudWatch metrics. A dashboard shows request rate, processing latency, error rates, and model drift. Alarms trigger SNS notifications to on-call engineers.

What a complete junior answer covers

  • High-level flow: API Gateway → Kinesis → Lambda → SQS → Decision.
  • Simple data model: transaction ID, user ID, amount, timestamp, and a boolean fraud flag.
  • Choice of Kinesis over a simple queue because it preserves ordering and supports replay.
  • Mention of DynamoDB for user profile lookup and S3 for raw event storage.
  • Basic latency reasoning (a warm Lambda runs in tens of milliseconds, a cold start can add hundreds, and each Kinesis hop adds propagation delay that has to fit in the budget).

Missing elements often include feature enrichment, model versioning, and any discussion of scaling beyond the base numbers.

What a senior answer adds

  • Feature store: details on how recent transaction aggregates are kept in Redis with TTLs, and why that beats scanning DynamoDB each time.
  • Model serving: containerized inference on Fargate for stable warm containers, reducing cold-start variance.
  • Back-pressure handling: using Kinesis shard limits and SQS dead-letter queues to avoid data loss.
  • Consistency trade-offs: eventual consistency of Redis vs. strong consistency of DynamoDB for different risk tiers.
  • Data retention policy: moving older raw events from S3 Standard to S3 Glacier after 30 days.
  • Security: IAM role separation, encryption at rest (KMS) and in transit (TLS).

What a staff answer adds

  • Failure modes: discuss what happens if Kinesis throttles, Lambda times out, or Redis crashes. Propose a fallback path that writes directly to SQS for batch scoring.
  • Operational tooling: automated canary deployments of model versions, chaos testing of the Redis cluster, and log aggregation with OpenSearch.
  • Cost analysis: rough estimate – Kinesis ~ $0.015 per shard-hour, Lambda ~ $0.000016 per GB-second, DynamoDB reads ~ $0.25 per million reads. Show that a 10 M/day workload stays under a few hundred dollars per month.
  • Trade-off defense: argue why a streaming approach beats a nightly batch job (real-time fraud loss reduction) even though it adds operational complexity.
  • Scalability roadmap: moving from Kinesis to MSK for higher throughput, sharding DynamoDB tables, and adding a Global Accelerator for multi-region low-latency ingestion.

Common mistakes that sink this question

MistakeWhy it hurts
Ignoring latencyFraud detection must be sub-second; a batch-only design fails the core requirement.
Over-engineering the model serving layerUsing a full ML platform for a simple rule-based filter adds unnecessary complexity and cost.
Forgetting back-pressureWithout queueing or throttling, spikes will cause data loss or timeouts, breaking the SLA.
No monitoringWithout metrics you cannot detect model drift or infrastructure failures, leading to silent degradation.
Mixing up consistency guaranteesRelying on eventually consistent stores for immediate block decisions can let fraud slip through.

Comparison of answer depth

LevelCoverage
JuniorHigh-level flow, basic components (API GW, Kinesis, Lambda, SQS), simple data model, latency sketch.
SeniorAdds feature enrichment (Redis, DynamoDB), model serving details, back-pressure handling, security, data lifecycle.
StaffAdds failure mode analysis, ops tooling, cost estimate, trade-off justification, scaling roadmap, and governance.

How to practice this

  1. Study the worked design for a fraud detection pipeline in the System Design AI library. Observe how each component is annotated with talking points.
  2. Draft your own answer on the canvas: start with the request path, then add enrichment, scoring, and decision layers.
  3. Use the rubric to self-evaluate: check that you covered the three answer levels, listed failure modes, and gave a cost sketch.

Questions people ask

What latency can a Lambda-based scoring service realistically achieve?

A warm Lambda container typically responds in 30-70 ms. Each streaming hop adds its own propagation delay (on the order of a couple of hundred milliseconds with a shared Kinesis consumer, less with enhanced fan-out), so add up every hop and check the total against the one-second budget.

Why choose DynamoDB over Aurora for user profiles?

DynamoDB provides single-digit millisecond reads at scale when the partition key spreads load evenly, as a user ID does, which is essential for per-transaction look-ups.

Can the pipeline survive a regional outage?

By replicating the API Gateway and Kinesis stream in multiple regions and using Global Accelerator, traffic can be routed to a standby region with minimal impact.

How often should the fraud model be retrained?

A common practice is to retrain weekly using the last 30 days of labeled data, then deploy the new model via a canary rollout.