System Design AI

Designs

Pick a system. Work through the problem. Compare your approach.

Create your own

Designs with worked examples 4

Ordered by design number

Company tags are community-reported. Counts on cards show how many people reported that design.

GPUsClear filters

102. Design a GPU Cluster Scheduler for Training Jobs
Thousands of GPUs shared by many teams: gang admission, topology-aware placement, quotas with lending, checkpointed preemption and failure recovery.
ClassicMedium
AI infrastructure AWS GPUs
+2
Kubernetes Scheduling
Worked design
Practice
105. Design a Distributed Training Platform
Train 1 B to 400 B models on 8 to 10,240 H100s: DDP, ZeRO and FSDP, tensor and pipeline parallel over NVLink and EFA, checkpoints, hot spares, MFU and goodput.
ClassicMedium
Machine learning AI infrastructure AWS
+3
GPUs Parallelism Training
Worked design
Practice
108. Design an LLM Inference Serving Platform
Open-weight LLMs from 8B to 405B served to many products over a streaming OpenAI-style API on AWS GPUs.
ClassicMedium
Machine learning Generative AI AI infrastructure
+4
AWS GPUs Inference LLMs
Worked design
Practice
117. Design a Self-Healing GPU Fleet
Detect, classify and repair failing GPU nodes (Xid errors, ECC, NVLink and EFA faults, stragglers, silent corruption) without draining the fleet.
ClassicMedium
Observability AI infrastructure AWS
+2
GPUs Reliability
Worked design
Practice

Each design brings together the brief, practice, examples and discussion.