105. Design a Distributed Training Platform
Train 1 B to 400 B models on 8 to 10,240 H100s: DDP, ZeRO and FSDP, tensor and pipeline parallel over NVLink and EFA, checkpoints, hot spares, MFU and goodput.
ClassicMedium
Designs
Pick a system. Work through the problem. Compare your approach.
Designs with worked examples 1
Ordered by design numberCompany tags are community-reported. Counts on cards show how many people reported that design.
GPUsClear filters