System Design AI

Designs

Pick a system. Work through the problem. Compare your approach.

Create your own

Classic designs 1

Ordered by design number

Company tags are community-reported. Counts on cards show how many people reported that design.

GPUsClear filters

108. Design an LLM Inference Serving Platform
Open-weight LLMs from 8B to 405B served to many products over a streaming OpenAI-style API on AWS GPUs.
ClassicMedium
Machine learning Generative AI AI infrastructure
+4
AWS GPUs Inference LLMs
Worked design
Practice

Each design brings together the brief, practice, examples and discussion.