System Design AI

Designs

Pick a system. Work through the problem. Compare your approach.

Create your own

All designs 1

Ordered by design number

Company tags are community-reported. Counts on cards show how many people reported that design.

BatchClear filters

115. Design an LLM Batch Inference Service
Millions of LLM prompts in JSONL files, one result per custom_id within 24 hours at about half the online price, on GPUs that come and go.
ClassicMedium
Machine learning Generative AI AI infrastructure
+4
AWS Batch Inference LLMs
Worked design
Practice

Each design brings together the brief, practice, examples and discussion.