Guided practice

Start with a template. Work through each step. Ask Coach when you need a second opinion.

All designs

All designs 8

Ordered by design number

Company tags are community-reported. Counts on cards show how many people reported that design.

LLMsClear filters

106. Design an LLM Chat Assistant like ChatGPT
A chat assistant on models we train and serve: next-token framing, three training stages, resumable SSE streams, prefix caching, token quotas, safety, evaluation.
ClassicMedium
Streaming Machine learning Generative AI
+2
AWS LLMs
Worked design
Practice
108. Design an LLM Inference Serving Platform
Open-weight LLMs from 8B to 405B served to many products over a streaming OpenAI-style API on AWS GPUs.
ClassicMedium
Machine learning Generative AI AI infrastructure
+4
AWS GPUs Inference LLMs
Worked design
Practice
115. Design an LLM Batch Inference Service
Millions of LLM prompts in JSONL files, one result per custom_id within 24 hours at about half the online price, on GPUs that come and go.
ClassicMedium
Machine learning Generative AI AI infrastructure
+4
AWS Batch Inference LLMs
Worked design
Practice
118. Design an LLM Gateway
One API in front of every model: token quotas, routing and fallbacks, caching, cost attribution, masked logs and guardrails, streamed without buffering.
ClassicMedium
Generative AI AI infrastructure AWS
+3
Gateway LLMs Rate limiting
Worked design
Practice
124. Design Multi-Tenant LoRA Adapter Serving
Serve 10,000 LoRA adapters for 2,000 tenants on one shared base model, with mixed-adapter batches, tiered adapter caches, affinity routing and base upgrades.
ClassicMedium
Machine learning Generative AI AI infrastructure
+4
AWS Inference LLMs LoRA
Worked design
Practice
125. Design a Prompt and Semantic Cache for LLMs
Exact, semantic and prefix caching in front of LLMs: scoped keys, distances with an error budget, tenant isolation, invalidation and measured savings.
ClassicMedium
Caching Generative AI AI infrastructure
+4
AWS Embeddings LLMs Retrieval
Worked design
Practice
131. Design Retrieval-Augmented Generation
Answers from private documents with citations: chunking, BM25 + vector retrieval, reranking, permission filters, grounding checks and evaluation.
ClassicMedium
Generative AI AWS LLMs
+2
RAG Retrieval
Worked design
Practice
132. Design Gmail Smart Compose
Next words within 100 ms of a keystroke: conditional language modelling, a distilled student on draft-pinned KV caches, private n-grams, DP and canary audits.
ClassicMedium
Machine learning Generative AI Autocomplete
+3
AWS Latency LLMs
Worked design
Practice

Self-paced by default. Your answers and canvas stay private. AI feedback runs only when you ask for it.

All designs 8

106. Design an LLM Chat Assistant like ChatGPT

108. Design an LLM Inference Serving Platform

115. Design an LLM Batch Inference Service

118. Design an LLM Gateway

124. Design Multi-Tenant LoRA Adapter Serving

125. Design a Prompt and Semantic Cache for LLMs

131. Design Retrieval-Augmented Generation

132. Design Gmail Smart Compose