83. Design a Vector Database
Nearest-neighbour search over a billion embeddings: HNSW, IVF-PQ and disk graphs, filters, segments, sharding, and hybrid search with re-ranking.
ClassicMedium
Guided practice
Start with a template. Work through each step. Ask Coach when you need a second opinion.
Designs with worked examples 24
Ordered by design numberCompany tags are community-reported. Counts on cards show how many people reported that design.
Generative AIClear filters
106. Design an LLM Chat Assistant like ChatGPT
A chat assistant on models we train and serve: next-token framing, three training stages, resumable SSE streams, prefix caching, token quotas, safety, evaluation.
ClassicMedium108. Design an LLM Inference Serving Platform
Open-weight LLMs from 8B to 405B served to many products over a streaming OpenAI-style API on AWS GPUs.
ClassicMedium114. Design a Web-Scale Training Data Pipeline
Petabytes of crawled pages into trillions of clean, deduplicated, tokenized and versioned training tokens, with opt-outs honoured.
ClassicMedium115. Design an LLM Batch Inference Service
Millions of LLM prompts in JSONL files, one result per custom_id within 24 hours at about half the online price, on GPUs that come and go.
ClassicMedium116. Design an LLM Pretraining Pipeline
Pretraining a base LLM on a fixed budget: sizing by 6ND and scaling laws, the data factory, tokenizer, stable runs, failures and evals.
ClassicMedium118. Design an LLM Gateway
One API in front of every model: token quotas, routing and fallbacks, caching, cost attribution, masked logs and guardrails, streamed without buffering.
ClassicMedium119. Design an RLHF and Preference-Tuning Pipeline
Preference tuning as a weekly loop: rater and AI labels, Bradley–Terry reward models, DPO and PPO with a KL leash, vLLM rollouts, reward-hacking checks.
ClassicMedium120. Design a Fine-Tuning Service (SFT and LoRA)
Tenants upload examples and get a tuned, gated model on the same API: LoRA and QLoRA arithmetic, chat templates, Kueue fair sharing, eval gates, multi-LoRA serving.
ClassicMedium121. Design an LLM Evaluation Platform
GPU batch generation behind a shared cache, sandboxed pass@k, calibrated judges, paired statistics, release gates, contamination checks.
ClassicMedium124. Design Multi-Tenant LoRA Adapter Serving
Serve 10,000 LoRA adapters for 2,000 tenants on one shared base model, with mixed-adapter batches, tiered adapter caches, affinity routing and base upgrades.
ClassicMedium125. Design a Prompt and Semantic Cache for LLMs
Exact, semantic and prefix caching in front of LLMs: scoped keys, distances with an error budget, tenant isolation, invalidation and measured savings.
ClassicMedium126. Design Model Distillation at Scale
A 70B teacher distilled into an 8B student: transfer sets, logit and on-policy distillation, per-slice gates, escalation routing, a refresh loop.
ClassicMedium127. Design a Model Quantization and Compression Pipeline
Every new LLM version shrunk to FP8, INT8 or INT4 with a smaller KV cache, gated against its BF16 parent and benchmarked on the serving GPU.
ClassicMedium128. Design a Synthetic Data Generation Pipeline
Teacher models write training data for smaller students: conditioned and evolved prompts, verified answers, dedup, decontamination, versioned datasets with lineage.
ClassicMedium131. Design Retrieval-Augmented Generation
Answers from private documents with citations: chunking, BM25 + vector retrieval, reranking, permission filters, grounding checks and evaluation.
ClassicMedium132. Design Gmail Smart Compose
Next words within 100 ms of a keystroke: conditional language modelling, a distilled student on draft-pinned KV caches, private n-grams, DP and canary audits.
ClassicMedium133. Design Google Translate
Translation for 20 to 240 languages: one multilingual encoder-decoder, mined and back-translated data, distilled students, COMET and MQM, quality-aware serving: one-hour boards for junior, senior and staff, with the theory.
ClassicMedium134. Design Text-to-Image Generation
Prompt to four images in seconds: latent diffusion, guidance, a distilled few-step student, recaptioned data, safety on both sides, watermarks and C2PA.
ClassicMedium135. Design Image Captioning
Captions read aloud as alt text: encoder, bridge and decoder, CLIP-filtered data, beam search, hallucination-aware evaluation, lanes on one GPU pool.
ClassicMedium136. Design Realistic Face Generation
Faces of people who do not exist: a style-based GAN, truncation, latent-space edits, selfie inversion, provenance and memorisation audits.
ClassicMedium144. Design Personalized Headshot Generation
Selfies in, professional headshots out: per-order LoRA on SDXL, an identity-encoder preview, face-match ranking, GPU pools, consent and deletion.
ClassicMedium145. Design High-Resolution Image Synthesis
Images at 2K–4K without paying for every pixel: latent diffusion, an SR cascade with noise augmentation, tiled decoding, distilled drafts, cost per megapixel: one-hour boards for junior, senior and staff, with the theory.
ClassicMedium146. Design Text-to-Video Generation
Short clips from a prompt: latent video diffusion, recaptioned data, spacetime attention, cascades, multi-GPU jobs, previews, fair queues, provenance.
ClassicMedium