The design library
Building blocks, complete architectures, and variations worth comparing. Search by the problem you want to solve or a component you want to understand.
Classic designOfficial
A 70B teacher distilled into an 8B student: transfer sets, logit and on-policy distillation, per-slice gates, escalation routing, a refresh loop: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Exact, semantic and prefix caching in front of LLMs: scoped keys, distances with an error budget, tenant isolation, invalidation and measured savings: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Serve 10,000 LoRA adapters for 2,000 tenants on one shared base model, with mixed-adapter batches, tiered adapter caches, affinity routing and base upgrades: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
A tuning service like Vizier: random and Bayesian search, ASHA and population-based training, trials on a shared GPU cluster: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Experiment tracking and a model registry: non-blocking logging, chunked metric curves, content-addressed artifacts, lineage and a promotion gate. One-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
GPU batch generation behind a shared cache, sandboxed pass@k, calibrated judges, paired statistics, release gates, contamination checks: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Tenants upload examples and get a tuned, gated model on the same API: LoRA and QLoRA arithmetic, chat templates, Kueue fair sharing, eval gates, multi-LoRA serving: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Preference tuning as a weekly loop: rater and AI labels, Bradley–Terry reward models, DPO and PPO with a KL leash, vLLM rollouts, reward-hacking checks: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
One API in front of every model: token quotas, routing and fallbacks, caching, cost attribution, masked logs and guardrails, streamed without buffering: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Detect, classify and repair failing GPU nodes (Xid errors, ECC, NVLink and EFA faults, stragglers, silent corruption) without draining the fleet: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Pretraining a base LLM on a fixed budget: sizing by 6ND and scaling laws, the data factory, tokenizer, stable runs, failures and evals: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Millions of LLM prompts in JSONL files, one result per custom_id within 24 hours at about half the online price, on GPUs that come and go: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Petabytes of crawled pages into trillions of clean, deduplicated, tokenized and versioned training tokens, with opt-outs honoured: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Route images, text, audio and model replies to thousands of annotators, buy quality with gold and consensus, cut cost with models, and ship versioned datasets: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Know within hours when hundreds of production models see broken inputs, drift or falling quality, before the labels arrive: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Save the 1 TB state of a 70 B model on 10,000 GPUs often enough that a failure costs minutes, without stalling training, and restore fast: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Choose 20 videos out of 10 million for 100 million daily users: implicit labels, two-tower retrieval, a multi-task ranker, bias and exploration: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Embed a billion chunks on GPUs, keep a k-NN index fresh through CDC with versioned writes and provable deletes, and migrate models blue-green: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Open-weight LLMs from 8B to 405B served to many products over a streaming OpenAI-style API on AWS GPUs: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
From registry to traffic for hundreds of models: gates, shadow and canary with automatic rollback, packed CPU and GPU pools, LLM serving, cells and audit: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
A chat assistant on models we train and serve: next-token framing, three training stages, resumable SSE streams, prefix caching, token quotas, safety, evaluation: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Train 1 B to 400 B models on 8 to 10,240 H100s: DDP, ZeRO and FSDP, tensor and pipeline parallel over NVLink and EFA, checkpoints, hot spares, MFU and goodput: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
Define a feature once, train on point-in-time correct history and serve 200 fresh values in under 10 ms: one-hour boards for junior, senior and staff, with the theory behind them.
Classic designOfficial
A 500 GB model on 1,000 GPU servers in minutes: chunks and hashes, a topology-aware swarm, signed manifests, bandwidth budgets and waves: one-hour boards for junior, senior and staff, with the theory behind them.