Classic design · Junior → Staff · by System Design AI

Design On-Device LLM Inference

A 3 B LLM on phones and laptops: bandwidth math, 4- and 2-bit weights, LoRA per feature, NPU runtimes, cohort rollouts, a private server: one-hour boards for junior, senior and staff, with the theory behind them.