169. Design a Real-Time Speech Transcription Pipeline
Live captions from a streaming Conformer-transducer: chunked attention with a per-stream cache, partials and finals, biasing, a post-call pass and a training loop.
Start with a template. Work through each step. Ask Coach when you need a second opinion.
Company tags are community-reported. Counts on cards show how many people reported that design.
Live captions from a streaming Conformer-transducer: chunked attention with a per-stream cache, partials and finals, biasing, a post-call pass and a training loop.
A cheap drafter guesses several tokens and a self-hosted 70B checks them in one pass, with rejection sampling keeping its output distribution exact.
A phone and browser agent that talks, calls tools and hands off: streaming ASR, LLM and TTS, turn detection, barge-in, a sub-second budget, telephony, simulated callers: one-hour boards for junior, senior and staff, with the theory.
Agents that call tools to finish multi-step tasks: the ReAct loop as a durable workflow, JSON-schema tool calls, microVM sandboxes, per-user OAuth, approvals, injection defences and pass^k evals: one-hour boards at three levels.
A coding assistant like Copilot: fill-in-the-middle completion in a 300 ms budget, prompts from open tabs and the repo, acceptance and persistence metrics, agents that edit and test: one-hour boards per level, with theory.
Search and cited answers across drives, wikis, tickets, chat and mail, trimmed to what each employee may open: ACL sync, identity mapping, hybrid retrieval, revocation deadlines and tenant keys, on junior, senior and staff boards.
A copilot that hears a live call and suggests answers within a second: two-channel capture, streaming recognition, a question detector, retrieval, a small LLM, summaries after the call, consent and cost at scale.
Guard LLM apps both ways: classifiers on prompts and streamed chunks, PII masking, injection and grounding checks, per-tenant policies, a red-team loop.
Prompt templates run side by side on several models over server-sent events, immutable versions with movable labels, an SDK cache that never blocks production, queued evaluations and CI gates: one-hour boards at three levels.
A weekly playlist of 30 never-played songs, plus home shelves, for 300 M listeners: implicit ALS, playlist and audio embeddings, a multi-task ranker, fair assembly and a bandit: three one-hour boards, with the theory behind them.
A vertical feed that picks each next video from what the viewer did seconds ago: labels at the swipe, multi-source retrieval, a multi-task ranker, online training and a fair start for new creators, as three one-hour boards.
Show what goes with a product, not what replaces it: directional co-purchase lift, item2vec, a cart-aware LambdaMART, feedback loops and cold start.
Rank 100 M products for a shopper’s query: BM25 plus semantic retrieval, LambdaMART on graded labels, position bias and revenue trade-offs.
Suggest completions on every keystroke in tens of milliseconds, ranked by the chance of a pick: a prefix index, a learning-to-rank model with context, fresh trends, safety filters and generation for unseen prefixes.
Recommend 25 of 15 million open jobs so members apply and posters answer: a two-sided score, GLMix personal effects, two-tower retrieval, application redistribution and fairness, on one-hour boards for three levels, with theory.
Decide which pushes and emails 400 M users get, when and how many: P(open) with a cap, a value model that prices unsubscribes, weekly budgets, send-time choice and uplift: one-hour boards for each level, with the theory.
Rank the stores that can deliver now by the chance of an order: session labels, live ETA and supply features, a multi-task ranker, new-merchant exploration, switchbacks: one-hour boards for junior, senior and staff, with theory.
Suggest creators to follow on a one-way graph of 40 billion edges: random walks and SALSA, follow communities, two-tower retrieval and a ranker for follows people keep; one-hour boards for junior, senior and staff, with theory.
Detect emerging topics per place within minutes from 500 M posts a day: sketches, seasonal bursts on distinct authors, events, spam and campaign filters: one-hour boards for junior, senior and staff, with the theory.
From millions of ads to one slot in 100 ms: boolean targeting, two-tower retrieval, calibrated pCTR and pCVR with late conversions, a second-price auction and pacing: one-hour boards for junior, senior and staff, with theory.
Build each member's homepage: which rows, their order, the titles and the image on each, for 100 M daily profiles. Row generators, a greedy page builder, 2D position bias, artwork bandits with replay, at three levels.
Score every card payment in under 100 ms and allow, challenge, review or block it: late, biased labels, velocity features, calibrated trees, actions priced in dollars, rings and adversaries: one-hour boards at three levels, with theory.
File billions of emails a day as inbox, spam or phishing when a lost real email costs most: sender reputation, SPF/DKIM/DMARC, trees over a robust text encoder, biased labels: one-hour boards for three levels, with the theory.
Warn before a phishing or malware page opens, minutes after it goes live: hash-prefix lists in the browser, a lexical pre-filter, rendered pages matched to brands, cloaking, takedowns: one-hour boards for three levels, with theory.
Catch fake and automated accounts at sign-up, while they act and in coordinated groups: challenges, behaviour models, graph trust, lockstep clusters, staged enforcement; one-hour boards for three levels, with theory.
Score every sign-in in under 50 ms and allow, step up or block: rare late labels, per-user familiarity, velocity against credential stuffing, friction budgets, passkeys and session risk. One-hour boards at three levels, with theory.
Find paid, incentivised and machine-written reviews and the rings behind them at 2 M reviews a day: behaviour, lockstep and dense-subgraph signals, staged enforcement, prevalence audits: boards for junior, senior and staff, with the theory.
Estimate each applicant's chance of default, then approve, size and price the loan with specific reasons for every decline: scorecards, monotone GBDT, reject inference, fair lending, one-hour boards and the theory behind them.
Check each ad, its landing page and its advertiser before it runs: a multimodal model with a head per policy, review by expected harm, LLMs for new policies, cloaking and prevalence: one-hour boards per level, with the theory.
Predict when a ride or a delivery arrives: a routing engine plus a learned residual, live traffic, quantile promises, deep and graph models, feedback loops: one-hour boards for junior, senior and staff, with the theory.