Classic design · Junior → Staff · by System Design AI

Design an Embedding API at 100,000 Requests a Second

Text in, unit vectors out for every product at 100,000 requests a second on AWS GPUs, batched by tokens and never mixing model versions: one-hour boards for junior, senior and staff, with the theory behind them.