Squeezing Every Millisecond: Dynamic Quantization for Low‑Latency Inference on Discord Engineering Feeds
When I first pushed a live text‑classification service for Discord’s engineering blog, the inference time was 80 ms per article and the model bloat…