phenomenon · first seen 6 Oct 2026
batch-invariant inference
Also: batch-induced nondeterminism, serving-context nondeterminism, batch invariance in LLM inference
Variation in model outputs caused by batch shape, hardware, or numerical precision changes during inference.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Same Text, Different Prediction: Serving-Context Nondeterminism in Text ClassifiersPapers · 6 Oct 2026 · Santhosh Kumar Kasa, Siva Rajesh Kasa, Sumit Negi
- BlockLock: Reproducible LLM Inference with Fixed Batch GeometryPapers · 4 Nov 2026 · Xiuyuan Qi, Pedro Reviriego, Javier Conde, Shanshan Liu
- How Much Determinism Can Fine-Tuning Buy? Margin Regularisation Against Batch-Induced Token FlipsPapers · 14 Nov 2026 · Charan Teja Nagisettygari, Siva Raja Sindiramutty
Latest sources
- How Much Determinism Can Fine-Tuning Buy? Margin Regularisation Against Batch-Induced Token FlipsPapers · 14 Nov 2026 · Charan Teja Nagisettygari, Siva Raja Sindiramutty
- BlockLock: Reproducible LLM Inference with Fixed Batch GeometryPapers · 4 Nov 2026 · Xiuyuan Qi, Pedro Reviriego, Javier Conde, Shanshan Liu
- Same Text, Different Prediction: Serving-Context Nondeterminism in Text ClassifiersPapers · 6 Oct 2026 · Santhosh Kumar Kasa, Siva Rajesh Kasa, Sumit Negi