phenomenon · first seen 9 Oct 2026
loss spikes in neural network training
Sudden, intermittent jumps in training loss observed during large model pre-training.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Simulating Training Spikes of Deep Neural Networks by Layer-wise Random Feature ModelsPapers · 9 Oct 2026 · Ran Tian, Courtney Paquette
Latest sources
- Simulating Training Spikes of Deep Neural Networks by Layer-wise Random Feature ModelsPapers · 9 Oct 2026 · Ran Tian, Courtney Paquette