trends.barnas.net

phenomenon · first seen 4 Oct 2026

data repetition in pretraining

Also: multi-epoch pretraining

Effects on language model loss and scaling when training data is seen multiple times during pretraining.

Mentions per week

0219 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

scaling law