method · first seen 29 Oct 2026
synthetic pre-pretraining
Also: pre-pretraining
Training language models on synthetic data as a stage before natural-language pretraining.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Rethinking Pre-pretrainingPapers · 29 Oct 2026 · Alexander Stern, Tim Rocktäschel, Acyr Locatelli, Laura Ruis
Latest sources
- Rethinking Pre-pretrainingPapers · 29 Oct 2026 · Alexander Stern, Tim Rocktäschel, Acyr Locatelli, Laura Ruis