method · first seen 8 Oct 2026
synthetic data for pretraining
Using machine-generated tasks and examples as part of a language model's pretraining corpus to improve downstream abilities.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Conditional Transfer from Controlled Pretraining Mixtures to CodePapers · 8 Oct 2026 · Ohad Rubin
- AFCS: Atomic-Fact-Conditioned Synthesis for Factual Knowledge PretrainingPapers · 14 Oct 2026 · Yunhao Wang, Feng Zhang, Tinghao Yu, Xing Wu, Jiang Zhou
Latest sources
- AFCS: Atomic-Fact-Conditioned Synthesis for Factual Knowledge PretrainingPapers · 14 Oct 2026 · Yunhao Wang, Feng Zhang, Tinghao Yu, Xing Wu, Jiang Zhou
- Conditional Transfer from Controlled Pretraining Mixtures to CodePapers · 8 Oct 2026 · Ohad Rubin