method · first seen 5 Oct 2026
masked language modeling
Also: MLM, BERT pre-training, masked language model, masked language model pretraining
Pretraining objective where randomly masked tokens are predicted from surrounding context.
Mentions per week
PapersCodeDiscussion
Earliest sources
- Domain adaptation of Russian ModernBERT for long legal documentsPapers · 5 Oct 2026 · I. Litvak, D. Gvozdetsky, F. Lashkin, V. Kirova, S. Lagutin, V. Volf, T. Maksiya…
- Forecasting Corpus-Sampling Disagreement in Masked Text ReconstructionPapers · 9 Oct 2026 · Hare Krishna
- Single-Sample Decoding of a Two-Layer Sound-Word Canvas Model for Thai Klon PadPapers · 10 Oct 2026 · Worasait Suwannik
- UltraBERT: Beyond the Limit of BERT Pre-TrainingPapers · 10 Oct 2026 · Jungwoo Park, Taewhoo Lee, Jaewoo Kang
- Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language ModelingPapers · 15 Oct 2026 · Tong Zhu, Zhe Xu, Zefeng He, Yunkai Chai, Xuyang Hu, Dehong Zheng, Yun Luo, Yafu…
- Prosodic Structure as a Critical Component in Phonemic Language Modelling at BabyLM ScalePapers · 16 Oct 2026 · Bill K. S. Ku, Janet B. Pierrehumbert
Latest sources
- NoLBERT-Calls: Reading Earnings Calls Without HindsightPapers · 15 Nov 2026 · Kevin Foley, Jonathan Hartadi, Shivesh Prakash, Hino Wirachapong Suwanphibun, Sw…
- MUSS: A Multi-scale and Sequence-based Model for Single-cell Gene RegulationPapers · 5 Nov 2026 · Kangjie Zheng, Amirhossein Vahidi, Liying Jin, Joseph Clarke, Vijaya Baskar MS,…
- QECGBERT: A Scalable ECG Foundation Model Based on Self-Supervised Language Learning via VQ-VAE Waveform TokenizationPapers · 25 Oct 2026 · Aruna Mohan, Burcu Ozek, Or Zilbershot, Fatemeh Afghah
- LAMB: A Modern Contextual Language Model for LatinPapers · 22 Oct 2026 · Aidan McCarthy
- BERT Is Already a Denoiser: Cheaply Unlocking Iterative Generation in Masked Language ModelsPapers · 19 Oct 2026 · Julio C. Amador Diaz Lopez
- Prosodic Structure as a Critical Component in Phonemic Language Modelling at BabyLM ScalePapers · 16 Oct 2026 · Bill K. S. Ku, Janet B. Pierrehumbert
- Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language ModelingPapers · 15 Oct 2026 · Tong Zhu, Zhe Xu, Zefeng He, Yunkai Chai, Xuyang Hu, Dehong Zheng, Yun Luo, Yafu…
- UltraBERT: Beyond the Limit of BERT Pre-TrainingPapers · 10 Oct 2026 · Jungwoo Park, Taewhoo Lee, Jaewoo Kang
- Single-Sample Decoding of a Two-Layer Sound-Word Canvas Model for Thai Klon PadPapers · 10 Oct 2026 · Worasait Suwannik
- Forecasting Corpus-Sampling Disagreement in Masked Text ReconstructionPapers · 9 Oct 2026 · Hare Krishna
- Domain adaptation of Russian ModernBERT for long legal documentsPapers · 5 Oct 2026 · I. Litvak, D. Gvozdetsky, F. Lashkin, V. Kirova, S. Lagutin, V. Volf, T. Maksiya…