method · first seen 8 Oct 2026
adaptive tokenization
Tokenization that learns segment boundaries dynamically rather than using a fixed vocabulary.
Mentions per week
PapersCodeDiscussion
Earliest sources
- AdaptTok-Indic: Script-Aware Adaptive Tokenization for Indic LanguagesPapers · 8 Oct 2026 · Binayak Mukherjeea, Moitreya Nandi, Arindam Panja
- BudgetPix: Compute-Adaptive Tokenization for Pixel-Space Image DiffusionPapers · 8 Oct 2026 · Ozgur Kara, Yujia Chen, Daniel Watson, David Forsyth, James Matthew Rehg, Wen-Sh…
- Differentiable Virtual-Slot Partitioning for Adaptive Tokenization in Vision TransformersPapers · 8 Nov 2026 · Gaoxiang He, Bin Zhu, Bo Xie, Xiaotong Yu, Yi Chen, Hongfei Yan
Latest sources
- Differentiable Virtual-Slot Partitioning for Adaptive Tokenization in Vision TransformersPapers · 8 Nov 2026 · Gaoxiang He, Bin Zhu, Bo Xie, Xiaotong Yu, Yi Chen, Hongfei Yan
- BudgetPix: Compute-Adaptive Tokenization for Pixel-Space Image DiffusionPapers · 8 Oct 2026 · Ozgur Kara, Yujia Chen, Daniel Watson, David Forsyth, James Matthew Rehg, Wen-Sh…
- AdaptTok-Indic: Script-Aware Adaptive Tokenization for Indic LanguagesPapers · 8 Oct 2026 · Binayak Mukherjeea, Moitreya Nandi, Arindam Panja