method · first seen 4 Oct 2026
compositional tokenization
Also: CoBPE
Tokenization that represents multi-word phrases as a base token plus reusable modifiers, reducing sequence length.
Mentions per week
PapersCodeDiscussion
Earliest sources
- More Than Words: Compositional Tokenization for Efficient Language ModelsPapers · 4 Oct 2026 · Yuval Reif, Guy Kaplan, Roy Schwartz
Latest sources
- More Than Words: Compositional Tokenization for Efficient Language ModelsPapers · 4 Oct 2026 · Yuval Reif, Guy Kaplan, Roy Schwartz