method · first seen 6 Oct 2026
FP4 quantization
Also: 4-bit floating point quantization
Representing weights or activations in 4-bit floating point format to reduce compute and memory cost.
FP4 quantization stores and computes weights, activations, KV caches or gradients in 4-bit floating point, typically with block-wise scale codes as in MXFP4 and NVFP4, a format that Blackwell-class GPUs execute natively in tensor cores. The new work in these sources moves beyond plain weight compression. Two papers, one with Qwen-affiliated authors, target FP4 rollouts for reinforcement learning on mixture-of-experts models. Others cover FP4 communication compression for model-parallel training, residue and low-rank corrections that recover accuracy (ResServe, BiLoCo), second-order scale search, and FP4 KV-cache attention for multi-head latent attention, including an NVIDIA paper. One open-source repo implements FP4 KV-cache decoding for vLLM.
Native FP4 hardware support turns accuracy-recovery methods into direct savings in serving and training cost. If these recipes hold up for RL post-training and attention kernels, FP4 could become the default precision for much of the LLM pipeline rather than an inference-only option.
Analyst note (emerging), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 32 on 16 Nov 2026 · 8 mentions in 28 days · 8 independent groups · papers → code · first seen 6 weeks ago · acceleration +0.44
Earliest sources
- TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language ModelsPapers · 6 Oct 2026 · Xin Wang, Hao Yu, Zhengyang Zhuge, Bochao Mao, Zheng Li, Junda Feng, Yuyan Luo,…
- BiLoCo: Binary Low-Rank Corrections for Post-Training QuantizationPapers · 23 Oct 2026 · David Jin, Beshr IslamBouli, Tarushii Goel, Han Guo, Yoon Kim
- TriTile: Three-Tier Attention for Video Diffusion TransformersPapers · 26 Oct 2026 · Yeonjae Kim, Omin Kwon, Seong Hoon Seo, Jae W. Lee
- FP4Quant: Efficient 4-Bit Communication Compression for Model-Parallel LLM TrainingPapers · 3 Nov 2026 · Man Liu, Xingchen Liu, Hairui Zhao, Guangming Tan, Dingwen Tao
- Aligning Training and Rollout Policies for End-to-End FP4 RL on Mixture-of-Experts LLMsPapers · 3 Nov 2026 · Hei Yi Mak, Hoang Le, Mehran Taghian Jazi, Yunke Peng, Huan Lin, Zhengzhi Tang,…
- OpenBitSys/BitDecoding-FP4Code · 3 Nov 2026 · OpenBitSys · 49 stars
Latest sources
- NVFP4-MLA: Unified FP4 KV-Cache Storage and Attention Computation for Multi-Head Latent AttentionPapers · 13 Nov 2026 · Chao Ni, Qi Zhang, Xiao Wang, Zongfei Jing, Mingyang Hao, Binghan Chen, Kefeng D…
- SoScale: Second-Order Scale Search for FP4 QuantizationPapers · 12 Nov 2026 · Pengxiang Zhao, Xing Li, Hui-Ling Zhen, Weizhe Lin, Xianzhi Yu, Zhenhua Dong
- ResServe: Accurate and Efficient FP4 LLM Inference with Residue CompensationPapers · 5 Nov 2026 · Shuo Rong, Ruibo Fan, Jiawen Wen, Xiaowen Chu
- OpenBitSys/BitDecoding-FP4Code · 3 Nov 2026 · OpenBitSys · 49 stars
- Aligning Training and Rollout Policies for End-to-End FP4 RL on Mixture-of-Experts LLMsPapers · 3 Nov 2026 · Hei Yi Mak, Hoang Le, Mehran Taghian Jazi, Yunke Peng, Huan Lin, Zhengzhi Tang,…
- FP4Quant: Efficient 4-Bit Communication Compression for Model-Parallel LLM TrainingPapers · 3 Nov 2026 · Man Liu, Xingchen Liu, Hairui Zhao, Guangming Tan, Dingwen Tao
- TriTile: Three-Tier Attention for Video Diffusion TransformersPapers · 26 Oct 2026 · Yeonjae Kim, Omin Kwon, Seong Hoon Seo, Jae W. Lee
- BiLoCo: Binary Low-Rank Corrections for Post-Training QuantizationPapers · 23 Oct 2026 · David Jin, Beshr IslamBouli, Tarushii Goel, Han Guo, Yoon Kim
- TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language ModelsPapers · 6 Oct 2026 · Xin Wang, Hao Yu, Zhengyang Zhuge, Bochao Mao, Zheng Li, Junda Feng, Yuyan Luo,…