method · first seen 4 Oct 2026
quantization
Also: FP4, FP8, INT8, compression, FP8 training, quantisation, 4-bit training
Reducing the numerical precision of model weights, activations, or gradients to lower-bit representations to save memory and compute.
Quantization lowers the numerical precision of weights, activations or gradients (for example to INT8, FP8 or 4-bit formats) to cut memory and compute costs. The sources show it as routine infrastructure, not a new idea. Papers cover a multi-precision LLM weight format built from refinement bit-planes, joint SVD pruning and quantization, INT8 deployment on microcontrollers, and an offline maternal-health assistant that uses quantized models. Repositories tag it as one ingredient in local inference engines, GGUF tooling and low-VRAM image generation. The variety of authors and settings reflects broad, ongoing use rather than a newly spreading technique.
The individual sub-methods (multi-precision prefix formats, differentiable bit-widths, FP4 training) are worth tracking. The umbrella term has been mainstream for years, so its rising count mostly mirrors overall activity in efficient inference.
Analyst note (established), 17 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 35 on 14 Nov 2026 · 85 mentions in 28 days · 85 independent groups · papers → code → discussion · first seen 6 weeks ago · acceleration -0.13
Earliest sources
- Investigating Model Compression for Neural Machine Translation in the Biomedical DomainPapers · 4 Oct 2026 · Maria Zafar, Souhail Bakkali, Rejwanul Haque
- drowzeys/keys-Mac-TensorFold-MiniMax-H3-MLXCode · 4 Oct 2026 · drowzeys · 31 stars
- StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMsPapers · 5 Oct 2026 · Zhe Wei, Mengqi Guo, Yuan Yuan, Jiunn Bin Lim, Boyi Pan, Michael Bi Mi
- Differentiable Bit-Widths: Co-optimizing Pruning and Quantization via SVD for Ultra-Efficient LLM CompressionPapers · 5 Oct 2026 · Hankyul Kang, Jongbin Ryu
- Anosognosia in LLMs: Probing Self-Awareness of Quantized Computational SubstratePapers · 5 Oct 2026 · Yoshihiro Izawa, Gouki Minegishi, Yoko Yamakata
- Scaling Down the Scaling Laws: Parameter Efficiency and Compute-Optimal Training in Resource-Constrained Large Language ModelsPapers · 5 Oct 2026 · Joe Dwyer
Latest sources
- Reasoning Again with a Quarter of the Experts: Retry First, Then TrainPapers · 16 Nov 2026 · Anik Jha
- manuelgscom/inside-llmCode · 15 Nov 2026 · manuelgscom · 92 stars
- Learning From the Best: What Makes Popular Hugging Face Models?Papers · 15 Nov 2026 · Yinan Wu, Zhou Yang, Bowen Xu, Bo Wang, Bach Le, David Lo
- Offline-First Quantized AI: Design and Evaluation of an On-Device Maternal-Health Assistant for Low-Connectivity EnvironmentsPapers · 15 Nov 2026 · Okenla AbdulQahhar Abimbola, Maurice Abraham John, Adepoju Jeremiah Moyondafoluw…
- Does synthetic pre-training pay off on a microcontroller? Accuracy, latency and energy across two MCUsPapers · 15 Nov 2026 · Achintya Yadav
- alexturkey/z-image-turbo-pascal-4gbCode · 14 Nov 2026 · alexturkey · 144 stars
- PBit1: One Programmable P-Bit per Parameter - A Finite-Control Representation of Pretrained NetworksPapers · 14 Nov 2026 · Omar Ramadan
- A predictive, per-engine energy roofline of the Apple Neural EnginePapers · 14 Nov 2026 · Spencer H. Bryngelson
- End‑to‑End 4‑Bit TrainingDiscussion · 13 Nov 2026 · stokedmartin · 24 points
- Choosing Quantized Code LLMs by File Size and Energy per Solved TaskPapers · 13 Nov 2026 · Aditi Patodiya
- User-Schedulable Approximations for Quantized Tensor ProgramsPapers · 12 Nov 2026 · Alex Reinking
- Advancing Quantization for R&R Models with Microscaling FormatsPapers · 12 Nov 2026 · Yavuz Yetim, Bor-Yiing Su, Zhe Wang, Arthur Liu, Pranav Sharma, Ying Wang, Randy…