architecture · first seen 15 Oct 2026
sparse activation functions
Also: ReLU
Activation functions like ReLU that output many exact zeros, enabling sparse storage of activations.
Mentions per week
PapersCodeDiscussion
Earliest sources
- BitSparse: Compact Activation Storage for ReLU Feed-Forward NetworksPapers · 15 Oct 2026 · Max Zhu, Bohang Sun, Pietro Lio, Jialin Yu
Latest sources
- BitSparse: Compact Activation Storage for ReLU Feed-Forward NetworksPapers · 15 Oct 2026 · Max Zhu, Bohang Sun, Pietro Lio, Jialin Yu