architecture · first seen 6 Oct 2026
pre-layer normalization
Also: Pre-Norm, Pre-LN transformer
A transformer design that applies layer normalization before each sublayer inside the residual branch.
Mentions per week
PapersCodeDiscussion
Earliest sources
- LayerRoPE: Dynamic Depth-wise Magnitude & Angular SuperpositionPapers · 6 Oct 2026 · Shikhar Srivastava, Christopher Kanan
- DepthGraft: Structural Regularization through Hierarchical Cross-Layer KV ReconstructionPapers · 2 Nov 2026 · Xiaohan Qin, Xiangdong Zhang, Yu Wang, Huaijin Wu, Zhuo Xia, Yebin Yang, Junchi…
Latest sources
- DepthGraft: Structural Regularization through Hierarchical Cross-Layer KV ReconstructionPapers · 2 Nov 2026 · Xiaohan Qin, Xiangdong Zhang, Yu Wang, Huaijin Wu, Zhuo Xia, Yebin Yang, Junchi…
- LayerRoPE: Dynamic Depth-wise Magnitude & Angular SuperpositionPapers · 6 Oct 2026 · Shikhar Srivastava, Christopher Kanan