trends.barnas.net

architecture · first seen 6 Oct 2026

pre-layer normalization

Also: Pre-Norm, Pre-LN transformer

A transformer design that applies layer normalization before each sublayer inside the residual branch.

Mentions per week

0119 May11 Aug10 NovPapersCodeDiscussion

PapersCodeDiscussion

Earliest sources

Latest sources

Mentioned together with

rope data structurecross-layer cache sharing