= LayerNorm in the PyTorch documentation
{c}
{title2=2.15}
https://docs.pytorch.org/docs/2.15/generated/torch.nn.LayerNorm.html
PyTorch 2.15, LayerNorm equation and Parameters/eps: epsilon is inside the square root after adding it to coordinate variance; correction=0 uses the divisor equal to the number of normalized coordinates. Parameters are learned per element when elementwise_affine is enabled; eps defaults to $10^{-5}$. Statistics come from the same input in training and evaluation modes. These documented conventions define the stabilized map used in the accompanying independent derivations.
Back to article page
