Transformer block
= Transformer block
A standard transformer block combines <multi-head attention> with a positionwise <feedforward neural network>, wrapping each sublayer in a <residual connection> and <layer normalization>. An encoder-decoder transformer's decoder also has a cross-attention sublayer. Pre-normalization and post-normalization variants differ in where <layer normalization> occurs relative to the sublayer and <residual connection>.