A standard transformer block combines multi-head attention with a positionwise feedforward neural network, wrapping each sublayer in a residual connection and layer normalization. An encoder-decoder transformer's decoder also has a cross-attention sublayer. Pre-normalization and post-normalization variants differ in where layer normalization occurs relative to the sublayer and residual connection.
New to topics? Read the docs here!