Transformer model
= Transformer model
A sequence model built from stacked <transformer blocks>, using <self-attention> to mix information between positions and positionwise <feedforward neural networks> to transform their representations. <Positional encoding> supplies sequence-order information. An autoregressive decoder uses <causal self-attention>; generation remains sequential even when training computations over known sequence positions are parallel.