A sequence model built from stacked transformer blocks, using self-attention to mix information between positions and positionwise feedforward neural networks to transform their representations. Positional encoding supplies sequence-order information. An autoregressive decoder uses causal self-attention; generation remains sequential even when training computations over known sequence positions are parallel.
New to topics? Read the docs here!