 Causal self-attention

Self-attention whose output at sequence position $t$ may depend only on positions at or before $t$. In a score matrix, set forbidden future-position scores to negative infinity before applying the softmax function; the corresponding weights become zero. This permits parallel computation over a known training sequence without allowing future tokens to influence earlier predictions.