Causal self-attention
= Causal self-attention
<Self-attention> whose output at sequence position $t$ may depend only on positions at or before $t$. In a score <matrix>, set forbidden future-position scores to negative infinity before applying the <softmax function>; the corresponding weights become zero. This permits parallel computation over a known training sequence without allowing future tokens to influence earlier predictions.