Positional encoding
= Positional encoding
Information representing sequence position or relative position, supplied to input representations or to attention scores. It can be fixed or learned. Without order information or a position-dependent mask, a <self-attention> layer using shared projections treats a simultaneous permutation of sequence positions equivariantly.