Information representing sequence position or relative position, supplied to input representations or to attention scores. It can be fixed or learned. Without order information or a position-dependent mask, a self-attention layer using shared projections treats a simultaneous permutation of sequence positions equivariantly.
New to topics? Read the docs here!