Multi-head attention
= Multi-head attention
Several independently learned query, key, and value projections apply <scaled dot-product attention> in parallel. Concatenating their outputs and applying a learned output projection allows different heads to mix sequence information using different learned representations.