= Ashish Vaswani
{c}
<Ashish Vaswani> is a <machine learning> researcher and coauthor of the 2017 paper introducing the <Transformer model>. The paper's shared-contribution statement credits him with jointly designing and implementing the first <Transformer models>. It explicitly states that author order is random, so his first position should not be read as sole credit for the architecture.<Attention Is All You Need by Ashish Vaswani and others>[Original byline and contribution footnote]{ref}
The <Transformer model> uses <self-attention> to combine information from different positions in a sequence. In <scaled dot-product attention>, a query is compared with keys using <dot products>, then the <softmax function> supplies weights for combining the corresponding values. The model connects this mechanism to a complete <neural network> for tasks such as <machine translation>.<Attention Is All You Need by Ashish Vaswani and others>[Sections3.2–3.2.3: attention computation and its uses]{ref}
An original one-dimensional example makes the weighting concrete. Choose query $q=1$, keys $k_1=0$, $k_2=\ln3$, and values $v_1=2$, $v_2=6$. With key dimension $d_k=1$, the scaled scores are $0$ and $\ln3$. The <softmax function> gives weights $1/4$ and $3/4$, so the output is $(1/4)2+(3/4)6=5$. The input values are mixed according to the query–key comparison; this invented example is not a learned translation result.
The original evaluation reported 28.4 <BLEU> for its large model on the English-to-German newstest2014 test. That is a result under a specific training and evaluation setup, rather than a universal score for all <Transformer models>.<Attention Is All You Need by Ashish Vaswani and others>[Section6.1 and Table2: big-model translation evaluation]{ref}
Back to article page
