A word embedding represents a word by a vector, usually learned so that relationships in the data become useful relationships between vectors. A vocabulary of words can be stored as a matrix with rows and columns: selecting a word selects one row. Unlike a separate indicator for each word, a dense representation can share predictive information across words.
For example, training a neural language model can place words occurring in similar contexts near one another, making evidence about “cat” useful when predicting contexts involving “dog”. This is statistical similarity, not a guarantee of synonymy: opposite words can occur in similar contexts too. The 2003 neural probabilistic model learned these vectors jointly with next-word probabilities.[ref]
A Neural Probabilistic Language Model by Yoshua Bengio and others. abstract and §2, shared word-feature matrix
New to topics? Read the docs here!