Yoshua Bengio coauthored a 2003 neural language model that learns word embeddings jointly with a probability model for word sequences. The aim is to share statistical information among related contexts rather than assign unrelated parameters to every observed sequence.[ref]
For example, a learned vector for a word can contribute to predictions in many contexts. Updating that vector through one training example can therefore affect predictions elsewhere, providing a mechanism for generalization. This is an explicit computational use of a distributed representation, not a claim that nearby vectors always encode the same linguistic relationship. Bengio also coauthored Deep Learning by Ian Goodfellow and others.
New to topics? Read the docs here!