= Sepp Hochreiter
{c}
<Sepp Hochreiter> is a <computer scientist> working on <machine learning> and <neural networks>. His institutional profile identifies <long short-term memory> among his research topics. The1997paper with <Jürgen Schmidhuber> connects his earlier analysis of decaying learning signals to a recurrent architecture designed to preserve useful error flow.<Sepp Hochreiter profile by JKU>[Institutional identification and research topics]{ref} <Long Short-Term Memory by Sepp Hochreiter and Jürgen Schmidhuber>[Abstract; Section3 and its account of the1991analysis]{ref}
A toy <chain rule> calculation explains the motivation. If a learning signal is multiplied by $1/2$ at each of twenty successive steps, its contribution is scaled by
$$
(1/2)^{20}=1/1048576\approx9.54\times10^{-7}.
$$
That small contribution can make an early input difficult to associate with a much later training signal. The example illustrates the <vanishing gradient problem>; it is not a measurement of every <recurrent neural network>.
The <long short-term memory> design introduced a controlled internal path with constant error flow, while multiplicative gates regulate access. Its artificial-data experiments included successful learning across delays exceeding1000discrete steps. The reported capability belongs to those experiments and their architecture-specific learning rules, rather than implying that any trained instance will remember every input indefinitely.<Long Short-Term Memory by Sepp Hochreiter and Jürgen Schmidhuber>[Abstract; Sections3–5: error-flow construction and artificial tasks]{ref}
\Image[https://www.jku.at/fileadmin/_processed_/a/4/csm_Sepp_Hochreiter_9f3c8a11bf.jpg]
{title=Sepp Hochreiter, identified on his JKU institutional profile}
{description=External institutional portrait from <Sepp Hochreiter profile by JKU>.}
Back to article page
