 Felix A. Gers

Felix A. Gers is a researcher in machine learning and recurrent neural networks. His institutional profile connects his research with long short-term memory; the full-name byline of his2000paper verifies the name used here.^{[ref] [ref]}

With Jürgen Schmidhuber and Fred Cummins, he introduced the adaptive LSTM forget gate to address continual input streams lacking externally marked reset points. The gate lets a LSTM memory cell learn when to reduce retained state, rather than requiring every sequence boundary to be supplied from outside.^[ref]

A scalar toy update makes selective retention tangible:Here $f_t$ is a retention multiplier and $u_t$ a new contribution. With $c_{t-1}=8$ and $u_t=0$, choices $f_t=1$, $0.25$ and $0$ retain $8$, $2$ and $0$ respectively. This arithmetic illustrates one role of a LSTM forget gate; it omits the other gates and training equations of a complete long short-term memory network. Forgetting stored state is also distinct from discarding all learned network parameters.