LSTM memory cell
= LSTM memory cell
{c}
{title2=$c_t$}
An \b[LSTM memory cell] is the state-carrying unit of a <long short-term memory> <recurrent neural network>. It maintains an internal memory $c_t$ whose update is controlled by gates, while an exposed state $h_t$ communicates selected information to the rest of the network. The memory and exposed output are different quantities.<Long Short-Term Memory by Sepp Hochreiter and Jürgen Schmidhuber>[Memory-cell construction and gated access to persistent state.]{ref}
In a simplified modern form, omitting peepholes,
$$
c_t=f_t\odot c_{t-1}+i_t\odot g_t,\qquad
h_t=o_t\odot\tanh(c_t).
$$
The <LSTM forget gate> $f_t$ regulates retention, the <LSTM input gate> $i_t$ regulates the candidate write $g_t$, and the <LSTM output gate> $o_t$ regulates exposure. The componentwise product $\odot$ allows different memory coordinates to make different decisions.<LSTM: A Search Space Odyssey by Klaus Greff and others>[Section II and Figure 1; the displayed simplified variant omits the baseline's peephole connections.]{ref}
For a scalar example, $c_{t-1}=0.8$, $f_t=0.9$, $i_t=0.2$ and $g_t=-0.5$ give $c_t=0.62$. If $o_t=0.1$, the output is only $h_t=0.1\tanh(0.62)\approx0.0551$. A small output can coexist with substantial stored state; it does not demonstrate memory erasure.
The additive state path was designed to help learning across long delays. The original 1997 architecture had input/output gating; adaptive forgetting was introduced later for continuing streams. Preserving a direct gradient path addresses an important learning difficulty but does not guarantee unlimited or perfectly reliable memory for every task.<Long Short-Term Memory by Sepp Hochreiter and Jürgen Schmidhuber>[Constant error flow and delayed-dependency experiments.]{ref}<Learning to Forget: Continual Prediction with LSTM by Felix A. Gers, Jürgen Schmidhuber and Fred Cummins>[Adaptive forget gate and continual prediction.]{ref}