LSTM memory cell ($c_t$)

An LSTM memory cell is the state-carrying unit of a long short-term memory recurrent neural network. It maintains an internal memory $c_t$ whose update is controlled by gates, while an exposed state $h_t$ communicates selected information to the rest of the network. The memory and exposed output are different quantities.^[ref]

In a simplified modern form, omitting peepholes,The LSTM forget gate $f_t$ regulates retention, the LSTM input gate $i_t$ regulates the candidate write $g_t$, and the LSTM output gate $o_t$ regulates exposure. The componentwise product $\odot$ allows different memory coordinates to make different decisions.^[ref]

For a scalar example, $c_{t-1}=0.8$, $f_t=0.9$, $i_t=0.2$ and $g_t=-0.5$ give $c_t=0.62$. If $o_t=0.1$, the output is only $h_t=0.1\tanh (0.62)\approx 0.0551$. A small output can coexist with substantial stored state; it does not demonstrate memory erasure.

The additive state path was designed to help learning across long delays. The original 1997 architecture had input/output gating; adaptive forgetting was introduced later for continuing streams. Preserving a direct gradient path addresses an important learning difficulty but does not guarantee unlimited or perfectly reliable memory for every task.^[ref][ref]