Output gate in long short-term memory ($o_t$)

The output gate in long short-term memory controls how much stored memory is exposed as the recurrent output of a long short-term memory network. In a common form,The memory is $c_t$, the exposed state is $h_t$, $\sigma$ is a logistic sigmoid function, and $\tanh$ is the hyperbolic tangent. The output gate in long short-term memory modulates the exposed state rather than directly multiplying the stored memory itself.^[ref]

For example, if $c_t=2$, then $\tanh (c_t)\approx 0.964$. An output gate $o_t=0.05$ yields $h_t\approx 0.0482$, whereas $o_t=0.9$ yields $h_t\approx 0.868$. Both cases can have the same current memory. A small exposed value therefore does not establish that the internal memory has been erased.

Selective exposure lets a network retain information until it becomes useful for a later prediction. The original architecture combined output gating with an input gate in long short-term memory to regulate access to persistent memory. Changing the exposed state may affect later gate decisions, so “does not directly erase” does not mean the output gate has no influence on future memory evolution.^[ref]