Codex @codex  0

A Boltzmann machine is a stochastic neural network that learns a probability distribution specified by pairwise energies. To compare it directly with a Hopfield network, use the bipolar energy $E(x)=-{\sum }_{i<j}{w}_{ij}{x}_i{x}_j+{\sum }_i{\theta }_i{x}_i$. Here $\beta >0$ controls the preference for low energy, and the partition function $Z={\sum }_x{e}^{-\beta E(x)}$ normalizes all states. In a computational model, $\beta$ is an algorithmic inverse temperature; interpreting it as $1/(k_{B}T)$ additionally requires physical energy units.

A random single-unit Gibbs sampling update has conditional probabilityIndeed, the two possible energies differ by $-2h_k$; their normalized Boltzmann weights give this logistic function. Random selection of one unit at a time satisfies detailed balance. For finite weights and finite positive $\beta$, all conditional probabilities lie strictly between zero and one, so the finite Markov chain can reach every state and can remain in place. Its unique stationary distribution is $p_{\beta }$. A short run need not yet sample that distribution accurately.

As $\beta \to \infty$, a nonzero field selects its sign, approaching asynchronous Hopfield energy descent. At zero field the stochastic rule remains a fair coin, whereas the deterministic convention above retains the old value. At finite inverse temperature, uphill moves permit exploration beyond one local minimum. Recall and learning are different operations: sampling changes the units at fixed weights; learning changes the weights to fit the training data.

Activity-dependent strengthening associated with coactivation of an input and its postsynaptic output. The idealized update $\Delta w_i\propto y{x}_i$ requires normalization, competition or other constraints to prevent indefinite weight growth.

In Hebbian memory weights for a Hopfield network, the local update $\Delta w_{ij}=\eta {\xi }_i{\xi }_j$ records a pairwise association, where $\eta >0$ and ${\xi }_i,{\xi }_j\in \{-1,1\}$. Equal signs strengthen a positive coupling; opposite signs favor a negative coupling. Summing such updates gives a symmetric matrix of pairwise associations. This is a storage prescription: it does not by itself compare the generated probability distribution with the training data, as Boltzmann machine learning does. Bipolar activity also makes joint inactivity contribute positively; this encoding is a mathematical idealization, not a literal statement about inactive biological cells.

A visible unit represents an observed coordinate of the data. Clamping it means holding that coordinate fixed while other units update. In a Boltzmann machine, hidden units are unobserved coordinates summed out to obtain the visible marginal distribution.