= Two-spin Boltzmann learning
{title2=$Z(J)=4\cosh J,\quad\ell'(J)=1/2-\tanh J$}
Take two visible bipolar units, no <hidden units>, zero thresholds, $\beta=1$, and $E_J(x)=-Jx_1x_2$. As original <training data>, assign probabilities $3/8$ each to $(+,+)$ and $(-,-)$ and $1/8$ each to $(+,-)$ and $(-,+)$. The data pair product averages to $1/2$. Enumeration of all four states gives two weights $e^J$ and two $e^{-J}$, hence the <partition function> $Z(J)=4\cosh J$ and
$$
\langle x_1x_2\rangle_-=\tanh J,\qquad
\ell(J)=\frac J2-\log(4\cosh J).
$$
Starting at $J=0$, the positive-minus-negative difference is $1/2-0$. A <gradient ascent> step of size $1/2$ gives $J=1/4$. Each aligned state now has <probability> $0.311230$ and each opposed state $0.188770$, bringing the model toward the data. The <log-likelihood> increases from $-1.386294$ to $-1.292224$. Solving $\ell'(J)=0$ gives $J_*=(\log3)/2$, which reproduces the data exactly. Since $\ell''(J)=-1/\cosh^2J<0$, this is the unique maximum. These are exact enumerations and their decimal evaluations, with no sampling or laboratory measurements.
\Image[/boltzmann-pair-learning.png]
{title=Exact probabilities and positive-minus-negative learning for two visible units}
{description=Left: data and model <probabilities> before and after one <gradient ascent> step. Right: the data pair-product average and the model average determine the <log-likelihood> <derivative>; their intersection gives the optimal coupling.}
Back to article page
