Two-spin Boltzmann learning ($Z(J)=4\cosh J,{\ell }^{\prime }(J)=1/2-\tanh J$)

Take two visible bipolar units, no hidden units, zero thresholds, $\beta =1$, and $E_J(x)=-J{x}_1{x}_2$. As original training data, assign probabilities $3/8$ each to $(+,+)$ and $(-,-)$ and $1/8$ each to $(+,-)$ and $(-,+)$. The data pair product averages to $1/2$. Enumeration of all four states gives two weights $e^J$ and two $e^{-J}$, hence the partition function $Z(J)=4\cosh J$ andStarting at $J=0$, the positive-minus-negative difference is $1/2-0$. A gradient ascent step of size $1/2$ gives $J=1/4$. Each aligned state now has probability $0.311230$ and each opposed state $0.188770$, bringing the model toward the data. The log-likelihood increases from $-1.386294$ to $-1.292224$. Solving ${\ell }^{\prime }(J)=0$ gives $J_{\ast }=(\log 3)/2$, which reproduces the data exactly. Since ${\ell }^{\prime \prime }(J)=-1/{\cosh }^{2}J<0$, this is the unique maximum. These are exact enumerations and their decimal evaluations, with no sampling or laboratory measurements.