Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 37 / 6 / i

Use the Bayes' theorem to multiply the Ising model prior by the conditionally independent normal likelihood factors. For fixed observed $x$, the posterior distribution isThere is no boundary wraparound: the neighbor sets use only actual horizontal and vertical edges inside the square. Each unordered edge is counted once. In particular, the prior retains the printed minus sign in front of $J$, and the likelihood retains the seventh power. The finite state space and finite observed values make every posterior weight positive and finite, with a finite positive normalizing sum.

The prior factors involving $s_i$ are $\exp (-J{s}_i{s}_j)$ for $j\in {\mathcal{N}}_i$. A likelihood factor centered at $k$ depends on $s_i$ precisely when $i\in {\mathcal{N}}_k$, equivalently $k\in {\mathcal{N}}_i$. Those factors areOnly incident prior edges and likelihoods centered at neighboring sites are affected. In particular, the likelihood centered at $i$ does not depend on $s_i$: its mean involves only the neighbors of $i$. These are the local likelihood factors in a hidden spin field. The resulting conditional dependence can extend two lattice steps, beyond the nearest-neighbor prior interactions. For $K=1$, both collections are empty.

For $k\in {\mathcal{N}}_i$, put $a_k^{(i)}={\sum }_{j\in {\mathcal{N}}_k\setminus \{i\}}{s}_j$ and $H_i={\sum }_{j\in {\mathcal{N}}_i}{s}_j$. Removing all factors independent of $s_i$ from the posterior distribution gives, for $u\in \{-1,1\}$,A stable implementation uses the difference of the log weights. The conditional log odds areso ${\pi }_i(+1\mid s_{-i},x)=1/(1+e^{-L_i})$, a logistic function. For very large $|L_i|$, evaluate this logistic expression using the sign of $L_i$ to avoid exponential overflow.

For Gibbs sampling for a finite hidden spin field, initialize any spin configuration. At every step choose $i\in D$ uniformly, draw a fresh uniform variable, set spin $i$ to $+1$ with the probability above and to $-1$ otherwise, and leave all remaining spins fixed. This Random-scan Gibbs sampler has the stated posterior distribution invariant by detailed balance of a random-scan Gibbs sampler. All conditional probabilities are strictly between zero and one. For the one-site case, the conditional probabilities are both $1/2$.

Apply the Monte Carlo estimator to the observable $h(s)=\exp ({\sum }_{i\in D}\sqrt{2+s_i})$:The theoretical justification is the ergodic theorem for a positive Harris recurrent Markov chain, not an independent-sample law of large numbers. The Markov chain has finite state space, is irreducible, and has the strictly positive posterior distribution as invariant law, so it is positive recurrent and Harris recurrent with counting measure. The observable is bounded: $e^{K^2}\le h(s)\le e^{\sqrt{3}{K}^2}$. Thus the estimator converges almost surely to the posterior expected value from any initial configuration.

Each state also has positive self-transition probability, so the chain is an aperiodic Markov chain. Finiteness then gives geometric convergence and the relevant central limit theorem; serial dependence determines the Markov chain Monte Carlo asymptotic variance if error bars are required. A fixed finite burn-in can be discarded without changing consistency, but independence of the retained states should not be assumed.

Fix any configurations $s$ and $s^{\prime }$. List the sites where they differ and update those sites in turn to their target signs. Every specified site has positive probability $1/K^2$ of being selected. At every intermediate configuration, both values of its full conditional distribution have positive probability, since all weights $w_i(\pm 1)$ are finite and strictly positive. The finite prescribed sequence therefore has positive probability and reaches $s^{\prime }$.

Thus every configuration is accessible from every other configuration, so the sampler is an irreducible Markov chain, or equivalently phi-irreducible with counting measure. If $s=s^{\prime }$, a single update that keeps its selected spin unchanged also has positive probability. This simultaneously proves positive self-transition probability and aperiodicity, including $K=1$. The sign of finite $J$ does not alter the accessibility argument.