 Solution

Restrict to the positive-weight support. From a state $\theta$, propose ${\theta }^{\prime }$ according to $q(\theta ,{\theta }^{\prime })$ and accept with probabilityOtherwise stay at $\theta$. For distinct states, the accepted transition kernel satisfiesThus detailed balance holds, and summing over $\theta$ proves stationarity. Irreducibility of the accepted kernel makes this stationary distribution unique. Proposal irreducibility by itself is insufficient if proposed edges have zero reverse probability and are always rejected.

For convergence of the distribution from an arbitrary start, also require aperiodicity, which can always be supplied by making the chain lazy. In a finite irreducible aperiodic chain, some power $P^m$ has all entries positive. Its rows then share a fixed positive probability component, so repeated application contracts total variation distance geometrically. This proves that the chain's distribution approaches $\pi$.

The stronger fact needed for averages isOne can justify this directly by returns to a chosen state. Successive excursions are independent identically distributed cycles by the Strong Markov property. In a finite irreducible chain, return times have finite mean: from every state there is a path of uniformly bounded length and uniformly positive probability to the chosen state, giving a geometric tail bound. Apply the strong law of large numbers to cycle lengths and accumulated rewards. Their ratio converges; the mean visits per cycle, normalized by the mean cycle length, form a stationary distribution and hence equal $\pi$. The unfinished final cycle is negligible. This proves the displayed average law, even when the chain is periodic. Under these conditions, Metropolis–Hastings delivers the required posterior expectations.