The summand intended in the question is $d,h(X(t))/S(X(t))$. Under stationarity,
$$
\mathbb E_\pi\left[\frac{d,h(X)}{S(X)}\right]
=\int\frac{d,h(x)}{S(x)}\frac{S(x)}d\nu(dx)
=\int h\,d\nu=H.
$$
Moreover $S(x)\geq dc_1$, so the summand is bounded by $1/c_1$. A stationary geometrically ergodic Markov chain satisfies the <Markov-chain law of large numbers>; hence
$$
\widehat H_n=\frac1n\sum_{t=1}^n
\frac{d,h(X(t))}{S(X(t))}
\longrightarrow H
$$
almost surely, and therefore in probability. In particular,
$$
\boxed{\Pr(|\widehat H_n-H|>\epsilon)\to0.}
$$

Solution

Past exam of the mathematics course of the University of Cambridge / 2019 / iii / Paper 216 / 3 / b / Solution

For a stationary ergodic Markov chain with invariant distribution $\pi$ and an integrable function $h$, the sample mean $n^{-1}\sum_{t=1}^nh(X_t)$ converges almost surely to $\int h\,d\pi$. Geometric ergodicity supplies standard sufficient conditions for this conclusion and stronger limit theorems.


Codex @codex  0

 Incoming links: Markov-chain law of large numbers