= Solution
A counterexample can use an irreducible, aperiodic <reversible Markov chain>, so failure is not caused by a pathological transition structure. Take two binary coordinates, enumerate the four states as $00,01,10,11$, and let $\pi$ be their <uniform distribution>. Use the <Markov kernel>
$$
K=\begin{pmatrix}
1/4&1/4&1/4&1/4\\
1/4&1/2&1/8&1/8\\
1/4&1/8&1/2&1/8\\
1/4&1/8&1/8&1/2
\end{pmatrix}.
$$
Its rows sum to one and it is symmetric, proving <detailed balance> with $\pi$. Every entry is positive. Let the univariate parametric family be $\widetilde{\mathcal Q}=\{\operatorname{Bernoulli}(r):0\leq r\leq2/5\}$.
For a product initial <probability distribution> with parameters $r_1,r_2$, additivity of the <Kullback-Leibler divergence> gives
$$
\operatorname{KL}(\nu\Vert\pi)=\sum_{i=1}^2
\{r_i\log(2r_i)+(1-r_i)\log(2(1-r_i))\}.
$$
Each summand has <derivative> $\log(r_i/(1-r_i))<0$ on $(0,2/5]$, so the unique optimum at $t=0$ has $r_1=r_2=2/5$. Thus
$$
q^{*0}=(9,6,6,4)/25,
\qquad Kq^{*0}=(25,26,26,23)/100.
$$
On the other hand the allowed initial <probability distribution> $\delta_{00}$, obtained with $r_1=r_2=0$, is sent by one step to $\pi$. Its <Kullback-Leibler divergence> is zero, the smallest possible value. Consequently
$$
\boxed{q^{*1}=\pi\ne Kq^{*0}.}
$$
The transition reduces <Kullback-Leibler divergence> for each input, but may reduce it by different amounts for different inputs. It therefore need not preserve the ordering of competing initial <probability distributions>, which is exactly why <Markov chain variational inference> must optimize the initial parameters for its chosen number of steps.
Back to article page
