= Solution
Write $\pi(x)=\mu(x\mid y)$. <Mean-field variational inference> minimizes the reverse <Kullback-Leibler divergence>
$$
\operatorname{KL}(q\Vert\pi)=\int q(x)\log\frac{q(x)}{\pi(x)}\,dx,
\qquad q(x)=\prod_{i=1}^pq_i(x_i),\quad \int q_i=1.
$$
Each $q_i$ is a <probability density function>; without further parametric restrictions, the optimization is over all such product <probability distributions>. Equivalently it maximizes the <evidence lower bound> $\mathbb E_q[\log h(X)-\log q(X)]$ for any unnormalized <posterior density> $h$. Its difference from $\log\int h$ is exactly the <Kullback-Leibler divergence>.
Fix $q_2,\ldots,q_p$ and write
$$
F(x_1)=\mathbb E_{q_{-1}}[\log\pi(x_1,X_{-1})],\qquad
Z_1=\int e^{F(u)}\,du.
$$
Assume $0<Z_1<\infty$ and that the displayed <expected values> and objective decomposition are well defined. The optimal factor is
$$
\boxed{q_1^*(x_1)=\frac{\exp\{\mathbb E_{q_{-1}}[\log\pi(x_1,X_{-1})]\}}{Z_1}.}
$$
Indeed, the part of the objective depending on $q_1$ is
$$
\int q_1\log q_1-\int q_1F
=\operatorname{KL}(q_1\Vert q_1^*)-\log Z_1.
$$
The remaining term $\sum_{i=2}^p\int q_i\log q_i$ is fixed. <Gibbs inequality> gives nonnegativity of the <Kullback-Leibler divergence>, with equality precisely at $q_1=q_1^*$ almost everywhere. This proves global optimality of that coordinate update; it does not assert global optimality of a sequence of coordinate updates for the full nonconvex product-family problem.
The printed upper index $k$ in the list of remaining coordinates is inconsistent with the dimension $p$; the natural interpretation is $k=p$. Also, if the exponential expression has zero or infinite <normalizing constant>, or the <expected values> are undefined, the usual coordinate formula needs additional support or integrability hypotheses. A restricted parametric factor family need not contain this unrestricted optimal factor.
Back to article page
