Past exam of the advanced topics in statistical machine learning course of the University of Oxford / 2025 / 1 / c / iii / Solution

The printed minimization of the evidence lower bound has no solution. To distinguish that literal result from the usual variational inference optimum, put $S={\sigma }_1^{-2}+{\sigma }_2^{-2}>0$. The preceding Kullback-Leibler divergence satisfiesIts unique global minimum occurs at ${\varphi }^2=2/S$. Since $\mathcal{L}=\log p(\mathcal{D})-\mathcal{J}$, this is the unique global maximum of the evidence lower bound:The optimal common variance is the harmonic mean of the two posterior variances. However, $\mathcal{J}(\varphi )\to +\infty$ as either $\varphi \downarrow 0$ or $\varphi \to \infty$. ThereforeThe source PDF confirms the minimization wording; replacing it by maximization would give the finite value above, consistently with part (i).