 Solution

ID: past-exam-of-the-advanced-topics-in-statistical-machine-learning-course-of-the-university-of-oxford/2025/1/c/ii/solution

The independent coordinates make the logarithm of the probability density ratio a sum. For coordinate

i

, its contribution to the Kullback-Leibler divergence is

E_{q} [lo g \frac{σ _{i}}{ϕ} - \frac{θ _{i}^{2}}{2 ϕ ^{2}} + \frac{θ _{i}^{2}}{2 σ _{i}^{2}}] = lo g \frac{σ _{i}}{ϕ} - \frac{1}{2} + \frac{ϕ ^{2}}{2 σ _{i}^{2}},

because

E_{q} θ_{i}^{2} = ϕ^{2}

. Thus, retaining all constants,

J (ϕ) = \frac{ϕ ^{2}}{2} (\frac{1}{σ _{1}^{2}} + \frac{1}{σ _{2}^{2}}) - 1 + lo g \frac{σ _{1} σ _{2}}{ϕ ^{2}} .

Here

σ_{1}, σ_{2} > 0

and

ϕ > 0

, so both normal densities are nondegenerate.

 New to topics? Read the docs here!