Past exam of the advanced topics in statistical machine learning course of the University of Oxford / 2025 / 1 / c / ii / Solution

The independent coordinates make the logarithm of the probability density ratio a sum. For coordinate

i

, its contribution to the Kullback-Leibler divergence is

E_{q} [lo g \frac{σ _{i}}{ϕ} - \frac{θ _{i}^{2}}{2 ϕ ^{2}} + \frac{θ _{i}^{2}}{2 σ _{i}^{2}}] = lo g \frac{σ _{i}}{ϕ} - \frac{1}{2} + \frac{ϕ ^{2}}{2 σ _{i}^{2}},

because

E_{q} θ_{i}^{2} = ϕ^{2}

. Thus, retaining all constants,

J (ϕ) = \frac{ϕ ^{2}}{2} (\frac{1}{σ _{1}^{2}} + \frac{1}{σ _{2}^{2}}) - 1 + lo g \frac{σ _{1} σ _{2}}{ϕ ^{2}} .

Here

σ_{1}, σ_{2} > 0

and

ϕ > 0

, so both normal densities are nondegenerate.

 Ancestors (13)

 View article source

 Discussion (0)

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.

  See all articles in the same topic Create my own version