θ ∣ y \sim N (θ, σ_{θ}^{2}), θ = \frac{τ ^{2}}{σ ^{2} + τ ^{2}} y, σ_{θ}^{2} = \frac{σ ^{2} τ ^{2}}{σ ^{2} + τ ^{2}} .

b

Solution

The evidence is the convolution of the likelihood with the prior:

Z = p (y) = \frac{1}{2 π ( σ ^{2} + τ ^{2} )} exp [- \frac{y ^{2}}{2 ( σ ^{2} + τ ^{2} )}] .

Since

E_{θ ∣ y} (y - θ)^{2} = (y - θ)^{2} + σ_{θ}^{2},

we obtain

E_{θ ∣ y} lo g L (θ) = - \frac{1}{2} lo g (2 π σ^{2}) - \frac{( y - θ ) ^{2} + σ _{θ}^{2}}{2 σ ^{2}} .

N (0, τ^{2})

D_{KL} {p (θ ∣ y) ∥ π} = \frac{1}{2} [\frac{σ _{θ}^{2} + θ ^{2}}{τ ^{2}} - 1 + lo g (\frac{τ ^{2}}{σ _{θ}^{2}})] .

Substituting

θ = τ^{2} y / (σ^{2} + τ^{2})

and

σ_{θ}^{2} = σ^{2} τ^{2} / (σ^{2} + τ^{2})

into parts c and d and simplifying gives

E_{θ ∣ y} lo g L (θ) - D_{KL} {p (θ ∣ y) ∥ π} = - \frac{1}{2} lo g {2 π (σ^{2} + τ^{2})} - \frac{y ^{2}}{2 ( σ ^{2} + τ ^{2} )} = lo g Z .

Thus the equality holds.

It holds generally whenever the posterior is well defined and the expectations exist. Bayes' theorem gives

lo g \frac{p ( θ ∣ y )}{π ( θ )} = lo g L (θ) - lo g Z .

Taking posterior expectations yields

D_{KL} {p (θ ∣ y) ∥ π} = E_{θ ∣ y} lo g L (θ) - lo g Z,

which rearranges to the claimed evidence decomposition.

There are no discussions about this article yet.

There are currently no matching articles.