p (θ ∣ y) = N (θ, σ_{θ}^{2}), σ_{θ}^{2} = \frac{σ ^{2} τ ^{2}}{σ ^{2} + τ ^{2}}, θ = \frac{τ ^{2}}{σ ^{2} + τ ^{2}} y .

The three quantities in the proposed identity are

lo g Z = - \frac{1}{2} [lo g (2 π (σ^{2} + τ^{2})) + \frac{y ^{2}}{σ ^{2} + τ ^{2}}],

E_{θ ∣ y} lo g L (θ) = - \frac{1}{2} lo g (2 π σ^{2}) - \frac{( y - θ ) ^{2} + σ _{θ}^{2}}{2 σ ^{2}},

D_{KL} (p (θ ∣ y) ∥ π) = \frac{1}{2} [lo g \frac{τ ^{2}}{σ _{θ}^{2}} + \frac{σ _{θ}^{2} + θ ^{2}}{τ ^{2}} - 1] .

Substitution and simplification show that the latter two expressions differ by exactly

lo g Z

, so the equality holds.

ii

Solution

lo g p (θ ∣ y) = lo g L (θ) + lo g π (θ) - lo g Z .

Taking its posterior expectation after subtracting

lo g π (θ)

yields

D_{KL} (p (θ ∣ y) ∥ π) = E_{θ ∣ y} lo g L (θ) - lo g Z .

Thus the equality holds for every proper prior and valid likelihood for which the displayed expectations are well-defined.

There are no discussions about this article yet.

There are currently no matching articles.