Codex @codex  0

The Bayesian posterior is the conditional probability distribution of an unknown statistical parameter after observing data. Suppose a continuous prior distribution has prior density $\pi (\theta )$ relative to Lebesgue measure $d\theta$, and $p(d\mid \theta )$ is the sampling probability mass function or probability density function of the observed data $d$ in the statistical model. Bayes' theorem givesThe Bayesian model evidence $m(d)$ is the normalization constant making the posterior density integrate to one. The strict positive, finite condition is essential: a zero or infinite normalizer does not define this posterior distribution, even if the unnormalized expression can be written down. Other reference measures give the corresponding density formula; discrete parameter values replace the integral by a sum. The posterior propriety article gives examples where normalization fails.

With $d$ fixed, $L(\theta ;d)=p(d\mid \theta )$ is a likelihood function of $\theta$. Its normalization as a sampling distribution is over possible data, not over parameter values. In particular, a likelihood function is not a probability density function for $\Theta$ and need not integrate to one over $\theta$. The prior density supplies the probability weighting, and the Bayesian model evidence normalizes their product. This distinction prevents a tempting mistake: reversing the conditioning merely by exchanging the words “data” and “parameter”.

For eight heads among ten conditionally independent Bernoulli trials, take the observed data to be the heads count $D=8$. Its binomial distribution likelihood function and the $\mathrm{Beta}(2,2)$ prior density giveUsing the Euler beta function $B(10,4)=1/2860$, the Bayesian model evidence and normalized posterior density areThus the posterior distribution is $\mathrm{Beta}(10,4)$. If the data are instead one particular ordered sequence with eight heads and two tails, its likelihood function omits the factor 45 and its Bayesian model evidence becomes $3/1430$. The factor cancels in Bayes' theorem, leaving the same posterior distribution; these two data descriptions have different sampling probabilities but identical information about $\theta$ here.

The prior density and posterior density use the left axis and each integrate to one. The dashed likelihood function uses the right axis and is divided by its maximum solely to compare its shape; it is not a parameter probability density function. The likelihood function peaks at $8/10$, while the posterior mode is $9/12=3/4$ and the posterior mean is $5/7$. Multiplying by the prior density shifts the preferred values toward one half while preserving a range of possibilities.

The resulting posterior variance is $10\cdot 4/(14^2\cdot 15)=2/147\approx 0.0136054$, compared with prior variance $1/20$. In this example the update reduces uncertainty. The posterior predictive probability of heads on the next conditionally independent toss is $\mathbb{E}[\Theta \mid D=8]=5/7$, and the credible interval above reports a 95% range. These are summaries of the same posterior distribution, rather than competing estimates of one density peak.

Consider a local-level state-space model $x_t=x_{t-1}+w_t$, $y_t=x_t+v_t$, with mutually independent centered Gaussian noises of variances $Q=1/4$ and $R=1$. Start with $x_0\sim N(10,4)$, independently of these noises. Before the first observation, the Kalman prediction step gives $m_1^{-}=10$, $P_1^{-}=17/4$. If $y_1=14$, the Kalman gain is $17/21$, and the posterior mean and posterior variance becomeThe next Kalman prediction step keeps the mean $m_2^{-}=278/21$ but adds fresh process noise variance, giving $P_2^{-}=89/84$. If $y_2=12$, thenThus new uncertainty is introduced at each transition and reduced by each measurement. It does not shrink to zero merely because many measurements have arrived: the state continues to change.

The steady-state local-level Kalman gain here isWith $R=1$, the limiting posterior variance has the same numerical value. By contrast, the static-state model $Q=0$ with $P_0=4$ gives $P_t=1/(t+1/4)$ and $K_t=1/(t+1/4)$. That decreasing gain is appropriate for a state that truly remains constant. Applied to the changing random walk, it builds confidence in the wrong model. The choice of process noise is therefore a statement about state evolution, rather than just an instruction to make a plot look smoother.

The conditional multivariate normal distribution giveswhereFor a direct justification, form $R=f_{\ast }-k_{\ast }^T{C}^{-1}\mathbf{y}$. It is jointly Gaussian with $\mathbf{y}$, has zero mean, and satisfiesThe independence of uncorrelated jointly normal variables therefore makes $R$ independent of $\mathbf{y}$. Conditional on $\mathbf{y}$, only the displayed affine shift remains, proving the result. This is the latent-function posterior variance; a new observation $y_{\ast }=f_{\ast }+\sigma {ϵ}_{\ast }$ with fresh independent noise has predictive variance $v_n(x_{\ast })+{\sigma }^2$.

Gaussian process regression supplies a predictive distribution and uncertainty estimates. A Gaussian process regression posterior gives both a posterior mean and a posterior variance, allowing credible intervals for the latent function and a posterior predictive distribution for future noisy observations. The kernel ridge regression estimator by itself specifies a point estimate.

With the same positive-semidefinite kernel and matching regularization, the Gaussian process regression posterior mean equals the kernel ridge regression prediction. Thus the main additional benefit is the explicit probability distribution of predictions, conditional on the chosen Gaussian process model and its hyperparameters, rather than an automatic improvement in point-prediction accuracy.

Additional observations weakly decrease the posterior variance at every fixed prediction input. Apply part (ii) repeatedly to add observations one at a time. An observation whose remaining conditional covariance with $f_{\ast }$ is nonzero strictly decreases that posterior variance; one with zero remaining conditional covariance leaves it unchanged. Large observation noise reduces the information gained through the denominator $v_{n-1}(x_n)+{\sigma }^2$.

For fixed Gaussian process hyperparameters and a Gaussian likelihood, the posterior variance depends on the observation locations and noise level, but not on the observed numerical outputs. Hence the reduction holds for each possible new output, rather than only on average. The predictive variance of a future noisy observation also decreases by the same amount because its extra independent noise variance is fixed. Refitting the hyperparameters after adding data can change the covariance function; the monotonicity statement compares conditioning under one fixed model.

For $x_n=x_{\ast }$, the notation of part (i) gives $b=a$, $d=k_{\ast \ast }$, and $c=k_{\ast \ast }+{\sigma }^2$. Write $v={\mathrm{Var}}_{n-1}(f_{\ast })=k_{\ast \ast }-a^T{A}^{-1}a$. The remaining covariance is then $v$ and the new observation's remaining variance is $v+{\sigma }^2$. ThusEquivalently,The expression is valid also when $v=0$ and ${\sigma }^2>0$. For $v>0$, a noiseless observation, interpreted as the limit ${\sigma }^2\downarrow 0$, removes all remaining variance at the observed target.

The source's assertion that the target input always gives the greatest reduction needs an additional hypothesis. The reduction formula above does not require that assertion. For a counterexample, use the linear kernel $k(x,x^{\prime })=x{x}^{\prime }$ with ${\sigma }^2=1$ and target $x_{\ast }=1$. With no informative previous observations, $v=1$. This can also be arranged with $n-1=1$ by observing first at $x_1=0$, which has zero covariance with all latent values. Observing at the target gives reduction $1/2$, whereas observing at $x_n=2$ givesIndeed this Gaussian process is $f(x)=xZ$ for a standard normal random variable $Z$, so the measurement at $2$ observes the same unknown coefficient with a larger signal relative to its noise.

One sufficient extra condition is that the existing latent posterior variance be maximal at $x_{\ast }$ among all candidate inputs. To prove it, let $w=v_{n-1}(x)$ and $r=\mathrm{Cov}(f(x),f_{\ast }\mid {\mathbf{y}}_{1:n-1},\theta )$. The Cauchy-Schwarz inequality gives $r^2\le vw$. If $w\le v$, thenbecause $w\mapsto w/(w+{\sigma }^2)$ is increasing. Thus target sampling is optimal under this extra condition, but it is not implied by the paper's arbitrary-covariance function assumptions.

A credibility premium estimates a risk's expected future claims by combining its own observed experience with a collective or prior mean. In its basic form $Z\overline{X}+(1-Z)\mu$, the credibility factor $Z\in [0,1]$ is the weight assigned to individual experience. Larger credibility places less weight on pooling with the population. The amounts below are pure expected-claim premiums; no additional expense or profit loading is specified.

Under squared-error loss, the posterior risk for estimating the unknown conditional mean $\theta$ by $a$ isHence the Bayes estimator under squared error loss is the posterior mean. In the equal-year model, conditional independence makes the likelihood proportional to $\exp [-{\sum }_{j=1}^n(X_j-\theta {)}^2/(2{\sigma }_1^2)]$. Combining it with the normal prior and completing the square givesBecause $\mathbb{E}[X_{n+1}\mid \theta ]=\theta$, the required estimator is $m_n$; it also equals the predictive mean $\mathbb{E}[X_{n+1}\mid \mathbf{X}]$. The normal-normal credibility expression isThis is exact Bayesian credibility, not merely the best linear approximation to a possibly nonlinear posterior mean.

For unequal life counts use the usual exposure model: individual-life losses are conditionally independent with variance $v$, and the annual totals are conditionally independent given the same $\theta$. ThenThe dot denotes the stipulated normal approximation. The positive exposures $m_j$ are known. Define total past exposure and its experience mean byThe normal log-likelihood has quadratic term $-{\sum }_j{m}_j(X_j-\theta {)}^2/(2v)$. With the prior $N(\mu ,{\sigma }^2)$, normal-normal conjugacy with unequal exposures gives posterior variance and meanTherefore the estimated pure premium per life for year $n+1$ has the normal-normal credibility with unequal exposures formIt is also an exact normal-model Bühlmann–Straub credibility estimate. The individual experience is exposure-weighted: the unweighted mean of the yearly per-life observations would generally discard the different precisions.

For positive $W,v,{\sigma }^2$, differentiation givesThus increasing the prior variance increases credibility, because greater between-risk heterogeneity makes the collective mean less representative of this risk. Increasing the individual process variance decreases credibility, because a fixed amount of observed experience becomes noisier. The future exposure supplies no new observation, so it does not enter $Z_W$. It only scales the total premium:The independent-life variance scaling is part of the usual exposure interpretation. Without it, a known variance $v$ for one life does not determine the variance of a total: a common pairwise conditional covariance $c$ would add $m_j(m_j-1)c$ to $m_{j}v$. Similarly, cross-year conditional dependence would change the joint likelihood. Those assumptions are implicit in applying the standard credibility model to the final paragraph.

For the independent normal distribution observations, the log likelihood isIts derivative is ${\sum }_i(X_i-\theta )$ and its second derivative is $-n<0$, so the unique maximum-likelihood estimate isFor the prior $\theta \sim N(\mu ,{\tau }^{-1})$ with $\tau >0$, multiplying prior density by likelihood and completing the square givesThus normal-normal conjugacy gives the posterior distributionFor quadratic loss, the conditional expected loss of action $a$ is the posterior variance plus $(a-\mathbb{E}[\theta \mid X]{)}^2$. Its unique minimizer is the posterior mean, so the Bayes estimator isThis weights the sample and prior means by their respective precisions.

Multiplying the Poisson distribution likelihood by the Jeffreys prior gives the posterior density kernelThus, in shape-rate convention,It is a proper posterior distribution even when $y_i=0$, provided $E_i>0$. The posterior mean and posterior variance are $(y_i+1/2)/E_i$ and $(y_i+1/2)/E_i^2$. This is Poisson-gamma conjugacy with unequal exposures, with the zero-rate prior interpreted as an improper kernel.

The new prior density contributes the factor ${\lambda }_i^{15}{e}^{-16{\lambda }_i}$. Multiplication by the Poisson distribution likelihood givesIts posterior mean displays the resulting shrinkage:The observed risk ratio $y_i/E_i$ is pulled towards one, most strongly for small exposures. The posterior variance is $(y_i+16)/(E_i+16{)}^2$.

The posterior predictive probability averages the conditional probability over the Bayesian posterior. Using its gamma distribution density,For fixed $a,b$ and $n/T\to \hat{\lambda }<\infty$,Consequently $p_0/e^{-n/T}\to 1$, and both tend to $e^{-\hat{\lambda }}$. This is reasonable because the posterior mean tends to the observed rate and the posterior variance tends to zero. Averaging $e^{-\lambda }$ then approaches evaluating it at the estimated rate: with abundant observations, predictive uncertainty about the rate vanishes.

Let $q_0=n_0$, $q_1=n_0/c^2$ and $y=\overline{y}$. These are the two prior precision parameters. Completing the square in $n(\beta -y{)}^2+q_i{\beta }^2$ gives the normal-normal conjugacy updateThus the posterior mean is a precision-weighted average of the observed mean and the prior center zero. The posterior variance is the reciprocal of the total precision.

For a vector parameter with finite second moments and posterior mean $m_y$, the posterior covariance matrix records its uncertainty and linear dependence after conditioning on the observations. Its diagonal entries are posterior variances. In a regular locally flat-prior normal distribution approximation it is the inverse observed information. A quadratic form has posterior expectation equal to the form at the posterior mean plus the trace of its matrix times this covariance.

A posterior moment is a moment computed under a posterior distribution. Its first raw moment is the posterior mean; its second central moment is the posterior variance.

A posterior variance is the variance of a parameter under its Bayesian posterior, namely $\mathbb{E}[(\theta -\mathbb{E}[\theta \mid y]{)}^2\mid y]$ when finite. It describes remaining uncertainty after observing data and is distinct from the repeated-sampling variance of a fitted estimator.

The variance of a predictive distribution quantifies spread of a future quantity conditional on the observed data and model assumptions. For a noisy future observation $Y_{\ast }=f_{\ast }+{ϵ}_{\ast }$ with fresh independent noise of variance ${\sigma }^2$, it is $\mathrm{Var}(f_{\ast }\mid \mathcal{D})+{\sigma }^2$. It should therefore be distinguished from the latent-function posterior variance.

For the state-space model $S_t=G{S}_{t-1}+w_t$, $X_t=F{S}_t+v_t$, assume independent Gaussian process noise and measurement noise of variances $W,V$, independent of the prior state and previous observations. A normal distribution for the filtered state, with mean $m$ and variance $P$, predicts mean $Gm$ and variance $R=G^{2}P+W$. Here $R$ denotes predicted state variance, rather than the observation-noise covariance notation used in the vector equations above. Gaussian conditional expectation then gives posterior mean $Gm+K(X-FGm)$ and posterior variance $RV/(F^{2}R+V)$, where the Kalman gain is $K=FR/(F^{2}R+V)$.

Mere weak white noise assumptions do not imply normal posterior distributions: independent process noise with a Rademacher distribution makes the next state a mixture of two shifted normal distributions. The Kalman filter can still describe a best affine state estimate under appropriate second-order assumptions, but that is distinct from an exact Gaussian conditional law.