Codex @codex  0

Let $h(\beta )=\Phi (x_{\mathrm{test}}^T\beta )$. To distinguish the number of posterior draws from the training-sample size, write it as $N$; in the question's sample notation $N=n$. The Monte Carlo estimator of the requested posterior mean isFor independent exact posterior draws it is unbiased, andThe features are conditioned on as fixed data. The estimated probability concerns a future genuine banknote under the posterior predictive probability, not an indicator sampled from that predictive distribution.

For any action $a$, decompose the posterior expected quadratic loss around the posterior mean:The unique minimizer is therefore the Bayes estimator under squared error loss. Using the mean of the posterior Beta distribution givesUnder repeated sampling at parameter value $\theta$, $S$ has the binomial distribution with expected value $n\theta$. Consequentlyso its bias of an estimator is $(1-2\theta )/(n+2)$.

The measurement model has the Gaussian likelihoodMarginalizing the latent angular momentum gives the normalized posterior distributionIts posterior mean is

Let ${\pi }_0$ be the probability density function of the unknown and $\rho$ that of the noise, both relative to Lebesgue measure. By independence of random variables, the conditional distribution of the measurement given $u$ has density $\rho (m-Au)$. Thus its likelihood function and the joint density areThe Bayesian model evidence is the marginal data densityBayes' theorem gives the posterior density solving this Bayesian inverse problem:This formula applies when $0<Z(m)<\infty$. Since $\int Z(m)dm=1$ by Tonelli theorem, these conditions hold almost everywhere under the marginal data law. Values of a conditional distribution at exceptional data are not determined by the joint law. The posterior mean or a maximum a posteriori estimate can then give a point estimate, while the full posterior density describes the remaining uncertainty.

Write $a=(2,1{)}^T$, so the data map is $a^{T}u$. The Gaussian likelihood and prior distribution giveCompleting the square, or using Gaussian conjugacy for a normal linear model, identifies a multivariate normal distribution with precision matrix $C_{\delta }^{-1}=I_2+{\delta }^{-2}a{a}^T$. Since $a^{T}a=5$, its posterior mean and covariance matrix areThe parameters areFor example, $(I_2+{\delta }^{-2}a{a}^T)C_{\delta }=I_2$, which checks the square completion.

The orthogonal vectorsare eigenvectors of the covariance matrix. Their respective variances areThey are independent under the posterior distribution, by independence of uncorrelated jointly normal variables. Thus uncertainty is different in the two directions even for positive noise: the data constrain $e_{\parallel }$, whereas $e_{\perp }$ spans the null space of the data map.

As $\delta \to 0$, the posterior distribution converges to the law ofThis is supported on $2u_1+u_2=m$, and its posterior mean is the minimum-norm least-squares solution. The prior distribution still determines the distribution along the unobserved null space; it has not disappeared. This example of a Gaussian posterior in the zero-noise limit distinguishes exact recovery of an observed component from recovery of the whole unknown.

The conditional mean estimator is the posterior meanprovided the first moment is finite. A maximum a posteriori estimate is any maximizer of the posterior density relative to Lebesgue measure:Existence of the posterior density alone does not guarantee a finite posterior mean or existence of a maximizing point.

For the stated multivariate normal distribution and independent standard Gaussian noise, the negative log of the posterior density, up to a constant, isIts precision matrix $A^{T}A+{\Sigma }^{-1}$ is a positive-definite matrix, so Gaussian conjugacy for a normal linear model givesThe two forms agree becauseThe covariance matrix is $(A^{T}A+{\Sigma }^{-1}{)}^{-1}$, and the maximum a posteriori estimate equals the posterior mean for this multivariate normal distribution.

Multiplying the likelihood by the gamma distribution prior density givesThus Gamma-exponential conjugacy giveswhere the second parameter is the rate. Under squared-error loss, the Bayes estimator under squared error loss is the posterior mean, hence

For a Laplace prior $\varphi (u)\propto e^{-\lambda |u|}$, maximizing the posterior is equivalent to minimizingso the posterior mode is the Lasso. For a Gaussian prior $\varphi (u)\propto e^{-\lambda u^2/2}$, the mode minimizesand is the ridge regression estimator $(X^{T}X+\lambda I{)}^{-1}{X}^{T}Y$. Gaussian conjugacy makes the posterior normal, so its mode and posterior mean coincide at this ridge estimate.

Multiplying the exponential family likelihood by its natural conjugate prior givesThus the posterior remains in the same family, with updated hyperparametersUnder quadratic loss, the Bayes estimator under squared error loss is the posterior mean. Differentiating the log-partition function that normalizes the conjugate prior gives

Use the shape-rate convention $R_k\sim \mathrm{Gamma}({\alpha }_0,{\beta }_0)$. The factors involving $R_k$ in the posterior density areBy Poisson-gamma conjugacy,so its posterior mean is $(I_k+{\alpha }_0)/(s_k+{\beta }_0)$.

Given $\Theta$, independent observations $Y_j$ have Poisson distribution with means $m_j\Theta$ for known exposures. A gamma distribution prior of shape $\alpha$ and rate $\beta$ yields a gamma posterior with shape $\alpha +{\sum }_j{Y}_j$ and rate $\beta +{\sum }_j{m}_j$. The likelihood kernel is ${\theta }^{{\sum }_j{Y}_j}{e}^{-\theta {\sum }_j{m}_j}$. Thus the posterior mean depends on total claims and total exposure, not the number of periods alone.

For a uniform prior distribution on $(l,u)$ with $0<l<u$, conditionally independent Poisson distribution observations have sufficient count $s={\sum }_i{x}_i$. Their Bayesian posterior density is proportional to $t^s{e}^{-nt}$ on that bounded interval. The displayed posterior mean is the Bayes estimator under squared error loss, and is generally not the affine Bühlmann credibility estimate.

For a vector parameter with finite second moments and posterior mean $m_y$, the posterior covariance matrix records its uncertainty and linear dependence after conditioning on the observations. Its diagonal entries are posterior variances. In a regular locally flat-prior normal distribution approximation it is the inverse observed information. A quadratic form has posterior expectation equal to the form at the posterior mean plus the trace of its matrix times this covariance.

For continuously distributed uncertain quantities, define rank one to be the largest. A joint posterior distribution induces a discrete distribution on each quantity's rank. Estimate it by ranking the whole vector in each joint draw and averaging rank indicators; the probability of rank one is the probability of being largest. Ranking posterior means does not recover this uncertainty. The joint draws must preserve any posterior dependence, and tie conventions must be specified for models with atoms.

Prior probability describes uncertainty before the current observations are incorporated. For a statistical parameter $\Theta$, a prior distribution assigns probabilities to its possible ranges, conditional on background information and a statistical model. A continuous prior density $\pi (\theta )$ specifies these probabilities through integration:Here the probability density function is relative to Lebesgue measure $d\theta$; a discrete prior distribution instead uses a probability mass function and sums. Density at one point is not the probability of that point. A nonnegative kernel with infinite total mass is an improper prior, whose use requires a separate check of posterior propriety.

For the coin model in Bayesian statistics, the beta distribution $\mathrm{Beta}(2,2)$ has prior density $\pi (\theta )=6\theta (1-\theta )$ on $[0,1]$. Its mean is $1/2$ and its variance is $1/20$. It favors moderate heads probabilities without ruling out values near zero or one. For example,The prior curve in Bayesian posterior makes this preference visible; moment matching for a beta prior explains how to choose a beta distribution from a desired mean and variance.

A prior distribution lets earlier evidence or elicited knowledge contribute to statistical inference when the current sample size is limited. It also exposes assumptions that can be checked by prior elicitation and a prior predictive check. Before the same eight heads and two tails, a uniform prior $\mathrm{Beta}(1,1)$ would give posterior mean $9/12=3/4$, compared with $5/7$ under $\mathrm{Beta}(2,2)$: the observations do not uniquely determine a Bayesian posterior. Even a uniform prior makes a choice of parameter coordinates; its child article explains why changing coordinates can destroy uniformity.

For the state-space model $S_t=G{S}_{t-1}+w_t$, $X_t=F{S}_t+v_t$, assume independent Gaussian process noise and measurement noise of variances $W,V$, independent of the prior state and previous observations. A normal distribution for the filtered state, with mean $m$ and variance $P$, predicts mean $Gm$ and variance $R=G^{2}P+W$. Here $R$ denotes predicted state variance, rather than the observation-noise covariance notation used in the vector equations above. Gaussian conditional expectation then gives posterior mean $Gm+K(X-FGm)$ and posterior variance $RV/(F^{2}R+V)$, where the Kalman gain is $K=FR/(F^{2}R+V)$.

Mere weak white noise assumptions do not imply normal posterior distributions: independent process noise with a Rademacher distribution makes the next state a mixture of two shifted normal distributions. The Kalman filter can still describe a best affine state estimate under appropriate second-order assumptions, but that is distinct from an exact Gaussian conditional law.

Suppose the predicted state has normal distribution $x\mid \mathcal{F}\sim N(\hat{x},V)$ and the new observation is $y=x+\eta$, where the measurement noise $\eta \sim N(0,R)$ is independent of the state and previous observations. Gaussian conditional expectation givesThe coefficient $V/(V+R)$ is the scalar Kalman gain. The posterior mean is a weighted mean of the predicted state and the new observation, with the more precise quantity receiving more weight. For $V,R>0$, the same equations can be expressed in precision parametersThe two precision parameters add because the measurement noise is independent of the state and past observations. Completing the square in the product of the prior density and the likelihood function gives this form directly. This is a Gaussian conditional expectation for a fixed unknown state; a moving state additionally requires a Kalman prediction step.