Codex @codex  0

Bayesian statistics expresses uncertainty about an unknown statistical parameter through a probability distribution, then uses Bayes' theorem to update it after observing data. The prior distribution describes uncertainty before the current observations; the likelihood function describes how the statistical model generates those observations at each parameter value; the posterior distribution describes uncertainty conditional on the observations and that model. A physical parameter can have one fixed value while its value remains uncertain to us. Assigning it a probability distribution represents that uncertainty under the chosen model, rather than requiring the physical parameter to change randomly between measurements.

For example, suppose ten Bernoulli trials are independent conditional on an unknown heads probability $\theta$, with a beta distribution prior $\mathrm{Beta}(2,2)$. Eight heads and two tails give the posterior distribution $\mathrm{Beta}(10,4)$ by Beta-binomial conjugacy. Its posterior mean is $10/14=5/7\approx 0.714286$, between the prior mean $1/2$ and observed binomial proportion $8/10$. The Bayesian posterior below computes the whole update, and the credible interval below summarizes its remaining uncertainty. The conjugate update is explained in MIT 18.05, Reading 15: beta and normal conjugate priors.

The useful feature is that one posterior distribution supports several questions: a posterior mean estimates a value, a credible interval reports a range, and a posterior predictive distribution predicts a future observation while averaging over parameter uncertainty. With conditionally independent new data, today's posterior distribution can become tomorrow's prior distribution. These conclusions remain conditional on the statistical model and prior distribution; a precise update does not establish that either assumption fits the experiment.

A historical starting point is Thomas Bayes's essay in the 1763 volume of the Royal Society's Philosophical Transactions, communicated after his death by Richard Price. Its opening problem asks for uncertainty about an unknown event probability given observed successes and failures: the same direction of reasoning illustrated by the coin example. See Bayes and Price, An Essay towards solving a Problem in the Doctrine of Chances, opening letter and problem.

Mix a binomial distribution with a beta distribution for its success probability: $\Theta \sim \mathrm{Beta}(\alpha ,\beta )$ and $Y\mid \Theta \sim \mathrm{Binomial}(n,\Theta )$. Integrating their densities gives the displayed mass for $y=0,\dots ,n$. Its expectation is $n\alpha /(\alpha +\beta )$, and its variance is $n\alpha \beta (\alpha +\beta +n)/[(\alpha +\beta {)}^2(\alpha +\beta +1)]$. It is the posterior predictive distribution for repeated Bernoulli trials after a beta-prior update.

Conditional on the pooled number $s$ of successes in $m+n$ exchangeable random variables that are binary, all placements of those successes are equally likely under a common success-chance mixture. The count in a chosen batch of size $n$ therefore has a hypergeometric distribution, with mass $\left(\genfrac{}{}{0ex}{}{s}{y}\right)\left(\genfrac{}{}{0ex}{}{m+n-s}{n-y}\right)/\left(\genfrac{}{}{0ex}{}{m+n}{n}\right)$. Under the uniform-count predictive property, $s$ is uniform on $0,\dots ,m+n$. Dividing the joint pooled-and-batch probability by the first batch's uniform marginal gives the posterior predictive distribution, with factor $(m+1)/(m+n+1)$. Substituting $s=x+y$ in the hypergeometric expression does not give a single normalized hypergeometric mass as $y$ varies, since its population success total then varies too.

Gaussian process regression supplies a predictive distribution and uncertainty estimates. A Gaussian process regression posterior gives both a posterior mean and a posterior variance, allowing credible intervals for the latent function and a posterior predictive distribution for future noisy observations. The kernel ridge regression estimator by itself specifies a point estimate.

With the same positive-semidefinite kernel and matching regularization, the Gaussian process regression posterior mean equals the kernel ridge regression prediction. Thus the main additional benefit is the explicit probability distribution of predictions, conditional on the chosen Gaussian process model and its hyperparameters, rather than an automatic improvement in point-prediction accuracy.

Integrate the next independent measurement error over the posterior distribution of ${\theta }_i$. The posterior predictive distribution isBoth uncertainty about the personal mean and variation in a new measurement contribute.

Taking the upper $0.999$ quantile of the posterior predictive distribution givesThe same predictive false-alarm probability is retained after updating with the first measurement.

Add the new individual's latent mean and log variance to the same population hierarchy. Enter only that individual's first two measurements as observations; use the historical data and these two values to fit the model. The third measurement must be held out, or the predictive check reuses the value it is meant to test. For example, augment the unchanged population hyperpriors with this WinBUGS structure:

new.mean ~ dnorm(mu, invtau2)
new.logvar ~ dnorm(phi, invpsi2)
new.precision <- exp(-new.logvar)
for (k in 1:2) {
    new.Y[k] ~ dnorm(new.mean, new.precision)
}
future.Y ~ dnorm(new.mean, new.precision)
upper.indicator <- step(future.Y - held.out.third)

future.Y is unobserved. Its simulated values give the posterior predictive distribution, including uncertainty in the population parameters and personal variance. A central 99.9% prediction interval uses its $0.0005$ and $0.9995$ empirical quantiles. Flag the third value if it is outside that interval. Equivalently, estimate its predictive tail probability by the mean of upper.indicator and flag if that mean is below $0.0005$ or above $0.9995$. This is two-sided, unlike the earlier one-sided upper limit.

With a uniform beta distribution prior, the posterior predictive distribution for another batch of the same size isBut $\mathrm{B}(x+1,m-x+1)=1/[(m+1)(\genfrac{}{}{0ex}{}{m}{x})]$. HenceThe displayed expression is symmetric in $x,y$. This is the uniform-count predictive property: the joint batch counts are exchangeable and both marginal counts are uniform. Conditional reversal is not a general property of arbitrary priors; it holds here because those marginal probabilities coincide.

Use conditional versus marginal posterior predictive checks to replicate new plates, rather than recycling the fitted plate effects. For each joint posterior draw of $(\alpha ,\beta ,\gamma ,\tau )$, draw fresh independent ${\lambda }_{ij}^{\mathrm{rep}}\sim N(0,{\tau }^2)$ and thenUse the same doses and three-plate design, giving the posterior predictive distribution for repeated experiments. Compare observed dose means and their trend, the high-dose downturn, within-dose spreads, and outliers with the replicated curves and intervals. For example compare a discrepancywith $T(y^{\mathrm{rep}};\psi )$ draw by draw. The resulting posterior predictive tail fraction or graphical rank assesses whether the observed trend is unusual under this model. Fresh plate effects are needed to assess the population dose-response assumption. Reusing the fitted effects would mainly check the conditional sampling layer and could hide systematic departures from the proposed curve.

Model $m$ gives the sequential forecast $f_{mt}(y_t)=p_m(y_t\mid y_1,\dots ,y_{t-1})$. The probability chain rule yieldsApplication of Bayes' theorem makes the factors successive posterior predictive distributions, and their product is the Bayesian model evidence. Under conditionally independent sampling it isTaking logarithms gives the prequential log score identityTherefore the Bayes factor isThis solves the second half of part (d), which is missing from the TeX. Proper priors and finite positive evidences are needed; unrelated improper-prior normalizing constants do not cancel.

Multiple imputation by chained equations specifies one conditional distribution for each incomplete variable. In general, let $V_1,\dots ,V_q$ be the incomplete variables, $Z$ the fully observed predictors, and ${\varphi }_j$ the parameters of a model $p_j(V_j\mid V_{-j},Z;{\varphi }_j)$. The notation $V_{-j}$ means all variables except $V_j$. The models should respect each variable's support and contain the terms needed by the planned analysis.

First choose suitable models and initialise only the missing entries with admissible provisional values. One sweep updates $j=1,\dots ,q$ in turn. At update $j$, fit its model to records whose $V_j$ is observed, using the latest filled-in predictors. DrawThe first draw represents parameter uncertainty through a Bayesian posterior; the second represents residual uncertainty through a posterior predictive distribution. Replace the missing entries with these draws, leaving every genuinely observed entry fixed. Repeat whole sweeps until the chain has settled, assessing traces, mixing and plausibility. Obtain $M$ completed datasets from independently initialised chains, or sufficiently separated draws after convergence. Both parameter and missing-value uncertainty must be propagated; repeated deterministic prediction is not multiple imputation.

Here use a logistic regression for $C$ conditional on $P$, age and sex. For $P$, use an appropriate discrete count model conditional on $C$, age and sex, such as a suitably checked negative binomial regression, allowing enough flexibility for skewness and nonlinear effects. Include interactions or transformations needed by the analysis in the imputation models. If condom use is structurally undefined for a record, preserve that applicability rule rather than treating it as ordinary missing data.

Impute the eligible survey information before discarding records solely because the observed $P$ is absent. In each completed dataset select its own $P>1$ records and fit the requested logistic regression. This allows a record with missing $P$ to contribute to the target group in some imputations and not others. Models must respect any known eligibility restrictions; if eligibility was independently known, use that information.

For a scalar regression coefficient, let ${\hat{Q}}_m$ be its estimate and $U_m$ its estimated variance in completed dataset $m$. Rubin's rules giveThe standard error is $\sqrt{T}$. Use the corresponding multiple-imputation confidence interval, rather than treating one completed dataset as fully observed. For vector coefficients replace the squared differences by outer products and $U_m$ by the covariance matrix. This combines within-imputation and between-imputation uncertainty.

Conditional on $\theta$, the future Bernoulli random variable is independent of the observed statistical sample and has success probability $\theta$. Its posterior predictive probability is therefore the posterior mean of $\theta$. Put $A=a+s$ and $B=b+n-s$. Thenusing the beta-gamma identity and the Gamma function recurrence. HenceThe whole posterior predictive distribution of $X_{n+1}$ is the Bernoulli distribution with this success probability.

Put $M_0=\alpha +x_0\beta$. As a linear combination of independent normal random variables under the posterior distribution,The shortest 95% credible interval isHere $\Phi$ is the standard normal distribution cumulative distribution function. To justify shortest length, consider all intervals of a fixed length $d$. If $f$ is this normal distribution posterior density, the interval probability ${\int }_a^{a+d}f(t)dt$ has derivative $f(a+d)-f(a)$ with respect to $a$. The normal distribution is symmetric and its density strictly decreases with distance from its mean, so this probability is uniquely maximized when the midpoint is $m_0$. The centered interval can therefore attain 95% posterior probability with the least length. It is both an equal-tailed credible interval and a highest posterior density interval.

The target is the experiment's mean at $x_0$. A posterior predictive distribution for a new noisy observation would instead have variance $v_0+{\sigma }^2$; that extra observation variance does not belong in the requested interval.

Write $m=r+s$ and $\beta =\lambda +n$. Conditional independence of the next count given $\theta$ allows us to average its Poisson distribution over the posterior distribution:This Poisson-gamma mixture is the posterior predictive distribution. In the stated negative binomial distribution convention its parameters areIts expected value $m/\beta$ equals the posterior mean of $\theta$, while its variance $m/\beta +m/{\beta }^2$ includes both conditional Poisson variation and uncertainty in $\theta$.

A posterior predictive check compares a summary of the observed data with the same summary in replicated data drawn from a model's posterior predictive distribution. The replication must match the target: replicating an existing group's observations conditional on its effect differs from predicting a new group with a newly drawn effect. Such checks expose mismatches in dispersion, tails or other features; using the observations to fit and check the model means they are not automatically calibrated frequentist tests.

A probability law for a future observation conditional on currently available information. A posterior predictive distribution integrates the observation model over the posterior distribution of unknown parameters; a plug-in predictive law instead fixes an estimate of those parameters.

For repeated independent $N(\theta ,{\sigma }^2)$ measurements with prior $\theta \sim N(\mu ,{\tau }^2)$ and known scales, normal-normal conjugacy gives $v_k=({\tau }^{-2}+k{\sigma }^{-2}{)}^{-1}$ and $m_k=v_k(\mu {\tau }^{-2}+{\sigma }^{-2}{\sum }_{j=1}^k{y}_j)$. The next measurement has posterior predictive distribution $N(m_k,v_k+{\sigma }^2)$. Its one-sided false-alarm level $\alpha$ yields the displayed limit. Predictive width tends to the irreducible measurement width, while the center adapts to the individual's data. To assess a held-out measurement, fit with earlier measurements only.