Codex @codex  0

The loss $|a-\theta |$ for estimating a real parameter $\theta$ by $a$. Its risk of a decision rule is the corresponding expected value. Unlike squared-error loss, its pointwise size is a metric, so the triangle inequality gives a direct overlap version of the Le Cam two-point lemma.

Bayesian credibility estimates a risk's conditional mean by its Bayesian posterior expected value under squared-error loss. When that posterior mean is an affine combination of exposure-weighted individual experience and a prior mean, the coefficient of the experience is its credibility factor. Exact Bayesian credibility is distinct from merely selecting the best affine estimator when the true posterior mean is nonlinear.

The binary Brier score applies squared-error loss to a reported success probability $p$ and the observed indicator $y\in \{0,1\}$. It is a loss, so smaller is better. Its values lie in $[0,1]$: predicting the outcome with certainty gives zero, while assigning it zero probability gives one. Unlike the logarithmic scoring rule loss, the binary Brier score remains bounded even for a confidently wrong forecast.

For a Bernoulli distribution with true success probability $q$, averaging the two squared losses givesThe report controls only $(p-q{)}^2$; the remaining term is the variance of the Bernoulli random variable. Consequently the Brier score is a strictly proper scoring rule. At $q=0.7$, the expected loss is $(p-0.7{)}^2+0.21$, with unique minimum $0.21$ at $p=0.7$. Reporting $p=0.2$ instead gives expected loss $0.46$, despite its better loss on the particular outcome $Y=0$ above.

For several possible categories, the Brier score sums the squared differences between reported probabilities and observed category indicators. With two categories that sum is $2(p-y{)}^2$, so its numerical values are twice the binary convention used here, with the same optimizing report. Brier's original ^[ref] averages these category-sum losses over forecasts. Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others (Section 3.1, Example 1, and Table 1) gives the category-sum and binary forms as negative rewards; negating them gives the losses described here.

Cross-validation estimates predictive performance by repeatedly fitting on one part of the data and evaluating on held-out observations. In K-fold cross-validation, divide training data into disjoint folds $I_1,\dots ,I_K$; $k(i)$ identifies observation $i$'s fold, and ${\hat{f}}^{(-k)}$ is fitted without fold $I_k$. The displayed average scores each observation using a rule that did not train on it. Weight by observation count when fold sizes differ. Leave-one-out cross-validation uses one observation per fold. ^[ref]

For the training error example, three folds contain indices $i$ with equal remainder modulo 3. Each fit uses eight observations. The squared-error loss averages for degrees 1, 2, and 7 are $0.724132$, $0.027192$, and $1.532790$ respectively. Select degree 2 and refit on all twelve training data observations. The folds reuse scarce data while challenging each candidate to predict unseen responses; they do not touch the final test set.

Any learned preprocessing, including variable selection, must also be fitted within each training fold to avoid data leakage. ^[ref] Repeated tuning can itself overfit the cross-validation score; nested cross-validation uses inner folds for model selection and outer folds for assessment of that entire procedure. ^[ref]

Folds use fewer training observations than the final refit, and their scores are dependent because training observations overlap. ^[ref] Splitting must match deployment: for time series forecasting, rolling-origin cross-validation trains on the past and evaluates later observations, preventing future information from entering a forecast. Ordinary random folds need not answer that forecasting question. ^[ref]

A gamma prior with shape $\alpha$ and rate $\beta$ combined with $n$ independent observations from an exponential distribution of rate $\theta$ gives the posteriorUnder squared-error loss, the Bayes estimator is its posterior mean $(\alpha +n)/(\beta +{\sum }_i{X}_i)$.

Suppose $X_1,\dots ,X_n$, $n\ge 1$, are independent and identically distributed with Laplace distribution $\mathrm{Laplace}(\mu ,b)$, where $b>0$. For realized values $x_1,\dots ,x_n$, the likelihood function and log-likelihood function areFor fixed $b$, maximum likelihood estimation of the location therefore minimizes the sum of absolute losses. This connects the Laplace distribution to robust statistics: each observation contributes a cost growing linearly with its distance, whereas squared-error loss grows quadratically.

Why does a sample median minimize $S$? Between successive order statistics, moving $a$ a little to the right increases each distance to a value on the left and decreases each distance to a value on the right. Hence the slope is the number on the left minus the number on the right. It changes from negative to positive at the middle. More precisely, if $x_{(1)}\le \cdots \le x_{(n)}$, all minimizing locations areFor an even sample size, the objective is flat throughout the interval between the two central order statistics. The midpoint convention for the sample median selects one solution; it does not prove uniqueness. ^[ref] This is the finite-sample counterpart of median minimizes expected absolute loss.

If $S(\hat{\mu })>0$, maximizing over $b>0$ givesThe derivative is positive below this value and negative above it. Moreover the maximized likelihood function is proportional to $S(\mu {)}^{-n}$ wherever $S(\mu )>0$, so the same minimizing locations give the joint maximum-likelihood estimates. ^[ref] The scale estimate divides by $n$; this formula is not an assertion that it is an unbiased estimator.

The boundary case matters: if all observations equal $c$, choose $\mu =c$. Then $S(c)=0$ and $L(c,b)=(2b{)}^{-n}\to \infty$ as $b\downarrow 0$. There is no finite joint maximum-likelihood estimate in the parameter space $b>0$. In particular, this always occurs for $n=1$. Writing $\hat{b}=0$ does not produce an admissible Laplace distribution.

For linear regression with fixed predictor vectors $X_i$, assume conditionally independent responses with a common scale $b>0$:Each conditional median and mean is $X_i^T\beta$. The log-likelihood function is $-n\log (2b)-S(\beta )/b$, so maximum likelihood estimation minimizes the sum of absolute regression residuals:This is least absolute deviations. With only an intercept, it reduces to the Laplace location and scale maximum likelihood problem, so a sample median is a fitted intercept. Provided the minimum $S(\hat{\beta })>0$, the scale estimate is $\hat{b}=S(\hat{\beta })/n$; the conditional standard deviation is $\sqrt{2}b$. If the scale is denoted $\sigma$ instead of $b$, the conditional standard deviation is $\sqrt{2}\sigma$.

Minimizing a sum of absolute losses does not automatically give a unique coefficient vector. Even an intercept-only fit to an even sample size can have an interval of solutions, and redundant predictor columns can introduce further nonuniqueness. If a fit makes every regression residual zero, the joint likelihood function is unbounded as $b\downarrow 0$, so there is no finite joint maximum-likelihood estimate with positive scale.

The linear growth of absolute loss limits sensitivity to an extreme response at fixed predictors compared with squared-error loss. It does not protect against every outlier: an extreme predictor can have large regression leverage. The numerical example above changes a response in an intercept-only problem, and makes no general claim of immunity to such predictor changes.

Conditionally independent normal distribution observations of mean $\theta$ and known variance ${\sigma }_1^2$, with normal prior mean $\mu$ and variance ${\sigma }_2^2$, give normal-normal conjugacy with known observation variance. Their posterior mean has the displayed exact credibility estimate form. It minimizes posterior squared-error loss and predicts the conditional mean of the next observation.

For an observed $x\ge 0$, the continuous uniform distribution likelihood is $p(x\mid \theta )={\theta }^{-1}{1}_{\{\theta \ge x\}}$, with $\theta >0$. Multiplying by the stated prior cancels the factor $\theta$, giving $e^{-\theta }{1}_{\{\theta \ge x\}}$. Its integral over the permitted parameter values is $e^{-x}$. Hence the posterior distribution isThis is the exponential posterior for a uniform endpoint. Its posterior mean is $x+1$ and its variance is one. The posterior expected squared-error loss decomposes asSince $c>0$, its unique minimum occurs at the Bayes estimate $\boxed{{\hat{\theta }}_B=x+1}$. The positive loss multiplier changes the risk, not the optimizing action.

For any prior density $\pi (p)$, the Bayes estimate under squared-error loss is the posterior mean:For the matched beta distribution, the integrals are beta integrals and their ratio isThe posterior mean minimizes posterior expected squared error because $\mathbb{E}[(p-a{)}^2\mid D]=\mathrm{Var}(p\mid D)+(a-\mathbb{E}[p\mid D]{)}^2$. The phrase Bayes point estimate depends on the loss: under absolute-error loss it would be a posterior median, while the posterior mode is a different summary and here equals $(14.8875-1)/(207.75-2)\approx 0.06750$.

A beta distribution is a convenient prior for a probability $p\in (0,1)$ and is conjugate to the binomial distribution of the death count. Match its mean $m=0.05$ and variance $v=0.02^2=0.0004$. For $p\sim \mathrm{Beta}(\alpha ,\beta )$,The moment matching for a beta prior formula givesConditional on $p$, use $D\sim \mathrm{Binomial}(90,p)$ and the observed count $D=9$. Multiplying its likelihood $p^9(1-p{)}^{81}$ by the prior density gives the Bayesian posterior through Beta-binomial conjugacy:Under squared-error loss, take its posterior mean as the point estimate. A 95% equal-tail credible interval is given by its 0.025 and 0.975 quantiles. Numerically,Matching two moments does not uniquely determine a prior distribution; the beta family is a justified convenient choice, not a conclusion forced by those moments. Its transfer to this hospital presumes that the historical between-hospital distribution is relevant to the present risk and case mix.

Use a hierarchical Bayesian model with binomial observation errors and a common distribution for the underlying risks:Give $(\alpha ,\beta )$ a shared hyperprior, or parameterize them by mean $m=\alpha /(\alpha +\beta )$ and concentration $\kappa =\alpha +\beta$ and put priors on $m\in (0,1)$ and $\kappa >0$. Conditional on these hyperparameters, the risks are independent and identically distributed; after integrating them out they are exchangeable random variables with shared uncertainty. The data estimate the overall risk level and between-hospital variation. Conditional Beta-binomial conjugacy givesand full Bayes point estimates under squared-error loss areAn Empirical Bayes method instead estimates the hyperparameters from the marginal beta-binomial distribution likelihoodand inserts their fitted values in the conditional posterior means. Both approaches provide partial pooling, stronger for smaller hospitals. Full Bayes also propagates hyperparameter uncertainty into the credible intervals. If systematic covariates are needed, one can place exchangeable residual effects in a logistic regression model rather than assuming a common unconditional risk distribution.

A credibility premium estimates a risk's expected future claims by combining its own observed experience with a collective or prior mean. In its basic form $Z\overline{X}+(1-Z)\mu$, the credibility factor $Z\in [0,1]$ is the weight assigned to individual experience. Larger credibility places less weight on pooling with the population. The amounts below are pure expected-claim premiums; no additional expense or profit loading is specified.

Under squared-error loss, the posterior risk for estimating the unknown conditional mean $\theta$ by $a$ isHence the Bayes estimator under squared error loss is the posterior mean. In the equal-year model, conditional independence makes the likelihood proportional to $\exp [-{\sum }_{j=1}^n(X_j-\theta {)}^2/(2{\sigma }_1^2)]$. Combining it with the normal prior and completing the square givesBecause $\mathbb{E}[X_{n+1}\mid \theta ]=\theta$, the required estimator is $m_n$; it also equals the predictive mean $\mathbb{E}[X_{n+1}\mid \mathbf{X}]$. The normal-normal credibility expression isThis is exact Bayesian credibility, not merely the best linear approximation to a possibly nonlinear posterior mean.

For unequal life counts use the usual exposure model: individual-life losses are conditionally independent with variance $v$, and the annual totals are conditionally independent given the same $\theta$. ThenThe dot denotes the stipulated normal approximation. The positive exposures $m_j$ are known. Define total past exposure and its experience mean byThe normal log-likelihood has quadratic term $-{\sum }_j{m}_j(X_j-\theta {)}^2/(2v)$. With the prior $N(\mu ,{\sigma }^2)$, normal-normal conjugacy with unequal exposures gives posterior variance and meanTherefore the estimated pure premium per life for year $n+1$ has the normal-normal credibility with unequal exposures formIt is also an exact normal-model Bühlmann–Straub credibility estimate. The individual experience is exposure-weighted: the unweighted mean of the yearly per-life observations would generally discard the different precisions.

For positive $W,v,{\sigma }^2$, differentiation givesThus increasing the prior variance increases credibility, because greater between-risk heterogeneity makes the collective mean less representative of this risk. Increasing the individual process variance decreases credibility, because a fixed amount of observed experience becomes noisier. The future exposure supplies no new observation, so it does not enter $Z_W$. It only scales the total premium:The independent-life variance scaling is part of the usual exposure interpretation. Without it, a known variance $v$ for one life does not determine the variance of a total: a common pairwise conditional covariance $c$ would add $m_j(m_j-1)c$ to $m_{j}v$. Similarly, cross-year conditional dependence would change the joint likelihood. Those assumptions are implicit in applying the standard credibility model to the final paragraph.

Under squared-error loss, the posterior expected loss decomposes asIt is uniquely minimized by the posterior mean. Integrating the Gamma posterior gives $\mathbb{E}[\theta \mid x]=b^2{\int }_0^{\infty }{\theta }^2{e}^{-b\theta }d\theta =2/b$. Hence the Bayes estimator is

A credibility estimate combines a risk's observed experience with collective or prior information, typically as $Z\overline{X}+(1-Z)m_0$. The credibility factor $Z\in [0,1]$ is the weight assigned to the individual experience: more reliable or more extensive data generally increases it. Bayesian credibility treats the risk parameter as random with a prior distribution and predicts future claims by averaging the conditional claim mean under its posterior distribution. Under squared-error loss this posterior predictive mean is optimal. It need not be an affine combination in every Bayesian model; the Poisson-gamma conjugacy calculation below yields that credibility form exactly, rather than as a best-linear approximation.

The uniform prior and likelihood give the posterior densitya Beta distribution with parameters $K+1,n-K+1$. Under squared-error loss for $\varphi$, the posterior expected loss is $\mathbb{E}[{\varphi }^2\mid K]-2a\mathbb{E}[\varphi \mid K]+a^2$, minimized uniquely at the posterior mean of $\varphi$, not at the product of the two posterior mean factors.

Using the supplied beta integral,Thus the Bayes estimate isThis remains positive even on the endpoint samples, unlike the extended MLE; it integrates over the posterior uncertainty in $\theta$.

A credibility estimate combines a risk's own observed experience with a collective or prior mean, in the form $Z{\hat{\mu }}_{\mathrm{experience}}+(1-Z){\mu }_{\mathrm{collective}}$. The credibility factor $Z\in [0,1]$ is the weight given to that risk's experience. Bayesian credibility starts with a prior distribution for a latent risk parameter, updates it using observations, and estimates the conditional risk mean by its Bayesian posterior expected value under squared-error loss. In conjugate models this posterior mean can have exactly the displayed credibility form.

For the unequal policy exposures, defineConditional independence of the binomial distribution observations gives the likelihood, up to a factor not depending on $\theta$,Multiplication by the beta distribution prior therefore gives Beta-binomial conjugacy:Both posterior parameters are positive, since $0\le s\le M$ and the prior parameters are positive.

The conditional target is $\mu (\theta )=m_{k+1}\theta$. To verify the quadratic-loss rule directly, for any proposed estimate $d$ and observed data $\mathcal{D}$,The variance term does not depend on $d$, so the unique Bayes estimator under squared error loss is the posterior mean. Applying the beta mean formula givesIt is also the posterior expected future claim count, by the tower property of conditional expectation.

Let $p_0=\alpha /(\alpha +\beta )$ be the collective prior claim probability and $\hat{p}=s/M$ the observed claims per policy. ThenTherefore the exact beta-binomial credibility with unequal exposures isThe individual experience must be weighted by exposure: $s/M$ is the policy-weighted frequency, not the unweighted average of $x_i/m_i$ when the $m_i$ differ. The factor multiplies next-year expected counts after scaling both component probabilities by $m_{k+1}$.

Conditional on $\Theta =\theta$, summing the individual independent Poisson counts gives $Y_j\sim \mathrm{Poisson}(m_j\theta )$. The between-year conditional independence makes the likelihood functionMultiplying by the gamma distribution prior density, proportional to ${\theta }^{\alpha -1}{e}^{-\beta \theta }$, gives the Poisson-gamma conjugacy with unequal exposures:For an action $t$, the posterior squared-error loss decomposes asHence the Bayes estimator under squared error loss is the posterior mean, givingFuture counts are independent of the observed years conditional on $\Theta$, so the law of total expectation then gives the posterior predictive expected countThe Bayesian and credibility estimates coincide exactly. This exact Bühlmann–Straub credibility for Poisson-gamma counts occurs because the posterior mean is already affine in the exposure-weighted experience, and therefore belongs to the class over which the credibility estimate minimizes mean squared error.

For fixed-design nonparametric regression, let $x_i=i/n$ and $K_h(u)=h^{-1}K(u/h)$. The degree-$p$ local polynomial estimator takes the fitted intercept ${\hat{m}}_h(x;p)={\hat{\beta }}_0$, whereUse a nonnegative regression kernel and an invertible local polynomial Gram matrix to obtain a unique fit. For $p=0$, solving the single weighted least squares normal equation gives the local constant estimator:At the interior point $x_0$, use the standard unit-integral regression kernel convention ${\mu }_0(K)=1$. Otherwise the leading variance below is $R(K)/(nh{\mu }_0(K{)}^2)$; unlike a kernel density estimator, the local constant estimator itself is unchanged by rescaling $K$.

The local constant estimator is a linear estimator in nonparametric regression with weights $W_i=K_h(x_i-x_0)/(n{s}_{0,h})$. The independent unit-variance errors implyTo control the bias of an estimator, differentiability at this one point is enough. WriteSymmetry makes ${\mu }_1(K)=0$, and the supplied moment approximation gives $s_{1,h}=o(h)$. Therefore the linear contribution to the bias of an estimator is $m^{\prime }(x_0)s_{1,h}/s_{0,h}=o(h)$. The remaining contribution has absolute value at most $\eta (h)h$, because only $|x_i-x_0|\le h$ receive positive weight. This is the interior first-order bias cancellation for local constant regression, and gives squared bias of an estimator $o(h^2)$. Combining with the variance provesFor fixed $\epsilon >0$, take $h_{\epsilon }={\epsilon }^{-1/3}{n}^{-1/3}$. It satisfies both asymptotic smoothing bandwidth conditions, and the displayed remainder, multiplied by $n^{2/3}$, tends to zero for this fixed $\epsilon$. ThusLetting $\epsilon \downarrow 0$ proves the optimized scaled pointwise error tends to zero. The order of these limits matters: no remainder uniform in $\epsilon$ has been assumed.

For the lower bound over the whole function class, the absence of a uniform bound on derivatives permits a particularly direct argument. It also avoids assuming a normal distribution for errors when the model only supplies their first two moments. Fix $n$ and $x_0$. Choose a smooth bump function $b$ on the interval, with $b(x_0)=1$, whose support contains no design point except possibly $x_0$. Such a bump exists because the design is finite; at an endpoint use the restriction of a smooth bump on the real line. Every $m_a=ab$, $a\in \mathbb{R}$, belongs to the differentiability class.

If $x_0$ is not a design point, all observation means are zero for every $a$. The Le Cam two-point lemma applied to $a=\pm A$ has total variation distance zero and gives a lower bound tending to infinity with $A$. In fact, the supremum of the mean squared error is infinite at every such $x_0$.

If $x_0$ is a design point, only its response depends on $a$. The resulting location family is the single observation $Y=a+ϵ$; all remaining observations are independent of $a$ and give no extra information. The single-observation location minimax bound isHere is a proof covering discrete as well as continuous errors. Give $a$ the prior distribution $U[-A,A]$, independently of the noise. Put $p_M=\mathbb{P}(|ϵ|\le M)$ and $v_M=\mathrm{Var}(ϵ\mid |ϵ|\le M)$. Consider an easier experiment in which an oracle also reveals whether $|ϵ|\le M$, and reveals $a$ itself on the complementary event. On the event $|ϵ|\le M$ and $|Y|\le A-M$, every possible truncated noise value places $a=Y-ϵ$ within $[-A,A]$. The flat prior distribution then makes the Bayesian posterior noise law exactly its truncated original law. The smallest conditional squared-error loss is its conditional variance $v_M$, attained by the posterior mean.

The probability of that event is at least $p_M(1-2M/A)$ for $A>2M$, by restricting further to $|a|\le A-2M$. Thus every rule's supremum risk function is at least the original Bayes risk, which is at least the easier experiment's Bayes risk, and therefore at least $p_M(1-2M/A)v_M$. Let $A\to \infty$ and then $M\to \infty$. Finite second moment and zero mean give $p_M{v}_M\to 1$. The rule $T(Y)=Y$ has constant mean squared error one, proving the equality.

Consequently, at every $x_0\in [0,1]$, the unrestricted differentiability class satisfies the stronger conclusionThus $c=1$ works for every $n\ge 1$. There is no conflict with the preceding fixed-function limit: the worst functions, including arbitrarily narrow bumps, can vary with $n$. This is the pointwise versus uniform risk distinction. A smoothness ball with a common derivative bound would pose a different minimax risk problem.

The specified gamma distribution uses shape $k$ and rate $\lambda$. Multiplying its density by the Poisson likelihood gives a posterior density proportional to ${\theta }^{k+S-1}{e}^{-(\lambda +n)\theta }$. Thus the conjugate prior calculation givesFor squared-error loss, the posterior expected loss isIt is minimized at the posterior mean, so the Bayes estimator isFor a fresh conditionally independent Poisson observation, the posterior predictive probability of zero is the posterior expectation of $e^{-\theta }$. Evaluating the gamma integral givesThe samples are independent conditional on $\theta$; integrating out the uncertain common parameter produces the predictive probability above.