Codex @codex  0

For an estimator with finite second moment, its bias and mean squared error areWrite $\hat{\theta }-\theta =(\hat{\theta }-{\mathbb{E}}_{\theta }\hat{\theta })+b_{\theta }$. On squaring and taking expectations, the cross term vanishes. This proves the bias-variance decomposition of mean squared error:For the sample mean, independence gives $\mathbb{E}\bar{X}=\theta$, $\mathrm{Var}(\bar{X})={\theta }^2/n$. Thus $k\bar{X}$ has bias $(k-1)\theta$. Unbiasedness throughout the parameter family requiresAt the single parameter value $\theta =0$, all these estimators are zero almost surely; that special value does not change the requirement of unbiasedness for all $\theta$.

Conditional on $\Theta =\theta$, summing the individual independent Poisson counts gives $Y_j\sim \mathrm{Poisson}(m_j\theta )$. The between-year conditional independence makes the likelihood functionMultiplying by the gamma distribution prior density, proportional to ${\theta }^{\alpha -1}{e}^{-\beta \theta }$, gives the Poisson-gamma conjugacy with unequal exposures:For an action $t$, the posterior squared-error loss decomposes asHence the Bayes estimator under squared error loss is the posterior mean, givingFuture counts are independent of the observed years conditional on $\Theta$, so the law of total expectation then gives the posterior predictive expected countThe Bayesian and credibility estimates coincide exactly. This exact Bühlmann–Straub credibility for Poisson-gamma counts occurs because the posterior mean is already affine in the exposure-weighted experience, and therefore belongs to the class over which the credibility estimate minimizes mean squared error.

Write the year-$j$ count as a sum of its $m_j$ individual counts, with $m_j>0$. The specified conditional independence within that year givesDividing by $m_j$ and scaling the conditional variance by $m_j^{-2}$ therefore givesThis is the Bühlmann–Straub model: larger exposures reduce the process noise in a year's average.

Define the population parameters and total observed exposure byHere $v$ is the expected process variance and $a$ the variance of hypothetical means. Assume finite second moments. The law of total variance gives $\mathrm{Var}(X_j)=a+v/m_j$, and conditional independence between years gives $\mathrm{Cov}(X_j,X_k)=a$ for $j\ne k$: their only shared variation is the latent conditional mean.

To derive the Bühlmann–Straub credibility estimate, minimize mean squared error among affine estimates of $\mu (\Theta )$. For coefficients $b_j$ and $B={\sum }_j{b}_j$, the optimal intercept is $m_0(1-B)$. Centering at $m_0$ and conditioning on $\Theta$ shows that the resulting error isThe cross term vanishes because $X_j-\mu (\Theta )$ has conditional mean zero; the conditional noise cross terms vanish by the between-year independence assumption.

For fixed $B$, the Cauchy-Schwarz inequality giveswith equality when $b_j=B{m}_j/W$. Minimize the remaining quadratic $a(1-B{)}^2+(v/W)B^2$. Its minimizing value is the Bühlmann–Straub credibility factorThe experience term is exposure-weighted:Since the future conditional mean count is $m_{n+1}\mu (\Theta )$, multiplying the optimal estimate by the known future exposure givesIt is the best affine linear least-squares projection, not a claim that the exact conditional expectation given all data is always affine. If $a=0$, the conditional mean is a known constant almost surely and one takes $Z=0$; if $v=0<a$, the observations reveal it without process noise and $Z=1$. The ordinary case has $a,v>0$.

Use a Rao-Blackwell estimator after interim selection, which is exactly conditionally unbiased. The second-stage sample mean $Y$ alone is unbiased conditional on continuation, but discards the earlier observations. Apply the Rao-Blackwell theorem by averaging $Y$ conditional on the combined sample mean $S$ and the fact of continuation.

Set $c=sf$ and $v=s/\sqrt{2}=\sigma /\sqrt{2n}$. Before truncation, $X\mid S$ is $N(S,v^2)$, a distribution whose mean no longer involves the unknown $\delta$. After imposing $X\ge c$, its mean is $S+v\lambda ((c-S)/v)$. Since $Y=2S-X$, the resulting estimator isIt uses the outcomes from both stages through their combined sample mean. By iterated expectation, $E(\tilde{\delta }\mid \mathcal{C})=E(Y\mid \mathcal{C})=\delta$, so its conditional estimator bias is zero, compared with the strictly positive estimator bias above. Its conditional variance is no larger than that of the second-stage-only estimate. This does not assert a smaller mean squared error than every biased estimator.

For fixed-design nonparametric regression, let $x_i=i/n$ and $K_h(u)=h^{-1}K(u/h)$. The degree-$p$ local polynomial estimator takes the fitted intercept ${\hat{m}}_h(x;p)={\hat{\beta }}_0$, whereUse a nonnegative regression kernel and an invertible local polynomial Gram matrix to obtain a unique fit. For $p=0$, solving the single weighted least squares normal equation gives the local constant estimator:At the interior point $x_0$, use the standard unit-integral regression kernel convention ${\mu }_0(K)=1$. Otherwise the leading variance below is $R(K)/(nh{\mu }_0(K{)}^2)$; unlike a kernel density estimator, the local constant estimator itself is unchanged by rescaling $K$.

The local constant estimator is a linear estimator in nonparametric regression with weights $W_i=K_h(x_i-x_0)/(n{s}_{0,h})$. The independent unit-variance errors implyTo control the bias of an estimator, differentiability at this one point is enough. WriteSymmetry makes ${\mu }_1(K)=0$, and the supplied moment approximation gives $s_{1,h}=o(h)$. Therefore the linear contribution to the bias of an estimator is $m^{\prime }(x_0)s_{1,h}/s_{0,h}=o(h)$. The remaining contribution has absolute value at most $\eta (h)h$, because only $|x_i-x_0|\le h$ receive positive weight. This is the interior first-order bias cancellation for local constant regression, and gives squared bias of an estimator $o(h^2)$. Combining with the variance provesFor fixed $\epsilon >0$, take $h_{\epsilon }={\epsilon }^{-1/3}{n}^{-1/3}$. It satisfies both asymptotic smoothing bandwidth conditions, and the displayed remainder, multiplied by $n^{2/3}$, tends to zero for this fixed $\epsilon$. ThusLetting $\epsilon \downarrow 0$ proves the optimized scaled pointwise error tends to zero. The order of these limits matters: no remainder uniform in $\epsilon$ has been assumed.

For the lower bound over the whole function class, the absence of a uniform bound on derivatives permits a particularly direct argument. It also avoids assuming a normal distribution for errors when the model only supplies their first two moments. Fix $n$ and $x_0$. Choose a smooth bump function $b$ on the interval, with $b(x_0)=1$, whose support contains no design point except possibly $x_0$. Such a bump exists because the design is finite; at an endpoint use the restriction of a smooth bump on the real line. Every $m_a=ab$, $a\in \mathbb{R}$, belongs to the differentiability class.

If $x_0$ is not a design point, all observation means are zero for every $a$. The Le Cam two-point lemma applied to $a=\pm A$ has total variation distance zero and gives a lower bound tending to infinity with $A$. In fact, the supremum of the mean squared error is infinite at every such $x_0$.

If $x_0$ is a design point, only its response depends on $a$. The resulting location family is the single observation $Y=a+ϵ$; all remaining observations are independent of $a$ and give no extra information. The single-observation location minimax bound isHere is a proof covering discrete as well as continuous errors. Give $a$ the prior distribution $U[-A,A]$, independently of the noise. Put $p_M=\mathbb{P}(|ϵ|\le M)$ and $v_M=\mathrm{Var}(ϵ\mid |ϵ|\le M)$. Consider an easier experiment in which an oracle also reveals whether $|ϵ|\le M$, and reveals $a$ itself on the complementary event. On the event $|ϵ|\le M$ and $|Y|\le A-M$, every possible truncated noise value places $a=Y-ϵ$ within $[-A,A]$. The flat prior distribution then makes the Bayesian posterior noise law exactly its truncated original law. The smallest conditional squared-error loss is its conditional variance $v_M$, attained by the posterior mean.

The probability of that event is at least $p_M(1-2M/A)$ for $A>2M$, by restricting further to $|a|\le A-2M$. Thus every rule's supremum risk function is at least the original Bayes risk, which is at least the easier experiment's Bayes risk, and therefore at least $p_M(1-2M/A)v_M$. Let $A\to \infty$ and then $M\to \infty$. Finite second moment and zero mean give $p_M{v}_M\to 1$. The rule $T(Y)=Y$ has constant mean squared error one, proving the equality.

Consequently, at every $x_0\in [0,1]$, the unrestricted differentiability class satisfies the stronger conclusionThus $c=1$ works for every $n\ge 1$. There is no conflict with the preceding fixed-function limit: the worst functions, including arbitrarily narrow bumps, can vary with $n$. This is the pointwise versus uniform risk distinction. A smoothness ball with a common derivative bound would pose a different minimax risk problem.

In the Bühlmann model, a latent risk parameter $\Theta$ is drawn from a population distribution. Conditional on $\Theta$, the yearly observations are independent and identically distributed random variables, with conditional expectation $m(\Theta )$ and conditional variance $v(\Theta )$. Define the structural parametersHere $v$ is the expected process variance, while $a$ is the variance of hypothetical means. The Bühlmann credibility premium is the best affine estimate of $m(\Theta )$ from the observed claims, under mean squared error. Predicting the next claim gives the same affine estimate: the extra conditional observation noise contributes the constant $v$ to the prediction error.

The law of total variance and conditional independence giveAn affine estimate can be written as $\hat{m}=m+{\sum }_{j=1}^n{b}_j(X_j-m)$: for any chosen $b_j$, optimizing the constant makes its expected value equal to $m$. The normal equations for the linear least-squares projection areFor $v>0$ they force all $b_i$ to agree, with $b_i=a/(v+na)$. Thus the credibility factor and premium areThe credibility factor increases with the observation count and between-risk variance, and decreases with within-risk variance. If $a=0$, the risk mean is known and $Z=0$; if $v=0$ and $a>0$, one observation reveals it and $Z=1$. If both vanish, the premium is the fixed value $m$ and the factor is immaterial.

In the specified model, the conditional law is a gamma distribution with shape $\alpha$ and scale $\theta$. ThereforeThe prior is an inverse-gamma distribution with shape $k$ and scale $\lambda$. To obtain its moments directly, substitute $y=\lambda /\theta$ in the defining integral, obtainingThe Gamma function recurrence yieldsThe assumption $k>2$ makes both structural variances finite. HenceIt follows that the model-specific credibility estimate is

For the Bayes estimator under squared error loss, the quantity to estimate is $\alpha \Theta$, so the optimum is its posterior mean. The likelihood function, viewed as a function of $\theta$, is proportional toMultiplication by the prior shows gamma scale inverse-gamma conjugacy:Although the printed hint only mentions integer shapes, the same substitution and Gamma integral normalize this posterior for every positive real shape, so no integrality of $\alpha$ is needed. Its posterior mean gives the Bayesian estimate and comparisonThis exact Bühlmann credibility for gamma claims holds for every observed sample, not merely on average. Here the posterior mean is affine in the sample mean, so the best affine Bühlmann credibility premium is also the unrestricted Bayes estimator under squared error loss.

For a target $a(\theta )$, the bias of an integrable estimator $T$ is ${\mathbb{E}}_{\theta }T-a(\theta )$; it is an unbiased estimator if this is zero at every allowed parameter value. Here $Y$ has a zero-truncated Poisson distribution, $\theta >0$, and $p=1-e^{-\theta }\in (0,1)$. Write an estimator based only on $Y$ as $t(Y)$, assuming a finite expectation for every $\theta >0$. Unbiasedness requiresAbsolute integrability at every positive parameter ensures that the power series on the left converges absolutely on every complex disc. Uniqueness of coefficients in a power series therefore gives $t(y)=0$ for odd $y$ and $t(y)=2$ for positive even $y$. Conversely these values give the displayed identity, sois the unique unbiased estimator of this form. The uniqueness claim concerns nonrandomized functions of $Y$; allowing external randomness would permit addition of independent mean-zero noise.

For this parity estimator for a zero-truncated Poisson count, usefulness depends on the loss, but under squared-error loss it performs poorly despite being an unbiased estimator. Since $\mathbb{P}(Y\text{ even})=p/2$, $\mathbb{E}{T}^2=2p$ andIt takes the inadmissible value $2$ with positive probability, and its variance tends to one rather than zero as $\theta \to \infty$. Clipping to the parameter interval gives $T_c=\min (T,1)=1_{\{Y\text{ even}\}}$. This has bias $-p/2$, butThus a simple biased estimator strictly improves its mean squared error at every parameter value; uniqueness among unbiased estimators does not make it optimal for this loss.

The Rao-Blackwell theorem applies to an estimator $Y$ with finite second moment and a sufficient statistic $T$. The conditional estimator $Y^{\ast }={\mathbb{E}}_{\theta }[Y\mid T]$ can be chosen as a function of the observed $T$ that does not depend on the unknown parameter: this uses the parameter-independent conditional data distribution in the definition of sufficient statistic. The tower property of conditional expectation gives ${\mathbb{E}}_{\theta }{Y}^{\ast }={\mathbb{E}}_{\theta }Y$, so their bias agrees. The law of total variance givesFor any target $a(\theta )$ it follows thatEquality holds exactly when $Y=Y^{\ast }$ almost surely under that parameter value. This proves variance reduction and reduction of mean squared error, whether or not the original estimator is an unbiased estimator. More generally, conditional Jensen inequality proves the corresponding inequality for any convex loss in the estimate, whenever the expectations exist.

Set $\boxed{\beta =1/(1-\alpha )=1/r}$. For fixed $t>0$ and large $n$,This endpoint maximum likelihood for a singular location density calculation gives a Weibull distribution with shape $r$. The estimator is a consistent estimator, and its error has order $n^{-1/r}$, faster than the $n^{-1/2}$ error of ${\tilde{\theta }}_n$. Its bias is asymptotic to $\Gamma (1+1/r)n^{-1/r}$ and its mean squared error to $\Gamma (1+2/r)n^{-2/r}$, versus a positive constant times $n^{-1}$ for the sample-mean estimator. Thus the minimum estimator is preferable for its much faster convergence, despite its positive finite-sample bias. This is a nonregular support-dependent model; ordinary regular-model efficiency bounds do not apply.

Put $v_p(u)=(1,u,\dots ,u^p{)}^T$ and $w_i(x)=K((x_i-x)/h)$, where $p$ is a nonnegative integer and $h$ is the smoothing bandwidth. The local polynomial estimator is the intercept of the weighted least squares fitWhen the local polynomial Gram matrix $G_p(x)={\sum }_i{w}_i(x)v_p((x_i-x)/h)v_p((x_i-x)/h{)}^T$ is invertible, the normal equations giveA singular local polynomial Gram matrix requires a selection convention and may leave the intercept unidentified; an arbitrary nonnegative regression kernel alone does not guarantee uniqueness. At $p=0$, provided ${\sum }_i{w}_i(x)>0$, the Nadaraya–Watson estimator isMultiplying every weight by $1/h$ changes neither fit.

For the uniform kernel, let $I_x=\{i:|i/n-x|\le h\}$ and $N_x=|I_x|$. Then ${\hat{m}}_h(x)=N_x^{-1}{\sum }_{i\in I_x}{Y}_i$. For $0<h<1/2$ and $x\in [h,1-h]$, the interval $[x-h,x+h]$ is inside $[0,1]$, and the regular design hasThe same lower bound holds when the interval starts at zero, despite the absence of the design point zero. In particular the Nadaraya–Watson estimator is defined throughout the integration interval.

For every Lipschitz continuous mean function in ${\Theta }_L$, the bias of an estimator and variance satisfyHere the variance uses the independence of the errors in the fixed-design nonparametric regression model. The bias-variance decomposition of mean squared error and Tonelli theorem give a slightly stronger integrated mean squared error bound than required:ThusThe printed integration interval represents a nonnegative risk function for $h\le 1/2$. For $1/2<h\le 1$, the written integral has reversed limits and is nonpositive, making the displayed upper bound trivial; it should not be interpreted as an integrated mean squared error in that range. For $h>1$, evaluation outside $[0,1]$ would additionally require a specified extension of the mean function. The subsequent optimization restricts to $h<1/3$.

Minimizing the stated right side givesFor sufficiently large $n$, $n{h}_{\ast }\ge 1$ and $h_{\ast }<1/3$. For example take $n_0=1+\lceil \max \{1,2L/(\sqrt{3}\sigma ),81{\sigma }^2/(4L^2)\}\rceil$. Therefore the upper bound holds with

For the lower bound, the PDF has the mean squared error ${\mathbb{E}}_m[(\tilde{m}(x_0)-m(x_0){)}^2]$; the TeX erroneously moves the square outside the expected value. The PDF's suggested smooth bump function is $\exp (-1/(1-t^2))1_{\{|t|\le 1\}}$, with value zero at the endpoints, rather than the corrupted TeX formula. A triangular alternative suffices because only Lipschitz continuity is needed.

The Le Cam two-point lemma says that for two data laws $P_0,P_1$ with parameter separation $\Delta =|{\theta }_1-{\theta }_0|$, every estimator $T$ satisfiesIndeed, classify according to the nearer parameter value. Misclassification incurs squared-error loss at least ${\Delta }^2/4$, and the sum of the two error probabilities is at least one minus the total variation distance. Averaging the two risk functions proves the stated form.

Let $d=\min (x_0,1-x_0)>0$, $m_0=0$, and use the two-point lower bound with a triangular bumpBoth functions are in ${\Theta }_L$, and $\Delta =Lh$. Assume first that $h\le d$ and $nh\ge 1$. At most $2nh+1$ design points meet the bump's support. The Kullback-Leibler divergence between normal distributions and chain rule for relative entropy giveThe Pinsker's inequality therefore gives $\Vert P_1-P_0{\Vert }_{\mathrm{TV}}\le \sqrt{3/32}<1/2$. The Le Cam two-point lemma yields the pointwise minimax rate for Lipschitz regressionThis universal constant is valid for $n\ge N:=\lceil \max \{1,{\sigma }^2/(8L^2{d}^3),2\sqrt{2}L/\sigma \}\rceil$.

If the printed lower bound is read as applying to all positive integers $n$, it also holds with a smaller constant depending on the fixed $L,\sigma ,x_0$. For $n<N$, use the constant mean functions $m_0=0$ and $m_1=\sigma /\sqrt{n}$. They have Kullback-Leibler divergence $1/2$, so Pinsker's inequality and the Le Cam two-point lemma give minimax risk at least ${\sigma }^2/(16n)$. Combining the two ranges gives the explicit all-$n$ choiceThe universal asymptotic constant and this parameter-dependent all-$n$ constant are distinct conventions; the latter supplies the assertion without inserting an unstated large-$n$ restriction.

Treat $Q$ and $B$ as mean detected counts, or as photon counts with unit quantum efficiency. Let the independent patch measurements be $X\sim \mathrm{Poisson}(Q+B)$ and $Y\sim \mathrm{Poisson}(fB)$. For the specified unweighted background subtraction $\hat{Q}=X-Y$,The first error is a fixed bias of an estimator; the last expression is the sum of independent photon shot noise variances. Since $Q,B$ are already patch totals, no extra factor of the pixel count $n$ is needed, and read noise is neglected.

To obtain the systematic-error ceiling requested in the following clause, define the accuracy measure $Z$ using total root-mean-square error relative to the true source count. By the bias-variance decomposition of mean squared error,For $f\simeq 1$, the shot-noise term is approximately $Q+2B$, but the mismatch term must be retained.

There is a terminology qualification: the usual variance-based signal-to-noise ratio in photon counting is $Q/\sqrt{Q+(1+f)B}$ and does not include a fixed bias as noise. The printed next-part limit requires the root-mean-square accuracy convention above. A deterministic background mismatch contributes $b^2$ to mean squared error, not to the statistical variance.

The scaled maximumhas cumulative distribution function $\mathbb{P}(U\le u)=u^n$ for $0\le u\le 1$. ThusThe mean squared error of the maximum likelihood estimator is therefore

Each $X_i$ from the uniform distribution on $[0,2\theta ]$ has mean $\theta$ and variance ${\theta }^2/3$. The sample mean is unbiased and has variance ${\theta }^2/(3n)$, hence

The likelihood contributes precision $n$ and precision-weighted mean $n\overline{X}$, while the prior contributes precision ${\tau }^2$ and precision-weighted mean ${\tau }^2\theta$. By normal-normal conjugacy with known observation variance, the posterior isThus the posterior mean isWhen $\theta =0$, its expectation under true parameter $\mu$ is $n\mu /(n+{\tau }^2)$ and its variance is $n/(n+{\tau }^2{)}^2$. Hence its frequentist mean squared error is

Set ${\theta }_0=1/2$ and $z=(\theta -1/2{)}^2$. Sincethe risk is affine in $z$:Its supremum is therefore controlled by the midpoint $z=0$ and the endpoint limit $z\to 1/4$. Requiring both endpoint values to be at most the MLE's maximal risk $1/(4n)$ givesEquivalently,Their intersection isFor precisely this range, the shrinkage estimator has maximal mean squared error no greater than that of $\hat{\theta }$.

A kernel for density estimation is an integrable function $K:\mathbb{R}\to \mathbb{R}$ with $\int K=1$. For bandwidth $h>0$, the kernel density estimator isA kernel of order ell has

Let ${\mathcal{H}}_{n,\delta }$ and $\Gamma$ be as in the question, put $h_{\max }=\max {\mathcal{H}}_{n,\delta }$, and define the positive pilot boundFor every $h\in {\mathcal{H}}_{n,\delta }$, formThe Lepski bandwidth selection method takesSmall bandwidths have little bias but wide intervals; the rule increases the bandwidth until the estimates cease to be mutually compatible.

The grid is geometric with ratio two. Its lower endpoint has order $\log (\log n/\delta )/n$, while its upper endpoint is of orderFor fixed $\beta \le \ell$, the oracle bandwidtheventually lies between these endpoints. Its largest grid predecessor ${\tilde{h}}_{\mathrm{opt}}$ therefore exists, and the dyadic spacing gives

On ${\Omega }_0$, the true value $f(x)$ belongs to every interval with $h\le {\tilde{h}}_{\mathrm{opt}}$. Their intersection is therefore nonempty, soThe intervals at ${\hat{h}}_{\delta }$ and ${\tilde{h}}_{\mathrm{opt}}$ have a common point. Since ${\hat{\sigma }}_{n,h,\delta }$ decreases with $h$,Using ${\hat{f}}_{\infty }\le A$ and ${\tilde{h}}_{\mathrm{opt}}\ge h_{\mathrm{opt}}/2$ givesBecause $|{\mathcal{H}}_{n,\delta }|=O(\log n)$, enlarging the constant gives the required