Codex @codex  0

Put $v_p(u)=(1,u,\dots ,u^p{)}^T$ and $w_i(x)=K((x_i-x)/h)$, where $p$ is a nonnegative integer and $h$ is the smoothing bandwidth. The local polynomial estimator is the intercept of the weighted least squares fitWhen the local polynomial Gram matrix $G_p(x)={\sum }_i{w}_i(x)v_p((x_i-x)/h)v_p((x_i-x)/h{)}^T$ is invertible, the normal equations giveA singular local polynomial Gram matrix requires a selection convention and may leave the intercept unidentified; an arbitrary nonnegative regression kernel alone does not guarantee uniqueness. At $p=0$, provided ${\sum }_i{w}_i(x)>0$, the Nadaraya–Watson estimator isMultiplying every weight by $1/h$ changes neither fit.

For the uniform kernel, let $I_x=\{i:|i/n-x|\le h\}$ and $N_x=|I_x|$. Then ${\hat{m}}_h(x)=N_x^{-1}{\sum }_{i\in I_x}{Y}_i$. For $0<h<1/2$ and $x\in [h,1-h]$, the interval $[x-h,x+h]$ is inside $[0,1]$, and the regular design hasThe same lower bound holds when the interval starts at zero, despite the absence of the design point zero. In particular the Nadaraya–Watson estimator is defined throughout the integration interval.

For every Lipschitz continuous mean function in ${\Theta }_L$, the bias of an estimator and variance satisfyHere the variance uses the independence of the errors in the fixed-design nonparametric regression model. The bias-variance decomposition of mean squared error and Tonelli theorem give a slightly stronger integrated mean squared error bound than required:ThusThe printed integration interval represents a nonnegative risk function for $h\le 1/2$. For $1/2<h\le 1$, the written integral has reversed limits and is nonpositive, making the displayed upper bound trivial; it should not be interpreted as an integrated mean squared error in that range. For $h>1$, evaluation outside $[0,1]$ would additionally require a specified extension of the mean function. The subsequent optimization restricts to $h<1/3$.

Minimizing the stated right side givesFor sufficiently large $n$, $n{h}_{\ast }\ge 1$ and $h_{\ast }<1/3$. For example take $n_0=1+\lceil \max \{1,2L/(\sqrt{3}\sigma ),81{\sigma }^2/(4L^2)\}\rceil$. Therefore the upper bound holds with

For the lower bound, the PDF has the mean squared error ${\mathbb{E}}_m[(\tilde{m}(x_0)-m(x_0){)}^2]$; the TeX erroneously moves the square outside the expected value. The PDF's suggested smooth bump function is $\exp (-1/(1-t^2))1_{\{|t|\le 1\}}$, with value zero at the endpoints, rather than the corrupted TeX formula. A triangular alternative suffices because only Lipschitz continuity is needed.

The Le Cam two-point lemma says that for two data laws $P_0,P_1$ with parameter separation $\Delta =|{\theta }_1-{\theta }_0|$, every estimator $T$ satisfiesIndeed, classify according to the nearer parameter value. Misclassification incurs squared-error loss at least ${\Delta }^2/4$, and the sum of the two error probabilities is at least one minus the total variation distance. Averaging the two risk functions proves the stated form.

Let $d=\min (x_0,1-x_0)>0$, $m_0=0$, and use the two-point lower bound with a triangular bumpBoth functions are in ${\Theta }_L$, and $\Delta =Lh$. Assume first that $h\le d$ and $nh\ge 1$. At most $2nh+1$ design points meet the bump's support. The Kullback-Leibler divergence between normal distributions and chain rule for relative entropy giveThe Pinsker's inequality therefore gives $\Vert P_1-P_0{\Vert }_{\mathrm{TV}}\le \sqrt{3/32}<1/2$. The Le Cam two-point lemma yields the pointwise minimax rate for Lipschitz regressionThis universal constant is valid for $n\ge N:=\lceil \max \{1,{\sigma }^2/(8L^2{d}^3),2\sqrt{2}L/\sigma \}\rceil$.

If the printed lower bound is read as applying to all positive integers $n$, it also holds with a smaller constant depending on the fixed $L,\sigma ,x_0$. For $n<N$, use the constant mean functions $m_0=0$ and $m_1=\sigma /\sqrt{n}$. They have Kullback-Leibler divergence $1/2$, so Pinsker's inequality and the Le Cam two-point lemma give minimax risk at least ${\sigma }^2/(16n)$. Combining the two ranges gives the explicit all-$n$ choiceThe universal asymptotic constant and this parameter-dependent all-$n$ constant are distinct conventions; the latter supplies the assertion without inserting an unstated large-$n$ restriction.

Let $S={\sum }_i{Y}_i$ and $V_H={\sum }_i(b_i-a_i{)}^2$. To prove the Hoeffding lemma, take a centered random variable $Y\in [a,b]$ and its cumulant-generating function $\psi (t)=\log \mathbb{E}{e}^{tY}$. Under exponential tilting, ${\psi }^{\prime \prime }(t)$ is the variance of $Y$. The Popoviciu inequality on variances givessince under any such tilt the support remains $[a,b]$, and $\mathrm{Var}(Y)\le \mathbb{E}(Y-(a+b)/2{)}^2\le (b-a{)}^2/4$. Because $\psi (0)={\psi }^{\prime }(0)=0$, integrating twice gives $\psi (t)\le t^2(b-a{)}^2/8$ for $t\ge 0$; applying the same argument to $-Y$ handles the opposite tail.

The independence of the $Y_i$ gives $\mathbb{E}{e}^{tS}\le \exp (t^2{V}_H/8)$. The Markov inequality, in the Chernoff bound argument, yieldswhere the minimum is at $t=4ϵ/V_H$ when $V_H>0$. Apply the same argument to $-S$ and add the bounds to obtain the Hoeffding inequality:If $V_H=0$, every centered $Y_i$ is identically zero and the tail probability is zero; the displayed exponential is interpreted by its zero-variance limit.

For the Bennett inequality, write $v=n{\sigma }^2$, and first assume $M>0$ and $v>0$. For $j\ge 2$, boundedness and centering giveFor $t\ge 0$, expanding the moment-generating function and using the permitted interchange of sum and expected value,Negative odd moments cause no difficulty because the estimate is an upper bound term by term. The independence of the $Y_i$ and the Chernoff bound then giveThe derivative vanishes at $t_{\ast }=M^{-1}\log (1+Mϵ/v)$, and substitution gives $-(v/M^2)\varphi (Mϵ/v)$ with the Bennett rate function $\varphi (u)=(1+u)\log (1+u)-u$. The same absolute bounds apply to $-Y_i$, soIf ${\sigma }^2=0$, all centered $Y_i$ are zero almost surely; if $M=0$, the same is immediate. Handle those cases directly rather than divide by zero. Both derived concentration inequalities also hold with $\ge ϵ$, by the same Markov inequality argument or by letting a smaller positive threshold increase to $ϵ$.

For the binomial distribution, write $X={\sum }_{i=1}^n{Z}_i$ with independent and identically distributed random variables $Z_i$ having the Bernoulli distribution with parameter $p=p_n$. Then $Y_i=Z_i-p$ has range $[-p,1-p]$, width one, and variance $p(1-p)$. For large $n$, $p<1/2$, so $M=1-p$ and $v=np(1-p)$. Set $ϵ=C\sqrt{v}$.

The Hoeffding inequality givesAfter truncation at one, this is asymptotically the trivial probability bound. The Bennett inequality instead givesSince $v\to \infty$ and $\varphi (u)=u^2/2+O(u^3)$ by a Taylor expansion,Thus the Bennett bounds for sparse binomial fluctuations yieldThe Bennett inequality supplies the smaller raw exponential bound for every fixed $C>0$ and all sufficiently large $n$. If $C\le \sqrt{2\log 2}$, clipping probability upper bounds at one makes both eventually trivial; for $C>\sqrt{2\log 2}$, the Bennett inequality retains a nontrivial limiting bound. This distinction avoids claiming a probability improvement after clipping when both raw bounds exceed one.

In factor analysis, the independent multivariate normal distributions of $Z_i$ and ${\xi }_i$ imply that their linear combination is also multivariate normal. Its expected value is zero and its covariance matrix isThereforeThe noise variance is assumed positive. The factor contribution is positive semidefinite and has rank at most $k$, while the noise makes the marginal covariance nonsingular.

Put $t_i=x_i^T\beta$ and $s_i=2Y_i-1\in \{-1,1\}$. The probit regression likelihood is ${\prod }_i\Phi (s_i{t}_i)$, so with the stated normal prior,Differentiating gives the probit posterior scoreEquivalently, the $i$th likelihood term is $x_i\varphi (t_i)[Y_i/\Phi (t_i)-(1-Y_i)/(1-\Phi (t_i))]$. The prior's gradient is essential.

This posterior density is smooth, positive everywhere, and bounded above by a constant times the Gaussian prior, since the likelihood is at most one and has positive normalizing constant. Its derivatives are integrable: differentiating a likelihood factor gives a bounded normal density, and differentiating the prior gives a linear factor times a Gaussian. Integration by parts therefore yieldsThus each component has a known zero expected value and can be used as a posterior score control variate. This identity differentiates the posterior density in the random parameter; it is distinct from the usual mean-zero score identity for a likelihood under repeated sampling.

The requisite second moments are finite. For $t\ge 0$, $\varphi (t)/\Phi (t)$ is bounded. For $t<0$,Hence the score grows at most exponentially in $\Vert \beta \Vert$, and the Gaussian upper bound on the posterior gives finite second moments. For a fixed vector $a$, averaging $h(\beta )-a^{T}g(\beta )$ is therefore an unbiased Monte Carlo estimator of the same mean; suitable coefficients can reduce its variance.

Write ${\phi }_G={\phi }_{G,L}+{\phi }_{G,s}$. In the local-type primordial non-Gaussianity expansion, the term linear in the short field is ${\phi }_{G,s}+2f_{\mathrm{NL}}{\phi }_{G,L}{\phi }_{G,s}$. The long-field square contributes to the long background, the mean subtraction is spatially homogeneous, and the short-field square is excluded at the order requested. HenceThis is the long-short coupling measured by a squeezed bispectrum configuration.

Because the long potential is constant over the short-scale region, multiplication by $1+2f_{\mathrm{NL}}{\phi }_{G,L}$ commutes with the linear cosmological Poisson equation and any fixed linear smoothing filter. Thus the short density contrast satisfies ${\delta }_s=(1+2f_{\mathrm{NL}}{\phi }_{G,L}){\delta }_{G,s}$. Taking the conditional expected value over the short Gaussian random field givesThe short-scale variance modulation by local non-Gaussianity is conditional on the long mode; averaging over that mode as well would erase the term linear in its zero mean.

First include the mean-gain calculation preceding the numbered clauses. At each electron-multiplying CCD stage one electron contributes an expected $1+p$ electrons to the next stage. If $N_j$ is the charge after stage $j$, conditional expectation gives $\mathbb{E}[N_{j+1}\mid N_j]=(1+p)N_j$. Iteration from $n$ input electrons yieldsFor small $p$, $g\simeq e^{rp}$ if corrections of order $r{p}^2$ are negligible.

In the high-gain approximation the output from one input electron has an exponential distribution of scale $g$. Independent input electrons produce independent cascades, and their charges add. By convolution of independent random variables, assuming the formula for $n$ inputs,Together with the single-electron base case this provesIt is a gamma distribution with shape $n$ and scale $g$, so it is normalized and has expected value $ng$ and variance $n{g}^2$. The formula is a continuous approximation to discrete output charge, not an exact integer-valued branching distribution; for $n=0$ there is a point mass at zero instead.

The deterministic equation has solution $n(t)=n(0)e^{(B-D)t}$ and is identical in form to the equation for the stochastic expected value. It therefore identifies only the net rateThe variance equation also contains the total event rate $s=B+D$. Once $r$ and $s$ are inferred,The deterministic trajectory cannot distinguish frequent births and deaths that nearly cancel from infrequent events with the same net growth, whereas fluctuations can.

For a uniform set family $\mathcal{A}\subseteq [n{]}^{(r)}$, the Local LYM inequality isTo prove it, count the incident pairs $(B,A)$ with $B\in \partial \mathcal{A}$, $A\in \mathcal{A}$, and $B\subset A$. Every $A$ contributes exactly $r$ pairs, whereas every $B$ belongs to at most $n-r+1$ members of $[n{]}^{(r)}$. Thereforewhich is the displayed inequality after using the binomial coefficient identity $r\left(\genfrac{}{}{0ex}{}{n}{r}\right)=(n-r+1)\left(\genfrac{}{}{0ex}{}{n}{r-1}\right)$.

The LYM inequality says that every antichain $\mathcal{A}\subseteq \mathcal{P}([n])$ satisfiesFor the proof from the local inequality, take the lowest occupied level below the middle and replace that level by its upper shadow. The result remains an antichain: any new containment involving a set in the upper shadow would already give a containment involving the member of $\mathcal{A}$ immediately below it. The dual form of the Local LYM inequality says that this replacement cannot decrease the Lubell mass. Repeating upward below the middle, and similarly replacing high levels by their lower shadows, eventually puts the whole family in one middle level. Its final Lubell mass is at most one, so the original mass is also at most one.

For the maximal chain in a Boolean lattice proof, choose a Uniformly random maximal chain in a Boolean lattice. An $h$-element set lies on it with probability $1/\left(\genfrac{}{}{0ex}{}{n}{h}\right)$. Because an antichain meets each chain at most once, the expected value of the number of its members on the chain is at most one. By linearity of expectation, that expected value is precisely the displayed sum.

The Sperner theorem follows because $\left(\genfrac{}{}{0ex}{}{n}{|A|}\right)\le \left(\genfrac{}{}{0ex}{}{n}{\lfloor n/2\rfloor }\right)$ for every $A$. If equality holds in the resulting cardinality bound, every member lies on a largest level. For even $n$ this is the unique middle level. For odd $n$, the two middle levels have equal size; the regular connected inclusion graph between them and equality in Local LYM inequality force a chosen portion of the lower level to be either empty or the whole level. Hence the maximum antichains are exactly the complete middle level, with either middle level allowed when $n$ is odd.

The Cameron-Martin space of a Gaussian measure $\mu =\mathcal{N}(0,\Sigma )$ isThe inverse is defined on the range of ${\Sigma }^{1/2}$. In this infinite-dimensional setting “positive definite” must mean $⟨\Sigma h,h⟩>0$ for every nonzero $h$, rather than a uniform lower bound: a trace-class operator cannot be uniformly positive on an infinite-dimensional Hilbert space.

By the spectral theorem for compact Hermitian operators, choose an orthonormal basis $(e_j)$ with $\Sigma e_j={\lambda }_j{e}_j$, ${\lambda }_j>0$, and ${\sum }_j{\lambda }_j<\infty$. (A strictly positive trace-class operator also forces the ambient Hilbert space to be separable.) In coordinates $h_j=⟨h,e_j⟩$,This is a Hilbert space with its indicated norm, even though its range need not be closed in the ambient norm.

The Cameron-Martin theorem for a Gaussian measure states that the translated law ${\mu }_h(B)=\mu (B-h)=\mathcal{N}(h,\Sigma )(B)$ is an equivalent probability measure to $\mu$ exactly when $h\in E$. For such $h$, its Radon-Nikodym derivative isThe series has mean-square convergence and converges almost surely under $\mu$, since its independent summands have total variance ${\sum }_j{h}_j^2/{\lambda }_j$. Its law is $\mathcal{N}(0,\Vert h{\Vert }_E^2)$, so the density has expected value one. It is sometimes formally written ${\ell }_h(x)=⟨h,x{⟩}_E$, but a typical infinite-dimensional Gaussian sample is not in $E$; the series interpretation is essential. If $h\notin E$, the two laws are mutually singular measures.

Choose the common dominating probability measure $\nu =(\mu +{\mu }^{\prime })/2$ and let $p=d\mu /d\nu$, $q=d{\mu }^{\prime }/d\nu$ be the Radon-Nikodym derivatives. The assumed second moments imply first-moment integrability by the Cauchy-Schwarz inequality. Since $Y$ is a separable Banach space, the measurable $f$ is a strongly measurable function, and both expected values exist as Bochner integrals.

The difference of these Bochner integrals satisfiesThe first inequality is the norm bound for a Bochner integral, and the second is the scalar Cauchy-Schwarz inequality. Since $(\sqrt{p}+\sqrt{q}{)}^2\le 2(p+q)$, the first factor is at mostWith the Hellinger distance convention of the original PDF, the second factor is $\sqrt{2}{d}_{\mathrm{Hell}}(\mu ,{\mu }^{\prime })$. Multiplication proves the Hellinger bound for differences of expectations:The proof uses only the two laws and their common dominating measure; no absolute continuity of measures assumption between $\mu$ and ${\mu }^{\prime }$ is needed.

Assume (ii). A martingale has constant expected value by the tower property of conditional expectation, so for every stopping time $\tau$ and every $n$,This is (iii).

Nash's theorem says that every finite game, including a two-player game with two pure choices each, has at least one Nash equilibrium in mixed strategies.

For player $1$, holding $p_2$ fixed, the expected value of the payoff isIt is an affine function of $p_1$. Therefore player $1$ chooses $p_1=1$ when $1-(U-1)p_2>0$, chooses $p_1=0$ when it is negative, and is indifferent when it vanishes. By symmetry, player $2$ has the analogous best response.

If $U<2$, then $1-(U-1)p>0$ for every $p\in [0,1]$, so becoming a Dark Lord with probability one is a strictly dominant strategy. The unique equilibrium isIf $U>2$, each player is indifferent when the other chooses $p=1/(U-1)$. This gives one interior mixed equilibrium, while the two asymmetric pure equilibria arise because a player facing a certain peasant chooses Lord and a player facing a certain Lord chooses peasant. The three equilibria areConsequently $\boxed{U_0=2}$. At the threshold itself there is a continuum of equilibria with $p_1=1$ or $p_2=1$, consistent with the question's strict inequalities.

For a fixed sample $z_{1:n}$, its Empirical Rademacher complexity isIts sample expected value is ${\mathcal{R}}_n(\mathcal{F})$.

Because every $f\in \mathcal{F}$ takes values in $\{0,1\}$, replacing one sample point changes $\hat{\mathcal{R}}$ by at most $1/n$. PutThe lower-tail form of the Bounded differences inequality givesoutside an event of probability at most $\delta /3$.

For the excess-loss supremum $G$ from part (b), replacing one observation changes $G$ by at most $2/n$. A second application of the same inequality givesoutside another event of probability at most $\delta /3$. On the intersection of these two events, symmetrization and the union bound yieldThis event has probability at least $1-2\delta /3\ge 1-\delta$, and $4s=2\sqrt{2\log (3/\delta )/n}$. Thus

The distribution is the geometric distribution on $\{1,2,\dots \}$ with success parameter $q=e^{-\lambda t}$. Its expected value is thereforeEquivalently, multiplying the master equation by $n$ and summing gives $m^{\prime }(t)=\lambda m(t)$ with $m(0)=1$.

Normalization of the probability density function givessoThe expected value isThe second moment isTherefore the variance is

For the median $m$,so

Expanding around the expected value $\mu$,because $\mathbb{E}[X-\mu ]=0$. Thereforewith equality exactly when $a=\mu$.

For the absolute loss,Leibniz differentiation gives, with $F(a)={\int }_{-\infty }^{a}f(x)dx$,Thus $H$ decreases while $F(a)<1/2$ and increases while $F(a)>1/2$. It is minimized at any median, characterized in the continuous case by

For points $x_1,\dots ,x_n$ and coefficients $c_1,\dots ,c_n$,The bound $|\hat{\varphi }|\le M$ ensures that this expected value is finite. Symmetry is immediate, so an expected outer product of a random feature map always defines a positive-semidefinite kernel.

Subtracting the expected value of $f$ does not alter either side, so suppose $\pi (f)=0$. Since reversibility makes $K$ a self-adjoint operator and stationarity makes it a contraction,The Discrete-time Poincaré inequality for a Markov kernel is therefore equivalent toApplying this inequality successively to $f,Kf,\dots ,K^{t-1}f$ yieldsConversely, the asserted variance contraction with $t=1$ rearranges to the Poincaré inequality. Hence the two statements are equivalent.

Write $p=(1+\mu {)}^{-1}$. Since $X$ and the geometric random variable $Z$ have the same expected value $\mu$,This is also the entropy deficit relative to the maximum entropy distribution on the nonnegative integers.

The random variables $X$ and $-Z$ are independent, so part a gives $H(X-Z)\ge H(-Z)=H(Z)$. Consequentlywhich is the required bound in terms of the Entropic Ruzsa distance.

To first order in the weak fluctuation, $n^2-1=2\mu W+O({\mu }^2)$, soForintroduce the signs $s=(1,-1,1,-1)$. The product rule gives