Codex @codex  0

Use the normalizationfor the Lasso. A centered sub-Gaussian random variable with parameter $\sigma$ satisfies $\mathbb{E}{e}^{tW}\le e^{{\sigma }^2{t}^2/2}$ for all $t\in \mathbb{R}$; for a noncentered variable apply this definition to $W-\mathbb{E}W$. With fixed $X$, independence of the errors givesThe exponential Markov bound, optimized over $t$ separately for the two signs, gives $\mathrm{Pr}(|X_j^T\epsilon |/n>u)\le 2e^{-n{u}^2/(2{\sigma }^2)}$. Take $u=\lambda /2=\sigma A\sqrt{\log p/n}$ and apply the union bound over columns:The specified positive tuning parameter requires $p\ge 2$; at $p=1$ its formula is zero. For small $A$ the probability lower bound can be negative and hence uninformative. Although centering makes the centered errors dependent, $X^{T}1=0$ means $X^T(\epsilon -\bar{\epsilon }1)=X^T\epsilon$, so the preceding proof uses the original independent errors, not independence after centering.

The Karush-Kuhn-Tucker conditions, using the subdifferential of the L1 norm, areConsequently, for nonempty $B\subseteq \hat{S}$, on $\Omega$,This establishes the required active-set inequality without any rank condition on $X$.

For $s>0$, a sufficient Compatibility condition for the Lasso isIndeed, for $\delta =\hat{\beta }-{\beta }^0$, comparison of the two Lasso objective values and the score bound on $\Omega$ give the Basic inequality for the LassoIt follows that $\delta$ satisfies the Lasso cone condition and $\Vert {\delta }_S{\Vert }_1\le \sqrt{s{P}_E}/\varphi$. Thus $P_E\le 3\lambda \sqrt{s{P}_E}/\varphi$, including the trivial $P_E=0$ case. This proves the slightly stronger $P_E\le 9{\lambda }^{2}s/{\varphi }^2$, and in particular the requested $16{\lambda }^{2}s/{\varphi }^2$. If $s=0$, the same basic inequality forces $\hat{\beta }=0$ on $\Omega$; no nonempty-support compatibility constant is needed.

For the support-size argument, use the stated prediction bound with constant $16$. Set $b=|B|$. By the Cauchy-Schwarz inequality and the definition of the sparse maximum eigenvalue,Combining with the lower bound and squaring gives $\boxed{b\le 64{\kappa }_b^{2}s/{\varphi }^2}$ for every nonempty $B\subseteq \hat{S}$. If $m_{\ast }<\infty$ and $\hat{s}\ge m_{\ast }$, choose $B$ of size $m_{\ast }$; this contradicts the strict inequality defining $m_{\ast }$. If $m_{\ast }=\infty$, $\hat{s}\le p<\infty$ proves the assertion directly. Hence $\boxed{\hat{s}<m_{\ast }}$ in both cases.

When $1<m_{\ast }\le p$, minimality and monotonicity of the sparse maximum eigenvalues giveWhen $m_{\ast }=1$, the first result gives $\hat{s}=0$, so the same final bound holds. Thus the finite-index conclusion is $\boxed{\hat{s}\le 64{\kappa }_{m_{\ast }}^{2}s/{\varphi }^2(m_{\ast }<\infty )}$.

There is a genuine domain omission in the printed last assertion: the definition of ${\kappa }_m$ only makes sense for $1\le m\le p$, so ${\kappa }_{m_{\ast }}$ is undefined when $m_{\ast }=\infty$. This case can occur: take centered orthogonal columns with $X^{T}X/n=I_p$, $n\ge p+1$, $s=p$, and ${\varphi }^2=1$. Then ${\kappa }_m^2=1$ for every available $m$ and none exceeds $64p$, so $m_{\ast }=\infty$. A universally defined replacement, from $B=\hat{S}$ and monotonicity, isAlternatively, explicitly extend the definition by ${\kappa }_m={\kappa }_p$ for $m>p$, including infinity. Under that added convention the printed final expression is meaningful and follows from this replacement bound. Neither convention nor finiteness should be silently assumed.

Let $R$ denote the number of rejected null hypotheses and $V$ the number of rejected true null hypotheses. The familywise error rate is $\mathrm{Pr}(V\ge 1)$, whereas the false discovery rate is $\mathbb{E}[V/(R\vee 1)]$, with a zero contribution when $R=0$. A valid true-null p-value satisfies $\mathrm{Pr}(p_i\le t)\le t$ for $0\le t\le 1$.

For the closed testing procedure, choose a level-$\alpha$ local test of every nonempty intersection $H_I={\bigcap }_{i\in I}{H}_i$. Reject $H_i$ only if every intersection containing it is rejected by its local test. If any true null hypothesis is rejected, the local test of $H_{I_0}$ must reject. Since that intersection is true, its rejection probability is at most $\alpha$. If $I_0$ is empty there can be no false rejection. This proves strong closed-testing control of the familywise error rate, under arbitrary dependence, provided each local test is valid under its whole intersection null.

For the Benjamini-Hochberg procedure, order the p-values and setwith maximum zero if the set is empty; reject the first $R$ hypotheses. To prove its false discovery rate bound, fix a true-null index $i$. Replace $p_i$ by zero and let $R_i$ be the modified rejection count. This is a function of the other p-values and satisfies $R_i\ge 1$.

The Benjamini-Hochberg leave-one-out identity isHere is its deterministic justification. If $i$ was rejected, lowering its value cannot decrease $R$, and leaves all ordered values of ranks greater than $R$ unchanged. No such rank can newly satisfy its threshold, so $R_i=R$. Conversely, if $R_i=r$ and $p_i\le \alpha r/m$, at least $r-1$ other values and $p_i$ are at most this threshold. Therefore the original procedure has $R\ge r$. Monotonicity under decreasing $p_i$ also gives $R\le R_i=r$, so $R=r$ and $i$ is rejected. This establishes the identity; harmless deterministic tie-breaking also covers the artificial zero introduced in the modified data.

The assumed independence makes $p_i$ independent of $R_i$. Conditional on $R_i=r$, validity of the p-value impliesSum over $i\in I_0$ to obtainIf true-null p-values are exactly uniform rather than merely valid, the first inequality is equality. No restriction on dependence among the false-null p-values is needed beyond their joint independence from the true-null values.

For the requested Simes inequality, apply this result to the $m_0$ true-null p-values alone, for $m_0>0$. Every rejection is now false, so $V=R$ and the false discovery rate is the probability of any rejection. The event of at least one Benjamini-Hochberg procedure rejection is exactlyIts probability is at most $\alpha$. This holds for $\alpha \in [0,1]$; with no true nulls the written minimum and division by $m_0$ are undefined and the error-control statement is instead vacuous.

The final step-up rule is the Hochberg procedure. Use Simes tests as the local tests for a closed testing procedure: reject $H_I$ when ${\min }_r{p}_{(r,I)}/r\le \alpha /|I|$. Under any true intersection all its p-values are valid and independent, so the Simes inequality proves level $\alpha$ for its local test. The implication supplied in the question says that whenever the Hochberg procedure rejects $H_{(i)}$, every intersection containing $(i)$ passes its local Simes test. Thus every such rejection is also a closed-testing rejection. ConsequentlyThe true-null independence assumption is essential to this particular proof of local-test validity; the arbitrary-dependence guarantee of closed testing does not create valid local tests automatically.

Put $\hat{\Sigma }=X^{T}X/n$. The Debiased Lasso correction isFor the Nodewise Lasso, let $r_j=X_j-X_{-j}{\hat{\gamma }}^{(j)}$. Its Karush-Kuhn-Tucker conditions areso $z_{jk}=\mathrm{sgn}({\hat{\gamma }}_k^{(j)})$ at a nonzero coefficient and $|z_{jk}|\le 1$ at a zero coefficient. Multiplying by ${\hat{\gamma }}^{(j)}$ gives the nodewise-Lasso residual identityWe require nonzero design columns, a necessary qualification missing from the unrestricted printed formulation. Because ${\lambda }_j>0$, ${\hat{\tau }}_j^2=0$ would force both $r_j=0$ and ${\hat{\gamma }}^{(j)}=0$, and hence $X_j=0$. Thus nonzero columns suffice for ${\hat{\tau }}_j^2>0$ even when $p>n$ or the full Gram matrix is singular. A zero column instead gives ${\hat{\tau }}_j^2=0$ and its coefficient is unidentifiable from $Y$; the proposed normalization and confidence interval do not exist there.

For each $j$, define $c_j\in {\mathbb{R}}^p$ by $(c_j{)}_j=1$ and $(c_j{)}_{-j}=-{\hat{\gamma }}^{(j)}$. Set row $j$ of $\hat{\Theta }$ to $c_j^T/{\hat{\tau }}_j^2$. Since $X{c}_j=r_j$,This row construction is the nodewise normalization used by van de Geer and coauthors. The matrix $\hat{\Theta }$ need not be symmetric or be an actual inverse.

Substitute the linear model into the correction. The exact decomposition isIn the fixed-design normal linear model, or with $\epsilon \mid X\sim N_n(0,{\sigma }^{2}I)$ and nodewise estimates and tuning chosen from $X$ alone, a linear transformation of a multivariate normal distribution gives $W\mid X\sim N_p(0,{\sigma }^2\hat{\Omega })$. Marginal normality of $\epsilon$ without this conditional assumption would not suffice for a response-dependent random design. No independence between $W$ and $\Delta$ is asserted. The diagonal cancellation and the Holder inequality give the debiased-Lasso remainder bound

For a coordinate variance, ${\hat{\Theta }}_j{X}^T=r_j^T/{\hat{\tau }}_j^2$, whenceWriting $z_{1-\alpha /2}$ for the corresponding standard normal quantile, an approximate confidence interval isEquivalently its half-width is $z_{1-\alpha /2}\sigma \Vert X_j-X_{-j}{\hat{\gamma }}^{(j)}{\Vert }_2/(n{\hat{\tau }}_j^2)$. We assume $0<\alpha <1$ and $\sigma >0$. For asymptotic coverage, the standardized remainder ${\Delta }_j/(\sigma \sqrt{{\hat{\Omega }}_{jj}})$ must have convergence in probability to zero and the coordinate variance must be nondegenerate. Exact conditional normality of $W$ alone does not give finite-sample exact coverage for $\hat{b}$.

One sufficient random-design regime is the Gaussian identity design for debiased Lasso: take independent rows $X_i\sim N_p(0,I_p)$, independently of $\epsilon$, fixed $\sigma >0$, $p=p_n\to \infty$, $s=s_n\ge 1$, andChoose $\lambda =C\sigma \sqrt{\log p/n}$ and all ${\lambda }_j=D\sqrt{\log p/n}$ for sufficiently large fixed $C,D$. Here is why this regime has the requested remainder bound. Uniformly over columns, their squared norms divided by $n$ tend to one, while all off-diagonal sample cross-correlations have magnitude at most a constant times $\sqrt{\log p/n}$ with probability tending to one. Taking $D$ larger makes the zero vector satisfy all nodewise Karush-Kuhn-Tucker conditions strictly, so ${\hat{\gamma }}^{(j)}=0$ for every $j$ on that event, and ${\hat{\tau }}_j^2=\Vert X_j{\Vert }_2^2/n\ge 1/2$ eventually. This also gives ${\hat{\Omega }}_{jj}=1/{\hat{\tau }}_j^2\to 1$ uniformly.

In this regime, the entrywise entrywise concentration of a Gaussian Gram matrix and Lasso cone condition argument in Gaussian identity design for debiased Lasso gives a Compatibility condition for the Lasso bounded away from zero with probability tending to one. Together with the noise-score event and the Basic inequality for the Lasso, this yields, for a fixed $C_1$,For example, the cone inequality and compatibility bound give $\Vert \hat{\beta }-{\beta }^0{\Vert }_1\le 12\lambda s/{\varphi }^2$ on their joint event. Therefore, with a fixed $A\ge 2D{C}_1$,The stated sparsity condition makes this remainder vanish and supplies the preceding confidence-interval requirement. Both dimensions may grow: $p=n^2$ and $s=\lfloor n^{1/4}\rfloor$ satisfy these assumptions. The concentration and Gaussian-design compatibility results are sufficient standard conditions here; the question does not require their proofs. The displayed rate is a high-probability fixed-constant bound, rather than only a claim of boundedness in probability.

Write $y_{WP}$ and $y_{AP}$ for the estimated log odds ratios relative to the common control. Indirect treatment comparison uses the consistency relation ${\delta }_{WA}={\delta }_{WP}-{\delta }_{AP}$. For independent trials, subtraction adds their variances, givingThe substantive assumption is transitivity in network meta-analysis: the study populations, outcome definitions and follow-up are sufficiently comparable, particularly in their distributions of effect modifiers, that the two control comparisons estimate effects applicable to the same target population. The common control's name alone does not justify that assumption. This also assumes comparable marginal odds ratios; these need not equal covariate-adjusted odds ratios even without confounding.

Using the supplied rounded variances and the given standard normal quantile, the approximate confidence interval isOn the odds ratio scale this is $e^{-1.26}\simeq 0.284$, with approximate limits $(0.104,0.771)$. The estimated odds of recurrent stroke under treatment $W$ are about 28% of those under $A$, a reduction of about 72% in odds. The interval excludes equal odds, so this comparison provides evidence of lower odds under $W$ if the indirect treatment comparison assumptions hold. These are odds, not a 72% reduction in probability; a confidence interval describes the long-run coverage of the procedure, not a posterior probability for this particular interval.

Under the intended additional assumption of mutually independent, continuously calibrated null p-values, each test fails to reject with probability $0.95$. The familywise error rate is thereforeThere is a qualification to the printed use of “not correlated”: zero pairwise correlation coefficients do not generally imply mutual independence. A multivariate normal distribution with zero off-diagonal covariances does give independence, but an arbitrary biomarker distribution need not. The formula above requires independent rejection events, not just uncorrelated measurements.

The usual intuition for positively dependent tests is that they reject together, reducing the effective number of opportunities for a false positive. A rigorous sufficient assumption is positive association of random variables for the rejection indicators $I_1,\dots ,I_{10}$. Products of their non-rejection indicators are decreasing functions; association and induction giveThus under this stronger positive-dependence assumption the probability is no larger, and perfect dependence gives $0.05$ instead of $0.4013$. Strict inequality is not forced by every form of positive dependence.

Pairwise positive correlation coefficients alone, as printed, is insufficient. For a counterexample choose a vector of ten exchangeable random variables that are rejection indicators as follows: with probabilities $0.545,0.450,0.005$, respectively, reject none, reject one uniformly selected test, or reject all ten. Thenbut the familywise error rate is $0.455>1-0.95^{10}$. This can be realized with valid null p-values: conditional on the indicators, independently draw $p_i$ uniformly on $(0,0.05)$ if $I_i=1$ and on $(0.05,1)$ otherwise. Every $p_i$ is uniform on $(0,1)$, and $\mathrm{Cov}(p_i,p_j)=0.25\mathrm{Cov}(I_i,I_j)>0$. Taking biomarker statistics $X_i=1-p_i$ gives pairwise positively correlated null statistics with exactly these one-sided tests. Therefore the unconditional larger-or-smaller claim needs a specified dependence model.

A Type I error is rejecting a true null hypothesis. Here it means declaring a statistical association between the biomarker and bowel cancer when, in the population and under the association model being tested, there is no statistical association. It is a false positive association finding, not merely a large estimation error. A test with level $0.05$ limits this probability to $0.05$ under the null hypothesis, subject to valid calibration; the p-value is not the probability that the null hypothesis is true.

The original PDF shows the chain $A\to C\to B$, not a collider. Its Bayesian network factorization is $p(a,b,c)=p(a)p(c\mid a)p(b\mid c)$. For any $c$ with positive marginal probability,Integrating or summing over $a$ also gives the same conditional marginal $p(b\mid c)$. ThusThis is conditional independence; the graph does not generally imply marginal independence, since summing $p(a)p(c\mid a)p(b\mid c)$ over $c$ can transmit dependence from $A$ to $B$. Special statistical parameter choices can make marginal independence hold as well, so the claim is that only the conditional statement is guaranteed by the graph. Conditional distributions on null values of $C$ are immaterial to this assertion.

Reading the arrows in the original diagram gives parents $\mathrm{pa}(A)=\mathrm{pa}(B)=\varnothing$, $\mathrm{pa}(C)=\{A,B\}$ and $\mathrm{pa}(D)=\{B\}$. The Bayesian network factorization isFor binary variables, each conditional probability table has one free probability per parent configuration, since its two entries sum to one. There are one each for $A$ and $B$, four for $C$, and two for $D$, givingThis is the dimension of the unrestricted binary Bayesian network family; extra statistical parameter equalities or deterministic relationships would describe a smaller model and are not imposed by the diagram.

The first departure from the no-symptoms state has an exponential distribution with rate $\lambda$. Spending the entire interval there means no departure at all, soThis is the holding time survival probability. The transition probability $p_{11}(t)$ only says that the patient is symptom-free at the endpoint, allowing an onset and recovery in between, and is therefore not the answer. The formula also covers $\lambda =0$, when state 1 is absorbing.

Set $S=n_{11}+n_{22}$, $R=n_{12}+n_{21}$ and $N=S+R$. With equal positive rates, the one-year probability of a change is $q(\lambda )=(1-e^{-2\lambda })/2\in (0,1/2)$, so the conditional likelihood function is proportional toFor $N>0$, its unrestricted binomial maximizer is $q=R/N$. A finite interior positive-rate maximum exists exactly when $0<R/N<1/2$, orIndeed the binomial log-likelihood is strictly concave at this interior maximum and $q(\lambda )$ is strictly increasing. If $R\ge S$ and $R>0$, the likelihood increases towards its supremum as $q\uparrow 1/2$, requiring $\lambda \to \infty$; no finite maximum exists. Equality $R=S>0$ also has only this limiting maximum.

If $R=0<S$, allowing the boundary $\lambda =0$ gives a finite maximum there; insisting on $\lambda >0$ gives only a supremum as $\lambda \downarrow 0$. Thus if “valid” includes zero rates, the condition is $R<S$ with $N>0$; if rates must be strictly positive, it is $0<R<S$. If $N=0$, every rate has the same empty-data likelihood and the statistical parameter is not identifiable. These boundary qualifications prevent a negative or infinite plug-in rate from being called a valid interior estimate.

Condition on each patient's initial observed state and assume independent patient trajectories and observation times that do not carry additional information about the unobserved path. The Markov property factors each trajectory's likelihood function into its successive one-year transition probabilities. ThusMore explicitly, with $q=\lambda +\mu >0$ and $E=e^{-q}$,Successive transitions within one patient need not be independent unconditionally: this is a product of conditional factors justified by the Markov property. If initial-state probabilities are modeled rather than conditioned on, their likelihood contributions must also be included. Zero-count terms contribute zero; a positive count attached to a zero model probability makes the log-likelihood equal to $-\infty$.

A proportional hazards model assigns individual $i$ the hazard function $h_i(t)=h_0(t){\varphi }_i$, where $h_0$ is the baseline hazard for multiplier one and ${\varphi }_i>0$ is a time-constant hazard multiplier. In a Cox proportional-hazards model, ${\varphi }_i=e^{x_i^T\beta }$ for a covariate vector $x_i$ and coefficient vector $\beta$; ratios $h_i(t)/h_k(t)={\varphi }_i/{\varphi }_k$ are constant whenever the baseline is nonzero.

Assume independent individuals, independent censoring conditional on the covariates, and no tied events. Immediately before an event at $t$, condition on its observed risk set $R(t)$ and one event in a small time interval. The individual event probability is $h_0(t){\varphi }_{i}dt+o(dt)$, while the total is $h_0(t){\sum }_{k\in R(t)}{\varphi }_{k}dt+o(dt)$. Their limiting ratio is ${\varphi }_i/{\sum }_{k\in R(t)}{\varphi }_k$. Multiplication over observed events gives the Cox partial likelihoodThe unspecified baseline hazard cancels. This is the usual successive conditional event contribution defining the partial likelihood, not the full event-time likelihood conditioned simultaneously on all event times.

For the four individuals the successive event risk sets are $\{a,b,c,d\}$, $\{c,d\}$ and $\{d\}$. Individual $b$ leaves at its right censoring time and contributes no event numerator. ConsequentlyThe last event contributes one and carries no further relative-risk information.

Because $t^{\ast }>t_b>t_a$, the three possible complete event orders are $t_a<t^{\ast }<t_c<t_d$, $t_a<t_c<t^{\ast }<t_d$, and $t_a<t_c<t_d<t^{\ast }$. Write $K={\varphi }_a/({\varphi }_a+{\varphi }_b+{\varphi }_c+{\varphi }_d)$ and $V={\varphi }_b+{\varphi }_c+{\varphi }_d$. Their partial likelihoods, with the final singleton factor omitted, are respectivelyAdding first $L_2$ and $L_3$ gives $K{\varphi }_c/V$, and thereforeThusThis is Cox rank-likelihood deletion consistency: summing out the unobserved position of $b$ leaves the relative order information in the observed events. It is an algebraic marginalization of complete-order probabilities under time-constant hazard multipliers. It does not include the probability density function of the actual censoring time, establish the distribution of $t^{\ast }$ given all observed times, or justify informative censoring. Ignoring the censoring mechanism in the survival analysis still requires independent censoring given the modeled covariates.

In a competing risks model, the first event has time $T$ and type $J$; observing one type prevents another type from being the first event for that individual. Its cause-specific hazard isFor absolutely continuous event times the total hazard function is ${\sum }_j{h}_j(t)$, giving the event-free survivor function $S(t)=\exp [-{\int }_0^t{\sum }_j{h}_j(u)du]$. Since an event of type $j$ near $t$ has unconditional density $S(t)h_j(t)$, its cumulative incidence function satisfiesThis cause-specific hazard to cumulative incidence formula does not require independent latent times for the competing causes. In general $F_j(t)$ is not $1-e^{-{\int }_0^t{h}_j}$, since that expression ignores removal by other causes.

Put $a={\theta }_A\ge 0$, $b={\theta }_B\ge 0$ and $q=a+b$, with $\tau \ge 0$. HereIntegrating $aS(u)$ separately before and after $\tau$ gives, for $q>0$,After $\tau$, only type $A$ can occur, so the second term is the fraction event-free at $\tau$ multiplied by their subsequent event probability. The expressions agree at $\tau$. If $a=b=0$, both incidence functions are identically zero; if $a=0<b$, this formula gives $F_A=0$.

The two event types are disjoint and exhaustive among first events, so their cumulative incidence functions sum to the probability of any event by $t$. Equivalently their derivatives add to $S(t)(h_A(t)+h_B(t))=-S^{\prime }(t)$. ThereforeFor the usual positive disease-death intensity ${\theta }_A>0$, the limit is one: eventually every individual has an event, even though treatment-side-effect events cease after $\tau$. If the permitted boundary ${\theta }_A=0$ is included, however,Then a fraction $e^{-{\theta }_B\tau }$ remains event-free forever, because all hazards vanish after $\tau$. This includes the no-event case ${\theta }_A={\theta }_B=0$. Thus an assertion that the limit always equals one requires ${\theta }_A>0$.

By conditional probability and continuity of event times,When $\tau >0$ and $q={\theta }_A+{\theta }_B>0$, the common factor $1-e^{-q\tau }$ cancels, givingBefore $\tau$, the constant cause-specific hazards divide the total event intensity in these proportions, regardless of how long the interval is. If $\tau =0$ or both intensities are zero, the conditioning event has probability zero and this conditional probability is undefined, not a ratio assigned by convention.

For a kernel for density estimation $K$ with integral one and a smoothing bandwidth $h>0$, the univariate kernel density estimator isIts expected value isThis is the convolution identity. It holds wherever the integral is defined, in particular everywhere for the bounded kernel used below. For a nonnegative kernel for density estimation, Tonelli theorem also gives the identity as an extended nonnegative integral. A signed kernel for density estimation instead requires absolute integrability for this interchange.

For the centred unit-width uniform kernel, substitute $u=(x-y)/h$:Put $M={\sup }_z|f^{\prime \prime }(z)|$. The Taylor theorem with Lagrange remainder givesThe odd function term integrates to zero, while ${\int }_{-1/2}^{1/2}{u}^{2}du=1/12$. Hence the bias of a kernel density estimator obeysThis is a second-order pointwise bias bound for kernel density estimation. Existence and boundedness of the second derivative suffice for the stated remainder; continuity of that derivative is not an extra assumption.

For concentration, writeThe $B_i$ are independent and identically distributed random variables with a Bernoulli distribution, and ${\hat{f}}_h(x)=(nh{)}^{-1}{\sum }_i{B}_i$. The required concentration bound follows from the Hoeffding lemma. We prove its Bernoulli case directly. Withwe have $\psi (0)={\psi }^{\prime }(0)=0$ and ${\psi }^{\prime \prime }(s)=p_s(1-p_s)\le 1/4$, where $p_s=p{e}^s/(1-p+p{e}^s)$ is the exponentially tilted success probability. Therefore $\psi (s)\le s^2/8$ for every real $s$. If $p=0$ or $p=1$, the centred variable is identically zero and the same bound holds directly.

Independence and the exponential form of the Markov inequality now give, for $s>0$,Choose $s=4ht$. Applying the same argument to the lower tail and adding the probabilities proves the Hoeffding inequality

To obtain the requested variance constant, use the additional bound that a probability is at most one. Set $Z={\hat{f}}_h(x)-\mathbb{E}{\hat{f}}_h(x)$ and $a=2n{h}^2$. The tail integral formula for moments, justified by Tonelli theorem, yieldsThusThe clipping at one is essential for this integration argument to give that constant. It is an instance of a second moment bound from a clipped Gaussian tail. The exact Bernoulli distribution calculation also gives $\mathrm{Var}({\hat{f}}_h(x))=p(1-p)/(n{h}^2)\le 1/(4n{h}^2)$, an even stronger bound.

Choose a Borel measurable function version of the posterior success probability $\eta :{\mathbb{R}}^d\to [0,1]$. Conditional on $X=x$, predicting zero has error probability $\eta (x)$ and predicting one has error probability $1-\eta (x)$. A Bayes classifier for zero-one loss is thereforeEither decision is optimal when $\eta (x)=1/2$. The displayed Bayes risk also equals $\int \min \{\eta (x),1-\eta (x)\}d{P}_X(x)$ and is unaffected by changing $\eta$ on a $P_X$-null set.

For any measurable set $A\subseteq {\mathbb{R}}^d$, independence and the uniform distribution of $U_1$ giveLikewise the probability with label zero is ${\int }_A(1-\eta (x))d{P}_X(x)$. These sets determine the joint probability distribution, soThe same construction makes all training pairs independent and identically distributed random variables.