Put $h=H^2(P_0,P_1)$ with the paper's normalized Hellinger distance. The Hellinger affinity isThe last bound follows from the Cauchy-Schwarz inequality and the unit integrals of the two probability density functions. Using a common dominating probability measure, for example $(P_0+P_1)/2$, makes the product calculation immediate. The probability density functions of the product measures factor, and Tonelli's theorem givesThe Squared Hellinger distance is independent of the choice of common dominating measure, so this calculation applies to the original distributions. We obtain the exact identity and then apply Bernoulli's inequality:For completeness, the scalar bound follows by induction: $(1-h{)}^1=1-h$, and multiplying $(1-h{)}^n\ge 1-nh$ by the nonnegative $1-h$ gives $(1-h{)}^{n+1}\ge 1-(n+1)h+n{h}^2\ge 1-(n+1)h$.

The statement necessarily treats $P_0,P_1$ as probability measures, as required by the sampling model $X\sim P_J$. The word "measures" in the introduction must be read with this normalization. Use the paper's convention $H^2(P,Q)=\frac{1}{2}\int (\sqrt{p}-\sqrt{q}{)}^{2}d\mu$ for the Squared Hellinger distance throughout this question.

A measurable binary hypothesis test $\psi$ is determined by the measurable set $A=\{x:\psi (x)=1\}$. Under the uniform prior on the two hypotheses, its error probability isSince the two measures have equal total mass,Indeed, the integral over any set is at most the integral of the positive part, and equality holds for $A=\{p_1>p_0\}$. This is also the total variation distance. The likelihood ratio rule choosing hypothesis $1$ precisely on that set therefore attains the infimum, provingArbitrary choices on the tie set $\{p_1=p_0\}$ leave the error unchanged. This is the exact link between total variation distance and optimal binary statistical hypothesis testing.

Let $a_1,\dots ,a_d>0$ be the eigenvalues of the positive-definite matrix $V_n$. Its trace satisfiesThe arithmetic-geometric mean inequality givesSince $\det V_0={\lambda }^d$, taking the logarithm provesThe determinant is largest, for a fixed trace, when the eigenvalues are equal; this is the geometric content of the estimate.

Work for an arbitrary realization of the sequence, and set $u_t=X_t^T{V}_{t-1}^{-1}{X}_t$. In the Loewner order,By the spectral theorem for real symmetric matrices, every eigenvalue of $V_{t-1}$ is at least $\lambda$. The matrix inverse therefore has eigenvalues at most ${\lambda }^{-1}$, andFor $0\le u\le 1$, put $g(u)=2\log (1+u)-u$. Since $g(0)=0$ and $g^{\prime }(u)=(1-u)/(1+u)\ge 0$, we have $u\le 2\log (1+u)$. The matrix determinant lemma now givesTaking logarithms and using a telescoping sum,This elliptical potential bound holds for every realization and every $n$, without a probabilistic exceptional event. The condition $\lambda \ge 1$ is what ensures $u_t\le 1$.

We give a finite chaining argument, keeping the constants explicit. If $\delta =0$, the metric space has just the point $t_0$ and both sides vanish. Suppose $\delta >0$, and writeChoose a smallest metric net $T_k\subseteq T$ of radius ${\epsilon }_k$, and maps ${\pi }_k:T\to T_k$ satisfying $d(t,{\pi }_k(t))\le {\epsilon }_k$. Set $T_0=\{t_0\}$ and ${\pi }_0(t)=t_0$. Since $T$ is finite, choose $K$ with ${\epsilon }_K$ smaller than the minimum distance between distinct points; then $T_K=T$ and ${\pi }_K(t)=t$.

For each $k$, retain only the pairs actually occurring along these approximation paths:The triangle inequality gives $d(u,v)\le {\epsilon }_k+{\epsilon }_{k-1}=3{\epsilon }_k$ for $(u,v)\in E_k$; for $k=1$ the same argument uses $d(t,t_0)\le \delta ={\epsilon }_0$. AlsoHere only monotonicity in the radius for the same space $T$ is used. No monotonicity under subsets is assumed.

For every pair in $E_k$, the increment has the Sub-Gamma random variable in the right tail bound with common parameters $9\nu {\epsilon }_k^2$ and $3c{\epsilon }_k$: for $0<s<(3c{\epsilon }_k{)}^{-1}$,Identical pairs give the zero increment, which obeys the same bound. Both orientations have finite exponential moments, so all increments are integrable random variables. Differentiating their two upper bounds at zero also gives $\mathbb{E}(X_u-X_v)=0$. Applying the maximum bound from part (b), including its $m_k=1$ case, givesThe telescoping sum along each path is exact:Take the supremum, bound it by the sum of the maxima, and take expected values. This provesFor either $f(u)=\sqrt{\log N(u;T,d)}$ or $f(u)=\log N(u;T,d)$, the function is nonnegative and nonincreasing. ThusCombining the estimates yields the required metric entropy bound:In particular, $C=12$ and $C^{\prime }=12$ work. The square-root and linear metric entropy terms arise respectively from the two terms in the sub-Gamma maximum bound. Because the chain ends exactly at $T$, no limiting sample-path regularity assumption is needed.

Use closed balls with centres belonging to the space being covered. The covering number iswith value $\infty$ if there is no finite metric net, and value zero for the empty space.

With this intrinsic-centre convention, the proposed monotonicity under subsets is false. Give $R=\{-1,0,1\}$ the usual distance, take $S=\{-1,1\}$, and set $\epsilon =3/2$. A ball centred at $0$ covers $R$, so $N(3/2;R,\rho )=1$. The two points of $S$ are distance $2$ apart, so a ball centred in $S$ contains only its centre, and $N(3/2;S,\rho )=2$. These values are also valid with open balls.

The obstruction is that the useful centre $0$ disappears on passing to the subspace. If centres were allowed anywhere in a fixed ambient space, monotonicity would hold. With intrinsic centres, one instead hasIndeed, discard covering balls missing $S$ and choose one point of $S$ from each remaining ball. The triangle inequality shows that every point of its intersection with $S$ is within $2\epsilon$ of the chosen point. This proves the replacement bound whenever the right side is finite; otherwise it is automatic.

The printed domain assertion is false. For example, take $K=\mathbb{Q}$ and $I=(4)$. Then ${\mathcal{O}}_K/I=\mathbb{Z}/4\mathbb{Z}$ has the nonzero zero divisor $2+(4)$, so it is not an integral domain and cannot be a principal ideal domain. The valid conclusion is that ${\mathcal{O}}_K/I$ is a principal ideal ring. This error is present in the original PDF, not just its transcription.

To prove the corrected conclusion, factorinto powers of distinct nonzero prime ideals. These powers are pairwise comaximal ideals. Indeed, a maximal ideal containing two such powers would contain both underlying prime ideals, forcing them to be equal. The Chinese remainder theorem givesEvery factor is a principal ideal ring by the preceding part. Every ideal $H$ in a finite direct product of rings is a product $H_1\times \cdots \times H_s$ of ideals in the factors: multiplication by the coordinate idempotent elements separates the coordinates of every element of $H$. Write $H_j=(h_j)$. Thenso $R/I$ is a principal ideal ring. This proves the principal ideal property of quotients of a Dedekind domain in the present setting. In fact, this finite quotient ring is an integral domain precisely when $I$ is a prime ideal, in which case it is a field.

For the final deduction, choose any $0\ne a\in I$ and apply the result to the nonzero proper ideal $(a)$. The ideal $I/(a)$ of $R/(a)$ is generated by one residue class $b+(a)$, which can be represented by $b\in I$. For each $x\in I$, some $r\in R$ satisfies $x-rb\in (a)$. HenceThis proves two generators for an ideal in a Dedekind domain; moreover, its first generator can be any prescribed nonzero element of $I$.

For every $i\ge 0$, the inclusion $P^{i+1}\subseteq P^i$ is strict: equality, followed by multiplication by the inverse of $P^i$, would give $P=R$. Hence one can choosePut $J=(\pi )+P^{i+1}$. ThenThe classification just proved, applied modulo $P^{i+1}$, leaves no ideal strictly between consecutive powers. It follows thatFor $i=0$, the same argument says $(\pi )+P=R$, using $\pi \notin P$ and maximality.

The extension to every $r\ge 1$ follows by mathematical induction. Suppose $P^i=(\pi )+P^{i+r}$. Multiplication of the initial equality by $P^r$ givesSubstitution yieldsThusTaking $r=n-i$ for $0\le i<n$ givesThis is a principal ideal of the quotient ring $R/P^n$. Its zero ideal is generated by zero, so every ideal of this quotient ring is a principal ideal.

By the correspondence theorem for ideals, an ideal of $R/P^n$ has the form $J/P^n$ with $P^n\subseteq J\subseteq R$. If $J\ne R$, apply unique factorization of ideals in a number field to $J$. Every prime ideal $Q$ dividing $J$ contains $J$ and hence $P^n$. Primality gives $P\subseteq Q$, and maximality gives $Q=P$. Thus $J=P^i$ for some $i\ge 1$.

The inclusion $P^n\subseteq P^i$ forces $i\le n$: if $i>n$, cancellation using the invertible fractional ideal $P^n$ would give $R\subseteq P^{i-n}$, which is impossible. The zero ideal corresponds to $i=n$, while the unit ideal corresponds to $i=0$. Consequently the nonzero proper ideals are exactlyWhen $n=1$, there are none, in agreement with $R/P$ being a field.

Put $\alpha =\sqrt{m_1}$, $\beta =\sqrt{m_2}$, andThe proposed integral basis for coprime biquadratic radicands congruent to two and one isEach entry is an algebraic integer, by the ring of integers of a quadratic field formula and closure under products. Their coefficient matrix in $1,\alpha ,\beta ,\alpha \beta$ has determinant $1/4$, so they are rationally linearly independent. It remains to show that they span every algebraic integer.

Write $\gamma =(a+b\alpha +c\beta +d\alpha \beta )/2$ initially with rational $a,b,c,d$. The field traces to $\mathbb{Q}(\alpha )$ and $\mathbb{Q}(\alpha \beta )$ are $a+b\alpha$ and $a+d\alpha \beta$. Since $m_1\equiv m_1{m}_2\equiv 2(\mathrm{mod}4)$, the corresponding integral basis in each subfield gives $a,b,d\in \mathbb{Z}$. The field trace to $\mathbb{Q}(\beta )$ is $a+c\beta$. Membership in $\mathbb{Z}[\omega ]$ says that $2a,2c$ are integers of equal parity. As $a$ is already an integer, this forces $c\in \mathbb{Z}$ too.

Now the field norm to $\mathbb{Q}(\beta )$ isBy the ring of integers of a quadratic field formula, $A/2$ is an integer and has the same parity as $ac-m_{1}bd$. Thus $A$ is even, giving $a\equiv c(\mathrm{mod}2)$, andUsing $m_1\equiv 2$, $m_2\equiv 1$ and $a\equiv c(\mathrm{mod}2)$, this becomesHence $b\equiv d(\mathrm{mod}2)$. Finally,an expression with integer coefficients. This proves the asserted integral basis without assuming a general theorem about products of elements of an integral basis.

Again put $\alpha =\sqrt{m_1}$ and $\beta =\sqrt{m_2}$. The ring of integers of each of the three quadratic subfields has integral basis $1,\alpha$, $1,\beta$, or $1,\alpha \beta$, respectively: their square-free radicands are respectively congruent to $3,2,2$ modulo four. The field traces of the algebraic integer $\gamma$ to those subfields areThey are algebraic integers, so comparison with each integral basis givesThis use of all three subfields is needed to control $d$ as well as the other coefficients.

The field norm to $\mathbb{Q}(\alpha )$ isIts two coefficients must be integers. The second condition says $ab$ is even, since $m_2$ is even. Reducing the numerator of the first coefficient modulo four givesReduction modulo two first gives $a\equiv b(\mathrm{mod}2)$. Since $ab$ is even, both $a$ and $b$ are even. The same congruence now gives $c^2+d^2\equiv 0(\mathrm{mod}2)$, or $c\equiv d(\mathrm{mod}2)$. Therefore every algebraic integer has the formwith four integer coefficients.

For the converse, $1,\alpha ,\beta$ are algebraic integers, andHere $4\mid 1+m_1$ and $2\mid m_2$, so both coefficients are integers. The monic polynomial $X^2-w^2$ and integral dependence is transitive show that $w$ is an algebraic integer. The four displayed generators are rationally linearly independent, since their coefficient matrix in $1,\alpha ,\beta ,\alpha \beta$ has determinant $1/2$. Hence the integral basis for coprime biquadratic radicands congruent to three and two isIn this formula $\sqrt{m_1{m}_2}$ denotes the product $\alpha \beta$ of the chosen square roots; this convention also handles negative radicands.

Put $\alpha =\sqrt{m_1}$, $\beta =\sqrt{m_2}$ and $F=\mathbb{Q}(\alpha )$. The hypotheses make $L/\mathbb{Q}$ a biquadratic extension. To check the degree of a field extension is four, if $\beta =x+y\alpha$ with rational $x,y$, squaring gives $xy=0$. The case $y=0$ would make $m_2$ a rational square; the case $x=0$ would make $m_2/m_1$ a rational square. Neither can occur for the specified distinct coprime square-free integers. Hence $[L:F]=2$.

The nonidentity field automorphism of $L/F$ fixes $\alpha$ and sends $\beta$ to $-\beta$. Thus, for $\gamma =(r+s\alpha )+(t+u\alpha )\beta$, the field norm and field trace areIf $\gamma$ is an algebraic integer, so is its conjugate, and their sum and product are algebraic integers. Conversely, put $T={\mathrm{Tr}}_{L/F}(\gamma )$ and $N=N_{L/F}(\gamma )$. The identityshows that $\gamma$ is integral over $\mathbb{Z}[T,N]$. If $T,N$ are algebraic integers, the ring generated by algebraic integers $\mathbb{Z}[T,N]$ is integral over $\mathbb{Z}$. Since integral dependence is transitive, $\gamma$ is an algebraic integer. This proves the trace and norm integrality criterion for a quadratic extension, including elements already in $F$.

For $x_n=x_{\ast }$, the notation of part (i) gives $b=a$, $d=k_{\ast \ast }$, and $c=k_{\ast \ast }+{\sigma }^2$. Write $v={\mathrm{Var}}_{n-1}(f_{\ast })=k_{\ast \ast }-a^T{A}^{-1}a$. The remaining covariance is then $v$ and the new observation's remaining variance is $v+{\sigma }^2$. ThusEquivalently,The expression is valid also when $v=0$ and ${\sigma }^2>0$. For $v>0$, a noiseless observation, interpreted as the limit ${\sigma }^2\downarrow 0$, removes all remaining variance at the observed target.

The source's assertion that the target input always gives the greatest reduction needs an additional hypothesis. The reduction formula above does not require that assertion. For a counterexample, use the linear kernel $k(x,x^{\prime })=x{x}^{\prime }$ with ${\sigma }^2=1$ and target $x_{\ast }=1$. With no informative previous observations, $v=1$. This can also be arranged with $n-1=1$ by observing first at $x_1=0$, which has zero covariance with all latent values. Observing at the target gives reduction $1/2$, whereas observing at $x_n=2$ givesIndeed this Gaussian process is $f(x)=xZ$ for a standard normal random variable $Z$, so the measurement at $2$ observes the same unknown coefficient with a larger signal relative to its noise.

One sufficient extra condition is that the existing latent posterior variance be maximal at $x_{\ast }$ among all candidate inputs. To prove it, let $w=v_{n-1}(x)$ and $r=\mathrm{Cov}(f(x),f_{\ast }\mid {\mathbf{y}}_{1:n-1},\theta )$. The Cauchy-Schwarz inequality gives $r^2\le vw$. If $w\le v$, thenbecause $w\mapsto w/(w+{\sigma }^2)$ is increasing. Thus target sampling is optimal under this extra condition, but it is not implied by the paper's arbitrary-covariance function assumptions.

Additional observations weakly decrease the posterior variance at every fixed prediction input. Apply part (ii) repeatedly to add observations one at a time. An observation whose remaining conditional covariance with $f_{\ast }$ is nonzero strictly decreases that posterior variance; one with zero remaining conditional covariance leaves it unchanged. Large observation noise reduces the information gained through the denominator $v_{n-1}(x_n)+{\sigma }^2$.

For fixed Gaussian process hyperparameters and a Gaussian likelihood, the posterior variance depends on the observation locations and noise level, but not on the observed numerical outputs. Hence the reduction holds for each possible new output, rather than only on average. The predictive variance of a future noisy observation also decreases by the same amount because its extra independent noise variance is fixed. Refitting the hyperparameters after adding data can change the covariance function; the monotonicity statement compares conditioning under one fixed model.

The Schur complement can be writtenbecause the first term is a conditional variance and ${\sigma }^2>0$. Therefore $c_1=s^{-1}>0$ and the squared term is nonnegative, soEquality holds exactly when the remaining conditional covariance $k(x_n,x_{\ast })-b^T{A}^{-1}a$ is zero. In particular, the reduction need not be strict even at a previously unseen input.

Use only the first $n-1$ observations to defineThus the full noisy covariance matrix is $C_n=\left(\begin{array}{cc}A& b\\ b^T& c\end{array}\right)$ and the full cross-covariance vector is $(a^T,d{)}^T$. SetThe scalar $s$ is the Schur complement of $A$. Substituting the block inverse into the Gaussian process regression posterior variance givesConsequently,andFor interpretation, after the first $n-1$ observations the remaining covariance between $f_{\ast }$ and $f(x_n)$ is $r=d-b^T{A}^{-1}a$, while the conditional variance of the new noisy observation is $s$. Thus the reduction is the squared remaining covariance divided by the remaining observation variance.

The conditional multivariate normal distribution giveswhereFor a direct justification, form $R=f_{\ast }-k_{\ast }^T{C}^{-1}\mathbf{y}$. It is jointly Gaussian with $\mathbf{y}$, has zero mean, and satisfiesThe independence of uncorrelated jointly normal variables therefore makes $R$ independent of $\mathbf{y}$. Conditional on $\mathbf{y}$, only the displayed affine shift remains, proving the result. This is the latent-function posterior variance; a new observation $y_{\ast }=f_{\ast }+\sigma {ϵ}_{\ast }$ with fresh independent noise has predictive variance $v_n(x_{\ast })+{\sigma }^2$.

Condition throughout on the training inputs and $x_{\ast }$. Interpret $\theta$ here as the fixed covariance function and noise hyperparameters. Assume observation errors ${ϵ}_i$ are independent standard normal random variables, independent of the Gaussian process $f$, and writeThe independent-error assumption is the standard Gaussian process regression interpretation of the observation model; the single-observation marginal noise law alone would not specify cross-observation noise correlations. We take ${\sigma }^2>0$ so that $C$ is a positive-definite matrix. More generally the inverse formulas below apply whenever the indicated noisy covariance matrices are invertible.

The Gaussian process prior makes $(f_{\ast },\mathbf{f})$ a Gaussian random vector. Adding the independent Gaussian noise vector to $\mathbf{f}$ preserves a multivariate normal distribution. Its mean is zero, its observation covariance matrix is $C$, and $\mathrm{Cov}(f_{\ast },y_i)=k(x_{\ast },x_i)$ because the noise is independent of $f_{\ast }$. ThereforeA degenerate latent variance is allowed; the displayed multivariate normal distribution then denotes a Gaussian law even if it has no full-dimensional probability density.

Gaussian process regression supplies a predictive distribution and uncertainty estimates. A Gaussian process regression posterior gives both a posterior mean and a posterior variance, allowing credible intervals for the latent function and a posterior predictive distribution for future noisy observations. The kernel ridge regression estimator by itself specifies a point estimate.

With the same positive-semidefinite kernel and matching regularization, the Gaussian process regression posterior mean equals the kernel ridge regression prediction. Thus the main additional benefit is the explicit probability distribution of predictions, conditional on the chosen Gaussian process model and its hyperparameters, rather than an automatic improvement in point-prediction accuracy.

A recurrent neural network processes positions through a recurrence such as $h_t=F(h_{t-1},x_t)$. Computing $h_t$ requires $h_{t-1}$, so the standard computation is sequential across sequence positions, although different examples in a batch can be processed in parallel. Information from position $s$ to $t$ must pass through $t-s$ recurrent steps, and backpropagated gradients involve products of that many step Jacobian matrices. This creates vanishing gradients or exploding gradients and can make long-range dependencies difficult to learn. Gating can mitigate these difficulties while retaining the recurrent dependence.

In a transformer block, all queries, keys, and values for an available input sequence can be formed in parallel. The self-attention weights are computed by matrix multiplication, so positions within one layer can be processed together; layers themselves remain sequential. With unrestricted self-attention, one position can directly receive information from any other position in one layer, giving a constant-length route across distant positions rather than one proportional to their separation. With causal self-attention, there is the same direct route to any permitted earlier position.

A transformer model supports parallel training across known sequence positions and short routes for long-range dependencies. Even a causal model can compute all training positions together when the entire training sequence is supplied and the attention mask excludes future positions. However, ordinary autoregressive generation still generates one new token at a time. Full self-attention also constructs $L^2$ pairwise scores, with quadratic time and storage in sequence length for the direct implementation; a standard recurrent neural network has linear sequential work in sequence length at fixed hidden width. Neither architecture guarantees that a useful long-range dependency will actually be learned.