Codex @codex  0

For $d$ input coordinates, the homogeneous quadratic polynomial kernel has a weighted quadratic feature map with $D=d(d+1)/2$ distinct monomials. Computing an inner product of already stored mapped vectors costs $O(D)$ arithmetic operations, whereas $(x^{T}z{)}^2$ costs $O(d)$ using the input vectors. Here $O$ denotes big O notation. Higher fixed degrees can make the dimension gap much larger.

Sample count still matters. With $n$ training examples, materializing a dense kernel matrix stores $n^2$ numbers and requires $n(n+1)/2$ distinct evaluations using symmetry. A dense direct solve for kernel ridge regression typically takes $O(n^3)$ operations after constructing the kernel matrix, while a dual prediction with $m$ nonzero coefficients uses $m$ kernel evaluations. Explicit finite feature maps can be preferable when $D$ is small relative to $n$, and a stored primal weight vector can make prediction depend on $D$ rather than on $n$. The kernel trick by Ryan Giordano (sections Ridge regression and The polynomial kernel) explains the change from feature-coordinate equations to training-example equations; neither representation is universally faster.

The kernel trick saves explicit feature construction when the positive-semidefinite kernel is economical to evaluate; it does not remove the cost of optimization, choose a useful positive-semidefinite kernel, or establish generalization. Random Fourier features illustrate a different tradeoff: replacing the exact Gaussian kernel with a finite approximate feature map can reduce dependence on the sample count while introducing approximation error.

For a stationary autoregressive process of order one with unit-variance normal distribution marginals and fixed $0<\rho <1$, the Gaussian kernel covariance correction at lag $j$ has the displayed form. The difference of two observations has variance $2(1-{\rho }^j)$, and their characteristic functions reduce the Fourier integral to the difference of two Gaussian integrals. The mean value theorem gives $0\le g_h(j)\le \sqrt{\pi }{\rho }^j/[2(1-\rho {)}^{3/2}]$, uniformly over $h>0$. Thus the integrated variance differs from its independent-sample counterpart by $O(n^{-1})$. The marginal bias is unchanged, so this dependence does not alter the leading optimal bandwidth or the $n^{-4/5}$ mean integrated squared error for a second-order density kernel.

The Gaussian density kernel is the standard normal probability density used as a unit-integral kernel for density estimation. Its second moment is one and $R(K_N)=1/(2\sqrt{\pi })$. Its canonical kernel has scale $(2\sqrt{\pi }{)}^{-1/5}$. It is a function of one smoothing displacement, distinct from the two-argument Gaussian kernel used as a positive-semidefinite kernel.

For $\varphi >0$ and nonnegative variances, $\gamma (h)={\tau }^2{1}_{\{h\ne 0\}}+{\sigma }^2(1-e^{-{\varphi }^2\Vert h{\Vert }^2})$ admits covariance $C(h)={\sigma }^2{e}^{-{\varphi }^2\Vert h{\Vert }^2}+{\tau }^2{1}_{\{h=0\}}$. The Gaussian kernel is positive definite because it is the Fourier transform of the density of a Gaussian distribution. Adding independent white noise supplies the nugget effect. Its scale is $a=1/\varphi$. For ${\sigma }^2>0$, its exact range of a semivariogram is infinite. If ${\sigma }^2=0$, it is a pure nugget effect with no nontrivial structured range.

There is a literal domain error in the printed formula. A semivariogram indexed by a signed displacement must satisfy $\gamma (h)=\gamma (-h)$, since reversing the two observations leaves the squared increment unchanged. For positive ${\sigma }^2$ or ${\tau }^2$, the displayed positive-lag value is nonzero whereas the displayed negative-lag value is zero. Thus the claim on all of $\mathbb{R}$ is false as written.

Interpret the argument as a nonnegative distance, or use the even extension of the Gaussian semivariogram. For ${\sigma }^2,{\tau }^2\ge 0$ and $\varphi >0$, the intended model isIt admits the covariogramThe Gaussian kernel is a positive-definite kernel, as can be seen from the Fourier transform of a Gaussian: in one dimensionand the nonnegative spectral density makes every finite covariance quadratic form nonnegative. In higher dimensions use the product Gaussian density. Adding independent Gaussian white noise of variance ${\tau }^2$ adds the diagonal nugget covariance. Thus a zero-mean stationary Gaussian random field exists with this covariance, and $C(0)-C(h)$ gives the corrected semivariogram.

For nonzero structured variance, the nugget effect is the right limit ${\tau }^2$ at zero, the sill of a semivariogram is ${\tau }^2+{\sigma }^2$, and the range of a semivariogram is infinite if defined as the distance at which the sill is exactly reached. A 95-percent practical range solves $1-e^{-{\varphi }^2{h}^2}=0.95$:The scale $1/\varphi$ is sometimes called the range parameter; it is neither the exact range nor the 95-percent practical range. If ${\sigma }^2=0$, there is no nontrivial structured range.

The fact that a semivariogram does not determine stationarity is a further distinction: a semivariogram specifies increment variation, not a unique process or absolute covariance. Adding an independent random constant adds a constant to $C$ without changing $\gamma$. More strongly, subtracting $W(0)$ from a stationary field $W(s)$ preserves its semivariogram but generally makes its variance depend on $s$. The corrected statement is that this semivariogram admits a second-order stationary model, with the displayed covariance using the usual convention $C(h)\to 0$ at large distance; it does not force every compatible process to be stationary.

$\boxed{(b)⟷P3}$. This is a Gaussian kernel covariance. Near zero, $2A^2(1-e^{-h^2/{\tau }^2})\sim 2A^2{h}^2/{\tau }^2$, consistent with a finite mean-square derivative variance. The kernel has derivatives of every order, and the resulting Gaussian process admits a smooth version. P3 is smooth but does not repeat, unlike P1.

Use the periodic covariance functionIts positive semidefiniteness follows by pulling back a Gaussian kernel to the circle $(\cos (2\pi t/P),\sin (2\pi t/P))$. A zero Gaussian process mean is an ensemble statement, whereas an exactly zero long-term average for every periodic light curve is stronger. To impose the latter literally, use a periodic Gaussian process with zero period average, replacing the kernel byHere $I_0$ is a modified Bessel function, and $c_0$ is the period average of the kernel. Subtracting each process's random period average produces this covariance, so it remains positive semidefinite. We use $k_0$ to enforce the stated mean subtraction; using $k_P$ gives the usual ensemble-zero-mean formulation and the same likelihood derivation.

Let $K_{ij}=k_0(t_i,t_j)$ and $R=\mathrm{diag}({\sigma }_1^2,\dots ,{\sigma }_N^2)$. The prior latent vector is $f\sim N(0,K)$ and the independent Gaussian noise is $\epsilon \sim N(0,R)$, so $y=f+\epsilon \sim N(0,V)$ with $V=K+R$. The Gaussian-process marginal likelihood isPositive measurement variances make $V$ positive definite even if $K$ is singular. Maximize $\ell (\mathcal{H})=-\frac{1}{2}{y}^T{V}^{-1}y-\frac{1}{2}\log |V|-\frac{N}{2}\log (2\pi )$ over all three hyperparameters, using a broad search in $P$ to find competing aliases, followed by joint optimization. Cholesky decomposition evaluates the likelihood without explicitly forming an inverse.

For a well-resolved interior maximum, let $J=-{\nabla }^2\ell (\hat{\mathcal{H}})$ be the Observed Fisher information in the coordinates $(A,\ell ,P)$. Then $\boxed{\hat{P}=(\hat{\mathcal{H}}{)}_P,{\sigma }_P\approx \sqrt{(J^{-1}{)}_{PP}}}$. Equivalently, the profile-likelihood curvature in $P$ gives the same local nuisance-adjusted variance; keeping the other hyperparameters fixed would generally underestimate it. A profile interval with ${\ell }_{\mathrm{prof}}(\hat{P})-{\ell }_{\mathrm{prof}}(P)=1/2$ is a local $1\sigma$ approximation. If the period likelihood is multimodal or poorly constrained, fit the complete posterior with proper hyperparameter priors and report its modes and marginal credible intervals rather than a single Hessian error bar. The sampling pattern can create period aliases that a single local search misses.

For a finite sample, the Gram matrix of $k_1{k}_2$ is the entrywise product of the two positive-semidefinite Gram matrices. The Schur product theorem makes it positive semidefinite, proving the product closure.

The Gaussian kernel with bandwidth ${\sigma }^2$ isFactor it asThe linear kernel $x^T{x}^{\prime }$ is positive semidefinite; products and nonnegative scalar multiples preserve positivity, as does multiplication by $a(x)a(x^{\prime })$. The partial sums are therefore kernels, and pointwise-limit closure proves that the Gaussian kernel is positive semidefinite.

Regard each centered random variable $g(x)$ as a vector $h_x$ in the Hilbert space $L^2(\mathbb{P})$. Thenso $k$ is the Gaussian kernel on the finite subset $\{h_x:x\in \mathcal{X}\}$ of that Hilbert space. More explicitly,Every power of the inner-product kernel is positive semidefinite, and the closure property of positive-semidefinite kernels under nonnegative sums, pointwise limits, and multiplication by one-variable factors proves that $k$ is positive definite.

The Gamma integral $u^{-1/2}={\pi }^{-1/2}{\int }_0^{\infty }{t}^{-1/2}{e}^{-ut}dt$ givesFor each $t>0$, the last factor is a Gaussian kernel, and the remaining weight is nonnegative. The diagonal integral equals ${\alpha }^{-1/2}<\infty$, so part ii shows that the displayed function is a positive-semidefinite kernel.

The Gaussian kernel with bandwidth ${\sigma }^2>0$ is

Suppose a feature map $\varphi :{\mathbb{R}}^d\to {\mathbb{R}}^p$ represented the Gaussian kernel. Choose $n>p$, a unit vector $e$, and points $x_i=iae$ for $i=0,\dots ,n-1$. Their kernel matrix isFor sufficiently large $a$, hence sufficiently small $r$, every row satisfiesThus $K$ is a symmetric strictly diagonally dominant matrix with positive diagonal and is therefore a positive-definite matrix, so $\mathrm{rank}K=n$.

On the other hand, if $\Phi$ is the $n\times p$ matrix whose $i$th row is $\varphi (x_i{)}^T$, then $K=\Phi {\Phi }^T$ and $\mathrm{rank}K\le p<n$, a contradiction. Hence every feature-space realization of the Gaussian kernel requires an infinite-dimensional vector space.

This covariance function is periodic with period $P>0$. Map time to the circle $u(t)=(\cos (2\pi t/P),\sin (2\pi t/P))$ and pull back a Gaussian kernel; since $\Vert u(t)-u(t^{\prime }){\Vert }^2=4{\sin }^2(\pi (t-t^{\prime })/P)$, this proves positive semidefiniteness. A Gaussian process with this kernel has $\mathrm{Var}(f(t+P)-f(t))=0$ and therefore repeats at every fixed phase. Replacing the inner $\pi$ by $2\pi$ changes the actual period to $P/2$.