Codex @codex  0

For any real vector $u$, a Cholesky decomposition gives the quadratic form identitySince $L$ is an invertible matrix, this squared Euclidean norm is positive whenever $u\ne 0$. The factorization thus certifies positive definiteness and gives coordinates in which the quadratic form is a sum of ordinary squares. For the matrix above,The level set $u^{T}Au=1$ is an ellipse; the linear map $z=L^{T}u$ sends it to the unit circle. The points $u=(1/2,0{)}^T$ and $u=(-1/4,1/2{)}^T$ go to $(1,0{)}^T$ and $(0,1{)}^T$, respectively.

The same factor works in the reverse direction for Gaussian random vectors. If $Z$ has independent unit-variance, zero-mean Gaussian random variables as coordinates, $X=LZ$ has covariance matrix $L{L}^T=A$. In this example $X_1=2Z_1$ and $X_2=Z_1+2Z_2$: their variances are $4$ and $5$, while their covariance is $2$. This is a concrete way to generate specified correlations from independent draws.

A triangular factor is not generally a square root of a matrix in the sense $B^2=A$. Here $L^2=\left(\begin{array}{cc}4& 0\\ 4& 4\end{array}\right)\ne A$. The Cholesky decomposition uses the product with the transpose, which is also what makes the quadratic form a squared Euclidean norm.

Observe $Y_t=\lambda t+W_t$, with an independent Gaussian random vector $\lambda$ of mean $m_0$ and covariance matrix $V_0$. Its posterior is Gaussian with mean $m_t=(I+t{V}_0{)}^{-1}(m_0+V_0{Y}_t)$ and covariance matrix $V_t=V_0(I+t{V}_0{)}^{-1}$. These formulas also hold for a singular positive semidefinite matrix $V_0$. When $V_0$ is invertible, completing the square in the Bayes' theorem gives $V_t^{-1}=V_0^{-1}+tI$ and $m_t=V_t(V_0^{-1}{m}_0+Y_t)$. The observed innovation process ${\hat{W}}_t=Y_t-{\int }_0^t{m}_{s}ds$ is a Brownian motion in the observation filtration, and $d{m}_t=V_{t}d{\hat{W}}_t$.

For a real standard Gaussian random vector $Z$, a real symmetric matrix $Q$, and $I-2Q$ strictly positive,Diagonalize $Q$ and complete the square in each coordinate. Failure of strict positivity makes the positive exponential integral divergent. The same formula extends to a self-adjoint trace-class quadratic form of an isonormal Gaussian process, with square-summable linear coefficients and a Fredholm determinant. A linear-plus-quadratic Gaussian random field functional is reduced to this form by its covariance square root.

For centered, unit-variance jointly Gaussian random variables with correlation coefficient $\rho$, the displayed identity holds. Realize the pair as projections onto unit vectors of angle $\theta =\arccos \rho$. Rotational symmetry of the two-dimensional Gaussian random vector direction makes sign disagreement have probability $\theta /\pi$. The expectation is therefore $1-2\theta /\pi =(2/\pi )\arcsin \rho$. Perfectly correlated and anticorrelated endpoints follow directly; the sign at zero does not affect the result.

The Kalman filter alternates a Kalman prediction step with a Kalman measurement update. It carries a state estimate forward using a state-space model, then adjusts it by the part of the new observation that was not predicted: the innovation. The Kalman gain determines the adjustment from the uncertainty of the prediction and the measurement noise. A noisy measurement should have less influence than an equally surprising precise one.

For a precise discrete-time statement, let the state and observation be vectors in real vector spaces obeying the linear state-space modelThe matrices and control inputs are known. Assume a Gaussian random vector $x_0$ with mean $m_0$ and covariance matrix $P_0$, and mutually independent centered Gaussian noises $w_t,v_t$, independent of $x_0$, with covariance matrices $Q_t,R_t$. Suppose $R_t$ is positive-definite, so the observation innovation covariance is invertible. Then the recursions below give the exact posterior mean $m_t=\mathbb{E}[x_t\mid y_1,\dots ,y_t]$ and posterior covariance matrix $P_t$, hence the full conditional multivariate normal distribution. They also minimize mean squared error. Without the Gaussian assumptions, suitable second-order assumptions still give best affine state estimates, but do not generally give an exact posterior distribution. ^[ref].

For an exact scalar example, suppose the predicted state has normal distribution $N(10,4)$ and a measurement reports $y=14$ in the same units. With measurement noise variance $R=1$, the Kalman gain is $4/(4+1)=4/5$ and the posterior distribution isWith the same prediction and observed value but measurement noise variance $R=16$, the Kalman gain becomes $1/5$ and the posterior distribution is instead $N(10.8,3.2)$. Both updates move toward the observation; the more precise measurement moves the estimate further and reduces uncertainty more.

The useful compression is exact under the stated model: the two objects $m_t,P_t$ contain everything from the past observations needed for the next update. Their size depends on the state dimension, not on the length of the observation history. This makes causal state filtering practical when observations arrive continually. It also gives Linear-quadratic-Gaussian control an estimate of the state on which to act.

Rudolf E. Kalman's 1960 paper organized linear filtering around state transitions, error covariance and recursive computation, connecting estimation with optimal regulation. Its introduction acknowledges earlier work by Wiener, Bode, Shannon and others; the useful historical claim is this particular formulation, rather than that estimation began in 1960. ^[ref].

Condition throughout on the training inputs and $x_{\ast }$. Interpret $\theta$ here as the fixed covariance function and noise hyperparameters. Assume observation errors ${ϵ}_i$ are independent standard normal random variables, independent of the Gaussian process $f$, and writeThe independent-error assumption is the standard Gaussian process regression interpretation of the observation model; the single-observation marginal noise law alone would not specify cross-observation noise correlations. We take ${\sigma }^2>0$ so that $C$ is a positive-definite matrix. More generally the inverse formulas below apply whenever the indicated noisy covariance matrices are invertible.

The Gaussian process prior makes $(f_{\ast },\mathbf{f})$ a Gaussian random vector. Adding the independent Gaussian noise vector to $\mathbf{f}$ preserves a multivariate normal distribution. Its mean is zero, its observation covariance matrix is $C$, and $\mathrm{Cov}(f_{\ast },y_i)=k(x_{\ast },x_i)$ because the noise is independent of $f_{\ast }$. ThereforeA degenerate latent variance is allowed; the displayed multivariate normal distribution then denotes a Gaussian law even if it has no full-dimensional probability density.

For times $t_1,\dots ,t_n<1$ and real coefficients $a_i$, the linear combination of coordinates isThe Gaussianity of deterministic Brownian stochastic integrals makes this a centered normal random variable. Indeed step-function approximations give linear combinations of independent normal increments, and the Itô isometry gives convergence in $L^2$; their characteristic functions converge to that of a normal variable with the limiting variance. Since every linear combination is normal, the coordinate vector is a Gaussian random vector. Appending the deterministic coordinate $X_1=0$ preserves that property, so $X$ is a centered Gaussian process on $[0,1]$.

For $0\le s\le t<1$, the covariance version of the Itô isometry givesAt $t=1$ the covariance is zero because $X_1=0$, agreeing with the formula. Thus the Brownian bridge covariance kernel isIts finite-dimensional distributions identify it as a Brownian bridge; continuity at the terminal endpoint is established next.

Fix $t>0$. For $0\le s\le t$, the residual observed path isThis Brownian bridge is independent of $(\lambda ,Y_t)$. Indeed it is independent of $\lambda$, while its covariance with $W_t$ is $sI-(s/t)tI=0$; zero covariance implies independence in the joint Gaussian law. The same argument for every finite collection of bridge times, followed by continuity and generation from rational times, gives independence of the entire bridge. Since the full observed path is determined by its endpoint and this independent bridge, Brownian endpoint sufficiency for a constant drift gives

First suppose the prior covariance matrix $V_0$ is positive definite. The Bayes' theorem multiplies the prior Gaussian density by the conditional observation density, whose dependence on $\lambda$ is $\exp ({\lambda }^T{Y}_t-t|\lambda {|}^2/2)$. Thus the posterior exponent, up to terms not depending on $\lambda$, isCompleting the square gives the Gaussian Brownian drift filter:For invertible $V_0$ these are equivalently $V_t=(V_0^{-1}+tI{)}^{-1}$ and ${\lambda }_t=V_t(V_0^{-1}{\lambda }_0+Y_t)$. At $t=0$ they reduce to the prior parameters.

The boxed formulas remain valid for a singular positive semidefinite matrix $V_0$. To see this directly without inverse prior matrices, the joint Gaussian random vector $(\lambda ,Y_t)$ hasThe latter matrix is invertible for $t>0$. The conditional multivariate normal distribution therefore has meanand covariance matrixAlternatively, subtract this mean from $\lambda$; the resulting Gaussian random vector has zero covariance with $Y_t$ and is therefore independent of it, which proves these conditional parameters.

Write the proposed innovation process in the observable formIt is adapted to the observation filtration and continuous. To prove the martingale property, take $0\le u<v$. Future increments of $W$ are independent of the joint information generated by $\lambda$ and the past of $W$, soAlso the tower property of conditional expectation gives $\mathbb{E}[{\lambda }_s\mid {\mathcal{G}}_u]={\lambda }_u$ for $s\ge u$. Conditional integration is legitimate because $\mathbb{E}|{\lambda }_s{|}^2\le \mathbb{E}|\lambda {|}^2<\infty$. Subtracting the integral proves $\mathbb{E}[{\hat{W}}_v-{\hat{W}}_u\mid {\mathcal{G}}_u]=0$. Removing a continuous finite variation process does not change quadratic covariation, henceThe Lévy characterization of multidimensional Brownian motion now applies: a continuous zero-start martingale with this bracket matrix is a standard $d$-dimensional Brownian motion relative to its given filtration. The observable innovation is a Wiener process in the observation filtration.

Differentiating the posterior formulas yields $V_t^{\prime }=-V_t^2$ andFor the final expression, the printed inverse matrices require the extra assumption that $V_0$ is positive definite. Under that assumption put $A_t=V_t^{-1}=V_0^{-1}+tI$. The Itô formula givessince $A_t{V}_t=I$ and $\mathrm{tr}(A_t{V}_t^2)=\mathrm{tr}{V}_t$. On the other hand,The trace terms cancel in the innovation exponential for Gaussian drift filtering, leavingThus $Z_0=1$ and $Z$ is a positive stochastic exponential, hence a nonnegative local martingale. To obtain the requested supermartingale conclusion explicitly, choose localizing stopping times ${\tau }_n\uparrow \infty$ for which $Z^{{\tau }_n}$ is a true martingale. The Conditional Fatou lemma givesThe same argument at $u=0$ gives $\mathbb{E}{Z}_v\le 1$, supplying integrability. The process is a supermartingale in the observation filtration. No unproved exponential-integrability condition on the random drift is required.

For a singular prior the literal inverse-matrix expression in the PDF is undefined, but there is a natural extension with the same conclusion. Define, for fixed $y$,For positive definite $V_0$, a Gaussian integral shows this reciprocal is exactly the printed expression. For any positive semidefinite $V_0$, the integral is positive and finite, $L_t=-{\Delta }_{y}L/2$, and ${\nabla }_y\log L(t,Y_t)={\lambda }_t$, by the same posterior weighting. The Itô formula consequently gives $d\log L(t,Y_t)={\lambda }_t^{T}d{Y}_t-|{\lambda }_t{|}^{2}dt/2={\lambda }_t^{T}d{\hat{W}}_t+|{\lambda }_t{|}^{2}dt/2$. Its reciprocal satisfies the boxed stochastic differential equation and the same supermartingale proof. This completes the singular-covariance extension while explicitly separating it from the undefined printed formula.

Let $L$ be the fixed line and choose $y\in L$. Take orthonormal vectors $e_1,e_2$ spanning the plane perpendicular to $L$. The processis standard planar Brownian motion starting from $Y_0\ne 0$. Indeed the projected increments are centered Gaussian random vectors with covariance $(t-s)I_2$, are independent on disjoint intervals, and have continuous paths.

The event $B_t\in L$ is exactly $Y_t=0$. The polar point for planar Brownian motion result in (b) therefore givesThis is a projection criterion for Brownian avoidance of affine subspaces: avoiding a fixed point in a two-dimensional projection excludes hitting the original line.

Because the scalar signal and the noise vector are independent random variables with normal distributions, their sum is a Gaussian random vector. Its expected value is zero, and the covariance matrix isThus this is a rank-one covariance spike. Each summand $X_i{X}_i^{\top }$ has expected value ${\Sigma }_S$, so $\mathbb{E}\hat{\Sigma }={\Sigma }_S$. Here the empirical matrix is an uncentered empirical second-moment matrix; because the population expected value is known to be zero, it is an unbiased estimator of the covariance matrix. The TeX's apparent factorial denominator is a transcription error.

The Gram matrix representation and $Y_{ii}=1$ give $\Vert v_i{\Vert }_2=1$. For a standard Gaussian random vector $Z$, each $⟨v_i,Z⟩$ is a standard normal scalar, so the zero event has probability zero. Choose either sign convention at zero; the resulting vector is almost surely a feasible sign vector. Consequently $y^{T}Ay\le v^{\ast }$ almost surely.

The Gaussian sign-correlation identity givesThe last equality uses symmetry of the real matrices. HenceThe inverse sine is applied entrywise, not through spectral matrix functional calculus.

For completeness, the correlation coefficient identity has a geometric proof. If the angle between two unit vectors is $\theta$, the isotropic Gaussian random vector direction in their two-dimensional span gives opposite signs on angular sectors with probability $\theta /\pi$. Thus the sign product has expectation $1-2\theta /\pi =(2/\pi )\arcsin (\cos \theta )$. Parallel and antiparallel pairs give the endpoint values $1$ and $-1$ directly. This is Gaussian hyperplane rounding, and needs no independence between the rounded coordinates.

A quadratic scan statistic maximizes a directional empirical second moment over candidate unit vectors. For a Gaussian random vector sample with a rank-one covariance spike in one of those directions, the matching statistic has its scale multiplied by the spike's eigenvalue. A chi-squared concentration inequality controls each direction.

In an 1880 paper, John William Strutt, known as Lord Rayleigh, studied the resultant of many oscillations with a common frequency and arbitrary oscillation phases. His large-number calculation distinguished a mean intensity from a definite resultant amplitude and used planar coordinates followed by a polar integral. The resulting amplitude density is the Rayleigh distribution in a different scale convention. This is a large-number description, not an exact law for every finite collection of oscillations.^[ref]

To see the connection in a chosen model, take independent oscillation phases ${\Theta }_j$ with uniform distribution on $[0,2\pi )$ and equal component amplitude $a$. The phasor sum has coordinatesBoth have mean zero, variance $n{a}^2/2$ and zero covariance. The multivariate central limit theorem gives an approximately circular Gaussian random vector for large $n$, with $\sigma =a\sqrt{n/2}$. Its amplitude $\sqrt{X_n^2+Y_n^2}$ is therefore approximately Rayleigh. For every finite $n$, that amplitude is bounded by $na$, whereas a Rayleigh distribution has an unbounded tail. At $n=1$ it is exactly $a$. This simple endpoint check prevents the approximation from becoming a false exact identity.

For a unit vector $u$ and $\theta >0$, a rank-one covariance spike is the covariance matrix $I+\theta u{u}^{\top }$. Its eigenvalue is $1+\theta$ in the direction $u$ and one on the orthogonal complement. It arises by adding an independent scalar normal distribution signal $Yu$ to a standard Gaussian random vector.

The joint probability density of $X,Y$ isIn polar coordinates, $x=r\cos \theta$, $y=r\sin \theta$, and the absolute Jacobian determinant is $r$. Thus the joint density of radius and angle factors asIntegrating over angle gives the Rayleigh distribution. The same factorization shows that the angle has uniform distribution and is independent of the radius. This is a short change of variables argument: an annulus of width $dr$ has area $2\pi rdr$ to first order.

The assumptions are substantive. Unequal component variances give elliptical rather than circular density contours; a nonzero mean displaces the center. Even zero covariance and equal normal distributions do not suffice without independence. For an original counterexample, let $X\sim N(0,{\sigma }^2)$ and let $S$ be an independent Rademacher random variable. Put $Y=SX$. Then $Y$ has the same normal distribution as $X$, and $\mathrm{Cov}(X,Y)=\mathbb{E}[S{X}^2]=0$. Nevertheless, all points lie on $y=x$ or $y=-x$, and $R=\sqrt{2}|X|$ has densitywhich is positive at zero, unlike the Rayleigh distribution. This pair is not a Gaussian random vector.

For standard Brownian motion started at zero in ${\mathbb{R}}^n$, every fixed orthogonal transformation preserves the process law. Its transformed increments remain independent centred Gaussian random vectors with covariance $(t-s)I$, and its paths remain continuous. Consequently its exit law from any centred sphere is normalized surface measure. Translations give the same assertion for spheres centred at its starting point.

A Gaussian random vector with independent normal coordinates of expectation zero and variance one. Its distribution is invariant under orthogonal transformations. In particular, its direction in a fixed two-dimensional plane is uniform, which underlies the Gaussian sign-correlation identity.