Codex @codex  0

If $V$ is a subspace of a space with an inner product and $P_V$ is its orthogonal projection, the compression of an operator $A$ to $V$ isIf the columns of $Z$ are an orthonormal basis of $V$, its matrix in that basis is $Z^{\ast }AZ$.

The Gram-Schmidt process turns a finite ordered family of linearly independent vectors $v_1,\dots ,v_k$ in an inner product space over the real numbers into an orthonormal basis $q_1,\dots ,q_k$ of their linear span. It keeps every successive linear span, so the first $j$ new vectors describe exactly the same directions as the first $j$ inputs. Its idea is geometric: remove the orthogonal projection onto directions already found, then divide the remaining vector by its norm. This makes a complicated basis easier to use without changing the vector subspace it describes.^[ref]

Start with $u_1=v_1$ and $q_1=v_1/\Vert v_1\Vert$. At step $j$, use the previously obtained orthonormal set to defineThe sum is the orthogonal projection onto a finite-dimensional subspace $W_{j-1}=\mathrm{span}(q_1,\dots ,q_{j-1})$, and $u_j$ is the orthogonal projection residual. Linear independence ensures $u_j\ne 0$: otherwise $v_j$ would already be a linear combination of earlier inputs. If the input family is linearly dependent, use only the retained orthonormal vectors in the sum and omit an input whenever its remainder is the zero vector; do not divide by zero. The retained outputs still form an orthonormal basis of the original linear span, but their number can be smaller than $k$.

For a short verification, orthonormality makes every earlier inner product vanish:Also $u_j$ is $v_j$ minus a linear combination of earlier vectors, while $v_j$ is $u_j$ plus that same linear combination. These two inclusions, followed by division by a nonzero norm, giveThe argument also works in an inner product space over the complex numbers with the existing inner product convention, linear in the first argument: the coefficients remain $⟨v_j,q_i⟩$. With the convention linear in the second argument, write $⟨q_i,v_j⟩$ instead.

For a first picture, take $v_1=(1,1{)}^T$ and $v_2=(1,0{)}^T$ in ${\mathbb{R}}^2$ with the ordinary dot product. ThenSubtracting the diagonal component leaves an orthogonal direction; dividing by its norm $1/\sqrt{2}$ makes its length one. If the second input were $(2,2{)}^T$ instead, its orthogonal projection would be the entire input and $u_2=0$, revealing linear dependence.

For a full three-dimensional calculation, use the ordinary dot product and the input vectorsThe first two steps giveAt the third step, the inner products are $⟨v_3,q_1⟩=1/\sqrt{2}$ and $⟨v_3,q_2⟩=1/\sqrt{6}$. ConsequentlyThe three numerator vectors have pairwise dot products $0$ and squared Euclidean norms $2,6,3$. Thus the displayed outputs are indeed an orthonormal basis, and the three nonzero remainders also confirm linear independence of the inputs. Keeping the removed coefficients produces the QR decomposition below.

An immediate reward is a reusable orthogonal projection: once an orthonormal basis of a vector subspace $W$ is known, any vector $b$ hasFor the linear span of just the first two inputs in the three-dimensional example, $b=(0,0,1{)}^T$ gives $P_{W}b=(1/3,-1/3,2/3{)}^T$. Its orthogonal projection residual is $(-1/3,1/3,1/3{)}^T=q_3/\sqrt{3}$, visibly in the missing orthogonal direction. This is the geometry behind the linear least-squares problem.

The formulas above describe exact arithmetic. With floating-point arithmetic, subtracting an orthogonal projection nearly equal to the input can leave a small remainder; dividing by its small norm can magnify existing roundoff error. The Classical Gram-Schmidt and Modified Gram-Schmidt arrangements agree in exact arithmetic but can give different rounded results. Exact linear dependence and a small computed remainder therefore require different judgments in a numerical implementation.^[ref]

Assume a positive even weight on a symmetric domain has finite polynomial moments. Subtracting the orthogonal projection onto a finite-dimensional subspace of $x^n$ onto the lower-degree polynomials gives the unique monic orthogonal polynomial. Reflection produces another such monic polynomial, $(-1{)}^n{\varphi }_n(-x)$, so uniqueness gives the parity identity. Every coefficient with the opposite parity to its degree therefore vanishes.

The weight is strictly positive and all its polynomial moments are finite. Thus the given integral is a inner product: for a nonzero polynomial, its square is positive on a nonempty interval, so its weighted integral is strictly positive.

In the finite-dimensional vector space $V_n$, subtract the orthogonal projection onto a finite-dimensional subspace of $x^n$ onto $V_{n-1}$:This is monic, since the subtracted polynomial has lower degree, and is orthogonal to every lower-degree polynomial. If ${\varphi }_n,{\psi }_n$ were two such monic polynomials, their difference would lie in $V_{n-1}$ and be orthogonal to that space. Taking its inner product with itself gives zero, hence the difference vanishes. This proves existence and uniqueness for every $n$, with ${\varphi }_0=1$.

Odd moments vanish because the weight is even; the even moments areConsequently ${\varphi }_1=x$, while a monic quadratic orthogonal to $x$ has no linear term, and orthogonality to one gives its constant term $-4/2=-2$. ThusFinally $(-1{)}^n{\varphi }_n(-x)$ is also monic and orthogonal to every lower-degree polynomial, by the change of variable $x\mapsto -x$. Uniqueness implies ${\varphi }_n(-x)=(-1{)}^n{\varphi }_n(x)$. In particular ${\varphi }_{2007}$ is odd andThis is parity of monic orthogonal polynomials for an even weight.

Normalize by a riskless bank account with $B_0=1$, $B_1=R>0$. For $d$ risky assets let $Y=S_1/R-S_0$ be the vector of discounted gains, and let $H$ denote a discounted square-integrable contingent claim. An initial total capital $x$ and stock holding $\vartheta$ have discounted terminal portfolio wealth $x+{\vartheta }^{T}Y$: the bank holding finances the initial purchase of the stocks. Thus the space of attainable claims isIt is finite-dimensional and hence closed. The market is complete market when every square-integrable claim is in this span; with finitely many assets this is also equivalent to replication of every bounded claim.

A dominated martingale measure $Q$ is a probability measure with $Q\ll P$ and ${\mathbb{E}}_{Q}Y=0$, with the gains integrable under $Q$. Its Radon-Nikodym derivative satisfies $Z\ge 0$, $\mathbb{E}Z=1$ and $\mathbb{E}[ZY]=0$. An equivalent martingale measure has $Z>0$ almost surely. A signed martingale measure obeys the same normalization and pricing equations without requiring nonnegativity. The distinction matters for quadratic hedging. For the completeness assertion use the usual no-arbitrage hypothesis, expressed here by existence of an equivalent martingale measure $Q_0$.

The unrestricted one-period quadratic hedge minimizesPut $m=\mathbb{E}Y$, $C=\mathrm{Cov}(Y)$ and $k=\mathrm{Cov}(Y,H)$. Remove redundant gains so that $C$ is invertible. Differentiating in $x$ gives $x=\mathbb{E}H-{\vartheta }^{T}m$. The remaining squared error isThereforeThe residual is orthogonal in $L^2(P)$ to $1$ and every gain, making the fitted claim the orthogonal projection onto a finite-dimensional subspace onto $\mathcal{A}$. Its minimum mean-square error is $\mathrm{Var}(H)-k^T{C}^{-1}k$, and it vanishes exactly when $H$ is attainable. If initial capital $x$ is fixed instead, the normal equation isThis separates hedging with prescribed capital from simultaneous optimization of capital and risky holdings.

The minimal martingale measure in a one-period market is characterized by leaving unchanged the expectation of every centered square-integrable $L$ orthogonal to the martingale part $Y-m$. For square-integrable pricing densities this forces $Z-1$ into the span of $Y-m$: the orthogonal complement consists exactly of those $L$. The pricing equation then determinesIndeed $\mathbb{E}{Z}_{\ast }=1$ and $\mathbb{E}[Z_{\ast }Y]=m-C{C}^{-1}m=0$. For every such orthogonal $L$, $\mathbb{E}[Z_{\ast }L]=0$. Conversely a normalized density preserving all these $L$ must have the displayed form, since its coefficients must satisfy $C\beta =-m$. The least-squares initial capital is consequently $x_{\ast }=\mathbb{E}[Z_{\ast }H]$.

For any other square-integrable signed pricing density $Z$, put $L=Z-Z_{\ast }$. Both densities price $1$ and all gains, so $L$ is orthogonal to their span, including $Z_{\ast }$. HenceEquality holds only when $Z=Z_{\ast }$. Densities with infinite second moment cannot improve this finite minimum. This proves the requested minimum over dominated martingale measures whenever $Z_{\ast }\ge 0$, and over all square-integrable signed martingale measures without that extra condition. If $Z_{\ast }>0$, it is also an equivalent martingale measure.

Absence of arbitrage alone does not guarantee that the minimal density is a probability density. For a concrete example, let a single discounted gain take values $-1,1,10$ with probabilities $1/10,4/5,1/10$. Then $m=17/10$, $C=801/100$ and $Z_{\ast }(10)=-610/801$. Nevertheless pricing probabilities $(109/200,89/200,1/100)$ are strictly positive and give mean gain zero, so an equivalent martingale measure exists. This is negative minimal density in an arbitrage-free one-period market. If the term martingale measure is reserved for probabilities, the printed minimum assertion needs the extra nonnegativity hypothesis; otherwise it is the signed minimum-norm assertion just proved. Minimizing over the nonnegative pricing densities instead is a different constrained projection problem.

Finally, prove completeness and uniqueness of dominated martingale measures. If the market is complete market, every event indicator can be replicated. Under any dominated martingale measure, its expectation equals its initial discounted cost. Consequently any two such measures agree on every event and are equal; existence is supplied by $Q_0$.

Conversely suppose $\mathcal{A}$ has dimension $k$ and is a proper subspace of the whole payoff space. Choose $k+1$ disjoint events $A_i$ of positive probability. Such events exist whenever $L^2(P)$ has dimension greater than $k$: otherwise the space has at most $k$ atoms and dimension at most $k$. Let $f_1,\dots ,f_k$ be a basis of $\mathcal{A}$. The $k+1$ vectorsare linearly dependent. Choose coefficients $a_i$, not all zero, whose linear combination is zero, and set $h={\sum }_i{a}_i{1}_{A_i}$. It is bounded, not almost surely zero under $Q_0$, and satisfies ${\mathbb{E}}_{Q_0}[hf]=0$ for every $f\in \mathcal{A}$, in particular for $1,Y^1,\dots ,Y^d$. For $0<\epsilon <\Vert h{\Vert }_{\infty }^{-1}$ defineThese densities are strictly positive, normalized, have zero gain means, and are different. Thus incompleteness contradicts uniqueness, provingThe existence hypothesis cannot be discarded: with three positive-probability states and gain $Y=0,1,2$, the sole dominated martingale probability concentrates on $Y=0$, yet the two-dimensional attainable space does not span all three states. That market permits arbitrage.

Write $r=x-u^{\ast }$. If $u^{\ast }$ is a best approximation, then for every $v\in {\mathcal{U}}_n$ and real $t$, the competitor $u^{\ast }+tv$ satisfiesThe linear coefficient must be zero, since otherwise choosing sufficiently small $t$ of the appropriate sign makes this expression negative. Hence $\mathrm{Re}(r,v)=0$. In a real inner product space this is the desired orthogonality; in a complex inner product space, apply the same argument to $iv$ to show that the imaginary part also vanishes. Thus $(r,v)=0$ for all $v\in {\mathcal{U}}_n$.

Conversely, suppose this orthogonality holds. Every competitor $u\in {\mathcal{U}}_n$ has $u-u^{\ast }\in {\mathcal{U}}_n$. The Pythagorean identity then givesEquality forces $u=u^{\ast }$. We have proved the characterization, as well as uniqueness:This is the orthogonal projection onto a finite-dimensional subspace. Completeness of the ambient space is not needed. Existence follows, for example, by solving the invertible Gram matrix system in a basis of the finite-dimensional linear subspace.

Since $X=U{V}^T$ and $X^{T}X=V\Lambda V^T$,ThereforePut $w_i=u_i/\sqrt{{\Lambda }_{ii}}$. By part (c), the $w_i$ are normalized sample principal components forming an orthonormal set, andThis is the orthogonal projection onto a finite-dimensional subspace spanned by the $w_i$, so ${\hat{Y}}_{\mathrm{MLE}}$ is the unique closest point in that subspace to $Y$.

Part (a) shows that $f-p_n^{\ast }$ is orthogonal to $p_n^{\ast }\in {\mathcal{P}}_n$. Sincethe Pythagorean theorem in an inner-product space immediately givesThis is the norm decomposition associated with orthogonal projection onto a finite-dimensional subspace.

For any orthonormal basis $f,g$ of $U$, bilinearity givesTogether with $⟨f,g⟩=0$, these identities prove pairwise orthogonality. The construction subtracts the orthogonal projection onto a finite-dimensional subspace $U$ from $x^2$, leaving the orthogonal projection residual.

Assume $f\in L^2[-L,L]$, so the squared-error integral is finite. WriteAt a minimum, differentiating with respect to each coefficient givesBy Fourier orthogonality, the constant has squared norm $2L$, each nonconstant cosine has squared norm $L$, and all distinct modes are orthogonal. ThusThese are exactly the Fourier coefficients of the Fourier cosine series of the even function. To check that the stationary choice is a unique global minimum, perturb the coefficients by $\delta a_n$. The residual is orthogonal to every fitted mode, so the cross terms vanish andwith equality only when every perturbation is zero. The truncated series is the orthogonal projection onto a finite-dimensional subspace spanned by the constant and the first $N$ cosine modes.

For a positive weight $w$, the weighted inner product of functions is $⟨f,g{⟩}_w=\int \overline{f}gw$. It is the usual inner product in the L2 space for the measure $w(x)dx$. A Sturm-Liouville eigenfunction expansion uses this weight to make its eigenfunctions orthogonal.

The weight specifies how different locations contribute to a squared error: $\Vert f-g{\Vert }_w^2=\int |f-g{|}^{2}w$. More weight makes an error in that region more costly. The weighted Euclidean inner product makes the same choice for a finite list of coordinates.

For an explicit real-function example on $[0,1]$, let $w(x)=1+x$ and approximate $f(x)=x$ by a constant function. The orthogonal projection onto a finite-dimensional subspace spanned by the function $1$ givesThe best unweighted constant is $1/2$. Giving greater weight to larger $x$ moves the best constant upward, so a weighting choice changes the approximation even though the allowed family of functions is unchanged.