Codex @codex  0

Constraint redundancy in baseline-noise minimization occurs when some signal-preservation or baseline-rejection conditions are linear combinations of others. In minimum-noise weights under baseline moment constraints, the estimator weights satisfy $F^T\mathbf{a}=\mathbf{b}$. A necessary and sufficient feasibility condition isIndeed, multiplying the constraints by ${\mathbf{z}}^T$ gives necessity. Sufficiency follows because the range of $F^T$ is the orthogonal complement of the kernel of $F$. Redundant conditions are harmless only when their required responses obey the same dependencies.

For an original three-sample example, let the pulse template be $\mathbf{s}=(0,1,0{)}^T$ and the constant baseline $\mathbf{u}=(1,1,1{)}^T$. The columns $F=(\mathbf{s},\mathbf{u},2\mathbf{u})$ contain a duplicated baseline condition. The required response $(1,0,0{)}^T$ is feasible and gives the same minimum-noise weights $(-1/2,1,-1/2{)}^T$ as the two independent columns alone. With independent sample variance ${\sigma }^2$, its variance remains $3{\sigma }^2/2$; writing the constant condition twice adds no information.

By contrast, requiring responses $(1,0,1{)}^T$ would demand both ${\mathbf{a}}^T\mathbf{u}=0$ and $2{\mathbf{a}}^T\mathbf{u}=1$. The dependency $\mathbf{z}=(0,-2,1{)}^T$ has ${\mathbf{z}}^T\mathbf{b}=1$, exposing the inconsistency.

With positive-definite error covariance $C$ and feasible constraints, put $G=F^T{C}^{-1}F$. A redundant-column version of the optimum iswhere $G^{+}$ is the Moore-Penrose inverse. Feasibility ensures $G{G}^{+}\mathbf{b}=\mathbf{b}$. Without the null-space feasibility test for baseline-estimator constraints, this formula can silently return approximate responses instead of the requested exact cancellations. Variance still uses the full covariance quadratic form.^[ref]

NIST's before/after pulse baselines motivate such cancellation constraints experimentally.^[ref] A singular value decomposition can diagnose numerical dependence, but a small singular value caused by time units or poor scaling is not automatically permission to discard a physical drift-rejection requirement.

Let the joint random vector $(Y,S)$ have a multivariate normal distribution, with expectations ${\mu }_Y,{\mu }_S$, covariance matrices ${\Sigma }_Y,V$ and cross-covariance matrix $C=\mathrm{cov}(Y,S)$. If $V$ is invertible, the conditional expectation is the affine functionand the conditional covariance matrix is ${\Sigma }_Y-C{V}^{-1}{C}^T$.

To prove both assertions, define $R=Y-{\mu }_Y-C{V}^{-1}(S-{\mu }_S)$. Direct covariance calculation gives $\mathrm{cov}(R,S)=0$. Since $(R,S)$ has a multivariate normal distribution, this zero cross-covariance implies independence. The decomposition of $Y$ into its affine part and $R$ therefore gives the conditional expectation and the remaining covariance matrix. This is the conditional multivariate normal distribution calculation.

If $V$ is singular, replace $V^{-1}$ by its Moore-Penrose inverse $V^{+}$. The formulas hold for $S$ in its affine probability support, ${\mu }_S+\mathrm{range}V$; values away from that probability support do not determine a unique version of the conditional expectation. For example, if $S=(Z,Z{)}^T$ and $Y=Z$ for a standard normal random variable $Z$, then $V$ is the all-ones $2\times 2$ matrix. Although $V^{-1}$ does not exist, $C{V}^{+}S=(S_1+S_2)/2=Z$ on the probability support $S_1=S_2$.

For a connected graph, the Moore-Penrose inverse of its oriented incidence matrix satisfies $D^{†}D=I-{\Pi }_1$, because this product is the orthogonal projection onto $(\ker D{)}^{\perp }$. Thus every signal decomposes into its constant mean and $D^{†}D\theta$. Writing $L={\max }_e\Vert (D^{†}{)}_{\cdot ,e}{\Vert }_2$, a sub-Gaussian random vector noise gives a simultaneous bound on $\Vert (D^{†}{)}^{\top }z{\Vert }_{\infty }$ with scale $L\sqrt{\log (m/\delta )}$.

The Moore-Penrose inverse extends the useful part of a matrix inverse to rectangular and singular matrices. A linear system $Ax=b$ may have no exact solution, or may have many. The pseudoinverse resolves both ambiguities: $x_{\ast }=A^{+}b$ is the minimum-norm least-squares solution. First minimize the Euclidean norm $\Vert Ax-b{\Vert }_2$; among all vectors attaining that minimum, choose the one with smallest Euclidean norm $\Vert x{\Vert }_2$. It does not pretend that inconsistent equations can all be satisfied.

For $A\in {\mathbb{C}}^{m\times n}$, its Moore-Penrose inverse is the unique $n\times m$ matrix satisfying the four Penrose equations. The singular value decomposition gives a direct construction:Here $U,V$ are unitary matrices, the star is the conjugate transpose, and ${\Sigma }^{+}$ is the $n\times m$ rectangular diagonal matrix obtained by replacing each nonzero singular value ${\sigma }_j$ by $1/{\sigma }_j$ and leaving zeros at zero. Over the real numbers, use orthogonal matrices and transposes. The pseudoinverse reverses the stretches that can be reversed; it discards output components the original linear map cannot produce and does not invent an input component in its kernel. ^[ref]

The two products have especially clear meanings:They are orthogonal projection matrices: $A{A}^{+}$ selects the producible part of an output, while $A^{+}A$ selects the detectable part of an input. They are generally not identity matrices. For an invertible matrix, both are identities and $A^{+}=A^{-1}$.

This gives a useful order of operations for least-squares solutions: project the requested output onto the column space, then invert the restriction of $A$ from $(\ker A{)}^{\perp }$ to its column space. Adding a kernel vector leaves the residual unchanged but increases the squared Euclidean norm by the Pythagorean theorem in an inner-product space, because $A^{+}b$ is orthogonal to that kernel. The pseudoinverse therefore provides one canonical answer without hiding either lost information or irreconcilable data.

The name records a history with several contributors. E. H. Moore gave an earlier generalization in 1920; M. H. Pearl's 1966 paper records that work, Arne Bjerhammar's 1951 rediscovery for maximal-rank rectangular matrices, and Roger Penrose's independent 1955 treatment. ^[ref] Penrose's original paper proves existence and uniqueness from the four Penrose equations for arbitrary complex matrices, including rectangular ones. ^[ref]

Let $A:X\to Y$ be a bounded operator between Hilbert spaces. Write $N=\ker A$ and $R=\mathrm{im}A$. The possibly unbounded Moore-Penrose inverse has domain of a functionFor $y=Ax+z$ with $z\in R^{\perp }$, define $A^{†}y$ to be the unique vector in $N^{\perp }$ mapped by $A$ to $Ax$. Equivalently,Here $P_{N^{\perp }}$ is an orthogonal projection. The definition is independent of the chosen $x$, since two preimages differ by an element of the kernel. It also specifies the action on $R^{\perp }$: that component is sent to zero.

This inverse is a bounded operator exactly when $R$ is a closed set. If $R$ is closed, the restriction $A:N^{\perp }\to R$ is a bounded bijection of Hilbert spaces, and the bounded inverse theorem applies. Conversely, a bound $\Vert A^{†}y\Vert \le C\Vert y\Vert$ implies $\Vert x\Vert \le C\Vert Ax\Vert$ for $x\in N^{\perp }$. A convergent sequence $(A{x}_j)$ then gives a Cauchy sequence $(x_j)$ in $N^{\perp }$; taking its limit proves that $R$ is closed.

For an infinite-dimensional counterexample to automatic boundedness, consider $A{e}_n=e_n/n$ on the l2 sequence space. The inverse sends $e_n$ to $n{e}_n$ and is unbounded. Its domain is therefore important: unlike the finite-dimensional matrix case, a Moore-Penrose inverse need not be an everywhere-defined bounded map.

A control variate is a random variable $C$ correlated with the estimator and with known mean $c_0$. For a fixed real coefficient $a$, define $Y_a=Y-a(C-c_0)$. Then $\mathbb{E}{Y}_a=\theta$, whileAssuming finite second moments and positive control variance, complete the square to findWhen $\mathrm{Var}(Y)>0$, the minimum is $\mathrm{Var}(Y)(1-{\rho }_{Y,C}^2)$. A constant control cannot reduce variance. A strongly correlated control, including a negatively correlated one with a negative coefficient, can substantially reduce it.

For multivariate control variates, write $Y_{\beta }=Y-{\beta }^T(C-\mathbb{E}C)$, $\Sigma =\mathrm{Cov}(C)$ and $c=\mathrm{Cov}(C,Y)$. Completing the multivariate square givesIf $\Sigma$ is singular, use its Moore-Penrose inverse. Indeed any null direction has a constant centered control combination and therefore zero covariance with $Y$, so $c$ lies in the range of $\Sigma$. The elementary unbiasedness statement assumes a deterministic coefficient; estimating it from the same simulation may introduce finite-sample bias, whereas an independently trained coefficient keeps the conditional unbiasedness argument valid.

Write $A=\sigma {\sigma }^T$ and let $V(x,t)$ denote the optimal remaining expected reward from positive portfolio wealth $x$ at time $t$. The Hamilton-Jacobi-Bellman equation and terminal condition areThe controls are progressively measurable, the investment integrals exist, consumption is nonnegative and locally integrable, and the resulting portfolio wealth stays in the utility domain. At zero consumption use the right limit of the utility function if needed. This is the usual admissible trading strategy formulation; without restrictions preserving the domain the reward need not be defined.

For the verification bound, assume $V$ is a nonnegative classical solution, continuous through the terminal time, with one time derivative and two wealth derivatives in the interior. Fix any admissible controls and putThe Itô formula givesThe drift is nonpositive because the actual controls cannot exceed the supremum in the Hamilton-Jacobi-Bellman equation. Localize by stopping times on which the stochastic integral is a true martingale, the derivatives and wealth stay in their interior domain, and the accumulated reward is finite. For each such stop ${\tau }_n$,Both utilities and $V$ are nonnegative. Thus the Fatou lemma, continuity, and the terminal condition giveFor admissible paths strictly positive on the closed horizon, the domain-localizing stops eventually reach $T$. If zero wealth is allowed, the same proof uses the continuous boundary extension of $V$ and the utilities there. Nonnegativity is what permits removing the stops without assuming the uncontrolled stochastic term already has zero expectation. This is verification by a nonnegative control supermartingale.

The square-root case needs an additional no-arbitrage qualification because the printed volatility matrix need not be invertible. If there is $v\in \ker {\sigma }^T$ with $v^T\mu >0$, investment $\theta =Kv$ has zero volatility and arbitrarily large positive drift. With any fixed positive consumption, sufficiently large $K$ preserves positive wealth and makes terminal square-root utility tend to infinity. For example, in one dimension $\sigma =0$, $\mu =1$, $\theta =K$, $C=1$ gives $X_s=x+(K-1)(s-t)$, so the value is infinite when $T>t$. A finite separated solution does not exist for all matrices allowed by the printed hypotheses.

The needed condition is $\mu \in \mathrm{Range}A$, equivalent to $\mu$ being orthogonal to $\ker {\sigma }^T$, because $A$ is a symmetric matrix and $\ker A=\ker {\sigma }^T$. Assume this condition and let $B=A^{+}$ be its Moore-Penrose inverse; if $A$ is invertible, simply use $B=A^{-1}$. Put $q={\mu }^{T}B\mu \ge 0$. Seek the square-root investment-consumption value in the formThen $V_x=g/\sqrt{x}$ and $V_{xx}=-g/(2x^{3/2})$. Completing the quadratic investment expression, using $AB\mu =\mu$, shows that its maximum is attained at ${\theta }^{\ast }=2xB\mu$ and has value $qg\sqrt{x}$:Any investment component in $\ker A$ has no effect. For consumption, set $z=\sqrt{c}$. Its contribution is the concave quadratic $2z-(g/\sqrt{x})z^2$, whose maximum is $\sqrt{x}/g$ at $c^{\ast }=x/g^2$. Substituting both maxima into the Hamilton-Jacobi-Bellman equation givesWith $h=g^2$, this becomes $h^{\prime }+qh+1=0$, $h(T)=1$. Solving this linear ordinary differential equation givesIn particular $h\ge 1$, so $g=\sqrt{h}$ stays positive, and the two controls areThese controls are admissible and actually attain the value. Indeed their portfolio wealth satisfiesSince $\Vert 2{\mu }^{T}B\sigma {\Vert }^2=4q$, the explicit positive solution starting at $(x,t)$ isIts finite-horizon moments of every positive order are finite. More directly,Differentiating $g(s)\mathbb{E}\sqrt{X_s}$ and using the equation for $g$ gives $\frac{d}{ds}(g(s)\mathbb{E}\sqrt{X_s})=-\mathbb{E}\sqrt{X_s}/g(s)$. Integrating from $t$ to $T$ therefore yieldsThis proves attainment without relying on an unjustified expectation of a local martingale. When $\mu$ is outside $\mathrm{Range}A$, the singular-volatility drift arbitrage counterexample applies instead.

Put $p=1-R$, $b=\mu -r1$, $A=\sigma {\sigma }^T$ and $q=R+\epsilon$. First assume the covariance matrix $A$ is invertible and initial portfolio wealth is positive. Introduce the accumulated fee processThe Itô formula and the self-financing portfolio equation giveThis converts the path-dependent payment into a terminal-utility problem with state $y$. The volatility-penalized terminal fee changes its drift as well as its spot volatility.

Let $J(t,y)$ be the manager's value function. Its Hamilton-Jacobi-Bellman equation, with terminal condition $J(T,y)=y^p/p$, isThe homogeneity of CRRA utility suggests $J(t,y)=f(t)y^p/p$. Since $y{J}_y=f(t)y^p$ and $y^2{J}_{yy}=-Rf(t)y^p$, the portfolio-dependent expression isIts coefficient is positive even when $p<0$. Completing the square givesThus the stock fractions are constant. Substitution into the Hamilton-Jacobi-Bellman equation yieldsIn particular, multiplying the fee by $a$ does not affect the optimal investment portfolio.

To prove global optimality rather than merely solve a formal Hamilton-Jacobi-Bellman equation, write $v_t={\sigma }^T{\pi }_t$, $v^{\ast }={\sigma }^T{\pi }^{\ast }$ and let $y^{\ast }$ be the fee under the proposed constant policy. Then $b^T\pi =q{v}^{\ast }\cdot v$. Define a probability $Q$ byThis density is a true martingale because $v^{\ast }$ is constant. By the Girsanov theorem, $W_t^Q=W_t-p{v}^{\ast }t$ is a Brownian motion under $Q$. The geometric Brownian motion formula for the fee ratio givesFor admissible, locally square-integrable exposures with finite accumulated quadratic variation, setThis stochastic exponential is a positive local martingale, so $E_Q{Z}_T\le 1$. Moreover,For $p>0$, the exponent $p/(1+\epsilon )$ lies between zero and one. The Jensen inequality for this concave function gives $E_Q[Z_T^{p/(1+\epsilon )}]\le 1$. For $p<0$, the power is convex and decreasing, so the Jensen inequality gives $E_Q[Z_T^{p/(1+\epsilon )}]\ge (E_Q{Z}_T{)}^{p/(1+\epsilon )}\ge 1$; an infinite moment only makes this inequality stronger. Dividing by $p$, with its corresponding sign, proves $E[U(y_T)]\le E[U(y_T^{\ast })]$ in both cases. Equality is achieved by the constant policy.

For an ordinary terminal-wealth agent with relative risk aversion $q$, the same Hamilton-Jacobi-Bellman equation calculation, with no fee penalty, maximizes $b^T\pi -q{\pi }^{T}A\pi /2$. It therefore gives exactly the same optimal investment portfolio. For $q\ne 1$, the preceding stochastic exponential and Jensen inequality verification applies with penalty zero and exponent $1-q$, proving global optimality for this agent too. The spot volatility penalty adds $\epsilon$ to effective relative risk aversion. If $q=1$, the printed expression for $U_0$ has a zero denominator; its appropriate limit, after subtracting an irrelevant constant, is $U_0(x)=\log x$, which still gives ${\pi }^{\ast }=A^{-1}b$. In this logarithmic case, the terminal log-wealth difference from the candidate equals $\log Z_T$ for a positive stochastic exponential $Z$, so $E[\log Z_T]\le \log E[Z_T]\le 0$ proves its optimality.

If $A$ is singular, the same conclusion holds in the absence of arbitrage, provided $b$ lies in the image of a linear map $x\mapsto Ax$: replace $A^{-1}$ by the Moore-Penrose inverse, and add any vector in the kernel of $A$ to ${\pi }^{\ast }$. These additions change neither returns nor Brownian portfolio exposures. If $b$ has a nonzero component in that kernel, a zero-volatility position has a nonzero excess return. Scaling its profitable sign gives arbitrage, so a finite attained optimum as claimed requires this degeneracy to be excluded.

Choose knots independently of the eventual validation responses, and let $b(x)=(b_1(x),\dots ,b_K(x){)}^T$ be a natural cubic spline basis as in part (a). A regression spline writes $m(x)=b(x{)}^T\beta$. Define the design matrix $B_{ij}=b_j(X_i)$. Under the stated normal linear model, maximum likelihood estimation for $\beta$ is the same as ordinary least squares:Differentiation gives the normal equations $B^{T}B\hat{\beta }=B^{T}Y$. If the design has full column rank,If it does not, a Moore-Penrose inverse gives the fitted values, but not every coefficient or off-sample prediction is identifiable. Merely choosing distinct knots does not guarantee full rank for an arbitrary set of observed predictor values.

More knots allow greater curvature and reduce approximation bias at the cost of larger estimation variance. Too many or poorly located knots can cause unstable fits. Equally spaced knots are simple; empirical predictor quantiles put more flexibility where observations are dense; subject knowledge can concentrate knots near likely changes. Both the number and positions can be compared using cross-validation or an appropriate penalized model-selection criterion. Adaptive knot selection must be included within validation, since treating a data-selected basis as prespecified understates its flexibility. Boundary knots should cover the region where inference is intended, and natural linear tails do not by themselves justify far-out extrapolation.

Write $z=(Z_{s_1},\dots ,Z_{s_n}{)}^T$, let ${\Sigma }_{ij}=C(\Vert s_i-s_j\Vert )$, let $c_i=C(\Vert s_i-s_0\Vert )$, and put $v_0=C(0)$. Assume the covariance matrix $\Sigma$ is invertible. Because the mean is known to be zero, every linear predictor $w^{T}z$ is unbiased. Its mean squared prediction error isCompleting the square yieldsThe positive-definite matrix property makes the last term nonnegative, proving the simple kriging formulasThe residual is uncorrelated with $z$. Since the joint law is multivariate normal, it is independent of $z$, so the same predictor and variance describe the conditional distribution. Consequently a 95-percent prediction interval isThere is no constraint that the weights sum to one: that is required for an unknown constant mean in ordinary kriging, not for the stated known-zero-mean problem. If $\Sigma$ is singular, use a Moore-Penrose inverse with the corresponding covariance compatibility condition. At an already observed location prediction of that very same field value has zero error; predicting an independent noisy replicate is a different target. Estimated covariance parameters make this a plug-in interval and generally add uncertainty that the formula does not include.

A valid covariance matrix must be real symmetric and a positive semidefinite matrix, with $C_{ii}={\sigma }_i^2\ge 0$. Conversely, every such matrix is the covariance of a possibly degenerate multivariate normal distribution. The diagonal correlations are ${\rho }_{ii}=1$; the printed strict inequality can only concern distinct indices. Pairwise bounds do not suffice: the $3$ by $3$ correlation matrix with diagonal $1$ and every off-diagonal entry $-0.9$ has eigenvalue $-0.8$ along $1$ and is invalid. Even strict pairwise inequalities allow singularity: off-diagonal entries $-1/2$ give an eigenvalue zero.

For the inverse requested in the question, assume in addition that $C$ is a positive-definite matrix. Write $x=({\hat{\mu }}_i)$, $\Lambda =C^{-1}$ and $S=1^T\Lambda 1>0$. Up to a constant, the log-likelihood is $-\frac{1}{2}(x-\mu 1{)}^T\Lambda (x-\mu 1)$. Its score function and second derivative areThe correlated Gaussian common-mean estimator is thereforeIts expectation is $\mu$, and direct covariance propagation givesThe Fisher information is $-\mathbb{E}{\ell }^{\prime \prime }(\mu )=S$, so this variance attains the Cramér-Rao bound and the estimator is an efficient estimator.

If $C$ is singular, an ordinary Lebesgue density and $C^{-1}$ are unavailable. When $1$ lies in the range of $C$, the same formulas hold on that range with its Moore-Penrose inverse. Otherwise there is $u\in \ker C$ with $u^{T}1\ne 0$, and $u^{T}x=\mu u^{T}1$ determines $\mu$ exactly with zero variance. The usual nonsingular Cramér–Rao calculation then does not apply.

The Moore-Penrose inverse acts coordinatewise ason the domainThe coordinates supported where $d_j=0$ form $\mathcal{R}(D{)}^{\perp }$ and are sent to zero. The inverse is continuous exactly when the nonzero diagonal entries are bounded away from zero,apart from the trivial all-zero operator, whose inverse is zero. Otherwise unit vectors along entries tending to zero show that $D^{†}$ is unbounded.

Let ${\Xi }_{ik}=⟨X_i,B_k⟩$ and define the roughness penalty matrixFor $c=(c_1,\dots ,c_K{)}^{\mathsf{T}}$, the loss is the quadratic functionIts normal equations areWhenever ${\Xi }^{\mathsf{T}}\Xi +\rho \Omega$ is positive definite, the unique minimizer isFor the usual choice $\rho \ge 0$, the penalty matrix is positive semidefinite, so full column rank of $\Xi$ suffices. Since the question permits arbitrary real $\rho$, a sufficiently negative value can make the quadratic form indefinite; then the loss is unbounded below and no minimizer exists. In the singular positive-semidefinite case, the Moore-Penrose inverse describes the minimum-norm solution whenever the normal equations are consistent.

Use a singular value decomposition $X=UD{V}^T$. ThenEach nonzero singular value $d$ contributes $d/(d^2+\lambda )\to d^{-1}$, while each zero one contributes zero. Thususing the Moore-Penrose inverse.

The normal equation for a linear inverse problem isIt has a solution exactly whenWhen solutions exist they formThey are unique exactly when $\ker A=\{0\}$, while $A^{†}f$ is always the unique solution in $(\ker A{)}^{\perp }$ and the solution of minimum norm. Here $A^{†}$ is the Moore-Penrose inverse.

The Euclidean norm is invariant under an orthogonal matrix, so the full singular value decomposition givesThe second term is independent of $x$; the first attains its unique minimum zero because $\Sigma$ is an invertible matrix. The linear least squares solution is thereforeHere $A^{+}=V{\Sigma }^{-1}{U}^T$ is the Moore-Penrose inverse, and the fitted vector is the orthogonal projection $A{x}_{\ast }=U{U}^{T}b$ onto the image of a linear map defined by $A$.

With the singular value decomposition supplied, forming $U^{T}b$ costs $O(mn)$ arithmetic operations, diagonal scaling costs $O(n)$, and multiplication by $V$ costs $O(n^2)$. Since $m\ge n$,One need not form $U_{\perp }$, $A^{+}$ or $A^{T}A$ to calculate $x_{\ast }$. Computing the needed thin singular value decomposition first has standard dense cost $O(m{n}^2)$; explicitly forming an entire $m\times m$ orthogonal completion can require additional work when $m$ is much larger than $n$.

For a rectangular diagonal matrix $D$, define $D^{+}\in {\mathbb{R}}^{n\times m}$ by transposing its shape and takingEntry by entry, $D{D}^{+}D=D$ and $D^{+}D{D}^{+}=D^{+}$. For $A=U\Sigma V^T$, orthogonality therefore givessince $A{A}^{+}A=U\Sigma {\Sigma }^{+}\Sigma V^T=A$, and similarly $A^{+}A{A}^{+}=A^{+}$. This candidate is the Moore-Penrose inverse. It also makes $A{A}^{+}$ and $A^{+}A$ symmetric, although those two extra conditions are not required by the question's stated definition. The two displayed conditions alone need not specify a unique generalized inverse.

Use the singular value decomposition and the orthogonal invariance of the Euclidean norm. Put $z=V^{T}x$ and $c=U^{T}b$. Since precisely ${\sigma }_1,\dots ,{\sigma }_r$ are positive,The second sum is independent of $x$ and cannot be removed. The first sum vanishes exactly when $z_j=c_j/{\sigma }_j$ for $1\le j\le r$, while $z_{r+1},\dots ,z_n$ are arbitrary. Thus every least-squares solution isThe unavoidable linear-system residual is the component of $b$ orthogonal to the column space of $A$:The minimum-norm least-squares solution sets all ${\alpha }_j=0$ and equals $A^{+}b$, where $A^{+}$ is the Moore-Penrose inverse. The squared minimum, rather than the minimum itself, is the second sum in the first display.

The Moore--Penrose inverse is characterized by

Here $A^{†}=A^{+}$ denotes the Moore-Penrose inverse, while the star denotes the conjugate transpose. The first two identities express consistency when applying and reversing the linear map; the last two make the resulting orthogonal projection matrices Hermitian. They must hold together. For example, $A=(12)$ and $B=(10{)}^T$ satisfy $ABA=A$ and $BAB=B$, butis not Hermitian. This $B$ is a right inverse, yet $Bb=(3,0{)}^T$ is not the minimum-norm least-squares solution for $b=3$. The last two Penrose equations impose the orthogonality that selects the pseudoinverse.

Pseudoinverse fitting with a parameter-dependent covariance range requires checking the allowed residual subspace separately for each fitted parameter. If the covariance is $C_{\theta }$ and the residual is ${\mathbf{r}}_{\theta }=\mathbf{y}-\mathbf{m}(\theta )$, the range restriction in singular-covariance spectral fitting becomesThe Moore-Penrose inverse quadratic ${\mathbf{r}}_{\theta }^T{C}_{\theta }^{+}{\mathbf{r}}_{\theta }$ alone does not enforce this condition. A projector computed at one trial parameter need not describe another trial's allowed noise.

For an original example, let ${\mathbf{u}}_{\theta }=(\cos \theta ,\sin \theta {)}^T$, zero mean and $C_{\theta }={\sigma }^2{\mathbf{u}}_{\theta }{\mathbf{u}}_{\theta }^T$. Noise lies on a line that rotates with $\theta$. With observation $\mathbf{y}=(2,0{)}^T$ and $\sigma =1$, the model at $\theta =0$ permits the observation and gives quadratic value four. At $\theta =\pi /2$, the pseudoinverse quadratic is zero, yet the observation is outside the allowed vertical line. Minimizing that quadratic without its range constraint would reward an impossible candidate.

The pinned SciPy documentation describes singular-covariance evaluation using pseudo-determinants, pseudoinverses and covariance rank; its implementation additionally applies a numerical support mask and returns zero density, or negative-infinite log density, off support.^[ref] Parameter-dependent supports also require care about which measure defines a density. Values defined on different subspaces are not automatically a common likelihood merely because each uses the same-looking formula.

A physically justified transverse noise resolves the idealized singularity. With perpendicular unit vector ${\mathbf{v}}_{\theta }$, use $C_{\theta ,ϵ}={\sigma }^2{\mathbf{u}}_{\theta }{\mathbf{u}}_{\theta }^T+{ϵ}^2{\mathbf{v}}_{\theta }{\mathbf{v}}_{\theta }^T$. For $ϵ=0.1$, the two quadratic values become four and four hundred. The determinant is the same at these orientations, so this full-rank Gaussian model correctly disfavors the transverse observation.

The transverse-noise limit of a rotating rank-one covariance model is an inference about a stated physical noise model, not permission to add arbitrary diagonal noise for numerical convenience. In spectral work, distinguish a parameter-dependent physical noise direction from a fixed deterministic rebinning matrix; the latter has a fixed output range when its input covariance is positive definite.