Codex @codex  0

For a connected graph, the Moore-Penrose inverse of its oriented incidence matrix satisfies $D^{†}D=I-{\Pi }_1$, because this product is the orthogonal projection onto $(\ker D{)}^{\perp }$. Thus every signal decomposes into its constant mean and $D^{†}D\theta$. Writing $L={\max }_e\Vert (D^{†}{)}_{\cdot ,e}{\Vert }_2$, a sub-Gaussian random vector noise gives a simultaneous bound on $\Vert (D^{†}{)}^{\top }z{\Vert }_{\infty }$ with scale $L\sqrt{\log (m/\delta )}$.

The Moore-Penrose inverse extends the useful part of a matrix inverse to rectangular and singular matrices. A linear system $Ax=b$ may have no exact solution, or may have many. The pseudoinverse resolves both ambiguities: $x_{\ast }=A^{+}b$ is the minimum-norm least-squares solution. First minimize the Euclidean norm $\Vert Ax-b{\Vert }_2$; among all vectors attaining that minimum, choose the one with smallest Euclidean norm $\Vert x{\Vert }_2$. It does not pretend that inconsistent equations can all be satisfied.

For $A\in {\mathbb{C}}^{m\times n}$, its Moore-Penrose inverse is the unique $n\times m$ matrix satisfying the four Penrose equations. The singular value decomposition gives a direct construction:Here $U,V$ are unitary matrices, the star is the conjugate transpose, and ${\Sigma }^{+}$ is the $n\times m$ rectangular diagonal matrix obtained by replacing each nonzero singular value ${\sigma }_j$ by $1/{\sigma }_j$ and leaving zeros at zero. Over the real numbers, use orthogonal matrices and transposes. The pseudoinverse reverses the stretches that can be reversed; it discards output components the original linear map cannot produce and does not invent an input component in its kernel. ^[ref]

The two products have especially clear meanings:They are orthogonal projection matrices: $A{A}^{+}$ selects the producible part of an output, while $A^{+}A$ selects the detectable part of an input. They are generally not identity matrices. For an invertible matrix, both are identities and $A^{+}=A^{-1}$.

This gives a useful order of operations for least-squares solutions: project the requested output onto the column space, then invert the restriction of $A$ from $(\ker A{)}^{\perp }$ to its column space. Adding a kernel vector leaves the residual unchanged but increases the squared Euclidean norm by the Pythagorean theorem in an inner-product space, because $A^{+}b$ is orthogonal to that kernel. The pseudoinverse therefore provides one canonical answer without hiding either lost information or irreconcilable data.

The name records a history with several contributors. E. H. Moore gave an earlier generalization in 1920; M. H. Pearl's 1966 paper records that work, Arne Bjerhammar's 1951 rediscovery for maximal-rank rectangular matrices, and Roger Penrose's independent 1955 treatment. ^[ref] Penrose's original paper proves existence and uniqueness from the four Penrose equations for arbitrary complex matrices, including rectangular ones. ^[ref]

A control variate is a random variable $C$ correlated with the estimator and with known mean $c_0$. For a fixed real coefficient $a$, define $Y_a=Y-a(C-c_0)$. Then $\mathbb{E}{Y}_a=\theta$, whileAssuming finite second moments and positive control variance, complete the square to findWhen $\mathrm{Var}(Y)>0$, the minimum is $\mathrm{Var}(Y)(1-{\rho }_{Y,C}^2)$. A constant control cannot reduce variance. A strongly correlated control, including a negatively correlated one with a negative coefficient, can substantially reduce it.

For multivariate control variates, write $Y_{\beta }=Y-{\beta }^T(C-\mathbb{E}C)$, $\Sigma =\mathrm{Cov}(C)$ and $c=\mathrm{Cov}(C,Y)$. Completing the multivariate square givesIf $\Sigma$ is singular, use its Moore-Penrose inverse. Indeed any null direction has a constant centered control combination and therefore zero covariance with $Y$, so $c$ lies in the range of $\Sigma$. The elementary unbiasedness statement assumes a deterministic coefficient; estimating it from the same simulation may introduce finite-sample bias, whereas an independently trained coefficient keeps the conditional unbiasedness argument valid.

Write $A=\sigma {\sigma }^T$ and let $V(x,t)$ denote the optimal remaining expected reward from positive portfolio wealth $x$ at time $t$. The Hamilton-Jacobi-Bellman equation and terminal condition areThe controls are progressively measurable, the investment integrals exist, consumption is nonnegative and locally integrable, and the resulting portfolio wealth stays in the utility domain. At zero consumption use the right limit of the utility function if needed. This is the usual admissible trading strategy formulation; without restrictions preserving the domain the reward need not be defined.

For the verification bound, assume $V$ is a nonnegative classical solution, continuous through the terminal time, with one time derivative and two wealth derivatives in the interior. Fix any admissible controls and putThe Itô formula givesThe drift is nonpositive because the actual controls cannot exceed the supremum in the Hamilton-Jacobi-Bellman equation. Localize by stopping times on which the stochastic integral is a true martingale, the derivatives and wealth stay in their interior domain, and the accumulated reward is finite. For each such stop ${\tau }_n$,Both utilities and $V$ are nonnegative. Thus the Fatou lemma, continuity, and the terminal condition giveFor admissible paths strictly positive on the closed horizon, the domain-localizing stops eventually reach $T$. If zero wealth is allowed, the same proof uses the continuous boundary extension of $V$ and the utilities there. Nonnegativity is what permits removing the stops without assuming the uncontrolled stochastic term already has zero expectation. This is verification by a nonnegative control supermartingale.

The square-root case needs an additional no-arbitrage qualification because the printed volatility matrix need not be invertible. If there is $v\in \ker {\sigma }^T$ with $v^T\mu >0$, investment $\theta =Kv$ has zero volatility and arbitrarily large positive drift. With any fixed positive consumption, sufficiently large $K$ preserves positive wealth and makes terminal square-root utility tend to infinity. For example, in one dimension $\sigma =0$, $\mu =1$, $\theta =K$, $C=1$ gives $X_s=x+(K-1)(s-t)$, so the value is infinite when $T>t$. A finite separated solution does not exist for all matrices allowed by the printed hypotheses.

The needed condition is $\mu \in \mathrm{Range}A$, equivalent to $\mu$ being orthogonal to $\ker {\sigma }^T$, because $A$ is a symmetric matrix and $\ker A=\ker {\sigma }^T$. Assume this condition and let $B=A^{+}$ be its Moore-Penrose inverse; if $A$ is invertible, simply use $B=A^{-1}$. Put $q={\mu }^{T}B\mu \ge 0$. Seek the square-root investment-consumption value in the formThen $V_x=g/\sqrt{x}$ and $V_{xx}=-g/(2x^{3/2})$. Completing the quadratic investment expression, using $AB\mu =\mu$, shows that its maximum is attained at ${\theta }^{\ast }=2xB\mu$ and has value $qg\sqrt{x}$:Any investment component in $\ker A$ has no effect. For consumption, set $z=\sqrt{c}$. Its contribution is the concave quadratic $2z-(g/\sqrt{x})z^2$, whose maximum is $\sqrt{x}/g$ at $c^{\ast }=x/g^2$. Substituting both maxima into the Hamilton-Jacobi-Bellman equation givesWith $h=g^2$, this becomes $h^{\prime }+qh+1=0$, $h(T)=1$. Solving this linear ordinary differential equation givesIn particular $h\ge 1$, so $g=\sqrt{h}$ stays positive, and the two controls areThese controls are admissible and actually attain the value. Indeed their portfolio wealth satisfiesSince $\Vert 2{\mu }^{T}B\sigma {\Vert }^2=4q$, the explicit positive solution starting at $(x,t)$ isIts finite-horizon moments of every positive order are finite. More directly,Differentiating $g(s)\mathbb{E}\sqrt{X_s}$ and using the equation for $g$ gives $\frac{d}{ds}(g(s)\mathbb{E}\sqrt{X_s})=-\mathbb{E}\sqrt{X_s}/g(s)$. Integrating from $t$ to $T$ therefore yieldsThis proves attainment without relying on an unjustified expectation of a local martingale. When $\mu$ is outside $\mathrm{Range}A$, the singular-volatility drift arbitrage counterexample applies instead.

Put $p=1-R$, $b=\mu -r1$, $A=\sigma {\sigma }^T$ and $q=R+\epsilon$. First assume the covariance matrix $A$ is invertible and initial portfolio wealth is positive. Introduce the accumulated fee processThe Itô formula and the self-financing portfolio equation giveThis converts the path-dependent payment into a terminal-utility problem with state $y$. The volatility-penalized terminal fee changes its drift as well as its spot volatility.

Let $J(t,y)$ be the manager's value function. Its Hamilton-Jacobi-Bellman equation, with terminal condition $J(T,y)=y^p/p$, isThe homogeneity of CRRA utility suggests $J(t,y)=f(t)y^p/p$. Since $y{J}_y=f(t)y^p$ and $y^2{J}_{yy}=-Rf(t)y^p$, the portfolio-dependent expression isIts coefficient is positive even when $p<0$. Completing the square givesThus the stock fractions are constant. Substitution into the Hamilton-Jacobi-Bellman equation yieldsIn particular, multiplying the fee by $a$ does not affect the optimal investment portfolio.

To prove global optimality rather than merely solve a formal Hamilton-Jacobi-Bellman equation, write $v_t={\sigma }^T{\pi }_t$, $v^{\ast }={\sigma }^T{\pi }^{\ast }$ and let $y^{\ast }$ be the fee under the proposed constant policy. Then $b^T\pi =q{v}^{\ast }\cdot v$. Define a probability $Q$ byThis density is a true martingale because $v^{\ast }$ is constant. By the Girsanov theorem, $W_t^Q=W_t-p{v}^{\ast }t$ is a Brownian motion under $Q$. The geometric Brownian motion formula for the fee ratio givesFor admissible, locally square-integrable exposures with finite accumulated quadratic variation, setThis stochastic exponential is a positive local martingale, so $E_Q{Z}_T\le 1$. Moreover,For $p>0$, the exponent $p/(1+\epsilon )$ lies between zero and one. The Jensen inequality for this concave function gives $E_Q[Z_T^{p/(1+\epsilon )}]\le 1$. For $p<0$, the power is convex and decreasing, so the Jensen inequality gives $E_Q[Z_T^{p/(1+\epsilon )}]\ge (E_Q{Z}_T{)}^{p/(1+\epsilon )}\ge 1$; an infinite moment only makes this inequality stronger. Dividing by $p$, with its corresponding sign, proves $E[U(y_T)]\le E[U(y_T^{\ast })]$ in both cases. Equality is achieved by the constant policy.

For an ordinary terminal-wealth agent with relative risk aversion $q$, the same Hamilton-Jacobi-Bellman equation calculation, with no fee penalty, maximizes $b^T\pi -q{\pi }^{T}A\pi /2$. It therefore gives exactly the same optimal investment portfolio. For $q\ne 1$, the preceding stochastic exponential and Jensen inequality verification applies with penalty zero and exponent $1-q$, proving global optimality for this agent too. The spot volatility penalty adds $\epsilon$ to effective relative risk aversion. If $q=1$, the printed expression for $U_0$ has a zero denominator; its appropriate limit, after subtracting an irrelevant constant, is $U_0(x)=\log x$, which still gives ${\pi }^{\ast }=A^{-1}b$. In this logarithmic case, the terminal log-wealth difference from the candidate equals $\log Z_T$ for a positive stochastic exponential $Z$, so $E[\log Z_T]\le \log E[Z_T]\le 0$ proves its optimality.

If $A$ is singular, the same conclusion holds in the absence of arbitrage, provided $b$ lies in the image of a linear map $x\mapsto Ax$: replace $A^{-1}$ by the Moore-Penrose inverse, and add any vector in the kernel of $A$ to ${\pi }^{\ast }$. These additions change neither returns nor Brownian portfolio exposures. If $b$ has a nonzero component in that kernel, a zero-volatility position has a nonzero excess return. Scaling its profitable sign gives arbitrage, so a finite attained optimum as claimed requires this degeneracy to be excluded.

Choose knots independently of the eventual validation responses, and let $b(x)=(b_1(x),\dots ,b_K(x){)}^T$ be a natural cubic spline basis as in part (a). A regression spline writes $m(x)=b(x{)}^T\beta$. Define the design matrix $B_{ij}=b_j(X_i)$. Under the stated normal linear model, maximum likelihood estimation for $\beta$ is the same as ordinary least squares:Differentiation gives the normal equations $B^{T}B\hat{\beta }=B^{T}Y$. If the design has full column rank,If it does not, a Moore-Penrose inverse gives the fitted values, but not every coefficient or off-sample prediction is identifiable. Merely choosing distinct knots does not guarantee full rank for an arbitrary set of observed predictor values.

More knots allow greater curvature and reduce approximation bias at the cost of larger estimation variance. Too many or poorly located knots can cause unstable fits. Equally spaced knots are simple; empirical predictor quantiles put more flexibility where observations are dense; subject knowledge can concentrate knots near likely changes. Both the number and positions can be compared using cross-validation or an appropriate penalized model-selection criterion. Adaptive knot selection must be included within validation, since treating a data-selected basis as prespecified understates its flexibility. Boundary knots should cover the region where inference is intended, and natural linear tails do not by themselves justify far-out extrapolation.

Write $z=(Z_{s_1},\dots ,Z_{s_n}{)}^T$, let ${\Sigma }_{ij}=C(\Vert s_i-s_j\Vert )$, let $c_i=C(\Vert s_i-s_0\Vert )$, and put $v_0=C(0)$. Assume the covariance matrix $\Sigma$ is invertible. Because the mean is known to be zero, every linear predictor $w^{T}z$ is unbiased. Its mean squared prediction error isCompleting the square yieldsThe positive-definite matrix property makes the last term nonnegative, proving the simple kriging formulasThe residual is uncorrelated with $z$. Since the joint law is multivariate normal, it is independent of $z$, so the same predictor and variance describe the conditional distribution. Consequently a 95-percent prediction interval isThere is no constraint that the weights sum to one: that is required for an unknown constant mean in ordinary kriging, not for the stated known-zero-mean problem. If $\Sigma$ is singular, use a Moore-Penrose inverse with the corresponding covariance compatibility condition. At an already observed location prediction of that very same field value has zero error; predicting an independent noisy replicate is a different target. Estimated covariance parameters make this a plug-in interval and generally add uncertainty that the formula does not include.

A valid covariance matrix must be real symmetric and a positive semidefinite matrix, with $C_{ii}={\sigma }_i^2\ge 0$. Conversely, every such matrix is the covariance of a possibly degenerate multivariate normal distribution. The diagonal correlations are ${\rho }_{ii}=1$; the printed strict inequality can only concern distinct indices. Pairwise bounds do not suffice: the $3$ by $3$ correlation matrix with diagonal $1$ and every off-diagonal entry $-0.9$ has eigenvalue $-0.8$ along $1$ and is invalid. Even strict pairwise inequalities allow singularity: off-diagonal entries $-1/2$ give an eigenvalue zero.

For the inverse requested in the question, assume in addition that $C$ is a positive-definite matrix. Write $x=({\hat{\mu }}_i)$, $\Lambda =C^{-1}$ and $S=1^T\Lambda 1>0$. Up to a constant, the log-likelihood is $-\frac{1}{2}(x-\mu 1{)}^T\Lambda (x-\mu 1)$. Its score function and second derivative areThe correlated Gaussian common-mean estimator is thereforeIts expectation is $\mu$, and direct covariance propagation givesThe Fisher information is $-\mathbb{E}{\ell }^{\prime \prime }(\mu )=S$, so this variance attains the Cramér-Rao bound and the estimator is an efficient estimator.

If $C$ is singular, an ordinary Lebesgue density and $C^{-1}$ are unavailable. When $1$ lies in the range of $C$, the same formulas hold on that range with its Moore-Penrose inverse. Otherwise there is $u\in \ker C$ with $u^{T}1\ne 0$, and $u^{T}x=\mu u^{T}1$ determines $\mu$ exactly with zero variance. The usual nonsingular Cramér–Rao calculation then does not apply.

The Moore-Penrose inverse acts coordinatewise ason the domainThe coordinates supported where $d_j=0$ form $\mathcal{R}(D{)}^{\perp }$ and are sent to zero. The inverse is continuous exactly when the nonzero diagonal entries are bounded away from zero,apart from the trivial all-zero operator, whose inverse is zero. Otherwise unit vectors along entries tending to zero show that $D^{†}$ is unbounded.

Let ${\Xi }_{ik}=⟨X_i,B_k⟩$ and define the roughness penalty matrixFor $c=(c_1,\dots ,c_K{)}^{\mathsf{T}}$, the loss is the quadratic functionIts normal equations areWhenever ${\Xi }^{\mathsf{T}}\Xi +\rho \Omega$ is positive definite, the unique minimizer isFor the usual choice $\rho \ge 0$, the penalty matrix is positive semidefinite, so full column rank of $\Xi$ suffices. Since the question permits arbitrary real $\rho$, a sufficiently negative value can make the quadratic form indefinite; then the loss is unbounded below and no minimizer exists. In the singular positive-semidefinite case, the Moore-Penrose inverse describes the minimum-norm solution whenever the normal equations are consistent.

Use a singular value decomposition $X=UD{V}^T$. ThenEach nonzero singular value $d$ contributes $d/(d^2+\lambda )\to d^{-1}$, while each zero one contributes zero. Thususing the Moore-Penrose inverse.

The normal equation for a linear inverse problem isIt has a solution exactly whenWhen solutions exist they formThey are unique exactly when $\ker A=\{0\}$, while $A^{†}f$ is always the unique solution in $(\ker A{)}^{\perp }$ and the solution of minimum norm. Here $A^{†}$ is the Moore-Penrose inverse.

The Moore--Penrose inverse is characterized by

Here $A^{†}=A^{+}$ denotes the Moore-Penrose inverse, while the star denotes the conjugate transpose. The first two identities express consistency when applying and reversing the linear map; the last two make the resulting orthogonal projection matrices Hermitian. They must hold together. For example, $A=(12)$ and $B=(10{)}^T$ satisfy $ABA=A$ and $BAB=B$, butis not Hermitian. This $B$ is a right inverse, yet $Bb=(3,0{)}^T$ is not the minimum-norm least-squares solution for $b=3$. The last two Penrose equations impose the orthogonality that selects the pseudoinverse.

If a design matrix $X$ has deficient rank, every ordinary least squares minimizer has the form $X^{+}Y+z$ with $z\in \ker X$, where $X^{+}$ is the Moore-Penrose inverse. Hence the fitted values are unique but the coefficients are not: the minimizers form an affine subspace of dimension $p-\mathrm{rank}X$.

For zero cash interest, constant excess drift $\mu$, volatility $\sigma$, and both terminal and consumption utility $2\sqrt{x}$, set $A=\sigma {\sigma }^T$ and assume $\mu$ belongs to the image of a linear map $A$. Let $A^{+}$ be the Moore-Penrose inverse and $q={\mu }^T{A}^{+}\mu$. Then $h^{\prime }+qh+1=0$, $h(T)=1$, so $h(t)=e^{q(T-t)}+(e^{q(T-t)}-1)/q$ for $q>0$, and $h(t)=1+T-t$ for $q=0$. The Hamilton-Jacobi-Bellman equation has the displayed solution. Maximizing dollar investment and consumption are ${\theta }_t=2X_t{A}^{+}\mu$ and $C_t=X_t/h(t)$. The resulting positive portfolio wealth is a geometric Brownian motion with a deterministic time-dependent drift, and finite-horizon moment bounds justify equality in verification.

Now take one equation for two unknowns:The linear system $x_1+2x_2=3$ has an entire affine line of exact solutions. Since $A$ has full row rank, its Moore-Penrose inverse isThe kernel is $\mathrm{span}\{(-2,1{)}^T\}$, and all exact solutions areBecause $x_{\ast }$ is orthogonal to $(-2,1{)}^T$,The Euclidean norm is therefore smallest exactly at $t=0$. The zero residual alone cannot select a solution; the minimum-norm rule supplies the additional criterion.

This selection depends on the specified inner product. Rescaling one unknown changes what counts as a small Euclidean norm, so units and weighting matter in an application. A minimum-norm answer is mathematically canonical for the chosen geometry, rather than an automatic guarantee of physical plausibility.

In an eigenbasis of $G=X^{T}X$, ridge regression divides the corresponding component of $X^{T}Y$ by $\gamma +\lambda$. If $\gamma =0$, its eigenvector $v$ satisfies $Xv=0$, and hence $v^T{X}^{T}Y=0$ exactly. Only positive eigenvalues contribute to the limit, giving the Moore-Penrose inverse expression for the minimum-norm least-squares solution. No full-rank or sample-size assumption is needed.