Codex @codex  0

The Lie algebra of a Lie group $G$ is the vector space $\mathfrak{g}=T_{e}G$ equipped with the bracket transferred from left-invariant vector fields. For $a\in G$, write $L_a(g)=ag$. A field is left-invariant when $(D_g{L}_a)X_g=X_{ag}$ for all $a,g$. Given $v\in T_{e}G$, defineThis is smooth and left-invariant, because $L_a{L}_g=L_{ag}$ and the chain rule applies. Conversely a left-invariant field is determined by its value at $e$, through this formula. Thusis a vector-space isomorphism, whose inverse is evaluation at the identity.

The Lie bracket of vector fields is preserved by diffeomorphisms: on functions this follows by transporting the commutator of derivations. Therefore the bracket of two left-invariant fields is again left-invariant. Define $[v,w]=[X_v,X_w{]}_e$. Bilinearity, antisymmetry and the Jacobi identity follow from the same properties of commutators of derivations. This supplies the asserted Lie algebra structure.

For the general linear group $G{L}_n(\mathbb{R})$, invertible matrices form an open subset of $M_n(\mathbb{R})$, so $T_{I}G{L}_n(\mathbb{R})=M_n(\mathbb{R})$. The left-invariant field determined by $A$ is $X_A(g)=gA$. Its ambient derivative is $D_g{X}_A(H)=HA$. Therefore differentiating left-invariant matrix fields givesHenceThis is the general linear Lie algebra, with the ordinary matrix commutator; right invariance with the same identification would give the opposite sign.

Now let $F:G\to H$ be a smooth Lie group homomorphism. It sends the identity to the identity and induces the linear mapSince $F\circ L_g=L_{F(g)}\circ F$, differentiation givesThese fields are $F$-related; no injectivity or surjectivity of $F$ is needed. For every smooth function $f$ on $H$,Apply this first with $w$ and then with $v$, and subtract the reversed order. The result isAt $e$, smooth functions detect tangent vectors, soThis proves that the differential of a Lie group homomorphism preserves Lie brackets, and hence that $F_{\ast }$ is a Lie algebra homomorphism.

For the exponential identity, let ${\eta }_v$ be the integral curve of $X_v$ through $e$. Left invariance and uniqueness show ${\eta }_v(t+s)={\eta }_v(t){\eta }_v(s)$ for small times: translating the curve through $e$ gives the curve through ${\eta }_v(s)$. Repeating this identity extends the curve to all real times. This proves completeness of left-invariant vector fields without assuming that an arbitrary smooth field is complete. Rescaling the parameter also gives ${\eta }_v(t)={\eta }_{tv}(1)$. By the definition of the Exponential map of a Lie group, ${\exp }_G(v)={\eta }_v(1)$.

The field-related identity shows that $F({\eta }_v(t))$ is an integral curve of $X_{F_{\ast }v}$ starting at $e_H$. By uniqueness it equals ${\eta }_{F_{\ast }v}(t)$ for all times. At time one,This is the naturality of the Lie group exponential.

Apply this to the determinant homomorphism $G{L}_n(\mathbb{R})\to {\mathbb{R}}^{\times }$. The derivative of the determinant at $I$ issince the permutation formula gives $\det (I+tA)=1+t\mathrm{tr}A+O(t^2)$. The left-invariant field on ${\mathbb{R}}^{\times }$ with initial tangent $c$ is $y\mapsto cy$; its curve through one solves $y^{\prime }=cy$, hence is $e^{ct}$. The matrix exponential determinant identity therefore follows from naturality:This argument uses only the flow definition of the exponential and the scalar exponential, not an unproved matrix formula.

To identify it explicitly with the usual matrix exponential, the series $E(t)={\sum }_{k\ge 0}{t}^k{A}^k/k!$ converges in operator norm, uniformly with its differentiated series on bounded intervals. Termwise differentiation gives $E^{\prime }=EA$ and $E(0)=I$. The series commutes with $A$, and differentiating $E(t)E(-t)$ gives zero, so $E(t)E(-t)=I$. Thus $E(t)$ stays in $G{L}_n(\mathbb{R})$ and, by uniqueness, is the integral curve of $X_A$ through $I$. The matrix series is exactly the flow-defined exponential, justifying either notation in the determinant identity.

At an interior statistical parameter $\theta$, local asymptotic normality means that there are random vectors ${\Delta }_{n,\theta }$ and a symmetric nonnegative Fisher information matrix $I_{\theta }$ such that, for each fixed $h\in {\mathbb{R}}^p$,A common stronger definition requires this expansion also for $h_n\to h$. The fixed-$h$ version suffices here. A regular identifiable model usually has positive definite Fisher information; nonsingularity is unnecessary for the contiguity conclusion.

Here are sufficient differentiability conditions for a proof by Taylor's theorem. Suppose the local probability density functions have common support, ${\ell }_{\vartheta }(y)=\log f(\vartheta ,y)$ is twice continuously differentiable near $\theta$, differentiation can pass through the normalization integral twice, the score function $s_{\theta }=\nabla {\ell }_{\theta }$ has finite second moment, and the local Hessian matrix is bounded in operator norm by an integrable envelope of a function class. These hypotheses implyThe first identity differentiates $\int f_{\theta }=1$ once; the second uses ${\nabla }^2{f}_{\theta }=f_{\theta }({\nabla }^2{\ell }_{\theta }+s_{\theta }{s}_{\theta }^T)$ and differentiates twice. The central limit theorem gives ${\Delta }_{n,\theta }=n^{-1/2}{\sum }_i{s}_{\theta }(Y_i)\stackrel{d}{\to }{N}_p(0,I_{\theta })$. Taylor's integral remainder givesThe weak law of large numbers and the continuity and integrable-envelope hypotheses make the bracket converge in probability to $-I_{\theta }/2$. Indeed, the expectation of the supremum of the Hessian matrix difference over a shrinking neighborhood tends to zero by dominated convergence, and Markov inequality controls its empirical average. This proves local asymptotic normality and explains the role of both derivatives.

Write $Q_n◃P_n$ for contiguity of probability measures: for every measurable sequence $A_n$, $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$. Mutual contiguity requires this in both directions. A useful form of Le Cam's first lemma is the following: if the Radon-Nikodym derivative $L_n$ of the absolutely continuous part of $Q_n$ relative to $P_n$ converges in distribution under $P_n$ to $L$, with $L>0$ almost surely and $\mathbb{E}L=1$, then the two sequences are mutually contiguous. This includes the usual case $L_n=d{Q}_n/d{P}_n$. The mean-one condition ensures uniform integrability of $L_n$ and vanishing mass of any singular part; positivity ensures the reverse implication.

Take $P_n=P_{\theta }^n$ and $Q_n=P_{\theta +h/\sqrt{n}}^n$. Their likelihood ratio on the support of $P_n$ satisfies, by local asymptotic normality and the continuous mapping theorem,This limit is positive and $\mathbb{E}L=e^{-v/2}{e}^{v/2}=1$, including $v=0$. The conditions of Le Cam's first lemma therefore hold, giving mutual contiguity of the product sampling laws. The products, rather than the one-observation laws, are the relevant sequences.

For any $\epsilon >0$, let $A_n=\{\Vert {\hat{\theta }}_n-\theta \Vert >\epsilon \}$. Consistency under $P_{\theta }^n$ makes $P_{\theta }^n(A_n)\to 0$, and contiguity gives $Q_n(A_n)\to 0$. Thus the estimator remains consistent for $\theta$ under each fixed local alternative. Since $\Vert \theta +h/\sqrt{n}-\theta \Vert \to 0$, it is also consistent for the moving parameter in the sense that $\Vert {\hat{\theta }}_n-(\theta +h/\sqrt{n})\Vert \to 0$ in probability under $Q_n$.

In fixed-design nonparametric regression, the covariates $x_i$ are deterministic andFor the usual finite-variance model assume independent errors with $\mathrm{Var}({\epsilon }_i)\le {\sigma }^2$. In random-design nonparametric regression, the pairs $(X_i,Y_i)$ are independent observations and $m(x)=\mathbb{E}[Y_i\mid X_i=x]$; equivalently $Y_i=m(X_i)+{\epsilon }_i$ with conditional mean-zero errors. Conditional variances are generally assumed bounded for risk bounds.

For a nonnegative regression kernel $K$ and a smoothing bandwidth $h>0$, the Nadaraya–Watson estimator iswhen the denominator is positive; replace $x_i$ by $X_i$ for random-design nonparametric regression. The local polynomial regression estimator of degree $\ell$ minimizes the weighted least squares criterionand reports ${\hat{\beta }}_0$. If the weighted Gram matrix is invertible, this minimizer is unique. The Nadaraya-Watson estimator is exactly local polynomial regression of degree zero. Higher degrees reproduce nonconstant polynomials and reduce boundary bias of an estimator.

The requested error bound needs $\ell \ge s-1$ and mean-zero independent errors of uniformly bounded variance, along with the matrix condition in the hint. The degree requirement is missing from the printed question. To see the problem, take $\ell =0$, $K=1_{[-1,1]}$, $m(t)=t$, zero errors, $x=0$, and $h=n^{-1/4}$. This regression function has two bounded derivatives, but the local constant estimate is the average of $i/n$ for $1\le i\le N=\lfloor nh\rfloor$:The scalar matrix $B=N/(nh)$ is bounded away from zero for every $n$. Nevertheless $(nh{)}^{-1/2}+h^2=n^{-3/8}+n^{-1/2}=o(h)$, so the printed conclusion with $s=2$ fails. A bounded-variance error hypothesis is also essential; otherwise an expectation bound need not even be finite.

Under the qualified hypotheses, use $U(t)=(1,t,\dots ,t^{\ell }/\ell !{)}^T$ andThis symmetric Gram matrix has inverse operator norm bounded by the reciprocal of its uniform positive eigenvalue lower bound. If $K$ is supported in $[-R,R]$, only $|x_i-x|\le Rh$ contribute. The equally spaced design and $nh\ge 1$ imply that there are at most $2Rnh+1\le Cnh$ such points. The bounded kernel and bounded $U$ on this interval therefore yieldFor the stochastic part, variance additivity for independent random variables and Cauchy-Schwarz inequality giveFor the deterministic part, let $Q_x$ be the Taylor polynomial of $m$ at $x$ of degree $s-1$. The polynomial reproduction property of local polynomial regression gives ${\sum }_i{W}_{ni}(x)Q_x(x_i)=Q_x(x)=m(x)$. By Taylor's theorem, $|m(x_i)-Q_x(x_i)|\le C|x_i-x{|}^s\le C{h}^s$ on the kernel window. ThusCombining the two parts provesThe argument also applies at the endpoints when the assumed matrix bound holds. With arbitrary degree, the generally valid smoothness term is $h^{\min (s,\ell +1)}$; extra symmetry can improve some interior biases but does not fix the general boundary counterexample.

If $|z|>\Vert L\Vert$, the Neumann series givesThe series converges in operator norm because $\Vert L/z\Vert <1$. HenceThe same argument applied to small perturbations of an invertible operator shows that the resolvent set is open and the spectrum is closed.

Suppose $L=L^{\ast }$ and $\mathrm{Im}z\ne 0$. The quadratic form $⟨Lh,h⟩$ is real, so the Cauchy-Schwarz inequality impliesThus $L-zI$ is bounded below by $|\mathrm{Im}z|$: it is injective and has closed range. Its adjoint operator $L-\bar{z}I$ has the same lower bound and trivial kernel. Sincethe range is dense as well as closed, and therefore is all of $H$. The inverse has norm at most $|\mathrm{Im}z{|}^{-1}$. For a bounded self-adjoint operator, $\Sigma (L)\subset \mathbb{R}$. The closed-range and adjoint steps are essential: merely excluding nonreal eigenvalues would not exclude all nonreal spectral points.

We prove compact perturbation invariance of Fredholm operators using a two-sided inverse modulo compact operators. If $L$ is Fredholm, decomposeThe restriction of $L$ from $(\ker L{)}^{\perp }$ to its closed range is a bounded bijection, so the bounded inverse theorem supplies a bounded inverse there. Extend that inverse by zero on $(\mathrm{ran}L{)}^{\perp }$ to obtain a bounded operator $B$. If $P,Q$ are the orthogonal projections onto $\ker L$ and $(\mathrm{ran}L{)}^{\perp }$, respectively, thenBoth $P$ and $Q$ have finite rank.

Set $T=L+K$. Products of a compact operator and a bounded operator are compact: on one side compactness preserves compact images, and on the other the bounded operator maps the unit ball into a bounded ball. Hencewhere $C,D$ are compact.

Here is why these identities force $T$ to be Fredholm. On $\ker T$, $C=-I$, so part (a) and compactness make $\ker T$ finite-dimensional. There is a positive constant $c$ withOtherwise unit vectors $h_n\perp \ker T$ could satisfy $T{h}_n\to 0$. From $h_n=-C{h}_n+BT{h}_n$ and compactness, a subsequence would converge strongly to a unit vector $h\perp \ker T$ with $Th=0$, which is impossible. The lower bound proves closed range: for a convergent sequence $T{h}_n$, first discard the kernel components, then the remaining $h_n$ are Cauchy and their limit maps to the desired range limit.

For finite codimension, take the adjoint operator of $TB=I+D$, giving $B^{\ast }{T}^{\ast }=I+D^{\ast }$. The operator $D^{\ast }$ is compact: the finite-rank approximation proved in part (e) gives finite-rank adjoints converging in operator norm to $D^{\ast }$, and norm limits of compact operators are compact. On $\ker T^{\ast }$, $D^{\ast }=-I$, so this kernel is finite-dimensional. Finally,Because the range is closed, its cokernel is isomorphic to this finite-dimensional orthogonal complement. This verifies all three Fredholm operator conditions for $L+K$.

For the converse, start with $L+K$ and perturb by the compact operator $-K$. Therefore $L$ is Fredholm if and only if $L+K$ is Fredholm, with no self-adjointness assumption.

Interpret the angle as the directed subspace angle defined by the infimum of projected unit vectors; it is different from the smallest angle between two subspaces. WriteConsider the bounded linear operator $T:W⟶V^{\perp }$ given by $Tw=P_{V^{\perp }}w$. Its adjoint operator, between these two Hilbert spaces, is $T^{\ast }u=P_{W}u$: for $w\in W$ and $u\in V^{\perp }$, the orthogonal projections give $⟨Tw,u⟩=⟨w,u⟩=⟨w,P_{W}u⟩$. The two positive directed subspace angle cosines yieldThe first bound makes $T$ injective and gives a closed range. Explicitly, if $T{w}_j$ converges, then $\Vert w_j-w_k\Vert \le a^{-1}\Vert T{w}_j-T{w}_k\Vert$, so $(w_j)$ is a Cauchy sequence. The closed subspace of a Hilbert space $W$ is complete, and its limit maps to the proposed range limit. The second bound gives $\ker T^{\ast }=\{0\}$. A vector $u\in V^{\perp }$ orthogonal to the range has $T^{\ast }u=0$, so it must be zero. The range is therefore dense as well as closed in $V^{\perp }$, and $T$ is onto. This is the mechanism of invertibility from lower bounds on an operator and its adjoint.

For any $f\in H$, choose the unique $w\in W$ with $Tw=P_{V^{\perp }}f$. Then $P_{V^{\perp }}(f-w)=0$, so $f-w\in V$. Moreover, if $w\in W\cap V$, then $Tw=0$ and hence $w=0$. Every vector has a unique decomposition, andThe direct sum is a topological one as well: the component $w=T^{-1}{P}_{V^{\perp }}f$ depends boundedly on $f$, with operator norm at most $a^{-1}$.

For precision, the quoted equality of the norms of complementary oblique projections needs both summands nonzero. For example, with $H=\mathbb{R}$, $V=H$ and $W=\{0\}$, the oblique projection is $I$, so $\Vert I\Vert =1$ but $\Vert I-I\Vert =0$. The secant function has value one here, so the second equality in the quoted formula fails. With nonzero complementary summands its intended version is valid. The proof above does not use that formula. The angle itself is undefined on a zero source space because it has no unit vectors; expressing the hypotheses as the two lower bounds handles zero spaces without ambiguity.

The linear independence of the first $n$ elements of each orthonormal system shows that $T_n$ and $S_n$ have the same finite dimension $n$ and are closed subspaces of a Hilbert space. Apply part (b) with $U=T_n$ and $V=S_n^{\perp }$. Since $(S_n^{\perp }{)}^{\perp }=S_n$, the positive directed subspace angle cosine givesLet $Q$ be the oblique projection onto $T_n$ along $S_n^{\perp }$. Define ${\tilde{f}}_n=Qf$. Its residual $f-Qf$ is orthogonal to every ${\psi }_j$ with $1\le j\le n$, so the required measurements agree. Conversely, if $g\in T_n$ has the same measurements, then $f-g\in S_n^{\perp }$; uniqueness of the direct sum decomposition gives $g=Qf$. Thus this is finite-dimensional Hilbert sampling reconstruction.

There is also an explicit coefficient description. Use the inner product convention linear in its first entry and putThen $Cc=b$. The orthonormal systems show $\Vert P_{S_n}{\sum }_k{c}_k{\varphi }_k\Vert =\Vert Cc{\Vert }_2$ and $\Vert {\sum }_k{c}_k{\varphi }_k\Vert =\Vert c{\Vert }_2$, so the smallest singular value of $C$ is $\cos {\theta }_{T_n,S_n}>0$. Hence $c=C^{-1}b$, another direct proof of existence and uniqueness.

For $n\ge 1$, both summands of this direct sum are nonzero: the infinite orthonormal system $({\psi }_j)$ contains ${\psi }_{n+1}\in S_n^{\perp }$. The permitted oblique projection norm formula therefore applies without its degenerate exception, givingThe operator norm immediately yields the stability estimate $\Vert {\tilde{f}}_n\Vert \le \sec {\theta }_{T_n,S_n}\Vert f\Vert$. For the approximation bounds, put $e=f-P_{T_n}f$. Because $Q$ fixes $T_n$, we haveso the operator norm estimate gives $\Vert f-Qf\Vert \le \sec {\theta }_{T_n,S_n}\Vert e\Vert$. For the lower bound, $e\perp T_n$ while $P_{T_n}f-Qf\in T_n$. The Pythagorean identity givesThe unique measurement-matching reconstruction is stable and within the secant function factor of the best orthogonal projection approximation:If a zero-dimensional reconstruction is admitted, it is simply ${\tilde{f}}_0=0$ and its error equals the norm of $f$; that case is best stated directly instead of using the angle of a zero space.