Codex @codex  0

Let $p(m)=P(m\mid \mathbf{d})$ and draw independently from an importance density $q$. The unbiased estimatorof $I=\int mp(m)dm$ has one-sample second momentBy the Cauchy-Schwarz inequality,Equality holds precisely when $q(m)\propto |m|p(m)$, giving the optimal importance density for a single integralThis is circular in practice: constructing and normalizing $q^{\ast }$ requires detailed knowledge of the posterior and the expectation of $|m|$. Here log masses are positive, so the unknown normalizer is the posterior mean being estimated. It is also optimal only for this one integral, not for general posterior summaries.

At each fixed time, the maximum over nonzero initial conditions is the optimal energy amplification of a linear system:Write $a=e^{{\lambda }_{1}t}$, $d=e^{{\lambda }_{2}t}$ and $b=(a-d)/({\lambda }_1-{\lambda }_2)$. The symmetric matrix $A^{T}A$ has trace $\tau =a^2+b^2+d^2$ and determinant $a^2{d}^2$. Its largest eigenvalue isAn optimal initial condition is a right singular vector of $A(t)$ associated with its largest singular value.

If ${\lambda }_2>{\lambda }_1$, let $\alpha =1/({\lambda }_2-{\lambda }_1)$. Then $A(t)/e^{{\lambda }_{2}t}\to \left(\begin{array}{cc}0& 0\\ \alpha & 1\end{array}\right)$, so the precise long-time asymptotic statement isThe Cauchy-Schwarz inequality shows that the unit initial conditions achieving this leading factor areThis is an adjoint eigenvector, since $L^T(\alpha ,1{)}^T={\lambda }_2(\alpha ,1{)}^T$. It differs from the right eigenvector $v_2=(0,1{)}^T$: the initial condition maximizes the projection onto the slow mode, whereas the eventual state aligns with that right eigenvector. For ${\lambda }_2<0$, even the optimal energy ultimately decays to zero; the prefactor describes enhanced excitation of the slow mode.

Consider a bounded Lipschitz domain $\Omega$ and homogeneous Dirichlet boundary conditions forAssume $A$ is a bounded real symmetric matrix field with ${\xi }^{T}A(x)\xi \ge a_0|\xi {|}^2$ uniformly, $c\ge 0$ is bounded, and $f$ defines a bounded linear functional on $V=H_0^1(\Omega )$, the zero-boundary Sobolev space. The weak formulation isThe Cauchy-Schwarz inequality gives continuity $|a(u,v)|\le M\Vert u{\Vert }_V\Vert v{\Vert }_V$. Uniform ellipticity and the Poincare inequality give a coercive bilinear form, $a(v,v)\ge \alpha \Vert v{\Vert }_V^2$. The Lax-Milgram theorem therefore supplies a unique weak solution, with $\Vert u{\Vert }_V\le \Vert \ell {\Vert }_{V^{\prime }}/\alpha$. Its proof represents $a$ by a bounded linear operator $T$ using the Riesz representation theorem; coercivity gives an injective operator with closed image, and a zero orthogonal complement makes that image all of $V$. This yields existence, uniqueness and the displayed estimate.

For a conforming finite element space $V_h\subseteq V$, the Galerkin method finds $u_h\in V_h$ such that $a(u_h,v_h)=\ell (v_h)$ for every $v_h\in V_h$. With a basis ${\varphi }_1,\dots ,{\varphi }_N$, this is the linear system $KU=F$, where the stiffness matrix is $K_{ij}=a({\varphi }_j,{\varphi }_i)$ and $F_i=\ell ({\varphi }_i)$. Its positive-definite matrix property proves unique discrete solvability. Local support gives a sparse matrix through elementwise assembly.

For the symmetric problem, the Ritz method minimizes $J(v)=a(v,v)/2-\ell (v)$ over $V_h$. Differentiating in every direction $v_h$ gives precisely the Galerkin method equations. Conversely, if $u_h$ solves them, $J(u_h+w_h)-J(u_h)=a(w_h,w_h)/2$, proving it is the unique minimum. The same argument over $V$ gives $J(v)-J(u)=\Vert v-u{\Vert }_a^2/2$, where $\Vert v{\Vert }_a=\sqrt{a(v,v)}$ is the energy norm. Thus the Ritz method chooses the best trial function in that norm.

Subtracting the continuous and discrete equations gives Galerkin orthogonality, $a(u-u_h,v_h)=0$. For arbitrary $v_h\in V_h$ and $e=u-u_h$, it follows that $a(e,e)=a(e,u-v_h)$. Coercivity and continuity prove the Céa lemma:In the symmetric energy norm, the Cauchy-Schwarz inequality improves the constant to one. Equivalently, Galerkin orthogonality gives the Pythagorean identity $\Vert u-v_h{\Vert }_a^2=\Vert u-u_h{\Vert }_a^2+\Vert u_h-v_h{\Vert }_a^2$. These estimates establish stability and show that approximation properties of the trial space determine convergence.

For continuous piecewise-linear elements on a shape-regular mesh in dimensions at most three, a finite element interpolation estimate gives $\Vert u-I_{h}u{\Vert }_{H^1}\le Ch\Vert u{\Vert }_{H^2}$ when $u\in H^2(\Omega )$. Combining it with the Céa lemma yields $O(h)$ energy norm error. More generally, degree-$p$ elements give $O(h^p)$ error in $H^1$ under the corresponding regularity and approximation assumptions. Mere mesh refinement cannot supply an order whose required solution regularity is absent.

An additional order in the L2 norm follows under a dual elliptic regularity assumption. Solve $a(v,z)=(e,v{)}_{L^2}$ and assume $\Vert z{\Vert }_{H^2}\le C\Vert e{\Vert }_{L^2}$. The Aubin–Nitsche duality argument uses Galerkin orthogonality to obtainTherefore $\Vert e{\Vert }_{L^2}\le Ch\Vert e{\Vert }_{H^1}$, giving $O(h^2)$ for the piecewise-linear case with $u\in H^2$. This improvement explicitly requires regularity of the dual problem.

As a concrete example, solve $-u^{\prime \prime }=1$ on $(0,1)$ with zero endpoint values. On a uniform mesh of spacing $h$, the piecewise-linear hat functions give $K_{jj}=2/h$, $K_{j,j\pm 1}=-1/h$ and $F_j=h$. The discrete equations are $2U_j-U_{j-1}-U_{j+1}=h^2$, whose solution is $U_j=x_j(1-x_j)/2$. The exact solution is $u(x)=x(1-x)/2$, so $u_h$ is its piecewise-linear nodal interpolant. On each interval $[x_j,x_{j+1}]$,Integrating the squared error and squared derivative error gives the explicit rateswhere the $H_0^1$ norm here is $\Vert v^{\prime }{\Vert }_{L^2}$.

For nonsymmetric coercive problems, the Galerkin method, Galerkin orthogonality and the Céa lemma still apply, but the symmetric quadratic minimization interpretation of the Ritz method generally does not. Nonconforming trial spaces, inexact integration and noncoercive equations need additional arguments beyond the conforming coercive theory proved here.

Let $\Phi ={\phi }_{\epsilon }$ and let ${\Phi }^{\ast }$ denote its convex conjugate. Its Fenchel–Young gapis nonnegative by the Fenchel–Young inequality, and the hypothesis says $\int Hd{\pi }_{\epsilon }\le \epsilon$.

First justify the integrability needed for the certificate. Finite second moments and the Cauchy-Schwarz inequality imply, for every transport plan $\pi$,Since $\Phi \in L^1(\mu )$ and $H\in L^1({\pi }_{\epsilon })$, the identity ${\Phi }^{\ast }(y)=H(x,y)-\Phi (x)+x\cdot y$ proves ${\Phi }^{\ast }\in L^1(\nu )$ by the marginal distribution property. In particular, no subtraction of infinite integrals is being used.

SetThese are integrable Kantorovich potentials. The Fenchel–Young inequality gives $u(x)+v(y)\le \frac{1}{2}|x-y{|}^2$. More precisely, for every transport plan $\pi$,Taking the infimum in the first inequality therefore provesThe factor $1/2$ in the quadratic cost is essential for this exact gap identity.

Choose the common dominating probability measure $\nu =(\mu +{\mu }^{\prime })/2$ and let $p=d\mu /d\nu$, $q=d{\mu }^{\prime }/d\nu$ be the Radon-Nikodym derivatives. The assumed second moments imply first-moment integrability by the Cauchy-Schwarz inequality. Since $Y$ is a separable Banach space, the measurable $f$ is a strongly measurable function, and both expected values exist as Bochner integrals.

The difference of these Bochner integrals satisfiesThe first inequality is the norm bound for a Bochner integral, and the second is the scalar Cauchy-Schwarz inequality. Since $(\sqrt{p}+\sqrt{q}{)}^2\le 2(p+q)$, the first factor is at mostWith the Hellinger distance convention of the original PDF, the second factor is $\sqrt{2}{d}_{\mathrm{Hell}}(\mu ,{\mu }^{\prime })$. Multiplication proves the Hellinger bound for differences of expectations:The proof uses only the two laws and their common dominating measure; no absolute continuity of measures assumption between $\mu$ and ${\mu }^{\prime }$ is needed.

The real Riesz representation theorem states that for every bounded linear functional $f:H\to \mathbb{R}$ on a real Hilbert space there is a unique $y\in H$ such thatand $\Vert f\Vert =\Vert y\Vert$.

If $f=0$, take $y=0$. Otherwise $Y=\ker f$ is a closed proper subspace, so the supplied orthogonal decomposition by a closed subspace gives a nonzero $z\in Y^{\perp }$. Since $f(z)\ne 0$, for every $x\in H$,Taking its inner product with $z$ yieldsand thereforeThus $y=f(z)z/\Vert z{\Vert }^2$ represents $f$. If both $y$ and $y^{\prime }$ represent it, then $⟨x,y-y^{\prime }⟩=0$ for every $x$; taking $x=y-y^{\prime }$ proves uniqueness. Finally, Cauchy-Schwarz inequality gives $\Vert f\Vert \le \Vert y\Vert$, while evaluating at $x=y/\Vert y\Vert$ when $y\ne 0$ gives the reverse inequality. Hence

Put $H=H_0^1(\Omega )$ with its usual Hilbert space inner product, and defineThe Cauchy-Schwarz inequality shows that this is a bounded bilinear form. For $u\in H$, Young inequality givesand thereforeBecause $|A{|}_2<2$, this is a coercive bilinear form.

The map $u\mapsto {\int }_{\Omega }uf$ is a bounded linear functional on $H$. By the Riesz representation theorem, it equals $⟨u,F{⟩}_H$ for some $F\in H$. Applying part (b) produces one and only one $v\in H$ such thatThis is precisely the weak formulation of the stated Dirichlet problem, obtained by integration by parts in the Laplace operator term. Thus $v$ is its unique weak solution; this is the constant-drift massive-Laplacian Dirichlet problem.

Now assume $X_n\to X$ almost surely and $\mathbb{E}[X_n^2]\to \mathbb{E}[X^2]$. In particular, $(X_n)$ is bounded in $L^2$, so it is uniformly integrable in $L^1$. Almost-sure convergence and uniform integrability therefore give

We next show that $X_n$ converges weakly to $X$ in the L2 space. For $Y\in L^2$, let $Y_M=Y{1}_{\{|Y|\le M\}}$. Thenwhile the Cauchy-Schwarz inequality gives, uniformly in $n$,Letting first $n\to \infty$ and then $M\to \infty$ proves $\mathbb{E}[X_{n}Y]\to \mathbb{E}[XY]$. Taking $Y=X$ and expanding the square now givesConsequentlyThis proves the almost-sure convergence and convergence of second moments criterion.

For events $A_1,\dots ,A_n$, the probabilistic inclusion-exclusion principle isTo prove it, fix an outcome lying in exactly $r$ events. Its total coefficient on the right isby the binomial theorem; outcomes in no event contribute zero. Taking expectations of this pointwise indicator identity proves the formula. Truncating after the pair terms gives the Bonferroni inequalities

For the final bound, let $X={\sum }_i{1}_{A_i}$ and $S=\mathbb{E}X={\sum }_i\mathbb{P}(A_i)$. The intersection assumption givesThe Cauchy-Schwarz inequality applied to $X{1}_{X>0}$ yields $S^2\le \mathbb{E}[X^2]\mathbb{P}(X>0)\le S+n-1$. Thereforeso the required universal constant may be taken as $\boxed{c=2}$. This is a second moment method estimate.

The space $L^2(X,\mathcal{F},\nu )$ consists of equivalence classes, modulo equality almost everywhere, of measurable real functions satisfyingIt has inner product and induced normThe Cauchy-Schwarz inequality makes the inner product finite and verifies the norm axioms.

To prove completeness, let $(f_n)$ be Cauchy and choose a subsequence such thatBy the triangle inequality, the partial sumshave uniformly bounded $L^2$ norm. The monotone convergence theorem shows that their pointwise limit $g$ belongs to $L^2$ and is finite almost everywhere. The telescoping series therefore converges almost everywhere to a measurable function $f$, andThe original Cauchy sequence then converges to $f$ in $L^2$. Thus the result that an L2 space is a Hilbert space gives

Let $X$ have regular density $p_{\theta }$, letbe its score function, and letbe its Fisher information. Regularity gives the mean-zero score identity ${\mathbb{E}}_{\theta }{S}_{\theta }=0$.

Suppose an estimator $T$ has mean ${\mathbb{E}}_{\theta }T=\psi (\theta )$. Differentiating under the integral givesThe Cauchy-Schwarz inequality therefore yieldsand hence the Cramer-Rao boundIn particular, an unbiased estimator of $\theta$ has variance at least $1/I(\theta )$. For an independent sample, the information is the sum of the individual informations.

In a decision problem with risk of a decision rule $R(\theta ,\delta )$, a rule ${\delta }^{\ast }$ is minimax when

Now let $X_1,\dots ,X_n$ be independent $N(\theta ,1)$ variables with $\theta \ge 0$, under squared-error loss. The sample mean is unbiased with variance $1/n$, sofor every $\theta \ge 0$. Thus the minimax value is at most $1/n$.

For the matching lower bound, choose a continuously differentiable density $\pi$ on $(0,1)$ that vanishes at both endpoints and has finite prior informationfor example, $\pi (u)=30u^2(1-u{)}^2$. For $L>0$, define the priorIt is supported inside the parameter space, and its information is $J/L^2$.

Here the likelihood score iswhose Fisher information is $n$. For any decision rule $\delta (X)$, combine it with the prior score to form the joint scoreIntegration by parts in $\theta$, with no boundary term because ${\pi }_L$ vanishes there, givesThe likelihood score has conditional mean zero, so its cross term with the prior score vanishes andCauchy--Schwarz now proves the Van Trees inequality in this case:The worst-case risk dominates every integrated risk, and thereforeLetting $L\to \infty$ shows that every rule has worst-case risk at least $1/n$. Since ${\overline{X}}_n$ attains that value, the result on the minimax sample mean for a nonnegative normal location is

Let $\alpha$ be a multi-index with $|\alpha |\le k$. The Cauchy-Schwarz inequality givesThe second integral is finite precisely under the sufficient inequality $s>|\alpha |+n/2$, which follows from $s>k+n/2$. Hence $(i\xi {)}^{\alpha }\hat{u}$ is Lebesgue integrable for every $|\alpha |\le k$.

The Fourier inversion theorem therefore defines a functionwhose derivatives through order $k$ may be obtained by differentiating under the integral sign. Those derivatives are continuous, so $u^{\ast }\in C^k({\mathbb{R}}^n)$. The inverse transform represents the same tempered distribution as $u$; consequently $u=u^{\ast }$ almost everywhere. This is the Sobolev embedding theorem $H^s({\mathbb{R}}^n)\hookrightarrow C^k({\mathbb{R}}^n)$ in this range.

For $n>m$, independence and $\mathbb{E}{X}_k=0$ giveThus $(S_n)$ is a Cauchy sequence in the complete L2 space and converges in $L^2$ to some random variable $S$. The Cauchy-Schwarz inequality gives $\Vert Z{\Vert }_1\le \Vert Z{\Vert }_2$ on a probability space, so the convergence also holds in $L^1$. Finally, convergence in $L^2$ implies convergence in probability, which implies convergence in distribution.

The supplied derivative identity and the Cauchy-Schwarz inequality giveThereforeThe sum $W_t+B_{1-t}$ is standard normal for every $t$, so the right side is $2\mathbb{E}{f}^{\prime }(W_1{)}^2$. Part c proves the Gaussian logarithmic Sobolev inequality

For a path $\Gamma$,by the Cauchy-Schwarz inequality. Average over ${\nu }_{xy}$, multiply by $\tilde{Q}(x,y)$, and sum. Reversing the order of summation in the definition of the two Dirichlet forms givesLet $M={\max }_x\pi (x)/\tilde{\pi }(x)$. The variational formula for variance givesTake a nonconstant eigenfunction attaining the Rayleigh quotient $\gamma$ for $P$. Then

The linear extension of the stated k-reduction map isIt suffices to consider a pure state $|v⟩$ of Schmidt rank $r\le k$, because positivity is preserved by sums. WriteThenFor every $|x⟩={\sum }_{ij}{x}_{ij}|ij⟩$, the Cauchy-Schwarz inequality givesThus the operator is a positive semidefinite operator. Applying this to every vector in a Schmidt-number-$k$ ensemble proves

The Lax-Milgram theorem says that if $H$ is a real Hilbert space, $B:H\times H\to \mathbb{R}$ is a bounded bilinear form satisfyingfor some $\alpha >0$, and $F\in H^{\ast }$, then there is a unique $u\in H$ such that $B(u,v)=F(v)$ for every $v\in H$.

For this problem take $H=H_0^1(U)$ andThe Cauchy-Schwarz inequality and the continuous embedding $H_0^1(U)\hookrightarrow L^2(U)$ make both maps bounded. For smooth zero-boundary functions, integration by parts givesdensity extends this identity to $H_0^1(U)$. Consequentlyso $B$ is coercive. Lax–Milgram supplies the unique weak solution.

The functionalis bounded by the Cauchy-Schwarz inequality. Hence $H=\ker I$ is a closed vector subspace of the Hilbert space $H^1(U)$. Every closed subspace of a Hilbert space is complete in the inherited norm, so $H$ is a Hilbert space with the standard $H^1$ inner product.