Write , let , let , and put . Assume the covariance matrix is invertible. Because the mean is known to be zero, every linear predictor is unbiased. Its mean squared prediction error is
Completing the square yields
The positive-definite matrix property makes the last term nonnegative, proving the simple kriging formulas
The residual is uncorrelated with . Since the joint law is multivariate normal, it is independent of , so the same predictor and variance describe the conditional distribution. Consequently a 95-percent prediction interval is
There is no constraint that the weights sum to one: that is required for an unknown constant mean in ordinary kriging, not for the stated known-zero-mean problem. If is singular, use a Moore-Penrose inverse with the corresponding covariance compatibility condition. At an already observed location prediction of that very same field value has zero error; predicting an independent noisy replicate is a different target. Estimated covariance parameters make this a plug-in interval and generally add uncertainty that the formula does not include.
Let have rows and let be the spatial covariance matrix with . Conditional on chosen covariance parameters, the best linear unbiased estimator of the drift coefficients is the generalized least squares estimator
The printed procedure first estimates the linear drift from depth ~ y, constructs a binned residual empirical semivariogram, fits a Gaussian semivariogram to it, and then uses the fitted covariance in generalized least squares trend estimation and universal kriging. The missing fit could be supplied as
gauss.model2 <- fit.variogram(smvg2, vgm(0.5, "Gau", 1, 0))
where those arguments are initial values, not the reported final estimates. Such semivariogram fitting typically minimizes a weighted sum of squared discrepancies between binned empirical values and model values; it is not automatically a joint Gaussian likelihood maximization. One can iterate: form residuals using the current drift, refit the covariance model, recompute the generalized least squares drift, and repeat until changes are small. The displayed commands themselves show a single sequence, not evidence that such iteration occurred.
A likelihood-based alternative estimates the two blocks coherently from
Substitute to obtain a profile log-likelihood, maximize over and with a positive-definite matrix , and recover the drift coefficients from the optimizing covariance. Restricted maximum likelihood adds the design determinant term and uses in place of in the normalizing term, up to a fixed-design constant. It can reduce bias from estimating the drift before the covariance. A zero fitted nugget is a legitimate boundary estimate, not proof that measurement error is absent. Estimate drift using covariance-weighted regression, and spatial dependence from drift-adjusted variation, with the two stages or likelihood explicitly distinguished.
Let contain independent complete observations of the -dimensional vector, and let be a Directed acyclic graph. A Gaussian Bayesian network uses, for node ,
Here is an intercept, the vector of parent coefficients, and the conditional variance; the vector collects these statistical parameters. Its likelihood function factors as
Choose a proper statistical parameter prior distribution and a graph prior distribution . Bayes theorem gives the Bayesian network structure score, up to the normalization common to graphs,
The integral is Bayesian model evidence: it averages over nuisance parameters rather than substituting their best-fitting values. A graph prior can favor sparse graphs, while the evidence balances fit against the amount of prior statistical parameter space that predicts the observations well. Compare these scores over admissible Directed acyclic graphs, using enumeration when feasible or a search procedure otherwise; search need not find a global maximum, and observational data need not identify a unique causal orientation.
A conjugate prior makes the integral analytic. For example, for take , , where is positive-definite matrix and ; these are independent local normal-inverse-gamma priors. Under prior independence between nodes, the evidence is a product of local regression evidences. Each posterior has the same family, so the local integral is the ratio of prior and posterior normalization constants, with the likelihood's constants included. This avoids costly numerical integration and makes local graph updates inexpensive. Hyperparameters must be specified coherently if score equivalence between observationally equivalent graphs is desired; arbitrary local priors do not automatically have that property.
Use the natural cubic spline convention appropriate to the interior design knots: the spline is linear on and . We also need . These points will be important for uniqueness.
A cubic spline is a function in whose restriction to each of , , through is a polynomial of degree at most three. A spline knot is a junction between consecutive pieces. For a natural cubic spline, the two exterior pieces are affine functions; equivalently, one uses the natural endpoint conditions on the spline over and then extends linearly with matching derivatives. Merely prescribing on otherwise arbitrary exterior cubic pieces is a different condition and does not provide the unique interpolant invoked in this question.
The natural cubic spline interpolant to values is the natural cubic spline satisfying . Its existence and uniqueness for are the interpolant fact allowed in the PDF. The interpolation map is a linear map: a linear combination of interpolants satisfies the same spline, boundary and value conditions, so uniqueness identifies it with the interpolant to the corresponding linear combination of value vectors.
Let , let have the same values, and put . Then . We prove the orthogonality
Apply integration by parts twice on each polynomial interval. On an interior interval, , and
The terms vanish because vanishes at the knots. The terms cancel at internal knots because and are continuous, and vanish at because is zero there. On the exterior intervals, identically. Summing therefore gives the claimed orthogonality without assuming that is continuous across knots.
Expanding the square now gives the minimum roughness property of the natural cubic spline interpolant
Equality forces everywhere, since is continuous. Thus is an affine function. It vanishes at two distinct design points, so , proving equality if and only if .
For the penalized fit, take the allowed spline roughness penalty matrix to be a symmetric matrix with ; thus it is a positive semidefinite matrix. If its supplied representative were not symmetric, its symmetric part defines the same quadratic form and is all that is needed. For any , let . Replacing it by preserves every fitted value and can only decrease the second derivative roughness penalty. Therefore it is enough to minimize
over . The matrix is a positive-definite matrix, since
Completing the quadratic or differentiating gives the unique solution
This is the cubic smoothing spline. More explicitly, the normal equations are , and
For an arbitrary function with fitted vector , the earlier orthogonality also gives
Both terms are nonnegative. Equality forces and, by the proved interpolation equality case, . This proves both existence in and uniqueness over the entire stated function class, not just over splines.
The argument is deterministic for each observed . The fixed-design nonparametric regression and homoscedasticity assumptions motivate the squared-error loss, but Gaussian errors are not required.
At least two distinct design points are required for uniqueness. Question 4 does not explicitly restate , and this hypothesis is necessary. With one design point, every function has zero residual and zero second derivative roughness penalty, for any real . Thus the printed unrestricted uniqueness conclusion would be false for . The corrected theorem assumes at least two distinct knots and the stated linear-tail convention. At , uniqueness over all of also fails; the stipulated eliminates that degeneracy.
Choose a basis of a conforming finite element space and write . Testing the variational problem with each basis function gives
These are also the equations for the Ritz method. The stiffness matrix is symmetric and a positive-definite matrix: for a nonzero coefficient vector , linear independence gives , and
Thus there is a unique coefficient vector.
Subtracting the exact and discrete variational problems proves Galerkin orthogonality, for every . In the energy norm , the Pythagorean identity yields
Therefore the Ritz method is the best approximation in the energy norm. The Céa lemma also gives , so any dense sequence of conforming trial spaces converges.
The quadratic form has symmetric matrix
whose characteristic polynomial is . All its eigenvalues are positive, so is positive definite and is an ellipsoid. The spectral theorem for real symmetric matrices gives semi-axis lengths
The nearest points lie in the eigendirection of the largest eigenvalue. Since is an -eigenvector, they are
Multiplication by gives
The numerator is real because is symmetric, while the denominator is strictly positive because is positive definite. Hence is real. As is real and singular, its kernel contains a nonzero real generalized eigenvector.
Use the integral normalization of the Chern connection curvature. For a holomorphic local frame , put . The fundamental form of a Hermitian holomorphic line bundle is
This follows from the local formula for the Chern connection on a line bundle, . It is a closed real (1, 1)-form representing the First Chern class. If , the formula becomes .
The Hermitian metric is positive when this is a positive real (1, 1)-form: for every nonzero ,
Equivalently, the matrix is a positive-definite matrix in every frame. Such a metric makes a positive holomorphic line bundle. Omitting the normalization factor changes none of these positivity conditions.
Use the convention that ridge regression minimizes . Its closed-form ridge regression estimator is
The factors and here specify the tuning convention. No intercept is needed for the centred responses and predictors.
For the two-component procedure, fix and put . Differentiating the objective with respect to its dense coefficient gives
Thus, with ,
The inverse exists because , even for a rank-deficient design. The sum of the fitted sparse and dense components is the Lava estimator.
Multiplying by and completing the square gives the exact profile calculation
Let . A singular value decomposition of shows that its eigenvalues are in nonzero singular directions and one in the orthogonal complement. Hence is a positive-definite matrix. Take its symmetric principal square root of a positive semidefinite matrix, . Profiling out therefore yields the Lasso reduction for the Lava estimator:
No restandardization of is needed: this is the exact transformed objective with its original penalty.
Write , , and . Then . Comparing the transformed Lasso objective at and gives the Basic inequality for the Lasso
On , Holder inequality bounds the score by . Using cancels the fitted penalty and yields the slow-rate prediction bound for the Lasso:
Finally, if , then , so
Consequently
Positive actually guarantees for every finite design; the factor measures how much prediction norm the profiling transformation can remove.
A valid covariance matrix must be real symmetric and a positive semidefinite matrix, with . Conversely, every such matrix is the covariance of a possibly degenerate multivariate normal distribution. The diagonal correlations are ; the printed strict inequality can only concern distinct indices. Pairwise bounds do not suffice: the by correlation matrix with diagonal and every off-diagonal entry has eigenvalue along and is invalid. Even strict pairwise inequalities allow singularity: off-diagonal entries give an eigenvalue zero.
For the inverse requested in the question, assume in addition that is a positive-definite matrix. Write , and . Up to a constant, the log-likelihood is . Its score function and second derivative are
The correlated Gaussian common-mean estimator is therefore
Its expectation is , and direct covariance propagation gives
The Fisher information is , so this variance attains the Cramer-Rao bound and the estimator is an efficient estimator.
If is singular, an ordinary Lebesgue density and are unavailable. When lies in the range of , the same formulas hold on that range with its Moore-Penrose inverse. Otherwise there is with , and determines exactly with zero variance. The usual nonsingular Cramér–Rao calculation then does not apply.
For a feasible vector, set . It is a positive semidefinite matrix, and
Thus every original feasible point supplies an equally valuable SDP feasible point. Dropping the rank-one restriction is the semidefinite relaxation of slab-constrained quadratic maximization, proving
The original PDF has ; the absolute values are missing from the extracted TeX and are essential for this relaxation.
For the subsequent finite rounding argument, the constraint vectors must span . Otherwise a nonzero common-kernel vector makes both and unbounded feasible families, so both objective suprema are infinite. Under spanning, is a positive-definite matrix and
The first inequality uses positive semidefinite trace nonnegativity. The SDP feasible set is closed and bounded, hence compact, so an optimum exists. Its trace is positive: a sufficiently small positive multiple of the identity is feasible when . These facts justify the optimum and nonzero denominator used below. Assume .
The trial space must be a subspace of , not merely of . The printed assumptions on the basis do not ensure square-integrable weak derivatives, and boundary traces are not defined for arbitrary L2 space functions. For example, is continuous, vanishes at both endpoints and belongs to , but is not integrable at zero. Assume the basis belongs to the zero-boundary Sobolev space, as it does for the hat functions in part (c).
Write . Differentiating the restricted energy with respect to , or imposing the Galerkin method with test function , gives
The derivative term is the usual diffusion stiffness matrix and the second is the mass matrix. The full matrix is real symmetric, and for every nonzero coefficient vector ,
because a basis is linearly independent. Thus it is a positive-definite matrix, so the restricted Ritz method has a unique solution and minimum.
The piecewise-linear hat functions have disjoint supports unless their nodes coincide or are neighbours. Direct integration on the two adjacent intervals gives
Since and the hat is symmetric about , its load is . Thus the reaction-diffusion finite element matrix gives the explicit equations
The coefficient matrix is a tridiagonal matrix and a positive-definite matrix by the energy identity in part (b), hence nonsingular. A separate spectral verification uses the sine basis with and gives
Both terms are positive for the Dirichlet modes. This also identifies precisely how the diffusion stiffness matrix and reaction mass matrix combine.
The starting point of a conforming finite element method is a weak formulation in a Hilbert space . For example, a homogeneous Dirichlet boundary condition on a bounded Lipschitz domain gives . One seeks
where is a bounded bilinear form, , and is a bounded linear functional. A conforming finite element space is finite dimensional and is typically built from functions that are polynomial on each element of a finite element mesh. With a basis , the Galerkin method imposes the same identity only for , giving
The stiffness matrix is sparse when basis supports overlap only locally. Local element integrals assemble its entries; the mass matrix represents an term. This separates the choice of trial functions from the variational principle determining their coefficients.
The fundamental well-posedness theorem is the Lax-Milgram theorem: if for some , there is a unique solution for each , with . The same theorem on gives a unique discrete solution. The constants must be independent of the mesh for uniform stability of a numerical method. Boundary conditions are built into the trial space when they are essential constraints; flux conditions arise through boundary terms in integration by parts and are natural constraints in the weak formulation.
When is symmetric and a coercive bilinear form, the Ritz method minimizes
over . Its first variation is exactly the Galerkin method, so Ritz minimization and Galerkin testing coincide for symmetric coercive problems. The energy norm turns the approximation into an orthogonal projection. Indeed, subtracting the exact and discrete weak equations gives Galerkin orthogonality, . For every the Pythagorean identity becomes
so
This exact best-approximation property is the central advantage of the Ritz method.
Symmetry is unnecessary for the Galerkin method. For any bounded, coercive form, the Céa lemma gives
To prove it, put . Galerkin orthogonality yields , so coercivity and boundedness give . Divide when and take the infimum. Thus approximation capability plus uniform coercivity gives convergence. In contrast, minimizing for a nonsymmetric form differentiates its symmetric part and generally does not solve the original weak equation.
A concrete symmetric example is the one-dimensional reaction-diffusion problem from Question 5. Continuous piecewise-linear hat functions give the diffusion stiffness matrix with diagonal and neighbouring entries , and the mass matrix with diagonal and neighbouring entries . Therefore and . Its positive-definite matrix is the coordinate form of the continuous energy. The exact smooth solution makes this an explicit test of the method, not just an abstract existence result.
For a nonsymmetric example, take positive , constant real , and
Its form is . Since , it has coercivity constant in the full norm, although it is not symmetric when . The Galerkin method remains well posed. On the uniform hat basis, , where , and . The skew-symmetric matrix cancels from , proving nonsingularity. A Ritz energy would omit precisely this advection contribution. Small may nevertheless make the approximation constant large and produce poorly resolved layers; algebraic solvability is not an accuracy guarantee.
Quantitative convergence uses a finite element interpolation estimate. On a shape-regular mesh, for piecewise-linear conforming elements and in the usual low-dimensional setting,
The Céa lemma therefore gives an energy error. The Aubin–Nitsche duality argument improves the L2 norm error when the dual elliptic problem has regularity: solve and use
Consequently
with constants depending on the solution and regularity bounds. The second estimate is conditional on dual regularity; corners or insufficient data regularity can reduce the rate.
The same Galerkin method handles evolution equations. For the homogeneous heat equation, the semidiscrete finite element heat equation is . Because is a positive-definite matrix and is a positive semidefinite matrix,
The decreasing quadratic form is exactly the L2 norm squared of the finite-element function. This separates spatial stability of a numerical method from the subsequent time integrator, just as in the Schrodinger equation example. Conformity, coercivity and approximation estimates together explain convergence.
A Lie group is a group and smooth manifold whose multiplication and inversion are smooth. A Lie algebra is a vector space with a bilinear alternating bracket satisfying the Jacobi identity. For a Matrix Lie group , a logarithmic chart of a matrix Lie group near sends to ; the matrix exponential is its local inverse, and the Baker--Campbell--Hausdorff formula makes the local group operations smooth.
For , consider the group commutator
Its logarithm takes values in the vector space , and expansion at gives
Thus is closed under the commutator. Bilinearity, alternation, and the Jacobi identity follow from matrix multiplication, so this proves that the Lie algebra of a matrix Lie group has
A principal bundle with structure group is a smooth fiber bundle with a free right -action, each fiber a single orbit, and equivariant local trivializations .
For the right action of the unitary group on , define
The matrix is a positive-definite matrix and a Hermitian matrix, and for . The polar decomposition of an invertible complex matrix gives the unique factorization
Consequently
is a diffeomorphism. If is a neighborhood of zero and , then is an open neighborhood of , it is a union of complete orbits, and
Moreover, two matrices have the same value of exactly when they differ by right multiplication by a unitary matrix. Thus induces the smooth identification
under which is projection . This is the general linear group modulo the unitary group, and is in fact a globally trivial principal -bundle.
Reflection in preserves the prescribed translational velocity and axial angular velocity. A force transforms as a polar vector, whereas a torque transforms as an axial vector; consequently . Reflection in reverses both imposed motions but preserves the normal force component. The linearity of Stokes flow then gives . Thus only and can be nonzero.
There is a sign inconsistency in the printed question. In right-handed coordinates, gives a rotational velocity at the sphere's lowest point. The printed shear and force formulas instead use the opposite rotational sense. Below, let be the right-handed component along ; the paper's displayed formulas are recovered by setting .
In the translating frame, the parabolic gap and leading boundary velocities are
The no-slip boundary condition and lubrication theory give
The gap is stationary in this frame, so . The Reynolds lubrication equation is therefore
Since and ,
Differentiating the velocity profile gives the leading shear stresses
The first expression becomes the printed formula after .
For the logarithmic lubrication resistance of a sphere near a wall, write . The logarithmic region is , where . Consequently
These follow from and , with a fixed small outer cutoff. The general logarithmic radial numerator is over ; the numerator in the printed integration hint appears to have a typographical error.
The force exerted on the fluid through the upper gap boundary includes pressure acting on its slope:
The pressure cancels the term, giving
Pressure exerts no torque about the centre of a sphere because its traction is radial. To logarithmic order, the shear has lever arm , so
The other components vanish by parity: involves an integral of the term, and has an integrand odd in . The combined hydrodynamic resistance matrix is
Its equal off-diagonal coefficients are required by the Lorentz reciprocal theorem for Stokes flow. It is a positive-definite matrix, as required by viscous dissipation.
In the rotational convention of the printed force formula, set and measure the scalar couple along , so . Then
For sedimentation under the buoyancy-adjusted weight , a uniform sphere is torque-free. Thus , and the force balance gives
The rotational scalar in the paper's displayed-force convention is . The sphere rolls in the right-handed sense but still slips: its lowest point has laboratory velocity . The signs and coefficients of the right-handed resistance agree with the rigid-wall terms of Bertin et al., equations (4.4)–(4.5), after reversing the forces and torques there from fluid-on-sphere to sphere-on-fluid.
The mass matrix is a positive-definite matrix, and integration by parts with periodic boundary conditions gives . Hence
This is energy conservation for semidiscrete Galerkin advection: . Thus the semidiscretization is stable and conserves its finite element L2 norm.
For a mesh-independent comparison with the grid norm, the Fourier symbol of is , between and . Consequently
The conserved energy therefore gives a uniform bound in as well. Equivalently, each Fourier mode evolves with the purely imaginary exponent . This concerns the semidiscrete method; a chosen time integrator must separately control those imaginary eigenvalues.
Consider a bounded Lipschitz domain and homogeneous Dirichlet boundary conditions for
Assume is a bounded real symmetric matrix field with uniformly, is bounded, and defines a bounded linear functional on , the zero-boundary Sobolev space. The weak formulation is
The Cauchy-Schwarz inequality gives continuity . Uniform ellipticity and the Poincare inequality give a coercive bilinear form, . The Lax-Milgram theorem therefore supplies a unique weak solution, with . Its proof represents by a bounded linear operator using the Riesz representation theorem; coercivity gives an injective operator with closed image, and a zero orthogonal complement makes that image all of . This yields existence, uniqueness and the displayed estimate.
For a conforming finite element space , the Galerkin method finds such that for every . With a basis , this is the linear system , where the stiffness matrix is and . Its positive-definite matrix property proves unique discrete solvability. Local support gives a sparse matrix through elementwise assembly.
For the symmetric problem, the Ritz method minimizes over . Differentiating in every direction gives precisely the Galerkin method equations. Conversely, if solves them, , proving it is the unique minimum. The same argument over gives , where is the energy norm. Thus the Ritz method chooses the best trial function in that norm.
Subtracting the continuous and discrete equations gives Galerkin orthogonality, . For arbitrary and , it follows that . Coercivity and continuity prove the Céa lemma:
In the symmetric energy norm, the Cauchy-Schwarz inequality improves the constant to one. Equivalently, Galerkin orthogonality gives the Pythagorean identity . These estimates establish stability and show that approximation properties of the trial space determine convergence.
For continuous piecewise-linear elements on a shape-regular mesh in dimensions at most three, a finite element interpolation estimate gives when . Combining it with the Céa lemma yields energy norm error. More generally, degree- elements give error in under the corresponding regularity and approximation assumptions. Mere mesh refinement cannot supply an order whose required solution regularity is absent.
An additional order in the L2 norm follows under a dual elliptic regularity assumption. Solve and assume . The Aubin–Nitsche duality argument uses Galerkin orthogonality to obtain
Therefore , giving for the piecewise-linear case with . This improvement explicitly requires regularity of the dual problem.
As a concrete example, solve on with zero endpoint values. On a uniform mesh of spacing , the piecewise-linear hat functions give , and . The discrete equations are , whose solution is . The exact solution is , so is its piecewise-linear nodal interpolant. On each interval ,
Integrating the squared error and squared derivative error gives the explicit rates
where the norm here is .
For nonsymmetric coercive problems, the Galerkin method, Galerkin orthogonality and the Céa lemma still apply, but the symmetric quadratic minimization interpretation of the Ritz method generally does not. Nonconforming trial spaces, inexact integration and noncoercive equations need additional arguments beyond the conforming coercive theory proved here.
The conditional mean estimator is the posterior mean
provided the first moment is finite. A maximum a posteriori estimate is any maximizer of the posterior density relative to Lebesgue measure:
Existence of the posterior density alone does not guarantee a finite posterior mean or existence of a maximizing point.
For the stated multivariate normal distribution and independent standard Gaussian noise, the negative log of the posterior density, up to a constant, is
Its precision matrix is a positive-definite matrix, so Gaussian conjugacy for a normal linear model gives
The two forms agree because
The covariance matrix is , and the maximum a posteriori estimate equals the posterior mean for this multivariate normal distribution.