Conditional mode of Gaussian random effects 2026-10-06
In a Gaussian linear mixed model with , the conditional multivariate normal distribution of given has mean and covariance matrix . When nonsingular, its mean is also its mode. Estimated parameters yield empirical conditional modes and shrink group deviations toward zero. At a zero variance component the associated effect is degenerate at zero; the covariance formula still applies, whereas formulas involving require limits.
Gaussian innovation likelihood 2026-10-14
In a linear Gaussian state-space model, let the predicted state mean and covariance be . The observation innovation process has conditional mean zero and covariance , with innovation . The conditional multivariate normal distribution gives the predictive observation density. Multiplying these conditional densities by the chain rule gives the displayed likelihood function; equivalently the innovations are independent Gaussian vectors. The Kalman filter supplies recursively, so one evaluates the exact observed-data likelihood function without integrating all latent states jointly.
Gaussian process regression 2026-10-14
Gaussian process regression estimates an unknown function by assigning its values a Gaussian process law, then conditioning on observed values, often with independent Gaussian noise. The covariance function expresses the proposed relation between nearby inputs; the conditional multivariate normal distribution gives the resulting Gaussian process regression posterior. Its conditional mean provides a prediction and its conditional variance quantifies uncertainty under the chosen model. The index set can represent spatial locations or other inputs as well as time.
Gaussian regression toward the mean 2026-10-14
Take the original synthetic model and , mutually independent random variables, with the second argument denoting variance. The pair has a multivariate normal distribution with mean zero, unit marginal variances, and Pearson correlation coefficient . Its conditional multivariate normal distribution isFor a first reading , the second reading has conditional expectation and conditional variance , hence standard deviation . There is still conditional probability of , a movement farther from zero, where is the standard normal distribution function. The line describes a conditional mean, not an individual trajectory.
A short derivation isolates . Its covariance with is zero and its variance is . Because has a multivariate normal distribution, zero covariance implies independence: independently of . This also explains why reversing the two measurements gives . The statistical association does not identify a direction of causal effect.
Kalman filter Created 2026-10-03 Updated 2026-10-14
The Kalman filter alternates a Kalman prediction step with a Kalman measurement update. It carries a state estimate forward using a state-space model, then adjusts it by the part of the new observation that was not predicted: the innovation. The Kalman gain determines the adjustment from the uncertainty of the prediction and the measurement noise. A noisy measurement should have less influence than an equally surprising precise one.
For a precise discrete-time statement, let the state and observation be vectors in real vector spaces obeying the linear state-space modelThe matrices and control inputs are known. Assume a Gaussian random vector with mean and covariance matrix , and mutually independent centered Gaussian noises , independent of , with covariance matrices . Suppose is positive-definite, so the observation innovation covariance is invertible. Then the recursions below give the exact posterior mean and posterior covariance matrix , hence the full conditional multivariate normal distribution. They also minimize mean squared error. Without the Gaussian assumptions, suitable second-order assumptions still give best affine state estimates, but do not generally give an exact posterior distribution. [ref]
https://www.cs.utexas.edu/~pstone/Courses/393Rfall15/readings/Welch%2BBishop-TR-95.pdf. Welch and Bishop, An Introduction to the Kalman Filter, section 1, UNC technical report TR 95-041, 2004 revision
.For an exact scalar example, suppose the predicted state has normal distribution and a measurement reports in the same units. With measurement noise variance , the Kalman gain is and the posterior distribution isWith the same prediction and observed value but measurement noise variance , the Kalman gain becomes and the posterior distribution is instead . Both updates move toward the observation; the more precise measurement moves the estimate further and reduces uncertainty more.
The same observed value produces different Kalman measurement updates when its uncertainty changes
. Both panels start from and observe . The green curve is the likelihood function normalized over the scalar state for this unit-coefficient observation model. The orange curve is the posterior density; its narrower width reflects reduced variance. The vertical dotted line marks the observed value, rather than a known true state. Details.The useful compression is exact under the stated model: the two objects contain everything from the past observations needed for the next update. Their size depends on the state dimension, not on the length of the observation history. This makes causal state filtering practical when observations arrive continually. It also gives Linear-quadratic-Gaussian control an estimate of the state on which to act.
Rudolf E. Kalman's 1960 paper organized linear filtering around state transitions, error covariance and recursive computation, connecting estimation with optimal regulation. Its introduction acknowledges earlier work by Wiener, Bode, Shannon and others; the useful historical claim is this particular formulation, rather than that estimation began in 1960. [ref]
https://www.cs.cmu.edu/~motionplanning/papers/sbp_papers/k/Kalman1960.pdf. Kalman, A New Approach to Linear Filtering and Prediction Problems, Journal of Basic Engineering 82 (1960), pp. 35–45
. Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 43 1 i Solution Created 2026-10-03 Updated 2026-10-14
Write the expected value as and assume the relevant covariance matrix is invertible. Remove the linear predictor from the first block by definingA linear image of a multivariate normal vector is again multivariate normal. Direct covariance calculation givesThus and are independent: uncorrelated jointly Gaussian variables are independent. Expanding the covariance matrix of , using , givesConditioning on translates by the fixed vector and leaves its covariance matrix unchanged. Consequently the conditional multivariate normal distribution isThe conditional covariance is the Schur complement and does not depend on .
Past exam of the mathematics course of the University of Cambridge 2005 iii Paper 46 1 iii Solution Created 2026-10-03 Updated 2026-10-14
Applying the conditional multivariate normal distribution to the two differences gives, more generally,At the result isBoth this law and the marginal law have mean zero, but the conditional variance is smaller. Specifically, with , the conditional variance is the marginal variance times . The assumptions imply , so the decrease is strict. Matching the breadth differences to zero gives information about the correlated length differences; it does not force the length differences themselves to be zero. The residual uncertainty remains positive. The original PDF has the between-son differences used here; the self-subtractions in the converted TeX are transcription errors.
Past exam of the mathematics course of the University of Cambridge 2005 iii Paper 46 1 Solution 2026-10-14
Write the partitioned mean as and assume is nonsingular, as required by the displayed inverse. For the conditional multivariate normal distribution, form the residualA linear transformation of a multivariate normal distribution is again normal. Direct calculation givesThe jointly normal blocks and are therefore independent: their joint characteristic function has no cross term in its quadratic exponent and factors into the two marginal characteristic functions. Conditioning leaves the distribution of unchanged. Reconstructing proves the full conditional law, including the requested Schur complement covariance:This describes a regular conditional law; it does not require the equality event to have positive probability.
Past exam of the mathematics course of the University of Cambridge 2007 iii Paper 42 3 Solution Created 2026-10-03 Updated 2026-10-14
Fix . For , the residual observed path isThis Brownian bridge is independent of . Indeed it is independent of , while its covariance with is ; zero covariance implies independence in the joint Gaussian law. The same argument for every finite collection of bridge times, followed by continuity and generation from rational times, gives independence of the entire bridge. Since the full observed path is determined by its endpoint and this independent bridge, Brownian endpoint sufficiency for a constant drift gives
First suppose the prior covariance matrix is positive definite. The Bayes' theorem multiplies the prior Gaussian density by the conditional observation density, whose dependence on is . Thus the posterior exponent, up to terms not depending on , isCompleting the square gives the Gaussian Brownian drift filter:For invertible these are equivalently and . At they reduce to the prior parameters.
The boxed formulas remain valid for a singular positive semidefinite matrix . To see this directly without inverse prior matrices, the joint Gaussian random vector hasThe latter matrix is invertible for . The conditional multivariate normal distribution therefore has meanand covariance matrixAlternatively, subtract this mean from ; the resulting Gaussian random vector has zero covariance with and is therefore independent of it, which proves these conditional parameters.
Write the proposed innovation process in the observable formIt is adapted to the observation filtration and continuous. To prove the martingale property, take . Future increments of are independent of the joint information generated by and the past of , soAlso the tower property of conditional expectation gives for . Conditional integration is legitimate because . Subtracting the integral proves . Removing a continuous finite variation process does not change quadratic covariation, henceThe Lévy characterization of multidimensional Brownian motion now applies: a continuous zero-start martingale with this bracket matrix is a standard -dimensional Brownian motion relative to its given filtration. The observable innovation is a Wiener process in the observation filtration.
Differentiating the posterior formulas yields andFor the final expression, the printed inverse matrices require the extra assumption that is positive definite. Under that assumption put . The Itô formula givessince and . On the other hand,The trace terms cancel in the innovation exponential for Gaussian drift filtering, leavingThus and is a positive stochastic exponential, hence a nonnegative local martingale. To obtain the requested supermartingale conclusion explicitly, choose localizing stopping times for which is a true martingale. The Conditional Fatou lemma givesThe same argument at gives , supplying integrability. The process is a supermartingale in the observation filtration. No unproved exponential-integrability condition on the random drift is required.
For a singular prior the literal inverse-matrix expression in the PDF is undefined, but there is a natural extension with the same conclusion. Define, for fixed ,For positive definite , a Gaussian integral shows this reciprocal is exactly the printed expression. For any positive semidefinite , the integral is positive and finite, , and , by the same posterior weighting. The Itô formula consequently gives . Its reciprocal satisfies the boxed stochastic differential equation and the same supermartingale proof. This completes the singular-covariance extension while explicitly separating it from the undefined printed formula.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 40 2 Solution Created 2026-10-03 Updated 2026-10-14
The state-space model as printed supplies only uncorrelated white noise. That is insufficient for the claimed normal conditional distributions. For example, take normal, independent process noises taking values equally often, and independent Gaussian observation noises. With and no previous observations, is a two-component shifted mixture distribution, not normal for . Its characteristic function is , which has zeros, whereas a normal characteristic function has none. Similarly a non-Gaussian observation noise prevents the asserted conditional normal law for .
The intended scalar Gaussian Kalman recursion requires and independent of each other and of the previous state and observation history. Assume these Gaussian independence conditions, , , and the given normal filtering law. Conditional on the past, is a sum of independent normals, henceThe new observation noise is independent of and the past, so . Write and . The independent normal pair shows directly thatThe covariance comes from the common term. Applying the conditional multivariate normal distribution formula with as the conditioned coordinate gives the Kalman filter updateThe final expression verifies nonnegativity of the posterior variance.
For , defineThe recursions become and . If , then and, with ,The nonnegative solution is the steady-state local-level Kalman gain:For the limit is zero; for it lies strictly between zero and one. These are exact conditional-law results under the additional Gaussian assumptions, not consequences of the weaker printed noise description alone.
Past exam of the mathematics course of the University of Cambridge 2010 iii Paper 33 2 Solution Created 2026-10-03 Updated 2026-10-14
Use the standard linear Gaussian state-space model convention: the initial state is independent of both mutually independent iid noise sequences. This independence from the initial state is needed for the usual forecast recursion. Let and write and . Taking conditional expectations in the state equation givesTo justify the covariance, the forecast error is . Its two terms have zero cross-covariance because is independent of the previous state and observations. Their covariances therefore add. In this Gaussian model the conditional error covariance depends on the parameters and time but not on the realized data, so it also equals the unconditional forecast-error covariance used in the notation of the question.
For completeness, derive the filtering step rather than leave the algorithm unspecified. Set and . Conditionally on the previous data, the observation has mean and covarianceDefine the innovation process and gain byThe state forecast error and innovation are jointly normal, with cross-covariance . The residual is uncorrelated with by the definition of , hence independent of it by joint normality. Its mean is zero and its covariance is . This proves the conditional multivariate normal distribution updateInitialize . For , predict using the first boxed pair, store that forecast, form , and update using the second boxed pair. The updated state then starts the next iteration. This is the Kalman filter, and supplies the entire forecast sequence in one forward pass.
The requested unconditional innovation moments areIn fact , with deterministic for fixed parameters. Thus the innovation is independent of the past observations, and hence the innovations are mutually independent. Equivalently, orthogonality to the past together with joint normality gives this independence.
The chain rule for densities now yields the Gaussian innovation likelihood, treating the specified initial mean and covariance as known:Both and are functions of the trial parameters, so the filtering recursion must be run again for each candidate . The ordinary density formula assumes positive definite; for example positive definite ensures this. Singular observation laws instead require densities on their Gaussian support.
Evaluate the log-likelihood numerically by the forward recursion, and maximize it over admissible , enforcing the covariance constraints. A Cholesky decomposition gives and the quadratic term , avoiding an explicit matrix inverse. Covariance parameterizations or constrained optimization keep positive semidefinite and the innovation covariance nonsingular. An attained maximizer is a maximum-likelihood estimator; in general this is a numerical optimization problem and neither uniqueness nor global convergence of a local optimizer is automatic.
Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 32 4 Solution Created 2026-10-03 Updated 2026-10-06
Put , the precision matrix. The conditional independence graph has an edge between and exactly when and are not conditionally independent given all the remaining components. For the multivariate normal distribution,Indeed, the conditional density of the pair given the others has its quadratic term determined by the corresponding two-by-two principal submatrix of ; its cross term vanishes exactly when , making the two conditional Gaussian variables independent. This is a Gaussian graphical model.
To derive the conditional law of one component, isolate in the multivariate normal density and complete the square. The terms involving it areThereforeEquivalently, its mean is and its variance is the Schur complement . These forms agree because the block of gives , and its block then identifies the variance. This proves the relevant conditional multivariate normal distribution formula.
In particular, the population regression of component on the other components has coefficientsintercept , and a Gaussian residual independent of the predictors with variance . These are the conditional regression coefficients from a Gaussian precision matrix. The coefficient is zero exactly when the corresponding edge is absent. Its counterpart has the same zero pattern and sign, but generally a different magnitude.
This motivates nodewise regression: for each , regress on , estimating an intercept as well. For sparse graph estimation, use the Nodewise Lasso criterionThe Lasso penalty encourages many fitted coefficients to be zero and remains usable when the number of predictors exceeds the sample size. Ordinary unpenalized sample regressions would usually give a dense estimated graph. Separate regressions can disagree about a pair, so an undirected graph can use the OR rule for nodewise graph selection, retaining an edge if either fitted direction is nonzero, or the more conservative AND rule for nodewise graph selection, retaining it only if both are nonzero. These are choices for combining estimated neighbourhoods, rather than identities forced by finite-sample fits.
The printed joint objective uses a Group Lasso penalty on the unordered pairsThis is paired group Lasso for Gaussian graph estimation. Each group represents a single undirected edge, coupling its two directional regressions. A sum of group norms encourages entire pairs to be zero; a squared group norm would give smooth ridge-type shrinkage instead of edge deletion. The constraint removes self-prediction, and the intercepts are not penalized.
The free symbols called in the objective are regression intercepts, not generally the marginal means of the original Gaussian vector. Profiling them out givesand replaces all columns by their centred versions in the squared-error terms. This is profiling intercepts in nodewise regression. Scaling predictors to comparable units can also help interpret a common penalty, although the displayed objective itself is well defined on the given data.
The grouped KKT conditions make the sparsity mechanism precise. With centred residuals , defineThe subdifferential of the Euclidean norm at zero is its closed unit ball, and away from zero is . Hence a minimizer satisfiesThese are the paired group Lasso optimality conditions. They jointly threshold the two directional scores, matching the population fact that both directions represent the same edge.
Estimate the graph by including exactly when . This graph is undirected by construction. The penalty does not impose , nor does it mathematically require both coordinates of every selected pair to be nonzero. That distinction is appropriate because the population regression coefficients themselves need not have equal magnitudes. The fitted coefficient matrix is not automatically a symmetric positive definite precision matrix; its pair supports, rather than its entries as a precision estimate, supply the requested conditional independence graph.
Past exam of the mathematics course of the University of Cambridge 2016 iii Paper 206 6 b Solution Created 2026-10-03 Updated 2026-10-06
For the Gaussian linear mixed model, temporarily regard as known, with here and . The joint multivariate normal distribution isBy the conditional multivariate normal distribution formula,A nonsingular Gaussian density has its mode at its mean. Thus the conditional mode of Gaussian random effects equals the conditional expectation. Replacing the unknown parameters by fitted values givesFor positive-definite and , an equivalent derivation minimizes the negative conditional log density,Differentiation yieldsThe inverse-covariance penalty shrinks furnace intercepts and slopes toward the population effects. For furnace , set and ; the same formula provides its two-component conditional mode from that furnace's observations, conditional on the fitted common parameters. If a variance component is zero, the covariance form remains meaningful and its corresponding random effect is zero, whereas the formula with needs a limiting interpretation. With known covariance parameters these are best linear unbiased predictors; plug-in variance estimates give their empirical version.
Past exam of the mathematics course of the University of Cambridge 2018 iii Paper 216 4 b Solution Created 2026-10-03 Updated 2026-10-05
For one factor, write for the loading vector and . Let be its current value and . The E-step of the expectation-maximization algorithm uses the conditional multivariate normal distribution to obtainThus . The terms of the expected complete-data log-likelihood that depend on a candidate areThis is a strictly concave quadratic in , since . Its unique maximizer isFor an explicit expression involving only the current loading and data, put . Substitution gives the EM update for a single-factor Gaussian modelNo empirical centering is applied because the model fixes the mean to zero. If initialized at zero, this update remains at zero; monotonicity alone does not guarantee that such a stationary point is a global maximum. This part is present in the PDF but missing from the TeX transcription.
Past exam of the mathematics course of the University of Cambridge 2018 iii Paper 219 3 iii Solution Created 2026-10-03 Updated 2026-10-05
At the fitted hyperparameters, define , , and , with the matrices formed by pairwise kernel evaluations. Conditional independence of observation noise and future latent values givesThe residual has zero covariance with , so the joint normal law makes it independent of . This derives the conditional multivariate normal distribution and the Gaussian process regression posterior:The covariance is a Schur complement and is positive semidefinite. When nonsingular, its joint density is . Repeated phases can make it singular, in which case the displayed normal law is interpreted on its support.
This predicts the true latent light curve, so no future measurement variance is added. For noisy future observations, add their independent noise covariance to . Fixing the hyperparameters omits their estimation uncertainty, as requested; integrating over their posterior would add it. The periodic model extrapolates by phase, even though all prediction times follow the observed times.
Past exam of the mathematics course of the University of Cambridge 2021 iii Paper 225 1 ii Solution 2026-09-28
Let . The Karhunen–Loève expansion and Gaussianity giveandThe conditional multivariate normal distribution formula yieldsIf is singular, the same formula uses its Moore-Penrose pseudoinverse.
Past exam of the mathematics course of the University of Cambridge 2022 iii Paper 205 5 Solution 2026-09-28
For disjoint nonempty index sets and , partition the mean and covariance conformably. The conditional multivariate normal distribution isIf the sets overlap, the shared coordinates are fixed and this formula applies to .
Partition into the and blocks. The block-inverse formula identifies the displayed conditional covariance withFor , inversion of the two-by-two precision block givesThe denominator is positive, so the conditional covariance vanishes exactly when . A Gaussian pair is independent exactly when it is uncorrelated, proving
Profiling the Gaussian likelihood over gives . Apart from constants and a positive factor, the negative log-likelihood for the precision matrix isIts derivative is , so the unpenalized minimizer is . The Graphical Lasso solveswith some conventions also penalizing the diagonal.
Past exam of the mathematics course of the University of Cambridge 2024 iii Paper 207 3 a Solution Created 2026-09-24 Updated 2026-09-25
Let have entries , let have entries , and let . The Gaussian process prior and independent Gaussian noise implyApplying the conditional multivariate normal distribution gives the Gaussian process regression posteriorwhere
Past exam of the mathematics course of the University of Cambridge 2026 iii Paper 205 5 Solution Created 2026-09-24 Updated 2026-09-25
The conditional multivariate normal distribution givesFor a jointly Gaussian vector, the residual is independent of the regressor, so with independent of .
The Square-root Lasso minimizesAt a nonzero residual , its KKT condition iswhich proves the required inequality.
Writing the response vector as , the reverse triangle inequality givesThe strong law of large numbers gives , hence .
Under , Gaussianity makes independent of and hence of . Conditionally on , . The remaining numerator term obeysCombining this with and Slutsky theorem proves the standard-normal limit.
Let be a real symmetric matrix with a positive-definite matrix. Its quadratic form satisfies the completing the square identityFor each fixed vector , the unique minimizing vector is , so the minimized value is . Also, is a positive-definite matrix exactly when is a positive-definite matrix, under the stated assumption on .[ref]
An original scalar example isHere , , , and the Schur complement is . Holding would give , but letting relax to reduces the quadratic form to . At the respective values are and . Eliminating a variable therefore changes the effective cost of the retained variable.
This interpretation supports quadratic optimization and the positive-definite symmetric matrix tests used in Cholesky decomposition. In a multivariate normal distribution, a covariance matrix also gives a conditional covariance through a Schur complement; the required probabilistic hypotheses are supplied by conditional multivariate normal distribution, rather than by the algebra alone.


