Sudakov-Fernique inequality Created 2026-10-06 Updated 2026-11-05
For centered multivariate normal distributions, if for every pair, then the displayed comparison holds. For a quick proof, make independent and set . Apply Gaussian integration by parts to . With softmax function weights and , the derivative of is . Since , let . Singular covariance matrices follow by adding identical small independent normal noise to both vectors and taking a limit.
Thermal-product input-distribution check 2026-11-05
A thermal-product input-distribution check tests whether the probability model used for a nonlinear thermal correction describes the available temperature and material information. In , with coefficient difference and temperature offset , the uncertainty depends on their joint behaviour, not just on two separately quoted standard uncertainties.
Start with the physical inputs: which thermometer calibration is shared, how coefficient estimates were obtained, and whether either quantity was inferred using the other. Translate those relationships into a joint distribution or an explicit dependence model. JCGM 101 distinguishes assigning input distributions from propagating them through a measurement model; numerical sampling cannot supply missing physical information.[ref]
An original counterexample shows why zero covariance is insufficient for the independent-input formula. Let be standard normal and defineBoth inputs have zero mean, with variances and , and their covariance is zero because the required Gaussian odd moments vanish. They are nevertheless dependent: is determined by .
Their product has zero mean and varianceusing Gaussian moments , , . Independence would instead give . With , and , the resulting standard deviation is about 22.36 nm rather than 10 nm. This is a deliberately constructed probability example, not a claim about typical thermometers.
Thermal-product dependence beyond covariance matters because higher moments enter the product variance. A multivariate normal distribution is a particular joint model whose moments are determined by its means and covariance; arbitrary inputs sharing those same summaries need not behave identically.
Check units, support, marginal shapes and dependence before simulation. Compare exact or limiting cases with a Monte Carlo method, and retain a sensitivity analysis for plausible alternative input models. The GUM's first-order rule requires higher-order treatment when nonlinearity matters.[ref] A small simulation error measures numerical precision conditional on the model, not proof that the model is physically correct.
Variational approximation gap 2026-11-05
The variational approximation gap is the part of the evidence lower bound deficit that remains even after optimizing separately for an observation within a specified variational family. With the generative model fixed and finite , defineThe equality uses the usual evidence lower bound identity for variational inference. A supremum allows for a best value that is not attained. The amortization gap separately measures failure of the shared inference rule to reach this value.[ref]
Inference Suboptimality in Variational Autoencoders by Chris Cremer and others. Table1; §3.1, equation8, decomposition for the ordinary variational-autoencoder bound
An original example shows what independence can cost. Stipulate a centred two-dimensional multivariate normal distribution as the posterior distribution, with covariance matrixRestrict the variational family to centred independent normal distributions, with . This is a simple mean-field variational inference restriction. Writing , the Kullback-Leibler divergence between normal distributions becomesEach variance contributes a term , minimized at . ConsequentlyFor , the best independent variances are both , and the unavoidable deficit is about nat. Matching both marginal variances to one instead would give about nat. The reverse Kullback-Leibler divergence therefore chooses narrower independent marginals here, while still failing to represent the correlation.
Enlarging the family to include the full covariance matrix permits and reduces this example's gap to zero. More generally, inclusion cannot increase the optimum approximation gap at a fixed model. Merely training the same shared rule longer cannot remove a strictly positive family restriction. Numerical improvements are not proofs that the infimum has been reached.
Wishart distribution Created 2026-10-14 Updated 2026-10-25
For integer , the sum of independent vectors has a Wishart distribution. Its positive-definite density extends to real and is proportional to . It is a conjugate prior for the precision matrix of a multivariate normal distribution. Its mean is .
The WinBUGS declaration
dwish(R, nu) uses ; the density convention is documented in the NIMBLE BUGS-language manual. For it is proper, while the expected inverse matrix does not exist: that expectation requires . Specifying a Wishart prior on precision is therefore a substantive covariance assumption.