The closure property of positive-semidefinite kernels gives three constructions, each seen directly from the feature maps .
- Nonnegative sums: , for . Its feature map is in the Hilbert space direct sum; expanding its inner product gives .
- Pointwise products: . In the Hilbert tensor product, use : the inner product of two such elementary tensors is the product of their two inner products.
- Input-dependent weighting: for any real-valued function . Its feature map is , so signs of are allowed. Nonnegative scalar multiplication is the special case of constant ; the weighting factor is itself a rank-one positive-semidefinite kernel.
Another useful consequence is pullback: if and is a positive-semidefinite kernel on , then has feature map .
For every , every choice , and every coefficient vector , the required condition isThus for the kernel matrix . For a feature map, this follows fromIn kernel methods, the term positive-definite kernel commonly uses this non-strict inequality. Strict positive definiteness is the stronger requirement that the expression be positive for distinct inputs and every nonzero ; it is not required here.
In this setting a positive-semidefinite kernel is an inner-product similarity: there exist a real Hilbert space and a feature map such thatThe feature map may be infinite-dimensional. Equivalently, is symmetric and every finite kernel matrix is a positive semidefinite matrix. The equivalence follows by forming the canonical feature space of a positive-semidefinite kernel and taking its Hilbert space completion.
The printed minimization of the evidence lower bound has no solution. To distinguish that literal result from the usual variational inference optimum, put . The preceding Kullback-Leibler divergence satisfiesIts unique global minimum occurs at . Since , this is the unique global maximum of the evidence lower bound:The optimal common variance is the harmonic mean of the two posterior variances. However, as either or . ThereforeThe source PDF confirms the minimization wording; replacing it by maximization would give the finite value above, consistently with part (i).
The independent coordinates make the logarithm of the probability density ratio a sum. For coordinate , its contribution to the Kullback-Leibler divergence isbecause . Thus, retaining all constants,Here and , so both normal densities are nondegenerate.
Use the factorization inside the evidence lower bound:HenceThe marginal likelihood is independent of the variational parameter , so maximizing the evidence lower bound is equivalent to minimizing the Kullback-Leibler divergence. Nonnegativity of the Kullback-Leibler divergence also gives .
For under the posterior density ,The independent-draw variance of a sample mean therefore yieldsSincewe again obtainThe optimum is again generally unavailable as an evaluable normalized density without knowing .
A proposal with too much mass where the posterior density is tiny makes reciprocal weights unstable. In particular, proposal tails that are too heavy relative to the posterior distribution can make infinite. This contrasts with ordinary importance sampling, whose dangerous denominator is . Choosing a normalized restricted to a well-covered posterior region is allowed, although a very small region can make most draws contribute zero and increase variance. Taking gives the Harmonic mean estimator of Bayesian model evidence, illustrating why very small likelihood values can cause severe instability.
For reciprocal importance sampling, use the same and but draw from the posterior density . Here the support requirement is reversed: the normalized density must vanish almost everywhere that . Thus may be confined to a subset of the posterior distribution's support. For ,Consequently,It is the estimate of the reciprocal that is an unbiased estimator. Taking its reciprocal does not generally produce an unbiased estimator of : inversion is nonlinear. In fact, when the average is strictly positive and the relevant expectation exists, Jensen's inequality gives .
With , , and as above, independence of the draws gives the variance of a sample mean:This formula allows infinite variance. To identify the optimum, write . ThenThe last equality uses . Its nonnegative integrand provesIndeed, all importance weights then equal . This is a formal optimum: evaluating the normalized posterior density generally already requires the unknown marginal likelihood.
Poor coverage of regions with substantial posterior mass produces large weights. In particular, excessively light proposal tails, a proposal that is too narrow, or a missed mode can make enormous or infinite. A proposal that is unnecessarily diffuse can also waste nearly all draws on negligible posterior probability. Omitting target support altogether violates unbiasedness, rather than merely increasing variance.
Put and . The support condition for importance sampling is essential: almost everywhere that . For one draw , the importance weight satisfiesBy linearity of expectation,This proves that it is an unbiased estimator. If the proposal omits a region of positive target mass, the expected value instead misses that region's contribution; the unconditional unbiasedness claim needs the stated support condition for importance sampling.
Evaluate Bayes' theorem at the posterior mode :The multivariate normal density in the Laplace approximation has height at its mean. ThereforeEquivalently, the approximate log marginal likelihood isThe determinant factor accounts for the volume of the locally plausible parameter region, which the peak likelihood alone does not measure.
Write . Choose an interior maximum a posteriori estimate, a globally maximizing posterior modeSuppose is twice continuously differentiable near and its negative Hessian matrix is a positive-definite matrix. The second-order Taylor expansion isExponentiating and normalizing this local approximation gives the Laplace approximationThus the approximate mean is the posterior mode, and the approximate covariance matrix is the inverse local curvature. This normal approximation is useful when most posterior probability is concentrated around this mode. A boundary mode, singular curvature, or several substantial modes requires a modified approximation.
The marginal likelihood integrates the likelihood function against the prior density:It is the normalizing constant in Bayes' theorem. For the subsequent posterior distribution and importance sampling calculations, assume a normalized prior density and .
Let count the parallel neighboring pairs, or frustrated bonds in an antiferromagnetic Ising ring. The periodic boundary conditions implyThere are antiparallel pairs, so : the number of antiparallel pairs must be even. For odd , must consequently be odd. Each frustrated bond in an antiferromagnetic Ising ring raises the energy by relative to a satisfied pair, givingThe minimum is attained at . Choose its position among the labeled bonds, choose either orientation of one Ising spin, then alternate across every other bond. This constructs exactly two spin configurations per position. ThusThis is geometric frustration: every local interaction prefers opposite Ising spins, but the closed odd ring prevents all preferences being satisfied at once. No particular location is preferred for the single frustrated bond in an antiferromagnetic Ising ring.
Each neighboring pair of Ising spins contributes at least to the energy, with equality exactly when the two Ising spins are opposite. For even , alternation closes consistently under the periodic boundary conditions. The ground states are thereforeHere is the ground-state degeneracy. Choosing fixes every other Ising spin, so these are all the ground states; translations do not give additional spin configurations. The two ground states are exchanged by spin inversion symmetry.
Take , so neighbouring operators act at distinct sites. The identityexpresses the Hamiltonian operator of this quantum XY chain in spin ladder operators. Substituting the Schwinger boson representation gives the requested quartic expression:Operators at different sites commute, allowing normal ordering without a correction here. For the first quartic term, the inverse discrete Fourier transforms giveHere is one when its two momenta agree modulo and zero otherwise; it expresses lattice momentum conservation. The other quartic term has the same ordered operators with phase . On the support of this Kronecker delta, modulo , so the two phases sum to . Thus the Fourier vertex of a Schwinger boson chain isAn equivalent three-sum expression sets modulo , so its coefficient is . The factor comes from four factors and one site sum, while the factor of two from the two exchange terms cancels the original . The formula is Hermitian: conjugation exchanges with and with , and the coefficient is unchanged after imposing lattice momentum conservation. It also preserves each local total boson number in its real-space form, hence preserves the physical constrained subspace.
For the specified positive-exponent discrete Fourier transform, the inverse discrete Fourier transforms have negative exponents:Every sum runs over , . Indeed, orthogonality of roots of unity givesso substituting the forward discrete Fourier transform into either proposed inverse recovers the original operator. Taking Hermitian conjugates supplies the inverse formulas for the creation operators.
Multiplying the inverse discrete Fourier transforms gives the local number operator combinationThis local expression includes off-diagonal momentum terms. Only after summing over sites does orthogonality of roots of unity giveIn a physical fixed- state the first boxed operator acts as at every site, by the local spin constraint in the Schwinger boson representation.
Let count the unequal neighbouring pairs, or domain walls. The energy is . A single domain wall is impossible with periodic boundary conditions: following all the other, equal-state bonds around the ring would force its endpoints to be equal too. Two domain walls are possible, splitting the ring into two nonempty uniform arcs carrying different states. Thus the first excited state hasChoose the two wall bonds in ways. For that choice, identify one of the resulting arcs by its smallest site label. Its state has choices and the other arc has . This counts each configuration exactly once, giving the two-domain-wall degeneracy of a Potts ringFor example, gives and ; the remaining two configurations are alternating and have four domain walls. The formula also covers , where the specified ring energy counts the two directed neighbouring bonds, and an unequal pair breaks both. If or , every allowed configuration is a ground state and there is no first excited state. For , the number of domain walls need not be even: three distinct states on a three-site ring give three, although one remains impossible.
Assume and ; the trivial exceptions are described below. In this periodic Potts chain, every satisfied bond contributes to the energy, and no bond can contribute less. All bonds can be satisfied simultaneously by choosing the same state at every site. Conversely, satisfying every bond forces this uniform configuration. Therefore the ground state energy and degeneracy factor areThe choices are distinct configurations on the labelled lattice: a global relabelling of the states is a symmetry, not an identification of microstates.
For commuting real scalar fields, remove the closed fermion loop sign and replace the Dirac propagators by scalar propagators. There are no spinor indices, hence no matrix trace. With two independent real scalar fields and interaction , the same normalization givesThe overall sign has reversed relative to part (i) because is now unaccompanied by a closed fermion loop sign. The same formula holds in the source's kernel convention, since there are again four propagators.
There is an additional convention to specify if the two symbols are intended to denote the same real scalar field. For the literal interaction , there are two identical connected Wick contractions between and , so the coefficient becomes . If the conventional interaction is instead , each vertex is and the loop has symmetry factor , giving coefficient . In both interpretations the closed fermion loop sign and spinor matrix trace disappear; the differing numerical factor comes from the interaction normalization and field identity.
There are unlisted articles, also show them or only show them.