EM likelihood monotonicity Created 2026-10-05 Updated 2026-10-14
For , the Jensen inequality givesThus any expectation-maximization algorithm M-step that increases cannot decrease the observed-data likelihood function.[ref]
For a short proof, assume common support and finite expressions, and keep the E-step conditional distribution fixed. The concavity of the logarithm and Jensen's inequality giveThis evidence lower bound equals . At , the ratio inside the expectation is the constant , so equality holds. During the M-step, the second term is constant; increasing raises a lower bound that touches the current log-likelihood. HenceIncreasing likelihood values alone do not establish convergence of the statistical parameters or attainment of a global maximum. In a differentiable finite mixture model, the gradient of equals the candidate-parameter gradient of evaluated at the current fit. Thus an interior fixed point of an exact M-step is a stationary point, which need not be a local maximum; the Identical-mean EM trap is an explicit counterexample. Small changes in log-likelihood can also occur before the means have settled, so inspecting both is useful.
For independent one-stop negative multinomial distribution observations, the natural estimates are and . Their sum is with strictly concave , so Jensen's inequality makes its expectation strictly below . Conditional category proportions split this bias between both estimates. With zero observed counts of a category, that estimate lies at zero; in a parameter space requiring strict positivity the likelihood supremum is not attained.
Independent-set bound for biregular graphs 2026-10-14
For a biregular graph with left vertices of degree and right vertices of degree , let be the indicator vector of a uniformly selected independent set. Use base-two information entropy. Shearer inequality gives . Conditional on , each right vertex with an empty selected neighbourhood is an independent fair binary choice, and the other right vertices are forbidden. Writing gives . For the neighbourhood patterns, assign weight to the zero pattern and weight one to all others. Jensen's inequality implies . Since , the bound follows from . If divides , the disjoint union of copies of the complete bipartite graph attains equality: each component has independent sets.
Integral fluctuation theorem 2026-10-14
Normalization of the reverse law gives an exact finite identity:The sum notation describes discrete paths; the corresponding path integral has the same change-of-variable argument. Jensen's inequality for the exponential function then gives , hence . Equivalently this mean is the Kullback-Leibler divergence between the forward path law and the reverse path law pulled back by . Equality requires those two laws to agree. [ref]
Entropy production along a stochastic trajectory and an integral fluctuation theorem by Udo Seifert. Seifert, 2005, Eqs. (17)–(18): integral relation and its mean inequality
Rare negative entropy production contributes disproportionately to this expectation through . Missing such events in a finite sample can spoil an estimate of the identity. Matching probability support is also essential: reverse paths absent from the forward law cannot be recovered by cancelling forward weights. This Second law of thermodynamics statement concerns an ensemble average; an individual negative value is compatible with it.
Jarzynski equality 2026-10-14
Integrating the Crooks fluctuation theorem over work and using normalization of the reverse probability distribution givesConsequently Jensen's inequality gives . Some realizations may nevertheless have . This is a work relation with equilibrium endpoint preparations, while the integral fluctuation theorem for total entropy production uses the actual final forward distribution. [ref]
The Entropy Production Fluctuation Theorem and the Nonequilibrium Work Relation for Free Energy Differences by Gavin E. Crooks. Crooks, 1999, Eq. (3) and derivation from Eqs. (10)–(11)
Jensen gap 2026-10-14
The Jensen gap is the nonnegative difference in Jensen's inequality, under its integrability assumptions. It quantifies how much applying a convex function before averaging differs from applying it afterward. For it is the variance of ; for an affine function it is zero. For general convex functions, it depends on both the function and the distribution of , and is not determined by the variance alone.
Jensen's inequality 2026-10-14
Jensen's inequality says that applying a convex function to a weighted mean cannot exceed the same weighted mean of its outputs. For nonnegative weights with and inputs in a convex set,Nonnegative weights are essential: this is a convex combination, rather than an arbitrary affine combination. For two inputs it is precisely the defining inequality for a convex function; repeatedly grouping inputs gives the finite weighted version.
Take , weights and , and inputs and . Their weighted mean is , so the two sides areThe Jensen gap is . The inequality measures the effect of replacing distinct inputs by one representative input before applying a nonlinear function.
The probabilistic form uses an integrable random variable taking values in an interval where is a convex function. If is also an integrable random variable and belongs to that interval, thenThe direction reverses for a concave function. For a strictly convex function, equality in the finite weighted form requires all inputs having positive weight to be equal; in the probabilistic form it requires to be constant almost surely. An affine function gives equality for any admissible distribution. [ref]
https://stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf. Boyd and Vandenberghe's slides, section 3.14, state the expectation form
.For the quadratic function and a random variable with a finite second moment, the difference between the two sides is exactly its variance:Consider two equally likely values and . Their expected value stays at as increases, but their variance is . Thus the expected value of the square grows from to . For the exponential function, the same two-point distribution gives , while . Both are examples of Jensen's inequality, but their Jensen gaps depend differently on the spread of the inputs.
Another useful choice is the concave function on positive inputs. Its version of Jensen's inequality gives . Applying the increasing exponential function shows that the weighted geometric mean cannot exceed the weighted arithmetic mean. For the equally weighted inputs and , these are and . This recovers the arithmetic-geometric mean inequality and illustrates how choosing one function can generate an entire family of bounds.
A short proof idea explains the probabilistic form. If has a subgradient at , the supporting affine function gives . Taking the expected value cancels the linear term because . This argument applies, for example, to finite convex functions on ; more general domains of a function need attention to endpoints and the existence of a subgradient.
Johan Jensen's 1906 paper “Sur les fonctions convexes et les inégalités entre les valeurs moyennes” is an early source for the connection between convex functions and inequalities for means. [ref]
https://zenodo.org/records/2371297. Jensen's original article and publication record, DOI 10.1007/BF02418571
. L2-compatible initialization of grid data 2026-10-14
An arbitrary initial function has no well-defined nodal point samples. Cell averages or a bounded projection to the discrete approximation space supply grid data independently of its pointwise representative. Jensen's inequality bounds the cell-average discrete L2 norm by the continuous L2 norm. Combining consistent bounded initialization, stability, and density of smooth data can extend convergence to rough initial functions, without claiming their nonexistent classical derivatives.
Lyapunov moment inequality Created 2026-10-06 Updated 2026-10-14
The Lyapunov moment inequality says that higher positive absolute moments give no smaller absolute moment sizes on a probability space. For a real random variable and ,allowing an infinite value. For the left expression is an Lp norm; below one it is a quasi-norm, so the usual triangle inequality terminology needs that distinction. The comparison uses a probability measure of total mass one; arbitrary unnormalized measure spaces need a factor depending on their total mass.
For a nonnegative count, the first moment size is its expected value, while the second is the square root of its second moment. Both ten-trial counts in second moment method have first moment size one. The independent count has second moment size , whereas the shared-outcome count has second moment size . The second moment distinguishes models with the same average because its square increases the influence of rare large counts.
More generally, for with a Bernoulli random variable of parameter ,It rises from one at toward ten as . Replacing by a parameter gives , which decreases toward zero for a fixed positive exponent when , making the rare-event and large-exponent limits different questions. For a scaled Rademacher random variable taking values and two with equal probabilities, every absolute moment size is exactly two: increasing the exponent does not distinguish signs when the absolute value is constant.
Lyapunov moment inequality reveals how higher moments emphasize large outcomes and when equality holds
. Left: the two original ten-trial models from second moment method have the same mean one, but different absolute moment sizes at exponents above one. The shared model has with . Right: equal masses at and two keep almost surely, giving equality for every pair of positive exponents. These are fully specified mathematical probability models. Details.For finite , apply the reversed Jensen's inequality to the strictly concave function and the nonnegative random variable :Taking the positive th root proves the claim. Equality in this finite case holds exactly when is constant almost surely. If the higher moment is infinite, the upper bound is infinite and remains valid. Exponent zero is excluded from the statement; logarithmic or geometric limiting summaries require separately stated conventions, especially when can vanish.
Past exam of the mathematics course of the University of Cambridge 2001 iii Paper 9 2 Solution Created 2026-10-03 Updated 2026-10-14
Use the Nevanlinna proximity function and the Nevanlinna integrated counting function for poles, counting multiplicity. If the pole order at zero is , thenBoundedness of the Nevanlinna characteristic refers to , equivalently to a uniform bound on for any fixed . This avoids the harmless origin normalization when .
The Nevanlinna first main theorem states, for every fixed finite value and every nonconstant meromorphic function on the disk,For , this is the definition of . The error is bounded independently of . Translation changes by , while Jensen's formula gives when near zero; these observations explain the first theorem.
The Nevanlinna second main theorem in the unit disc states that for distinct values on the Riemann sphere,as outside an exceptional set satisfying . The truncated Nevanlinna counting function counts each distinct preimage once. The boundary-distance error cannot simply be suppressed: on the disk it need not be . The disk error and exceptional-set estimates follow from the logarithmic-derivative estimates in mathweb.tifr.res.in/Documents/Publications/Lectures/tifr17.pdf .
Suppose first that is bounded. Since , its poles satisfy the Blaschke condition, counting their orders: let in their integrated count and use . Let be their Blaschke product, including the origin factor if necessary. Then extends to a holomorphic function on the whole disk, and givesHere the last bound follows from bounded and the uniform lower bound on .
The nonnegative subharmonic function has a finite harmonic majorant. Indeed its Poisson integrals on expanding disks dominate it by the maximum principle for subharmonic functions, increase with , and have . The Harnack inequality for harmonic functions bounds them on every compact subdisk, so their increasing limit is a finite harmonic function . Since the disk is simply connected, has a harmonic conjugate, giving a holomorphic function with . ConsequentlyThis proves the constructive half of the quotient characterization of bounded characteristic.
Conversely, if with bounded holomorphic functions and , each pole of is a zero of of at least the same order. The elementary inequality and the Nevanlinna first main theorem giveAny cancellation at an origin zero contributes only a bounded multiple of on , which is included in this . The zero function is immediate; constant nonzero is handled directly. Thus bounded characteristic is exactly a quotient of two bounded analytic functions.
An unbounded example with bounded characteristic isIt is a ratio of the bounded holomorphic functions and . More explicitly, Jensen's formula gives the mean of as zero, so , although .
Finally, set . Pointwise . Jensen's inequality for the concave logarithm givesThere is no pole term because is holomorphic. Hence the stated integral bound implies bounded characteristic. This argument in fact works for every .
Past exam of the mathematics course of the University of Cambridge 2002 iii Paper 57 3 a Solution Created 2026-10-03 Updated 2026-10-14
Use fractions of all farms, so and . Assume farm size is fixed, contacts mix randomly, infectious periods have size-independent exponential mean , and the farm-size distribution has finite moments and . Write . The farm-size stratified SIR model with susceptibility proportional to size isHere includes the common contact-rate constant; using frequency-dependent mixing avoids introducing an extra total-farm factor. An infectious farm of any size generates infections in initially susceptible size- farms. The next-generation matrix has rank one, giving the basic reproduction numberThe comparison SIR model replaces every size by , so it has the same basic reproduction number and the same initial exponential growth rate .
As the epidemic develops, put . For negligible initial infection, . Large farms leave the susceptible class faster. If with , differentiation givesThe mean size of newly infected farms is , larger than the susceptible mean unless all sizes agree. If , thenRecovery cancels from this equation because its rate is size-independent. Starting with representative infectives, initially rises toward the size-biased mean ; later depletion of large susceptible farms lowers and usually lowers . It is incorrect to assert that the mean size of infectives must decrease from every possible initial condition. In the growing linearized mode, , so its mean is already .
The aggregate growth equation is . The falling susceptible mean adds a depletion effect absent from the homogeneous SIR model. The final size relation for an epidemic, with vanishing initial seed and attack fraction , isBy Jensen's inequality, the right side is at most , so is no larger than the homogeneous attack fraction. For a common small representative seed the illustrated epidemic is flatter and infects fewer farms; exact pointwise ordering of infected fractions for every time and arbitrary initial seed is not a consequence of equality of .
Farm-size heterogeneity: epidemic prevalence and mean livestock counts for the three transmission assumptions
. Details. The original sketch uses equally many farms of sizes 1 and 3, recovery rate , and a representative infected seed of . Each homogeneous comparison has . The left, middle and right columns correspond to parts (a), (b) and (c). The top row shows infected farm fractions, and the bottom row distinguishes the mean sizes of susceptible and infected farms. These are illustrative trajectories under the stated model, not universal empirical outbreak curves.
Past exam of the mathematics course of the University of Cambridge 2004 ib Paper 1 21H Solution Created 2026-09-24 Updated 2026-10-14
The Rao-Blackwell theorem states that if an estimator has finite second moment and is a sufficient statistic, then can be chosen without knowing the parameter. It has the same expectation as and no larger variance. In particular an unbiased estimator remains unbiased and its squared-error risk cannot increase. Sufficiency makes the conditional distribution, hence the function of used here, parameter-independent.
The tower property gives . The law of total variance givesso the variance decreases, with equality exactly when is already a function of almost surely. Since the bias is unchanged, the same identity proves the squared-error risk claim. More generally the corresponding convex-loss inequality follows from conditional Jensen's inequality.
For the given geometric distribution, the joint mass is on positive-integer sample vectors. Thus is a one-dimensional sufficient statistic. Directly, conditional on , every positive-integer composition of into parts has the same probability; there are such compositions, independently of .
The simple estimator is unbiased for , since . For , compositions with are counted by subtracting one from their first part, giving when . Consequently the Rao-Blackwell estimator of a geometric failure probability isAt its value is zero, as it must be when all observations are one. For the possible one-observation case, use ; the displayed fraction would otherwise be undefined at . The conditional-expectation argument proves unbiasedness without needing to sum the negative-binomial mass explicitly.
Past exam of the mathematics course of the University of Cambridge 2004 iii Paper 11 6 Solution Created 2026-10-03 Updated 2026-10-14
Use discrete Shannon entropy with base-two logarithms. The standard facts needed are the chain rule for information entropy, conditioning reduces entropy, nonnegativity of conditional entropy, and the fact that a uniform distribution on outcomes has Shannon entropy .
Order the coordinates by their indices and put . Applying the chain rule within a coordinate subset and then adding the missing earlier conditioning coordinates givesOn summing over the family, coordinate occurs at least times. Since each is nonnegative,This proves Shearer's inequality without any independence assumption on the coordinates. For countable alphabets, apply the finite-valued assertion to refining finite coordinate partitions; the standard monotone approximation of Shannon entropy gives the same result with extended values. Differential entropy is not the Shannon entropy intended here.
For the graph application, let its bipartition be , with degrees on and on , where . Choose an independent set uniformly from all such sets, and let be its vertex-indicator vector. Then . For put and .
Each vertex of belongs to exactly of these neighbourhoods. Shearer's inequality therefore gives . Conditional on , a vertex of is forced absent if any neighbor was selected, and otherwise is a free fair binary choice. There are no edges within , and the conditional independent-set distribution is uniform on all these free choices. Henceand
There are possible neighborhood patterns. Give the zero pattern weight and every other pattern weight one. For its distribution , concavity of the logarithm, or Jensen's inequality, givesThis establishes the local weighted-entropy estimate rather than just counting neighborhood patterns.
Edge counting gives ; together with , it yields . Exponentiating the Shannon entropy estimate proves the independent-set bound for biregular graphs:The complete bipartite graph has exactly independent sets, since one chooses a subset of one side and subtracts the double-counted empty set. Disjoint unions of such components attain equality. If exactly one degree parameter is zero, edge counting forces the graph to be edgeless and the displayed bound equals , so it remains valid. If , the denominator is undefined; the edgeless graph has exactly independent sets.
Past exam of the mathematics course of the University of Cambridge 2006 ia Paper 2 3F Solution Created 2026-09-24 Updated 2026-10-14
A real function on an interval is a convex function iffor all in its domain and . For a finite-valued random variable with values and probabilities , Jensen's inequality statesApply this with equal probabilities to and the convex function on the nonnegative axis:Multiplying by proves the sharp two-term convex power bound. Conversely taking forces . ThusFor the inequality is equality for every pair; for equality occurs when .
Past exam of the mathematics course of the University of Cambridge 2008 ib Paper 2 19H Solution Created 2026-09-24 Updated 2026-10-14
Write . Differentiating the probability generating function at givesExpanding its denominator as a geometric series and then applying the binomial theorem givesCoefficient identification proves , a one-stop negative multinomial distribution.
For an independent identically distributed sample let and . Ignoring parameter-independent factors, the log-likelihood is . When both sums are positive, its stationary equations are and . Together with , these giveThe Hessian is , negative definite in this case, so the stationary point is the unique global maximum.
There is an important boundary qualification: if , the derivative with respect to is , so the likelihood increases as decreases to zero. The displayed formula then gives a boundary estimator, not an attained MLE in the stated open parameter space . The analogous issue holds for . The formulas are MLEs on the extension allowing zero category probabilities, and interior MLEs when both counts are positive.
For bias, put and . Then and with . Its second derivative is , and is nondegenerate. Strict Jensen's inequality givesIn fact both estimates are downward biased. Conditional on , the first count is binomial with success probability ; hence conditional on , is also binomial with that probability. ThereforeThis proves the finite-sample bias in a negative multinomial probability estimator, using the boundary-extended formulas whenever necessary.
Finally, the strong law of large numbers applies because the nonnegative variables have finite means: almost surely and . Continuity of the estimator function then givesand similarly for . Thus the estimates are strongly consistent despite finite-sample bias. Almost surely both sample sums eventually become positive, so the open-parameter MLE issue affects finite samples rather than this asymptotic conclusion.
Past exam of the mathematics course of the University of Cambridge 2008 iii Paper 39 2 iii Solution Created 2026-10-03 Updated 2026-10-14
Let be the unique equivalent martingale measure. The risk-neutral pricing formula isFor the usual nonnegative strike and maturities , the monotonic bank account gives . ThereforeThe second inequality follows directly since the conditional expectation of a positive part is at least both zero and the conditional expectation of its argument; it is also conditional Jensen's inequality. Taking expectations proves , the maturity monotonicity of calls with nonnegative strikes.
If , then pointwise; positive discounting and expectation prove monotonicity of a European call price in strike. For , the pointwise inequalitygives convexity of a European call price in strike after the same operations. HenceThe maturity assertion uses ; it need not hold for a negative strike. For example, in the deterministic complete market with , a negative-strike call has price , which decreases with . The monotonicity assertions are non-strict, as equality can occur.
Past exam of the mathematics course of the University of Cambridge 2010 ia Paper 2 3F a Solution Created 2026-09-24 Updated 2026-10-14
For , on , so is a convex function; for it is linear and also convex. The equally weighted random variable has expected value . Applying Jensen's inequality givesMultiplying by provesFor equality requires all , by strict convexity; for equality holds for every admissible vector.
Past exam of the mathematics course of the University of Cambridge 2010 iii Paper 12 4 i Solution Created 2026-10-03 Updated 2026-10-14
Let be the number of independent sets of the biregular graph . Choose one such set uniformly and write . All information entropies below use logarithms to base two, soWe will prove .
We state and justify the entropy facts used. For a finite-valued random variable , with . Its conditional entropy is the mean of this quantity over the conditioning variable. Factoring the joint probabilities proves the chain rule for information entropy, . Conditioning reduces entropy: . To see this, condition on and use concavity of to compare the mixture of the conditional distributions given with their mean entropy.
The precise Shearer inequality needed here is: if subsets of the indices of a finite random vector cover every index at least times, then . To prove it, order the indices. The chain rule for information entropy expandsThe inequality is conditioning reduces entropy. Summing over counts each nonnegative conditional term at least times and proves the assertion.
For every , its vertex neighbourhood has vertices. Every lies in exactly of these neighbourhoods. Hence Shearer inequality givesLet , the probability that no neighbour of is selected. Conditional on a particular selected subset of , a vertex with a selected neighbour is forbidden, and all other vertices of are available. Because there are no edges within , every subset of the available vertices is a valid completion. Uniform choice of makes these completions equally likely. The available coordinates are therefore independent fair binary choices, each of entropy one, and the forbidden coordinates are zero. Averaging their number givesBy the chain rule for information entropy,It remains to bound each bracket. Let , a binary vector of length , with distribution on , and put . Assign weight to the all-zero vector and weight to every nonzero vector. ThenThe first inequality is Jensen's inequality for the concave logarithm; the weights sum to . This explicitly handles zero probabilities and does not assume the neighbourhood indicators are independent.
There are right vertices, so combining the bounds and using givesExponentiating proves the required independent-set bound for biregular graphs. The proof uses both degree constraints: supplies the neighbourhood-cover multiplicity and the conditional-completion weight, while determines the number of neighbourhood patterns.


