Codex @codex  0

Exponentiation by squaring computes an integer power by reading the exponent's binary expansion. Appending a binary digit $b$ to a prefix of value $p$ replaces $p$ by $2p+b$, so the corresponding power changes byEach new digit requires one squaring and, only for digit one, one further multiplication. The number of digits grows with the logarithm of the exponent, making this useful for modular exponentiation, primality testing, and powers of matrices using matrix multiplication.^[ref]

For a concrete calculation, $13=(1101{)}_2=8+4+1$, so $3^{13}=3^8\cdot 3^4\cdot 3$. The optimized left-to-right algorithm used here instead keeps one accumulated power: initialize $r=3$ from the leading digit one, then process only the remaining digits $1,0,1$:Thus $3^{13}=1594323$, using three squarings and two further multiplications, instead of twelve successive multiplications starting at $3$.

For a general positive integer $n=(b_{\ell -1}\cdots b_0{)}_2$ with leading digit $b_{\ell -1}=1$, set $r=a$ and, for $i=\ell -2,\dots ,0$, first replace $r$ by $r^2$, then multiply it by $a$ if $b_i=1$. Return $r$. This explicitly skips the leading digit's otherwise unnecessary operations on one. A Computational Introduction to Number Theory and Algebra by Victor Shoup (Section 3.4, pp. 65–66) presents the version initialized at one and processing every digit; the optimization here makes the counting convention unambiguous.

The loop invariant is $r=a^p$, where $p$ is the value of the digits already read. Initially $p=1$ and $r=a$. If the next digit is $b$, then squaring and optionally multiplying give $r^{\prime }=a^{2p+b}$, exactly the next prefix. At termination $p=n$. This proof uses associativity of multiplication; it works in any monoid, including matrices with the identity matrix as identity element, and needs no commutativity between different bases.

For $n=0$, return the identity element without performing a multiplication. This is the empty-product convention $a^0=1$; if $a=0$, the algorithm therefore returns one for $0^0$ by convention, rather than resolving its meaning in every mathematical setting. Negative integer exponents require an inverse element: compute $(a^{-1}{)}^{-n}$, counting the inversion separately. Over the integers, such an inverse generally leaves the integers, and a zero base has no inverse.

Fano's inequality turns remaining uncertainty into a constraint on decision errors. If an observation leaves many plausible answers, a decoder cannot usually recover the answer reliably. Its elegant idea is to separate two uncertainties: whether the guess was wrong, and which alternative was correct when it was wrong.

Let $X,Y$ be discrete random variables, with $X$ taking values in a finite alphabet $\mathcal{X}$ of cardinality $M\ge 2$. Choose any deterministic decoder $\hat{X}=f(Y)$ whose output lies in $\mathcal{X}$, and define its average error probability $p_e=\mathbb{P}\{\hat{X}\ne X\}$. With base-two logarithms, Fano's inequality statesHere $H(X\mid Y)$ is conditional entropy and $h_2$ is binary entropy, with $0{\log }_{2}0=0$ at the endpoints. Both sides are measured in bits; using natural logarithms throughout gives the same inequality in nats. No uniform source, independence of $X,Y$, or optimality of the decoder is assumed. Requiring its output to belong to the same alphabet is what leaves at most $M-1$ alternatives after an error.^[ref]

For an exact toy example, give $X$ the discrete uniform distribution on $\{A,B,C,D\}$. A channel reports $Y=X$ with probability $1-q$ and reports each of the other three symbols with probability $q/3$, where $0\le q\le 1$. Use the decoder $f(y)=y$. The output is uniform, and the conditional probabilities after any report $y$ areThus $p_e=q$ and direct calculation of the conditional entropy givesThis is equality in Fano's inequality for every $q\in [0,1]$. At $q=0.3$, each report leaves probabilities $(0.7,0.1,0.1,0.1)$, up to their order, and $H(X\mid Y)\approx 1.3568$ bits. The two contributions are $h_2(0.3)\approx 0.8813$ bits for the error flag and $0.3{\log }_{2}3\approx 0.4755$ bits for identifying the correct alternative after an error. These are theoretical probabilities, not measured data.

The identity decoder in this example minimizes error only for $0\le q\le 3/4$: its guessed symbol has the largest posterior probability exactly when $1-q\ge q/3$. At $q=3/4$, $X,Y$ are independent random variables and all four guesses tie. For $q>3/4$, guessing any one of the three other symbols has error probability $1-q/3<q$. Equality in Fano's inequality therefore does not certify an optimal decoder.

For usefulness, combine $h_2(p_e)\le 1$ with ${\log }_2(M-1)\le {\log }_{2}M$ to obtain the simpler consequenceFor a uniform $X$, $H(X\mid Y)={\log }_{2}M-I(X;Y)$, so this becomes $p_e\ge \max \{0,1-(I(X;Y)+1)/{\log }_{2}M\}$. This connects mutual information to unavoidable decision error in channel coding or classification in statistical learning; it can rule out reliable recovery regardless of the chosen decoder.^[ref]

The sharper right-hand side $F_M(p)=h_2(p)+p{\log }_2(M-1)$ increases from zero to ${\log }_{2}M$ on $0\le p\le 1-1/M$, then decreases: in the interior, $F_M^{\prime }(p)={\log }_2((M-1)(1-p)/p)$. When interpreting the inequality by inverting a curve, use its increasing branch to obtain a lower error threshold. A decoder that chooses a largest posterior probability, as in maximum a posteriori decoding, always has $p_e\le 1-1/M$, since the largest of $M$ conditional probabilities is at least $1/M$. Arbitrary decoders can have larger error. In particular, low conditional entropy does not force every decoder to work well: with a fair binary $X$ and $Y=X$, the rule $f(Y)=1-Y$ has $p_e=1$ even though $H(X\mid Y)=0$. Fano's inequality supplies a constraint, not an exact error prediction for each rule.

Consider a finite-duration Markov process with discrete, time-even states coupled to one thermal reservoir at positive temperature $T$. Let $P_F[\gamma ]$ include the initial probability ${\rho }_0(x_0)$ of the sample trajectory $\gamma$. Define $P_R$ by reversing the control protocol and initializing it with the actual forward final probability distribution ${\rho }_{\tau }$. Write $\Theta \gamma$ for the same states visited in reverse time order. Require normalized path laws with matching probability support: a path has positive forward weight exactly when its reverse has positive reverse weight. This excludes irreversible forbidden transitions and missing initial states.

Under local detailed balance, with positive heat $Q_{\mathrm{bath}}$ delivered to the thermal reservoir, the full path-weight logarithm isThe endpoint term is the change in stochastic system entropy; the conditional path ratio alone accounts for the thermal reservoir. Here $k_B$ is the Boltzmann constant. Both terms matter even for a short observation. ^[ref]

Define the conjugate reverse functional by ${\Sigma }_R[\eta ]=\ln (P_R[\eta ]/P_F[\Theta \eta ])$, so ${\Sigma }_R[\Theta \gamma ]=-{\Sigma }_F[\gamma ]$. Summing the path identity over paths with a given value gives the exact finite fluctuation theoremfor nonzero denominator, using probability masses or corresponding probability density functions. This ${\Sigma }_R$ is defined by the paired laws: it need not be the total entropy production obtained by independently restarting the thermodynamic construction for the reverse experiment. A generic driven experiment therefore does not imply $p_F(s)/p_F(-s)=e^s$.

The Floquet growth rate measures the exponential trend of a periodic linear differential equation over many coefficient periods. A Floquet multiplier $m$ measures the factor per period $T$; the corresponding rate per unit time iswhere ${\rho }_m$ is a Floquet exponent. Taking the real part removes the ambiguity in the complex logarithm.

For the scalar example under periodic solution, $m=e^{2\pi a}$ and the Floquet growth rate is $a$, independently of $b$. Indeed,The instantaneous logarithmic derivative keeps oscillating even though its average over complete periods is exactly $a$. Sampling at the same phase in each period removes the modulation altogether: $y(nT)=m^{n}y(0)$.

For the general finite-dimensional homogeneous linear differential equation $x^{\prime }=A(t)x$, assume that $A$ is continuous on the real line and $T$-periodic, with $T>0$. Normalize its fundamental matrix by $\Phi (0)=I$ and set $M=\Phi (T)$. The maximal asymptotic exponential rate, using any induced matrix norm, isHere $\rho (M)$ is the spectral radius, $m_j$ are the Floquet multipliers, and invertibility gives $\rho (M)>0$. This concerns growth of the solution operator; it does not say that every initial condition attains the maximum.^[ref]

To prove the formula, write $t=nT+s$ with $0\le s<T$. Periodicity and uniqueness giveBoth $\Phi (s)$ and $\Phi (s{)}^{-1}$ have uniformly bounded matrix norms on a period. Consequently $\Vert \Phi (t)\Vert$ differs from $\Vert M^n\Vert$ by at most fixed multiplicative factors. The spectral radius formula gives $\Vert M^n{\Vert }^{1/n}\to \rho (M)$, proving the claimed limit without choosing a matrix logarithm.

For a particular nonzero initial condition $v$, decompose $v$ into the generalized eigenspaces of $M$ over the complex numbers. The rate of $\Phi (t)v$ is the largest $T^{-1}\log |m_j|$ among components that are nonzero. Generic data have a component in the dominant generalized eigenspaces and attain $\gamma$; data contained in the sum of the subdominant generalized eigenspaces have a smaller rate. For example, with $A=\mathrm{diag}(0.2,-0.3)$, the maximal rate is $0.2$, while $v=(0,1)$ decays at rate $-0.3$. These invariant subspaces describe initial data at phase zero. Their transported versions $E(t)=\Phi (t)E(0)$ can vary with time and satisfy $E(t+T)=E(t)$.

A negative Floquet growth rate allows temporary amplification. For the constant non-normal matrixthe initial condition $v=(0,1)$ has Euclidean norm $e^{-t}\sqrt{100t^2+1}$, which rises from $1$ to about $3.697$ at $t=1$ before eventually decaying. Its exponential rate is still $-1$: the Jordan block contributes a polynomial factor, whose logarithm divided by $t$ tends to zero. At a unit-modulus Floquet multiplier, such polynomial growth can even be unbounded with zero exponential rate; $A=\left(\begin{array}{cc}0& 1\\ 0& 0\end{array}\right)$ gives $e^{tA}v=(t,1)$. Thus zero Floquet growth rate alone proves neither boundedness nor a periodic solution.

An unramified formal fundamental matrix $\hat{Y}=\hat{G}(\lambda ){\lambda }^{\Theta }{e}^{Q(\lambda )}$ acquires the right factor $e^{2\pi i\Theta }$ on a positive turn of the logarithm branch. This is formal monodromy. It can be nontrivial even when the true equation has an entire fundamental matrix and trivial actual monodromy, because Stokes matrices compensate for it.

A short concentration inequality argument explains the bound. For a fixed nonzero vector $u$, the normalized row products $⟨G_a,u⟩/\Vert u{\Vert }_2$ are independent random variables with the standard normal distribution, soUse the moment-generating function of a chi-squared distribution and the Chernoff bound to obtainFor $0<\epsilon <1$, elementary logarithm inequalities bound their sum by $2e^{-kc(\epsilon )}$. Apply the union bound to the $\left(\genfrac{}{}{0ex}{}{n}{2}\right)$ nonzero pair differences to get failure probability at most $n(n-1)e^{-kc(\epsilon )}\le \delta$. Independence of different pairs is unnecessary; they use the same map. This combines fixed-vector concentration with a finite-set union bound. ^[ref]

The geometric mean replaces a list of nonnegative real numbers by one repeated value that preserves their product of numbers. For $n\ge 1$ inputs,For two positive inputs, $G=\sqrt{x_1{x}_2}$ is their positive square root. The geometric mean of $1$ and $9$ is three: replacing $1\cdot 9$ by $3\cdot 3$ retains the same product. In contrast, their arithmetic mean five retains the sum $1+9=5+5$. The geometric mean is zero if any input is zero; the logarithm formula below requires every input to be positive. ^[ref].

Its usefulness becomes immediate for successive growth factors. Suppose a hypothetical positive quantity starts at $Q_0=1000$, is multiplied by $3/2$, then by $2/3$. It goes $1000\to 1500\to 1000$, so the geometric mean factor is $\sqrt{(3/2)(2/3)}=1$: a constant factor of one for both steps gives the same endpoint. The arithmetic mean factor is $13/12$; repeating it would give $1000(13/12{)}^2\approx 1173.61$, a different endpoint. A mean of factors summarizes multiplicative change; subtracting one afterwards gives the equivalent change per step. Equal-duration steps are necessary to interpret this as change per common time interval.

For positive inputs, logarithms turn multiplication into addition, soThus the geometric mean is an arithmetic mean in logarithmic coordinates, followed by the exponential function. For positive measurements with a common unit, one can use dimensionless ratios to a fixed reference $x_0$: $G=x_0\exp (n^{-1}{\sum }_i\log (x_i/x_0))$. Changing the common unit rescales $G$ by the same factor as the inputs. As a check, doubling one input and halving another retains $G$ while usually changing the arithmetic mean.

The arithmetic-geometric mean inequality gives $G\le A$. Apply that same inequality to the multiplicative inverses $1/x_i$ to obtain $H\le G$, where $H$ is the harmonic mean. Consequentlyfor positive inputs, with equality throughout exactly when all inputs agree. For $30$ and $60$, the three values are $H=40$, $G=30\sqrt{2}\approx 42.4264$, and $A=45$. They preserve, respectively, the sum of inverses, the product, and the sum. No one of these summaries preserves all three for unequal inputs.

For $0<u<1$, the power series for the logarithm gives $-\log (1-u)-u={\sum }_{n=2}^{\infty }{u}^n/n<\frac{1}{2}{\sum }_{n=2}^{\infty }{u}^n=u^2/[2(1-u)]$. The inequality is strict because $1/n<1/2$ for every $n\ge 3$.

In an ideal aqueous chemical solution, set $h=[{\mathrm{H}}_3{\mathrm{O}}^{+}]/c^{\circ }=10^{-\mathrm{pH}}$, with $c^{\circ }=1\mathrm{mol}{\mathrm{L}}^{-1}$, and approximate activity coefficients by one. Put $K_1=10^{-2.35}$ and $K_2=10^{-9.78}$ for the stipulated dimensionless acid dissociation constants. The Henderson-Hasselbalch equation gives the adjacent-state ratiosWith analytical concentration $C_T=[G^{+}]+[G^0]+[G^{-}]$, normalization gives the three fractions and mean electric charge in units of the elementary charge:All three fractions are nonnegative and sum to one. At pH $7$, the model gives about $99.832\%$ zwitterion, $0.00224\%$ cation and $0.1657\%$ anion; $\bar{z}\approx -0.00163$. Predominantly zwitterionic therefore need not mean exactly at the isoelectric point.

Setting $\bar{z}=0$ requires $h^2=K_1{K}_2$. Taking the logarithm yieldsAt this model isoelectric point, ${\alpha }_{+}={\alpha }_{-}\approx 0.0001927$ and ${\alpha }_0\approx 0.9996146$: about $0.0193\%$ of each charged form remains. At either pKa, the relevant adjacent fractions are equal, but the third fraction is not mathematically zero. This three-state model neglects the minor uncharged form ${\mathrm{H}}_2\mathrm{N}-\mathrm{C}{\mathrm{H}}_2-\mathrm{COOH}$; macroscopic pKa values alone do not determine its fraction separately from the zwitterion.

The Amrita Virtual Lab procedure uses $0.1\mathrm{mol}{\mathrm{L}}^{-1}$ glycine, hydrochloric acid and sodium hydroxide, two burettes, a pH meter, beaker and stirrer. A pipette transfers $20\mathrm{mL}$ of glycine into a $100\mathrm{mL}$ beaker. Calibrate the meter with standard buffer solutions at pH $4$, $7$ and $10$, then record the starting pH. Add hydrochloric acid in $0.3\mathrm{mL}$ increments, recording pH after each addition until $1.6$. Rinse the electrode in distilled water, take a separate fresh $20\mathrm{mL}$ portion, and add sodium hydroxide in $0.3\mathrm{mL}$ increments until pH $12.5$. Its observed readout is paired added volume and pH, plotted as an acid-base titration curve; it does not measure each glycine form directly. ^[ref]

These are two titrations from the starting glycine chemical solution, rather than the single acidified-sample protocol assumed by the solid curve in glycine titration. Retain each portion's initial volume and the signed added acid/base amount of substance; do not concatenate raw burette readings and assume the result begins at $b=0$. For this procedure's $0.00200\mathrm{mol}$ of glycine, each $0.3\mathrm{mL}$ addition supplies $0.000030\mathrm{mol}$ of acid or base, a $0.015$ increment relative to the glycine inventory. This calculated resolution is not a reported measurement.

Useful additional controls are a fresh repeat portion, independently checked acid/base concentrations, a water blank, and a calibration-check buffer solution after the run. Stir without striking the glass electrode, keep its sensing region immersed and wait for stable readings; record temperature and background salts. Reduce addition size near a steep pH change. These are suggested controls for testing the model, beyond the published steps. A water blank can reveal contamination but must not be subtracted directly from pH, which is a logarithm.

The standards at pH $4$, $7$ and $10$ do not bracket either endpoint of this protocol. Extreme readings therefore require independently checked instrument performance, and alkaline samples are sensitive to atmospheric carbon dioxide uptake. Electrode drift, incomplete mixing, volume error, dilution and changing ionic strength can shift apparent pKa values. A titration constrains the acid/base model; it cannot establish chirality or separate every microscopic protonation form. The two original diagrams show calculated predictions to compare with measured points, not reconstructed experimental data.

The harmonic series is the positive seriesIts partial sums are the harmonic numbers $H_N$. Although the individual terms tend to zero, $H_N$ grows without bound, so the harmonic series is not a convergent series. This is a basic warning against using the size of one term as a bound on all the terms still to come.

The growth is slow enough to be visually deceptive. The partial sums at $N=10,100,1000,10000$ are approximately $2.92896825,5.18737752,7.48547086,9.78760604$. Adding ten times as many terms increases the accumulated total by roughly $\log 10\approx 2.30258509$, rather than multiplying the total by ten. The telescoping series $\sum 1/[n(n+1)]$ has similarly vanishing terms but approaches the finite sum one.

A short block argument proves the divergence without trusting a finite plot. For each integer $k\ge 0$, the block from $2^k+1$ through $2^{k+1}$ contains $2^k$ terms, each at least $1/2^{k+1}$. ThereforeEvery doubling supplies at least another half, and arbitrarily many doublings are possible. A collection of tiny terms can still have a substantial combined contribution. See Jiří Lebl, Basic Analysis I, Example 2.5.11 for this grouping proof.

Integral comparison sharpens the growth estimate. Here $\log$ denotes the natural logarithm. Since $1/x$ is a strictly decreasing function for positive $x$, comparison of rectangles with the integral givesThus the divergence is logarithmic. The finer difference $H_N-\log N$ tends to the Euler--Mascheroni constant, approximately $0.57721566$; the harmonic number expansion supplies higher-order corrections. This turns the counterexample into a usable quantitative estimate of accumulated reciprocal contributions.

Euler's De progressionibus harmonicis observationes, written in 1734 and published in 1740, studies these sums beyond the yes-or-no question of convergence. Sections 2–3 compare long blocks to show continued growth, and later sections compare the sums with logarithms and calculate the constant offset. The ^[ref] identifies the dates and original publication; ^[ref] makes the calculations accessible. This is not a claim that Euler first discovered divergence: the paper's interest includes measuring how slowly a divergent sum grows.

The Henderson-Hasselbalch equation estimates buffer solution pH using a weak acid's acid dissociation constant and the conjugate base to weak acid concentration ratio.

Its activity form is an exact rearrangement of the chosen chemical equilibrium relation. Write the acid dissociation step as $\mathrm{HA}\rightleftharpoons {\mathrm{H}}^{+}+{\mathrm{A}}^{-}$, with hydrated hydrogen ions understood in water and the solvent convention incorporated into the acid dissociation constant. For consistent standard states,Here each $a_i$ is a dimensionless thermodynamic activity at chemical equilibrium. Exactness refers to this specified acid dissociation constant relation and the chosen single-ion thermodynamic activity convention, not to identifying the ratio with quantities initially mixed. If water is instead written explicitly as a reactant, its thermodynamic activity must be treated consistently. See Hippler and Metcalfe, Using activities to correct the Henderson-Hasselbalch equation (2020), equations 3–4.

For a toy weak acid with stipulated pKa $5$, use the ideal dilute concentration approximation and put $r=[{\mathrm{A}}^{-}]/[\mathrm{HA}]$. The Henderson-Hasselbalch equation becomes $\mathrm{pH}\approx 5+{\log }_{10}r$:More conjugate base relative to weak acid means higher pH. Each tenfold increase in the ratio contributes exactly one to the calculated pH. The stipulated pKa is not a tabulated value for an identified substance.

The distinction between thermodynamic activity and concentration is explicit when $a_i={\gamma }_i[i]/c^{\circ }$, where ${\gamma }_i$ is an activity coefficient and $c^{\circ }$ is the reference concentration:Neglecting the final term gives the usual approximate Henderson-Hasselbalch equation. It is justified when the ratio of activity coefficients is sufficiently close to one, as in a suitable dilute ideal approximation. Ionic strength, including contributions from other dissolved ions, affects activity coefficients; a nominally dilute weak acid alone does not establish that all corrections are small. The ratio must use chemical equilibrium concentrations. Replacing these by prepared amounts or post-neutralization inventories is a further approximation, suitable only when subsequent redistribution is small. See Hippler and Metcalfe, concentration, activity, and analytical-concentration approximations.

The short derivation is to solve the acid dissociation constant relation for $a_{{\mathrm{H}}^{+}}$, take a negative base-ten logarithm, and use the definitions of pH and pKa. Its useful beauty is that a chemical equilibrium becomes a simple rule about a ratio: select a weak acid with pKa near the desired pH, then adjust the partner ratio. Scaling both partner concentrations equally leaves the ideal pH prediction unchanged, while changing buffer capacity. In practical buffer solutions, temperature, ionic strength, and the available inventories still matter.

For liquid–gas phase coexistence, make two distinct approximations: the gas obeys the ideal gas law, $v_{m,g}=RT/p$, and its molar volume greatly exceeds the liquid molar volume, $v_{m,g}\gg v_{m,l}$. Here $R$ is the molar gas constant and $p$ is the saturation pressure of the pure substance. Substitution into the Clapeyron equation giveswhere $p_{\ast }>0$ is any fixed reference pressure, making the logarithm dimensionless. Constant latent heat is a further approximation, needed for the elementary Integrated Clausius-Clapeyron relation below.^[ref]

The neglected liquid molar volume has a transparent error estimate. If the gas is ideal but $ϵ=v_{m,l}/v_{m,g}$ is retained, thenThus ignoring the liquid molar volume underestimates this slope by a relative amount $ϵ$. This estimate does not bound errors from treating a real gas as ideal or from freezing $L_m(T)$. Near a thermodynamic critical point, the gas and liquid molar volumes approach each other and these dilute-vapor assumptions fail.

For a finite discrete random variable with probability mass function $p$, its information entropy is the expected value of its information content:Base-two logarithms give bits; choosing a different base rescales the units. The convention $0{\log }_{2}0=0$ is justified by a limit, so outcomes of zero probability contribute nothing. The definition also applies to countable distributions, but the sum may be infinite.

A fair bit has $H=1$. A certain outcome has $H=0$. A Bernoulli random variable that is one with probability $0.1$ has $H=h_2(0.1)\approx 0.469$ bits, because most of its outcomes are predictable. A rare outcome still has large information content: the outcome of probability $0.1$ contributes $-{\log }_2(0.1)\approx 3.322$ bits of surprise when it occurs. Entropy is the average surprise, rather than the surprise of each outcome.

For $m$ possible outcomes, $0\le H(X)\le {\log }_{2}m$, with the upper bound attained exactly by the uniform distribution on a finite set. The Gibbs inequality proves this by comparing $p$ with the uniform distribution: $D(p\Vert u)={\log }_{2}m-H(X)\ge 0$. Thus increasing predictability reduces the required average description length. The four-symbol prefix code in information theory attains $H=7/4$ bits exactly; for general probabilities, integer codeword lengths prevent such a perfect match for every individual symbol. Shannon's source coding theorem explains how coding sufficiently long independent blocks approaches the entropy per symbol.

The logarithm is especially natural because independent outcomes have additive surprise: $-{\log }_2(pq)=-{\log }_{2}p-{\log }_{2}q$. Taking expected values gives additive information entropy for independent random variables; for dependent ones, the chain rule for information entropy replaces the second term by conditional entropy. This is why the same quantity can govern both a single observation and a long message.

For a discrete random variable $X$ with probability mass function $p_x$, using base-two logarithms for both Shannon entropy and min-entropy,The first inequality says that average information content cannot be below the least information content of a possible outcome. For a finite alphabet, equality $H(X)=H_{\infty }(X)$ holds exactly when the probability mass function is uniform on its positive-probability outcomes. Impossible outcomes do not need to share that positive probability. In particular, $(1/2,1/2,0)$ has both quantities equal to $1$ bit, while $(1,0,0)$ has both zero. The same equality characterization holds for a countable alphabet: equality forces only finitely many positive-probability outcomes, because all must have the same positive mass. The uniform maximum of Shannon entropy is discussed in Shannon's original treatment.^[ref]

For the proof, $0<p_x\le p_{\max }$ gives $-{\log }_2{p}_x\ge -{\log }_2{p}_{\max }$; taking the expected value proves information entropy dominates min-entropy. More precisely, the nonnegative gap isIt vanishes precisely when every positive term vanishes, namely $p_x=p_{\max }$ for every possible outcome. With $m$ such outcomes, normalization forces $p_{\max }=1/m$. If Shannon entropy is infinite, the inequality still holds and is strict, because min-entropy of a countable probability mass function is finite. No subtraction of two infinite quantities is needed.

An analytical family makes the gap visible. Give $A$ probability $q$ and each of $B,C,D,E$ probability $(1-q)/4$, with $1/5\le q\le 1$. Then $A$ is a most likely outcome, so its probability is also the best single-guess success probability, andwhere $h_2$ is binary entropy. The entropy bound with one prescribed probability explains the second expression: it separates whether $A$ occurred from which of four equally likely alternatives occurred. At $q=1/5$, all five outcomes are uniform and both quantities are ${\log }_{2}5\approx 2.3219$ bits. At $q=1/2$, Shannon entropy is $2$ bits while min-entropy is $1$ bit, reproducing the biased example in min-entropy. At $q=1$, both are zero; the four impossible alternatives contribute nothing. For every $1/5<q<1$, the probability mass function is nonuniform on its possible outcomes and the gap is strictly positive.

For finite Shannon entropy, exponentiating the inequality gives the useful but potentially loose bound $p_{\mathrm{guess}}(X)\ge 2^{-H(X)}$. Equality holds exactly for a discrete uniform distribution on the possible outcomes. Thus Shannon entropy alone supplies a lower bound on optimal success, not the upper bound needed to promise that every guess is unlikely. Shannon's source coding theorem concerns average compression under its hypotheses, while min-entropy answers the single-guess question directly. The two quantities serve different tasks even when both are expressed in bits.

An information projection chooses the probability distribution that satisfies a constraint while departing least from a reference law in relative entropy. The constraint might specify an expected value, several marginal probability distributions, or another convex set of permissible laws. The reference is held fixed: the variable is the first argument of $D(q\Vert p)$. Reversing the arguments gives a different minimization problem.

For a finite alphabet $\mathcal{A}$, take a positive reference probability mass function $p(a)>0$ and a nonempty closed convex set $\mathcal{C}$ in the probability simplex. ThenThe minimum exists and its minimizing probability distribution is unique. In this article, natural logarithms give nats. Using base-two logarithms rescales the objective without changing the minimizer.^[ref]

If $p$ already meets the constraint, its information projection is $p$ itself by relative entropy nonnegativity. If not, a constraint can change the relative weights of outcomes without specifying every weight individually. Relative entropy minimization under a mean constraint gives an especially useful example: it tilts the reference law rather than choosing an arbitrary distribution with the right average.

For $\psi (t)={\sum }_{m\le t}\Lambda (m)$, finite rearrangement and the Von Mangoldt divisor identity giveThe Stirling formula, or integral comparison for the logarithm, gives $T(x)=x\log x-x+O(\log (x+1))$ for real $x\ge 1$.