Codex @codex  0

A buffer solution contains appreciable quantities of a weak acid and its conjugate base, and resists changes in pH caused by small quantities of added chemical acid or chemical base. The conjugate base accepts added protons; the weak acid supplies protons when a chemical base consumes them. Consuming most of either partner undermines the buffering action. See Flowers et al., Chemistry 2e, buffers.

For a separate toy calculation, stipulate pKa $5$ for a weak acid $\mathrm{HA}$; this is an input, not measured material data. Initially $1\mathrm{L}$ of buffer solution contains $0.10\mathrm{mol}$ of $\mathrm{HA}$ and $0.10\mathrm{mol}$ of its conjugate base ${\mathrm{A}}^{-}$. In the ideal concentration approximation, the Henderson-Hasselbalch equation gives initial pH $5$. Add $0.010\mathrm{mol}$ of hydroxide with negligible volume change. The dominant neutralization has one-to-one stoichiometry:Each mole of consumed hydroxide removes one mole of weak acid and forms one mole of conjugate base and water. Thus the inventories after neutralization areThe shared volume cancels from the concentration ratio. The Henderson-Hasselbalch equation therefore predictsThis small increase illustrates how conversion between abundant partners absorbs added chemical base. The calculation neglects the tiny free hydronium and hydroxide inventories, changes during re-establishment of chemical equilibrium, self-ionization of water, and activity coefficient corrections. The displayed inventories approximate the final chemical species distribution; they are not an exact solution of all chemical equilibrium conditions.

A buffer solution has a finite buffer capacity: its weak acid inventory supplies protection against added chemical base, and its conjugate base inventory supplies protection against added chemical acid. Here an addition approaching $0.10\mathrm{mol}$ of hydroxide uses up the available $\mathrm{HA}$ in the dominant stoichiometry calculation. Close to that limit, the neglected chemical equilibrium contributions matter; once $\mathrm{HA}$ is exhausted, inserting a zero denominator into the Henderson-Hasselbalch equation cannot predict the actual pH. Excess hydroxide and the remaining chemical species require a fresh chemical equilibrium calculation. This is why a buffer solution stabilizes conditions for chemical reactions over a finite range of additions. See Flowers et al., Chemistry 2e, buffer capacity and added base.

For the pair $\mathrm{HA}/{\mathrm{A}}^{-}$, rearranging the acid dissociation constant gives the activity relationWith thermodynamic activity of water approximated by one and ideal dilute thermodynamic activities for both partners, this yields the familiar Henderson-Hasselbalch equation:The concentrations are at chemical equilibrium; using initial values requires negligible equilibration changes. Below, also neglect self-ionization of water and competing chemical reactions.

Stipulate $\mathrm{p}{K}_a=4.76$ for an illustrative weak acid, without identifying it with a measured substance. A conjugate base to weak acid concentration ratio of two givesEqual concentrations give a ratio of one, whose logarithm is zero, hence pH $4.76$. Multiplying the ratio by ten raises pH by one: ratios $0.1$, $1$, and $10$ give pH values $3.76$, $4.76$, and $5.76$. The ratio controls the ideal prediction, while the absolute quantities control how much added chemical acid or chemical base the buffer solution can accommodate.

A catalyst makes a chemical reaction proceed faster by participating in an alternative reaction mechanism and being regenerated. This makes repeated conversion possible: the same catalyst can help many successive reactants become reaction products. An enzyme is a biological catalyst. The overall standard change in Gibbs free energy is unchanged; see the IUPAC Gold Book: catalyst.

Regeneration does not mean that the catalyst sits unchanged throughout every step. For example, a schematic reaction mechanism can temporarily bind A to a catalyst C in a reaction intermediate AC:Adding the steps cancels both the reaction intermediate AC and the catalyst C from the net chemical equation. AC is produced and later consumed; C is used and later recovered. These letters describe possible roles, not a measured reaction mechanism. The alternative route is useful only if its chemical kinetics permits faster conversion. See Flowers et al., Chemistry 2e, §12.7: catalysts and alternative mechanisms.

For a worked barrier example, consider an idealized reversible A-to-B conversion at fixed temperature $T=300\mathrm{K}$. Stipulate standard molar Gibbs free energy levels $G_{\mathrm{A}}^{\circ }=0$ and $G_{\mathrm{B}}^{\circ }=-20\mathrm{kJ}\mathrm{mo}{\mathrm{l}}^{-1}$, with effective transition state levels $80$ without a catalyst and $50\mathrm{kJ}\mathrm{mo}{\mathrm{l}}^{-1}$ with a catalyst. Use the same standard states and the same zero for both routes; the catalytic route includes the regenerated C at both endpoints. Unlike the separate potential-energy profile example, this illustration uses Gibbs free energy at one specified temperature.

The Gibbs energy of activation is measured from the starting endpoint in the chosen direction. Thus the four barriers areBoth directional barriers fall by $30\mathrm{kJ}\mathrm{mo}{\mathrm{l}}^{-1}$. The endpoint difference stays ${\Delta }_r{G}^{\circ }=-20\mathrm{kJ}\mathrm{mo}{\mathrm{l}}^{-1}$.

To turn those barriers into rate constants, use the Eyring form of transition state theory for this effective first-order reaction pair, with a common prefactor and unit probability of successful passage:Here $k_{\mathrm{B}}$ is the Boltzmann constant, $h$ is the Planck constant, and $R$ is the molar gas constant. The common prefactor cancels from the comparisons. These Gibbs energies of activation are not the empirical activation energy $E_a$ of the Arrhenius equation. With the stipulated value $R=8.314462618\mathrm{J}\mathrm{mo}{\mathrm{l}}^{-1}{\mathrm{K}}^{-1}$, the speedup isThe exponent is a dimensionless quantity: convert $30\mathrm{kJ}\mathrm{mo}{\mathrm{l}}^{-1}$ to $30000\mathrm{J}\mathrm{mo}{\mathrm{l}}^{-1}$ before dividing by $RT$. The exponential function explains why a modest barrier reduction can have a dramatic kinetic effect. This number follows from the chosen free-energy barriers and common prefactor; it is not a universal catalytic multiplier or a prediction across temperatures.

Assume ideal thermodynamic activities with a common concentration standard state, and directional rate laws $J_f=k_f[\mathrm{A}]$ and $J_r=k_r[\mathrm{B}]$. At chemical equilibrium, equal reaction rates give $K=[\mathrm{B}]/[\mathrm{A}]=k_f/k_r$ for this model. The barriers giveThe catalyst speeds up both directions, so it changes how quickly chemical equilibrium is approached without changing this equilibrium constant. For a general chemical reaction, the thermodynamic relation is $K=\exp [-{\Delta }_r{G}^{\circ }/(RT)]$ with consistent standard states; an arbitrary multistep reaction mechanism need not permit $K=k_f/k_r$ for two measured rate constants. At the same temperature, pressure, and thermodynamic activities, a catalyst also leaves the actual overall reaction change in Gibbs free energy unchanged. See 13.3 Shifting Equilibria: Le Châtelier’s Principle (§13.3: a catalyst does not shift equilibrium). To drive an unfavourable overall conversion, a process such as energy coupling must change the combined transformation.

For finite discrete random variables, the mutual information supplied by two observations equals the information supplied by the first plus the additional conditional mutual information supplied by the second:The order changes the individual contributions but leaves the total unchanged.^[ref] To prove the first identity, add and subtract the intermediate conditional entropy:With base-two logarithms, all contributions are in bits. Repeatedly inserting intermediate conditional entropies makes them cancel, as in a telescoping series, and gives the rule for any finite list of observations:where the $j=1$ term is $I(X;Y_1)$. The same calculation with $W$ already known gives $I(X;Y,Z\mid W)=I(X;Y\mid W)+I(X;Z\mid Y,W)$.

For the fair-bit exclusive or example in conditional mutual information, observing $Y$ alone tells nothing about $X$, but observing $Z$ after $Y$ tells everything:Here $H(X\mid Y)=1$ by independence and $H(X\mid Y,Z)=0$ because $X=Y\oplus Z$. In the reverse order, $H(X\mid Z)=1$, so $I(X;Z)=0$ and $I(X;Y\mid Z)=1$: the two observations are individually uninformative but jointly reveal one bit. Simply adding their separate mutual informations would miss all of it.

The chain rule is useful for measuring the value of each new observation without double counting information already supplied. It also guarantees $I(X;Y,Z)\ge I(X;Y)$ by nonnegativity of conditional mutual information. Keeping $Y$ and adding $Z$ can help, even though discarding $Y$ and retaining only a processed $Z$ is constrained by the data processing inequality. Equating the two observation orders gives the precise identitywhich explains why there is no general rule that conditioning decreases mutual information.

Channel capacity is the best reliable information rate available through a specified communication channel. For a discrete memoryless channel with finite input and output alphabets, fixed transition probabilities $W(y\mid x)$, and no input-cost constraint, it isin bits per channel use, when the mutual information uses base-two logarithms. The maximization varies the input probability distribution, while keeping the communication channel fixed. A large input alphabet or a large output information entropy alone does not determine how much information gets through: the receiver needs information about the input, rather than merely unpredictable output.

The operational meaning comes from the Shannon second coding theorem. A block code represents one of $M$ possible messages by a word of length $n$, giving a rate ${\log }_{2}M/n$ bits per use. Take the message label from the uniform distribution on a finite set of those $M$ choices when measuring average decoding-error probability. For every fixed rate $R<C$, there are sequences of such block codes whose decoding-error probability tends to zero as $n$ grows; a rate above $C$ cannot have vanishing decoding-error probability. This is an asymptotic limit, rather than a promise that an arbitrary finite block code at a rate below $C$ works well. See MIT 18.310, Shannon's Noisy Coding Theorem, sections 1–2.

For a binary symmetric channel with crossover probability $p=0.1$, the binary symmetric channel capacity is approximately $0.53100441$ bits per use. Thus a binary code rate $R=1/2$ lies below the limit, while $R=3/4$ lies above it. At $R=1/2$, two transmitted symbols represent one message bit on average; the redundancy helps distinguish messages after noise. This does not split individual physical symbols into fractional pieces.

The result is useful because it separates two tasks. Computing channel capacity establishes the target imposed by the transition probabilities; constructing an effective block code attempts to approach that target with acceptable delay and decoding effort. The surprise is that reliability can improve indefinitely without forcing every positive rate to disappear. In his 1948 paper, Shannon defines noisy channel capacity in section 12 and develops its operational significance in section 13, Theorem 11. His original formulation uses information per unit time; the fixed-duration channel-use convention here expresses the same rate question per symbol. ^[ref].

The continuous angular geometric mean of the positive boundary complex modulus isHere the empty product is one. This differs from the angular arithmetic mean $A_f(R)=(2\pi {)}^{-1}{\int }_0^{2\pi }|f(R{e}^{i\theta })|d\theta$. In general $G_f(R)\le A_f(R)$ by the concavity of the logarithm; equality requires constant boundary complex modulus. For a holomorphic function with no zeros of a function in the closed disk, Jensen's formula gives $G_f(R)=|f(0)|$, even when the boundary complex modulus varies with angle. Thus replacing the mean of a logarithm by the logarithm of an arithmetic mean loses the exact zero identity.

Under the hypotheses of Jensen's formula, the circle log-mean is the angular average of the natural logarithm of the complex modulus. The factor $1/(2\pi )$ normalizes the integral to a mean. It averages values at equally weighted angles on the circle, and Jensen's formula computes it from $|f(0)|$ and the interior zeros of a function.

The Clément-Desormes experiment infers heat capacity ratio from a rapid pressure reduction followed by reheating. Use a rigid bottle, pump or bellows, a quick release valve, and a U-tube pressure gauge connected to the bottle and atmosphere. First pump to a small excess pressure, close the inlet, and wait for thermal equilibrium with the room at $T_0$. Measure $p_1>p_a$, where $p_a$ is local absolute atmospheric pressure. Briefly open the release valve until the bottle reaches $p_a$, then close it promptly. The remaining gas is cooler than the room. As it warms back to $T_0$ at fixed bottle volume, its pressure rises to $p_3$, with $p_a<p_3<p_1$ in the ideal procedure. The rebound reveals cooling during release. ^[ref]

The bottle loses molecules while the valve is open: it is not a closed system whose geometrical volume expands. In the well-mixed, negligible-heat transfer discharge model with constant heat capacity ratio, the remaining gas obeys $T_2/T_0=(p_a/p_1{)}^{(\gamma -1)/\gamma }$. Subsequent heating at fixed amount of substance and bottle volume gives $p_3/p_a=T_0/T_2$ by the ideal gas law. Eliminating the unmeasured $T_2$ yieldsThe logarithms use absolute pressure ratios. The approximation requires small gauge pressures $\Delta p_i\ll p_a$; $h_i$ are signed height differences between the two liquid surfaces, and the final equality uses the same liquid in both readings with hydrostatic equilibrium. It is not an instruction to insert gauge readings into the logarithmic formula. For synthetic states generated with $p_a=100000\mathrm{Pa}$, $p_1=102000\mathrm{Pa}$ and $\gamma =1.4$, the model gives $p_3\simeq 100567.39\mathrm{Pa}$. The logarithmic inference recovers $1.4$; the small-pressure approximation gives approximately $1.3961$.

Wait for a stable initial pressure after pumping and for a stable final pressure after closing. Vary the valve-open interval and initial excess pressure across repeated trials: closing too early leaves residual excess pressure, while leaving the valve open during warming lets more gas escape and suppresses the rebound. Test the seal for leaks, and read both liquid levels consistently. In a variant with incomplete release, measure the actual immediate post-release absolute pressure $p_2$ and replace the numerator by $\ln (p_1/p_2)$. The Zurich laboratory manual explains that liquid-column oscillations can delay this reading until appreciable reheating has occurred; its proposed backward extrapolation assumes an exponential thermal relaxation and must be checked against the recorded readings. These controls separate a thermal signal from valve timing, instrumental lag and leakage. ^[ref]

The cluster expansion organizes the logarithm of a dilute interacting system's canonical partition function as a sum over connected clusters built from Mayer bonds.

Any deterministic comparison sort correct for all orders of $n$ distinct sort keys needs at least $\lceil {\log }_2(n!)\rceil$ comparisons on some input. Here $n!$ is a factorial, ${\log }_2$ a logarithm, and the brackets denote the ceiling function. This is a worst-case requirement, not a requirement on every input.^[ref]

For three distinct sort keys, the diagram has one leaf for each possible ordering of the original entries $a,b,c$. Some paths need two comparisons, others three; two comparisons cannot distinguish all six orders.

For the proof outline, represent adaptive comparisons by a binary tree: distinct sort keys give only two outcomes per comparison. A leaf fixes an output permutation; one leaf cannot correctly handle two different strict orders. Hence there must be at least $n!$ leaves. Maximum depth $h$ permits at most $2^h$ leaves, so $2^h\ge n!$ gives the bound. Even a three-way comparator has only two reachable outcomes on this distinct-key family.

The growth follows without a precise approximation to the factorial. At least $\lfloor n/2\rfloor$ of its factors are at least $n/2$, so for $n\ge 2$,The brackets denote the floor function. This gives a constant times $n\log n$ lower bound for large $n$, matching the worst-case comparison growth of merge sort.

Distinct sort keys are essential to the $n!$ counting argument. Identical sort keys do not create $n!$ different strict orders. If all sort keys coincide, stable sorting returns the arrival order. This does not remove the worst-case bound for a comparison sort required to handle arbitrary distinct-key inputs.

The logarithm is strictly concave on positive real numbers, as its second derivative is negative. Thus the Jensen inequality gives $\int \log fd\mu \le \log \int fd\mu$ for a probability measure and an integrable positive $f$, whenever the left side is defined. It also controls the negative logarithmic part of convex combinations of strictly positive functions.

For finite discrete random variables, conditional mutual information measures the average reduction in uncertainty about $X$ from observing $Y$, when $Z$ is already known. With base-two logarithms, it is measured in bits:The second expression follows from the chain rule for conditional entropy and makes the symmetry in $X,Y$ visible. Equivalently, compute the mutual information separately in each conditional probability distribution and then average:Here $D$ is the Kullback-Leibler divergence, also computed with base-two logarithms; zero-probability terms contribute zero. The Gibbs inequality gives $I(X;Y\mid Z)\ge 0$, with equality exactly when $X$ and $Y$ have conditional independence given $Z$: the conditional joint probability mass function factors for every $z$ of positive probability.^[ref]

For a fully computed example, take independent random variables $X,Y$ that are fair bits, and let $Z=X\oplus Y$, where $\oplus$ is exclusive or. The complete joint probability distribution has only these four positive entries:

Every pair $(x,y)$ has probability $1/4=p_X(x)p_Y(y)$, so $I(X;Y)=4(1/4){\log }_{2}1=0$. But $p_Z(0)=p_Z(1)=1/2$. Given $Z=0$, the pairs $(0,0),(1,1)$ each have conditional probability $1/2$, while both conditional marginals are fair. ThusGiven $Z=1$, the pairs $(0,1),(1,0)$ give the same calculation. ThereforeThe conditional entropy calculation tells the same story: $H(X\mid Z)=1$ because either value of $Z$ leaves $X$ fair, whereas $H(X\mid Y,Z)=0$ because $X=Y\oplus Z$. Learning the exclusive or does not reveal either bit alone; it reveals whether they agree, so learning one then determines the other.

This example distinguishes conditional independence from ordinary independence. It also disproves the claim that conditioning always reduces mutual information, despite conditioning reduces entropy. There is no general ordering between $I(X;Y\mid Z)$ and $I(X;Y)$: the example above increases it from zero to one, while taking $Y=X$ and $Z=X$ for a fair bit decreases it from one to zero. The chain rule for mutual information explains why both are possible.^[ref]

Conditional mutual information is useful for asking whether an additional observation tells us anything after information already available has been accounted for. Its zero case expresses conditional independence exactly; in a Markov chain, this is the step that turns the chain rule for mutual information into the data processing inequality.

For finite discrete random variables $X,Y$, knowing $Y$ reduces the information entropy of $X$ on average:Equality holds exactly when $X$ and $Y$ are independent random variables. The average over observations is essential: $H(X\mid Y=y)$ can exceed $H(X)$ for a particular $y$. All logarithms here have base two, giving bits, and zero-probability terms contribute zero.^[ref]

For Bayesian inference, this compares a prior distribution with the posterior distributions produced by possible observations. An unexpected report can make previously unlikely alternatives plausible, while useful reports reduce uncertainty overall. For lossless source coding, an observation shared by encoder and decoder lets them choose descriptions for the appropriate conditional distribution; sending that observation also has a cost. The example below makes both the saving and that cost explicit.

In A Mathematical Theory of Communication by Claude Shannon (1948), the definition of conditional entropy is already an average over observations, followed by the inequality and its independence condition in section 6, property 6. The phrase “knowledge reduces uncertainty” must be read with that averaging convention.^[ref]

For a system coupled to a thermal reservoir at fixed temperature $T$, prepare the forward process in the initial canonical ensemble and the reverse process in the final control parameter's canonical ensemble. Reverse the protocol, assume microscopic reversibility and matching path probability support, and count $W$ as work done on the system. With Helmholtz free energy difference $\Delta F=F_{\mathrm{final}}-F_{\mathrm{initial}}$ and $\beta =1/(k_{B}T)$,The final forward distribution need not already be in thermal equilibrium. Its reverse preparation is prescribed equilibrium preparation, rather than the actual forward final distribution used in finite trajectory entropy production. Thus the paired path logarithm $\beta (W-\Delta F)$ is dissipated work in thermal units, not automatically the instantaneous total stochastic entropy production at the end of an arbitrary driven protocol. ^{[ref] [ref]}

Cross-entropy measures the average logarithmic loss incurred when outcomes follow one probability distribution $P$, but a model assigns probabilities using another probability distribution $Q$. For a finite alphabet, write their probability mass functions as $p_i$ and $q_i$. With base-two logarithms,The expectation is under the true or stipulated source law $P$, rather than the model $Q$. Terms with $p_i=0$ contribute nothing. If $p_i>0$ but $q_i=0$ for any outcome, the cross-entropy is $+\infty$: the model rules out an event that can occur. The result is measured in bits; natural logarithms give nats.

The useful identity is cross-entropy equals entropy plus relative entropy:The source's information entropy $H(P)$ is the baseline uncertainty. The Kullback-Leibler divergence is the extra average cost of a mismatched model, with the true law first and model second. Thus optimizing $Q$ for a fixed $P$ reduces the mismatch, rather than changing the source's own uncertainty. For finite alphabets, the minimum is $H(P)$, attained exactly at $Q=P$.

Take a three-symbol source with $P=(1/2,1/4,1/4)$. A matched binary prefix code assigns $A\mapsto 0$, $B\mapsto 10$, $C\mapsto 11$, with lengths $(1,2,2)$. A mismatched prefix code instead assigns $A\mapsto 10$, $B\mapsto 11$, $C\mapsto 0$, with lengths $(2,2,1)$, corresponding to $Q=(1/4,1/4,1/2)$. Both prefix codes decode exactly, but the most common symbol is shorter in the matched code. Under the same source $P$,The mismatched code costs an extra (1/4) bit per source symbol on average. No information is lost; the penalty is inefficient representation of this source. The mismatch penalty in lossless coding explains when this coding interpretation is exact.

Cross-entropy is also useful when evaluating a predictive distribution. A report that places tiny probability on the realized outcome receives a large logarithmic loss, even when it usually predicts the most common outcome correctly. The loss convention is the negative of the reward convention in the logarithmic scoring rule. Its expected loss is uniquely minimized by the correct probability distribution, making it a strictly proper scoring rule. Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others treats this connection in §3.1, Example 3; its discussion uses rewards, so the signs are reversed for the losses here.^[ref]

I. J. Good's Rational Decisions by I. J. Good appeared in 1952. Its published summary describes a method for encouraging accurate probability estimates and a connection with information theory. Strictly Proper Scoring Rules, Prediction, and Estimation by Tilmann Gneiting and others traces the logarithmic scoring rule at least as far back as this 1952 paper.^[ref]

For a parametric statistical model with probability mass function $q_{\theta }$ and an independent and identically distributed model, minimizing empirical cross-entropy is maximum likelihood estimation. The logarithm is strictly increasing, so maximizing ${\prod }_t{q}_{\theta }(X_t)$ is equivalent to maximizing ${\sum }_t{\log }_2{q}_{\theta }(X_t)$. Negating and dividing by $n$ gives the displayed minimization. Restricting the model family can prevent the optimum from matching the empirical distribution, and a finite optimizing parameter need not exist. This algebra does not by itself establish accuracy on new data.^[ref]

Suppose $T\ge 0$ has an absolutely continuous probability distribution, with survival function $S(0)=1$. On an interval where $S(t)>0$, its cumulative hazard isHere $h$ is the hazard function; its integral is dimensionless because the units of rate and time cancel. The identity follows by integrating the derivative of $\log S$, so mere continuity of the survival function is not enough: the absolute continuity assumption supplies a probability density function. If $S$ reaches zero, interpret $-\log S$ as $+\infty$. The normalization $S(0)=1$ also matters; an initial probability mass at zero would require an initial term $-\log S(0)$.

Accumulating hazard functions adds on the $H$ scale and multiplies survival factors on the $S$ scale. For $S(a)>0$ and $b>a$,For instance, a cumulative hazard increment of $2$ gives event probability $1-e^{-2}\approx 0.864665$, rather than a probability of $2$. With $h(t)=0.2$ per year, $H(5)=1$ and $S(5)=e^{-1}\approx 0.367879$. With the increasing hazard function $h(t)=0.04t$ per year when $t$ is expressed in years, $H(t)=0.02t^2$, so $H(5)=0.5$ gives $S(5)=e^{-0.5}\approx 0.606531$. Its coefficient has units of inverse years squared.

The constant hazard function produces an exponential distribution; the increasing example is a Weibull distribution. Their survival functions both decrease, although one hazard function stays constant and the other increases. At year ten both have $H(10)=2$ and $S(10)=e^{-2}$ despite different earlier histories. The accumulation identity is described in Lu Tian's Stanford notes, Section 1.1.

For a discrete hazard $q_j$ at each possible event time $t_j$, the exact counterpart is $S(t)={\prod }_{t_j\le t}(1-q_j)$. Taking its negative logarithm sums $-\log (1-q_j)$, rather than $q_j$. Therefore exponentiating a Nelson–Aalen estimator does not give exactly the Kaplan–Meier estimator; small-jump comparison of cumulative hazard estimators quantifies the difference.