Codex @codex  0

The Gamma function extends factorials to a continuous argument. Its defining Gamma integral iswhere $t^{z-1}=\exp ((z-1)\log t)$ uses the real natural logarithm for $t>0$. The Gamma function recurrence gives its meromorphic continuation to the complex plane, with simple poles at $0,-1,-2,\dots$ and no zeros. See Gamma function residue at a nonpositive integer and Gamma function has no zeros for proof outlines.^[ref]

For nonnegative integers $n$, the factorial identity has a shift of one:Thus $\Gamma (n)=(n-1)!$ for positive integers $n$. Half-integer arguments supply a different familiar constant:The first value follows from the Gaussian integral; the second follows from the Gamma function recurrence.^[ref]

The plot shows only positive real arguments; the gamma function also accepts complex numbers away from its poles. Matching the factorials alone does not select a unique interpolation, even if the Gamma function recurrence is imposed: $f(x)=\Gamma (x)\exp (\sin (2\pi x))$ has the same positive-integer values and the same recurrence relation. The Bohr-Mollerup theorem supplies the extra condition that selects the gamma function.

The useful idea is to replace a discrete product of numbers by an integral whose exponent varies continuously. The Gamma integral then evaluates whole families of integrals at once, provides the normalizing constant for the Gamma distribution in probability theory, and appears in physics, for example in the Bose integral. Its large-positive-argument growth reconnects this continuous description to factorials through Stirling's formula.^[ref]

Historically, Leonhard Euler studied the interpolation of factorials in his paper E19, written in 1729 and published in 1738. His integral over $(0,1)$ becomes the Gamma integral after a change of variables, with the same one-step shift in the argument.^[ref]

Draw a $k\times D$ random matrix $G$ whose entries are independent random variables with the standard normal distribution, and setThe factor $1/\sqrt{k}$ makes the expected value of the squared L2 norm equal to the original squared L2 norm. This Gaussian Johnson–Lindenstrauss transform is a linear map, generally not an orthogonal projection: its rows are not required to be orthogonal vectors. ^[ref]

For the fixed finite set $S$, let $0<\delta <1$ be an allowed failure probability, and defineWith $\log$ denoting the natural logarithm, a sufficient integer target dimension isThen one draw of this specified random matrix satisfies all the squared Euclidean distance bounds simultaneously with probability at least $1-\delta$. Since $c(\epsilon )\ge {\epsilon }^2/12$, the simpler sufficient bound $k\ge \lceil 12{\epsilon }^{-2}\log (n(n-1)/\delta )\rceil$ also works. Both are sufficient bounds for this argument; they do not claim the smallest possible target dimension.

This is a probability statement about a map drawn independently of the fixed finite set. It implies existence because the success probability is positive, but permits failed draws. The set cannot be chosen after inspecting the map with this same guarantee. In particular, no single draw preserves every possible finite set in the ambient Euclidean space when $k<D$.

For a probability mass function $Q$ of full support on a finite alphabet and a real function $g$,where $D_e$ uses natural logarithms. The unique maximizing probability mass function is the exponential tilt $P^{\ast }(x)=Q(x)e^{g(x)}/{\mathbb{E}}_Q{e}^g$.

The standard Gumbel distribution for maxima has distribution function $\Lambda (z)=e^{-e^{-z}}$ for every real $z$. Its probability density function is the derivative of its distribution function:This article uses the maxima convention throughout. Reflecting a standard Gumbel distribution random variable $Z$ to $-Z$ gives the minima convention, with distribution function $1-\exp (-e^z)$. The two conventions have opposite directions of asymmetry, so their formulas must not be interchanged. ^[ref].

For a location $\mu \in \mathbb{R}$ and positive scale $b>0$, let $Y=\mu +bZ$. This location-scale family has cumulative distribution function and probability density functionSolving $\Lambda (z)=p$ gives the standard quantile function $q(p)=-\log (-\log p)$, $0<p<1$, and hence the standard medianThe median of $Y$ is therefore $\mu -b\log (\log 2)$. Location $\mu$ is not the median: at $x=\mu$ the cumulative distribution function is $e^{-1}\approx 0.367879$.

To derive the Gumbel distribution as an extreme-value limit, take the maximum of independent exponential random variables with rate $\lambda >0$ and setThe exact distribution function isThe finite-$n$ lower boundary is required because $M_n\ge 0$. For each fixed real $z$, eventually $z>-\log n$, and the natural logarithm of the first branch satisfiesExponentiating proves convergence in distribution to the standard Gumbel distribution. It is the familiar limit $(1-u/n{)}^n\to e^{-u}$ with $u=e^{-z}$: a simple calculation reveals a whole limiting shape. The name honors Emil Gumbel; Siegrist gives this exponential-maxima derivation and the standard Gumbel distribution formulas. ^[ref].

For the toy delay model $\lambda =1{\mathrm{s}}^{-1}$, $n=100$, the limiting Gumbel distribution approximation to $M_{100}$ has $\mu =(\log 100)\mathrm{s}$ and $b=1\mathrm{s}$. Its predicted median iscompared with the exact median $-\log (1-2^{-1/100})\mathrm{s}\approx 4.975147\mathrm{s}$. The approximation is about $0.003464$ seconds lower in this example. It is not an exact finite-sample probability distribution: the approximating Gumbel distribution assigns positive probability to negative delays, while the exact sample maximum cannot be negative.

The maximum domain of attraction of the Gumbel distribution is characterized by an exponential tail ratio on the scale of a Gumbel auxiliary function. This allows many original probability distributions to share the limiting shape, but it does not make the centering $\log n$ appropriate to all of them.

The Haldane mapping function connects additive genetic map distance to endpoint recombination fraction in a toy model with no crossover interference. Specify the model directly along a transmitted chromatid: the exchange count $N$ in the interval has a Poisson distribution with expected value $d$, where $d$ is the distance in morgans. Counts in disjoint intervals are independent random variables. This is the transmitted-chromatid convention; $d$ is not the mean number of exchanges across all four chromatids in meiosis.

An odd exchange count changes parental phase. Summing the odd terms of the Poisson distribution givesFor $0\le r<1/2$, the inverse and short-interval approximation areHere $\log$ is the natural logarithm. At $d=0.2$, the recombination fraction is $0.16483997698$; at $r=0.2$, the distance is $0.2554128119$ morgans, or about $25.5413$ centimorgans. The linear approximation $r\approx d$ is useful only near zero. The recombination fraction approaches $1/2$ as $d$ grows; $r=1/2$ has no finite inverse in this model.

The model is a useful baseline, not a universal rule for every interval during meiosis. Crossover interference can change the relationship between distance and recombination fraction. For the count convention and assumptions, see Weeks, Tang and Kwon (2009): genetic map functions and exchange-count conventions. Haldane's original paper is bibliographically recorded as The combination of linkage values, and the calculation of distances between the loci of linked factors, Journal of Genetics 8 (1919), 299–309, in Selected Genetics Papers of J. B. S. Haldane, edited by K. Dronamraju (1990), contents.

The harmonic series is the positive seriesIts partial sums are the harmonic numbers $H_N$. Although the individual terms tend to zero, $H_N$ grows without bound, so the harmonic series is not a convergent series. This is a basic warning against using the size of one term as a bound on all the terms still to come.

The growth is slow enough to be visually deceptive. The partial sums at $N=10,100,1000,10000$ are approximately $2.92896825,5.18737752,7.48547086,9.78760604$. Adding ten times as many terms increases the accumulated total by roughly $\log 10\approx 2.30258509$, rather than multiplying the total by ten. The telescoping series $\sum 1/[n(n+1)]$ has similarly vanishing terms but approaches the finite sum one.

A short block argument proves the divergence without trusting a finite plot. For each integer $k\ge 0$, the block from $2^k+1$ through $2^{k+1}$ contains $2^k$ terms, each at least $1/2^{k+1}$. ThereforeEvery doubling supplies at least another half, and arbitrarily many doublings are possible. A collection of tiny terms can still have a substantial combined contribution. See Jiří Lebl, Basic Analysis I, Example 2.5.11 for this grouping proof.

Integral comparison sharpens the growth estimate. Here $\log$ denotes the natural logarithm. Since $1/x$ is a strictly decreasing function for positive $x$, comparison of rectangles with the integral givesThus the divergence is logarithmic. The finer difference $H_N-\log N$ tends to the Euler--Mascheroni constant, approximately $0.57721566$; the harmonic number expansion supplies higher-order corrections. This turns the counterexample into a usable quantitative estimate of accumulated reciprocal contributions.

Euler's De progressionibus harmonicis observationes, written in 1734 and published in 1740, studies these sums beyond the yes-or-no question of convergence. Sections 2–3 compare long blocks to show continued growth, and later sections compare the sums with logarithms and calculate the constant offset. The ^[ref] identifies the dates and original publication; ^[ref] makes the calculations accessible. This is not a claim that Euler first discovered divergence: the paper's interest includes measuring how slowly a divergent sum grows.

Let $B_1,\dots ,B_n$ be independent and identically distributed Bernoulli random variables with success probability $p$, and let ${\hat{p}}_n=n^{-1}{\sum }_i{B}_i$ be their observed frequency, a sample mean. Since every interval width is one, $V=n$. Substituting $t=n\epsilon$ into the Hoeffding inequality givesFor tolerance $0<\epsilon <1$ and failure probability $0<\delta <1$, solving $2e^{-2n{\epsilon }^2}\le \delta$ yields the sufficient sample sizeHere $\log$ is the natural logarithm. For $\epsilon =0.05$ and $\delta =0.05$, $n=738$ guarantees an error smaller than $0.05$ with probability at least $0.95$, for every $p\in [0,1]$. The two-sided bound at this sample size is approximately $0.0499440$. A specified single direction instead needs $n\ge \lceil \log (1/\delta )/(2{\epsilon }^2)\rceil$, or $600$ for these choices. Halving the tolerance multiplies the unrounded sufficient sample size by four; reducing the failure probability tenfold adds $\log (10)/(2{\epsilon }^2)$ before rounding.

This is a conservative sufficient sample size, not a necessary minimum. For a specified $p$, the count $K={\sum }_i{B}_i$ has a binomial distribution, whose exact upper tail probability isFor $k>np$, the Hoeffding inequality bounds this by $\exp [-2(k-np{)}^2/n]$. An arbitrary frequency threshold uses $k=\lceil n(p+\epsilon )\rceil$; if $k>n$, the event is impossible. The discreteness of the count and its variance $np(1-p)$ can make the exact tail probability substantially smaller. At $n=100$, $p=1/2$, $k=60$, the exact upper tail probability is about $0.0284440$, while the bound is $e^{-2}\approx 0.135335$. At $p=1/10$, $k=20$, the same deviation $\epsilon =0.1$ gives the same bound but an exact upper tail probability about $0.00197856$.

An information projection chooses the probability distribution that satisfies a constraint while departing least from a reference law in relative entropy. The constraint might specify an expected value, several marginal probability distributions, or another convex set of permissible laws. The reference is held fixed: the variable is the first argument of $D(q\Vert p)$. Reversing the arguments gives a different minimization problem.

For a finite alphabet $\mathcal{A}$, take a positive reference probability mass function $p(a)>0$ and a nonempty closed convex set $\mathcal{C}$ in the probability simplex. ThenThe minimum exists and its minimizing probability distribution is unique. In this article, natural logarithms give nats. Using base-two logarithms rescales the objective without changing the minimizer.^[ref]

If $p$ already meets the constraint, its information projection is $p$ itself by relative entropy nonnegativity. If not, a constraint can change the relative weights of outcomes without specifying every weight individually. Relative entropy minimization under a mean constraint gives an especially useful example: it tilts the reference law rather than choosing an arbitrary distribution with the right average.

For a reversible thermodynamic process that changes the volume of a fixed amount of ideal gas at absolute temperature $T$, the work done by the gas isHere $n$ is the amount of substance, $R$ the molar gas constant, $p$ the pressure, and $\ln$ the natural logarithm. The ideal gas law supplies $p=nRT/V$; the integral is the signed area under the pressure-volume curve. Isothermal expansion gives positive work; isothermal compression gives negative work. A convention counting work done on the gas instead has $W_{\mathrm{on}}=-W_{\mathrm{by}}$.^[ref]

For a calculated example, let $n=1\mathrm{mol}$, $T=300\mathrm{K}$ and $V_i=10\mathrm{L}$, $V_f=20\mathrm{L}$. The pressure falls from approximately $249.4$ to $124.7\mathrm{kPa}$, andTraversing the same isothermal process backward changes both transfer signs: $W_{\mathrm{by},\mathrm{rev}}=Q\approx -1.729\mathrm{kJ}$. Neither direction changes internal energy of an ideal gas. These stipulated values illustrate the energy balance; they are not experimental measurements.

The shaded geometric area is positive in both panels; the direction of the integral supplies the sign of work. Knowing only the two equilibrium endpoints does not justify shading this curve for an irreversible process. For an idealized piston moving against a specified external resisting pressure, boundary work is $W_{\mathrm{by}}=\int p_{\mathrm{ext}}dV$. Substitution of the equilibrium gas pressure requires the appropriate reversible limit. In free expansion into vacuum, $p_{\mathrm{ext}}=0$ and this boundary work is zero.

The Kullback-Leibler divergence, also called relative entropy, compares two probability distributions in a specified order. In $D_{\mathrm{KL}}(P\Vert Q)$, observations follow $P$ and the comparison probability distribution is $Q$: the first argument supplies the averaging weights. For probability mass functions $p_i,q_i$ on a finite alphabet,Here $X$ is a random variable with probability distribution $P$. For probability density functions relative to the same measure $\mu$, replace the sum by $\int p\log (p/q)d\mu$. These formulas use natural logarithms unless stated otherwise. The expected value is under $P$, even though $Q$ appears in the denominator. ^[ref]

Take $P=(0.75,0.25)$ and $Q=(0.5,0.5)$ on two outcomes. Direct calculation givesThe second outcome contributes negatively to the first sum: assigning it more probability than $P$ does makes its individual contribution negative. Nevertheless, relative entropy nonnegativity guarantees a nonnegative total. Reversing the arguments changes both the likelihood ratio and the averaging probabilities; taking an absolute value inside the logarithm would define a different quantity.

The units depend on the logarithm: natural logarithms give nats, whereas base-two logarithms give bits. Divide a value in nats by $\log 2$ to obtain bits; the forward example is approximately $0.188722$ bits. Changing the base rescales every Kullback-Leibler divergence by the same positive factor. Since the two calculated directions differ, Kullback-Leibler divergence is asymmetric and cannot be a metric. It also need not satisfy the triangle inequality. ^[ref]

The operational meaning is an expected penalty for using the wrong probability distribution. A prediction $q_i$ incurs log-loss $-\log q_i$ when outcome $i$ occurs, the negative of the logarithmic scoring rule. Under $P$, its expected value is the cross-entropy $H(P,Q)=-{\sum }_i{p}_i\log q_i$. Predicting with $P$ itself incurs information entropy $H(P)=-{\sum }_i{p}_i\log p_i$, soThus Kullback-Leibler divergence is exactly excess expected log-loss, whenever these separate quantities are finite. ^[ref]

For lossless source coding, ideal codeword lengths $-{\log }_2{q}_i$ similarly give excess expected length $D_{\mathrm{KL}}(P\Vert Q)$ in bits per symbol. Actual prefix codes require integer codeword lengths; Shannon's source coding theorem explains how long blocks approach ideal per-symbol lengths. The interpretation concerns an average, not a fractional physical bit attached to every outcome. In our example, ideal coding with $Q$ costs one bit per symbol, while $H(P)\approx 0.811278$ bits; the difference is exactly the computed $0.188722$ bits. ^[ref]

Zeros require explicit conventions. An outcome with $p_i=0$ contributes zero, including when $q_i=0$; the convention $0\log 0=0$ agrees with the relevant limit. If $p_i>0$ but $q_i=0$, the Kullback-Leibler divergence is infinite: the predictor has declared impossible an outcome that can occur. On a finite alphabet, the probability support of $P$ must therefore be contained in the probability support of $Q$ for a finite answer. For example, $P=(1,0)$ and $Q=(0.5,0.5)$ give $D(P\Vert Q)=\log 2$, but $D(Q\Vert P)=+\infty$. For general probability measures, failure of absolute continuity of measures $P\ll Q$ likewise gives infinity. Even $P\ll Q$ does not guarantee a finite integral. ^[ref]

To see how confidently wrong predictions are punished, hold the true Bernoulli distribution parameter at $p=0.75$ and vary the predicted parameter $q$. Its binary relative entropy isIt vanishes at $q=0.75$, equals approximately $2.894055$ nats at $q=0.01$, and equals $0.596495$ nats at $q=0.99$. Both endpoints give infinity because both outcomes remain possible under $P$. Predicting almost no successes is especially costly because successes actually occur three quarters of the time. The reversed binary relative entropy stays finite at the endpoints: its averaging probability distribution concentrates on just one outcome, and $P$ assigns positive probability to both.

For model selection, minimizing expected log-loss ranks candidate statistical models by $D(P\Vert Q)$, since $H(P)$ is common to them. The true probability distribution is usually unknown, so one estimates predictive performance using fresh observations or cross-validation. Fitting and assessing on the same data can reward overfitting; a small fitted Kullback-Leibler divergence alone does not establish useful prediction.

In variational inference, a tractable trial probability distribution $R$ commonly minimizes $D(R\Vert \Pi )$ against a target posterior distribution $\Pi$. Maximizing the evidence lower bound accomplishes this without evaluating the target's normalization constant. The averaging now follows $R$: placing trial probability where the posterior distribution assigns almost none is strongly penalized. The chosen direction and trial family therefore matter. ^[ref]

These comparisons do not by themselves establish causal inference. Two statistical models can reproduce the same observed probability distribution while predicting different responses to an intervention. Choosing the smaller Kullback-Leibler divergence therefore needs additional assumptions or experimental evidence before supporting a causal claim. ^[ref]

The original reference is S. Kullback and R. A. Leibler's 1951 paper ^[ref], published in The Annals of Mathematical Statistics 22(1), pages 79–86. Its title connects this comparison with sufficient statistics: the historical setting was statistical information, before today's widespread applications in prediction and variational inference.

Within the ideal-vapor Clausius-Clapeyron approximation, a local slope of the natural logarithm of saturation pressure against inverse temperature estimates the molar latent heat:In the stipulated vapor-pressure calculation, the slope is $-4811.16\mathrm{K}$, which recovers $40.0\mathrm{kJ}\mathrm{mo}{\mathrm{l}}^{-1}$. For variable $L_m(T)$, a two-point estimate gives an average weighted by $T^{-2}$, rather than the latent heat at an arbitrary midpoint. Differentiating noisy pressure measurements amplifies their errors. Outside the ideal-vapor regime, use independently determined molar volumes in $L_m=T\Delta v_{m}dp/dT$; a straight-looking graph alone cannot establish that the gas is ideal.

For $Y\sim N(\mu ,k{\mu }^2)$ with $\mu \ne 0$ and fixed $k>0$, represent $Y=\mu (1+\sqrt{k}Z)$ with $Z$ having the standard normal distribution. Taking the natural logarithm of the absolute value gives the displayed identity. Its random second term has a common distribution, independent of $\mu$, so this variance-stabilizing transformation gives an exactly constant variance. That variance is finite: the normal density is bounded near zero and ${\int }_0^1|\log t{|}^{2}dt<\infty$, while normal tails control logarithmic growth at infinity. The transformation is defined almost surely, loses the sign of the mean, and does not produce an exact normal distribution. A zero mean makes the original observation identically zero and is outside its domain.

For a positive real base $b\ne 1$, the logarithm is the inverse function of the exponential function $t\mapsto b^t$:It asks which exponent produces a given positive number. Thus ${\log }_{2}8=3$ and ${\log }_2(1/8)=-3$. The base restrictions matter: $b=1$ would produce only one value, while a negative base does not define $b^t$ as a real-valued function for every real exponent.

The useful transformation isFor example, $8\cdot 16=128$ corresponds to $3+4=7$ in base two. Products become sums and ratios become differences. The logarithm does not generally convert a sum into a sum of logarithms. Base conversion uses the natural logarithm: ${\log }_{b}x=\ln x/\ln b$. It also gives $d({\log }_{b}x)/dx=1/(x\ln b)$, positive when $b>1$ and negative when $0<b<1$.

Historically, John Napier's 1614 Mirifici logarithmorum canonis descriptio presented logarithmic tables and their use, documented in the Smithsonian Libraries' digitized original. Tables made multiplication-heavy calculations more manageable by replacing products with addition and a lookup. The 1619 Constructio title page below belongs to a later publication from the same work on logarithms. These historical publications preceded the modern inverse function and integral presentation used here.

The natural logarithm has the parameter-integral representationFor $0<t<1$, the identity $(t^x-1)/\ln t={\int }_0^x{t}^{s}ds$ shows convergence: the absolute value is at most $|x|t^{\min (x,0)}$, which is integrable when $x>-1$. At $t=1$, the apparent singularity is removable, with limiting value $x$.

Write the left side as $I(x)$. On a compact parameter interval $c\le x\le d$ with $c>-1$, its differentiated integrand $t^x$ is bounded by the integrable function $t^c$. The dominated convergence theorem justifies differentiation under the integral sign, giving $I^{\prime }(x)={\int }_0^1{t}^{x}dt=1/(1+x)$. Since $I(0)=0$, the fundamental theorem of calculus gives $I(x)=\ln (1+x)$. At $x=-1$, the original improper integral diverges as $-{\int }_{0}dt/[t|\ln t|]$, and it also diverges for $x<-1$.

For positive endpoints $a,b$, halfway along a logarithmic scale meansHere $\beta >1$ is the base. Equivalently, using the natural logarithm, $2\ln (m/x_0)=\ln (a/x_0)+\ln (b/x_0)$. The product of numbers identity for logarithms gives $m^2=ab$, so the midpoint is the geometric mean $\sqrt{ab}$. The two endpoints lie at equal multiplicative distances because $m/a=b/m$.

For $a=10$ and $b=100$, this midpoint is $\sqrt{1000}\approx 31.6228$, giving a factor of $\sqrt{10}$ on each side. The arithmetic mean $55$ is halfway on a linear scale, but its two ratios are $55/10=5.5$ and $100/55\approx 1.81818$. Thus reading halfway between the marks $10$ and $100$ as $55$ on a logarithmic scale gives the wrong value.

A logarithmic scale places a positive quantity $x$ according to a logarithm, so equal ratios occupy equal distances. For example, $1,2,4,8$ are equally spaced under the natural logarithm because every successive difference is $\ln 2$. An ordinary linear scale instead gives equal distances to equal differences: $1,2,3,4$.

For a base $b>1$ and a positive reference $x_0$, define the axis positionThe reference has the same units as $x$, making $x/x_0$ a dimensionless quantity. The distance between two positions isMultiplying both values by the same positive factor preserves that distance. Replacing $x_0$ by $c{x}_0$ translates every position by $-{\log }_{b}c$; changing the base rescales positions through ${\log }_{b}z=\ln z/\ln b$. Neither change alters which ratios receive equal gaps.

In base ten, the values $1,10,100,1000$ occupy positions $0,1,2,3$ when $x_0=1$. The differences between neighboring values grow from nine to ninety to nine hundred, but every step spans one order of magnitude. The values $2,20,200$ occupy positions $0.30103,1.30103,2.30103$: the same spacing shifted by ${\log }_{10}2$. This lets one plot small and large positive quantities together while keeping equal proportional changes equally visible.

A logarithmic scale is a rule for position, not an instruction to relabel the tick marks with their logarithms. A mark labelled $100$ still means the original value one hundred. Ordinary real logarithms exclude zero and negative inputs. Omitting such values or clipping them to a plotting boundary does not give them genuine logarithmic positions; the plotting choice needs to be clear.^[ref]

The logarithmic scoring rule rewards the natural logarithm of the reported probability mass or probability density at the observed outcome. The title uses the reward convention. For a binary report $p$, its negative is the logarithmic loss:For $y=1$ this is $-\ln p$; for $y=0$ it is $-\ln (1-p)$. At an endpoint, use these outcome-specific forms: assigning zero probability to the outcome incurs infinite loss, while assigning it probability one gives zero. The natural logarithm therefore makes a confidently wrong binary forecast far more costly than the bounded Brier score does.

Under the stipulated Bernoulli distribution with $q=0.7$, the expected logarithmic loss isIts unique minimum is at $p=0.7$, where the expected loss is approximately $0.610864$. At $p=0.2$ it is approximately $1.193550$. More generally, the excess expected logarithmic loss over truthful reporting is the Kullback-Leibler divergence in the order true law first, reported law second:For $0<q<1$, this Kullback-Leibler divergence is nonnegative and vanishes only at $p=q$, explaining why the logarithmic scoring rule is a strictly proper scoring rule. The same Kullback-Leibler divergence identity applies to probability density functions relative to a common measure when the truthful expected logarithmic loss is finite and the comparison expected value is well defined.^[ref] For sequential forecasts, the existing prequential log score identity explains how logarithmic rewards add over observations.

For $x>0$, concavity of the natural logarithm givesEquivalently, ${\log }_{b}x\le ({\log }_{b}e)(x-1)$ for $b>1$.

The log-likelihood is the natural logarithm of the likelihood function:For positive finite likelihood, the natural logarithm is a strictly increasing function, so the maximizing parameter values are unchanged. Set $\ell =-\infty$ where $L=0$; the same ordering then includes zero-likelihood boundaries whenever a positive finite maximum exists. For independent and identically distributed random variables, a product becomes a sum,This makes derivatives simpler and avoids forming products of tiny numbers in floating-point arithmetic. These equal-maximizer and additive formulas underlie the worked calculations in MIT 18.05, Reading 10b, section 3.1.

For the ordered sample of eight successes and two failures in likelihood function, on $0<p<1$,The score equation ${\ell }^{\prime }(p)=0$ gives $8(1-p)=2p$, hence $\hat{p}=8/10=0.8$. The negative second derivative proves strict concavity, so this is the unique interior maximum. Both counts are positive, so $L(0)=L(1)=0$, while $L(p)>0$ inside: the same point is the unique global maximum on $[0,1]$. Counting only successes adds $\log 45$ to the log-likelihood, leaving both derivatives and the MLE unchanged. For all-success or all-failure samples, the boundary analysis in maximum-likelihood estimator applies instead of an interior score equation.

The two panels below compare $k=2,5,8$ successes out of ten Bernoulli trials. Their peaks are at $\hat{p}=k/10$. The left panel uses relative height; the right panel uses the log-likelihood difference from each peak. These are the same comparison on two scales:A difference of $-2$ means relative height $e^{-2}$, and a difference of $-10$ means $e^{-10}$; the log-likelihood scale makes small relative heights easier to see. Neither vertical axis is a parameter probability density function. The zero likelihood endpoints have log-likelihood $-\infty$ and are outside the finite vertical range in the right panel.

Multiplying a likelihood function by a positive parameter-independent constant adds a constant to its log-likelihood. Subtracting its peak removes that constant. This is why the ordered-data and count-data curves coincide in this figure after rescaling. When differentiable, the gradient of the log-likelihood is the score function; the negative Hessian matrix is the observed information. A solution of a score equation still needs checking against other stationary points and boundaries: maximizing the log-likelihood does not automatically make it concave.

For an event of probability $p$, the log odds are the natural logarithm of its odds: