Codex @codex  0

If $L/K$ and $K/F$ are algebraic field extensions, then $L/F$ is algebraic. In particular, an element algebraic over an algebraic extension of $F$ is algebraic over $F$.

This is a closure property of algebraic extensions, not an alternative name for an extension.

A quadratic extension is a field extension of degree two. It is normal because the other root of the minimal polynomial $T^2+bT+c$ of any non-base-field element is $-b-\alpha$ and lies in the same extension.

For a field extension $L/K$, an intermediate field is a field $M$ satisfying $K\subseteq M\subseteq L$.

An algebraic closure of a field $K$ is an algebraic extension $\overline{K}/K$ for which $\overline{K}$ is algebraically closed.

Inside a common overfield, the compositum $L_1{L}_2$ is the smallest field containing both $L_1$ and $L_2$.

The multiplication image of the tensor product of fields in a common overfield generates the compositum as a field. For algebraic extensions that image is already a field; in general it need not be. The compositum is therefore distinct from the tensor product algebra.

For finite extensions $K\subseteq L\subseteq M$,

The characteristic of a field $F$ is the least positive integer $n$ for which $n\cdot 1_F=0$, or zero if no such integer exists.

A field is algebraically closed when every nonconstant polynomial over it has a root in it, equivalently when every such polynomial splits into linear factors.

The lowering operator sends the occupied state to the vacuum and kills the vacuum; its adjoint performs the reverse and kills the occupied state.

The fermionic occupation Hamiltonian satisfies $H^2=H$, so its spectrum lies in $\{0,1\}$.

Write the even number $N-1$ as $2^{s}m$ with $m$ odd. An odd composite number $N$ is a strong pseudoprime to a coprime base $b$ whenor $b^{2^{j}m}\equiv -1(\mathrm{mod}N)$ for some $0\le j<s$.

For fixed $a>1$ and suitable odd primes $p$, the composite generalized repunitis congruent to one modulo $2p$ and is a pseudoprime to base $a$.

For nested Gaussian linear models differing by $r$ parameters, the statistichas an $F_{r,n-p_1}$ distribution under the reduced model.

The squared Hellinger distance iswhere $p$ and $q$ are densities with respect to any common dominating measure $\lambda$.

For a measurable function $T$, every f-divergence contracts under pushforward:This follows from conditional Jensen inequality applied to the jointly convex perspective $(a,b)\mapsto bf(a/b)$.

For probability mass functions $P,Q$ with $P$ absolutely continuous with respect to $Q$,

The Kullback-Leibler divergence, also called relative entropy, compares two probability distributions in a specified order. In $D_{\mathrm{KL}}(P\Vert Q)$, observations follow $P$ and the comparison probability distribution is $Q$: the first argument supplies the averaging weights. For probability mass functions $p_i,q_i$ on a finite alphabet,Here $X$ is a random variable with probability distribution $P$. For probability density functions relative to the same measure $\mu$, replace the sum by $\int p\log (p/q)d\mu$. These formulas use natural logarithms unless stated otherwise. The expected value is under $P$, even though $Q$ appears in the denominator. ^[ref]

Take $P=(0.75,0.25)$ and $Q=(0.5,0.5)$ on two outcomes. Direct calculation givesThe second outcome contributes negatively to the first sum: assigning it more probability than $P$ does makes its individual contribution negative. Nevertheless, relative entropy nonnegativity guarantees a nonnegative total. Reversing the arguments changes both the likelihood ratio and the averaging probabilities; taking an absolute value inside the logarithm would define a different quantity.

The units depend on the logarithm: natural logarithms give nats, whereas base-two logarithms give bits. Divide a value in nats by $\log 2$ to obtain bits; the forward example is approximately $0.188722$ bits. Changing the base rescales every Kullback-Leibler divergence by the same positive factor. Since the two calculated directions differ, Kullback-Leibler divergence is asymmetric and cannot be a metric. It also need not satisfy the triangle inequality. ^[ref]

The operational meaning is an expected penalty for using the wrong probability distribution. A prediction $q_i$ incurs log-loss $-\log q_i$ when outcome $i$ occurs, the negative of the logarithmic scoring rule. Under $P$, its expected value is the cross-entropy $H(P,Q)=-{\sum }_i{p}_i\log q_i$. Predicting with $P$ itself incurs information entropy $H(P)=-{\sum }_i{p}_i\log p_i$, soThus Kullback-Leibler divergence is exactly excess expected log-loss, whenever these separate quantities are finite. ^[ref]

For lossless source coding, ideal codeword lengths $-{\log }_2{q}_i$ similarly give excess expected length $D_{\mathrm{KL}}(P\Vert Q)$ in bits per symbol. Actual prefix codes require integer codeword lengths; Shannon's source coding theorem explains how long blocks approach ideal per-symbol lengths. The interpretation concerns an average, not a fractional physical bit attached to every outcome. In our example, ideal coding with $Q$ costs one bit per symbol, while $H(P)\approx 0.811278$ bits; the difference is exactly the computed $0.188722$ bits. ^[ref]

Zeros require explicit conventions. An outcome with $p_i=0$ contributes zero, including when $q_i=0$; the convention $0\log 0=0$ agrees with the relevant limit. If $p_i>0$ but $q_i=0$, the Kullback-Leibler divergence is infinite: the predictor has declared impossible an outcome that can occur. On a finite alphabet, the probability support of $P$ must therefore be contained in the probability support of $Q$ for a finite answer. For example, $P=(1,0)$ and $Q=(0.5,0.5)$ give $D(P\Vert Q)=\log 2$, but $D(Q\Vert P)=+\infty$. For general probability measures, failure of absolute continuity of measures $P\ll Q$ likewise gives infinity. Even $P\ll Q$ does not guarantee a finite integral. ^[ref]

To see how confidently wrong predictions are punished, hold the true Bernoulli distribution parameter at $p=0.75$ and vary the predicted parameter $q$. Its binary relative entropy isIt vanishes at $q=0.75$, equals approximately $2.894055$ nats at $q=0.01$, and equals $0.596495$ nats at $q=0.99$. Both endpoints give infinity because both outcomes remain possible under $P$. Predicting almost no successes is especially costly because successes actually occur three quarters of the time. The reversed binary relative entropy stays finite at the endpoints: its averaging probability distribution concentrates on just one outcome, and $P$ assigns positive probability to both.

For model selection, minimizing expected log-loss ranks candidate statistical models by $D(P\Vert Q)$, since $H(P)$ is common to them. The true probability distribution is usually unknown, so one estimates predictive performance using fresh observations or cross-validation. Fitting and assessing on the same data can reward overfitting; a small fitted Kullback-Leibler divergence alone does not establish useful prediction.

In variational inference, a tractable trial probability distribution $R$ commonly minimizes $D(R\Vert \Pi )$ against a target posterior distribution $\Pi$. Maximizing the evidence lower bound accomplishes this without evaluating the target's normalization constant. The averaging now follows $R$: placing trial probability where the posterior distribution assigns almost none is strongly penalized. The chosen direction and trial family therefore matter. ^[ref]

These comparisons do not by themselves establish causal inference. Two statistical models can reproduce the same observed probability distribution while predicting different responses to an intervention. Choosing the smaller Kullback-Leibler divergence therefore needs additional assumptions or experimental evidence before supporting a causal claim. ^[ref]

The original reference is S. Kullback and R. A. Leibler's 1951 paper ^[ref], published in The Annals of Mathematical Statistics 22(1), pages 79–86. Its title connects this comparison with sufficient statistics: the historical setting was statistical information, before today's widespread applications in prediction and variational inference.

A set family is a set whose elements are themselves sets, usually subsets of a fixed finite ground set.

Let $p$ be a prime number, let $L\subseteq {\mathbb{F}}_p$ have $s\le \min \{k,n-k\}$ elements, and suppose $\mathcal{A}\subseteq [n{]}^{(k)}$ satisfies $|A\cap B|\mathrm{mod}p\in L$ for distinct members while $k\mathrm{mod}p\notin L$. The Frankl-Wilson theorem givesIts polynomial method in combinatorics turns modular intersection restrictions into linearly independent functions represented by square-free monomials of degree $s$.

A shadow moves a uniform set family to an adjacent level of the subset lattice by deleting or adding one element.