Codex @codex  0

For finite discrete random variables, mutual information is the reduction of information entropy provided by observing the other variable:The last expression makes the symmetry $I(X;Y)=I(Y;X)$ visible. It is nonnegative and is zero exactly when $X$ and $Y$ are independent random variables. One way to see both facts iswhere $D$ is the Kullback-Leibler divergence with base-two logarithms. The Gibbs inequality gives nonnegativity; equality means that the joint probability distribution factors. The statement extends to more general distributions via relative entropy, but differences of infinite information entropies must not be used as if they were finite numbers.

A binary symmetric channel gives a useful example. Let $X$ be a fair bit, let $N$ be an independent Bernoulli random variable with error probability $q$, and set $Y=X\oplus N$, where $\oplus$ is exclusive or. The output is also a fair bit, but knowing the input leaves just the error uncertainty:Here $h_2$ is binary entropy. At $q=0.1$, $h_2(q)\approx 0.469$, so the output carries about $0.531$ bits of mutual information about the input per use. This is an average uncertainty reduction, not a claim that each observed symbol can be divided into fractional physical bits.

At $q=1/2$, every output is equally likely for either input, so $I(X;Y)=0$. At $q=1$, the output is always the inverted input: knowing this rule lets the receiver recover the input exactly, so $I(X;Y)=1$ again. A high raw error rate need not mean low mutual information when the errors are predictable. For this particular channel the fair input is optimal, giving the binary symmetric channel capacity $1-h_2(q)$.

Mutual information detects dependence that ordinary correlation can miss. Let $X$ be uniform on $\{-1,0,1\}$ and $Y=X^2$. Symmetry gives $\mathbb{E}[X]=\mathbb{E}[XY]=0$, hence zero covariance and zero correlation, while $Y$ is completely determined by $X$. Thus $I(X;Y)=H(Y)=h_2(1/3)>0$. This distinction is useful whenever a relationship is nonlinear: knowing one variable can reveal another without a linear trend.

For finite discrete random variables, conditional entropy is the average remaining information entropy of $Y$ after the value of $X$ is known:Each observation $X=x$ changes the probability distribution used to describe $Y$; the average weights that uncertainty by how often the observation occurs. An observation can increase uncertainty for one particular outcome, even though conditioning reduces entropy on average for finite variables.

For example, let $Y$ choose $A,B,C,D$ with probabilities $1/2,1/4,1/8,1/8$, and let $X$ report whether $Y=A$. If it reports yes, there is no remaining uncertainty. Otherwise the conditional probabilities are $1/2,1/4,1/4$, whose information entropy is $3/2$ bits. The two reports each occur with probability $1/2$, so $H(Y\mid X)=3/4$ of a bit. The original uncertainty was $7/4$ bits: this observation removes exactly one bit on average.

If $Y$ is a function of $X$, then $H(Y\mid X)=0$: the observation determines the answer. If $X$ and $Y$ are independent random variables, then $H(Y\mid X)=H(Y)$: the observation gives no improvement. These endpoints make conditional entropy useful for separating relevant observations from irrelevant ones. They also lead to the chain rule for information entropy, $H(X,Y)=H(X)+H(Y\mid X)$, which counts the first uncertainty and the remaining uncertainty without double counting.

An alphabet is a finite nonempty set of symbols from which strings or source outputs are formed.

Dirichlet separation in a rectangular box gives products of sine waves and energy proportional to $n_x^2/a^2+n_y^2/b^2+n_z^2/c^2$.

The infinite square well supplies the one-dimensional factors of this rectangular-box construction.

A divergence-free vector field satisfies $\nabla \cdot u=0$. It preserves volume under its flow, and its transport operator is skew-symmetric when boundary fluxes vanish.

When $y=\lambda x$ with $\lambda >1$, the integral is dominated by its lower endpoint. Setting $t=y+s$ and applying Watson's lemma gives coefficients rational in $\lambda -1$.

For fixed finite $y$ and $x\to \infty$, the omitted integral from zero to $y$ is negligible, so $\gamma (x,y)\sim \Gamma (x)$ and Stirling's formula applies.

Repeated integration by parts generates successive inverse powers of a large endpoint, with a remainder expressed by the same integral with shifted parameters.

The interaction Hamiltonian generates interaction-picture time evolution. To leading order for many cubic field interactions it is the negative of the cubic interaction Lagrangian, although derivative interactions require the canonical Legendre transform to justify this relation.

A static fluid in a uniform downward gravitational field obeys $dp/dz=-\rho g$.

For constant gravitational acceleration and temperature,

For a hyperbolic triangle, $\sinh a/\sin \alpha =\sinh b/\sin \beta =\sinh c/\sin \gamma$.

A hyperbolic triangle is bounded by three hyperbolic geodesics and may have vertices on the ideal boundary.

A Lambert quadrilateral is a hyperbolic quadrilateral with three right angles. If its remaining angle is $\theta$ and the two sides incident with that angle have lengths $a,b$, then

The hyperboloid model of the curvature-minus-one hyperbolic plane isIts metric is the restriction of the Minkowski bilinear form to tangent planes, and its geodesics are intersections with two-dimensional linear subspaces through the origin.

The map $z\mapsto i(1+z)/(1-z)$ is an isometry from the Poincare disc to the upper half-plane.

The upper half-plane carries the hyperbolic metric $(d{x}^2+d{y}^2)/y^2$.

The hyperbolic plane is the simply connected two-dimensional geometry of constant curvature minus one.

If every Jacobian eigenvalue has negative real part, the equilibrium is locally asymptotically stable.

The $n=2$ eigenspace has basis