For finite-valued random vectors, the conditional entropy is the average of the entropies of the conditional distributions:Use $0{\log }_{2}0=0$ and omit zero-probability conditioning events. Equivalently,The base two makes all information quantities measured in bits; changing the logarithm base scales them by a positive constant and preserves the inequalities.

Make a rooted binary tree with an edge labelled zero for the first branch and one for the second branch at each interior node. Put each source symbol at the leaf reached by following the successive digits of its codeword. The prefix property ensures no assigned word ends at an ancestor of another assigned word.

The route of a codeword is its root-to-leaf path; its length is the number of edges on that path, equal to the number of binary digits in the word. For an optimal tree with positive leaf probabilities, every interior node has two children: a node with just one child could be contracted, shortening all words below it and strictly reducing the expected length. The diagram in the earlier Huffman calculation is one such full binary tree.

A binary prefix code assigns each source symbol a distinct finite binary word, with no assigned word a prefix of another. It is instantaneous because a decoder can identify the symbol as soon as its last digit arrives, without reading digits from the next word. Such a code is optimal when it minimizesamong all binary prefix codes for that source distribution. Symbols of zero probability may be removed from the optimization.

Let $X$ be the original input, $U$ the output of the first channel, and $Y$ the final output. The independent erasure flag is observable from $Y$. The mutual information chain rule, conditioning on that flag, givesThe same uniform input maximizes $I(X;U)$ as in the binary symmetric channel. ThusThis scaling works because the erasure event is independent of both the input and the first channel noise.

The capacity of a discrete memoryless channel is $C={\max }_{P_X}I(X;Y)$, in bits per channel use. For the binary symmetric channel with crossover probability $p$, $H(Y\mid X)=h(p)$ and $H(Y)\le 1$. A uniform input makes the output uniform, attainingFor the binary erasure channel, let $E$ be its erasure indicator. This flag is visible in the output and independent of the input. If there is no erasure, the output reveals the input; otherwise it reveals nothing. Consequentlyagain attained by a uniform input. These formulas include crossover probabilities zero and one, and complete erasure.

The entropy chain rule givesConditional entropy is nonnegative, so $I(X;Y)\le \min (H(X),H(Y))$. For finite entropies, if $H(X)\le H(Y)$, equality holds exactly when $H(X\mid Y)=0$, meaning $X$ is a function of $Y$ almost surely. If $H(Y)\le H(X)$, interchange the variables. When their entropies are equal, equality requires both functional relations. Zero conditional entropy has this characterization because each conditional probability distribution must be a point mass.

Write $p_{xy}=\mathbb{P}(X=x,Y=y)$, $p_x={\sum }_y{p}_{xy}$, and $q_y={\sum }_x{p}_{xy}$. In bits the mutual information isTerms with zero joint probability contribute zero. On its positive support apply $-\log t\ge 1-t$ with $t=p_x{q}_y/p_{xy}$:Equality requires equality in every logarithmic inequality and no product mass outside the joint support, hence $p_{xy}=p_x{q}_y$ everywhere. Conversely that factorization makes every term zero. Thus mutual information is nonnegative and vanishes exactly for independence. For countable alphabets the same conclusion follows from the relative-entropy inequality, allowing the value $+\infty$.

Minimum-distance decoding with Hamming distance chooses a codeword minimizing the number of positions in which it differs from the received word. Ties may be resolved by a fixed rule.

Maximum-likelihood decoding chooses a codeword maximizing $p(y\mid x)$ for the received word $y$. It uses the likelihood alone, without weighting by the prior probabilities of the codewords.

Maximum a posteriori decoding chooses an input with the largest posterior probability given the received word. By Bayes' theorem this maximizes $P(X=x)p(y\mid x)$ over candidate codewords, minimizing the average probability of a decoding error.

The continuous representative of $u$ makes $x\mapsto 2xu(x{)}^3$ continuous on the closed interval. The integrated formula therefore gives a continuously differentiable representative of $u^{\prime }$, and hence $u\in C^2([0,1])$ withDifferentiate once more. The right side is continuously differentiable because $u\in C^2$, andThus $\boxed{u\in C^3([0,1])}$. These formulas give continuous one-sided derivatives at the endpoints as well. Repeated bootstrapping would actually give smoothness of every order.

In one dimension, $u\in W^{1,2}$ has an absolutely continuous bounded representative. For any smooth zero-endpoint test function $\eta$, the functions $u+t\eta$ are admissible and arbitrarily close to $u$ for small $t$. Local minimality forces the first variation to vanish. Differentiation under the integral is valid because $u,\eta$ are bounded and the derivative terms are integrable, giving the weak Euler-Lagrange equationIn distributional form this says $(2u^{\prime }{)}^{\prime }=4x{u}^3$. The fundamental theorem for weak derivatives, or the fundamental lemma applied after subtracting an antiderivative, gives a constant $C$ withThis is the integrated Euler-Lagrange equation. The boundary difference $u(1)-u(0)=2$ determinesThe weak identity extends to all zero-trace $W^{1,2}$ test functions by density.

Here $I(0)=1/2$. First restrict $\Vert u^{\prime }{\Vert }_{\infty }\le \pi /12$. On that slope interval, the second derivative of $h(p)=1-\cos (\pi /3+p)$ is $\cos (\pi /3+p)\ge \cos (5\pi /12)>1/4$. Taylor's theorem givesThe linear term integrates to zero because ${\int }_0^1{u}^{\prime }=0$. Since $x^3\le 1$, the Poincaré inequality yieldsThe coefficient is positive. Thus zero is a strict weak local minimizer.

For failure of strong minimality, take a triangular bump supported on $[1/2-\epsilon ,1/2+\epsilon ]$, with slopes $+2\pi$ and $-2\pi$, and zero outside. Its height is $2\pi \epsilon \to 0$. Almost everywhere its slope is $0$ or $\pm 2\pi$, so the cosine term has exactly its zero-function value everywhere. The potential term is strictly negative on the bump. ThereforeHence it is not a strong local minimizer. The derivatives of these bumps stay large, so they do not contradict weak local minimality.

For any zero-endpoint admissible $u$, use $|\sin u|\le |u|$ and the sharp Poincaré inequality on $(0,1)$:The coefficient is positive, and equality forces $u^{\prime }=0$, hence $u=0$ by its endpoint values. ThusIt is therefore both a strict strong and a strict weak local minimizer.

A strong local minimizer of a variational integral is an admissible $\bar{u}$ for which there is $\epsilon >0$ such thatThe competitor's derivative need not be close. This compares with a larger family of competitors, which is why it is the stronger minimality property.

A weak local minimizer of a variational integral is an admissible $\bar{u}$ for which there is $\epsilon >0$ such thatBoth the function and its first derivative must remain uniformly close, up to the almost-everywhere convention for the derivative.

If an admissible $u$ attained the infimum zero, each nonnegative term of its integrand would vanish almost everywhere. In particular $|u{|}^{2011}=0$ almost everywhere, so $u=0$ in its Sobolev space and its weak derivative is zero. But then the remaining term is $(-64{)}^2=4096$ almost everywhere, giving $I(u)=4096$, a contradiction. Thus the infimum is zero but no global minimizer exists. The oscillatory sequence has no convergence of derivatives compatible with substitution into this nonconvex energy.

The integrand is nonnegative, so $\inf I\ge 0$. For each positive integer $n$, setThis is a continuous piecewise affine sawtooth with zero endpoints, $0\le u_n\le 1/n$, and $u_n^{\prime }=\pm 2$ almost everywhere. It belongs to $W^{1,\infty }$, and $(u_n^{\prime }{)}^6=64$ almost everywhere. ThereforeTogether with nonnegativity this proves $\boxed{\underset{\mathcal{A}}{\inf }I=0}$. The functions converge uniformly to zero while their derivatives retain magnitude two; small amplitude does not force small gradient.

A global minimizer is an admissible function $\bar{u}$ for whichEquivalently it attains the infimum over the whole admissible class. Its endpoint traces must equal the prescribed boundary values; comparing only with nearby functions would define a local rather than a global notion.

A strictly convex function satisfies the strict inequalityEndpoint and coincident-point cases are excluded because they necessarily give equality.