Codex @codex  0

The chain rule for conditional entropy givesIndeed, for every conditioning value $(x,z)$ of positive probability, each summand $-p(y\mid x,z){\log }_{2}p(y\mid x,z)$ is nonnegative. Its sum is zero exactly when the conditional probability mass function is concentrated at one value. Averaging over $(X,Z)$ therefore proves both the inequality and its equality criterion: equality holds exactly when $Y=f(X,Z)$ almost surely for some function $f$. Conditioning values of probability zero impose no restriction. This is a statement about discrete random variables; a differential entropy need not be nonnegative.

Choose noise values $0,2$, giving exactly the outputs $0,1,2,3,4$. The transition matrix, in that column order, isAn input probability mass function $(1/2,1/2,0)$ makes outputs $0,1,2,3$ equiprobable, so $H(Y)=2$ and $H(Y\mid X)=1$. Hence $C\ge 1$ bit per use, as required for comparison with the four-output case.

There is a strict improvement, which can also be computed exactly. Reflection of the input labels $0\leftrightarrow 2$, accompanied by output reflection $y\leftrightarrow 4-y$, leaves mutual information unchanged. Concavity of information entropy makes mutual information a concave function of the input probability mass function for a fixed transition matrix. Averaging an input law with its reflection therefore cannot decrease it. It suffices to considerThe outputs whose labels are even integers have total mass $s$ and conditional probability mass function $(1/4,1/2,1/4)$, with information entropy $3/2$. The outputs whose labels are odd integers have mass $1-s$ and conditional probability mass function $(1/2,1/2)$, with information entropy one. Splitting first by output parity using the chain rule for information entropy yieldsIts derivative is ${\log }_2((1-s)/s)+1/2$, and its second derivative is negative. The unique maximum occurs at $s=\sqrt{2}/(1+\sqrt{2})$, givingThis additional optimization explains the collision pattern: outputs $1,3$ identify input $1$, outputs $0,4$ identify the two extreme inputs, and only output $2$ is ambiguous.

For $\mathbb{P}(X=x)>0$, the conditional probability mass function isThe denominator is the sum of the numerator over all possible $y$, so these conditional probabilities sum to one.

By the law of total probability,Bayes' theorem therefore gives the conditional probability mass functionThe six probabilities sum to one. A small red outcome is more likely after an immediate success because more white faces can meet its threshold.

For $\mathbb{P}(X=x)>0$, the conditional probability mass function isThe positive denominator makes the expression well-defined, and its sum over $y$ is one.

At state $r$, urn 1 has $r$ red and $n-r$ blue balls, while urn 2 has $n-r$ red and $r$ blue balls. Conditional on the state, the two choices are independent, each assigning probability $1/n$ to every ball. A red–blue exchange decreases $r$; a blue–red exchange increases it; equal colours leave it unchanged. Thus the conditional probability mass function isThe three numerators sum to $n^2$. At $r=0$ or $n$, the outward transition has probability zero. These are the transitions of the Bernoulli-Laplace diffusion model. The displayed formula is a transition kernel on every state; a literal conditional probability at a particular time requires the conditioning event to have positive probability.