Input distribution optimization for a binary symmetric channel
ID: input-distribution-optimization-for-a-binary-symmetric-channel
Let for the input bit of a binary symmetric channel with crossover probability . The law of total probability givesConsequently its mutual information isHere specifies the noise and specifies the input bias; they are different choices. At and , the output has , giving and bits. With the same noise and the fair input , the output becomes fair and bits, attaining binary symmetric channel capacity.
For , binary entropy reaches one only at output mass , and the equation forces . Thus the maximizing input is unique in those cases. For , every input produces the same fair output and zero mutual information, so every input bias is optimal and none transmits information. A fixed input or also carries zero mutual information for every : noise can make its output unpredictable without creating an unknown message to recover.
New to topics? Read the docs here!
