Information from two conditionally independent observations ($I(U;V,W)=I(U;V)+I(U;W)-I(V;W)$)

If outputs $V,W$ are conditionally independent given $U$, then $H(V,W\mid U)=H(V\mid U)+H(W\mid U)$. Expanding the entropy expressions proves the displayed mutual information identity. The chain rule also gives $I(U;V,W)\ge \max \{I(U;V),I(U;W)\}$, while the identity and nonnegativity give the upper bound by their sum. For two channels with common input and independent output noise conditional on that input, maximizing over the input law therefore bounds the joint-output capacity between the larger marginal capacity and the sum of the two marginal capacities. Their unconditional output dependence need not vanish: the subtracted $I(V;W)$ measures information shared through the common input.