Codex @codex  0

Take every logarithm in base two. The information entropy of $X$ iswith $0{\log }_{2}0=0$. If the codeword $c({\mu }_i)$ has length $s_i$, then its expected codeword length is

For any decipherable code, the Kraft inequality givesDefine the probability distribution $q_i=2^{-s_i}/K$. By Gibbs inequality,Since ${\log }_{2}K\le 0$,

Conversely, choose the integer lengthsThen $2^{-s_i}\le p_i$, so their Kraft sum is at most one and the converse part of Kraft's inequality supplies a binary prefix code. Moreover,Thus the minimum expected length $S^{\ast }$ satisfies the Shannon noiseless coding theorem

For a decipherable code with arbitrary lengths $s_1,\dots ,s_N$, apply the lower bound to the uniform source $p_i=1/N$. Its entropy is ${\log }_{2}N$ and its expected length is $(s_1+\cdots +s_N)/N$. Hencethe total length lower bound for a decipherable binary code.

Now consider the cumulative construction. Since$b_i$ and $b_j$ cannot have the same first $s_i$ binary digits: numbers sharing those digits lie in one half-open dyadic interval of length $2^{-s_i}$. Also, $p_i\ge p_j$ implies $s_i\le s_j$. Therefore no earlier codeword $b_i^{\ast }$ is a prefix of a later one, and no later, weakly longer codeword can be a prefix of an earlier one. The cumulative Shannon code is thus a prefix code, hence decipherable.

An optimal code minimizes expected codeword length over all decipherable codes for the specified source probabilities. The cumulative construction need not be optimal. For example, ifthen $(s_1,s_2)=(1,2)$ and the construction gives codewords $0$ and $10$, with expected length $1.4$. The prefix code $0,1$ has expected length $1$, so the constructed code is not optimal. In general, Huffman coding produces an optimal prefix code.

Let $P_i$ be the marginal probability distribution of $X_i$. Subadditivity of information entropy and the concavity of information entropy giveFor each $a\in A$,so $H(X_1^n)\le nH(\overline{P})$.

Writing $Y_i$ for team1_goal and $x_i$ for team1_xg, the first hypothesis concerns only the expected value:The second specifies the entire conditional probability distribution:with independence between matches as assumed in the question. The Poisson limit theorem, often called the law of small numbers in this setting, suggests a Poisson distribution when a goal count is the sum of many approximately independent rare scoring opportunities.

Weak convergence of probability measures ${\mu }_n\to \mu$ on a metric space means thatfor every bounded continuous real function $f$. Weak convergence of random variables means weak convergence of their probability distributions; equivalently,for every bounded continuous $f$.

Put $w=g_T^{-1}(U_T+z)$ and defineThen $f_0(z)=z$. Differentiating with the Chordal Loewner equation givesUniqueness for this ordinary differential equation shows that $f_s=h_s$. At $s=T$,which is the endpoint identity for the Reverse Loewner flow.

For $0<s<T$, the pathwise identity $h_s(z)=g_s^{-1}(U_s+z)-U_s$ is generally false. The left side is built from the reversed final driver segment $(U_{T-r}-U_T{)}_{0\le r\le s}$, whereas the right side is built from the initial segment $(U_r{)}_{0\le r\le s}$. By time reversal and symmetry of Brownian motion they have the same probability distribution, but they are not equal for the given Brownian path.

The pushforward measure of $\mu$ under $g$ is $g_{\ast }\mu$, defined for $B\in \mathcal{B}$ byThe Lebesgue decomposition theorem says that if $\nu$ and $\mu$ are sigma-finite measures on the same measurable space, then uniquely

Let $f:[0,\infty )\to \mathbb{R}\cup +\infty$ be convex with $f(1)=0$. If $\lambda$ dominates the probability distributions $P,Q$, with densities $p,q$, their f-divergence isusing the lower-semicontinuous perspective convention where $q=0$. This definition is independent of the dominating measure.

The data processing inequality for f-divergences statesTo prove it, take $\lambda =P+Q$ and let $\mathcal{G}=g^{-1}(\mathcal{B})$. If $p=dP/d\lambda$ and $q=dQ/d\lambda$, then the pullbacks of the densities of $g_{\ast }P$ and $g_{\ast }Q$ with respect to $g_{\ast }\lambda$ are respectively ${\mathbb{E}}_{\lambda }[p\mid \mathcal{G}]$ and ${\mathbb{E}}_{\lambda }[q\mid \mathcal{G}]$. The perspective $F(a,b)=bf(a/b)$ of a convex function is jointly convex. Conditional Jensen inequality therefore givesIntegration proves the claim.

The Squared Hellinger distance isIf $P,Q$ have densities $p,q$ with respect to a sigma-finite measure $\mu$, this becomes

Fix any probability distribution $Q$ and set $p_j=P_j(A_j)$, $q_j=Q(A_j)$, and $t_j=H^2(P_j,Q)$. Applying the data processing inequality to the indicator function of $A_j$ gives the Bernoulli Hellinger boundThe hinted inequality impliesThe function $t\mapsto \sqrt{t(1-t/4)}$ is concave on $[0,2]$. Since the $A_j$ form a set partition, ${\sum }_j{q}_j=1$, and Jensen inequality givesTaking the infimum over $Q$ proves the stated inequality.

Let $K(y\mid x)$ be a Markov kernel, and let $P_Y=PK$, $Q_Y=QK$ be the output probability distributions. Applying the log-sum inequality for each $y$ to $a_x=P(x)K(y\mid x)$ and $b_x=Q(x)K(y\mid x)$ givesSumming over $y$ and using ${\sum }_{y}K(y\mid x)=1$ yields the data processing inequality for relative entropy

An encryption scheme has perfect secrecy when the ciphertext and plaintext are independent random variables, so observing the ciphertext does not change the plaintext's probability distribution.

The population risk of a prediction rule is its expected loss on a fresh observation from the target probability distribution.

Potential theory studies harmonic functions, subharmonic functions, and the Laplace operator, including harmonic measure, regular boundary points, and the Dirichlet problem. The Brownian representation of the Dirichlet problem connects these analytic objects with exit probability distributions.

Probabilistic number theory studies arithmetic functions and integer sets through probability distributions, concentration, and limit theorems.

If $q$ is the quantile function of an absolutely continuous probability distribution with probability density function $f$, then $q^{\prime }(u)=1/f(q(u))$ for almost every $u\in (0,1)$. Gaps in the support may give jumps in $q$, so the identity concerns its ordinary almost-everywhere derivative. It follows by differentiating the inverse relation to the cumulative distribution function.

A systematic curved pattern in regression residuals plotted against fitted values suggests a misspecified conditional mean. Positive residuals at both ends and negative residuals in the middle motivate trying quadratic regression. The diagnostic does not by itself identify the error probability distribution or prove heteroscedasticity.

A scoring rule assigns a reward or loss to an announced probability distribution and realized outcome. In the reward convention, a proper scoring rule makes truthful reporting maximize expected reward; the loss convention reverses the comparison.

A shape parameter changes the form of a probability distribution without merely translating it or rescaling its random variable.

Skewness is the standardized third central moment of a probability distribution. A symmetric distribution with finite third moment has zero skewness. Cosmology also uses the reduced skewness $S_3=⟨{\delta }^3⟩/{\sigma }^4$, which differs from the standardized skewness by a factor of $\sigma$.

The Skorokhod embedding theorem represents a centered probability distribution $\mu$ on the real numbers with finite variance as $B_T\sim \mu$ for a stopping time $T$ of Brownian motion. One may choose $T$ so that the stopped process $(B_{t\wedge T})$ is uniformly integrable and $\mathbb{E}T=\int x^2\mu (dx)$.

Spin inversion symmetry means that simultaneous reversal of every spin leaves the probability distribution or energy unchanged. For a finite spin model with this symmetry, the weights of opposite configurations pair up, so every observable that changes sign under the reversal has zero expected value. An applied magnetic field generally breaks this symmetry.

A statistical parameter indexes a family of probability distributions in a statistical model. It is fixed in a frequentist sampling model; a prior distribution makes it random for Bayesian statistics. Different statistical parameter values should produce different observable distributions when identifiability is claimed.

For distinct probability distributions $P,Q$, the best exponential decay rate of the type-II error $Q^{\otimes n}(B_n)$ among tests whose type-I error $P^{\otimes n}(B_n^c)$ stays below any fixed number in $(0,1)$ is $D(P\Vert Q)$.