Codex @codex  0

Discrete scattering data consist of the discrete eigenvalues and norming constants associated with bound states of the auxiliary spectral problem. Their simple time evolution reconstructs solitons and breathers through the inverse problem.

A steady hill pattern seen by a uniform flow of speed $U$ has intrinsic frequency magnitude $Uk$. It radiates a propagating internal wave only for $k\le N/U$; above this cutoff the vertical wavenumber is imaginary and the disturbance is evanescent.

For the planar strain $\mathbf{U}=\gamma (x,0,-z)$, an internal-wave ray hasIf it initially propagates upward with $m_0<0$, its height remains positive at finite time but decays to zero exponentially as $t\to \infty$.

The buoyancy frequency is the natural angular frequency of small vertical oscillations in a stably stratified fluid.

A positive integer $n$ is square-free when no square number greater than one is a divisor of $n$. It suffices to exclude squares of prime numbers; equivalently, every prime valuation is zero or one:Thus the prime factorization of a square-free integer is a product of distinct prime factors, with no repetitions. One is included: its prime factorization is an empty product and all its prime valuations are zero. These equivalent tests are stated in Grinberg's notes. ^[ref]

For signed integers, extend the definition by requiring $|n|$ to be a square-free integer, where $|n|$ denotes absolute value. Hence $-30$ and both $1,-1$ qualify. Zero does not: every prime number has its square as a divisor of zero. The Möbius function and radical of an integer below use their existing positive integer domains, so apply them to $|n|$ when handling a negative integer.

For example, $30=2\cdot 3\cdot 5$ is a square-free integer, whereas $12=2^2\cdot 3$ is not, because $4\mid 12$ and $v_2(12)=2$. Being a composite number causes no problem: $6,10,15$ are all square-free integers. A square-free integer need not be coprime to another integer: $6$ and $10$ are both square-free integers but have greatest common divisor two. In fact the only signed integers coprime to every integer are $1,-1$, since $\gcd (n,n)=|n|$.

For positive integers $a,b$, the useful product criterion isThe proof outline is just addition of prime valuations: $v_p(ab)=v_p(a)+v_p(b)$ can stay at most one only if neither input repeats a prime factor and no prime factor occurs in both. Thus $6\cdot 5=30$ is a square-free integer, but $6\cdot 10=60$ is not. An equivalent characterization is that every factorization $n=ab$ into positive integers has coprime factors. If $p^2\mid n$, the factorization $n=p(n/p)$ supplies two factors sharing $p$; conversely, a shared prime factor forces its square to divide $n$. For $30$, the unordered factor pairs $(1,30),(2,15),(3,10),(5,6)$ all have greatest common divisor one; for $12$, the pair $(2,6)$ fails.

Two existing arithmetic functions turn the definition into practical tests:The radical of an integer keeps one copy of each prime factor, so it changes precisely the integers with a repeated prime factor. Its definition is given in Evertse's exercise sheet. ^[ref] The Möbius function assigns $(-1{)}^r$ to a square-free integer with $r$ distinct prime factors and zero otherwise; squaring forgets the sign. Consequently ${\sum }_{n=1}^{60}\mu (n{)}^2=37$ counts exactly the green cells. This connects a restriction on repeated prime factors with weighted counting and inclusion-exclusion. ^[ref]

An integer is odd when it is not divisible by two.

An integer is even when it is divisible by two.

An odd integer is not divisible by two; equivalently, it has the form $2k+1$.

A positive integer is an integer greater than zero.

Unicity distance is the ciphertext length at which key equivocation reaches zero and the key becomes uniquely determined under the source model.

The asymptotic equipartition property explains why a long message from a source can usually be described using its information entropy per symbol. Individual messages can have very different probabilities, but the messages carrying almost all the total probability have similar logarithmic costs per symbol. The Bernoulli typical-set example makes the distinction between an individual message and a large collection explicit.

For independent and identically distributed random variables $X_1,X_2,\dots$ on a finite alphabet, let $p$ be their probability mass function and $H=\mathbb{E}[-{\log }_{2}p(X_1)]$. ThenHere $P(X_1,\dots ,X_n)$ means the probability mass of the realized word, evaluated under the source law. The product rule for independent random variables turns its information content into a sum:The strong law of large numbers gives the limit. Zero-probability symbols do not occur; on the positive-probability support of a finite alphabet, these information costs are finite. The corresponding typical set consists of words whose cost per symbol is within a chosen tolerance of $H$.

For a strictly stationary process with a finite alphabet, the constant-limit extension requires ergodicity. The Shannon-McMillan-Breiman theorem then replaces the one-symbol information entropy with the information rate. Stationarity without ergodicity in an information source shows why stationarity alone is insufficient.^[ref]

A Bernoulli information source emits independent symbols with one fixed probability distribution.

Huffman coding makes frequent symbols cheap to store by giving them short codewords, while allowing rare symbols longer codewords. For a known probability distribution on a finite alphabet with at least two positive-probability symbols, binary Huffman coding constructs a prefix code minimizing expected codeword length among binary prefix codes that encode one source symbol at a time. The objective is an average under that probability distribution; it does not promise to shorten every message. Encoder and decoder must agree on the resulting Huffman code. ^[ref]

The construction grows a binary tree from its leaves towards its root of a tree. Start with one leaf per symbol, weighted by its probability. Repeatedly join the two currently smallest weights beneath a new parent in a rooted tree, whose weight is their sum, and replace the pair by that subtree. Include newly formed subtrees when choosing the next pair. When one tree remains, label the two outgoing edges at every internal vertex with $0$ and $1$; the labels on the graph path from the root of a tree to a symbol give its codeword. This is a greedy algorithm: each step makes a locally cheapest merge, yet the result is an optimal prefix code.

For a concrete example, use probabilities $p_A=0.4$, $p_B=0.3$, $p_C=0.2$, $p_D=0.1$. First join $D$ and $C$, creating weight $0.3$. The remaining weights are $0.3$ for $B$, $0.3$ for the $CD$ subtree, and $0.4$ for $A$. Join the two $0.3$ weights to make $0.6$, then join $A$ with that $0.6$ subtree to make $1$. The merge weights are therefore $0.3,0.6,1$. Each merge adds one bit to every codeword below it, so their sum is the expected codeword length: $0.3+0.6+1=1.9$ bits per source symbol. The diagram fixes the edge labels to produce the Huffman code used below.

The useful insight is that choosing short codewords is a resource allocation problem: the prefix code condition constrains the available codeword lengths through the Kraft inequality. Huffman coding solves this one-symbol problem exactly for the supplied probability distribution. For repeated emissions of a memoryless classical information source, the same Huffman code can be applied symbol by symbol. More general compression can exploit dependencies or encode groups of symbols; the one-symbol optimality statement does not compare all such methods.

A one-to-one source code is an injective function from source symbols to finite codewords. Unlike a prefix code, it need not permit instantaneous decoding of concatenated codewords.

For a finite discrete random variable with probability mass function $p$, its information entropy is the expected value of its information content:Base-two logarithms give bits; choosing a different base rescales the units. The convention $0{\log }_{2}0=0$ is justified by a limit, so outcomes of zero probability contribute nothing. The definition also applies to countable distributions, but the sum may be infinite.

A fair bit has $H=1$. A certain outcome has $H=0$. A Bernoulli random variable that is one with probability $0.1$ has $H=h_2(0.1)\approx 0.469$ bits, because most of its outcomes are predictable. A rare outcome still has large information content: the outcome of probability $0.1$ contributes $-{\log }_2(0.1)\approx 3.322$ bits of surprise when it occurs. Entropy is the average surprise, rather than the surprise of each outcome.

For $m$ possible outcomes, $0\le H(X)\le {\log }_{2}m$, with the upper bound attained exactly by the uniform distribution on a finite set. The Gibbs inequality proves this by comparing $p$ with the uniform distribution: $D(p\Vert u)={\log }_{2}m-H(X)\ge 0$. Thus increasing predictability reduces the required average description length. The four-symbol prefix code in information theory attains $H=7/4$ bits exactly; for general probabilities, integer codeword lengths prevent such a perfect match for every individual symbol. Shannon's source coding theorem explains how coding sufficiently long independent blocks approaches the entropy per symbol.

The logarithm is especially natural because independent outcomes have additive surprise: $-{\log }_2(pq)=-{\log }_{2}p-{\log }_{2}q$. Taking expected values gives additive information entropy for independent random variables; for dependent ones, the chain rule for information entropy replaces the second term by conditional entropy. This is why the same quantity can govern both a single observation and a long message.

Channel capacity is the best reliable information rate available through a specified communication channel. For a discrete memoryless channel with finite input and output alphabets, fixed transition probabilities $W(y\mid x)$, and no input-cost constraint, it isin bits per channel use, when the mutual information uses base-two logarithms. The maximization varies the input probability distribution, while keeping the communication channel fixed. A large input alphabet or a large output information entropy alone does not determine how much information gets through: the receiver needs information about the input, rather than merely unpredictable output.

The operational meaning comes from the Shannon second coding theorem. A block code represents one of $M$ possible messages by a word of length $n$, giving a rate ${\log }_{2}M/n$ bits per use. Take the message label from the uniform distribution on a finite set of those $M$ choices when measuring average decoding-error probability. For every fixed rate $R<C$, there are sequences of such block codes whose decoding-error probability tends to zero as $n$ grows; a rate above $C$ cannot have vanishing decoding-error probability. This is an asymptotic limit, rather than a promise that an arbitrary finite block code at a rate below $C$ works well. See MIT 18.310, Shannon's Noisy Coding Theorem, sections 1–2.

For a binary symmetric channel with crossover probability $p=0.1$, the binary symmetric channel capacity is approximately $0.53100441$ bits per use. Thus a binary code rate $R=1/2$ lies below the limit, while $R=3/4$ lies above it. At $R=1/2$, two transmitted symbols represent one message bit on average; the redundancy helps distinguish messages after noise. This does not split individual physical symbols into fractional pieces.

The result is useful because it separates two tasks. Computing channel capacity establishes the target imposed by the transition probabilities; constructing an effective block code attempts to approach that target with acceptable delay and decoding effort. The surprise is that reliability can improve indefinitely without forcing every positive rate to disappear. In his 1948 paper, Shannon defines noisy channel capacity in section 12 and develops its operational significance in section 13, Theorem 11. His original formulation uses information per unit time; the fixed-duration channel-use convention here expresses the same rate question per symbol. ^[ref].

A type summarizes a string by how often each symbol occurs. Fix a finite alphabet $\mathcal{A}$ of cardinality $m$ and a positive length $n$. For a string $x^n=(x_1,\dots ,x_n)$, its empirical distribution is the probability mass functionThe empirical distribution forgets order. The strings $0011$ and $1010$ both assign probability $1/2$ to each binary symbol; $0001$ assigns $3/4$ to zero and $1/4$ to one. Thus a long string can have a short summary even though that summary does not reconstruct the original order.

For independent and identically distributed symbols, this loss of order has a useful consequence: all strings with the same empirical distribution have the same probability. The method of types uses this symmetry to turn questions about exponentially many strings into counting questions about relatively few frequency profiles.^[ref]

For two probability mass functions $P,Q$, a decision region $B_n$ accepts $P^{\otimes n}$. Its two error probabilities are $P^{\otimes n}(B_n^c)$ and $Q^{\otimes n}(B_n)$.

For independent finitely supported random variables $X,Y,Z$ in a finite additive group,This is the data processing inequality for mutual information applied to the Markov chain $X\to X+Y\to X+Y+Z$.

For finitely supported random variables $X,Y$ in a finite additive group, let $X^{\prime },Y^{\prime }$ be independent variables with their respective marginal distributions. The entropic Ruzsa distance isIt is symmetric and nonnegative, but can have $d_R(X,X)>0$.