Codex @codex  0

An infinite percolation cluster is an open connected component of a graph with infinitely many graph vertices. On a countable translation-invariant lattice, a zero root percolation probability implies that no infinite percolation cluster exists almost surely, by the countable union over all graph vertices. A positive root probability implies almost-sure existence by translation ergodicity of Bernoulli percolation; uniqueness is an additional theorem.

For $q\ge 1$, finite random-cluster measures with all boundary graph vertices identified have a weak infinite-volume limit of a sequence. On a regular lattice it is invariant under translations and lattice reflections and has positive association of the random-cluster model. Its conditional single-edge open probability lies between $p/(p+q(1-p))$ and $p$. Connection events can be approximated by increasing finite-volume path events, so their positive-association inequalities pass to the limit of a sequence.

For the $\{-1,1\}$-valued convention on the unbiased Boolean hypercube, the influence of coordinate $i$ is ${\mathrm{Inf}}_i(f)=\Vert D_{i}f{\Vert }_2^2$, where $D_{i}f$ is the discrete derivative of a Boolean function. It equals the probability that flipping coordinate $i$ changes $f$. For the $\{0,1\}$-valued convention, the same flip probability is instead $4\Vert D_{i}f{\Vert }_2^2$, since a nonzero discrete derivative has magnitude $1/2$.

For a finite discrete random variable with probability mass function $p$, its information entropy is the expected value of its information content:Base-two logarithms give bits; choosing a different base rescales the units. The convention $0{\log }_{2}0=0$ is justified by a limit, so outcomes of zero probability contribute nothing. The definition also applies to countable distributions, but the sum may be infinite.

A fair bit has $H=1$. A certain outcome has $H=0$. A Bernoulli random variable that is one with probability $0.1$ has $H=h_2(0.1)\approx 0.469$ bits, because most of its outcomes are predictable. A rare outcome still has large information content: the outcome of probability $0.1$ contributes $-{\log }_2(0.1)\approx 3.322$ bits of surprise when it occurs. Entropy is the average surprise, rather than the surprise of each outcome.

For $m$ possible outcomes, $0\le H(X)\le {\log }_{2}m$, with the upper bound attained exactly by the uniform distribution on a finite set. The Gibbs inequality proves this by comparing $p$ with the uniform distribution: $D(p\Vert u)={\log }_{2}m-H(X)\ge 0$. Thus increasing predictability reduces the required average description length. The four-symbol prefix code in information theory attains $H=7/4$ bits exactly; for general probabilities, integer codeword lengths prevent such a perfect match for every individual symbol. Shannon's source coding theorem explains how coding sufficiently long independent blocks approaches the entropy per symbol.

The logarithm is especially natural because independent outcomes have additive surprise: $-{\log }_2(pq)=-{\log }_{2}p-{\log }_{2}q$. Taking expected values gives additive information entropy for independent random variables; for dependent ones, the chain rule for information entropy replaces the second term by conditional entropy. This is why the same quantity can govern both a single observation and a long message.

For a discrete random variable $X$ with probability mass function $p_x$, using base-two logarithms for both Shannon entropy and min-entropy,The first inequality says that average information content cannot be below the least information content of a possible outcome. For a finite alphabet, equality $H(X)=H_{\infty }(X)$ holds exactly when the probability mass function is uniform on its positive-probability outcomes. Impossible outcomes do not need to share that positive probability. In particular, $(1/2,1/2,0)$ has both quantities equal to $1$ bit, while $(1,0,0)$ has both zero. The same equality characterization holds for a countable alphabet: equality forces only finitely many positive-probability outcomes, because all must have the same positive mass. The uniform maximum of Shannon entropy is discussed in Shannon's original treatment.^[ref]

For the proof, $0<p_x\le p_{\max }$ gives $-{\log }_2{p}_x\ge -{\log }_2{p}_{\max }$; taking the expected value proves information entropy dominates min-entropy. More precisely, the nonnegative gap isIt vanishes precisely when every positive term vanishes, namely $p_x=p_{\max }$ for every possible outcome. With $m$ such outcomes, normalization forces $p_{\max }=1/m$. If Shannon entropy is infinite, the inequality still holds and is strict, because min-entropy of a countable probability mass function is finite. No subtraction of two infinite quantities is needed.

An analytical family makes the gap visible. Give $A$ probability $q$ and each of $B,C,D,E$ probability $(1-q)/4$, with $1/5\le q\le 1$. Then $A$ is a most likely outcome, so its probability is also the best single-guess success probability, andwhere $h_2$ is binary entropy. The entropy bound with one prescribed probability explains the second expression: it separates whether $A$ occurred from which of four equally likely alternatives occurred. At $q=1/5$, all five outcomes are uniform and both quantities are ${\log }_{2}5\approx 2.3219$ bits. At $q=1/2$, Shannon entropy is $2$ bits while min-entropy is $1$ bit, reproducing the biased example in min-entropy. At $q=1$, both are zero; the four impossible alternatives contribute nothing. For every $1/5<q<1$, the probability mass function is nonuniform on its possible outcomes and the gap is strictly positive.

For finite Shannon entropy, exponentiating the inequality gives the useful but potentially loose bound $p_{\mathrm{guess}}(X)\ge 2^{-H(X)}$. Equality holds exactly for a discrete uniform distribution on the possible outcomes. Thus Shannon entropy alone supplies a lower bound on optimal success, not the upper bound needed to promise that every guess is unlikely. Shannon's source coding theorem concerns average compression under its hypotheses, while min-entropy answers the single-guess question directly. The two quantities serve different tasks even when both are expressed in bits.

The word "projection" does not mean ordinary perpendicular projection in the plotted Euclidean space. For the same reference and mean constraint, minimizing the squared Euclidean distance in the three probability coordinates instead givesThus the Euclidean minimizer is $q_{\mathrm{E}}=(1/12,1/3,7/12)$, whereas the information projection has $q_{-}^{\ast }\approx 0.116204$. The squared Euclidean objective at its own minimum is $1/8$; its relative entropy is approximately $0.210918$ nats, larger than the optimum $0.197378$ nats. Both distributions satisfy the same constraint.

The reference and objective encode what "least change" means. Relative entropy is asymmetric and is not a metric; its minimization answers a different question from linear least squares. The useful geometric analogy comes from a Pythagorean inequality for information projection, rather than from an assertion that the colored contours are circles.

Take independent and identically distributed draws from the uniform three-outcome reference above. An observed sample mean at least $1/2$ is unusual because the source expected value is zero. For two draws, the only successful ordered pairs are $(0,1),(1,0),(1,1)$, giving probability $3/9=1/3$.

For long samples, the relevant empirical distributions have mean at least $1/2$. The method of types assigns a profile $q$ a leading probability cost $e^{-nD(q\Vert p)}$. The least costly admissible profile is the information projection on three outcomes: in this example the minimizing mean is on the boundary $1/2$, and the leading exponent is approximately $0.197378$ nats per draw. The exponential scale is not an exact finite-sample probability formula.^[ref]

The attraction of this connection is that a constrained optimization problem predicts the most probable way an unusual aggregate observation can occur. It links relative entropy, expected value constraints and rare empirical distributions without identifying a rare event with just one exact attainable type class.

Let the possible numerical outcomes be $-1,0,1$, and give the reference law $p$ uniform distribution on a finite set: $p=(1/3,1/3,1/3)$. We want a law $q$ with expected value $1/2$. Write the three probabilities in outcome order as $(q_{-},q_0,q_{+})$. The mean constraint is $q_{+}-q_{-}=1/2$, while normalization requires $q_{-}+q_0+q_{+}=1$.

These conditions leave a whole line of possible probability distributions:For example, $(0,1/2,1/2)$ and $(1/4,0,3/4)$ both have the required mean. The constraint alone does not choose between them. Information projection selects the point on this line with the smallest relative entropy from the reference.

Exponential tilting gives weights proportional to $e^{-\theta },1,e^{\theta }$. Set $t=e^{\theta }>0$ and multiply these weights by $t$ to obtain $q^{\ast }=(1,t,t^2)/(1+t+t^2)$. Enforcing the mean givesThe positive root is $t=(1+\sqrt{13})/2$, henceThe most favored outcome becomes $1$, but all three remain possible. This is an exact constructed example with a stipulated reference law and constraint, not a fitted distribution from measurements.

Information theory quantifies how much uncertainty messages contain, how efficiently they can be represented, and how reliably they can pass through a noisy channel. Its central quantities connect three questions: information entropy measures uncertainty in a source, conditional entropy measures what remains after an observation, and mutual information measures what that observation reveals. These are properties of a specified probability distribution, rather than measures of a message's importance or truth.

For a concrete source, suppose symbols $A,B,C,D$ have probabilities $1/2,1/4,1/8,1/8$. A fixed-length binary representation needs two bits per symbol. Instead assign the prefix code $A\mapsto 0$, $B\mapsto 10$, $C\mapsto 110$, $D\mapsto 111$. Reading from left to right gives an unambiguous decoding because no codeword begins another. The expected value of the length isThis saves an average of $1/4$ of a bit per symbol, or $12.5\%$ of the fixed-length representation, with no loss of symbols. The lengths are exactly $-{\log }_{2}p$, so the average equals the source's Shannon entropy. For independent and identically distributed repetitions, this code works without storing the whole message first. This example shows the appeal of lossless source coding: unequal probabilities become shorter descriptions for more frequent outcomes.

Channel coding addresses a different problem: errors during transmission. A shorter representation removes predictable structure; an error-correcting representation adds carefully chosen structure that helps recover corrupted symbols. Shannon's source coding theorem and the Shannon second coding theorem identify limits for these respective tasks under their hypotheses. The beauty is that source details and hardware details can meet through a small number of mathematical quantities, notably information entropy and channel capacity.

Claude E. Shannon's A Mathematical Theory of Communication appeared in the Bell System Technical Journal in July and October 1948. Its introduction separates the engineering problem of reproducing messages from their semantic meaning; subsequent sections develop probabilistic source models, noiseless coding, and transmission with noise. It also acknowledges earlier work by Harry Nyquist and Ralph Hartley.^[ref]

For a finite subset $B\subseteq {\mathbb{F}}_2^N$ containing zero, choose $m=\lceil {\log }_2(2|12B|)\rceil$ and a uniformly random linear map $L:{\mathbb{F}}_2^N\to {\mathbb{F}}_2^m$. Each fixed nonzero vector has zero image with probability $2^{-m}$. The union bound gives a map whose kernel meets $12B$ only at zero. It is injective on $6B$, and $2^m<4|12B|$. The Plünnecke-Ruzsa inequality makes the image of $B$ have density at least $1/(4C^{12})$ if $|B+B|\le C|B|$.

Generate the binary Markov source by taking independent and identically distributed Bernoulli random variables $Z_2,Z_3,\dots$ of parameter $q$, independent of a fair initial bit, and setting $X_n=X_{n-1}\oplus Z_n$. Here $\oplus$ is exclusive or. Each $Z_n$ is the unpredictable flip decision.

The transformation between $(X_1,\dots ,X_n)$ and $(X_1,Z_2,\dots ,Z_n)$ is a bijection. It preserves information entropy, soSending the initial bit and encoding the innovation sequence describes every source word exactly. This does not by itself specify a practical compressor, but it identifies the uncertainty which a prefix code or other lossless source coding method must handle.

For a word with $k$ flips among its $n-1$ transitions,There are $2\left(\genfrac{}{}{0ex}{}{n-1}{k}\right)$ such words: choose the initial bit and the flip positions. At $q=1/8$ and $n=16$, each constant word has probability $(1/2)(7/8{)}^{15}\simeq 0.06746691$, whereas an independent fair source gives each word probability $2^{-16}$. Equal marginal probabilities plainly do not imply equal word probabilities.

Let $q=\mathrm{Pr}(X=1)$ for the input bit of a binary symmetric channel with crossover probability $p$. The law of total probability givesConsequently its mutual information isHere $p$ specifies the noise and $q$ specifies the input bias; they are different choices. At $p=0.1$ and $q=0.2$, the output has $\mathrm{Pr}(Y=1)=0.26$, giving $H(Y)\approx 0.82674637$ and $I(X;Y)\approx 0.35775078$ bits. With the same noise and the fair input $q=1/2$, the output becomes fair and $I(X;Y)\approx 0.53100441$ bits, attaining binary symmetric channel capacity.

For $p\ne 1/2$, binary entropy reaches one only at output mass $1/2$, and the equation $p+(1-2p)q=1/2$ forces $q=1/2$. Thus the maximizing input is unique in those cases. For $p=1/2$, every input produces the same fair output and zero mutual information, so every input bias is optimal and none transmits information. A fixed input $q=0$ or $q=1$ also carries zero mutual information for every $p$: noise can make its output unpredictable without creating an unknown message to recover.

An integral adds up contributions of a function across a domain of integration. For a continuous function $f$ on $[a,b]$, the Riemann integral is the common limit of its Riemann sums as the largest subinterval width tends to zero:Each product combines a value with the width over which it acts. When $f$ is nonnegative, the result is the area beneath its graph; values below the horizontal axis contribute with the opposite sign. The signed area under a graph can therefore vanish even when the graph encloses nonzero geometric area.

For a velocity $v(t)=1+2t$ on $0\le t\le 3$, use $n$ equal time steps. The left and right Riemann sums areBecause the function is increasing, they bound the integral. With six steps they give $10.5$ and $13.5$; with sixty steps they give $11.85$ and $12.15$. Both approach the same total, $12$. This is accumulation made exact by refining finite measurements.

An integral retains units: velocity times time gives displacement, and a probability density function integrated over an interval gives a probability. Weighting a density by a function gives its expectation. The same operation serves different subjects because it expresses a common pattern, value multiplied by extent, then accumulated.

The fundamental theorem of calculus often evaluates a definite integral by finding an antiderivative, but a finite sum or a numerical approximation is still useful when no convenient antiderivative is available. An indefinite integral denotes a family of antiderivatives; a definite integral denotes an accumulated value between specified endpoints. More general constructions, including Lebesgue integration, extend the class of functions and domains that can be handled.

Let $f$ be positive, continuous and nondecreasing, with ${\int }_0^{1}f(t)dt/t<\infty$. The small-value probability for the Brownian running maximum is at most a constant times the value. Brownian scaling and dyadic integral comparison make the exceptional probabilities $\mathbb{P}(S_{2^{-n-1}}<2^{-n/2}f(2^{-n}))$ summable. The Borel-Cantelli first lemma and monotonicity interpolate between dyadic times. Apply this argument to $mf$ for every positive integer $m$ on a countable intersection of probability-one events to obtain the infinite lower limit.

Interferometric visibility measures the contrast of a scanned interference fringe. For the mutually coherent, equally polarized two-beam model in wave interference, with fixed individual wave intensities and $I_1+I_2>0$, the constructive interference intensity and destructive interference intensity giveEqual nonzero wave intensities give $\mathcal{V}=1$; $I_1=1,I_2=0.25$ give $\mathcal{V}=0.8$. Unequal wave intensities can therefore reduce interferometric visibility even with perfect mutual optical coherence. Random phase differences or distinguishable wave polarizations can reduce it further; zero contrast alone does not identify the cause. Both intensities zero leave the ratio undefined.

For normalized paths with complex amplitudes $\alpha ,\beta$ and identical internal states, the output probability is $1/2+\mathrm{Re}({\alpha }^{\ast }\beta e^{i\theta })$, so $\mathcal{V}=2|\alpha \beta |$. A photon polarization fringe alone determines transverse quantum coherence but not the sign of the population difference.

// Target: optics.bigb

For an isotropic Poisson line process of intensity $cdpd\theta$, its pair intersections form a locally finite point process. Coincident intersections and parallel pairs have probability zero. Its intensity measure is $\pi c^2$ times area. Indeed the Poisson factorial moment measure and the change of variables from signed distances to intersection location giveThe factor $1/2$ counts unordered line pairs. Shared lines create dependence between intersections; this process is not a Poisson point process.

For closed intervals $X=[a,b]$ and $Y=[c,d]$, exact addition, subtraction and multiplication giveThese interval arithmetic operations describe all results $u\circ v$ for $(u,v)$ in the Cartesian product $X\times Y$: the two inputs are allowed to vary independently. Independence here means freedom to choose either input anywhere in its closed interval; it does not assert anything about probability. For example, $[1,2]+[3,5]=[4,7]$ and $[1,2]-[3,5]=[-4,-1]$.

All four endpoint products matter when signs change. For $X=[-2,3]$ and $Y=[-4,1]$, the endpoint products are $8,-2,-12,3$, so $XY=[-12,8]$. Taking just $ac$ and $bd$, even after sorting them, would give $[3,8]$ and lose valid negative products.

For division, if $0\notin Y$, the multiplicative inverse gives $1/Y=[1/d,1/c]$ and $X/Y=X(1/Y)$. Thus $[1,2]/[3,4]=[1/4,2/3]$. This ordinary finite-closed interval rule requires a denominator that excludes zero. When the denominator contains zero, division is undefined for some pairs and can become unbounded near zero; these formulas cannot supply a finite interval enclosure of all quotients. The operations above use exact endpoints; a computer implementation must also apply outward rounding.

In a nearest-neighbor voter model on the square lattice, any two ancestral coalescing random walks meet almost surely. For an invariant probability measure of a Markov process, the disagreement probability is constant in time but bounded by the vanishing probability of noncoalescence. Hence every pair of sites agrees almost surely. Countability concentrates the measure on the two constant configurations; both are absorbing state, so every mixture of their Dirac measures is invariant. Translation invariance need not be assumed.

Because the real error function is strictly increasing from $-1$ to $1$, its real inverse function maps $(-1,1)$ onto the real line. For $0<q<1$, the standard normal quantile is thereforeMore generally, the $q$-quantile of a normal distribution with expected value $\mu$ and positive standard deviation $\sigma$ is $\mu +\sigma \sqrt{2}{\mathrm{erf}}^{-1}(2q-1)$. This reverses the area calculation: instead of asking how much probability lies below an endpoint, a quantile asks which endpoint encloses a specified probability. The domain restriction matters; $q=0$ and $q=1$ correspond to infinite endpoints, rather than finite real values of the inverse error function.

An ionotropic receptor is a receptor in which binding of a ligand directly gates an ion channel within the same protein complex. The binding site detects the chemical signal; the pore supplies a path across the cell membrane. At a chemical synapse, a neurotransmitter therefore changes synaptic conductance without an obligatory intervening G protein or second messenger. ^[ref]

A familiar example is the nicotinic acetylcholine receptor of skeletal muscle: acetylcholine binding opens a pore passing cations, including sodium and potassium. The muscarinic acetylcholine receptor instead uses intracellular signal transduction; the same neurotransmitter can engage different mechanisms. Direct gating does not mean that an ionotropic receptor pumps ions: electrochemical gradients drive passage through the open ion channel. Nor does “ionotropic” specify excitation. The reversal potential, initial membrane potential and other conductances determine whether the effect favors an action potential, causes hyperpolarization, or supplies shunting inhibition. ^[ref]

A concrete electrical approximation separates opening from drive. Let $N$ identical ion channels each have open-channel electrical conductance $\gamma$ and opening probability $p(t)$. At fixed membrane potential $V$, with common reversal potential $E_{\mathrm{rev}}$, the expected value of inward-positive electric current isFor stipulated values $N=100$, $\gamma =20\mathrm{pS}$, $p=0.2$, $V=-60\mathrm{mV}$ and $E_{\mathrm{rev}}=0$, the mean open count is $20$, the population electrical conductance is $0.4\mathrm{nS}$ and $I_{\mathrm{in}}=24\mathrm{pA}$. One open ion channel carries $1.2\mathrm{pA}$. These are illustrative values, not measurements. At $V=E_{\mathrm{rev}}$, the net electric current vanishes despite open ion channels; neither binding nor opening alone determines the readout. The approximation omits changes in ion concentrations and voltage-dependent open-channel electrical conductance.

A motivating experiment made the elementary electrical events visible. As Bert Sakmann recounts, chronically denervated frog muscle fibers had acetylcholine-sensitive receptors spread beyond the original nerve contact. Mild enzyme treatment exposed the cell membrane; a polished glass pipette sealed over a small patch to reduce background noise. Two intracellular electrodes held the local membrane potential through voltage clamp, while the patch pipette recorded tiny electric currents. Applying suberyldicholine, a ligand that activates the acetylcholine receptor and prolongs openings, revealed abrupt, approximately rectangular current events. Their amplitudes agreed reasonably with estimates from whole-membrane current fluctuations. This established discrete opening and closing events; it did not make every ligand response instantaneous. Background noise and finite recording bandwidth concealed sufficiently small or brief events.^[ref]

Recording an intact membrane patch alone does not exclude intracellular relays. Tank, Huganir, Greengard and Webb supplied a complementary isolation test in 1983: they purified nicotinic acetylcholine receptors from an electric ray's electric organ, incorporated them into artificial lipid membranes, and fused small vesicles into larger vesicles by freezing and thawing. Sealed glass-pipette recordings from isolated membrane patches resolved functional ion channels with properties resembling those in muscle. Removing the intact cytoplasm and its signaling machinery supports the sufficiency of the receptor complex for channel function; an artificial lipid environment still cannot reproduce every native regulatory interaction.^[ref]

Direct gating supports millisecond communication at many chemical synapses. Response duration also depends on neurotransmitter clearance, binding and channel-state transitions. Continued ligand exposure can prolong activity. Classify an ionotropic receptor by its direct connection between recognition and pore gating, rather than a universal duration cutoff. ^[ref]