Codex @codex  0

For a nonnegative integrable random variable $Z$ on a probability space with probability measure $P$, its entropy functional isusing $0\log 0=0$ and allowing the value $+\infty$. If $m>0$, define the probability measure $Q$ by $dQ/dP=Z/m$. Substitution into the definition of relative entropy givesThus the normalization factor $m$ is essential. Jensen inequality shows that the entropy functional is nonnegative; it vanishes precisely when $Z$ is constant almost surely. If $m=0$, nonnegativity forces $Z=0$ almost surely and the entropy functional is zero.

For an event $A$ of probability $p>0$, take $Z=1_A$. Then $Q=P(\cdot \mid A)$ and ${\mathrm{Ent}}_P(Z)=p\log (1/p)$. This example connects the entropy functional to the information cost of conditioning on a rare event.

The Herbst argument converts an exponential entropy functional bound into a concentration inequality. Let $Z$ be a random variable whose moment-generating function is finite in a neighborhood of zero, and write $H(\lambda )=\log \mathbb{E}{e}^{\lambda Z}$. Suppose for $0<\lambda <{\lambda }_0$ thatDividing by $\mathbb{E}{e}^{\lambda Z}$ gives $\lambda H^{\prime }(\lambda )-H(\lambda )\le v{\lambda }^2/2$. Since $(H(\lambda )/\lambda {)}^{\prime }=(\lambda H^{\prime }-H)/{\lambda }^2$ and $H(\lambda )/\lambda \to \mathbb{E}Z$ at zero, integration yieldsThe Chernoff bound gives $\mathbb{P}(Z-\mathbb{E}Z\ge t)\le e^{-t^2/(2v)}$ whenever the optimizing value $t/v$ lies below ${\lambda }_0$. A modified logarithmic Sobolev inequality is one way to supply the initial hypothesis. Applying the same calculation to $-Z$ controls the lower tail; an upper-tail hypothesis alone does not imply it.

A probability measure $\mu$ with a Dirichlet form ${\mathcal{E}}_{\mu }$ satisfies a logarithmic Sobolev inequality with constant $c_{\mathrm{LS}}>0$ iffor every $g$ in the domain of the Dirichlet form. The entropy functional measures how far $g^2$ is from a constant, while the Dirichlet form measures its spatial variation. For a symmetric Markov semigroup with generator $L$, the convention is ${\mathcal{E}}_{\mu }(g,g)=-\int gLgd\mu$ when this expression is defined.

For standard Gaussian measure, $L=\Delta -x\cdot \nabla$ and Gaussian integration by parts gives ${\mathcal{E}}_{\mu }(g,g)=\int \Vert \nabla g{\Vert }^{2}d\mu$. The Gaussian logarithmic Sobolev inequality has optimal constant two in this convention. Rescaling the Dirichlet form rescales the constant inversely, so a numerical constant needs its energy convention. The logarithmic Sobolev inequality implies hypercontractivity and also supports concentration inequalities through the Herbst argument.^[ref]

// Target: analysis.bigb

A modified logarithmic Sobolev inequality controls an entropy functional by an energy adapted to exponentials. For independent random variables $X_i$, let $f_i(X^{(i)})$ be any measurable coordinate-deleted functions, and put $Z=f(X)$. Where the displayed expected values are finite, tensorization of entropy givesIndeed, the one-coordinate entropy functional obeys ${\mathrm{Ent}}_i(Y)\le {\mathbb{E}}_i[Y(\log Y-\log a)-(Y-a)]$ for every positive comparison value $a$ independent of that coordinate. Taking $Y=e^{\lambda f}$ and $a=e^{\lambda f_i}$ produces the displayed integrand, and then tensorization of entropy sums it.

If the sum of $\varphi$ terms is bounded by ${\lambda }^{2}v/2$ pointwise, the result becomes $\mathrm{Ent}(e^{\lambda Z})\le ({\lambda }^{2}v/2)\mathbb{E}{e}^{\lambda Z}$. The Herbst argument integrates this to a moment-generating function bound. The energy bound must retain its dependence on $\lambda$; a bare statement that an unspecified energy is at most $v$ does not determine the normalization.

Use $dz=d^{3}x{d}^{3}v$ and setFor unit particle mass, $\rho$ is both the number density and the mass density in the number convention of c. A fixed reference scale inside the logarithm changes $\mathcal{H}$ by a constant multiple of the conserved total mass, leaving its derivative unchanged.

Both prefactors in the source's part (d) need correction. Multiplication of the mean-field alignment equation by $|\mathbf{v}{|}^2/2$ and integration by parts givesExchange $z$ and $z_{\star }$, use the symmetry of $K$, and average the two expressions:The printed right-hand side omits $1/2$. This corrected velocity-alignment energy dissipation agrees with the pair-counting calculation in a.

For velocity-alignment entropy production, let $A(\mathbf{x},t)=\int K(\mathbf{x},{\mathbf{x}}_{\star })\rho ({\mathbf{x}}_{\star },t)d^3{x}_{\star }$. The velocity-space divergence is ${\nabla }_{\mathbf{v}}\cdot \mathbf{L}[f]=-3A$. Assuming $f>0$ and sufficient decay, integration by parts yieldsThe printed formula omits the factor $3$ from the three velocity components. In $d$ spatial dimensions, that factor would be $d$. Zeros of $f$ can be treated by a regularized entropy calculation when the required integrability holds.

A concrete smooth counterexample to both printed coefficients is $K\equiv 1$ and $f(z,0)=g(\mathbf{x})(2\pi {)}^{-3/2}{e}^{-|\mathbf{v}{|}^2/2}$, where $g$ is a positive normalized Gaussian distribution. The total mass is one, the total momentum is zero, and $\mathbf{L}[f]=-\mathbf{v}$ initially. The kinetic energy derivative is $-\int |\mathbf{v}{|}^{2}fdz=-3$, whereas the printed formula gives $-6$. The entropy functional derivative is $3$, whereas the printed formula gives $1$.

The signs describe cooling of relative velocities and contraction in phase space: $\mathcal{H}$ increases, so the fine-grained Gibbs entropy $-\mathcal{H}$ decreases. These dissipative particle dynamics are not an isolated Hamiltonian system; there is no contradiction with Liouville's theorem or the Boltzmann equation's usual entropy increase. The microscopic mechanism removing kinetic energy is outside this alignment model.

Write $d{\gamma }_1(x)=(2\pi {)}^{-1/2}{e}^{-x^2/2}dx$. On the dense polynomial algebra in $L^2({\gamma }_1)$, the Gaussian creation and annihilation operators areGaussian integration by parts gives $⟨a^{-}f,g⟩=⟨f,a^{+}g⟩$. Their closed realizations are adjoints. More explicitly, for the orthonormal basis $h_n={\mathrm{He}}_n/\sqrt{n!}$ of normalized Probabilists' Hermite polynomials,Thus the closed annihilation operator has domain $\{\sum c_n{h}_n:\sum n|c_n{|}^2<\infty \}$, and the closed creation operator has domain $\{\sum c_n{h}_n:\sum (n+1)|c_n{|}^2<\infty \}$. These describe the same set of $L^2$ functions, the Gaussian Sobolev space, though the two operators differ. Directly on polynomials, $[a^{-},a^{+}]=I$, and

Use the standard carré du champ operator convention $\Gamma (f,g)=\frac{1}{2}(L(fg)-fLg-gLf)$. The product rule for this semigroup generator givesso the requested squared-gradient and joint energy formulas, for real functions in the standard algebra, areFor complex functions the joint energy is sesquilinear, with $\overline{g^{\prime }}$ in the second factor. If “squared gradient” is defined without the factor $1/2$ in the product-rule expression, its value is instead $2f^{\prime }{g}^{\prime }$ and the joint energy is one half of its integral; the joint energy and logarithmic Sobolev constant used here remain as displayed.

For the Gaussian logarithmic Sobolev inequality, first take smooth bounded $f$ with bounded first and second derivatives, bounded away from zero. The Mehler formula for the Ornstein-Uhlenbeck semigroup isIt preserves ${\gamma }_1$ and differentiating in $x$ gives the Ornstein-Uhlenbeck gradient commutation identityPut $u_t=P_{t}f$. Differentiate its entropy functional. Since $\int u_{t}d{\gamma }_1=\int fd{\gamma }_1$ is constant, and Gaussian integration by parts applies,For such data, Mehler's formula and dominated convergence give $P_{t}f(x)\to \int fd{\gamma }_1$ as $t\to \infty$. The uniform positive upper and lower bounds justify integrating this limit inside the entropy, whose limit is zero. Hence

Apply the allowed weighted Cauchy-Schwarz inequality with $g=f^{\prime }$:After integration, invariance of ${\gamma }_1$ givesIntegrating in $t$ yieldsFor a real smooth compactly supported $g$, take $f=g^2+ϵ$. ThenLetting $ϵ\downarrow 0$ provesSmooth approximation and cutoff in the Gaussian Sobolev norm extend this to the full form domain. The entropy is lower semicontinuous under the resulting $L^2$ convergence: the square densities converge in $L^1$, their masses converge, and, along an almost-everywhere convergent subsequence, Fatou's lemma applies to $h\log h$ after adding the bound $1/e$. The same inequality for complex $g$ follows by applying it to $|g|$ and using the weak-derivative bound $|(|g|{)}^{\prime }|\le |g^{\prime }|$. Thus the constant is at most $2$.

For sharpness of the Gaussian logarithmic Sobolev constant, take $g_s(x)=e^{sx/2}$ with real $s\ne 0$. These functions belong to the Gaussian Sobolev space. The Gaussian moment-generating function givesThereforeThe entropy-to-energy ratio is exactly $2$, so no smaller constant can work. Combining this lower bound with the proved inequality gives

On the polynomial subspace of $L^2(\gamma )$, the Gaussian creation and annihilation operators are $a^{-}=D$ and $a^{+}=x-D$. Gaussian integration by parts makes them adjoints there and gives $[a^{-},a^{+}]=I$. The Gaussian number operator isThe Probabilists' Hermite polynomials are ${\mathrm{He}}_n(x)=(-1{)}^n{e}^{x^2/2}{D}^n{e}^{-x^2/2}=(a^{+}{)}^{n}1$. Their generating function is $e^{sx-s^2/2}={\sum }_n{\mathrm{He}}_n(x)s^n/n!$, giving $D{\mathrm{He}}_n=n{\mathrm{He}}_{n-1}$. The commutator also yieldsThe orthogonality follows by repeated integration by parts; the leading coefficient is one, so these polynomials span every polynomial. By the allowed density assumption $h_n={\mathrm{He}}_n/\sqrt{n!}$ is an orthonormal basis. Define the closed number operator by $N\sum c_n{h}_n=\sum n{c}_n{h}_n$ on $\sum n^2|c_n{|}^2<\infty$. This diagonal multiplication operator is self-adjoint and positive semidefinite; polynomial truncations show it is the closure of the polynomial operator.

The Ornstein-Uhlenbeck semigroup is $P_t=e^{-tN}$, or $P_{t}f={\sum }_n{e}^{-nt}{c}_n{h}_n$. It also has the Mehler formula for the Ornstein-Uhlenbeck semigroupTo verify the formula, apply its right side to $e^{sx-s^2/2}$: the Gaussian moment-generating function makes the result $e^{s{e}^{-t}x-s^2{e}^{-2t}/2}$, proving the eigenvalue identity on every Hermite polynomial. Positivity and invariance of Gaussian measure give $L^2$ contraction by Jensen inequality, so polynomial density extends the equality to all $L^2(\gamma )$. The coefficient expansion givesFor continuously differentiable $f$ with bounded derivative, differentiate the Mehler expectation by the dominated convergence theorem to get the Ornstein-Uhlenbeck gradient commutation identityThe same identity extends to the Gaussian Sobolev form domain. Mehler's formula also gives pointwise convergence to the Gaussian mean for such $f$, since bounded derivative permits at most linear growth.

The Gaussian Dirichlet energy is the energy form of $N$:For polynomials this follows from $N=(a^{-}{)}^{\ast }{a}^{-}$, and closure extends it to the form domain. For the given continuously differentiable $f$, Gaussian integration by parts yields $⟨f^{\prime },h_{n-1}⟩=\sqrt{n}{c}_n$, so Parseval identity proves the derivative-energy equality directly. Bounded derivative puts $f$ in the Gaussian Sobolev space, hence in that form domain; it need not be in the full operator domain, so $⟨f,Nf⟩$ is not always a legitimate initial definition.

Use the entropy functional ${\mathrm{Ent}}_{\gamma }(h)=\int h\log hd\gamma -(\int hd\gamma )\log \int hd\gamma$. Begin with bounded smooth $h>0$ bounded away from zero, and put $h_t=P_{t}h$. Invariance and Gaussian integration by parts give the Ornstein-Uhlenbeck entropy dissipation identityThe gradient identity and the permitted weighted Cauchy-Schwarz inequality, applied with weight $h$, implyThe entropy tends to zero as $t\to \infty$ by Mehler's formula and bounded convergence. Integrating in time and using invariance yields ${\mathrm{Ent}}_{\gamma }(h)\le \frac{1}{2}\int |h^{\prime }{|}^2/hd\gamma$. Set $h=f^2$ to obtain the sharp Gaussian logarithmic Sobolev inequalityFor the stated possibly unbounded $f$, apply the argument to smooth positive clipped approximations with a common positive lower bound and uniformly bounded derivatives. They can converge pointwise together with their derivatives and have a common linear-growth bound. Since Gaussian measure integrates every polynomial moment, dominated convergence theorem passes both the entropy and the derivative energy to the limit. This also justifies use of the supplied inequality when $2f{f}^{\prime }$ itself is unbounded. The normalization $\Vert f{\Vert }_1=1$ concerns $f$, and does not imply $\int f^{2}d\gamma =1$; the second term in the entropy definition must be retained.

LetThe entropy functional satisfiesHence the assumed inequality givesBecause $\mathbb{E}X=0$, $\psi (\lambda )/\lambda \to 0$ as $\lambda \to 0$. Integrating from zero to $\lambda$ when $\lambda >0$, and from $\lambda$ to zero and then multiplying by the negative number $\lambda$ when $\lambda <0$, gives in both casesThus $\mathbb{E}{e}^{\lambda X}\le e^{\nu {\lambda }^2/2}$ for every real $\lambda$, which is precisely the sub-Gaussian random variable bound with variance parameter $\nu$. This integration is the Herbst argument.

For a nonnegative function $Z$ of independent random variables, entropy is at most the sum of its conditional coordinate entropies. This tensorization is a basic step of the entropy method.