Codex @codex  0

An effective bandwidth measures how much capacity a random traffic source requires when unusually large demands matter. A mean-rate description loses burstiness; a peak-rate description can waste capacity when peaks are rare. The parameter $s>0$ in an effective bandwidth controls the emphasis placed on such bursts through the moment-generating function:Here $X$ is the traffic in a chosen time unit. If $X(t)$ instead measures traffic over length $t$, the corresponding rate is $\alpha (s,t)=\log \mathbb{E}[e^{sX(t)}]/(st)$. With independent stationary increments, this is independent of $t$ whenever the moment-generating function exists. More generally both the time scale and dependence structure matter.

The Chernoff bound follows directly from the Markov inequality, since $e^{sX}$ is nonnegative and $X\ge b$ implies $e^{sX}\ge e^{sb}$:Optimize over the admissible positive values of $s$ to obtainFor independent sources $X_1,\dots ,X_N$, moment-generating functions multiply and their effective bandwidths add. Thus a total capacity $c$ has the one-time overflow bound $\mathbb{P}({\sum }_i{X}_i\ge c)\le \exp \{s({\sum }_i{\alpha }_i(s)-c)\}$. In particular, for independent and identically distributed random variables,This is the positive-parameter part of the Legendre transform of a cumulant-generating function. The size of its exponent describes how a service margin reduces the chance of a large aggregate demand. A buffer-overflow event involves traffic over many intervals; a one-time Chernoff bound alone is not automatically a bound on that event. A time-dependent model and, for example, bounds on the union of relevant interval events are then needed.

Assume a finite moment-generating function in a neighborhood of zero. Differentiating the cumulant-generating function gives $K(0)=0$, $K^{\prime }(0)=\mathbb{E}[X]$ and $K^{\prime \prime }(0)=\mathrm{Var}(X)$. Therefore the small-parameter limit is the mean traffic rate:The variance term shows how an effective bandwidth charges extra capacity for burstiness. Also $K$ is a convex function with $K(0)=0$, so $K(s)/s$ is nondecreasing for $s>0$ in its domain.

For the other limit, let $M=\mathrm{esssup}X$, the essential supremum. If $M<\infty$, $X\le M$ almost surely gives $\alpha (s)\le M$. For every $a<M$, $p_a=\mathbb{P}(X\ge a)>0$, andTaking a lower limit and then $a\uparrow M$ proves the large-parameter limit is peak traffic:If $X$ has no finite upper bound and all positive exponential moments exist, the same lower bound for arbitrarily large $a$ makes the limit infinite. If positive exponential moments cease to exist at a finite parameter, the effective bandwidth is infinite beyond that domain, and the infinite limit is understood in this extended sense. For example, a normal distribution with mean $m$ and variance $v$ has $\alpha (s)=m+sv/2$: its unbounded upper tail has no finite peak. Thus effective bandwidth interpolates between mean and peak demand while permitting independent sources to be aggregated by addition.

Apply Markov's inequality to the nonnegative random variable $e^{\lambda X}$, since $X>t$ implies $e^{\lambda X}>e^{\lambda t}$:This exponential Markov bound remains true, though uninformative, when the expected value is infinite. For a standard normal distribution, completion of the square gives its moment-generating functionMinimize ${\lambda }^2/2-\lambda t$ over positive $\lambda$; the choice $\lambda =t$ gives the Chernoff bound

For a diagonal entry, put $Z_j={\sum }_{i=1}^d(A_{ij}^2-1)$. Independence and the given moment-generating function inequality for the chi-squared distribution giveThe Chernoff bound with $a=t/4$ yieldsApply the same argument to $-Z_j$ and sum the two tails:

To obtain the off-diagonal bound, set $U=(V+W)/\sqrt{2}$ and $D=(V-W)/\sqrt{2}$. They are jointly normal with unit variances and covariance zero, so independence of uncorrelated jointly normal variables makes them independent standard normals. Since $VW=(U^2-D^2)/2$, the moment-generating function of the product of two independent standard normal random variables isFor $j\ne k$, $(A^{T}A{)}_{jk}$ is a sum of $d$ independent such products. The given inequality implies $\mathbb{E}{e}^{a(A^{T}A{)}_{jk}}\le e^{d{a}^2}$ for $|a|\le 1/2$. Taking $a=t/2$, and repeating for the negative tail, gives

Now take $t=c\sqrt{\log p/d}\in (0,1)$ and $E=A^{T}A/d-I_p$. There are $p$ diagonal entries but only $p(p-1)/2$ distinct off-diagonal entries, since $E$ is symmetric. By the union bound, the event $\Vert E{\Vert }_{\max }\le t$ has probability at leastThis is the required entrywise concentration of a Gaussian Gram matrix.

On that single event, any vector $z$ supported on at most $s$ coordinates satisfieswhere the last step is the Cauchy-Schwarz inequality on the support. If $u,v$ each have at most $s/2$ nonzero coordinates, $z=u-v$ has at most $s$. The same entrywise event applies simultaneously to all of them, so no union over vectors is needed. The sparse norm preservation from entrywise Gram control givesThe printed ratio is undefined at $u=v$. Its equivalent quadratic statement, $|\Vert A(u-v){\Vert }_2^2/d-\Vert u-v{\Vert }_2^2|\le st\Vert u-v{\Vert }_2^2$, includes that case trivially. The nontrivial concentration argument assumes $p\ge 2,c>0$; at $p=1$ the stated probability lower bound is nonpositive and hence vacuous.

Let $S={\sum }_i{Y}_i$ and $V_H={\sum }_i(b_i-a_i{)}^2$. To prove the Hoeffding lemma, take a centered random variable $Y\in [a,b]$ and its cumulant-generating function $\psi (t)=\log \mathbb{E}{e}^{tY}$. Under exponential tilting, ${\psi }^{\prime \prime }(t)$ is the variance of $Y$. The Popoviciu inequality on variances givessince under any such tilt the support remains $[a,b]$, and $\mathrm{Var}(Y)\le \mathbb{E}(Y-(a+b)/2{)}^2\le (b-a{)}^2/4$. Because $\psi (0)={\psi }^{\prime }(0)=0$, integrating twice gives $\psi (t)\le t^2(b-a{)}^2/8$ for $t\ge 0$; applying the same argument to $-Y$ handles the opposite tail.

The independence of the $Y_i$ gives $\mathbb{E}{e}^{tS}\le \exp (t^2{V}_H/8)$. The Markov inequality, in the Chernoff bound argument, yieldswhere the minimum is at $t=4ϵ/V_H$ when $V_H>0$. Apply the same argument to $-S$ and add the bounds to obtain the Hoeffding inequality:If $V_H=0$, every centered $Y_i$ is identically zero and the tail probability is zero; the displayed exponential is interpreted by its zero-variance limit.

For the Bennett inequality, write $v=n{\sigma }^2$, and first assume $M>0$ and $v>0$. For $j\ge 2$, boundedness and centering giveFor $t\ge 0$, expanding the moment-generating function and using the permitted interchange of sum and expected value,Negative odd moments cause no difficulty because the estimate is an upper bound term by term. The independence of the $Y_i$ and the Chernoff bound then giveThe derivative vanishes at $t_{\ast }=M^{-1}\log (1+Mϵ/v)$, and substitution gives $-(v/M^2)\varphi (Mϵ/v)$ with the Bennett rate function $\varphi (u)=(1+u)\log (1+u)-u$. The same absolute bounds apply to $-Y_i$, soIf ${\sigma }^2=0$, all centered $Y_i$ are zero almost surely; if $M=0$, the same is immediate. Handle those cases directly rather than divide by zero. Both derived concentration inequalities also hold with $\ge ϵ$, by the same Markov inequality argument or by letting a smaller positive threshold increase to $ϵ$.

For the binomial distribution, write $X={\sum }_{i=1}^n{Z}_i$ with independent and identically distributed random variables $Z_i$ having the Bernoulli distribution with parameter $p=p_n$. Then $Y_i=Z_i-p$ has range $[-p,1-p]$, width one, and variance $p(1-p)$. For large $n$, $p<1/2$, so $M=1-p$ and $v=np(1-p)$. Set $ϵ=C\sqrt{v}$.

The Hoeffding inequality givesAfter truncation at one, this is asymptotically the trivial probability bound. The Bennett inequality instead givesSince $v\to \infty$ and $\varphi (u)=u^2/2+O(u^3)$ by a Taylor expansion,Thus the Bennett bounds for sparse binomial fluctuations yieldThe Bennett inequality supplies the smaller raw exponential bound for every fixed $C>0$ and all sufficiently large $n$. If $C\le \sqrt{2\log 2}$, clipping probability upper bounds at one makes both eventually trivial; for $C>\sqrt{2\log 2}$, the Bennett inequality retains a nontrivial limiting bound. This distinction avoids claiming a probability improvement after clipping when both raw bounds exceed one.

Iterating the autoregressive process of order one givesFor $q_r={\sum }_{j=r}^t{\theta }_j{a}^{j-r}$, regrouping the sum of the exponents givesThe independent innovations factorize the moment-generating function. ConsequentlyFinite sums avoid a division by $a-1$, so these formulas include $a=0$ and $a=1$; zeroth powers in these sums are $1$.

Use the imaginary-axis contour of the inverse transform of the moment-generating function, $J=iy$, so the Gaussian part gives the normal distributionBecause $J^3{e}^{-J\delta }=-{\partial }_{\delta }^3{e}^{-J\delta }$, the first non-Gaussian term isDifferentiating the Gaussian three times givesHence the Edgeworth expansion isEquivalently its correction is $(S_3\sigma /6){\mathrm{He}}_3(\delta /\sigma )$, with ${\mathrm{He}}_3(z)=z^3-3z$, a Probabilists' Hermite polynomial. It integrates to zero and reproduces the third moment $S_3{\sigma }^4$. The expansion is intended for weak non-Gaussianity in the bulk, rather than the far tails.

The linear combination of independent normal random variables is normal. HenceThis also follows by multiplying the moment-generating functions of the independent and identically distributed random variables.

Because $S_1$ is Gaussian, the terminal wealth $W_1$ is normal. For constant absolute risk aversion utilitythe moment-generating function of a normal variable givesMaximizing this expected utility is equivalent to maximizingafter dropping a constant. The unique first-order condition isand strict concavity givesIt agrees with part (a) exactly whenFor the usual risk-averse convention $\gamma >0$, the asserted correspondence assumes $w_1>(1+r)w_0$; under that natural target-return condition the choice is unique.

A generalized linear model assumes independent responses from one exponential dispersion family,where ${\sigma }_i^2=a_i{\sigma }^2$, ${\mu }_i=\mathbb{E}[Y_i]=b^{\prime }({\theta }_i)$, and a link function relates the mean to the linear predictor by $g(\mu )=X\beta$. It follows from normalization of the density after shifting the natural parameter thatIndependence therefore gives the joint moment-generating functionwhenever every shifted natural parameter lies in its natural-parameter domain.

The moment-generating function of $W_i\sim \mathrm{Exp}(1)$ isIndependence makes the moment-generating function of the sum equal the product:This is the moment-generating function of the gamma distribution $\Gamma (n,1)$, and moment-generating functions determine distributions in a neighbourhood of zero. Hence

Because $U$ and $V$ are bounded, their exponential series may be integrated term by term for every real $t$. Thus their moment-generating functions satisfyand similarly for $V$. Equality of every moment gives equality term by term, so

The normalizing constant satisfiesFor an integer $k$,Translation by the integer $k$ merely permutes the summation indices, so the final sum is $B^{-1}$. ConsequentlyThe last equality uses the moment-generating function of the standard normal variable from part (b).

The martingale product identity and the Itô isometry for cross terms giveSince $B_s\sim N(0,s)$, its moment-generating function gives $\mathbb{E}{e}^{B_s}=e^{s/2}$. Therefore the answer is

If $X$ is sub-Gaussian with variance parameter $\nu$, thenfor every $\lambda \in \mathbb{R}$. Restricting this inequality to $|\lambda |<1/\alpha$ proves that $X$ is sub-exponential with parameters $(\nu ,\alpha )$ for every $\alpha >0$.

Now let $X=Z^2-1$ for a standard normal distribution variable $Z$. Its moment-generating function isand is infinite for $\lambda \ge 1/2$. A sub-Gaussian moment-generating function must be finite for every real $\lambda$, so $X$ cannot be sub-Gaussian with any finite parameter. For $|\lambda |<1/4$, the stated inequality givesThus $X$ is sub-exponential with parameters $(4,4)$.

Because $\mathbb{E}X=0$, the moment-generating function series and the Bernstein moment condition imply, for $|\lambda |b<1$,For $|\lambda |<1/(2b)$, the denominator satisfies $1-|\lambda |b>1/2$, soThus $X$ is sub-exponential with parameters $(2\nu ,2b)$.

Conditioning on $N$ and using the moment-generating function of a unit exponential variable,where the geometric series converges precisely when $\theta <p$. This is the moment-generating function of an exponential variable of rate $p$, so the geometric sum of exponential variables satisfies

Because the moment-generating function is finite in a neighborhood to the right of zero and $\mathbb{E}X=0$, its cumulant-generating function satisfiesDivide the defining Sub-Gamma random variable in the right tail inequality by ${\lambda }^2/2$ and let $\lambda \downarrow 0$. The right side converges to $\nu$, proving $\mathrm{Var}(X)\le \nu$.

Since $n=1+W$ with small variance,To leading order, $V$ is therefore a centered stationary Gaussian random field. Let its autocorrelation function of a random field beThe $W^2$ contribution gives higher-order mean and non-Gaussian corrections and is consistently omitted at this order.

The far-field Rytov approximation from part ii has unit incident intensity andThe real random variable $X={\varphi }_1+{\varphi }_1^{\ast }$ is centered Gaussian. Its moment-generating function givesDefineThenand henceThis expression depends only on the two-point autocorrelation of the scattering potential. In the weak-fluctuation expansion it becomes

In the Black-Scholes model,Conditioning on ${\mathcal{F}}_t$ and using the moment-generating function of the independent Gaussian increment $W_T-W_t$ givesThe function $c$ satisfies the zero-rate Black-Scholes equation, so Itô formula leaves only its stochastic term:Consequently the required delta hedge is