Codex @codex  0

The Lévy continuity theorem states that if characteristic functions ${\phi }_n$ of probability distributions ${\mu }_n$ converge pointwise on $\mathbb{R}$ to $\phi$, and $\phi$ is continuous at zero, then $\phi$ is the characteristic function of a unique probability distribution $\mu$ andHere the arrow denotes weak convergence of probability measures, equivalently convergence in distribution of associated random variables. Conversely, weak convergence of probability measures implies pointwise convergence of their characteristic functions to the characteristic function of the limit. The continuity-at-zero condition is essential to ensure that the pointwise limit is a characteristic function.

Write $A_n=n^{-1}{\sum }_{j=1}^n{X}_j$ and $\phi (t)=\mathbb{E}{e}^{it{X}_1}$. The given two-sided limit $|f(u)|/u\to 0$ implies $|f(u)|/|u|\to 0$, so $f(u)=o(|u|)$. For each fixed real $t$, independence and the characteristic function of a sum of independent variables giveSet $z_n=iat/n+f(t/n)$. For $t\ne 0$, $n{z}_n\to iat$ and $z_n=O(n^{-1})$. The local Taylor expansion of the complex logarithm at $1$ givesThus $\mathbb{E}{e}^{it{A}_n}\to e^{iat}$; at $t=0$ the identity is immediate. The limit is continuous at zero and is the characteristic function of the constant random variable $a$. The Lévy continuity theorem proves the weak law from a characteristic-function expansion:No integrability hypothesis has been used. The conclusion is also convergence in probability by (d)'s convergence in distribution to a constant implies convergence in probability. If the word “constant” were to allow complex $a$, the characteristic function symmetry $\phi (-t)=\overline{\phi (t)}$ forces $a=\overline{a}$, so it is necessarily real.

Multiplication by the integrating factor $e^{t/2}$ and the Itô product rule give the Ornstein-Uhlenbeck processThe two summands are independent centered normal random variables, because $X_0$ is independent of the Brownian motion and the integrand is deterministic. By the Itô isometry, their variances sum to $e^{-t}+{\int }_0^t{e}^{-(t-u)}du=1$. ThusFor $t\ge s$, split the Itô integral at $s$ to obtain $X_t=e^{-(t-s)/2}{X}_s+{\int }_s^t{e}^{-(t-u)/2}d{B}_u$, where the latter term is centered and independent of $X_s$. ConsequentlyIn particular this initialization makes the Gaussian process a stationary process.

Use the standard definition of a local martingale, including an integrable initial value, and let $T_n\uparrow \infty$ be a localizing sequence. If $X\ge 0$, the Fatou lemma gives $\mathbb{E}{X}_t\le {\mathrm{liminf}}_n\mathbb{E}{X}_{t\wedge T_n}=\mathbb{E}{X}_0<\infty$. For $s\le t$, the Conditional Fatou lemma givesFor clarity, the Conditional Fatou lemma itself follows by setting $U_m={\inf }_{n\ge m}{U}_n$ for nonnegative $U_n$, applying the conditional version of the monotone convergence theorem to $U_m\uparrow \mathrm{liminf}{U}_n$, and observing $\mathbb{E}(U_m\mid {\mathcal{F}}_s)\le {\inf }_{n\ge m}\mathbb{E}(U_n\mid {\mathcal{F}}_s)$. Hence a nonnegative local martingale is a supermartingale.

If $|X_t|\le C$ simultaneously for all $t$, with a deterministic $C<\infty$, the dominated convergence theorem applied to the stopped martingale identity gives, for each $A\in {\mathcal{F}}_s$,This is exactly the conditional expectation identity, so a bounded local martingale is a true martingale. The same proof works for deterministic bounds on every finite time interval; merely having a finite random pathwise bound supplies no dominating integrable random variable and is a different convention.

Let $S={\sum }_i{Y}_i$ and $V_H={\sum }_i(b_i-a_i{)}^2$. To prove the Hoeffding lemma, take a centered random variable $Y\in [a,b]$ and its cumulant-generating function $\psi (t)=\log \mathbb{E}{e}^{tY}$. Under exponential tilting, ${\psi }^{\prime \prime }(t)$ is the variance of $Y$. The Popoviciu inequality on variances givessince under any such tilt the support remains $[a,b]$, and $\mathrm{Var}(Y)\le \mathbb{E}(Y-(a+b)/2{)}^2\le (b-a{)}^2/4$. Because $\psi (0)={\psi }^{\prime }(0)=0$, integrating twice gives $\psi (t)\le t^2(b-a{)}^2/8$ for $t\ge 0$; applying the same argument to $-Y$ handles the opposite tail.

The independence of the $Y_i$ gives $\mathbb{E}{e}^{tS}\le \exp (t^2{V}_H/8)$. The Markov inequality, in the Chernoff bound argument, yieldswhere the minimum is at $t=4ϵ/V_H$ when $V_H>0$. Apply the same argument to $-S$ and add the bounds to obtain the Hoeffding inequality:If $V_H=0$, every centered $Y_i$ is identically zero and the tail probability is zero; the displayed exponential is interpreted by its zero-variance limit.

For the Bennett inequality, write $v=n{\sigma }^2$, and first assume $M>0$ and $v>0$. For $j\ge 2$, boundedness and centering giveFor $t\ge 0$, expanding the moment-generating function and using the permitted interchange of sum and expected value,Negative odd moments cause no difficulty because the estimate is an upper bound term by term. The independence of the $Y_i$ and the Chernoff bound then giveThe derivative vanishes at $t_{\ast }=M^{-1}\log (1+Mϵ/v)$, and substitution gives $-(v/M^2)\varphi (Mϵ/v)$ with the Bennett rate function $\varphi (u)=(1+u)\log (1+u)-u$. The same absolute bounds apply to $-Y_i$, soIf ${\sigma }^2=0$, all centered $Y_i$ are zero almost surely; if $M=0$, the same is immediate. Handle those cases directly rather than divide by zero. Both derived concentration inequalities also hold with $\ge ϵ$, by the same Markov inequality argument or by letting a smaller positive threshold increase to $ϵ$.

For the binomial distribution, write $X={\sum }_{i=1}^n{Z}_i$ with independent and identically distributed random variables $Z_i$ having the Bernoulli distribution with parameter $p=p_n$. Then $Y_i=Z_i-p$ has range $[-p,1-p]$, width one, and variance $p(1-p)$. For large $n$, $p<1/2$, so $M=1-p$ and $v=np(1-p)$. Set $ϵ=C\sqrt{v}$.

The Hoeffding inequality givesAfter truncation at one, this is asymptotically the trivial probability bound. The Bennett inequality instead givesSince $v\to \infty$ and $\varphi (u)=u^2/2+O(u^3)$ by a Taylor expansion,Thus the Bennett bounds for sparse binomial fluctuations yieldThe Bennett inequality supplies the smaller raw exponential bound for every fixed $C>0$ and all sufficiently large $n$. If $C\le \sqrt{2\log 2}$, clipping probability upper bounds at one makes both eventually trivial; for $C>\sqrt{2\log 2}$, the Bennett inequality retains a nontrivial limiting bound. This distinction avoids claiming a probability improvement after clipping when both raw bounds exceed one.

A pricing kernel is a strictly positive random variable $R$ such that $R{P}_1^i$ is integrable andIt is a one-period state-price density, converting a terminal payoff into its initial price. Strict positivity means $R>0$ almost surely, not merely $R\ge 0$. When cash has price $1$ at both dates, this identity forces $\mathbb{E}R=1$, so $d\mathbb{Q}=Rd\mathbb{P}$ defines an equivalent martingale measure.

For any holdings $h$, the pricing kernel identity implies $V_0=\mathbb{E}[R{V}_1]$. If $V_1\ge 0$ almost surely, then $V_0\ge 0$, so an arbitrage cannot have $V_0<0$. If also $V_0=0$, then the nonnegative integrable random variable $R{V}_1$ has zero expectation and hence is zero almost surely. Since $R>0$ almost surely, $V_1=0$ almost surely, ruling out a positive terminal gain. Therefore a pricing kernel rules out arbitrage.

Let ${\tau }_n\uparrow \infty$ be a localizing sequence for the discrete-time local martingale $X$. For $t\ge 1$, the increment of the stopped martingale isand $\{{\tau }_n\ge t\}\in {\mathcal{F}}_{t-1}$. Thus, for any $A\in {\mathcal{F}}_{t-1}$,The assumed integrability gives the dominating random variable $|X_t|+|X_{t-1}|$. The dominated convergence theorem now yields $\mathbb{E}[1_A(X_t-X_{t-1})]=0$, or $\mathbb{E}[X_t\mid {\mathcal{F}}_{t-1}]=X_{t-1}$. Iterating the tower property of conditional expectation proves an integrable discrete-time local martingale is a true martingale. The proof uses the discrete-time increment indicator; the conclusion does not extend to continuous time.

Fix the maturity horizon. Since $YB$ is a nonnegative local martingale and $r$ is bounded, $\mathbb{E}{Y}_T<\infty$: $B_T\ge B_0{e}^{-\Vert r{\Vert }_{\infty }T}$ and $\mathbb{E}(Y_T{B}_T)\le B_0$. The bounded nonnegative payoff thus makes $Y_T{\xi }_T$ integrable. SetThe Brownian martingale representation theorem yields $N_t=N_0+{\int }_0^t{\zeta }_{s}d{W}_s$, using its locally square-integrable version for an integrable terminal random variable. The Itô formula for $N/Y$ givesChoose the replicating strategyIts self-financing portfolio equation has exactly the displayed drift and diffusion, because $\mu -r=\sigma \lambda$. All coefficients are locally integrable after stopping; the holdings are predictable in the augmented natural Brownian filtration. This construction has $X_t\ge 0$ and $X_T={\xi }_T$, so it is an admissible replicating strategy under the question's nonnegative-wealth convention.

For any other nonnegative self-financing portfolio $\hat{X}$ replicating the same payoff, part (b) implies ${\hat{X}}_0\ge \mathbb{E}[Y_T{\xi }_T]$. The constructed strategy attains equality, since $Y_0=1$ and $N_0=\mathbb{E}[Y_T{\xi }_T]$ in the augmented natural Brownian filtration. HenceThis is deflator-based claim replication; it does not require upgrading the local martingale deflator to a true martingale density.

Write $N_m=|B(m)|$ and $\theta =\theta (p)>0$. By translation invariance and linearity of expectation,Let $r={\mathbb{P}}_p(R(m)\ge \theta N_m/2)$. Since $0\le R(m)\le N_m$, splitting the expectation at the threshold givesRearranging produces the slightly stronger estimateOnly the mean and boundedness of the random variable are used; independence of the vertex-membership events is unnecessary.

For finite-valued discrete random variables, the conditional entropy isThe chain rule for information entropy in two orders givesThereforeusing nonnegativity of conditional entropy and the fact that conditioning reduces entropy. This proves the required inequality.

For Fano's inequality, let $X$ take values in an alphabet of size $m$, let $\hat{X}=g(Y)$ be any estimator, and putBecause $E$ is determined by $(X,Y)$,Now $H(E\mid Y)\le H(E)=h_2(p_e)$. If $E=0$, $X$ is determined by $Y$; if $E=1$, at most $m-1$ values remain possible. HenceCombining the bounds yields

The quantile function $X_{\mu }=F_{\mu }^{-1}$ is nondecreasing, hence a Borel measurable function. More explicitly, for every $x\in \mathbb{R}$,Thus $X_{\mu }$ is a random variable, and because Lebesgue measure is being used on $(0,1)$,Its cumulative distribution function is therefore $F_{\mu }$, soThis is inverse transform sampling.

The chain rule for information entropy and conditioning reduces entropy giveThus subadditivity of information entropy holds for two finitely valued random variables. Applying this inequality to $(X_1,\dots ,X_{n-1})$ and $X_n$ and then inducting gives

For an integrable random variable $X$ and a sub-sigma-algebra $\mathcal{G}\subseteq \mathcal{F}$, the conditional expectation $Y=\mathbb{E}[X\mid \mathcal{G}]$ is an integrable, $\mathcal{G}$-measurable random variable satisfyingfor every $G\in \mathcal{G}$.

Yes. Almost-sure convergence givesalmost surely. Since the random variables are nonnegative, Fatou lemma applies and givesThe inequalities remain valid in the extended nonnegative reals, so no integrability assumption beyond nonnegativity is required.

Uniform orientation on the unit sphere means that probability is proportional to the solid-angle elementBecause the minor axis is unoriented, it is enough to take $0\le i\le \pi /2$. After integration over the azimuth and normalization,Equivalently, the random variable $\mu =\cos i$ has the uniform distribution on $[0,1]$.

For a nonnegative random variable $Y$ and $a>0$, the Markov inequality isIndeed, $Y\ge a{1}_{\{Y\ge a\}}$ pointwise. Taking expectations and dividing by $a$ proves the claim.

A centered random variable $X$ is sub-Gaussian with parameter $a>0$ whenfor every $t\in \mathbb{R}$.

The Bernstein concentration inequality for products of sub-Gaussian variables quoted in the course says that if each coordinate of the identically distributed pairs $(U_i,V_i)$ is sub-Gaussian with parameter $\sigma /4$, thenSince $U^{T}V={\sum }_i{U}_i{V}_i$, this is the required bound. It follows by observing that a product of sub-Gaussian variables is sub-exponential and applying Bernstein's inequality to the independent centered products.

For any vector $\delta$ admissible in the definition of ${\varphi }_{\Sigma }^2$, one has $\Vert \delta {\Vert }_1\le 4$. The entrywise maximum norm bound therefore impliesBy the definition of the compatibility constant, ${\delta }^T\Sigma \delta \ge {\varphi }_{\Sigma }^2/s$, and henceTaking the infimum proves ${\varphi }_{\Theta }^2\ge {\varphi }_{\Sigma }^2/2$. This is the stability of a compatibility constant under entrywise perturbation.

Put $C=X^{T}X/n$. Applying the product concentration bound with the stated $t$ and using $t\le {\sigma }^2/3$ gives, for every $j,k$,There are $p(p+1)/2$ distinct entries in the symmetric matrix, so the union bound shows that the eventhas probability at least $1-p/(p+1{)}^2\ge p/(p+1)$.

On $\mathcal{E}$, $|C_{jj}-1|\le t$. Since $|{\Sigma }_{jk}|\le 1$ by Cauchy-Schwarz, normalization of the sample columns givesChoosing one coordinate of $S$ in the infimum shows ${\varphi }_{\Sigma }^2\le s$, so the assumed bound on $t$ is below one and, more precisely,The perturbation result now gives ${\varphi }_{\hat{\Sigma }}^2\ge {\varphi }_{\Sigma }^2/2$ throughout $\mathcal{E}$, and therefore

Write $S_n={\sum }_{j=1}^n{Y}_j$ for the partial sum. Fix an integer $M\ge 1$. The tail-sum formula for expectation, applied to the nonnegative random variable $|Y_1|$, givesbecause $\mathbb{E}|Y_1|=\infty$. The events $\{|Y_n|>Mn\}$ are independent, so the second Borel--Cantelli lemma shows that $|Y_n|/n>M$ infinitely often with probability one. Taking the countable intersection over $M\in \mathbb{N}$ yieldsalmost surely.

Now $Y_n=S_n-S_{n-1}$, and the triangle inequality givesIf the limit superior of $|S_n|/n$ were finite, the right-hand side would have finite limit superior, contradicting the preceding conclusion. Thereforealmost surely.

The Markov inequality states that for a nonnegative random variable $Y$ and $a>0$,For $t>0$, the event $X\ge x$ is the same as $e^{tX}\ge e^{tx}$. Applying Markov's inequality to the nonnegative variable $e^{tX}$ gives the exponential Markov bound