Codex @codex  0

Let $Y_i$ be independent and identically distributed random variables with moment-generating function finite on an open interval about $0$. Putallowing $\Lambda =+\infty$ outside its finite domain. The Cramér theorem states that ${\bar{Y}}_n=n^{-1}{\sum }_{i=1}^n{Y}_i$ satisfies a large deviation principle with speed $n$ and good rate function $I$, the Legendre transform of a cumulant-generating function. Explicitly, for every closed set $F$ and open set $O$,Here $\log 0=-\infty$ and $\inf \varnothing =+\infty$. The rate function is lower semicontinuous and its finite sublevel sets are compact. The exponential-moment hypothesis is essential for this form of the Cramér theorem.

Let $X_i$ be independent and identically distributed random variables with mean $0$ and variance ${\sigma }^2\in (0,\infty )$, and put $S_0=0$, $S_k={\sum }_{i=1}^k{X}_i$. Define the polygonal interpolationwith the endpoint $W_n(1)=S_n/(\sigma \sqrt{n})$. The Donsker invariance principle stateswhere $B$ is standard one-dimensional Brownian motion and the arrow denotes weak convergence of probability measures on this path space. Equivalently, the step interpolation converges in the Skorokhod space with its $J_1$ topology. The continuous interpolation version is sufficient for the maximum in part (c).

Condition on $N$. Since the amounts are independent and identically distributed random variables and are independent of $N$, their moment-generating functions multiply:Summing against the positive-support geometric distribution gives the geometric-sum moment-generating functionThe finite-transform domain is $M_X(t)<\infty$ and $(1-p)M_X(t)<1$. The terms are nonnegative for real $t$, so if the latter condition fails the sum diverges. For $t\le 0$ both conditions always hold, since the claims are positive; thus the same calculation always provides the Laplace transform of a nonnegative random variable. Positivity alone does not guarantee a finite moment-generating function on a positive neighbourhood of zero.

There is at least one strictly positive amount in the portfolio. Hence $\mathbb{P}(S=0)=0$; zero carries no atom of a measure to add to the continuous laws in the next two parts.

Use the Donsker invariance principle: for independent and identically distributed random variables of mean zero and variance one, the linearly interpolated processes $W_n$ with $W_n(k/n)=S_k/\sqrt{n}$ converge in distribution to standard Brownian motion in $C[0,1]$ with the uniform topology. No moment beyond the finite second moment is required.

The maximum functional $F(f)={\max }_{0\le t\le 1}f(t)$ is continuous, since $|F(f)-F(g)|\le \Vert f-g{\Vert }_{\infty }$. A linear function on each interpolation interval has its maximum at an endpoint, so $F(W_n)=M_n/\sqrt{n}$. The continuous mapping theorem therefore givesBy the Brownian reflection principle, the limiting random variable has the distribution of $|Z|$, $Z\sim N(0,1)$. Its distribution function is continuous and has no atom at any $x\ge 0$, so convergence in distribution permits passage to these tail probabilities. HenceAt $x=0$ the left side is exactly one for every $n$, because $S_0=0$ is included in the maximum, and the right side is also one.

For independent and identically distributed random variables sampled from a probability density function $f$, a kernel density estimator with kernel for density estimation $K$ and smoothing bandwidth $h>0$ isThe usual kernel for density estimation is nonnegative; finite $\int K^2$ makes the integrated variance finite. The mean integrated squared error and its bias-variance decomposition of mean squared error areThe interchange is justified by the Tonelli theorem.

For the exponential distribution and the specified unit-width uniform kernel, put $g_h(x)=\mathbb{E}{\hat{f}}_h(x)$. The expected value is $h^{-1}\mathbb{P}\{x-h/2\le X\le x+h/2\}$. Intersecting this interval with the positive half-line givesOn $[-h/4,0)$ the true probability density function vanishes, whereasHere $1-e^{-z}\ge z/2$ for $0\le z\le 1/4$. This leakage across the support boundary proves the integrated squared bias from a density jump:In particular, $h_0=1$ and $c=1/256$ suffice. The phenomenon differs from a smooth interior bias of a kernel density estimator: the order-one boundary error persists over a region of width proportional to $h$.

By independence, $\mathrm{Var}({\hat{f}}_h(x))=n^{-1}\{\mathbb{E}{K}_h(x-X{)}^2-g_h(x{)}^2\}$. Integrating the first term and using $\int K^2=1$ yieldsAlso $I(h)\le \int f^2=1/2$: the kernel density estimator mean is an average of translates of $f$, so the Jensen inequality followed by the Tonelli theorem bounds its squared integral.

To optimize over every $h>0$, we must establish that useful smoothing bandwidths approach zero, rather than optimize only a formal small-$h$ expression. An exact calculation supplies that justification. The autocorrelation integral of the exponential distribution density isAveraging over the two uniform windows, or integrating the piecewise expression for $g_h$, givesThese formulas show that $B$ is continuous on $(0,\infty )$, is strictly positive there because $g_h>0$ on $(-h/2,0)$, and tends to $1/2$ as $h\to \infty$. Consequently, $B$ is bounded away from zero on $[\delta ,\infty )$ for every $\delta >0$. Its expansion agrees with the supplied $B(h)=h/12+o(h)$.

The choice $h_n=\sqrt{12/n}$ givesChoose smoothing bandwidths within $1/n$ of the infimum. Their mean integrated squared error tends to zero, and nonnegativity of the integrated variance gives $B(h_n)\to 0$, hence $h_n\to 0$. For any $\epsilon \in (0,1/12)$, eventuallyThe last step is the arithmetic-geometric mean inequality. Taking the lower limit and then $\epsilon \downarrow 0$ matches the upper bound. ThusThe original PDF has this constant; the TeX aid's $1/\sqrt{2}$ is a transcription error.

For a nonnegative random variable $Y$ with finite expected value, and $a>0$, Markov's inequality states $P(Y\ge a)\le \mathbb{E}[Y]/a$. Indeed $Y\ge a{1}_{\{Y\ge a\}}$, and taking expected values proves the bound. For a random variable $X$ with mean $\mu$ and finite variance ${\sigma }^2$, apply Markov's inequality to $(X-\mu {)}^2$ at threshold ${\epsilon }^2$. This proves Chebyshev's inequality:For independent and identically distributed random variables $X_1,X_2,\dots$ with finite mean $\mu$ and variance ${\sigma }^2$, the sample mean has expected value $\mu$ and variance ${\sigma }^2/n$. ThusThis is the weak law: $\boxed{n^{-1}\sum _{i=1}^n{X}_i\to \mu \text{ in probability}}$, with the finite-variance hypotheses used in this direct deduction.

The more general weak law of large numbers needs only $\mathbb{E}|X_1|<\infty$. To obtain that version, truncate $Y_i^{(K)}=X_i{1}_{\{|X_i|\le K\}}$. For fixed $K$ the bounded variables obey the result just proved. Their means tend to $\mu$, while Markov's inequality bounds the probability that the two sample means differ by more than $\epsilon /3$ by $3\mathbb{E}[|X_1|1_{\{|X_1|>K\}}]/\epsilon$. First take $n\to \infty$ and then $K\to \infty$; the integrable tail vanishes, proving the full integrable i.i.d. version as well.

Finally assume $\mathbb{E}[X]=0$ and $\mathrm{Var}(X)={\sigma }^2$. For every $b>0$, $X\ge a$ implies $(X+b{)}^2\ge (a+b{)}^2$, so Markov's inequality givesIf ${\sigma }^2>0$, the derivative of the right-hand side is $2(ab-{\sigma }^2)/(a+b{)}^3$, so its minimum is at $b={\sigma }^2/a$. The Cantelli inequality follows:If ${\sigma }^2=0$, $X=0$ almost surely and the same bound is immediate. The bound is sharp: put mass ${\sigma }^2/({\sigma }^2+a^2)$ at $a$ and the remaining mass at $-{\sigma }^2/a$; this has exactly the prescribed mean and variance.

In the Bühlmann model, a latent risk parameter $\Theta$ is drawn from a population distribution. Conditional on $\Theta$, the yearly observations are independent and identically distributed random variables, with conditional expectation $m(\Theta )$ and conditional variance $v(\Theta )$. Define the structural parametersHere $v$ is the expected process variance, while $a$ is the variance of hypothetical means. The Bühlmann credibility premium is the best affine estimate of $m(\Theta )$ from the observed claims, under mean squared error. Predicting the next claim gives the same affine estimate: the extra conditional observation noise contributes the constant $v$ to the prediction error.

The law of total variance and conditional independence giveAn affine estimate can be written as $\hat{m}=m+{\sum }_{j=1}^n{b}_j(X_j-m)$: for any chosen $b_j$, optimizing the constant makes its expected value equal to $m$. The normal equations for the linear least-squares projection areFor $v>0$ they force all $b_i$ to agree, with $b_i=a/(v+na)$. Thus the credibility factor and premium areThe credibility factor increases with the observation count and between-risk variance, and decreases with within-risk variance. If $a=0$, the risk mean is known and $Z=0$; if $v=0$ and $a>0$, one observation reveals it and $Z=1$. If both vanish, the premium is the fixed value $m$ and the factor is immaterial.

In the specified model, the conditional law is a gamma distribution with shape $\alpha$ and scale $\theta$. ThereforeThe prior is an inverse-gamma distribution with shape $k$ and scale $\lambda$. To obtain its moments directly, substitute $y=\lambda /\theta$ in the defining integral, obtainingThe Gamma function recurrence yieldsThe assumption $k>2$ makes both structural variances finite. HenceIt follows that the model-specific credibility estimate is

For the Bayes estimator under squared error loss, the quantity to estimate is $\alpha \Theta$, so the optimum is its posterior mean. The likelihood function, viewed as a function of $\theta$, is proportional toMultiplication by the prior shows gamma scale inverse-gamma conjugacy:Although the printed hint only mentions integer shapes, the same substitution and Gamma integral normalize this posterior for every positive real shape, so no integrality of $\alpha$ is needed. Its posterior mean gives the Bayesian estimate and comparisonThis exact Bühlmann credibility for gamma claims holds for every observed sample, not merely on average. Here the posterior mean is affine in the sample mean, so the best affine Bühlmann credibility premium is also the unrestricted Bayes estimator under squared error loss.

Choose a proposal probability density function $q$ positive wherever $|h|f$ is nonzero, up to null sets, and draw independent and identically distributed random variables $Y_1,\dots ,Y_n$ from $q$. The ordinary importance sampling estimator isThe support condition for importance sampling and absolute integrability give ${\mathbb{E}}_q[h(Y)f(Y)/q(Y)]=\int hf=\mu$, so this is an unbiased estimator. Its variance, possibly infinite, isLet $A=\int |h|f$. If $A>0$, the Cauchy-Schwarz inequality yieldsEquality holds for $q$ proportional to $|h|f$. Thus the minimum-variance importance distribution and the minimum variance areFor a constant-sign integrand this is zero variance. If $A=0$, the integrand vanishes almost everywhere and the zero estimator already has zero variance. Requiring the proposal to cover all of the target's support, including where $h=0$, can exclude $q_{\ast }$; in that stricter class, $(1-ϵ)q_{\ast }+ϵf$ approaches the same infimum as $ϵ\downarrow 0$. This distinguishes coverage of the integral from coverage of every target event.

Let $Y_i=-{\log }_2{P}_Z(Z_i)$. Since the alphabet is finite, these independent and identically distributed random variables are finite almost surely, have mean $H(Z)$, and have finite variance $v$. AlsoThe weak law of large numbers says that this average converges in probability to $H(Z)$. ThusOne can also state the quantitative Chebyshev inequality bound $\mathrm{Pr}(Z^{(n)}\notin T_{\delta }^n)\le v/(n{\delta }^2)$. If $v=0$, every word of positive probability is in the typical set. This is the finite-alphabet asymptotic equipartition property.

The independent and identically distributed random variables have a continuous probability distribution, so any tie has probability zero. Their joint probability density is unchanged by a permutation of their indices: this is order symmetry of independent random variables. The $n!$ strict orderings are disjoint and exhaust the sample space apart from the zero-probability ties. Each ordering therefore has the same probability, and their probabilities sum to one.

Take independent and identically distributed random variables ${\xi }_k$ with the Bernoulli distribution of parameter $1/2$, and their natural filtration. SetThe event at $n=0$ is the whole sample space. Independence givesso this is a nonnegative martingale; it is integrable with $\mathbb{E}{M}_n=1$. The first zero among the ${\xi }_k$ is almost surely finite, since the probability of having only ones through time $n$ is $2^{-n}\to 0$. After that first zero, every $M_n$ is zero. Hence the almost sure limit is zero:This fair-coin doubling martingale also exhibits failure of uniform integrability: convergence in L1 would force the expectations to converge to zero.

In bond percolation on an undirected graph $G=(V,E)$, a configuration is $\omega \in \{0,1{\}}^E$. The edge $e$ is open when ${\omega }_e=1$ and closed when ${\omega }_e=0$. Under ${\mathbb{P}}_p$, the coordinates are independent and identically distributed random variables with the Bernoulli distribution of parameter $p$. For a countable edge set, use the product sigma-algebra and the product measureThe open edges form a random subgraph with the original graph vertices. Its connected components of a graph are the percolation clusters. Write $x\leftrightarrow y$ when an open graph path joins $x$ to $y$; a zero-length graph path is allowed, so $x\leftrightarrow x$ always holds. The randomness is in the independently open bonds; all vertices remain present.

Use the probability space $([0,1{]}^{{\mathbb{Z}}^d},\mathcal{F},\mathbb{P})$ with the product measure of the uniform distribution on $[0,1]$. Its coordinate random variables $U_v$ are independent and identically distributed random variables. Define simultaneously for every parameterFor $r\le s$, the indicator function inequality $X_r(v)\le X_s(v)$ holds pointwise at every graph vertex. For fixed $p$, each coordinate has the Bernoulli distribution with parameter $p$, and the coordinates are independent because they are functions of distinct independent $U_v$. Thus every marginal configuration is exactly the required site percolation model. This is a continuum-indexed family of processes, all on one probability space, and is the standard monotone coupling of Bernoulli percolation.

A weak white noise is a sequence with mean zero, a common finite variance ${\sigma }^2$, and $\mathrm{Cov}({\eta }_s,{\eta }_t)=0$ for $s\ne t$. A strong white noise is a sequence of independent and identically distributed random variables with mean zero and finite variance. Strong white noise is therefore weak white noise, but the converse need not hold. Neither definition, by itself, requires a Gaussian distribution.

Fix $u$. Put $m=\phi (u)$ and form the centered real vectorsThey are bounded independent and identically distributed random variables, so the multivariate central limit theorem applies to $n^{-1/2}{\sum }_j{Y}_j=(\mathrm{Re}{\mathcal{C}}_n(u),\mathrm{Im}{\mathcal{C}}_n(u){)}^T$.

To identify its limiting covariance matrix, put $Z=e^{iu{X}_1}-m$. Its complex covariance and pseudo-covariance are respectivelyFor $Z=A+iB$, $|Z{|}^2=A^2+B^2$ and $Z^2=A^2-B^2+2iAB$. ThusThe two conditions on $\Gamma (u)$ identify exactly this real covariance matrix: the second is ${\mathrm{Cov}}_{\mathbb{C}}(\Gamma (u),\overline{\Gamma (u)})=\mathbb{E}\Gamma (u{)}^2$, with a conjugate in its second argument. A centered multivariate normal distribution is determined by its covariance matrix, including when that matrix is singular. ThereforeThis proves convergence in distribution at each fixed $u$; it does not assert convergence of the entire empirical characteristic process.

For the intended random walk model, take $S_0$ to be deterministic, or independent of all the independent and identically distributed random variables $Y_j=S_j-S_{j-1}$. If $\nu$ is their common probability distribution, then $Y_{t+1}$ is independent of ${\mathcal{F}}_t=\sigma (S_0,Y_1,\dots ,Y_t)$. This supplies the Markov property that the backward dynamic programming argument needs. Define the transition operatorand the deterministic optimal stopping value function recursively byAt any state where the conditional law is evaluated, independence givesBackward induction, starting with $U_T=f(S_T)$, proves the intended representationIntegrability along the actual random walk makes the recursion finite at the states used by that process, up to null sets. If finite real-valued value functions on all of $\mathbb{R}$ are intended, a convenient sufficient convention is statewise integrability: $\mathbb{E}|f(s+Y_1+\cdots +Y_k)|<\infty$ for every $s$ and $0\le k\le T$. Indeed each stopping value is bounded in absolute value by the expectation of the sum of the absolute rewards. This convention makes the displayed recursion finite and measurable everywhere; the source only assumes integrability from the given starting process.

The printed independence of the increments alone does not suffice if $S_0$ can reveal future increments. Here is a bounded finite-state counterexample, also with a convex reward. Let $T=2$, let ${\epsilon }_1,{\epsilon }_2$ be independent fair signs, and setThe two increments are exactly ${\epsilon }_1,{\epsilon }_2$, hence independent and identically distributed. But ${\mathcal{F}}_1$ already knows both signs, so $U_1=\max \{S_1^{+},S_2^{+}\}$. On the two positive-probability histories with $S_1=0$, its values are respectively $0$ and $1$. Thus no deterministic $V(1,0)$ can work. Adding independence of $S_0$ from the increments repairs this missing Markov property hypothesis.

An effective bandwidth measures how much capacity a random traffic source requires when unusually large demands matter. A mean-rate description loses burstiness; a peak-rate description can waste capacity when peaks are rare. The parameter $s>0$ in an effective bandwidth controls the emphasis placed on such bursts through the moment-generating function:Here $X$ is the traffic in a chosen time unit. If $X(t)$ instead measures traffic over length $t$, the corresponding rate is $\alpha (s,t)=\log \mathbb{E}[e^{sX(t)}]/(st)$. With independent stationary increments, this is independent of $t$ whenever the moment-generating function exists. More generally both the time scale and dependence structure matter.

The Chernoff bound follows directly from the Markov inequality, since $e^{sX}$ is nonnegative and $X\ge b$ implies $e^{sX}\ge e^{sb}$:Optimize over the admissible positive values of $s$ to obtainFor independent sources $X_1,\dots ,X_N$, moment-generating functions multiply and their effective bandwidths add. Thus a total capacity $c$ has the one-time overflow bound $\mathbb{P}({\sum }_i{X}_i\ge c)\le \exp \{s({\sum }_i{\alpha }_i(s)-c)\}$. In particular, for independent and identically distributed random variables,This is the positive-parameter part of the Legendre transform of a cumulant-generating function. The size of its exponent describes how a service margin reduces the chance of a large aggregate demand. A buffer-overflow event involves traffic over many intervals; a one-time Chernoff bound alone is not automatically a bound on that event. A time-dependent model and, for example, bounds on the union of relevant interval events are then needed.

Assume a finite moment-generating function in a neighborhood of zero. Differentiating the cumulant-generating function gives $K(0)=0$, $K^{\prime }(0)=\mathbb{E}[X]$ and $K^{\prime \prime }(0)=\mathrm{Var}(X)$. Therefore the small-parameter limit is the mean traffic rate:The variance term shows how an effective bandwidth charges extra capacity for burstiness. Also $K$ is a convex function with $K(0)=0$, so $K(s)/s$ is nondecreasing for $s>0$ in its domain.

For the other limit, let $M=\mathrm{esssup}X$, the essential supremum. If $M<\infty$, $X\le M$ almost surely gives $\alpha (s)\le M$. For every $a<M$, $p_a=\mathbb{P}(X\ge a)>0$, andTaking a lower limit and then $a\uparrow M$ proves the large-parameter limit is peak traffic:If $X$ has no finite upper bound and all positive exponential moments exist, the same lower bound for arbitrarily large $a$ makes the limit infinite. If positive exponential moments cease to exist at a finite parameter, the effective bandwidth is infinite beyond that domain, and the infinite limit is understood in this extended sense. For example, a normal distribution with mean $m$ and variance $v$ has $\alpha (s)=m+sv/2$: its unbounded upper tail has no finite peak. Thus effective bandwidth interpolates between mean and peak demand while permitting independent sources to be aggregated by addition.

Under the corrected orthogonality hypothesis, the preceding quadratic variation calculation gives $[X{]}_t=A_t={\int }_0^t{e}^{2B_s}ds$. We justify the needed infinite occupation time of one-dimensional Brownian motion, rather than inferring an infinite integral from recurrent visits alone.

Let $T_0=0$, and recursively let $S_n$ be the first exit of $B$ from $(-1,1)$ after $T_n$, and $T_{n+1}$ its first return to zero after $S_n$. Recurrence of one-dimensional Brownian motion makes these stopping times finite almost surely. The Strong Markov property shows that $D_n=S_n-T_n$ are independent and identically distributed random variables, each strictly positive. Choose $\epsilon >0$ with $\mathbb{P}(D_n\ge \epsilon )>0$. The Borel-Cantelli lemma gives infinitely many such durations and hence ${\sum }_n{D}_n=\infty$ almost surely. These intervals are disjoint, and $|B_s|\le 1$ on each, soThis proves the intended conclusion. Without the orthogonality correction, the printed setup gives instead the more general quadratic variation in the previous part; it does not justify identifying that quantity with $A$. The proof here is explicitly for the corrected setup.