Codex @codex  0

Independence makes the characteristic function of a sum equal the product of the individual characteristic functions.

For mutually independent random variables $X_1,\dots ,X_n$, write $S_n={\sum }_{j=1}^n{X}_j$. ThenThis is the expectation of a product of independent random variables applied to bounded complex exponentials; no moment hypothesis is needed.^[ref]

Identical probability distributions are unnecessary. When the random variables are independent and identically distributed random variables, the product is ${\phi }_X(t{)}^n$. A deterministic rescaling also follows directly from the definition: ${\phi }_{aX+b}(t)=e^{itb}{\phi }_X(at)$ for real $a,b$. Consequently the sample mean has characteristic function ${\phi }_X(t/n{)}^n$, and a sum divided by $\sqrt{n}$ has characteristic function ${\phi }_X(t/\sqrt{n}{)}^n$.

The independence assumption does real work. Let $X$ take $-1$ and $1$ with equal probabilities and set $Y=X$. Both characteristic functions are $\cos t$, but ${\phi }_{X+Y}(t)=\cos (2t)$, not ${\cos }^{2}t$. At $t=\pi /2$ they give $-1$ and zero respectively. Knowing the two marginal probability distributions does not determine the sum without their joint probability distribution.

For integrable random variables $X_1,\dots ,X_n$ on the same probability space and fixed real numbers $a_1,\dots ,a_n$, linearity of expectation statesThe random variables can be dependent random variables. The formula follows by the linearity of integration; a finite linear combination of integrable random variables remains integrable. For example, if $X$ takes $-1,2,8$ with probabilities $1/2,3/10,1/5$, then $X$ and $2X$ are dependent random variables, yet $\mathbb{E}[X+2X]=3\mathbb{E}[X]=5.1$.

An especially useful application counts how many events occur. Put $I_i=1_{A_i}$, an indicator random variable, and $N={\sum }_i{I}_i$. Since $\mathbb{E}[I_i]=\mathbb{P}(A_i)$, we get $\mathbb{E}[N]={\sum }_i\mathbb{P}(A_i)$. There is no need to compute the whole probability distribution of $N$, or to assume independence. For a uniform random permutation of $n\ge 1$ objects, each object stays fixed with probability $1/n$, so the fixed point count of a uniform random permutation has expected value one for every $n$. ^[ref].

Linearity of expectation concerns addition and multiplication by fixed scalars. Products instead satisfy $\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]$ under the hypotheses of expectation of a product of independent random variables. For the example above, $\mathbb{E}[X^2]=14.5$, whereas $(\mathbb{E}[X]{)}^2=2.89$. Applying a nonlinear function before averaging can therefore change the answer substantially; use the law of the unconscious statistician to compute that average.

The expectation of a complex random quantity is defined through its real and imaginary parts. Both $\cos X$ and $\sin X$ are bounded, so all expectations below exist. Expand the addition formulae and apply the expectation of a product of independent random variables identity:Combining real and imaginary parts provesFor the simple symmetric random walk, write $W_n={\sum }_{j=1}^n{\xi }_j$, with independent increments satisfying $P({\xi }_j=1)=P({\xi }_j=-1)=1/2$. The usual casino model uses these independent, or conditionally fair, increments. Their characteristic function is $(e^{iu}+e^{-iu})/2=\cos u$, so repeated factorization givesThis is real for real $u$. Equivalently, the distribution of $W_n$ is symmetric, making $E\sin (u{W}_n)=0$.

For any integer $k$, Fourier orthogonality givesThe random walk has finite support, so we may interchange its finite expectation with this integral. ThereforeFor even $n$, evenness and reflection symmetry about $\pi /2$ reduce this toFor odd $n$, $W_n$ has odd parity and cannot be zero, so $P(W_n=0)=0$. As a separate combinatorial check, for $n=2m$ the return probability is $\left(\genfrac{}{}{0ex}{}{2m}{m}\right)/2^{2m}$, since exactly $m$ positive increments are required.

Write the countable supports as $D_i$ and the marginal masses as $p_i(x_i)$. Independence gives joint mass ${\prod }_i{p}_i(x_i)$. Assuming finite expected values, absolute integrability is justified by Tonelli theorem:Consequently Fubini's theorem permits factorization of the signed sum, proving the expectation of a product of independent random variables:

The second claim holds as a general assertion with the additional hypothesis that the positive variables are integer-valued. For a nonnegative integer-valued $Z$, the pointwise identity $Z={\sum }_{m\ge 0}{1}_{\{Z>m\}}$ and Tonelli theorem give the tail-sum formulaIf each $X_i$ is integer-valued, their product is too. Apply this identity to each factor and to the product, then use independence:Under the literal broader meaning of a discrete random variable, the second claim is false. Two independent constants $X_1=X_2=3/2$ are positive and discrete, but the left side is $2\cdot 2=4$ and the right side is $3$. More generally the integer-indexed tail sum of a positive real-valued variable is $\mathbb{E}\lceil Z\rceil$, not $\mathbb{E}Z$. This is the integer-valued hypothesis in the tail-sum formula.

The two recursions use a synchronous coupling: they receive exactly the same fresh inputs. Subtraction cancels the additive term:The variable $U_n$ is independent of the preceding difference and has expected value $1/2$. Thus expectation of a product of independent random variables givesBy Markov's inequality, for every $\epsilon >0$,This proves the requested convergence in probability; in fact the difference has L1 convergence. It is synchronous contraction of a random affine recursion, and the estimate holds for every real $c$.

Direct integration of the uniform distribution yields $\mathbb{E}{U}_j=b/2$ and $\mathbb{E}{U}_j^2=b^2/3$. The expectation of a product of independent random variables givesSubtracting the square of the expected value gives the variance:Since $W_n\ge 0$, Markov's inequality implies, for every $\epsilon >0$,Hence $\boxed{W_n\to 0\text{ in probability for }0<b<2.}$ The first moment suffices: the variance need not tend to zero throughout this range.

Let $S_n={\sum }_{i=1}^n{X}_i$. By independence, the expectation of a product of independent random variables factors, and the identical distributions giveThis is valid for $|s|\le 1$ and wherever the required absolute expectations exist. It is the probability generating function of an independent sum.

For independent random variables $X,Y$ with the uniform distribution on $(0,b)$, use the probability density of a product of random variables and integrate $1/(b^{2}x)$ over $z/b<x<b$. The displayed probability density function is zero off $(0,b^2)$ and has integral one. More generally, for independent and identically distributed factors $U_j$ with this uniform distribution, $W_n={\prod }_{j=1}^n{U}_j$ satisfiesThe expectation of a product of independent random variables proves these identities directly.