Joint probability distribution

The joint probability distribution of random variables $X_1,\dots ,X_n$ is the probability distribution of the random vector $(X_1,\dots ,X_n)$. It records both the marginal distributions and their dependence.

A family of random variables is exchangeable if its joint probability distribution is unchanged by finite permutations of its coordinates. Independent and identically distributed random variables are exchangeable, but exchangeable random variables need not be independent random variables.

For an infinite sequence of real exchangeable random variables, conditional on its invariant sigma-algebra of an exchangeable sequence, the coordinates are independent and identically distributed. In particular, for Borel sets $A_0,\dots ,A_p$,Here is the core proof. Let ${\mathcal{S}}_m$ denote invariance under permutations of coordinates $0,\dots ,m$. Symmetrization as conditional expectation identifies the empirical average of each indicator with its conditional expectation on ${\mathcal{S}}_m$. The reverse martingale convergence theorem gives its limit conditional on $\mathcal{S}$. Multiplying the averages sums over all ordered coordinate tuples, including repetitions. Symmetrization of the product itself sums over the distinct tuples. The fraction of repeated tuples is at most $\left(\genfrac{}{}{0ex}{}{p+1}{2}\right)/(m+1)$, so these bounded averages have the same limit. This proves the displayed factorization. Permutation invariance also makes the individual conditional laws identical. A regular conditional distribution on the real line packages that common law as a random probability measure, giving the equivalent mixture-of-iid formulation.

On the coordinate product space of a sequence, the invariant sigma-algebra consists of measurable events unchanged by every finite permutation of coordinates. Its definition is pointwise and does not depend on the probability law. An exchangeable law makes each such permutation measure preserving. events in the tail sigma-algebra are invariant because any finite permutation fixes every sufficiently late coordinate. Conversely, exchangeable invariant events are tail events modulo null sets.

For a real sequence of exchangeable random variables, every event $A$ in its invariant sigma-algebra of an exchangeable sequence agrees almost surely with an event of its tail sigma-algebra. Put $F=1_A$. Approximate $F$ in $L^1$ by its conditional expectation on the first $n+1$ coordinates. Swap that block with the next $n+1$ coordinates. Invariance of $A$ and the law shows that the conditional expectation on the second block is an equally good approximation. It is measurable with respect to the tail after index $n$. The L1 contraction of conditional expectation then bounds $\Vert F-\mathbb{E}[F\mid {\mathcal{G}}_n]{\Vert }_1$ by twice the approximation error. The reverse martingale convergence theorem gives $F=\mathbb{E}[F\mid {\bigcap }_n{\mathcal{G}}_n]$ almost surely. Thresholding the latter at $1/2$ supplies the required tail event. Together with the reverse inclusion, the two sigma-algebras have identical probability completions.

If independent real random variables $X$ and $Y$ have densities $f_X$ and $f_Y$, their sum has the convolution density

Let $X$ take integer values with probabilities $p_n$, and let $Y_{\sigma }$ be an independent centered normal distribution with positive standard deviation $\sigma$. Conditioning on $X$ gives the probability density function of their sum as $g_{\sigma }(z)={\sum }_n{p}_n{\phi }_{\sigma }(z-n)$, where ${\phi }_{\sigma }$ is the normal density. Nonnegativity and Tonelli's theorem show that this density integrates to one. For small $\sigma$, isolated atoms become narrow smooth peaks with their original probability masses.

For each fixed real $t$, dominated convergence givessince $X/\sigma \to 0$ almost surely and ${\phi }_1$ is bounded. Thus every fixed integer-valued distribution acquires a Gaussian central profile on this expanding scale, without any moment assumption on $X$. This is not a claim about uniform relative error in the tails or about finite moments. For example, if $p_n=3/({\pi }^2{n}^2)$ for $n\ne 0$ and $p_0=0$, the expectation of $|X|$ is infinite. Since $\mathbb{E}|Y_{\sigma }|<\infty$, the triangle inequality forces $\mathbb{E}|X+Y_{\sigma }|=\infty$ too. Symmetry then gives equal infinite positive and negative first moments, so there is no ordinary mean. The second moment is also infinite. A Gaussian-looking center can therefore coexist with a heavy-tailed distribution.

Let $X$ have the Standard Cauchy distribution and let $Y$, independently, have the uniform distribution on $[-h,h]$, where $h>0$. The convolution of probability densities gives the probability density of $Z=X+Y$ asThis is an even function. Its derivative isso its unique mode is zero. Its tails satisfy $p_h(z)\sim 1/(\pi z^2)$ as $|z|\to \infty$: multiplying the integral by $z^2$ makes the integrand converge uniformly to $1$ on the bounded interval $[-h,h]$. Consequently both positive and negative parts of the first moment diverge, so its expected value does not exist. The symmetric Cauchy principal value of that moment is zero, which is a different assertion. Adding a bounded independent displacement smooths the probability density without removing the divergent first moment of the Cauchy distribution.

For a taxi starting at any vertex of a rectangle with side lengths $s,t$, a uniform pickup vertex gives approach-distance law uniform on $0,s,t,s+t$. Conditional on that pickup, a uniform distinct destination gives passenger-distance law uniform on $s,t,s+t$, independent of the pickup vertex. Hence the two distance variables are independent despite sharing that vertex. Their convolution has twelve equally likely pairs. Its mean is $7(s+t)/6$ and its variance is $(s^2+t^2)/4+2(s^2+t^2-st)/9$. Repeated numerical sums combine their multiplicities when some distances coincide.

A marginal distribution is the distribution of a selected coordinate or subvector of a random vector, obtained from the joint distribution by integrating or summing over the remaining coordinates.

Marginalization sums or integrates a joint probability distribution over unwanted coordinates to obtain a marginal distribution.

A marginal density is a probability density function for one component of a joint probability distribution. Integrating the joint probability density over the other coordinates gives it; for two components, $f_X(x)=\int f_{X,Y}(x,y)dy$.