Independent random variables

Random variables $X_1,\dots ,X_n$ are independent when their generated sigma-algebras are independent sigma-algebras. Equivalently, their joint distribution is the product of their marginal distributions.

Random variables are pairwise independent when every two distinct members are independent. This is weaker than joint independence, but it is enough to make all off-diagonal covariances vanish for square-integrable variables.

Random variables $X$ and $Y$ are conditionally independent given $Z$ when their conditional joint distribution factorizes into their two conditional marginal distributions almost surely.

Let $A,B$ be indicator random variables with conditional independence given a random variable $T$, and suppose $\mathbb{E}[A\mid T]=\mathbb{E}[B\mid T]=g(T)$. Then the law of total expectation gives $\mathbb{E}[AB]=\mathbb{E}[g(T{)}^2]$ and $\mathbb{E}[A]=\mathbb{E}[B]=\mathbb{E}[g(T)]$. Thus their covariance is the variance of $g(T)$ and is nonnegative. The result explains correlated component evolutionary states in a coeval binary population without correlation between initial masses.

For a random vector indexed by vertices $V$, its conditional independence graph is the undirected graph with $u-v$ absent exactly when $Z_u\perp Z_v\mid Z_{V\setminus \{u,v\}}$. In a nonsingular multivariate normal distribution, its edges are the nonzero off-diagonal entries of the precision matrix.

Fit the conditional law of each variable given all others and use its predictor support to estimate its neighbours. In a Gaussian graphical model, this is linear regression with coefficients given by conditional regression coefficients from a Gaussian precision matrix. The Nodewise Lasso supplies sparse fits; the OR rule for nodewise graph selection and AND rule for nodewise graph selection combine possibly disagreeing directional supports into an undirected graph.

Minimizing the squared error over each free regression intercept gives the displayed formula. Substitution replaces all data columns by centred columns. The regression intercept is generally not the marginal mean of the response when predictor means are nonzero; the distinction matters in conditional regression coefficients from a Gaussian precision matrix.

The intersection rule retains an edge only when both directions of nodewise regression select it. It is more conservative than the OR rule for nodewise graph selection; the distinction matters because separate finite-sample fits can disagree.

The union rule retains an edge whenever either of the two nodewise regression fits selects the other variable. It is more inclusive than the AND rule for nodewise graph selection and makes an undirected edge decision from potentially asymmetric estimated neighbourhoods.

For a nondegenerate multivariate normal distribution, the conditional independence graph is read from zeros in its precision matrix. Conditional independence of two components given all others is equivalent to a zero cross term in their conditional Gaussian quadratic density, hence to ${\Omega }_{jk}=0$. It does not generally correspond to a zero in the covariance matrix.

For Gaussian data, regress each variable on all remaining variables using Lasso. Nonzero coefficients estimate its neighbours in the conditional independence graph. Symmetrize the separate regressions using either the union or the intersection of their selected directed neighbourhoods.

If D-separation and conditional independence are equivalent for a Directed acyclic graph, its moral graph is its conditional independence graph. After conditioning on all other vertices, an active path can only be a direct edge or a two-edge collider through a common child: two adjacent interior vertices cannot both be colliders. These are exactly the moral edges.

The contraction axiom says thatIt follows directly by multiplying the corresponding conditional-density factorizations.

The decomposition axiom says that $X\perp (Y,W)\mid Z$ implies both $X\perp Y\mid Z$ and $X\perp W\mid Z$. It follows by marginalizing the joint conditional distribution over the discarded variable.

To test $X\perp \perp Y\mid Z$, regress $X$ and $Y$ on $Z$ and average the products of their residuals. Dividing the scaled average by its empirical second moment produces the generalized covariance measure statistic. Small mean-square nuisance-regression errors and a product-rate condition make it asymptotically pivotal.

If the numerator of the generalized covariance measure statistic obeys a central limit theorem with variance $\mathbb{E}({\epsilon }^2{\xi }^2)$ and its empirical residual-product second moment converges in probability to the same positive quantity, the Slutsky theorem makes the studentized statistic converge in distribution to $N(0,1)$.

If $X_i$ are independent centered square-integrable random variables in a Hilbert space, then $\mathbb{E}⟨X_i,X_j⟩=0$ for $i\ne j$, and hence

A family of independent random variables is independent and identically distributed when every variable has the same probability distribution.