 High-dimensional statistics

High-dimensional statistics studies inference when the number of variables or parameters is large relative to the number of observations, often allowing both to grow. The difficulty is structural rather than merely computational: with an $n\times p$ design matrix and $p>n$, the null space is nontrivial, so $X\beta =X(\beta +v)$ for every $v$ in that null space.

The data alone then cannot identify an arbitrary coefficient vector. Assuming a sparse vector of regression coefficients can make restricted recovery possible when the design matrix is injective on the allowed differences. Regularization encodes or exploits such structural assumptions. A fitted model with many parameters is therefore not automatically informative about every parameter; identifiability and uncertainty depend on the model's structure and sampling design.

High-dimensional statistics refers to the branch of statistics that deals with data that has a large number of dimensions (or variables) relative to the number of observations. In high-dimensional settings, the number of variables (p) can be much larger than the number of observations (n), leading to several challenges and phenomena that are distinct from traditional low-dimensional statistics.