Debiased-Lasso asymptotic normality

In $Y=X{\beta }^0+\epsilon$, put $\hat{\Sigma }=X^{\mathsf{T}}X/n$ and choose a matrix $M$ from $X$ alone. The corrected Lasso estimatehas the exact decompositionIf $\eta =\Vert I-M\hat{\Sigma }{\Vert }_{\max }$, every remainder coordinate is bounded by $\sqrt{n}\eta \Vert \hat{\beta }-{\beta }^0{\Vert }_1$. Thus a sufficiently accurate approximate inverse of a Gram matrix and a small coefficient L1 norm error make the remainder negligible.^[ref]

Given $X$, Gaussian noise $\epsilon \sim N(0,{\sigma }^{2}I)$ makes the first term exactly normal with covariance matrix ${\sigma }^{2}M\hat{\Sigma }{M}^{\mathsf{T}}$. For coordinate $j$, let $v_j=\sigma \sqrt{(M\hat{\Sigma }{M}^{\mathsf{T}}{)}_{jj}}>0$. If the remainder divided by $v_j$ tends to zero in probability, then $\sqrt{n}({\tilde{\beta }}_j-{\beta }_j^0)/v_j$ converges to a standard normal distribution. Replacing $\sigma$ requires a consistent estimator; non-Gaussian errors require a corresponding central limit theorem.

In the normal linear model, assume $\epsilon \sim N_n(0,{\sigma }^{2}I)$ is independent of $X$. Independent rows of $X$ with multivariate normal distribution $N_p(0,I_p)$ give a simple random design for Debiased Lasso. Let $p\to \infty$, $\log p=o(n)$ and $s\log p=o(\sqrt{n})$, with fixed positive noise scale. Choose the Lasso penalty and all Nodewise Lasso penalties as sufficiently large constant multiples of $\sqrt{\log p/n}$, including the noise scale in the former. Uniform sample cross-correlations are bounded by this scale, so zero is the nodewise solution with probability tending to one. Column squared norms divided by $n$ converge uniformly to one. To check the Compatibility condition for the Lasso directly, put $\hat{\Sigma }=X^{T}X/n$ and $\eta ={\max }_{j,k}|{\hat{\Sigma }}_{jk}-1_{\{j=k\}}|$. On the Lasso cone condition, $\Vert \delta {\Vert }_1\le 4\Vert {\delta }_S{\Vert }_1$, and the Cauchy-Schwarz inequality givesFor $s\ge 1$, entrywise concentration of a Gaussian Gram matrix gives $\eta \le D_0\sqrt{\log p/n}$ with probability tending to one for sufficiently large fixed $D_0$. The stated sparsity condition implies $s\eta \to 0$ on this event, so one can take ${\varphi }^2\ge 1/2$ eventually. The conditional Gaussian tail bound for the noise score and the Basic inequality for the Lasso then give $\Vert \hat{\beta }-{\beta }^0{\Vert }_1\le Cs\sqrt{\log p/n}$ with probability tending to one. Consequently the debiased-Lasso remainder bound is at most $As\log p/\sqrt{n}$ with probability tending to one, for a fixed constant $A$. These assumptions allow both $p$ and $s$ to grow, for example $p=n^2$, $s=\lfloor n^{1/4}\rfloor$.