Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 205 / 4

The Square-root Lasso estimator with regularization parameter $\gamma >0$ isUnlike the ordinary Lasso, its tuning parameter does not require prior knowledge of the noise standard deviation $\sigma$.

One standard construction uses the Debiased Lasso. Starting from the Square-root Lasso estimate $\hat{\beta }$, estimate a vector ${\hat{\Theta }}_j$ that approximately inverts the $j$th column of the empirical Gram matrix $\hat{\Sigma }=X^{T}X/n$, for example by a Nodewise Lasso. Defineand estimate $\sigma$ by $\hat{\sigma }=\Vert Y-X\hat{\beta }{\Vert }_2/\sqrt{n}$. The approximate two-sided level-$\alpha$ test rejects $H_j^{\beta }$ whenwhere $z_{1-\alpha /2}$ is a standard normal quantile.

Sufficient high-dimensional conditions include a Compatibility condition for the Lasso bounded away from zero, $\gamma \asymp \sqrt{\log p/n}$, $\log p=o(n)$, andtogether with the corresponding sparsity and consistency conditions for the nodewise inverse-Gram estimate. Under these assumptions the Debiased-Lasso asymptotic normality makes the rejection probability under $H_j^{\beta }$ tend to $\alpha$.

If $B=\varnothing$, then for every $j$ at least one of $H_j^{\beta }$ and $H_j^{\eta }$ is true. Sincevalidity of the p-value for whichever component null is true implies $\mathbb{P}(q_j\le t)\le t$. Therefore the union bound givesThis is the Bonferroni correction for the composite intersection alternatives.

Let $I_0=B^c$ and $m_0=|I_0|$. Each $q_j$ with $j\in I_0$ is a valid p-value by the argument in part c. If the Holm step-down procedure selects any index from $I_0$, let $r$ be the rank of the first such index. All $r-1$ earlier selections belong to $B$, soSelection through rank $r$ impliesConsequentlyThus the procedure controls the familywise error rate without requiring independence among the p-values.