Model Selection With Lasso-Zero: Adding Straw to the Haystack to Better Find Needles

The high-dimensional linear model y=Xβ0+ϵ is considered and the focus is put on the problem of recovering the support <i>S</i>⁰ of the sparse vector β0. We introduce a new l1-based estimator, called <i>Lasso-Zero</i>, whose novelty resides in the repeated use of noise dictionaries concatenated to <i>X</i> for overfitting the response. Lasso-Zero is an extension of thresholded basis pursuit (TBP), for which we prove sign consistency for correlated Gaussian designs. Both theoretical and empirical results motivate the use of noise dictionaries to improve TBP when the coefficients’ amplitude is low. To select the threshold of Lasso-Zero, we suggest to employ a pivotal version of the quantile universal threshold (QUT) that exploits a byproduct of Lasso-Zero to avoid the need to estimate the noise level. Numerical simulations show that Lasso-Zero tuned by QUT performs well in terms of support recovery and provides an excellent trade-off between high power and few false discoveries compared to competitors. Supplemental materials for this article are available online.