Tuning Parameter Selection for the Adaptive Lasso Using ERIC

The adaptive Lasso is a commonly applied penalty for variable selection in regression modeling. Like all penalties though, its performance depends critically on the choice of the tuning parameter. One method for choosing the tuning parameter is via information criteria, such as those based on AIC and BIC. However, these criteria were developed for use with unpenalized maximum likelihood estimators, and it is not clear that they take into account the effects of penalization. In this article, we propose the extended regularized information criterion (ERIC) for choosing the tuning parameter in adaptive Lasso regression. ERIC extends the BIC to account for the effect of applying the adaptive Lasso on the bias-variance tradeoff. This leads to a criterion whose penalty for model complexity is itself a function of the tuning parameter. We show the tuning parameter chosen by ERIC is selection consistent when the number of variables grows with sample size, and that this consistency holds in a wider range of contexts compared to using BIC to choose the tuning parameter. Simulation show that ERIC can significantly outperform BIC and other information criteria proposed (for choosing the tuning parameter) in selecting the true model. For ultra high-dimensional data (<i>p</i> &gt; <i>n</i>), we consider a two-stage approach combining sure independence screening with adaptive Lasso regression using ERIC, which is selection consistent and performs strongly in simulation. Supplementary materials for this article are available online.

自适应Lasso（Adaptive Lasso）是回归建模中用于变量选择的一类常用惩罚项。与所有惩罚项一样，其性能高度依赖于调优参数的选取。选择调优参数的一类方法是基于信息准则，例如基于赤池信息准则（AIC）和贝叶斯信息准则（BIC）的准则。然而，这些准则最初是为非惩罚极大似然估计量设计的，尚不清楚它们是否考虑了惩罚带来的影响。本文提出了扩展正则化信息准则（ERIC），用于自适应Lasso回归中的调优参数选择。ERIC对BIC进行了拓展，以刻画自适应Lasso在偏差-方差权衡中产生的影响，由此得到的准则中，模型复杂度惩罚项本身即为调优参数的函数。我们证明了当变量个数随样本量增长时，ERIC选取的调优参数具有选择一致性，且相较于使用BIC选取调优参数的情形，该一致性在更广的场景下成立。仿真实验表明，在选择真实模型方面，ERIC的性能显著优于BIC及其他已提出的用于调优参数选择的信息准则。针对超高维数据（<i>p</i> &gt; <i>n</i>），本文考虑了一种结合确定性独立筛选（Sure Independence Screening）与采用ERIC的自适应Lasso回归的两阶段方法，该方法具有选择一致性且在仿真实验中表现优异。本文的补充材料可在线获取。