An Iterative Sparse-Group Lasso

In high-dimensional supervised learning problems, sparsity constraints in the solution often lead to better performance and interpretability of the results. For problems in which covariates are grouped and sparse structure are desired, both on group and within group levels, the sparse-group lasso (SGL) regularization method has proved to be very efficient. Under its simplest formulation, the solution provided by this method depends on two weight parameters that control the penalization on the coefficients. Selecting these weight parameters represents a major challenge. In most of the applications of the SGL, this problem is left aside, and the parameters are either fixed based on a prior information about the data, or chosen to minimize some error function in a grid of possible values. However, an appropriate choice of the parameters deserves more attention, considering that it plays a key role in the structure and interpretation of the solution. In this sense, we present a gradient-free coordinate descent algorithm that automatically selects the regularization parameters of the SGL. We focus on a more general formulation of this problem, which also includes individual penalizations for each group. The advantages of our approach are illustrated using both real and synthetic datasets. Supplementary materials for this article are available online.

在高维监督学习任务中，求解过程引入稀疏性约束通常能提升模型性能并增强结果的可解释性。针对协变量存在分组且需要同时实现组层面与组内层面稀疏结构的任务，稀疏组套索（sparse-group lasso, SGL）正则化方法已被证实极具实用性。在其最基础的形式下，该方法得到的求解结果依赖于两个用于控制系数惩罚项的权重参数，而这两个参数的选取是一项核心挑战。在绝大多数稀疏组套索的实际应用中，这一参数选取问题常被忽略，研究者要么基于数据先验信息固定参数取值，要么通过在候选值网格中最小化某一误差函数来选择参数。然而，参数的合理选取对求解结果的结构与可解释性起到关键作用，因此值得投入更多关注。基于此，本文提出一种无需梯度的坐标下降算法，可自动选取稀疏组套索的正则化参数。本文针对该问题的更通用形式展开研究，该形式同时支持对每个分组施加单独的惩罚项。本文通过真实数据集与合成数据集验证了所提方法的优势。本文的补充材料可在线获取。