Supervised Predictive Modeling of High-dimensional Data with Group l0-norm Constrained Neural Networks

To flexibly handle high-dimensional data, this paper develops a new sparsity-constrained optimization approach. By leveraging group l0-norm constrained neural networks, the proposed approach aims to simultaneously extract crucial features and estimate the underlying model function with statistically guaranteed accuracy. Under some mild conditions, we establish statistical theories of the proposed method for a general class of nonparametric regression-type loss functions. Moreover, two iterative greedy selection algorithms, which iterate between a standard gradient descent step and a hard thresholding step with or without debiasing, are presented to implement the computation. Convergence guarantees and sparsity recovery capabilities of these algorithms are rigorously examined. Empirical validation through a series of comprehensive experiments conducted on both real-world and synthetic datasets underscores the superior performance of our proposed estimator. In various scenarios, the proposed approach surpasses conventional Lasso-based sparse learning methods in terms of variable selection accuracy and prediction performance, thereby highlighting its efficacy in practical applications.