Higher Order Accurate Symmetric Bootstrap Confidence Intervals in High Dimensional Penalized Regression

This article develops methodology for higher order accurate two-sided Bootstrap confidence intervals (CIs) in high dimensional penalized regression models using the Bootstrap. We consider a large class of penalized regression methods that satisfy the Oracle property of Fan and Li and a stronger variant of it, called the Strong Oracle property. While second order accuracy of the Bootstrap is known for both classes, it is typically not sufficient to guarantee better accuracy of two-sided Bootstrap CIs over their Oracle limit based counterparts. In this article, we show that for penalization methods with the strong Oracle property (called “Class I” methods here), a variant of the two-sided symmetric Bootstrap CI method, originally proposed by Hall in the traditional fixed dimensional case, can attain an accuracy level of O(n−2) in sample size n, even if the dimension of the regression model grows at an arbitrary polynomial rate in n. On the other end, for penalized methods with only the Oracle property (called “Class II” methods here), two-sided symmetric Bootstrap CIs can achieve the same O(n−2) level of accuracy in such high dimensions only after some nontrivial modification. Consequently, the proposed methodology can be used to construct accurate two-sided CIs for the relevant regression parameters in very high dimensional regression models with a much smaller sample size than what has been considered possible in the literature. We also report results from a simulation study and illustrate the methodology with a real data example. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.