Estimation of Linear Functionals in High Dimensional Linear Models: From Sparsity to Non-sparsity

High dimensional linear models are commonly used in practice. In many applications, one is interested in linear transformations β⊤x of regression coefficients β∈Rp, where <i>x</i> is a specific point and is not required to be identically distributed as the training data. One common approach is the plug-in technique which first estimates β, then plugs the estimator in the linear transformation for prediction. Despite its popularity, estimation of β can be difficult for high dimensional problems. Commonly used assumptions in the literature include that the signal of coefficients β is sparse and predictors are weakly correlated. These assumptions, however, may not be easily verified, and can be violated in practice. When β is non-sparse or predictors are strongly correlated, estimation of β can be very difficult. In this paper, we propose a novel pointwise estimator for linear transformations of β. This new estimator greatly relaxes the common assumptions for high dimensional problems, and is adaptive to the degree of sparsity of β and strength of correlations among the predictors. In particular, β can be sparse or non-sparse and predictors can be strongly or weakly correlated. The proposed method is simple for implementation. Numerical and theoretical results demonstrate the competitive advantages of the proposed method for a wide range of problems.