Bootstrap Model Averaging

Model averaging has garnered significant attention in recent years for its ability to combine information from multiple models. A critical challenge in frequentist model averaging is determining the appropriate weight vector. The bootstrap method, well-known for its desirable properties, offers a promising solution. In this paper, we propose a bootstrap model averaging approach that selects weights by minimizing a bootstrap-based criterion. Notably, our weight selection criterion can also be interpreted as bootstrap aggregating. When all candidate models are misspecified, we show that the resulting estimator is asymptotically optimal, achieving the minimum possible squared error loss. Furthermore, we establish the convergence rate of the bootstrap weights toward the theoretically optimal weights. In scenarios where correct candidate models exist within a nested set and the number of covariates is fixed, we derive the limiting distribution of our proposed model averaging estimator. Through simulation studies and empirical applications, we demonstrate that our proposed method often outperforms other commonly used model selection and model averaging techniques, and other bootstrap-based variants.