Kernel Averaging Estimators

The issue of bandwidth selection is a fundamental model selection problem stemming from the uncertainty about the smoothness of the regression. In this article, we advocate a model averaging approach to circumvent the problem caused by this uncertainty. Our new approach involves averaging across a series of Nadaraya-Watson kernel estimators each under a different bandwidth, with weights for these different estimators chosen such that a least-squares cross-validation criterion is minimized. We prove that the resultant combined-kernel estimator achieves the smallest possible asymptotic aggregate squared error. The superiority of the new estimator over estimators based on widely accepted conventional bandwidth choices in finite samples is demonstrated in a simulation study and a real data example.