Bayesian elastic net regression

We propose the method for estimating the value of the L2 penalty parameter, λ_2, of elastic net linear regression model using Bayesian analysis. The value of λ_2 is specified through Bayes factor. We compare the performance of the value of λ_2 based on Bayes factor to the value of λ_2 chosen by 10-fold cross-validation method. Simulation studies and real data examples show that the value of λ_2 based on Bayes factor performs better in prediction accuracy. The value of λ_2 based on Bayes factor can be used for adaptive elastic net estimator where the adaptive weight is included in the L1 penalty. We study the performance of two adaptive elastic net estimation methods where the adaptive weights are constructed using elastic net and least squares estimators. Simulation studies show that two adaptive weights perform differently. When the elastic net estimator is used, the adaptive elastic net performs best in estimation accuracy and variable selection performance. If the least squares estimator is used, the adaptive elastic net has the prediction performance better than using the other adaptive weight. We study the performance of the Bayesian variable selection for elastic net linear regression model (BVS) using two different priors: the penalty parameters λ_1 and λ_2 are estimated by the 10-fold cross-validation, and the penalty parameter λ_2 is based on Bayes factor. The variable selection result of BVS differs from elastic net. The BVS performs both variable selection and group selection where the pair of predictors which are highly correlated with the response variable is included into the optimal model whereas some pair of predictors which are highly correlated with the response variable is excluded from the elastic net model. The BVS is more parsimonious than the elastic net. For BVS method, the prior for the penalty parameters λ_1 and λ_2 estimated by the 10-fold cross-validation method is the best.