A Hybrid Regression Model for improving prediction accuracy

The main disadvantage of Regression tree is that it assigns the same predicted value, average value, for all the tuples which satisfies the same corresponding splitting criterion. K-Nearest Neighbors (KNN) is sensitive to irrelevant or redundant features because all features contribute to the similarity. In this paper a hybrid regression model based on Regression tree (RT) and KNN is proposed which overcomes the above two problems. The performance of proposed model is compared with KNN for 10 types of distance measures. The performance of proposed model is also compared with RT, K-Nearest neighbour regression (KNN), Support Vector Regression (SVR) through a Monte Carlo simulation study. The simulation result indicates that hybrid model outperforms all other regression model irrespective of sample size when the observations are from normal distributions as well as t-distributions. The working of the proposed model is illustrated for a real-life application on global warming data of Delhi.