Model building using forward selection and machine learning

A common challenge in large-scale data sets with numerous predictors is multicollinearity, which can degrade the performance and interpretability of predictive models. Although dimensionality reduction techniques like Principal Component Analysis (PCA) can mitigate multicollinearity, they transform original variables into new uncorrelated variables, often reducing the model interpretability. For this reason, feature selection techniques are generally preferred in domains where understanding of the original predictors is essential. This paper presents a novel feature selection method that combines machine learning techniques with forward feature selection to effectively address multicollinearity while preserving the accuracy of prediction. The proposed method is evaluated on three diverse data sets from the maritime, healthcare, and environmental domains. Regression models were used to assess predictive performance based on adjusted R2, Root Mean Square Error and Mean Absolute Error. The results demonstrate that the proposed method achieves high performance comparable to traditional feature selection techniques while preventing multicollinearity. This balance between predictive power and feature stability makes it especially suitable for high-dimensional data sets, where redundant or highly correlated variables are prevalent. The approach supports more robust and interpretable models, highlighting its potential for widespread application in complex, data-intensive domains.