Data-Driven Decision Model for Machine Learning Model Selection

Context: Machine learning methods are easily accessible and widely used, given how straightforward they are to employ on a predictive modeling dataset. While the performance of these libraries is stable, the challenge exists for a data scientist to choose the correct combinations of models for their machine learning domain related task. Besides sufficient performance, we consider many machine learning model related features.
Method: To address this challenge, we present a meta-model for the decision-making process in the context of machine learning model selection. the creation of this decision model adopts a systematic research approach, combining systematic literature review , expert interviews, case studies, and design science to investigate machine learning model selection approaches. Where the systematic literature review enables us to gather and analyze relevant information from existing literature. The expert interviews allow a critical approach to our collected data. the case studies help us to assess the practical applicability of our findings. And the design science allows for the finalization of a decision model.
Results: Our study analyzed 43 common models across 72 common features. We provide a comprehensive machine learning taxonomy, featuring machine learning paradigms, approaches, and domains. We provide insights into potential model combinations, trends in model selection, quality concerns, evaluation measures, and frequently used datasets for training and evaluating these models.
Contribution: Our study shows the effectiveness and robustness of our decision model, contributing practical insight and comprehensive understanding to the field of machine learning model selection. We highlight the importance of further development of the decision model, to improve its accuracy and scope beyond its current state.