Data-Efficient Machine Learning for Small Tabular Datasets: A Comparison Study

This study compares three machine learning algorithms (Logistic Regression, Random Forest, XGBoost) across four healthcare and finance datasets with sample sizes ranging from 303 to 30,000. Evaluation metrics include accuracy, F1-score, and ROC-AUC. Key findings: Logistic Regression outperforms ensemble methods on small datasets (<500 samples), while Random Forest dominates on larger datasets. XGBoost failed to achieve best performance on any dataset. Results provide practical guidance for algorithm selection in resource-constrained environments.