Model evaluation results of "Evaluating the Use of Intra-Class Rarity Measures to Improve Classification Performance for Rare Samples"

This dataset includes all evaluation results of the master thesis "Evaluating the Use of Intra-Class Rarity Measures to Improve Classification Performance for Rare Samples".AbstractIntra-class rare samples, data points that are rare relative to their assigned class label, are often underrepresented during model training. Consequently, machine learning models tend to prioritize dominant patterns at the expense of uncommon yet meaningful ones. This work explores whether explicitly measuring and incorporating intra-class rarity into model training can improve the classification of such samples. We propose and evaluate two intra-class rarity measures: L²class, which is based on class neighborhood composition, and CB-LoOP, which is derived from a probabilistic outlier score. Scores from these measures are integrated into a random forest classifier via two strategies: adjusted bootstrap sampling and sample weighting. Experiments on 65 datasets across 32 parameter settings demonstrated that adjusted bootstrap sampling with L²class notably improved classification of intra-class rare samples, while preserving the overall performance of the model.Dataset/ResultsThe results in this repository are derived using the model evaluation pipeline available at https://github.com/jannewer/icr-evaluation-pipeline.Each run consists of three files: A config file with the parameter settings of the run, a file containing the results of the baseline model (rf_{$run-id}_combined_results.csv) and a file containing the results of the proposed model (icr-rf_{$run-id}_combined_results.csv) with the configuration in the config file.All runs are also included in all_evaluation_results.zip for easier download.Futher informationThe proposed model is available at https://github.com/jannewer/intra-class-rare-learn and can be installed from PyPI.The thesis is available at request (contact [e-mail]).