Performance measures of machine-learning based classifiers applied to date set 4 (Fig 4) with either the full set of variables 1,…,20 or a reduced set [1,…,10,16,…20] from which variables in which the groups differed with respect to their central means [11,…,15] were omitted.

Two different machine-learning methods (classification and regression trees (CART) and random forests (RF)) were applied to the artificially generated data set 4, which comprises two groups with sizes of n = 1000 cases and d = 20 variables. The results represent the medians of test performance measures from 100-fold cross-validation runs using random splits of the data set into training data (2/3 of the data set) and test data (1/3 of the data set). In addition, a negative control data set was created by permutating the variables from the training data set, with the expectation that the machine learning algorithms should not perform group assignment better than chance when trained with such data; otherwise, there could be overfitting involved.