Figure 4 - data

Figure 4. Comparison between best performing ML models. a) F1 against threshold from the 10-best models of each architecture as evaluated in the test set, and the 10-best filters of all possible parametric combinations. Each line represents the performance of one trained model, colored by its maximal F1 (mean from all sessions is plotted in dark color). b) F1-scores for the best model of panel A, and ensemble model of panel e). Thresholds used are: 0.4 for XGBoost, 0.5 for SVM, 0.4 for LSTM, 0.1 for CNN2D, 0.5 for CNN1D, 4.5SD for Filter. c) Stability index for the same models as in panel b) (bottom), and the stability index vs the F1 (top). d) Similarity between predicted events of different architectures. Models are the same as in panels b)-c). To measure the similarity, the mean F1 across test sessions have been computed, using detected events in the y-axis as detections, and detected events in the x-axis as ground truth. Note the similarity between LSTM and 1D-CNN (white *), and that by XGBoost against SVM, LSTM and 1D-CNN (white +). e) Ensemble model, trained using the output of the best models of the machine learning architectures. Weights were: w1=-0.11 (XGBOOST); w2=-1.56 (SVM); w3=5.33 (LSTM); w4= 2.03 (2D-CNN); w5= 4.07 (1D-CNN); bias= -4.97. On the right, mean F1 score (line) ±95% confidence interval (shadow) for test sessions. Maximum F1-score and stability index for test sessions has been included in panels b) and c) to facilitate comparison with the rest of the methods.