Navas-Olive, Rubio, et al. (2023). Figure 4 - data

Figure 4. Comparison between best performing ML models. a) F1 against threshold from the 10-best models of each architecture as evaluated in the test set, and the 10-best filters of all possible parametric combinations. Each line represents the performance of one trained model, colored by its maximal F1 (mean from all sessions is plotted in dark color). b) F1-scores for the best model of panel A, and ensemble model of panel e). Thresholds used are: 0.4 for XGBoost, 0.5 for SVM, 0.4 for LSTM, 0.1 for CNN2D, 0.5 for CNN1D, 4.5SD for Filter. c) Stability index for the same models as in panel b) (bottom), and the stability index vs the F1 (top). d) Similarity between predicted events of different architectures. Models are the same as in panels b)-c). To measure the similarity, the mean F1 across test sessions have been computed, using detected events in the y-axis as detections, and detected events in the x-axis as ground truth. Note the similarity between LSTM and 1D-CNN (white *), and that by XGBoost against SVM, LSTM and 1D-CNN (white +). e) Ensemble model, trained using the output of the best models of the machine learning architectures. Weights were: w1=-0.11 (XGBOOST); w2=-1.56 (SVM); w3=5.33 (LSTM); w4= 2.03 (2D-CNN); w5= 4.07 (1D-CNN); bias= -4.97. On the right, mean F1 score (line) ±95% confidence interval (shadow) for test sessions. Maximum F1-score and stability index for test sessions has been included in panels b) and c) to facilitate comparison with the rest of the methods.

图4 表现最优机器学习（Machine Learning, ML）模型对比。a) 以测试集评估得到的各模型架构Top10模型，以及所有参数组合下的Top10滤波器的F1分数（F1-score）随阈值的变化情况。每条曲线代表一个训练好的模型的性能，按其最大F1分数着色，所有测试轮次的均值以深色绘制。b) 子图a)中的最优模型与子图e)中的集成模型的F1分数。所用阈值设置为：XGBoost（极限梯度提升树）0.4、SVM（支持向量机，Support Vector Machine）0.5、LSTM（长短期记忆网络，Long Short-Term Memory）0.4、CNN2D（二维卷积神经网络，Two-Dimensional Convolutional Neural Network）0.1、CNN1D（一维卷积神经网络，One-Dimensional Convolutional Neural Network）0.5、滤波器（Filter）4.5倍标准差（4.5SD）。c) 与子图b)相同的模型的稳定性指数（Stability index），下方子图为该指数，上方子图为稳定性指数与F1分数的关系。d) 不同模型架构的预测事件间的相似性。所用模型与子图b)-c)一致。为衡量相似性，以纵轴的检测事件作为预测结果、横轴的检测事件作为真实标签，计算各测试轮次的平均F1分数。需注意：LSTM与一维卷积神经网络的相似性以白色星号标注，XGBoost与SVM、LSTM及一维卷积神经网络的相似性以白色加号标注。e) 集成模型：基于各机器学习模型架构的最优模型的输出进行训练所得。其权重参数为：w₁=-0.11（XGBOOST）、w₂=-1.56（SVM）、w₃=5.33（LSTM）、w₄=2.03（2D-CNN）、w₅=4.07（1D-CNN），偏置项bias=-4.97。右侧为测试轮次的平均F1分数（曲线）±95%置信区间（阴影区域）。为便于与其余方法对比，子图b)和c)已纳入测试轮次的最大F1分数与稳定性指数。