模型可信性、鲁棒性、公平性、可解释性及不确定性评测数据集

本数据集通过对多源数据的采集与综合处理，构建了模型可信性、鲁棒性、公平性、可解释性及不确定性的评测数据集。其中，可信性与不确定性数据集基于广泛用于手写数字识别任务的MNIST数据集，为模型可信性评测提供了稳定可靠的基准。公平性数据集则通过对Adult数据集进行模型训练，并采用PGD白盒攻击增加数据偏见，结合本项目自研的五种偏见检测方法，覆盖性别、种族、年龄等多维度偏见，为偏见检测算法的优化与研究提供支持。可解释性数据集选取了ImageNet的val集，凭借其丰富的图像类别和高质量样本，用于评估模型在图像识别任务中的可解释性表现。鲁棒性数据集则是基于国际标准数据集的对抗训练，生成鲁棒验证模型并进行测试得到鲁棒验证数据。整个数据集采集与处理过程严格遵循质量控制流程，确保了数据的高质量与可靠性，为模型多维度评测提供了有力支撑。

This dataset is developed through the collection and comprehensive processing of multi-source data, serving as an evaluation benchmark covering five key dimensions of model performance: trustworthiness, robustness, fairness, interpretability, and uncertainty. Specifically, the trustworthiness and uncertainty subset is built upon the widely used MNIST dataset for handwritten digit recognition tasks, providing a stable and reliable benchmark for model trustworthiness evaluation. The fairness subset is constructed by training models on the Adult dataset, introducing data biases via PGD white-box attacks, and combining five self-developed bias detection methods of this project. It covers multi-dimensional biases including gender, race, age and other attributes, providing support for the optimization and research of bias detection algorithms. The interpretability subset selects the ImageNet validation set, which boasts abundant image categories and high-quality samples, and is used to evaluate the interpretability performance of models in image recognition tasks. The robustness subset is based on adversarial training on international standard datasets, where robust verification models are generated and tested to obtain robust verification data. The entire collection and processing workflow of this dataset strictly follows standardized quality control procedures, ensuring the high quality and reliability of the data, and providing strong support for multi-dimensional model evaluation.

搜集汇总

背景与挑战

背景概述

该数据集通过对MNIST、Adult和ImageNet等标准数据源进行综合处理，构建了覆盖模型可信性、鲁棒性、公平性、可解释性及不确定性的评测数据集。它包含偏见检测、鲁棒验证和可解释性评估等维度，严格遵循质量控制以确保数据可靠性。整体旨在为人工智能模型的多维度性能评测提供基准支持。