Speech Robust Bench (SRB)

Speech Robust Bench (SRB) 是一个全面的基准，用于评估自动语音识别（ASR）模型对多样化干扰的鲁棒性。SRB包含69种输入扰动，模拟ASR模型在物理和数字世界中可能遇到的多种干扰。该数据集由卡内基梅隆大学语言技术研究所开发，旨在通过提供标准化和可比较的鲁棒性评估，推动ASR模型的研究进展。SRB不仅用于评估模型的预测准确性，还用于分析模型在不同人口子群体（如英语和西班牙语使用者，以及男性和女性）中的鲁棒性差异，从而揭示模型在公平性方面的潜在问题。

Speech Robust Bench (SRB) is a comprehensive benchmark for evaluating the robustness of automatic speech recognition (ASR) models against diverse perturbations. SRB encompasses 69 types of input perturbations that simulate a wide range of disturbances that ASR models may encounter in both physical and digital worlds. Developed by the Language Technologies Institute at Carnegie Mellon University, this dataset aims to advance ASR research by providing standardized and comparable robustness assessments. SRB is not only utilized to evaluate the predictive accuracy of models, but also to analyze disparities in model robustness across diverse demographic subgroups such as English and Spanish speakers, as well as male and female users, thus uncovering potential fairness-related issues in these models.