rsbench

rsbench是由特伦托大学开发的综合性基准套件，旨在系统评估模型在学习和推理任务中对推理捷径（RSs）的影响。该数据集包含多种类型的任务，如算术、逻辑和高风险任务，并提供数据生成器以评估分布外场景。rsbench不仅支持神经网络模型，还包括神经符号模型，通过实施质量评估指标和新的形式验证程序，帮助研究者理解和改进模型在复杂任务中的表现。该数据集适用于解决模型在学习和推理过程中可能出现的推理捷径问题，特别是在高风险应用中，如自动驾驶决策。

rsbench is a comprehensive benchmark suite developed by the University of Trento, which aims to systematically evaluate the impact of reasoning shortcuts (RSs) on models during learning and inference tasks. This dataset includes a wide range of task types, such as arithmetic, logical, and high-stakes tasks, and offers data generators to assess out-of-distribution scenarios. rsbench supports not only neural network models but also neurosymbolic models. By implementing quality evaluation metrics and novel formal verification procedures, it helps researchers understand and improve model performance on complex tasks. This dataset is designed to address the issue of reasoning shortcuts that may emerge during model learning and inference processes, especially in high-stakes applications such as autonomous driving decision-making.