EssentialAI/bbh_adv
收藏官方服务：
资源简介：
BBH对抗性数据集是一个用于评估模型在复杂推理任务中反射能力的诊断数据集。它基于Big-Bench Hard (BBH) 基准，包含多种多步骤推理任务，如逻辑谜题、物体操作和几何描述。该数据集通过引入旨在模仿语言模型常见失败模式的误导性链式推理解释来构建对抗性示例，挑战模型批判性地评估推理步骤并避免被误导。
The BBH Adversarial dataset is a diagnostic dataset designed to evaluate a models capacity for reflection in complex reasoning tasks. It is based on the Big-Bench Hard (BBH) benchmark, which includes a diverse suite of multi-step reasoning tasks such as logical puzzles, object manipulation, and geometric descriptions. The dataset introduces adversarial examples with misleading Chain-of-Thought (CoT) explanations that mimic common failure modes of language models, challenging the model to critically assess reasoning steps and avoid being misled.
提供机构：
EssentialAI
