Salesforce/WAFER-QA

WAFER-QA是一个用于评估大型语言模型在对抗性反馈下的鲁棒性的基准数据集。它包含两部分：有上下文的问题和无上下文的问题。每个样例都包括一个问题、一个正确答案、一个表示是否有在线反证证据的布尔值、一个由证据支持的备选答案、支持证据或上下文（带有源URL）、原始数据集来源以及多项选择题的选项（对于开放式问题则为空）。

WAFER-QA is a benchmark for evaluating the resilience of large language models against factually supported deceptive feedback. It consists of two parts: questions with context and questions without context. Each sample includes a question, a correct answer, a boolean indicating the presence of online counterevidence, an alternative answer supported by evidence, supporting evidence or context (with source URLs), the original dataset source, and multiple-choice options (for multiple-choice QA; empty for open-ended QA).