sanskxr02/Beacon
收藏官方服务：
资源简介：
Beacon数据集旨在通过一种新颖的单轮强制选择评估范式，测量大型语言模型中的谄媚偏见。该数据集包含420个经过精心挑选的提示，每个提示都配有一个原则性回应和一个谄媚性替代回应。专家注释使得可以在批判性思维和流畅性方面进行细致的行为分析。
The Beacon dataset is designed to measure sycophantic bias in Large Language Models (LLMs) through a novel single-turn forced-choice evaluation paradigm. It consists of 420 carefully curated prompts, each paired with a principled response and a sycophantic alternative. Expert annotations enable fine-grained behavioral analysis on dimensions of Critical Thinking and Fluency.
提供机构：
sanskxr02
