notdaniel1234/sycophancy-guard

Sycophancy Guard是一个用于文本分类的数据集，专注于检测语言模型中的奉承行为。它基于TruthfulQA数据集和Llama-3-8B模型的隐藏状态，通过推理时线性探针分类来区分regressive sycophancy（模型从正确答案转向同意事实错误的用户反驳）和progressive sycophancy（模型转向同意事实正确的用户反驳）。数据集包含处理后的数据对、隐藏状态文件（如layer_25.npy）、训练和评估脚本，以及复现实验所需的资源。数据集旨在支持奉承行为对齐、可解释性、探针和LLM评估研究，适用于单轮反驳对话场景。Sycophancy Guard is a dataset for text classification, focusing on detecting sycophancy behavior in language models. It is based on TruthfulQA and hidden states from the Llama-3-8B model, using inference-time linear probes to classify regressive sycophancy (where the model flips from a correct answer to agree with a factually wrong user pushback) and progressive sycophancy (where the model flips toward agreement with a factually correct user pushback). The dataset includes processed data pairs, hidden state files (e.g., layer_25.npy), scripts for training and evaluation, and resources for reproducing experiments. It is designed for research on sycophancy alignment, interpretability, probing, and LLM evaluation in single-turn pushback dialogues.