microsoft/WildFeedback

WildFeedback是一个基于真实世界用户与ChatGPT互动的偏好数据集。它不同于依赖AI生成排名的合成数据集，而是通过对话中自然发生的用户反馈信号捕获真实的人类偏好。该数据集旨在利用直接的用户输入来提高大型语言模型（LLM）与实际人类价值的一致性。数据集由Microsoft Research策划，包含20,281个偏好对，源自148,715次多轮对话。每个偏好对包括原始用户提示、基于反馈提取的用户偏好、与用户期望一致的偏好回应以及引发不满的非偏好回应。此外，还包括SAT/DSAT注释，提供诸如领域、用户意图、对话状态跟踪等信息。

WildFeedback is a preference dataset constructed from real-world user interactions with ChatGPT. Unlike synthetic datasets that rely solely on AI-generated rankings, WildFeedback captures authentic human preferences through naturally occurring user feedback signals in conversation. The dataset is designed to improve the alignment of large language models (LLMs) with actual human values by leveraging direct user input. The dataset curated by Microsoft Research includes 20,281 preference pairs from 148,715 multi-turn conversations, each containing the original user prompt, extracted user preferences based on feedback, a preferred response aligning with user expectations, and a dispreferred response that triggered dissatisfaction. Additionally, it includes SAT/DSAT annotations with information such as domain, user intent, and dialogue state tracking.