lmsys/lmsys-arena-human-preference-55k

--- license: apache-2.0 task_categories: - text-classification language: - en pretty_name: LMSYS Chatbot Arena Human Preference Predictions size_categories: - 10K<n<100K --- Dataset for [Kaggle competition](https://www.kaggle.com/competitions/lmsys-chatbot-arena/overview) on predicting human preference on Chatbot Arena battles. The training dataset includes over 55,000 real-world user and LLM conversations and user preferences across over 70 state-of-the-art LLMs, such as GPT-4, Claude 2, Llama 2, Gemini, and Mistral models. Each sample represents a battle consisting of 2 LLMs which answer the same question, with a user label of either prefer model A, prefer model B, tie, or tie (both bad). ### Citation Please cite the following paper if you find our leaderboard or dataset helpful. ``` @misc{chiang2024chatbot, title={Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference}, author={Wei-Lin Chiang and Lianmin Zheng and Ying Sheng and Anastasios Nikolas Angelopoulos and Tianle Li and Dacheng Li and Hao Zhang and Banghua Zhu and Michael Jordan and Joseph E. Gonzalez and Ion Stoica}, year={2024}, eprint={2403.04132}, archivePrefix={arXiv}, primaryClass={cs.AI} } ```

原始信息汇总

数据集概述

基本信息

• 许可证: Apache-2.0
• 任务类别: 文本分类
• 语言: 英语
• 美观名称: LMSYS Chatbot Arena Human Preference Predictions
• 大小类别: 10K<n<100K

数据集描述

• 目的: 用于预测Chatbot Arena战斗中人类偏好的Kaggle竞赛数据集。
• 内容: 包含超过55,000条真实世界的用户与LLM（如GPT-4, Claude 2, Llama 2, Gemini, Mistral等）的对话及用户偏好。
• 样本结构: 每个样本代表一场战斗，包含两个LLM对同一问题的回答，以及用户的偏好标签（偏好模型A、偏好模型B、平局或两者均不佳）。

引用信息

若使用此数据集或其排行榜，请引用以下文献：

@misc{chiang2024chatbot, title={Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference}, author={Wei-Lin Chiang and Lianmin Zheng and Ying Sheng and Anastasios Nikolas Angelopoulos and Tianle Li and Dacheng Li and Hao Zhang and Banghua Zhu and Michael Jordan and Joseph E. Gonzalez and Ion Stoica}, year={2024}, eprint={2403.04132}, archivePrefix={arXiv}, primaryClass={cs.AI} }