allenai/WildBench

WildBench是一个用于评估大型语言模型（LLMs）在更具挑战性任务上的基准测试，这些任务更能代表现实世界的应用。数据集中的示例来自AI2 WildChat项目收集的真实用户数据。WildBench提供了细粒度的注释，包括任务类型和评估响应质量的检查清单，并使用长度惩罚的Elo评分来确保响应质量不偏向于较长的输出。WildBench旨在提供一个更现实和更具挑战性的基准测试，以评估LLMs在多样性和复杂性方面的表现。数据集包含1024个示例，并且将不断添加新的示例和更新评估方法。WildBench is a benchmark designed for evaluating large language models (LLMs) on more challenging tasks that better align with real-world application scenarios. The examples in the dataset are derived from real user interaction data collected by the AI2 WildChat project. WildBench provides fine-grained annotations including task types and checklists for evaluating response quality, and employs length-penalized Elo scoring to avoid biasing response quality toward longer outputs. WildBench aims to offer a more realistic and challenging benchmark to assess LLMs' performance across dimensions of diversity and complexity. The dataset currently contains 1024 examples, and new instances as well as updated evaluation methodologies will be continuously added over time.

原始信息汇总

数据集概述

名称: WildBench

来源: WildBench是WildChat数据集的一个子集。

授权: 该数据集已根据AI2的ImpACT许可公开发布，作为低风险制品。

使用限制: 严格禁止使用WildChat数据造成伤害。