livebench_language

# Dataset Card for "livebench/language" LiveBench is a benchmark for LLMs designed with test set contamination and objective evaluation in mind. It has the following properties: - LiveBench is designed to limit potential contamination by releasing new questions monthly, as well as having questions based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses. - Each question has verifiable, objective ground-truth answers, allowing hard questions to be scored accurately and automatically, without the use of an LLM judge. - LiveBench currently contains a set of 18 diverse tasks across 6 categories, and we will release new, harder tasks over time. This is the instruction_following category of livebench. See more in our [paper](https://arxiv.org/abs/2406.19314), [leaderboard](https://livebench.ai/), and [datasheet](https://github.com/LiveBench/LiveBench/blob/main/docs/DATASHEET.md).# 「livebench/language」数据集卡片 LiveBench 是一款专为大语言模型（Large Language Model，LLM）设计的基准测试集，其研发初衷为规避测试集污染问题并实现客观评估。该基准具备以下特性： - 该基准通过每月发布全新题目，以及基于最新公开数据集、arXiv论文、新闻文章与IMDb电影剧情概要构建题目，以此限制潜在的测试集污染。 - 每道题目均配有可验证的客观标准答案，无需借助大语言模型作为评判器，即可对高难度题目实现精准且自动化的评分。 - 当前LiveBench 涵盖6个类别下共18项多样化任务，后续还将逐步发布难度更高的全新任务。 本数据集属于LiveBench的指令跟随（instruction_following）类别。 更多详情可参阅我们的[论文](https://arxiv.org/abs/2406.19314)、[排行榜](https://livebench.ai/)以及[数据集说明文档](https://github.com/LiveBench/LiveBench/blob/main/docs/DATASHEET.md)。

搜集汇总

背景与挑战

背景概述

LiveBench是一个专为大型语言模型设计的基准测试，旨在通过每月发布新问题、基于近期数据源和提供客观真实答案来减少测试集污染并实现自动评估。该数据集包含18个多样化任务，覆盖6个类别，当前部分专注于指令跟随任务。