lthn/livebench-model_judgment

---
dataset_info:
features:
- name: question_id
dtype: string
- name: task
dtype: string
- name: model
dtype: string
- name: score
dtype: float64
- name: turn
dtype: int64
- name: tstamp
dtype: float64
- name: category
dtype: string
splits:
- name: leaderboard
num_bytes: 8856866
num_examples: 60372
download_size: 737444
dataset_size: 8856866
configs:
- config_name: default
data_files:
- split: leaderboard
path: data/leaderboard-*
arxiv: 2406.19314
---
# Dataset Card for "livebench/model_judgment"
LiveBench is a benchmark for LLMs designed with test set contamination and objective evaluation in mind. It has the following properties:
- LiveBench is designed to limit potential contamination by releasing new questions monthly, as well as having questions based on recently-released datasets, arXiv papers, news articles, and IMDb movie synopses.
- Each question has verifiable, objective ground-truth answers, allowing hard questions to be scored accurately and automatically, without the use of an LLM judge.
- LiveBench currently contains a set of 18 diverse tasks across 6 categories, and we will release new, harder tasks over time.

This dataset contains all model judgments (scores) currently used to create the [leaderboard](https://livebench.ai/). Our github readme contains instructions for downloading the model judgments (specifically see the section for download_leaderboard.py).

For more information, see our [paper](https://arxiv.org/abs/2406.19314).