distilabel-internal-testing/mt-bench-eval-critique

---
dataset_info:
features:
- name: idx
dtype: int64
- name: response_source
dtype: string
- name: category
dtype: string
- name: instruction
dtype: string
- name: response
dtype: string
splits:
- name: train
num_bytes: 970116
num_examples: 320
download_size: 398377
dataset_size: 970116
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

## Description

This dataset is used to check criticon prompts/responses while testing, it contains instructions/responses from mt_bench_eval, as extracted from: https://github.com/kaistAI/prometheus/blob/main/evaluation/benchmark/data/mt_bench_eval.json

The dataset has been obtained cleaning the data with:

```python
import re
import pandas as pd
from datasets import Dataset

df = pd.read_json("mt_bench_eval.json", lines=True)

ds = Dataset.from_pandas(df, preserve_index=False)

def prepare_ds(example):
example["instruction"].split("###The instruction to evaluate:")
pat1 = r'###The instruction to evaluate:(.*?)###Response to evaluate:'
pat2 = r'###Response to evaluate:(.*?)###Reference Answer'
matches_1 = re.search(pat1, example["instruction"], re.DOTALL)
matches_2 = re.search(pat2, example["instruction"], re.DOTALL)
instruction = response = None
if matches_1:
instruction = matches_1.group(1).strip()
if matches_2:
response = matches_2.group(1).strip()
return {
"instruction": instruction,
"response": response
}

ds = ds.map(prepare_ds)
```