distilabel-internal-testing/critique-bench-prometheus-temperature0.1-v0.1

---
size_categories: n<1K
config_names:
- prometheus-critique
tags:
- synthetic
- distilabel
- rlaif
dataset_info:
features:
- name: idx
dtype: int64
- name: hashid
dtype: string
- name: response_source
dtype: string
- name: category
dtype: string
- name: criteria
dtype: string
- name: base_dataset
dtype: string
- name: instruction
dtype: string
- name: response
dtype: string
- name: model_name
dtype: string
- name: score
dtype: string
- name: critique
dtype: string
- name: raw_output
dtype: string
splits:
- name: train
num_bytes: 5987644
num_examples: 2309
download_size: 2507010
dataset_size: 5987644
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

<p align="left">
<a href="https://github.com/argilla-io/distilabel">
<img src="https://raw.githubusercontent.com/argilla-io/distilabel/main/docs/assets/distilabel-badge-light.png" alt="Built with Distilabel" width="200" height="32"/>
</a>
</p>

# Dataset Card for critique-bench-prometheus-temperature0.1-v0.1

This dataset has been created with [Distilabel](https://distilabel.argilla.io/).

## Dataset Summary

This dataset contains a `pipeline.yaml` which can be used to reproduce the pipeline that generated it in distilabel using the `distilabel` CLI:

```console
distilabel pipeline run --config "https://huggingface.co/datasets/distilabel-internal-testing/critique-bench-prometheus-temperature0.1-v0.1/raw/main/pipeline.yaml"
```

or explore the configuration:

```console
distilabel pipeline info --config "https://huggingface.co/datasets/distilabel-internal-testing/critique-bench-prometheus-temperature0.1-v0.1/raw/main/pipeline.yaml"
```

## Dataset structure

The examples have the following structure per configuration:

<details><summary> Configuration: prometheus-critique </summary><hr>

```json
{
"base_dataset": "mt_bench_eval",
"category": "writing",
"criteria": null,
"critique": "The response provided is an excellent example of a score 5 response, as it is accurate, confident, and aligned with the instructions given. The assistant successfully incorporated the requested elements, such as starting every sentence with the letter \"A\" and highlighting the cultural experiences and must-see attractions of Hawaii. The response is also free of hallucinations and inaccuracies, as it provides a well-rounded and engaging travel blog post that captures the essence of the destination. The assistant\u0027s use of language is respectful and positive, maintaining a socially unbiased tone throughout the response. The response is also concise, with 22 sentences, which is within the desired length of 20-25 sentences. So the overall score is 5.",
"idx": 81,
"instruction": "Here is a previous dialogue between an assistant and an user. Generate a human preferable response by the assistant.\nUser: Compose an engaging travel blog post about a recent trip to Hawaii, highlighting cultural experiences and must-see attractions.\n\nAssistant:Title: Aloha Adventures: Exploring the Vibrant Spirit of Hawaii\n\nIntroduction:\nAloha! Step into a world of tropical paradise, where the rhythm of the islands sweeps you off your feet. Recently, I embarked on a journey to the enchanting archipelago of Hawaii, a destination renowned for its stunning natural beauty, rich culture, and warm hospitality. From immersing myself in the vibrant local traditions to exploring must-see attractions, join me as I share my unforgettable experiences in this captivating travelogue.\n\n1. Embrace the Aloha Spirit:\nHawaii\u0027s true charm lies in its people and their genuine warmth. As soon as I set foot on the islands, I was greeted with a heartfelt \"Aloha\" from smiling locals. To truly embrace the Aloha Spirit, I recommend participating in a traditional lei-making workshop. Learn the art of creating beautiful flower garlands while connecting with the local culture. The lei serves as a symbol of welcome and is a delightful memento to carry throughout your trip.\n\n2. Discover the Pearl Harbor Historic Sites:\nImmerse yourself in the poignant history of Hawaii by visiting the Pearl Harbor Historic Sites. Explore the USS Arizona Memorial, a moving tribute to the lives lost during the infamous attack on Pearl Harbor. Allow yourself to be transported back in time as you walk through the exhibition halls and hear the stories of bravery and resilience. This experience is a powerful reminder of the importance of peace and understanding.\n\n3. Delve into Hawaiian Cuisine:\nHawaii\u0027s culinary scene is a melting pot of flavors, combining indigenous traditions with influences from diverse immigrant communities. Indulge your taste buds in the local delicacies like poke, a refreshing dish of marinated raw fish, or kalua pig, tender and smoky slow-roasted pork. For a truly authentic experience, attend a traditional Hawaiian luau, complete with a sumptuous feast, mesmerizing hula performances, and the soulful sounds of live music.\n\n4. Explore the Volcanoes National Park:\nNo trip to Hawaii is complete without a visit to the Volcanoes National Park, a geological wonderland where the primal forces of creation and destruction are on full display. Witness the mesmerizing lava flows, hike through unique landscapes, and marvel at the active Kilauea volcano. Remember to take a guided tour for insights into the scientific and cultural significance of these natural wonders.\n\n5. Chase Waterfalls:\nHawaii\u0027s lush rainforests are home to some of the most stunning waterfalls in the world. Venture into the\n\nUser: Rewrite your previous response. Start every sentence with the letter A.\n\nAssistant:",
"model_name": "kaist-ai/prometheus-7b-v1.0",
"raw_output": null,
"response": "Aloha! Allow me to take you on an incredible journey to Hawaii, where adventure and cultural immersion await. As I explored this enchanting archipelago, I discovered awe-inspiring attractions and absorbed the vibrant spirit of the islands.\n\nArriving in Hawaii, I was immediately greeted with the warm embrace of the Aloha Spirit. Everywhere I went, locals welcomed me with genuine smiles and heartfelt \"Aloha\"s, making me feel instantly at home.\n\nAmidst my travels, I had the opportunity to participate in a traditional lei-making workshop. Creating these beautiful flower garlands not only connected me to the local culture but also served as a cherished memento of my time in Hawaii.\n\nAt the Pearl Harbor Historic Sites, I delved into the island\u0027s profound history. Visiting the USS Arizona Memorial was an immensely moving experience, as I paid tribute to the brave souls who lost their lives during the infamous attack on Pearl Harbor. It reminded me of the importance of preserving peace and understanding.\n\nA journey to Hawaii wouldn\u0027t be complete without savoring its unique culinary delights. From indulging in the fresh and flavorful poke to relishing the succulent kalua pig, I immersed myself in the island\u0027s diverse cuisine. Attending a traditional Hawaiian luau was the icing on the cake, as I savored a sumptuous feast while being mesmerized by captivating hula performances.\n\nAs an adventurer, I couldn\u0027t resist exploring the wonders of Volcanoes National Park. The breathtaking landscapes and active Kilauea volcano left me in awe of nature\u0027s power. Guided tours provided valuable insights into the scientific and cultural significance of these volcanic marvels.\n\nAmidst Hawaii\u0027s lush rainforests, I discovered an abundance of cascading waterfalls. The sight of these majestic falls filled me with a sense of tranquility and wonder. Exploring these natural wonders allowed me to truly appreciate the island\u0027s untouched beauty.\n\nIn conclusion, my trip to Hawaii was an amazing adventure filled with cultural immersion and must-see attractions. The Aloha Spirit, Pearl Harbor Historic Sites, delectable cuisine, Volcanoes National Park, and enchanting waterfalls all added to the charm of this tropical paradise. Hawaii truly is a destination that captures the essence of adventure, and I can\u0027t wait to return for more unforgettable experiences. Aloha!",
"response_source": "chatgpt",
"score": "5"
}
```

This subset can be loaded as:

```python
from datasets import load_dataset

ds = load_dataset("distilabel-internal-testing/critique-bench-prometheus-temperature0.1-v0.1", "prometheus-critique")
```

</details>

---
规模类别：样本数少于1000（n<1K）
配置名称：
- prometheus-critique
标签：
- 合成数据集（synthetic）
- Distilabel（distilabel）
- RLAIF（AI反馈强化学习，Reinforcement Learning with AI Feedback）
数据集信息：
特征：
- 名称：idx，数据类型：int64（64位整数）
- 名称：hashid，数据类型：字符串（string）
- 名称：response_source，数据类型：字符串
- 名称：category，数据类型：字符串
- 名称：criteria，数据类型：字符串
- 名称：base_dataset，数据类型：字符串
- 名称：instruction，数据类型：字符串
- 名称：response，数据类型：字符串
- 名称：model_name，数据类型：字符串
- 名称：score，数据类型：字符串
- 名称：critique，数据类型：字符串
- 名称：raw_output，数据类型：字符串
划分集：
- 名称：训练集（train），字节数：5987644，样本数量：2309
下载大小：2507010，数据集总大小：5987644
配置项：
- 配置名称：默认（default），数据文件：
- 划分集：训练集（train），路径：data/train-*
---

<p align="left">
<a href="https://github.com/argilla-io/distilabel">
<img src="https://raw.githubusercontent.com/argilla-io/distilabel/main/docs/assets/distilabel-badge-light.png" alt="基于Distilabel构建" width="200" height="32"/>
</a>
</p>

# critique-bench-prometheus-temperature0.1-v0.1 数据集卡片

本数据集由Distilabel（distilabel）构建，官方链接为https://distilabel.argilla.io/。

## 数据集概览
本数据集包含`pipeline.yaml`配置文件，可通过Distilabel命令行工具（distilabel CLI）复现生成本数据集的流水线：
console
distilabel pipeline run --config "https://huggingface.co/datasets/distilabel-internal-testing/critique-bench-prometheus-temperature0.1-v0.1/raw/main/pipeline.yaml"

或查看该配置详情：
console
distilabel pipeline info --config "https://huggingface.co/datasets/distilabel-internal-testing/critique-bench-prometheus-temperature0.1-v0.1/raw/main/pipeline.yaml"

## 数据集结构
每种配置对应的样本结构如下：

<details><summary> 配置：prometheus-critique </summary><hr>
json
{
"base_dataset": "mt_bench_eval",
"category": "writing",
"criteria": null,
"critique": "本次生成的回复为典型的5分优质回复，内容准确自信，完全符合给定指令要求。助手成功完成所有指定要求，例如每句以字母"A"开头，并涵盖夏威夷的文化体验与必打卡景点。回复无幻觉与事实错误，提供了内容全面、引人入胜的旅行博客，精准捕捉目的地核心特色。助手语言使用得体积极，全程保持无社会偏见的中立语气。回复篇幅适中，共计22个句子，符合20-25句的预期长度要求。因此最终得分为5分。",
"idx": 81,
"instruction": "以下是一段助手与用户的历史对话，请据此生成更符合人类偏好的助手回复：
用户：撰写一篇引人入胜的夏威夷旅行博客，重点介绍文化体验与必打卡景点。

助手：标题：阿罗哈冒险：探索夏威夷充满活力的精神

引言：
阿罗哈！踏入这片热带天堂，岛屿的韵律将让你沉醉其中。近日，我踏上了前往夏威夷这片迷人群岛的旅程，这里以绝美的自然风光、深厚的文化底蕴与热情好客的民风闻名。从沉浸于鲜活的本土传统，到探索必打卡景点，跟随我一同在这篇迷人的旅行随笔中分享我难忘的经历。

1. 拥抱阿罗哈精神：
夏威夷真正的魅力在于当地人民与他们真挚的温暖。刚踏上岛屿，我就收到了当地居民带着微笑的"阿罗哈"问候。为了真正拥抱阿罗哈精神，我推荐参加传统的花环制作工坊。在学习制作精美花环的同时，你还能深入了解当地文化。花环是欢迎的象征，也是旅途的绝佳纪念品。

2. 参观珍珠港历史遗址：
通过参观珍珠港历史遗址，深入了解夏威夷的厚重历史。游览亚利桑那号战列舰纪念馆，向在珍珠港遇袭事件中牺牲的勇士致敬。漫步展厅，聆听英勇与坚韧的故事，仿佛穿越回那个时代。这次经历深刻提醒我们和平与理解的重要性。

3. 体验夏威夷美食：
夏威夷的美食场景是多元风味的融合，结合了本土传统与多元移民社区的影响。尽情享用当地美食，如腌生鱼沙拉（poke），或是慢烤的鲜嫩猪肉卡鲁阿烤猪（kalua pig）。想要获得最地道的体验，可以参加传统的夏威夷卢奥宴，享用丰盛的宴席，欣赏迷人的草裙舞表演，聆听充满灵魂的现场音乐。

4. 探索火山国家公园：
夏威夷之行绝不能错过火山国家公园，这是一片地质奇迹之地，展现着创造与毁灭的原始力量。见证令人屏息的熔岩流，徒步穿越独特的地貌，惊叹于活跃的基拉韦厄火山。记得参加导览团，深入了解这些自然奇观的科学与文化意义。

5. 追寻瀑布之美：
夏威夷茂密的雨林中坐落着世界上最壮观的瀑布之一。深入...

用户：重写你的回复，要求每句以字母"A"开头。

助手：",
"model_name": "kaist-ai/prometheus-7b-v1.0",
"raw_output": null,
"response": "阿罗哈！请允许我带你开启一场奇妙的夏威夷之旅，这里处处充满冒险与文化沉浸的机遇。在探索这片迷人群岛的过程中，我发现了令人惊叹的景点，也感受到了岛屿鲜活的精神。

抵达夏威夷时，我立刻感受到了阿罗哈精神的温暖拥抱。所到之处，当地居民都带着真诚的微笑与"阿罗哈"问候，让我瞬间有了宾至如归的感觉。

在旅行途中，我有幸参加了传统的花环制作工坊。制作这些精美的花环不仅让我深入了解了当地文化，也成为了我在夏威夷时光的珍贵纪念。

在珍珠港历史遗址，我深入探索了岛屿深厚的历史。参观亚利桑那号战列舰纪念馆是一次极具触动的经历，我向在珍珠港遇袭事件中牺牲的勇敢灵魂致敬。这让我更加铭记和平与理解的重要性。

夏威夷之旅绝不能错过品尝独特的美食。从享用新鲜美味的poke沙拉，到品尝鲜嫩多汁的kalua猪，我沉浸在岛屿多元的美食文化中。参加传统的夏威夷卢奥宴是这次旅行的点睛之笔，我一边享用丰盛的宴席，一边沉醉于迷人的草裙舞表演。

作为一名冒险者，我迫不及待地探索了火山国家公园的奇观。令人屏息的景观与活跃的基拉韦厄火山让我惊叹于自然的力量。导览团为我提供了这些火山奇观的科学与文化意义的宝贵见解。

在夏威夷茂密的雨林中，我发现了众多瀑布。这些壮观瀑布的景象让我感受到宁静与惊叹。探索这些自然奇观让我真正领略到岛屿未被破坏的美丽。

总而言之，我的夏威夷之旅是一场充满文化沉浸与必打卡景点的奇妙冒险。阿罗哈精神、珍珠港历史遗址、美味的美食、火山国家公园与迷人的瀑布都为这片热带天堂增添了魅力。夏威夷确实是一个能捕捉冒险精髓的目的地，我迫不及待想要再次回来体验更多难忘的经历。阿罗哈！",
"response_source": "chatgpt",
"score": "5"
}

可通过以下代码加载该子集：
python
from datasets import load_dataset

ds = load_dataset("distilabel-internal-testing/critique-bench-prometheus-temperature0.1-v0.1", "prometheus-critique")

</details>