ReBatch/ultrafeedback_nl

---
dataset_info:
features:
- name: prompt
dtype: string
- name: chosen
list:
- name: content
dtype: string
- name: role
dtype: string
- name: rejected
list:
- name: content
dtype: string
- name: role
dtype: string
- name: prompt_id
dtype: string
- name: messages
list:
- name: content
dtype: string
- name: role
dtype: string
splits:
- name: train_prefs
num_bytes: 429608574.0
num_examples: 69932
- name: test_prefs
num_bytes: 8943665.0
num_examples: 1475
download_size: 203640550
dataset_size: 438552239.0
configs:
- config_name: default
data_files:
- split: train_prefs
path: data/train_prefs-*
- split: test_prefs
path: data/test_prefs-*
---

# Dataset Card for ultrafeedback_nl

## Dataset Description

This dataset is a combination 2 datasets. The first is a tranlsation of [HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) using a MarianMT model.
It contains both Chosen and rejected versions of multi-turn chat conversations between a user and an assistant.
The second consists of the DPO HQ samples from [BramVanroy/ultra_feedback_dutch_cleaned](https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch_cleaned). Which are in turn recreated and cleaned samples from HuggingFaceH4/ultrafeedback_binarized using gpt-4.

## Dataset Structure

The dataset has two splits; Only the prefs splits of the original dataset were translated. This is because the datasets was created for DPO finetuning.

| Source | train_prefs | test_prefs |
|:-------:|:-------:|:-----------:|
| Total | 69932 | 1475 |
| HuggingFaceH4/ultrafeedback_binarized | 61135 | 2000 |
| BramVanroy/ultra_feedback_dutch_cleaned dpo_hq | 9186 | 484 |

## Usage

```python
from datasets import load_dataset

ds = load_dataset("ReBatch/ultrafeedback_nl")

```

## Translation

The ultrafeedback_binarized dataset was translated using [vvn/en-to-dutch-marianmt](https://huggingface.co/vvn/en-to-dutch-marianmt).
A fine-Tuned MarianMT translation model for translating text from English to Dutch.

原始信息汇总

数据集概述

数据集描述

该数据集包含两个部分：

1. 使用MarianMT模型翻译的HuggingFaceH4/ultrafeedback_binarized数据集。
2. 来自BramVanroy/ultra_feedback_dutch_cleaned的DPO HQ样本，这些样本是使用gpt-4重新创建和清理的。

数据集结构

特征

• prompt: 字符串类型
• chosen: 列表类型，包含两个子特征：
  • content: 字符串类型
  • role: 字符串类型
• rejected: 列表类型，包含两个子特征：
  • content: 字符串类型
  • role: 字符串类型
• prompt_id: 字符串类型
• messages: 列表类型，包含两个子特征：
  • content: 字符串类型
  • role: 字符串类型

分割

• train_prefs:
  • 字节数: 429608574.0
  • 样本数: 69932
• test_prefs:
  • 字节数: 8943665.0
  • 样本数: 1475

大小

• 下载大小: 203640550
• 数据集大小: 438552239.0

配置

• default:
  • train_prefs: 路径为data/train_prefs-*
  • test_prefs: 路径为data/test_prefs-*

使用方法

python from datasets import load_dataset

ds = load_dataset("ReBatch/ultrafeedback_nl")

翻译

使用vvn/en-to-dutch-marianmt模型将ultrafeedback_binarized数据集从英语翻译为荷兰语。