leduo123/assignment4-preferences

---
dataset_info:
features:
- name: prompt
dtype: string
- name: chosen
dtype: string
- name: rejected
dtype: string
splits:
- name: train
num_bytes: 135314
num_examples: 50
download_size: 90303
dataset_size: 135314
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---
# Assignment 4 Preference Dataset

This dataset was created for Assignment 4.

It contains 50 preference training examples built from instructions sampled from the LIMA dataset. For each instruction, 5 candidate responses were generated using `Qwen/Qwen2.5-7B-Instruct`, and the responses were ranked with `llm-blender/PairRM`. The highest-ranked response was used as `chosen`, and the lowest-ranked response was used as `rejected`.

## Fields

- `prompt`: the chat-formatted training prompt
- `chosen`: the preferred response
- `rejected`: the less preferred response

## Usage
This dataset is intended for preference-based fine-tuning methods such as DPO.