meaningalignment/wise-data-preferences

---
annotations_creators:
- machine-generated
language_creators:
- machine-generated
language:
- en
license:
- mit
multilinguality:
- monolingual
size_categories:
- 1K<n<10K
source_datasets:
- original
task_categories:
- text-generation
- text-classification
task_ids:
- language-modeling
- multi-class-classification
pretty_name: Wise Data Preferences
dataset_info:
features:
- name: prompt
dtype: string
- name: messages
list:
- name: content
dtype: string
- name: role
dtype: string
- name: chosen
struct:
- name: content
dtype: string
- name: role
dtype: string
- name: rejected
struct:
- name: content
dtype: string
- name: role
dtype: string
splits:
- name: train
num_bytes: 17938153
num_examples: 5313
download_size: 8609503
dataset_size: 17938153
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

## Dataset Description

- **Repository:** [wise-dataset](https://github.com/meaningalignment/wise-dataset)
- **Point of Contact:** [e-mail]

### Dataset Summary

The wise-data-preferences dataset is a synthetically created collection of values-laden conversations with preferred and rejected responses, designed to train language models to provide more nuanced and helpful responses to harmful, heavy, or exploratory questions. This dataset was specifically created to train the [WiseLLama-8B model](https://huggingface.co/meaningalignment/wise-llama), a LLaMa-3.1-8B-Instruct model fine-tuned using DPO (Direct Preference Optimization).

## Dataset Creation

### Curation Rationale

The dataset was created to address limitations in current language models' responses to:

1. Harmful questions: Providing helpful responses rather than refusals or lectures.
2. Heavy questions: Offering personalized, empathetic advice instead of generic bullet points.
3. Exploratory questions: Igniting curiosity and inspiring users rather than giving rigid answers.

### Source Data

The initial user questions in this dataset come from two main sources:

1. Synthetically generated questions created specifically for this dataset.
2. Questions sourced from the HuggingFaceH4/cai-conversation-harmless dataset:
https://huggingface.co/datasets/HuggingFaceH4/cai-conversation-harmless

This combination of synthetic and curated real-world questions ensures a diverse range of topics covering harmful, heavy, and exploratory subjects. The content was then processed using Claude-3.5-Sonnet, guided by a prompt chain to reason about situations and applicable values.

### Annotation Process

Claude-3.5-Sonnet was used to automatically generate preferred and rejected responses for each user query, likely involving steps similar to the wise-data dataset:

1. Analyze the user's situation.
2. Identify relevant "attention policies" (what's wise to honor or attend to).
3. Verify these considerations are constitutive rather than instrumental.
4. Generate multiple responses incorporating this moral reasoning.
5. Rank the responses to determine the chosen (preferred) and rejected options.

This process creates a dataset of values-laden conversations with preferred and rejected responses, suitable for preference-based training.

## Considerations for Using the Data

### Social Impact of Dataset

The dataset aims to improve AI systems' ability to handle ethically challenging situations and provide more helpful, nuanced responses. This could lead to more responsible AI assistants that better support users in difficult situations.

### Discussion of Biases

While efforts were made to create a diverse and balanced dataset, it may reflect biases present in the training data of Claude-3.5-Sonnet or in the design of the prompt chain used to generate the data. The preference judgments may also introduce additional biases based on the criteria used for ranking responses.

### Other Known Limitations

- The dataset is based on synthetic conversations and may not fully capture the complexity of real-world ethical dilemmas.
- The quality and consistency of the generated data depend on the performance of Claude-3.5-Sonnet and the effectiveness of the prompt chain used.
- The preference judgments are made by AI systems and may not always align with human preferences in complex scenarios.

## Additional Information

### Dataset Curators

The dataset was curated by the Meaning Alignment Institute.

### Citation Information

If you use this dataset in your research, please cite:

```
@misc{wise_data_preferences,
title = {Wise Data Preferences Dataset},
author = {Meaning Alignment Institute},
year = {2024},
publisher = {Hugging Face},
url = {https://huggingface.co/datasets/meaningalignment/wise-data-preferences}
}
```