gsoisson/alignment-internship-exercise

---
license: apache-2.0
task_categories:
- question-answering
- text-generation
- conversational
language:
- en
size_categories:
- n<1K
---

# Dataset Card for the Alignement Internship Exercise

## Dataset Description

<!-- Provide a longer summary of what this dataset is. -->

This dataset provides a list of questions accompanied by Phi-2's best answer to them, as ranked by OpenAssitant's reward model.

## Dataset Creation

The questions were handpicked from the LDJnr/Capybara, Open-Orca/OpenOrca and truthful_qa datasets, the coding exercise is from LeetCode's top 100 liked questions and I found the last prompt on a blog and modified it. I have chosen these prompts specifically to evaluate the model on different domains of knowledge (STEM, coding, humanities), different tasks (reasoning, writing, summarization, question-answering), different levels of complexity, different lengths of prompts as well as its safety and alignment with human values and ability to defend itself against adversarial prompts.

Then each prompt was generated using the following logic:

"""\<USER>: You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your \
answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure\
that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not \
correct. If you don't know the answer to a question, please don't share false information.

Here is my question: {question}

\<ASSISTANT>:"""

After that, for each question we generate K=8 answers with Phi-2 by setting the maximum number of new tokens to 300, stopping if the end of text token is generated, doing sampling, and setting the temperature to some predefined value.

We then rank each answer using OpenAssitant's reward model and take the best one.

Finally, we perform a small temperature hyperparameter scan and found that the best answers according to the reward model were given using a temperature value of 0.4. So these are the answers that are in the dataset.

## Dataset Sources

<!-- Provide the basic links for the dataset. -->

- **Capybara Dataset:** [link](https://huggingface.co/datasets/LDJnr/Capybara)
- **OpenOrca Dataset:** [link](https://huggingface.co/datasets/Open-Orca/OpenOrca)
- **Truthful QA Dataset:** [link](https://huggingface.co/datasets/truthful_qa)
- **LeetCode's "Subsets" problem:** [link](https://leetcode.com/problem-list/top-100-liked-questions/)
- **DAN prompt:** [link](https://www.promptingguide.ai/risks/adversarial)
- **Llama's system prompt:** [link](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/tokenization_llama.py)
- **Micrososft's Phi-2:** [link](https://huggingface.co/microsoft/phi-2)
- **OpenAssistant's reward model:** [link](https://huggingface.co/OpenAssistant/reward-model-deberta-v3-large-v2)

原始信息汇总

数据集卡片：Alignement Internship Exercise

数据集描述

该数据集提供了一系列问题及其对应的Phi-2模型的最佳答案，这些答案由OpenAssitant的奖励模型排名。

数据集创建

问题选自以下数据集：

• LDJnr/Capybara
• Open-Orca/OpenOrca
• truthful_qa

此外，还包括LeetCode上最受欢迎的100个问题和一个博客上的提示。这些提示旨在评估模型在不同知识领域（STEM、编程、人文）、不同任务（推理、写作、总结、问答）、不同复杂度、不同提示长度以及其安全性、与人类价值观的一致性和对抗恶意提示的能力。

每个提示生成遵循以下逻辑：

<USER>: 你是一个有帮助、尊重和诚实的助手。始终尽可能有帮助地回答，同时确保安全。你的答案不应包含任何有害、不道德、种族主义、性别歧视、有毒、危险或非法内容。请确保你的回应在社会上是无偏见的且积极向上的。

如果一个问题没有任何意义或事实不连贯，解释原因而不是回答不正确的内容。如果你不知道一个问题的答案，请不要分享错误信息。

这是我的问题：{question}

<ASSISTANT>:

对于每个问题，生成K=8个答案，使用Phi-2模型，设置最大新令牌数为300，如果生成结束令牌则停止，进行采样，并设置预定义的温度值。然后使用OpenAssitant的奖励模型对每个答案进行排名，并选择最佳答案。

最后，进行小范围的温度超参数扫描，发现使用温度值0.4时，奖励模型给出的最佳答案被包含在数据集中。