HelpSteer-filtered

# HelpSteer-filtered

This dataset is a highly filtered version of the [nvidia/HelpSteer](https://huggingface.co/datasets/nvidia/HelpSteer) dataset.

# ❓ How this dataset was filtered:

1. I calculated the sum of the columns `["helpfulness," "correctness," "coherence," "complexity," "verbosity"]` and created a new column named `sum`.

2. I changed some column names and added a **empty column** to match the Alpaca format.

3. The dataset was then filtered to include only those entries with a sum greater than or equal to 16.

# 🧐 More Information

You can find more information about the unfiltered dataset here:

- [nvidia/HelpSteer](https://huggingface.co/datasets/nvidia/HelpSteer)

# HelpSteer-filtered
本数据集为[nvidia/HelpSteer](https://huggingface.co/datasets/nvidia/HelpSteer)数据集的高精度过滤版本。

# ❓ 数据集过滤流程
1. 对`["有用性（helpfulness）", "正确性（correctness）", "连贯性（coherence）", "复杂度（complexity）", "冗长度（verbosity）"]`各列的数值进行求和运算，新增一列并命名为`sum`。
2. 修改部分列名，并新增一个**空列**以适配Alpaca格式（Alpaca）。
3. 随后对数据集进行过滤，仅保留`sum`列取值大于或等于16的样本条目。

# 🧐 补充信息
您可通过以下链接获取原始未过滤数据集的更多信息：
- [nvidia/HelpSteer](https://huggingface.co/datasets/nvidia/HelpSteer)

搜集汇总

背景与挑战

背景概述

HelpSteer-filtered是nvidia/HelpSteer数据集的高度过滤版本，通过计算多个评分列的总和并筛选总和大于等于16的条目生成，同时调整了列名以匹配Alpaca格式。