ontocord/falcon_subset_100k

---
license: odc-by
---

### Description of Dataset
This is a subset of the Falcon Refinedweb dataset, with tagging performed using the taggers in the [ontocord/riverbed](https://huggingface.co/datasets/ontocord/riverbed) repository. This subset includes approximately 100K items (485MB or approximately 100M tokens). The items have been filtered for moderately high Red-Pajama 1 filter scores (0.65) and Textbook quality scores (0.65).
Tags are added to the beginning of some of the items when they have been detected to be science based, how-to, legal, etc. The original tagging was performed by Ontocord for the Aurora-M project, and subsequently further tagged for textbook quality by @kenhktsui [here](https://huggingface.co/datasets/kenhktsui/falcon_subset_quality_score_v1). It was further filtered to create this subset.

### Original Refinedweb Dataset
The authors of Refinedweb writes:
> RefinedWeb is built through stringent filtering and large-scale deduplication of CommonCrawl; we found models trained on RefinedWeb to achieve performance in-line or better than models trained on curated datasets, while only relying on web data.
See the 📓 [paper on arXiv](https://arxiv.org/abs/2306.01116) for more details on Refinedweb itself.

### Purpose
The purpose of the dataset is to investigate filtering of data and creating performant mixtures of data for model training.

### Disclaimer
This dataset is provided for **research purposes only**. We make no claims to ownership of the data and release any of our annotations under ODC-By because that was the original RefinedWeb license. We believe usage of web crawled data is permitted under fair use, but you use this dataset at your own risks, and we disclaim all warranties and liabilities including warranties for non-infringement.

NO PERSONALLY IDENTIFIABLE INFORMATION REDACTION WAS PERFORMED. ALL SUCH INFORMATION ARE FROM THE ORIGINAL DATASET AND THE CORRESPONDING WEB PAGE.

### Contact Us
Please reach out to us in the discussion link above if you have questions.

原始信息汇总

数据集描述

本数据集是Falcon Refinedweb数据集的一个子集，通过ontocord/riverbed仓库中的标记器进行标记。该子集包含约10万个项目（485MB或约100M令牌），经过过滤以达到中等高度的Red-Pajama 1过滤器评分（0.65）和教科书质量评分（0.65）。某些项目在检测到科学、操作指南、法律等内容时，会在其开头添加标签。原始标记由Ontocord为Aurora-M项目执行，随后由@kenhktsui 此处进一步标记教科书质量，并进一步过滤以创建此子集。

数据集目的

该数据集旨在研究数据过滤和创建用于模型训练的高性能数据混合。

免责声明

本数据集仅供研究目的使用。我们不声称拥有数据的所有权，并根据原始RefinedWeb许可证发布我们的任何注释为ODC-By。我们相信使用网络爬取数据属于合理使用范畴，但您使用此数据集需自行承担风险，我们不承担所有保证和责任，包括不侵权的保证。

未进行个人身份信息重编。所有此类信息均来自原始数据集和相应的网页。