opendatalab/SlimPajama-Meta-rater-Professionalism-30B
收藏Hugging Face2025-06-15 更新2025-07-05 收录
下载链接：
https://hf-mirror.com/datasets/opendatalab/SlimPajama-Meta-rater-Professionalism-30B
下载链接
链接失效反馈官方服务：
资源简介：
这个数据集包含了从SlimPajama-627B语料库中使用专业性维度筛选出的前30B个token。每个文档都由一个经过ModernBERT模型微调的评估器进行评分和过滤，以评估理解文本所需的专业知识和专业程度。数据集涵盖了CommonCrawl、C4、GitHub、书籍、ArXiv、维基百科和StackExchange等领域的文本。
This dataset contains the top 30B tokens from the SlimPajama-627B corpus, selected using the Professionalism dimension of the PRRC framework. Each document in this subset is scored and filtered by a ModernBERT-based rater fine-tuned to assess the degree of professional knowledge and expertise required to comprehend the text, covering domains such as CommonCrawl, C4, GitHub, Books, ArXiv, Wikipedia, and StackExchange.
提供机构：
opendatalab
