科技文献预训练语料集

该数据集主要根据预训练语言模型的研究需要，遴选8个领域的科技文献数据，数据从参与课题的研究单位中国科学院文献情报中心、北京万方数据股份有限公司和广州奥凯信息咨询有限公司数据集中遴选，然后经过格式化处理，生成JSON格式文件。数据采集环节由中国科学院文献情报中心、北京万方数据股份有限公司和广州奥凯信息咨询有限公司主要负责，从现有数据集中提供基础数据。数据加工环节主要由中国科学院自动化研究所、中国科学院计算机网络信息中心和广州奥凯信息咨询有限公司负责，根据预训练模型需要的语料情况，筛选出高质量的数据。数据处理加工环节不需要特定型号的仪器设备。

This dataset is primarily curated to meet the research requirements of pre-trained language models, and it selects scientific literature data across 8 domains. The source data is selected from the datasets of three participating research institutions: the National Science Library of the Chinese Academy of Sciences, Beijing Wanfang Data Co., Ltd., and Guangzhou Aokai Information Consulting Co., Ltd., and then formatted into JSON-format files. The data collection phase was mainly undertaken by the aforementioned three institutions, which provided basic data from their respective existing datasets. The data processing and curation phase was primarily undertaken by the Institute of Automation of the Chinese Academy of Sciences, the Computer Network Information Center of the Chinese Academy of Sciences, and Guangzhou Aokai Information Consulting Co., Ltd., where high-quality data was screened in accordance with the corpus requirements of pre-trained language models. No specific instruments or equipment are required throughout the entire data processing and curation process.

搜集汇总

背景与挑战

背景概述

该数据集是为预训练语言模型研究而构建的科技文献语料库，涵盖8个领域的数据，由中国科学院文献情报中心等机构提供基础数据，并经过格式化处理为JSON格式。数据集总规模为163.61GB，包含318个文件，旨在支持大模型训练和自然语言处理应用。