开源连续学习长文档分类数据集
收藏资源简介：
该数据集由exAAPD、exLitCovid、exMeSH和exPFD四个针对不同学术领域的子数据集整合而成，共包含约110,359篇高保真结构化长文档数据。其中，exAAPD涵盖49,326篇源自arXiv的计算机科学论文，涉及cs.cv、cs.it等54个学科类别，并具备明确的时间演进特性；exLitCovid包含14,831篇关于新冠病毒的PubMed Central论文，精细划分为预防、治疗、机制等8个核心研究专题；exMeSH包含15,966篇基于MeSH词表扩展的医学论文，涵盖心血管、糖尿病等11类顶级疾病；exPFD则包含30,236篇跨地理、政治、经济、心理等7个领域的学术文档，并具备自然的类别不均衡分布特征。全套数据集采用JSON格式存储，完整保留了文档的层级结构信息，能够支持连续学习设定下的跨领域知识迁移、长文本语义建模及新类发现研究。数据构建方案：该数据集基于Python程序化工具从arXiv、PubMed及相关公开学术数据库进行自动化采集，并通过分词、去噪及结构化解析等流程处理而成。
This dataset is compiled from four sub-datasets targeting distinct academic domains, namely exAAPD, exLitCovid, exMeSH, and exPFD, and contains approximately 110,359 high-fidelity structured long documents. Specifically, exAAPD includes 49,326 computer science papers sourced from arXiv, covering 54 academic disciplines such as cs.cv and cs.it, and exhibits clear temporal evolution characteristics. exLitCovid contains 14,831 PubMed Central papers focused on COVID-19, which are finely categorized into 8 core research topics including prevention, treatment, and pathogenesis. exMeSH consists of 15,966 medical papers expanded based on the MeSH thesaurus, covering 11 top-level disease categories such as cardiovascular diseases and diabetes. exPFD includes 30,236 academic documents spanning 7 fields such as geography, politics, economics, and psychology, and has a natural class-imbalanced distribution. The entire dataset is stored in JSON format, fully preserving the hierarchical structural information of the documents, and can support research on cross-domain knowledge transfer, long-text semantic modeling, and novel category discovery under the continual learning setup. Data construction pipeline: This dataset was automatically collected from arXiv, PubMed and other relevant public academic databases using Python-based programmatic tools, and processed through procedures such as tokenization, denoising, and structured parsing.
