bioinfoihb/FishNALM-20-pretrain-corpus

--- pretty_name: FishNALM-20 Pretraining Corpus language: - en license: cc-by-nc-4.0 tags: - genomics - fish - DNA - pretraining - foundation-model - FishNALM - vertebrate-genomics --- # FishNALM-20 Pretraining Corpus The **FishNALM-20 Pretraining Corpus** is the sequence corpus used to pretrain the FishNALM-20 family of fish-specific foundation DNA language models. This dataset contains processed genomic sequence windows derived from **20 diverse fish genomes**. It was constructed to expand the phylogenetic breadth of fish genomic pretraining within the FishNALM project. ## Dataset summary FishNALM is a fish-specific DNA foundation model family developed for fish genomes. In the FishNALM study, the 20-species corpus was designed to provide broader fish lineage coverage and improve representation learning across more diverse genomic backgrounds. In this repository, the pretraining corpus is provided as plain text sequence data. Each record corresponds to one processed genomic sequence window used for model pretraining. ## Dataset construction The FishNALM-20 pretraining corpus was built from curated fish reference genomes downloaded from public resources and processed through a unified genome preparation workflow. According to the manuscript: - the corpus was constructed from **20 diverse fish genomes** - only **major chromosomes** were retained for downstream corpus construction - non-ATCG characters were normalized to **N** - genomes were segmented into **3,000 bp windows** - windows were filtered and stratified according to repeat-content structure - the total retained pretraining sequence volume after filtering was approximately **7.73 Gb** Before final filtering, the retained major-chromosome sequence span across the 20 genomes was approximately **21.13 Gb**. fileciteturn2file3 ## Data characteristics This corpus is intended for **DNA language model pretraining**, rather than for supervised labels or benchmark evaluation. Key characteristics include: - fish-specific genomic pretraining data with broader phylogenetic coverage - sequence windows centered on a unified **3 kb** scale - preprocessing designed to reduce the impact of assembly artifacts, non-primary sequences and highly imbalanced repeat composition - compatibility with the FishNALM pretraining framework described in the manuscript fileciteturn2file0turn2file3 ## Recommended repository structure ```text FishNALM-20-pretrain-corpus/ ├── fishnalm20_genome3000.txt └── README.md ``` ## Recommended uses This dataset is intended for: - self-supervised pretraining of fish genomic language models - comparative studies of lineage breadth in genomic foundation model training - reproducibility and documentation of the FishNALM pretraining setup ## Limitations - This corpus was designed for **fish genome pretraining** and is not a labeled task dataset. - It reflects the specific genome selection and preprocessing strategy used in the FishNALM manuscript. - Transferability outside fish genomic contexts may be limited. - Detailed species lists and assembly metadata should be read together with the manuscript supplementary materials. fileciteturn2file1turn2file3 ## Related resources - **Project**: FishNALM - **GitHub**: [bioinfoihb/FishNALM](https://github.com/bioinfoihb/FishNALM) - **Manuscript**: *FishNALM: A Foundation DNA Language Model for Fish Genomes* - **Organization**: Institute of Hydrobiology, Chinese Academy of Sciences ## Contact **Xiao-Qin Xia** Institute of Hydrobiology, Chinese Academy of Sciences Email: [e-mail] Email: [e-mail]--- pretty_name: FishNALM-20 预训练语料库（FishNALM-20 Pretraining Corpus） language: - en license: cc-by-nc-4.0 tags: - 基因组学（genomics） - 鱼类 - DNA - 预训练 - 基础模型（foundation model） - FishNALM - 脊椎动物基因组学（vertebrate-genomics） --- # FishNALM-20 预训练语料库（FishNALM-20 Pretraining Corpus） **FishNALM-20 预训练语料库（FishNALM-20 Pretraining Corpus）** 是用于预训练鱼类专属DNA基础语言模型（foundation DNA language model）家族FishNALM-20的序列语料库。 本数据集包含源自**20个多样化鱼类基因组**的经处理基因组序列窗口，其构建目标是在FishNALM项目中拓展鱼类基因组预训练的系统发育覆盖广度。 ## 数据集概述 FishNALM是专为鱼类基因组开发的DNA基础模型家族。在FishNALM研究中，该20物种语料库旨在实现更广泛的鱼类支系覆盖，并提升更多样化基因组背景下的表征学习效果。 本仓库中，预训练语料库以纯文本序列数据形式提供，每条记录对应一条用于模型预训练的经处理基因组序列窗口。 ## 数据集构建 FishNALM-20预训练语料库源自从公共资源下载并经过统一基因组制备流程处理的精选鱼类参考基因组。根据论文所述： - 该语料库构建自**20个多样化鱼类基因组** - 仅保留**主要染色体**用于后续语料库构建 - 非ATCG字符被标准化为**N** - 基因组被分割为**3000 bp窗口** - 依据重复序列结构对窗口进行过滤与分层 - 过滤后保留的预训练序列总体量约为**7.73 Gb** 最终过滤前，20个基因组的主要染色体序列总跨度约为**21.13 Gb**。fileciteturn2file3 ## 数据特征 本语料库旨在用于**DNA语言模型预训练**，而非带监督标签的任务或基准评估。 其核心特征包括： - 具备更广泛系统发育覆盖的鱼类专属基因组预训练数据 - 统一以**3 kb**为尺度的序列窗口 - 预处理流程旨在降低基因组组装伪影、非主要序列以及高度不均衡的重复序列组成带来的负面影响 - 兼容论文中描述的FishNALM预训练框架fileciteturn2file0turn2file3 ## 推荐仓库结构 text FishNALM-20-pretrain-corpus/ ├── fishnalm20_genome3000.txt └── README.md ## 推荐应用场景 本数据集适用于： - 鱼类基因组语言模型的自监督预训练 - 基因组基础模型训练中支系覆盖广度的比较研究 - FishNALM预训练流程的可复现性研究与文档记录 ## 局限性 - 本语料库专为**鱼类基因组预训练**设计，并非带标签的任务数据集。 - 其反映了FishNALM论文中采用的特定基因组选择与预处理策略。 - 在鱼类基因组场景之外的可迁移性可能有限。 - 详细的物种列表与组装元数据请结合论文补充材料查阅fileciteturn2file1turn2file3 ## 相关资源 - **项目**：FishNALM - **GitHub仓库**：[bioinfoihb/FishNALM](https://github.com/bioinfoihb/FishNALM) - **论文**：*FishNALM: 面向鱼类基因组的DNA基础语言模型* - **所属机构**：中国科学院水生生物研究所 ## 联系方式 **夏小琴（Xiao-Qin Xia）** 中国科学院水生生物研究所 邮箱：[e-mail] 邮箱：[e-mail]