Data_Sheet_1_ProkBERT family: genomic language models for microbiome applications.PDF

BackgroundIn the evolving landscape of microbiology and microbiome analysis, the integration of machine learning is crucial for understanding complex microbial interactions, and predicting and recognizing novel functionalities within extensive datasets. However, the effectiveness of these methods in microbiology faces challenges due to the complex and heterogeneous nature of microbial data, further complicated by low signal-to-noise ratios, context-dependency, and a significant shortage of appropriately labeled datasets. This study introduces the ProkBERT model family, a collection of large language models, designed for genomic tasks. It provides a generalizable sequence representation for nucleotide sequences, learned from unlabeled genome data. This approach helps overcome the above-mentioned limitations in the field, thereby improving our understanding of microbial ecosystems and their impact on health and disease.
MethodsProkBERT models are based on transfer learning and self-supervised methodologies, enabling them to use the abundant yet complex microbial data effectively. The introduction of the novel Local Context-Aware (LCA) tokenization technique marks a significant advancement, allowing ProkBERT to overcome the contextual limitations of traditional transformer models. This methodology not only retains rich local context but also demonstrates remarkable adaptability across various bioinformatics tasks.
ResultsIn practical applications such as promoter prediction and phage identification, the ProkBERT models show superior performance. For promoter prediction tasks, the top-performing model achieved a Matthews Correlation Coefficient (MCC) of 0.74 for E. coli and 0.62 in mixed-species contexts. In phage identification, ProkBERT models consistently outperformed established tools like VirSorter2 and DeepVirFinder, achieving an MCC of 0.85. These results underscore the models' exceptional accuracy and generalizability in both supervised and unsupervised tasks.
ConclusionsThe ProkBERT model family is a compact yet powerful tool in the field of microbiology and bioinformatics. Its capacity for rapid, accurate analyses and its adaptability across a spectrum of tasks marks a significant advancement in machine learning applications in microbiology. The models are available on GitHub (https://github.com/nbrg-ppcu/prokbert) and HuggingFace (https://huggingface.co/nerualbioinfo) providing an accessible tool for the community.

背景：在持续演进的微生物学与微生物组分析研究领域中，机器学习的整合对于解析复杂的微生物互作关系、从海量数据集中预测并识别新型功能而言至关重要。然而此类方法在微生物学领域的应用效果却面临诸多挑战：微生物数据本身兼具复杂性与异质性，加之信噪比偏低、存在上下文依赖性，且适配的标注数据集严重匮乏，使得应用难度进一步攀升。本研究推出了ProkBERT模型家族——一类专为基因组任务设计的大语言模型（Large Language Model）集合，通过对未标注基因组数据进行预训练，实现了通用化的核苷酸序列表征能力。该方案有助于克服上述领域痛点，进而深化我们对微生物生态系统及其对健康与疾病影响的认知。
方法：ProkBERT模型基于迁移学习与自监督学习方法构建，可有效利用体量庞大但结构复杂的微生物数据。本研究提出的新型局部上下文感知（Local Context-Aware, LCA）Token化技术是一项重要突破，使ProkBERT得以突破传统Transformer模型的上下文局限。该方法不仅能够保留丰富的局部上下文信息，还在各类生物信息学任务中展现出出色的泛用性。
结果：在启动子预测、噬菌体识别等实际应用场景中，ProkBERT模型均展现出卓越性能。在启动子预测任务中，性能最优的模型在大肠杆菌数据集上的马修斯相关系数（Matthews Correlation Coefficient, MCC）可达0.74，在混合物种场景下则为0.62。在噬菌体识别任务中，ProkBERT模型的表现持续优于VirSorter2、DeepVirFinder等现有主流工具，其MCC值可达0.85。上述结果充分证明，该模型在监督学习与无监督学习任务中均具备极高的准确性与泛化能力。
结论：ProkBERT模型家族是微生物学与生物信息学领域一款轻量化却功能强劲的工具。其能够实现快速精准的分析，且可适配多样任务场景，标志着机器学习在微生物学领域的应用取得了重要进展。该模型已在GitHub（https://github.com/nbrg-ppcu/prokbert）与HuggingFace（https://huggingface.co/nerualbioinfo）平台公开，可为学界提供便捷可用的研究工具。