A deep learning model based on the BERT pre-trained model to predict the antiproliferative activity of anti-cancer chemical compounds

Identifying new compounds with minimal side effects to enhance patients’ quality of life is the ultimate goal of drug discovery. Due to the expensive and time-consuming nature of experimental investigations and the scarcity of data in traditional QSAR studies, deep transfer learning models, such as the BERT model, have recently been suggested. This study evaluated the model’s performance in predicting the anti-proliferative activity of five cancer cell lines (HeLa, MCF7, MDA-MB231, PC3, and MDA-MB) using over 3,000 synthesized molecules from PubChem. The results indicated that the model could predict the class of designed small molecules with acceptable accuracy for most cell lines, except for PC3 and MDA-MB. The model’s performance was further tested on an in-house dataset of approximately 25 small molecules per cell line, based on IC50 values. The model accurately predicted the biological activity class for HeLa with an accuracy of 0.77±0.4 and demonstrated acceptable performance for MCF7 and MDA-MB231, with accuracy between 0.56 and 0.66. However, the results were less reliable for PC3 and HepG2. In conclusion, the ChemBERTa fine-tuned model shows potential for predicting outcomes on in-house datasets.

开发副作用最小的新型化合物以提升患者生活质量，是药物研发的终极目标。鉴于实验研究成本高昂且耗时漫长，且传统定量构效关系（Quantitative Structure-Activity Relationship, QSAR）研究存在数据稀缺问题，深度迁移学习模型（如BERT模型，即Bidirectional Encoder Representations from Transformers）近年来被应用于该领域。本研究利用PubChem数据库中超过3000个合成小分子，评估了该模型对5种癌细胞系（HeLa、MCF7、MDA-MB231、PC3及MDA-MB）抗增殖活性的预测性能。结果显示，除PC3与MDA-MB细胞系外，该模型对多数癌细胞系的设计小分子类别预测精度均达到可接受水平。本研究进一步基于半最大效应浓度（Half Maximal Inhibitory Concentration, IC50）值，在每个细胞系约含25个小分子的内部自有数据集上测试了该模型的性能。该模型对HeLa细胞系的生物活性类别预测准确率达0.77±0.4，表现精准；对MCF7与MDA-MB231细胞系的预测性能亦处于可接受范围，准确率介于0.56至0.66之间。但针对PC3与HepG2细胞系的预测结果可靠性相对较低。综上，经微调的ChemBERTa模型在内部自有数据集的活性预测任务中展现出应用潜力。