Solving the RNA design problem with reinforcement learning

We use reinforcement learning to train an agent for computational RNA design: given a target secondary structure, design a sequence that folds to that structure in silico. Our agent uses a novel graph convolutional architecture allowing a single model to be applied to arbitrary target structures of any length. After training it on randomly generated targets, we test it on the Eterna100 benchmark and find it outperforms all previous algorithms. Analysis of its solutions shows it has successfully learned some advanced strategies identified by players of the game Eterna, allowing it to solve some very difficult structures. On the other hand, it has failed to learn other strategies, possibly because they were not required for the targets in the training set. This suggests the possibility that future improvements to the training protocol may yield further gains in performance.

我们采用强化学习（reinforcement learning）训练用于计算RNA设计的AI智能体（AI Agent）：给定目标二级结构，设计可在计算机模拟（in silico）条件下折叠为该结构的RNA序列。我们的AI智能体采用一种新颖的图卷积（graph convolutional）架构，使得单个模型可适配任意长度的任意目标二级结构。在随机生成的目标结构上完成训练后，我们在Eterna100基准测试集上对该模型进行评测，发现其性能优于所有此前提出的算法。对其生成的解决方案进行分析可知，该模型已成功学习到Eterna游戏玩家所总结的部分高级策略，从而能够攻克部分极具挑战性的结构设计任务。但另一方面，该模型未能掌握其余相关策略，这可能是因为训练集的目标结构并不需要用到这些策略。这表明，未来对训练协议进行优化改进，有望进一步提升模型的整体性能。