HOAX: a hyperparameter optimisation algorithm explorer for neural networks

Computational chemistry has become an important tool to predict and understand molecular properties and reactions. Even though recent years have seen a significant growth in new algorithms and computational methods that speed up quantum chemical calculations, the bottleneck for trajectory-based methods to study photo-induced processes is still the huge number of electronic structure calculations. In this work, we present an innovative solution, in which the amount of electronic structure calculations is drastically reduced, by employing machine learning algorithms and methods borrowed from the realm of artificial intelligence. However, applying these algorithms effectively requires finding optimal hyperparameters, which remains a challenge itself. Here we present an automated user-friendly framework, HOAX, to perform the hyperparameter optimisation for neural networks, which bypasses the need for a lengthy manual process. The neural network-generated potential energy surfaces (PESs) reduce the computational costs compared to the ab initio-based PESs. We perform a comparative investigation on the performance of different hyperparameter optimiziation algorithms, namely grid search, simulated annealing, genetic algorithm, and Bayesian optimizer in finding the optimal hyperparameters necessary for constructing the well-performing neural network in order to fit the PESs of small organic molecules. Our results show that this automated toolkit not only facilitate a straightforward way to perform the hyperparameter optimisation but also the resulting neural networks-based generated PESs are in reasonable agreement with the ab initio-based PESs.

计算化学（Computational Chemistry）现已成为预测与解析分子性质及化学反应的关键工具。尽管近年来涌现出大量可加速量子化学计算的新型算法与计算方法，但基于轨迹的光诱导过程研究方法仍面临核心瓶颈：电子结构计算的体量过于庞大。本研究提出一项创新性解决方案：通过引入机器学习算法与人工智能领域的相关技术，大幅削减了电子结构计算的工作量。然而，高效应用这类算法需先确定最优超参数（hyperparameters），而这本身亦是一项极具挑战的工作。为此，本研究推出一款自动化且易用的框架HOAX，可针对神经网络（neural networks）完成超参数优化，无需冗长的手动调参流程。相较于基于从头算（ab initio）的势能面（potential energy surfaces），神经网络生成的势能面可显著降低计算成本。本研究针对多款超参数优化算法开展了对比性能测试，包括网格搜索（grid search）、模拟退火（simulated annealing）、遗传算法（genetic algorithm）以及贝叶斯优化器（Bayesian optimizer），以筛选出构建高性能神经网络所需的最优超参数，进而拟合小型有机分子的势能面。研究结果表明，这款自动化工具包不仅为超参数优化提供了简便易行的实现路径，其生成的基于神经网络的势能面也与基于从头算的势能面具有良好的一致性。