GeoT: A Geometry-Aware Transformer for Reliable Molecular Property Prediction and Chemically Interpretable Representation Learning

In recent years, molecular representation learning has
emerged
as a key area of focus in various chemical tasks. However, many existing
models fail to fully consider the geometric information on molecular
structures, resulting in less intuitive representations. Moreover,
the widely used message passing mechanism is limited to providing
the interpretation of experimental results from a chemical perspective.
To address these challenges, we introduce a novel transformer-based
framework for molecular representation learning, named the geometry-aware
transformer (GeoT). The GeoT learns molecular graph structures through
attention-based mechanisms specifically designed to offer reliable
interpretability as well as molecular property prediction. Consequently,
the GeoT can generate attention maps of the interatomic relationships
associated with training objectives. In addition, the GeoT demonstrates
performance comparable to that of MPNN-based models while achieving
reduced computational complexity. Our comprehensive experiments, including
an empirical simulation, reveal that the GeoT effectively learns chemical
insights into molecular structures, bridging the gap between artificial
intelligence and molecular sciences.