Replacing the final prediction layer of end-to-end trained Transformer Networks with gradient boosting models improves performance.

The table displays the performance metrics for the end-to-end (E2E) trained Transformer Networks and for a gradient boosting model for the prediction of enzyme-substrate pairs. The E2E Transformer Network was trained with different numbers of hidden layers and different numbers of nodes in the hidden layers for its fully-connected neural network on top of the attention blocks. The gradient boosting model was trained with the learned joint protein-small molecule embeddings as its only input. Arrows next to the metric names (↑) indicate that higher values correspond to better model performance.
(XLSX)