Deep Learning-Based Conformal Prediction of Toxicity

Predictive
modeling for toxicity can help reduce risks in a range
of applications and potentially serve as the basis for regulatory
decisions. However, the utility of these predictions can be limited
if the associated uncertainty is not adequately quantified. With recent
studies showing great promise for deep learning-based models also
for toxicity predictions, we investigate the combination of deep learning-based
predictors with the conformal prediction framework to generate highly
predictive models with well-defined uncertainties. We use a range
of deep feedforward neural networks and graph neural networks in a
conformal prediction setting and evaluate their performance on data
from the Tox21 challenge. We also compare the results from the conformal
predictors to those of the underlying machine learning models. The
results indicate that highly predictive models can be obtained that
result in very efficient conformal predictors even at high confidence
levels. Taken together, our results highlight the utility of conformal
predictors as a convenient way to deliver toxicity predictions with
confidence, adding both statistical guarantees on the model performance
as well as better predictions of the minority class compared to the
underlying models.