Conformal Regression for Quantitative Structure–Activity Relationship ModelingQuantifying Prediction Uncertainty

Making predictions
with an associated confidence is highly desirable
as it facilitates decision making and resource prioritization. Conformal
regression is a machine learning framework that allows the user to
define the required confidence and delivers predictions that are guaranteed
to be correct to the selected extent. In this study, we apply conformal
regression to model molecular properties and bioactivity values and
investigate different ways to scale the resultant prediction intervals
to create as efficient (i.e., narrow) regressors as possible. Different
algorithms to estimate the prediction uncertainty were used to normalize
the prediction ranges, and the different approaches were evaluated
on 29 publicly available data sets. Our results show that the most
efficient conformal regressors are obtained when using the natural
exponential of the ensemble standard deviation from the underlying
random forest to scale the prediction intervals, but other approaches
were almost as efficient. This approach afforded an average prediction
range of 1.65 pIC50 units at the 80% confidence level when applied
to bioactivity modeling. The choice of nonconformity function has
a pronounced impact on the average prediction range with a difference
of close to one log unit in bioactivity between the tightest and widest
prediction range. Overall, conformal regression is a robust approach
to generate bioactivity predictions with associated confidence.