Reliable Estimation of Prediction Uncertainty for Physicochemical Property Models

One of the major
challenges in computational science is to determine
the uncertainty of a virtual measurement, that is the prediction of
an observable based on calculations. As highly accurate first-principles
calculations are in general unfeasible for most physical systems,
one usually resorts to parameteric property models of observables,
which require calibration by incorporating reference data. The resulting
predictions and their uncertainties are sensitive to systematic errors
such as inconsistent reference data, parametric model assumptions,
or inadequate computational methods. Here, we discuss the calibration
of property models in the light of bootstrapping, a sampling method
that can be employed for identifying systematic errors and for reliable
estimation of the prediction uncertainty. We apply bootstrapping to
assess a linear property model linking the 57Fe Mössbauer
isomer shift to the contact electron density at the iron nucleus for
a diverse set of 44 molecular iron compounds. The contact electron
density is calculated with 12 density functionals across Jacob’s
ladder (PWLDA, BP86, BLYP, PW91, PBE, M06-L, TPSS, B3LYP, B3PW91,
PBE0, M06, TPSSh). We provide systematic-error diagnostics and reliable,
locally resolved uncertainties for isomer-shift predictions. Pure
and hybrid density functionals yield average prediction uncertainties
of 0.06–0.08 mm s–1 and 0.04–0.05
mm s–1, respectively, the latter being close to
the average experimental uncertainty of 0.02 mm s–1. Furthermore, we show that both model parameters and prediction
uncertainty depend significantly on the composition and number of
reference data points. Accordingly, we suggest that rankings of density
functionals based on performance measures (e.g., the squared coefficient
of correlation, r2, or the root-mean-square
error, RMSE) should not be inferred from a single data set. This study
presents the first statistically rigorous calibration analysis for
theoretical Mössbauer spectroscopy, which is of general applicability
for physicochemical property models and not restricted to isomer-shift
predictions. We provide the statistically meaningful reference data
set MIS39 and a new calibration of the isomer shift based on the PBE0
functional.