Bayesian Optimization Methods for Nonlinear Model Calibration

This work develops and compares seven Gaussian process
Bayesian
optimization (GPBO) methods for calibrating nonlinear models. We demonstrate
through ten (non)linear parameter estimation examples that new BO
methods using GP emulators of (computationally expensive) models accurately
recovered parameters in 67% of the benchmarking instances compared
to 28% for standard GPBO, which uses GP models for the loss objective.
When considering noisy or stochastic expensive models, emulator GPBO
finds the true parameters in 62% of the instances compared to approximately
0% for gradient-based nonlinear least-squares. We show that GPBO is
more efficient than other popular derivative-free search algorithms,
including genetic algorithms, the Nelder–Mead algorithm, or
the simplicial homology global optimization algorithm. We recommend
emulator GPBO with either an expected improvement Monte Carlo approximation
or an expected value of the sum of squared errors acquisition function.
Finally, we discuss future opportunities to improve these methods
and consider applications to expensive stochastic models (e.g., molecular
simulations).