Implementation of a Bayesian Optimization Framework for Interconnected Systems

Bayesian optimization (BO) is an effective paradigm for
the optimization
of expensive-to-sample systems. Standard BO learns the performance
of a system f(x) by using a Gaussian
Process (GP) model; this treats the system as a black box and limits
its ability to exploit available structural knowledge (e.g., physics
and sparse interconnections in a complex system). Gray-box modeling,
wherein the performance function is treated as a composition of known
and unknown intermediate functions f(x, y(x)) (where y(x) is a GP model), offers a solution to this limitation;
however, generating an analytical probability density for f from the Gaussian density of y(x) is often an intractable problem (e.g., when f is nonlinear). Previous work has handled this issue by using sampling
techniques or by solving an auxiliary problem over an augmented space
where the values of y(x) are constrained
by confidence intervals derived from the GP models; such solutions
are computationally intensive. In this work, we provide a detailed
implementation of a recently proposed gray-box BO paradigm, BOIS,
that uses adaptive linearizations of f to obtain
analytical expressions for the statistical moments of the composite
function. We show that the BOIS approach enables the exploitation
of structural knowledge, such as that arising in interconnected systems
as well as systems that embed multiple GP models and combinations
of physics and GP models. We benchmark the effectiveness of BOIS against
standard BO and existing gray-box BO algorithms using a pair of case
studies focused on chemical process optimization and design. Our results
indicate that BOIS performs as well as or better than existing gray-box
methods, while also being less computationally intensive.