Optimization Techniques for Inference and Parameter Recovery

Applications involving data often contain uncertainty due to imprecise measurements or prior numerical computations. Inverse problems in statistics and machine learning arise when one seeks to estimate underlying parameters of noisy outputs. This is often achieved through posterior distribution approximation for sample generation, which can be a challenging task when characterized by multiple modes. We present an adaptive annealing scheduler AdaAnn to facilitate this task, which automatically adjusts temperature increments based on expected change in the Kullback-Leibler divergence between two annealed distributions. Incorporating this technique with a surrogate model for computationally expensive model evaluations led to the development of LINFA, a Python Library for Inference with Normalizing Flow and Annealing. Additionally, inexact parameters can lead to uncertainty in the structure of a solution set when solving parameterized polynomial systems in numerical algebraic geometry. We present methodologies for robust interpretation regarding the number of finite solutions, existence of higher-dimensional components, and number of irreducible components, along with their multiplicities, by searching for nearby parameters on exceptional sets. This dissertation discusses several numerical examples, including applications in ordinary differential equations and kinematics.