Improving Legacy Optimization Systems with Benchmarking

Software applications using MIP optimization to solve business problems are typical in industrial settings. Despite the prevalence and importance of these systems, there is limited research on how to improve the performance of legacy systems instead of designing new systems. In practice, the costs and risks of designing new systems can outweigh the potential benefits of implementing a new research technique. We propose a novel framework for improving the performance of legacy MIP systems without altering the model. In this research, we proposed a novel framework to perform benchmarking and compare MIP solvers. We evaluated 3 commercial solvers, and 3 open-source solvers with default configuration. We used the MIPLIB 2017 benchmark collection to test 240 instances and a real-world MIP model for pilots scheduling at NetJets. In total, we ran 1,820 experiments in batches parallelly for each solver. All our experiments would take up to 89 days to get done, but using OSC Cardinal cluster resources, we were able to get them in less than 3 days. To continue our experiments, we are working in a new experiment for parameter tuning of solvers; with this approach, we will need to do around 90,000 new experiments, which would take up to 5 years to get all the results, but by parallelizing jobs using Supercomputer, we estimate we can get all the results in 4 days.