Designing empirical experiments to compare interactive multiobjective optimization methods

Interactive multiobjective optimization methods operate iteratively so that a decision maker directs the solution process by providing preference information, and only solutions of interest are generated. These methods limit the amount of information considered in each iteration and support the decision maker in learning about the trade-offs. Many interactive methods have been developed, and they differ in technical aspects and the type of preference information used. Finding the most appropriate method for a problem to be solved is challenging, and supporting the selection is crucial. Published research lacks information on the conducted experiments’ specifics (e.g. questions asked), making it impossible to replicate them. We discuss the challenges of conducting experiments and offer realistic means to compare interactive methods. We propose a novel questionnaire and experimental design and, as proof of concept, apply them in comparing two methods. We also develop user interfaces for these methods and introduce a sustainability problem with multiple objectives. The proposed experimental setup is reusable, enabling further experiments.

交互式多目标优化方法（Interactive Multiobjective Optimization Method）以迭代方式运行，决策者可通过提供偏好信息主导求解流程，仅生成决策者感兴趣的解。此类方法会限制每次迭代中处理的信息量，辅助决策者理解各目标间的权衡关系。目前已提出多种交互式优化方法，它们在技术细节与所采用的偏好信息类型上存在差异。为待求解问题挑选最合适的方法颇具挑战性，因此为方法选择提供支持至关重要。已发表的相关研究往往缺失已开展实验的具体细节（例如实验中询问的问题），导致无法复现相关实验。本文探讨了开展此类实验所面临的挑战，并提供了可用于对比交互式多目标优化方法的实用方案。本文提出了一种全新的调查问卷与实验设计方案，并以概念验证（Proof of Concept）为目的，将其应用于两种交互式优化方法的对比实验中。此外，本文还为这两种方法开发了用户界面（User Interface，UI），并引入了一个多目标可持续性问题。所提出的实验框架具备可复用性，可支持后续相关实验的开展。