Designing empirical experiments to compare interactive multiobjective optimization methods

Interactive multiobjective optimization methods operate iteratively so that a decision maker directs the solution process by providing preference information, and only solutions of interest are generated. These methods limit the amount of information considered in each iteration and support the decision maker in learning about the trade-offs. Many interactive methods have been developed, and they differ in technical aspects and the type of preference information used. Finding the most appropriate method for a problem to be solved is challenging, and supporting the selection is crucial. Published research lacks information on the conducted experiments’ specifics (e.g. questions asked), making it impossible to replicate them. We discuss the challenges of conducting experiments and offer realistic means to compare interactive methods. We propose a novel questionnaire and experimental design and, as proof of concept, apply them in comparing two methods. We also develop user interfaces for these methods and introduce a sustainability problem with multiple objectives. The proposed experimental setup is reusable, enabling further experiments.

交互式多目标优化方法（Interactive multiobjective optimization methods）以迭代方式运行，决策者通过提供偏好信息主导整个求解流程，仅生成决策者关注的可行解。此类方法会限制每次迭代中需考量的信息量，辅助决策者明晰各目标间的权衡关系。目前已涌现出多种交互式多目标优化方法，其技术实现细节与所采用的偏好信息类型均存在差异。为待求解问题遴选适配的方法颇具难度，因此辅助方法选择环节至关重要。已发表的相关研究往往缺失实验细节（如所使用的问卷问题），导致无法复现相关实验成果。本文探讨了开展此类实验面临的诸多挑战，并提出了可行的交互式方法对比方案。本文提出了一套全新的问卷设计与实验框架，并以概念验证为目标，将其应用于两种交互式优化方法的对比实验。此外，本文还为这两种方法开发了配套用户界面，并引入了一个多目标可持续性问题作为测试场景。本文提出的实验框架具备可复用性，可为后续相关研究提供实验基础。