Paradox-free analysis for comparing the performance of optimization algorithms

Numerical comparison serves as a major tool in evaluating the performance of optimization algorithms, especially nondeterministic algorithms, but existing methods may suffer from a ‘cycle ranking’ paradox and/or a ‘survival of the nonfittest paradox. This paper searches for paradox-free data analysis methods for numerical comparison. It is discovered that a class of sufficient conditions exist for designing paradox-free analysis. Rigorous modeling and deduction are applied to a class of profile methods employing a filter. It is thus further discovered and proven that algorithm-independent filter conditions can prevent cycle ranking and survival of non-fittest paradoxes from occurring. By adopting an algorithm-independent filter, popular profile methods such as the ‘modified data profile method’, ‘the accuracy profile method’, and ‘the operational characteristics zones method’ can be paradox free in comparing or bench-marking the performance of optimization algorithms.

数值比较作为评估优化算法性能的主要工具，尤其在非确定性算法的评估中占据重要地位，然而，现有的方法可能遭受‘循环排名悖论’及‘适者生存悖论’的困扰。本文致力于寻求无悖论的数据分析方法以进行数值比较。研究发现，存在一类充分条件可用于设计无悖论的分析方法。通过对一类采用滤波器的轮廓方法进行严谨的建模与推导，进一步发现并证明了算法无关的滤波条件能够防止循环排名和适者生存悖论的发生。通过采用算法无关的滤波条件，如‘改进的数据轮廓法’、‘准确性轮廓法’以及‘操作特性区域法’等流行轮廓方法，在比较或基准测试优化算法性能时，可以避免悖论的出现。