Code and data underlying the paper: "Aligning object detector bounding boxes with human preference"

Previous work shows that humans tend to prefer large bounding boxes over small bounding boxes with the same IoU. However, we show here that commonly used object detectors predict large and small boxes equally often. In this work, we investigate how to align automatic detected object boxes with human preference and study whether this improves human quality perception. We evaluate the performance of three commonly used object detectors through a user study with more than 120 participants. We find that humans prefer object detections that are upscaled with factors of 1.5 or 2, even if the corresponding AP is close to 0. Motivated by this result, we propose an asymmetric bounding box regression loss that encourages large over small predicted bounding boxes. Our evaluation study shows that object detectors fine-tuned with the asymmetric loss are better aligned with human preference and are preferred over fixed scaling factors. In this repository, we provide our code, including the evaluation of object detection size and the implementation of the user studies. We share a Google drive link to the images used in our user studies.

已有研究表明，在交并比 (Intersection over Union, IoU) 相同的前提下，人类更偏好尺寸更大的边界框而非尺寸更小的边界框。然而本研究发现，当前主流的目标检测器 (object detector) 对大小边界框的预测频次并无显著差异。本研究旨在探索如何使自动检测得到的目标边界框与人类偏好对齐，并验证该对齐策略是否能够提升人类对检测结果的质量感知。我们通过覆盖120余名参与者的用户研究，对三款常用目标检测器的性能进行了评估。研究结果显示，即便对应的平均精度 (Average Precision, AP) 趋近于0，人类仍更青睐将检测框按1.5倍或2倍比例放大后的目标检测结果。基于上述发现，我们提出了一种非对称边界框回归损失函数，该函数会引导模型优先生成更大尺寸的预测边界框。后续评估实验表明，使用该非对称损失进行微调后的目标检测器，其检测结果与人类偏好的对齐程度更高，且相较于固定比例放大的检测框更受人类青睐。本代码仓库提供了完整的实验代码，包括目标检测框尺寸评估模块以及用户研究的相关实现。我们还公开了用户研究中所用图像的谷歌云端硬盘 (Google Drive) 共享链接。