基于轻量级网络的眼睛注视估计角膜反射的实时定位和匹配

眼部注视估计在人机交互中具有很大的潜力。文献中一种流行的眼部注视估计方案是使用一些红外（IR）灯光来照亮眼睛，同时一个IR摄像机捕捉图像。这个方案的关键步骤是定位由IR灯光照亮的角膜上的闪光点（称为角膜反射）并将其与相应的IR灯光匹配。然而，角膜反射通常很暗甚至不存在，再加上其他图像质量问题，注视估计系统的准确性和连续性可能会受到严重影响。为了解决上述问题，本文设计了一个新的注视估计硬件系统，并提出了一个轻量级的深度神经网络，用于实时定位和匹配角膜反射，这可以简单地部署在设计好的硬件系统中。受关键点检测的启发，这个网络可以同时定位角膜反射并将其与相应的IR灯光匹配。它还通过一个注意模块合并了定位瞳孔中心和定位角膜反射的两个任务。实验结果显示，与现有最先进的方法相比，所提议的网络在定位和匹配角膜反射方面取得了更好的性能。此外，我们设计的系统能够为实时应用提供准确且连续的注视估计。

Eye gaze estimation holds great potential in human-computer interaction (HCI). A popular eye gaze estimation scheme in the literature uses several infrared (IR) lights to illuminate the eyes, while an IR camera captures the images. The key step of this scheme is to locate the bright spots on the cornea illuminated by IR lights (called corneal reflections) and match them with their corresponding IR lights. However, corneal reflections are often dim or even absent, coupled with other image quality issues, which may severely impair the accuracy and continuity of gaze estimation systems. To address the aforementioned issues, this paper designs a novel gaze estimation hardware system and proposes a lightweight deep neural network for real-time localization and matching of corneal reflections, which can be easily deployed on the designed hardware system. Inspired by keypoint detection, this network can simultaneously locate corneal reflections and match them with their corresponding IR lights. It also integrates the two tasks of locating the pupil center and corneal reflections via an attention module. Experimental results show that compared with existing state-of-the-art methods, the proposed network achieves better performance in localizing and matching corneal reflections. Furthermore, the system we designed can provide accurate and continuous gaze estimation for real-time applications.

搜集汇总

背景与挑战

背景概述

该数据集专注于眼睛注视估计领域，提供了一个基于轻量级网络实现角膜反射实时定位和匹配的解决方案。数据集包含一个约2.1 GB的文件，发布于2023年，由中国科学院脑科学与智能技术卓越创新中心提供，旨在通过改进硬件系统和深度学习模型，提升注视估计的准确性和连续性，适用于实时人机交互应用。