NVGaze (NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation)

训练数据集的质量，多样性和大小是基于学习的凝视估计器的关键因素。我们创建了两个满足这些标准的数据集，用于红外照明下的近眼注视估计: 使用解剖学信息的眼睛和面部模型的合成数据集，其面部形状，注视方向，瞳孔和虹膜，肤色和外部条件 (200万图像在1280x960)，以及与35个主题 (640x480的250万张图像) 一起收集的真实数据集。使用我们的数据集，我们训练了一个用于凝视估计的神经网络，在被排除在训练之外的真实对象的宽30 × 40度视场上实现2.06 (/- 0.44) 度的准确性，并且在针对一个真实对象明确训练时，0.5度最佳情况准确性 (跨同一视场)。我们还训练了网络的一种变体来执行瞳孔估计，显示出比以前的方法更高的鲁棒性。与以前的网络相比，我们的网络需要更少的卷积层，从而实现了亚毫秒的延迟。

The quality, diversity, and size of training datasets are critical factors for learning-based gaze estimators. We developed two datasets that meet these criteria for near-eye gaze estimation under infrared illumination: a synthetic dataset using anatomical eye and facial models, which covers facial shapes, gaze directions, pupils and irises, skin tones, and external conditions (2 million images at 1280×960 resolution), and a real-world dataset collected with 35 participants (2.5 million images at 640×480 resolution). Using our datasets, we trained a neural network for gaze estimation, which achieved an accuracy of 2.06 (±0.44) degrees on a wide 30°×40° field of view for real subjects excluded from the training set, and a best-case accuracy of 0.5 degrees when explicitly trained for a single real subject (across the same field of view). We also trained a variant of our network to perform pupil estimation, which demonstrated higher robustness than previous methods. Compared with prior networks, our network requires fewer convolutional layers, enabling sub-millisecond latency.

搜集汇总

背景与挑战

背景概述

NVGaze是一个用于低延迟、近眼凝视估计的数据集，包含基于解剖学信息的合成数据（200万张图像）和真实收集数据（来自35个对象的250万张图像）。该数据集旨在训练高效的神经网络，在真实对象上实现高精度凝视估计（最佳0.5度）和亚毫秒级延迟，适用于红外照明下的近眼应用。