EVE (End-to-end Video-based Eye-tracking)

仅从图像中估计视线是一项具有挑战性的任务，在很大程度上是由于无法观察到的特定于人的因素。实现高精度通常需要来自测试用户的标记数据，这在实际应用中可能无法实现。我们观察到，用户正在看的东西与用户眼睛的外观之间存在很强的关系。针对这种理解，我们提出了一种新颖的数据集和附带方法，旨在明确学习这些语义和时间关系。我们的视频数据集包括时间同步的屏幕记录，面向用户的摄像机视图和眼睛凝视数据，这允许在时间凝视跟踪以及无标签的凝视细化新基准。重要的是，我们证明了来自视觉刺激和眼睛图像的信息融合可以导致实现与通过监督个性化获得的文献报道的数字相似的性能。我们的最终方法在我们提出的EVE数据集上产生了显着的性能改进，注视点估计的改进高达28% (导致角度误差的2.49)，为纯粹从网络摄像头传感器实现基于屏幕的高精度眼睛跟踪铺平了道路。

Estimating gaze solely from images is a challenging task, largely attributable to unobservable person-specific factors. Achieving high accuracy typically requires labeled data from test users, which may not be feasible in real-world applications. We observed a strong correlation between what a user is looking at and the appearance of their eyes. Building on this insight, we propose a novel dataset and accompanying method designed to explicitly learn these semantic and temporal relationships. Our video dataset includes time-synchronized screen recordings, user-facing camera views, and eye gaze data, enabling new benchmarks for temporal gaze tracking and unlabeled gaze refinement. Importantly, we demonstrate that fusing information from visual stimuli and eye images can yield performance comparable to the figures reported in literature obtained via supervised personalization. Our final method achieves significant performance improvements on our proposed EVE dataset, with gaze estimation accuracy improved by up to 28% (resulting in an angular error of 2.49), paving the way for screen-based high-precision eye tracking purely from webcam sensors.

搜集汇总

背景与挑战

背景概述

EVE数据集由苏黎世联邦理工学院于2020年发布，包含时间同步的屏幕记录、用户摄像机视图和眼睛凝视数据，用于无标签视线估计研究。该数据集通过融合视觉刺激和眼睛图像信息，实现了高达28%的注视点估计性能改进，角度误差降至2.49度，为基于网络摄像头的高精度眼动跟踪提供了新基准。