EyeGuide - From Gaze Data to Instance Segmentation

Obtaining precise instance-level segmentations is a challenging task in machine learning. Especially for objects with complex and non-convex geometries or with partial occlusions scribble, bounding boxes or user clicks are often provided to guide the segmentation. In the paper "EyeGuide - From Gaze Data to Instance Segmentation", we explore the usage of a remote eye tracking system to generate gaze data as an additional input for object segmentation models. The gaze data is recorded during routine image inspections (i.e. without giving a particular task) and is used as an additional input to train neural networks. The training and evaluation was done on the PascalVOC2012 train and a part of the Cellpose dataset. The corresponding raw gaze data are published here.