Behavior shapes retinal motion statistics during natural locomotion

Walking through an environment generates retinal motion, which humans rely on to perform a variety of visual tasks. Retinal motion patterns are determined by an interconnected set of factors, including gaze location, gaze stabilization, the structure of the environment, and the walker’s goals. The characteristics of these motion signals have important consequences for neural organization and behavior. However, to date, there are no empirical in situ measurements of how combined eye and body movements interact with real 3D environments to shape the statistics of retinal motion signals. Here, we collect measurements of the eyes, the body, and the 3D environment during locomotion. We describe properties of the resulting retinal motion patterns. We explain how these patterns are shaped by gaze location in the world, as well as by behavior, and how they may provide a template for the way motion sensitivity and receptive field properties vary across the visual field.

Methods
Raw data (not included in this dataset):

Pupil Labs Core eye tracker (World facing camera RGB video, infrared eye camera video)
Shadow Motion Capture System (IMU based motion capture data, sensor orientation, and relative position data)

Processed (included if indicated):

Eye tracker data processed with Pupil Capture software provides world facing camera relative to 3D gaze vectors
World-facing video processed with Meshroom to provide camera position and orientation, and 3D mesh terrain reconstruction
Custom MATLAB code used to align 3D gaze vector aligned to estimated camera position, provides approximated eyeball center, eye direction, relative to 3D mesh (gaze data included in dataset)
Custom Python code used with Blender to compute eye perspective depth images
Custom MATLAB is used to approximate retinal motion given depth image + eye translation + eye rotation measurement. (Retinal motion histograms included)