COVEE: A Dataset for Cognitive Modeling with Video, Electroencephalography, and Eye Tracker

We present COVEE, a dataset for Cognitive mOdeling with Video, Electroencephalography, and Eye tracker. COVEE aims to be a valuable resource for advancing research on human cognition and the real-time estimation of cognitive load (CL) and situation awareness (SA) in complex operational environments. COVEE subsumes around 26 hours of multimodal data collected in carefully designed human subject experiments in realistic task settings, along with self-reported measures for CL and SA. In total, 23 human subjects performed multiple tasks with varying complexity in their role as remote operators of unmanned aerial vehicles. To establish baselines, extensive experiments are conducted using various modern deep learning architectures and pre-trained large vision models. Insights gained from the experiments show the potential of COVEE for future research and real-world applications related to brain-computer interfaces, assisted technologies, or other forms of real-time behavioral intervention. Both raw and processed experiment data, along with all codes, are published for the highest reproducibility (https:\/\/github.com\/Purdue-AIDA3\/CM-COVEE).