Cognitector2024

Attention monitoring via facial expressions has relied on unimodal datasets, which lack the context and task-related information for harnessing emotions, suffer from imbalance and exhibit annotation biases. By context-aware, emotions of individuals elicited during the data gathering, however, had no connection with any cognitive processing. Predominantly, the Dataset for Affective States in E-Environments (DAiSEE) dataset has been adopted as a common dataset for most studies on attention and engagement, though it has been employed in machine learning and deep learning related studies for engagement or attention monitoring. In this paper, we present the first facial-expression dataset explicitly designed for monitoring based on the proven cognitive experiment, the trail-making test(TMT)-A. TMT-A is a standard cognitive task that directly correlates with attention and executive function. Experiments with convolutional neural network-based and vision-transformer models were conducted on the new datasets, as the number of participants was only 43. There were two classes chosen as average and fast attention rates.  Comparison with other open-source datasets which are connected to attention monitoring was presented.

基于面部表情的注意力监测研究此前多依赖单模态数据集（unimodal datasets），此类数据集缺乏情绪分析所需的场景关联与任务相关信息，且存在样本不平衡与标注偏差问题。然而，即便采用上下文感知的数据采集策略，此前研究中采集到的个体诱发情绪，仍未与任何认知加工过程建立关联。目前，多数注意力与投入度相关研究普遍采用电子环境下情感状态数据集（Dataset for Affective States in E-Environments, DAiSEE）作为基准数据集，该数据集已被广泛应用于机器学习与深度学习领域的投入度或注意力监测相关研究中。本文提出了首个专为注意力监测设计的面部表情数据集，其构建基于经过验证的经典认知实验——连线测验A版（Trail-Making Test-A, TMT-A）。连线测验A版是一项与注意力及执行功能直接相关的标准认知任务。由于本次研究的参与者仅为43人，研究团队基于卷积神经网络（Convolutional Neural Network, CNN）与视觉Transformer（Vision Transformer）模型，对该新数据集开展了相关实验。本次数据集将注意力速率划分为平均速率与快速速率两个类别。此外，本文还将该数据集与其他面向注意力监测的公开开源数据集进行了对比分析。