Multi-Modal User Modeling for Task Guidance (MUTMG): A Dataset for Real-Time Assistance with Stress and Interruption Dynamics
收藏资源简介：
We introduce the Multi-Modal User Modeling for Task Guidance (MUMTG) to support the development of AI models for such purposes. The dataset is created through human-subjects studies with users performing search and assembly procedures with help from virtual instructions provided by either a head-worn AR headset or a laptop screen. The data-collection study uses a game-like scenario to guide participants through six guided tasks that vary in difficulty. Within each group, we manipulated task duration and induced several stress triggers to increase the task cognitive demand for three tasks. The dataset includes physiological data such as electrodermal activity, temperature, heart rate, pupil dilation, and gaze. We also collected subjective self-report ratings regarding task workload and emotional responses after each task. We offer this rich dataset as a valuable resource to facilitate the development of user models for task guidance in highly demanding contexts.
我们提出了面向任务指导的多模态用户建模（Multi-Modal User Modeling for Task Guidance, MUMTG）数据集，旨在支撑此类AI模型的开发工作。该数据集通过人类受试者研究构建而成：受试者可借助头戴式增强现实（Augmented Reality, AR）头显或笔记本屏幕提供的虚拟指导，完成搜索与装配操作流程。本次数据采集研究采用类游戏化场景，引导受试者完成六项难度各异的指导性任务。在每组任务中，我们通过调整任务时长并设置多种应激触发因素，提升了其中三项任务的认知负荷水平。该数据集包含皮肤电活动、体温、心率、瞳孔扩张及注视数据等生理数据。此外，我们还在每项任务结束后，收集了受试者关于任务负荷与情绪反应的主观自评评分。我们公开该丰富数据集作为宝贵资源，以助力高负荷场景下面向任务指导的用户建模研究与开发工作。
