NeuroEngage

This dataset contains raw fMRI data of human-human and human-robot bidirectional free conversations with varying levels of conversational engagement from a total of 50 participants (N HHI = 30, N HRI = 20). Each participant held three 10-minute conversations in Swedish with either a human confederate or a robot operated via a VR telepresence interface.

Each functional run contains 11 min 45 sec of recording with 1.2 sec TR. The functional run structure was as follows: fixation cross (15 sec) -> conversation (10 min) -> message to wrap up the discussion -> remaining of conversation (20 sec, not included in the rest of the data) -> blank screen (2 sec) -> fixation cross (15 sec).

Directory sourcedata contains multimodal behavioral data (eye tracking, segmented audio and video).

Details on the technical setup for replication can be found in the comments below. Details of the experiment are reported in: E. Torubarova, C. Arvidsson, J. Berrebi, J. Uddén, and A. Pereira, “Neuroengage: A multimodal dataset integrating fmri for analyzing conversational engagement in human-human and human-robot interactions,” in Proceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction (HRI), IEEE, 2025

Notes:
- sub-09, sub-10, sub-17, sub-37, sub-39 are excluded from the dataset due to various procedural issues.
- sub-16_run-03 contains 3 minutes of conversation
- some data types (audio or eye tracking) are missing for some of the participants. For details, see participants.tsv
- anatomical volumes were defaced using pydeface 2.0.2

[Preprocessed data and questionnaire results: to be uploaded]