Multimodal-Human-Robot-Teaching-Dataset

Summary: This dataset was collected as part of the CISC (Collaborative Intelligence for Safety-Critical Systems) project under the Marie Skłodowska-Curie ITN programme. The data supports research in human-machine interaction and programming by demonstration in safety-critical environments.

It contains multimodal recordings, including eye-tracking data, human posture information (captured via a vision-based system), robot trajectory data, and questionnaire responses related to usability and cognitive load.

The dataset was collected through experiments involving human participants performing teaching tasks with a collaborative robot. All data has been anonymized in line with ethical approvals and is suitable for analysis in human factors, ergonomics, computer vision, and AI feedback systems.

This dataset can be used for academic research in machine learning, human-robot collaboration, and system evaluation studies.

Please cite this dataset if used in publications. Funding for this work was provided by the EU Horizon 2020 programme under Grant Agreement No.955901.

Multimodal Human-Robot Teaching Dataset: This dataset captures multimodal interaction data collected during a study on teaching robots through human demonstration. The data includes eye tracking, motion tracking, robot trajectory, and subjective evaluations from participants performing structured teaching tasks under different feedback conditions.

Experiment Overview: Participants(N=28) were asked to perform Pick-and-Place and Sliding tasks using a teaching-by-demonstration interface in a controlled experimental setup. Each participant experienced one of four feedback conditions: No Feedback-NF (Control) Visual Feedback-VF Descrptive Feedback-DF Mixed Feedback-MF Data was collected across multiple modalities to investigate how humans learn and teach robots, with an emphasis on usability, attention, and ergonomic performance.

participants.csv: Participant demographics (age, gender) subjective_responses.csv: NASA-TLX and SUS questionnaire responses per condition and task EyeTracking/: Gaze data and fixation metrics (participant → condition → task) MotionTracking/: Motion capture data

Use Case: This dataset can support research in: Human-robot interaction Cognitive load estimation Programming-by-demonstration systems Eye tracking analysis Multimodal machine learning

License: This dataset is shared under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Citation If you use this dataset, please cite our related data paper (DOI to be added soon).