The AFFECT-HRI data set: physiological data for affective computing in human-robot interaction with anthropomorphic service robots

We provide a comprehensive data set AFFECT-HRI containing physiological data labeled with human affect (i.e., mood and emotion) gathered during an empirical study consisting of a complex human-robot interaction (HRI). A realistic retail scenario served as an experimental environment. In prior research, we showed the necessity to combine the expertise of the research fields of psychology, computer science, and law in the design of a responsible human-centered HRI. Therefore, we implemented five conditions (neutral, transparency, liability, moral, and immoral) covering the perspectives from these three research fields and used two different anthropomorphic service robots. Our study followed a multi-method approach, resulting in a data set containing and combining objective physiological sensor data with subjective human-affect assessments. Additionally, the data set includes insights from 146 participants regarding affect, demographics, and socio-technical questionnaire ratings, as well as robot gestures and robot speech. Our study can be split into three scenes: a consultation regarding products, a request for sensitive personal information while opening a customer account, and a successful or failing handover when buying a mold remover. Thus, this data set offers for the first time the possibility to prove established or develop new emotion recognition methods and technological capabilities for HRI. Further, our data set provides the possibility to combine affective computing with research about robot behavior (gestures, speech, and handover), liability (questionnaire), transparency (questionnaire), and psychological aspects, allowing an encompassing, human-centered view of HRI.

The detailed data descriptor has been published in Nature Scientific Data. For more details on the data set, please check the paper below.

Please cite the following paper if the dataset is used in a publication:Heinisch, J.S., Kirchhoff, J., Busch, P. et al. Physiological data for affective computing in HRI with anthropomorphic service robots: the AFFECT-HRI data set. Sci Data 11, 333 (2024). https://doi.org/10.1038/s41597-024-03128-z

AcknowledgementsThis research was conducted as part of RoboTrust, a project of the Centre Responsible Digitality, supported by the Hessian Minister for Digital Strategy and Innovation. The authors would like to thank all participants for their participation in the study. We particularly want to thank Ruth Stock-Homburg for her support and for making Elenoide available. Further, we want to thank Mona Kegel, Vignesh Prasad, and all the research assistants who supported the study. We also thank the leap in time lab for serving as study location. A special thanks goes to Amer Altizini, who supported us by helping to prepare the data for publication. We want to thank Niklas Jungermann for his valuable comments on the statistical evaluation.