A Multimodal Dataset for Identifying Opportune Moments in Smart Home Environments

Abstract

As smart speakers become increasingly integrated into everyday life, there is growing interest in leveraging them to proactively deliver personalized services in smart home environments. In particular, delivering such proactive services at opportune moments, times when users are most receptive and willing to interact with provided service, can enhance user experience and engagement. However, delivering such services at inappropriate times can disturb users. Despite several studies on identifying opportune moments for proactive services, there is still a lack of publicly available datasets specifically designed to identify opportune moments in real-world home settings. To address this, we present a dataset, a real-world contextual dataset collected from 27 participants with diverse age groups over a three-week in-the-wild field study. The dataset was collected using a proactive service, SpeechMaster, implemented on commercial smart speakers. Our dataset consists of 2905 cases including contextual factors such as temporal, spatial, and behavioral contexts, and surrounding image and sound recordings. We believe that our dataset will advance the understanding and detection of opportune moments in smart home environments, grounded in rich contextual data collected in real-world scenarios. 

For those who need access to the image and sound data, please refer to the following link:https://gofile.me/7muuE/QBgcZ8zVE