A Multimodal-context Interruptibility Dataset for Proactive Services on Smart Speakers

Abstract As smart speakers become increasingly integrated into everyday home life, there is growing interest in leveraging thesespeakers to proactively deliver personalized proactive services. To enhance user experience and engagement of proactiveservices, a key challenge is identifying opportune moments, user contexts when users are mostly interruptible to engage inproactive services. Researchers have identified such moments by exploring interruptibility across various user contexts (inwhich contexts users are more interruptible), and several datasets have been released with interruptibility labels. However,no publicly available dataset has focused specifically on interruptibility within home environments. To fill this gap, we presentINPROSH, a dataset collected from 26 participants over a three-week in-the-wild field study. Participants used proactive servicesvia smart speakers in their home. INPROSH comprises 2,830 cases, each annotated with interruptibility labels and enriched withcontextual information, including temporal, spatial, and user-behavioral contexts, as well as surrounding image and soundrecordings near the smart speakers. We believe INPROSH will support deeper understanding and more accurate detection ofinterruptibility in home environments.