SocialPulse: On-Device Detection of Social Interactions in Naturalistic Settings Using Smartwatch Multimodal Sensing

The dataset is associated with our paper, "SocialPulse: On-Device Detection of Social Interactions in Naturalistic Settings Using Smartwatch Multimodal Sensing", which is currently under review. This dataset contains log-mel spectrogram features extracted from audio, collected in real-time through our on-watch system. Audio Feature Dataset.zip After unzipping this file, there are multiple participant-level files with names such as "PB03__Smartwatch_AudioDatum.pkl", where PB03 denotes the participant ID. Each file follows this naming pattern and begins with the corresponding participant ID. Each pkl file contains two columns: T: the timestamp log melspectrogram: the log-mel spectrogram feature values Note: Zenodo may display some macOS-generated metadata files under the __MACOSX folder. These files are not part of the dataset and can be ignored in case it is found after downloading. In our check on a MacBook Pro, these files did not appear after downloading and unzipping "Audio Feature Dataset.zip". Labels.zip This archive contains a single pkl file with three columns: T, P_ID, and is_during_interaction?. T: This column contains the timestamp. To retrieve the corresponding audio feature, search within the audio feature file for the same participant ID in "Audio Feature Dataset.zip". Use the audio feature from the same participant file (from "Audio Feature Dataset.zip") that was recorded within 16 seconds of the timestamp in T. P_ID: This column indicates the participant ID. is_during_interaction?: This column contains the binary interaction label. A value of 0 indicates that no interaction occurred at that timestamp, while a value of 1 indicates that an interaction occurred. Data Collection Process For details about the system, audio recording process, and data collection procedures, please refer to our paper [link removed to maintain anonymity]. Codes All code used for preprocessing, analysis, labeling, and model development will be available here [link to be added before May 1]. NOTE Recipients of the dataset must not use the data, alone or together with any other information, to re-identify any participant. If a recipient inadvertently finds any identifiable information or identifies a participant, they must promptly notify PI [anonymous for review]. Contact If you have any questions about the dataset, you may contact [anonymous for review].