EarSet: A Multi-Modal In-Ear Dataset

EarSet aims at providing the research community with a novel, multi-modal, dataset, which, for the first time, will allow studying of the impact of body and head/face movements on both the morphology of the PPG wave captured at the ear, as well as on the vital signs estimation. To accurately collect in-ear PPG data, coupled with a 6 degrees-of-freedom (DoF) motion signature, we prototyped and built a flexible research platform for in-the-ear data collection. The platform is centered around a novel ear-tip design which includes a 3-channel PPG (green, red, infrared) and a 6-axis (accelerometer, gyroscope) motion sensor (IMU) co-located on the same ear-tip. This allows the simultaneous collection of spatially distant (i.e., one tip in the left and one in the right ear) PPG data at multiple wavelengths and the corresponding motion signature, for a total of 18 data streams. Inspired by the Facial Action Coding Systems (FACS), we consider a set of potential sources of motion artifact (MA) caused by natural facial and head movements. Specifically, we gather data on 16 different head and facial motions - head movements (nodding, shaking, tilting), eyes movements (vertical eyes movements, horizontal eyes movements, brow raiser, brow lowerer, right eye wink, left eye wink), and mouth movements (lip puller, chin raiser, mouth stretch, speaking, chewing). We also collect motion and PPG data under activities, of different intensities, which entail the movement of the entire body (walking and running). Together with in-ear PPG and IMU data, we collect several vital signs including, heart rate, heart rate variability, breathing rate, and raw ECG, from a medical-grade chest device. With approximately 17 hours of data from 30 participants of mixed gender and ethnicity (mean age: 28.9 years, standard deviation: 6.11 years), our dataset empowers the research community to analyze the morphological characteristics of in-ear PPG signals with respect to motion, device positioning (left ear, right ear), as well as a set of configuration parameters and their corresponding data quality/power consumption trade-off. We envision such a dataset could open the door to innovative filtering techniques to mitigate, and eventually eliminate, the impact of MA on in-ear PPG. We ran a set of preliminary analyses on the data, considering both handcrafted features, as well as a DNN (Deep Neural Network) approach. Ultimately, we observe statistically significant morphological differences in the PPG signal across different types of motions when compared to a situation where there is no motion. We also discuss a 3-classes classification task and show how full-body motions and head/face motions can be discriminated from a still baseline (and among themselves). These preliminary results represent the first step towards the detection of corrupted PPG segments and show the importance of studying how head/face movements impact PPG signals in the ear. To the best of our knowledge, this is the first in-ear PPG dataset that covers a wide range of full-body and head/facial motion artifacts. Being able to study the signal quality and motion artifacts under such circumstances will serve as a reference for future research in the field, acting as a stepping stone to fully enable PPG-equipped earables.

EarSet 旨在为研究社区提供一款新颖的多模态数据集，这也是首个可用于研究身体、头部/面部运动对耳部采集的光电容积脉搏波描记法（PPG）波形形态，以及生命体征估算影响的数据集。为精准采集入耳式PPG数据并搭配六自由度（DoF）运动特征，我们研发并搭建了一套灵活的入耳数据采集研究平台。该平台以创新的耳尖设计为核心，耳尖上集成了三通道PPG传感器（绿光、红光、红外光）以及六轴（加速度计、陀螺仪）运动传感器（惯性测量单元，IMU）。这使得我们能够同时采集多波长下、空间位置分离的（即左耳和右耳各配置一个耳尖）PPG数据及对应运动特征，总计包含18个数据流。
受面部动作编码系统（FACS）启发，我们梳理了由自然面部和头部运动引发的运动伪影（MA）潜在来源。具体而言，我们采集了16种不同的头部和面部运动数据：头部运动（点头、摇头、侧头）、眼部运动（垂直眼球运动、水平眼球运动、抬眉、降眉、右眼眨眼、左眼眨眼）以及口腔运动（抿唇、抬下巴、伸嘴、说话、咀嚼）。我们还采集了不同强度的全身运动（行走、跑步）场景下的运动及PPG数据。
除入耳式PPG和IMU数据外，我们还通过医用级胸部设备采集了多项生命体征数据，包括心率、心率变异性、呼吸频率以及原始心电图（ECG）。本数据集共包含来自30名不同性别、不同种族参与者的约17小时数据（平均年龄：28.9岁，标准差：6.11岁），可为研究人员分析入耳式PPG信号的形态特征提供支撑，分析维度涵盖运动、设备佩戴位置（左耳、右耳），以及一系列配置参数及其对应的数据质量与功耗之间的权衡关系。
我们期望该数据集能够为研发创新的滤波技术铺平道路，以缓解并最终消除运动伪影对入耳式PPG信号的影响。我们针对该数据集开展了一系列初步分析，既考虑了手工设计特征，也采用了深度神经网络（DNN）方法。最终我们观察到：与静止状态相比，不同类型运动下的PPG信号存在具有统计学意义的显著形态差异。我们还探讨了一项三分类任务，并展示了如何将全身运动与头部/面部运动从静止基线状态中区分出来（同时也能在这两类运动之间进行区分）。这些初步结果是实现受损PPG片段检测的第一步，也证明了研究头部/面部运动如何影响耳部PPG信号的重要性。
据我们所知，这是首个覆盖大范围全身及头部/面部运动伪影的入耳式PPG数据集。在上述场景下研究信号质量与运动伪影的影响，可为该领域的未来研究提供参考，成为全面赋能搭载PPG的耳戴式可穿戴设备的重要基石。

搜集汇总

背景与挑战

背景概述

EarSet是一个创新的多模态耳内数据集，首次提供了研究身体和头部/面部运动对PPG信号影响的全面数据。它包含来自30名参与者的约17小时数据，涵盖了多种运动类型和生命体征，为研究运动伪影对PPG信号的影响提供了宝贵资源。