EarSet: A Multi-Modal In-Ear Dataset

EarSet aims at providing the research community with a novel, multi-modal, dataset, which, for the first time, will allow studying of the impact of body and head/face movements on both the morphology of the PPG wave captured at the ear, as well as on the vital signs estimation. To accurately collect in-ear PPG data, coupled with a 6 degrees-of-freedom (DoF) motion signature, we prototyped and built a flexible research platform for in-the-ear data collection. The platform is centered around a novel ear-tip design which includes a 3-channel PPG (green, red, infrared) and a 6-axis (accelerometer, gyroscope) motion sensor (IMU) co-located on the same ear-tip. This allows the simultaneous collection of spatially distant (i.e., one tip in the left and one in the right ear) PPG data at multiple wavelengths and the corresponding motion signature, for a total of 18 data streams. 
Inspired by the Facial Action Coding Systems (FACS), we consider a set of potential sources of motion artifact (MA) caused by natural facial and head movements.  Specifically, we gather data on 16 different head and facial motions - head movements (nodding, shaking, tilting), eyes movements (vertical eyes movements, horizontal eyes movements, brow raiser, brow lowerer, right eye wink, left eye wink), and mouth movements (lip puller, chin raiser, mouth stretch, speaking, chewing).
We also collect motion and PPG data under activities, of different intensities, which entail the movement of the entire body (walking and running). Together with in-ear PPG and IMU data, we collect several vital signs including, heart rate, heart rate variability, breathing rate, and raw ECG, from a medical-grade chest device.

With approximately 17 hours of data from 30 participants of mixed gender and ethnicity (mean age: 28.9 years, standard deviation: 6.11 years), our dataset empowers the research community to analyze the morphological characteristics of in-ear PPG signals with respect to motion, device positioning (left ear, right ear), as well as a set of configuration parameters and their corresponding data quality/power consumption trade-off. We envision such a dataset could open the door to innovative filtering techniques to mitigate, and eventually eliminate, the impact of MA on in-ear PPG. We ran a set of preliminary analyses on the data, considering both handcrafted features, as well as a DNN (Deep Neural Network) approach. Ultimately, we observe statistically significant morphological differences in the PPG signal across different types of motions when compared to a situation where there is no motion. We also discuss a 3-classes classification task and show how full-body motions and head/face motions can be discriminated from a still baseline (and among themselves). These preliminary results represent the first step towards the detection of corrupted PPG segments and show the importance of studying how head/face movements impact PPG signals in the ear. 

To the best of our knowledge, this is the first in-ear PPG dataset that covers a wide range of full-body and head/facial motion artifacts. Being able to study the signal quality and motion artifacts under such circumstances will serve as a reference for future research in the field, acting as a stepping stone to fully enable PPG-equipped earables.

EarSet 旨在为科研社群提供一款新颖的多模态数据集，这也是首个可用于研究身体、头部/面部运动对耳部采集的光电容积脉搏波（PPG，Photoplethysmography）波形形态，以及生命体征估算两方面影响的数据集。为精准采集入耳式PPG数据并同步获取六自由度（6DoF）运动特征，我们研发并搭建了一套用于入耳式数据采集的灵活研究平台。该平台以创新的耳尖设计为核心，集成了三通道PPG传感器（绿光、红光、红外光）以及六轴（加速度计、陀螺仪）运动传感器（惯性测量单元（IMU，Inertial Measurement Unit）），二者共置于同一耳尖之上。这一设计支持同时采集空间上分离的（即左耳与右耳各一个耳尖）多波长PPG数据及其对应的运动特征，总计涵盖18个数据流。
受面部动作编码系统（FACS，Facial Action Coding Systems）启发，我们梳理了由自然面部与头部运动引发的各类潜在运动伪影（MA，Motion Artifact）来源。具体而言，我们采集了16种不同的头部与面部运动数据：头部运动（点头、摇头、偏头）、眼部运动（垂直眼球运动、水平眼球运动、抬眉、降眉、右眼眨眼、左眼眨眼）以及口部运动（拉唇、抬下巴、唇部拉伸、讲话、咀嚼）。
我们还采集了不同强度的全身运动（行走、跑步）场景下的运动与PPG数据。除入耳式PPG与IMU数据外，我们还通过医疗级胸部设备采集了多项生命体征数据，包括心率、心率变异性、呼吸频率以及原始心电图（ECG，Electrocardiogram）。
本数据集共收录30名不同性别、种族参与者的约17小时数据，参与者平均年龄为28.9岁，标准差为6.11岁。该数据集可助力科研社群分析入耳式PPG信号的形态特征，分析维度涵盖运动、设备放置位置（左耳、右耳），以及一系列配置参数及其对应的数据质量与功耗间的权衡关系。我们期望该数据集能够为开发创新性滤波技术以缓解并最终消除运动伪影对入耳式PPG的影响提供支撑。我们针对数据集开展了一系列初步分析，既考虑了人工设计特征，也采用了深度神经网络（DNN，Deep Neural Network）方法。最终我们观察到，相较于无运动的基准场景，不同运动类型下的PPG信号存在统计学意义上的显著形态差异。我们还探讨了一个三类分类任务，并展示了如何将全身运动与头部/面部运动从静止基准状态中区分出来（且各类运动之间也可相互区分）。这些初步结果是检测受损PPG片段的第一步，也证实了研究头部/面部运动如何影响耳部PPG信号的重要性。
据我们所知，这是首个涵盖大范围全身及头部/面部运动伪影的入耳式PPG数据集。能够在上述场景下研究信号质量与运动伪影，将为该领域的未来研究提供参考，成为全面普及搭载PPG技术的耳部可穿戴设备的重要基石。