Robust In-Vehicle Heartbeat Detection Using Multimodal Signal Fusion

Continuous in-vehicle health monitoring enables the earlier detection of diseases. Therefore, we developed a redundant sensor system for heartbeat measurement, which is composed of an electrocardiogram (ECG), and photoplethysmogram (PPG), and a camera. The signals are recorded in the different driving scenarios: i. rest (5 min), ii. city (15 min), iii. highway (15 min), and vi. rural (15 min). We recorded the ECG, PPG, RGB video, and reference ECG of 19 subjects in a research car (Tiguan, Volkswagen AG, Wolfsburg, Germany). To extract the cheek face segment, we integrated a face recognition engine, which detects in real-time the position of face landmarks. From this face segment, we extracted the RGB channel of the RGB video. In summary, the recorded data provides information about the usable time for medical analysis during driving.