Uday3114/mcd_rppg

---
license: cc-by-4.0
size_categories:
- 1K<n<10K
tags:
- medical
- video
- ecg
- ppg
task_categories:
- other
---

# MCD-rPPG: Multi-Camera Dataset for Remote Photoplethysmography

This repository contains the dataset from the paper ["Gaze into the Heart: A Multi-View Video Dataset for rPPG and Health Biomarkers Estimation"](https://dl.acm.org/doi/10.1145/3746027.3758255).

The MCD-rPPG dataset is available on the Hugging Face Hub: [**MCD-rPPG Dataset**](https://huggingface.co/datasets/kyegorov/mcd_rppg)

The presented large-scale multimodal MCD-rPPG dataset is designed for remote photoplethysmography (rPPG) and health biomarker estimation from video. The dataset includes synchronized video recordings from three cameras at different angles, PPG and ECG signals, and extended health metrics (arterial blood pressure, oxygen saturation, stress level, etc.) for 600 subjects in both resting and post-exercise states.

We also provide an efficient multi-task neural network model that estimates the pulse wave signal and other biomarkers from facial video in real-time, even on a CPU.

## The MCD-rPPG Dataset

The dataset contains:
* **3600 video recordings** (600 subjects × 2 states × 3 cameras)
* **Synchronized PPG** (100 Hz) and ECG signals
* **13 health biomarkers**: systolic/diastolic pressure, oxygen saturation, temperature, glucose, glycated hemoglobin, cholesterol, respiratory rate, arterial stiffness, stress level (PSM-25), age, sex, BMI.
* **Multi-view videos**: frontal webcam, FullHD camcorder, mobile phone camera.
*
## Fast Baseline Model

We propose an efficient multi-task model that:
* Processes video in **real-time on a CPU** (up to 13% faster than leading models).
* Estimates the **PPG signal** and **10+ health biomarkers** simultaneously.
* Is lightweight (~4 MB) and uses domain-specific preprocessing suitable for low-power devices.

The model architecture combines domain-specific preprocessing (ROI selection on the face) with a convolutional network (1D Feature Pyramid Network).

## Code and Sample Usage

See GitHub repository [https://github.com/ksyegorov/mcd_rppg](https://github.com/ksyegorov/mcd_rppg)

To get started with the code and reproduce experiments, follow these steps:

1. **Clone the repository:**
```bash
git clone https://github.com/ksyegorov/mcd_rppg.git
cd mcd_rppg/
```

2. **Install dependencies.** Using a virtual environment is recommended.
```bash
pip install -r requirements.txt
```

3. **Run the notebooks** you are interested in (e.g., `train_SCNN_8roi_mcd_rppg.ipynb`) for training or reproducing experiments. Remember to download the MCD-rPPG dataset first.

## Results and Comparison

The tables below show key results of our model (Ours) compared to state-of-the-art (SOTA) alternatives. MAE (Mean Absolute Error) is calculated for the PPG signal and Heart Rate (HR).

**Table: Model performance comparison (MAE) in cross-dataset scenarios**
*(Summary of results from the paper)*

| Model | ... | MCD-rPPG (HR MAE) | ... |
|----------------|-----|-------------------|-----|
| PBV | ... | 15.37 | ... |
| OMIT | ... | 4.78 | ... |
| POS | ... | 3.80 | ... |
| PhysFormer | ... | 4.08 | ... |
| **Ours** | ... | **4.86** | ... |

**Table: Performance for different camera views and inference speed**

| Model | CPU Inference (s) | Size (Mb) | Frontal PPG MAE | Side PPG MAE |
|----------------|-------------------|-----------|-----------------|--------------|
| POS | 0.26 | 0 | 0.87 | 1.25 |
| PhysFormer | 0.93 | 28.4 | 0.46 | 0.97 |
| **Ours** | **0.15** | **3.9** | 0.68 | 1.10 |

Complete results, including biomarker evaluation, are presented in the paper.

## Citation

If you use the MCD-rPPG dataset or code from this repository, please cite our work:

```bibtex
@inproceedings{10.1145/3746027.3758255,
author = {Egorov, Konstantin and Botman, Stepan and Blinov, Pavel and Zubkova, Galina and Ivaschenko, Anton and Kolsanov, Alexander and Savchenko, Andrey},
title = {Gaze into the Heart: A Multi-View Video Dataset for rPPG and Health Biomarkers Estimation},
year = {2025},
isbn = {9798400720352},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3746027.3758255},
doi = {10.1145/3746027.3758255},
abstract = {Progress in remote PhotoPlethysmoGraphy (rPPG) is limited by the critical issues of existing publicly available datasets: small size, privacy concerns with facial videos, and lack of diversity in conditions. The paper introduces a novel, comprehensive, large-scale multi-view video dataset for rPPG and health biomarkers estimation. Our dataset comprises 3600 synchronized video recordings from 600 subjects, captured under varied conditions (resting and post-exercise) using multiple consumer-grade cameras at different angles. To enable multimodal analysis of physiological states, each recording is paired with a 100 Hz PPG signal and extended health metrics, such as electrocardiogram, arterial blood pressure, biomarkers, temperature, oxygen saturation, respiratory rate, and stress level. Using this data, we train an efficient rPPG model and compare its quality with existing approaches in cross-dataset scenarios. The public release of our dataset and model should significantly speed up the progress in the development of AI medical assistants.},
booktitle = {Proceedings of the 33rd ACM International Conference on Multimedia},
pages = {13053–13059},
numpages = {7},
keywords = {biosignals, rppg, telemedicine, video},
location = {Dublin, Ireland},
series = {MM '25}
}
```

---
许可证：知识共享署名4.0（CC BY 4.0）协议
样本量范围：1000 < 样本量 < 10000
标签：
- 医疗
- 视频
- 心电图（Electrocardiogram, ECG）
- 光电容积描记法（Photoplethysmography, PPG）
任务类别：
- 其他
---

# MCD-rPPG：面向远程光电容积描记法的多摄像头数据集

本仓库包含论文《凝视心灵：面向远程光电容积描记法与健康生物标志物估计的多视角视频数据集》配套的数据集，论文链接：https://dl.acm.org/doi/10.1145/3746027.3758255

MCD-rPPG数据集已在Hugging Face Hub平台发布：[**MCD-rPPG 数据集**](https://huggingface.co/datasets/kyegorov/mcd_rppg)

本次发布的大规模多模态MCD-rPPG数据集专为基于视频的远程光电容积描记法（remote Photoplethysmography, rPPG）与健康生物标志物估计任务设计。该数据集包含600名受试者在静息状态与运动后状态下的多视角同步视频记录、PPG信号与ECG信号，以及扩展健康指标（动脉血压、血氧饱和度、应激水平等）。

本项目同时提供一款高效多任务神经网络模型，可基于面部视频实时估计脉搏波信号与其他生物标志物，甚至可在CPU上运行。

## MCD-rPPG 数据集

该数据集包含：
* **3600段视频记录**（600名受试者 × 2种状态 × 3个摄像头）
* **同步PPG信号（采样率100Hz）与ECG信号**
* **13项健康生物标志物**：收缩压/舒张压、血氧饱和度、体温、血糖、糖化血红蛋白、胆固醇、呼吸频率、动脉僵硬度、应激水平（PSM-25量表）、年龄、性别、身体质量指数（Body Mass Index, BMI）
* **多视角视频**：前置网络摄像头、全高清数码摄像机、智能手机摄像头。

## 高效基准模型

我们提出一款高效多任务模型，具备以下特性：
* 可在**CPU上实时处理视频**（比主流模型快最高13%）。
* 可同时估计**PPG信号**与**10项以上健康生物标志物**。
* 模型体积小巧（约4 MB），并采用适配低功耗设备的领域专属预处理流程。

该模型架构将领域专属预处理（面部感兴趣区域ROI选择）与卷积网络（一维特征金字塔网络，1D Feature Pyramid Network）相结合。

## 代码与示例使用方法

详见GitHub仓库：https://github.com/ksyegorov/mcd_rppg

若需使用代码并复现实验，请按照以下步骤操作：
1. **克隆仓库：**
bash
git clone https://github.com/ksyegorov/mcd_rppg.git
cd mcd_rppg/

2. **安装依赖项**，推荐使用虚拟环境进行配置：
bash
pip install -r requirements.txt

3. **运行所需的Jupyter Notebook**（例如`train_SCNN_8roi_mcd_rppg.ipynb`）以完成模型训练或实验复现，请务必提前下载MCD-rPPG数据集。

## 实验结果与对比

下表展示了本文提出模型（下称"Ours"）与当前主流（State-of-the-Art, SOTA）方法的关键对比结果。本实验以平均绝对误差（Mean Absolute Error, MAE）作为PPG信号与心率（Heart Rate, HR）的评估指标。

**表1：跨数据集场景下的模型性能对比（MAE）**
*(论文实验结果摘要)*

| 模型 | ... | MCD-rPPG（HR MAE） | ... |
|----------------|-----|-------------------|-----|
| PBV | ... | 15.37 | ... |
| OMIT | ... | 4.78 | ... |
| POS | ... | 3.80 | ... |
| PhysFormer | ... | 4.08 | ... |
| **Ours** | ... | **4.86** | ... |

**表2：不同摄像头视角下的模型性能与推理速度**

| 模型 | CPU推理耗时（s） | 模型体积（MB） | 前置视角PPG MAE | 侧面视角PPG MAE |
|----------------|-------------------|-----------|-----------------|--------------|
| POS | 0.26 | 0 | 0.87 | 1.25 |
| PhysFormer | 0.93 | 28.4 | 0.46 | 0.97 |
| **Ours** | **0.15** | **3.9** | 0.68 | 1.10 |

完整实验结果（含生物标志物评估）详见论文原文。

## 引用声明

若您使用MCD-rPPG数据集或本仓库中的代码，请引用本文成果：

bibtex
@inproceedings{10.1145/3746027.3758255,
author = {Egorov, Konstantin and Botman, Stepan and Blinov, Pavel and Zubkova, Galina and Ivaschenko, Anton and Kolsanov, Alexander and Savchenko, Andrey},
title = {Gaze into the Heart: A Multi-View Video Dataset for rPPG and Health Biomarkers Estimation},
year = {2025},
isbn = {9798400720352},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3746027.3758255},
doi = {10.1145/3746027.3758255},
abstract = {当前公开的远程光电容积描记法（remote Photoplethysmography, rPPG）相关数据集存在样本量偏小、面部视频涉及隐私问题、场景多样性不足等关键缺陷，限制了该领域的研究进展。本文提出一款全新的大规模多视角视频数据集，用于rPPG与健康生物标志物估计任务。本数据集包含600名受试者在不同条件下（静息状态与运动后状态）通过多台消费级多角度摄像头采集的3600段同步视频记录。为支持生理状态的多模态分析，每段视频均同步搭配100Hz采样率的PPG信号与扩展健康指标，包括心电图、动脉血压、各类生物标志物、体温、血氧饱和度、呼吸频率与应激水平。基于该数据集，我们训练了一款高效rPPG模型，并在跨数据集场景下与现有方法进行性能对比。本数据集与模型的公开发布将显著推动AI医疗助手领域的研发进度。},
booktitle = {Proceedings of the 33rd ACM International Conference on Multimedia},
pages = {13053–13059},
numpages = {7},
keywords = {生物信号, rPPG, 远程医疗, 视频},
location = {Dublin, Ireland},
series = {MM '25}
}