FusionPhys: a multi-source rPPG method for estimating heart rate from facial videos

Remote photoplethysmography (rPPG) is a contactless technique for estimating heart rate from facial video sequences. By analyzing subtle color variations in the skin that are associated with blood volume changes, rPPG enables physiological monitoring without requiring wearable sensors or direct physical contact. However, despite its potential for remote health monitoring, classical signal processing methods, such as Green and POS, as well as deep learning approaches such as DeepPhys, remain sensitive to real-world challenges. Subject motion, illumination changes, facial pose variations, and reduced image quality can significantly affect the quality of the extracted physiological signal and reduce estimation accuracy.

To address these limitations, we propose FusionPhys, a multi-source architecture designed to improve the robustness of rPPG-based heart rate estimation. The proposed model exploits three complementary sources of information extracted from facial videos: raw facial appearance, motion variations between successive frames, and pulse-related signals estimated by classical rPPG methods. These streams provide different but complementary perspectives on the physiological information contained in the video. They are integrated through a neural attention-based fusion module, which learns to emphasize the most informative sources depending on the recording conditions.

The proposed approach is evaluated on the UBFC-rPPG dataset using four standard metrics: mean absolute error (MAE), root mean square error (RMSE), Pearson correlation coefficient, and signal-to-noise ratio (SNR). The results show consistent improvements over baseline methods, supporting the potential of multi-source fusion for improving the robustness, reliability, and deployability of rPPG systems in real-world scenarios.