pith. sign in

arxiv: 2606.21511 · v1 · pith:CTSWBFNKnew · submitted 2026-06-19 · 📡 eess.IV · cs.CV

A Skin-Tone-Aware Dual-Representation Remote Photoplethysmography Framework for Contactless Respiratory Rate Estimation

Trishna Saikia , Anup Kumar Gupta , Puneet Gupta , Pasi Liljeberg This is my paper

Pith reviewed 2026-06-26 12:35 UTC · model grok-4.3

classification 📡 eess.IV cs.CV
keywords remote photoplethysmographyrespiratory rate estimationskin-tone awarenessEulerian representationLagrangian representationcontactless monitoringcontrastive lossRR-rPPG dataset
0
0 comments X

The pith

A skin-tone-aware dual-representation rPPG framework estimates respiratory rate from facial videos with up to 42.1 percent lower error than prior methods.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that existing rPPG techniques designed for heart rate can be adapted and improved specifically for respiratory rate by adding skin-tone awareness to RGB projections, a denoising network for motion signals, and a contrastive loss that aligns two video representations without depending on signal phase. This would matter if true because respiratory rate indicates pulmonary and cardiovascular health yet most measurement methods require physical contact, while video-based remote methods could enable easier monitoring. The authors introduce a new dataset called RR-rPPG with Indian demographic representation to test the approach. Evaluation on this dataset and COHFACE shows the framework consistently beats comparison methods. The work also plans to release code and data to support further remote respiratory monitoring research.

Core claim

The authors claim that a skin-tone-aware dynamic RGB signal projection captures respiratory information in the Eulerian representation, a denoising network mitigates non-respiratory motion sensitivity in the Lagrangian representation, and a phase-independent contrastive loss enables the two representations to collaboratively learn respiratory rate information, resulting in consistent outperformance of comparison methods and up to a 42.1% reduction in mean absolute error on the RR-rPPG and COHFACE datasets. The framework shows the value of jointly using these representations, while the new RR-rPPG dataset supplies a diverse benchmark resource for remote respiratory monitoring.

What carries the argument

Skin-tone-aware dynamic RGB signal projection combined with phase-independent contrastive loss that lets Eulerian and denoised Lagrangian representations collaborate on respiratory signals.

If this is right

  • Respiratory rate estimation accuracy improves when rPPG methods incorporate skin-tone awareness and dual representations instead of relying on fixed heart-rate projections.
  • The denoising network reduces the impact of motion artifacts on Lagrangian signals used for respiration.
  • The phase-independent contrastive loss allows Eulerian and Lagrangian views to reinforce each other for respiratory information.
  • RR-rPPG supplies a public benchmark dataset with Indian demographic representation for contactless respiratory monitoring research.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method could support camera-based respiratory monitoring in consumer devices for populations with varied skin tones.
  • Similar dual-representation designs might apply to other contactless vital-sign tasks such as blood oxygen estimation.
  • Real-world deployment would require testing under uncontrolled lighting and larger head movements to confirm the components remain effective.

Load-bearing premise

The skin-tone-aware dynamic projection and phase-independent contrastive loss isolate respiratory signals from facial videos rather than fitting to dataset-specific noise or motion patterns.

What would settle it

An ablation study on an independent dataset with simultaneous ground-truth respiration from a chest belt sensor that measures whether removing the skin-tone projection or the contrastive loss eliminates the reported error reduction.

Figures

Figures reproduced from arXiv: 2606.21511 by Anup Kumar Gupta, Pasi Liljeberg, Puneet Gupta, Trishna Saikia.

Figure 1
Figure 1. Figure 1: Overview of the proposed method, ELITE-RR. It combines two complementary representations: Eulerian and Lagrangian. In the Eulerian branch (top), the facial ROI is divided into patches. Each patch’s skin-tone-aware features guide a projection network (𝑝 ) to transform RGB signals into reliable physiological signals. In the Lagrangian branch (bottom), vertical displacements of facial landmarks are tracked t… view at source ↗
Figure 2
Figure 2. Figure 2: Architecture of the denoising network 𝑑 . The input Lagrangian signal is first perturbed with a small Gaussian noise term, 𝑠𝑙 = 𝑠𝑙 +(0, 𝜖2 ), to improve robustness during training. The encoder consists of four 1D convolutional layers followed by a fully connected projection to a 32-dimensional latent representation. The decoder reconstructs the denoised signal through a fully connected expansion and four… view at source ↗
Figure 4
Figure 4. Figure 4: Representative samples from the proposed RR-rPPG dataset. Each column presents a subject’s facial frame (top) together with the corresponding ground-truth respiration signal (blue) and pulse signal (red) (bottom). The subject(s) in the figure gave permission and consent for the use of their identifiable images [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Fitzpatrick Skin-Tone Distribution in the (a) RR￾rPPG and (b) COHFACE Datasets. 4. Experimental Analysis 4.1. Dataset Description Proposed Dataset: We introduce the RR-rPPG dataset, com￾prising 119 facial video recordings from participants aged between 18 and 40 years, including 26 females and 93 males. Alongside each video, corresponding ground truth respira￾tory and pulse signals were collected as shown … view at source ↗
Figure 6
Figure 6. Figure 6: Performance of the proposed method across different video clip lengths, with optimal results observed at 12 seconds [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Bland–Altman plot of the proposed ELITE-RR method on COHFACE + RR-rPPG dataset. The solid line represents the mean difference, while the dashed lines denote the limits of agreement. across a broad range of RR. In addition, a large proportion of the samples exhibit relatively small estimation errors, with approximately 81% having an absolute error less than or equal to 5 BrPM, and all samples falling within… view at source ↗
Figure 8
Figure 8. Figure 8: MAE across Fitzpatrick skin-tone scales for different training and testing dataset combinations (“X → Y” denotes that the model was trained on dataset X and tested on dataset Y). certain training conditions, primarily due to the limited number of samples in these groups, as shown in [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
read the original abstract

Respiratory rate is a vital indicator of pulmonary and cardiovascular health, yet conventional methods for estimating respiratory rate are often intrusive due to their contact-based nature. Remote photoplethysmography offers a promising non-contact alternative and has been widely used for heart rate estimation; however, its potential for respiratory rate estimation remains underexplored. Existing methods typically adapt green and chrominance-based projections originally designed for heart rate estimation, which only partially capture respiratory dynamics. Most prior work focuses on the Eulerian representation with fixed or empirically selected RGB projections. To address these gaps, we propose a skin-tone-aware dynamic RGB signal projection that captures respiratory information. To mitigate the sensitivity of the Lagrangian representation to non-respiratory motion, we introduce a denoising network for motion-based remote photoplethysmography signals. We further design a phase-independent contrastive loss that enables Eulerian and Lagrangian representations to collaboratively learn respiratory rate information. We also introduce RR-rPPG, a respiratory-rate facial video dataset with Indian demographic representation. We evaluate the method on RR-rPPG and the publicly available COHFACE dataset, where it consistently outperforms comparison methods and achieves up to a 42.1% reduction in mean absolute error across the evaluated settings. The proposed framework demonstrates the effectiveness of jointly leveraging skin-tone-aware Eulerian and denoised Lagrangian representations for contactless respiratory rate estimation from facial videos. In addition, RR-rPPG contributes a diverse benchmark resource for future research in remote respiratory monitoring. The code and dataset will be made publicly available upon paper acceptance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces a skin-tone-aware dual-representation rPPG framework for contactless respiratory rate (RR) estimation from facial videos. Key contributions include a dynamic RGB projection that accounts for skin tone, a denoising network to stabilize Lagrangian motion-based signals, a phase-independent contrastive loss to align Eulerian and Lagrangian representations, and the new RR-rPPG dataset with Indian demographic diversity. The method is evaluated on RR-rPPG and COHFACE, reporting consistent outperformance over baselines with up to 42.1% MAE reduction; code and dataset are to be released publicly.

Significance. If the empirical results hold under rigorous validation, the work advances inclusive remote respiratory monitoring by explicitly addressing skin-tone bias and fusing complementary Eulerian/Lagrangian signals. The new RR-rPPG dataset supplies needed demographic diversity, and the public release of code and data constitutes a concrete community resource. These elements strengthen the paper's potential impact beyond incremental method tweaks.

major comments (2)
  1. [§4 and §5] §4 (Proposed Method) and §5 (Experiments): the central performance claim (up to 42.1% MAE reduction) rests on the joint contribution of the skin-tone-aware projection, denoising network, and contrastive loss, yet no ablation results isolating each component are described. Without these, it remains possible that gains derive primarily from the new dataset rather than the proposed modules.
  2. [§5.2] §5.2 (Evaluation on COHFACE and RR-rPPG): the manuscript reports MAE improvements but supplies no error bars, cross-validation details, or statistical significance tests (e.g., paired t-tests or Wilcoxon tests) for the differences versus baselines. This weakens the assertion of consistent outperformance across settings.
minor comments (2)
  1. [Abstract] Abstract: the quantitative claim of 42.1% MAE reduction should be accompanied by the number of subjects, exact baselines, and evaluation protocol to allow immediate assessment of scope.
  2. [§3] Notation throughout: ensure consistent capitalization and abbreviation of 'Eulerian' and 'Lagrangian' representations; minor inconsistencies appear in the method description.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback and positive recommendation for minor revision. We address the major comments below and will incorporate the necessary additions in the revised manuscript.

read point-by-point responses

  1. Referee: [§4 and §5] §4 (Proposed Method) and §5 (Experiments): the central performance claim (up to 42.1% MAE reduction) rests on the joint contribution of the skin-tone-aware projection, denoising network, and contrastive loss, yet no ablation results isolating each component are described. Without these, it remains possible that gains derive primarily from the new dataset rather than the proposed modules.

    Authors: We agree that ablation studies are important to isolate the contributions of each proposed component. In the revised manuscript, we will add detailed ablation experiments in §5 that evaluate the impact of the skin-tone-aware dynamic RGB projection, the denoising network, and the phase-independent contrastive loss individually and in combination. These will include comparisons to a baseline using only the new RR-rPPG dataset without the proposed modules to demonstrate that the performance improvements are attributable to the framework rather than the dataset alone. revision: yes

  2. Referee: [§5.2] §5.2 (Evaluation on COHFACE and RR-rPPG): the manuscript reports MAE improvements but supplies no error bars, cross-validation details, or statistical significance tests (e.g., paired t-tests or Wilcoxon tests) for the differences versus baselines. This weakens the assertion of consistent outperformance across settings.

    Authors: We acknowledge the need for more rigorous statistical reporting. In the revision, we will include error bars (e.g., standard deviations across cross-validation folds or subjects) in the tables and figures of §5.2. Additionally, we will provide details on the cross-validation strategy and perform statistical significance tests, such as paired t-tests or Wilcoxon signed-rank tests, to confirm that the observed MAE reductions are statistically significant compared to the baselines. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper's central claims rest on empirical outperformance of a proposed skin-tone-aware dynamic projection, denoising network, and phase-independent contrastive loss, evaluated on a newly collected RR-rPPG dataset plus the public COHFACE set. No equations, fitted parameters, or self-citations are shown to reduce the reported MAE reductions to quantities defined by construction from the same inputs. The derivation chain is self-contained via standard supervised learning components and new data collection rather than self-definitional mappings or load-bearing self-citations.

Axiom & Free-Parameter Ledger

3 free parameters · 1 axioms · 0 invented entities

The central claim rests on the unverified effectiveness of the proposed neural components and the assumption that the new dataset captures representative respiratory dynamics; multiple network weights are fitted to data with no parameter count or regularization details supplied.

free parameters (3)
  • dynamic RGB projection parameters
    Weights or selection rules for skin-tone-aware color projection are learned or tuned from data.
  • denoising network weights
    Parameters of the motion-denoising network are fitted during training.
  • contrastive loss hyperparameters
    Temperature or margin parameters in the phase-independent contrastive loss are chosen or fitted.
axioms (1)
  • domain assumption Standard neural-network training converges to representations that isolate respiratory frequency from motion and illumination noise
    Invoked by the introduction of the denoising network and contrastive loss without further justification in the abstract.

pith-pipeline@v0.9.1-grok · 5830 in / 1252 out tokens · 25349 ms · 2026-06-26T12:35:52.447709+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 1 linked inside Pith

  1. [1]

    JongSongRyu,SunCholHong,ShiliLiang,SinIlPak,QingyueChen, and Shifeng Yan. A measurement of illumination variation-resistant noncontact heart rate based on the combination of singular spectrum analysis and sub-band method.Computer Methods and Programs in Biomedicine, 200:105824, 2021

    2021

  2. [2]

    Evaluation of contactless respiratory rate measurement: Thermogra- phy vs

    Wang Liao, Chen Zhang, Maik Rosenberger, and Gunther Notni. Evaluation of contactless respiratory rate measurement: Thermogra- phy vs. rPPG.Measurement: Sensors, page 101647, 2025

    2025

  3. [3]

    Availability and performance of face based non-contact methods for heart rate and oxygen saturation estimations: A systematic review

    AnkitGupta,AntonioGRavelo-Garcia,andFernandoMorgadoDias. Availability and performance of face based non-contact methods for heart rate and oxygen saturation estimations: A systematic review. Computer Methods and Programs in Biomedicine,219:106771,2022

    2022

  4. [4]

    RADIANT: Better rPPG Estimation Using Signal Embed- dings and Transformer

    Anup Kumar Gupta, Rupesh Kumar, Lokendra Birla, and Puneet Gupta. RADIANT: Better rPPG Estimation Using Signal Embed- dings and Transformer. InIEEE Winter Conference on Applications of Computer Vision, pages 4976–4986, 2023

    2023

  5. [5]

    SHINE: Synergizing transformers with contrastive learning for thriving rPPG-based SpO2 estimation.Expert Systems with Applications, page 129190, 2025

    Vaidehi Agarwal, Trishna Saikia, Anup Kumar Gupta, and Puneet Gupta. SHINE: Synergizing transformers with contrastive learning for thriving rPPG-based SpO2 estimation.Expert Systems with Applications, page 129190, 2025

    2025

  6. [6]

    Zhihong Lin, Jingjing Luo, Hongbo Wang, Rong Shen, Zhaozheng Wang, and Zan Fang. Facial ippg heatmap patterns based on period- aware autoencoder show association with carotid atherosclerosis to- wardsnon-contacthemodynamicassessment.Computer Methods and Programs in Biomedicine, page 109508, 2026

    2026

  7. [7]

    ALPINE: Improving Remote Heart Rate Estimation Using ContrastiveLearning

    Lokendra Birla, Sneha Shukla, Anup Kumar Gupta, and Puneet Gupta. ALPINE: Improving Remote Heart Rate Estimation Using ContrastiveLearning. InIEEE Winter Conference on Applications of Computer Vision, pages 5029–5038, 2023

    2023

  8. [8]

    Weakly supervised rPPG estimation for respiratory rate estimation

    Jingda Du, Si-Qi Liu, Bochao Zhang, and Pong C Yuen. Weakly supervised rPPG estimation for respiratory rate estimation. In IEEE/CVF International Conference on Computer Vision Workshops, 2021

    2021

  9. [9]

    An as- sessment of algorithms to estimate respiratory rate from the remote photoplethysmogram

    Duncan Luguern, Simon Perche, Yannick Benezeth, et al. An as- sessment of algorithms to estimate respiratory rate from the remote photoplethysmogram. InIEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020

    2020

  10. [10]

    Multimodal breathing rate estimation using facial motion and rPPG from RGB camera

    MigyeongGwak,KoroshVatanparvar,LiZhu,NafiulRashid,Mohsin Ahmed, Jungmok Bae, Jilong Kuang, and Alex Gao. Multimodal breathing rate estimation using facial motion and rPPG from RGB camera. InIEEE International Conference on Acoustics, Speech and Signal Processing, pages 2046–2050, 2024

    2046

  11. [11]

    HREADAI: heart rate estimation from face mask videos by consolidating Eulerian and Lagrangian approaches.IEEE Transac- tions on Instrumentation and Measurement, 73:1–11, 2023

    Trishna Saikia, Lokendra Birla, Anup Kumar Gupta, and Puneet Gupta. HREADAI: heart rate estimation from face mask videos by consolidating Eulerian and Lagrangian approaches.IEEE Transac- tions on Instrumentation and Measurement, 73:1–11, 2023

    2023

  12. [12]

    Respiration rate estimation from remote PPG via camera in presence of non-voluntary artifacts

    Korosh Vatanparvar, Migyeong Gwak, Li Zhu, Jilong Kuang, and Alex Gao. Respiration rate estimation from remote PPG via camera in presence of non-voluntary artifacts. InIEEE EMBS International Conference on Wearable and Implantable Body Sensor Networks, pages 1–4, 2022

    2022

  13. [13]

    Robust pulse rate from chrominance-based rPPG.IEEE Transactions on Biomedical Engi- neering, 60(10):2878–2886, 2013

    Gerard De Haan and Vincent Jeanne. Robust pulse rate from chrominance-based rPPG.IEEE Transactions on Biomedical Engi- neering, 60(10):2878–2886, 2013

    2013

  14. [14]

    Robust respirationdetectionfromremotephotoplethysmography.Biomedical Optics Express, 7(12):4941–4957, 2016

    Mark Van Gastel, Sander Stuijk, and Gerard De Haan. Robust respirationdetectionfromremotephotoplethysmography.Biomedical Optics Express, 7(12):4941–4957, 2016

    2016

  15. [15]

    A comparative study of princi- pled rPPG-based pulse rate tracking algorithms for fitness activities

    Qiang Zhu, Chau-Wai Wong, Zachary McBride Lazri, Mingliang Chen, Chang-Hong Fu, and Min Wu. A comparative study of princi- pled rPPG-based pulse rate tracking algorithms for fitness activities. IEEE Transactions on Biomedical Engineering,72(1):152–165,2024

    2024

  16. [16]

    Respiratory rate estimation from face videos

    Mingliang Chen, Qiang Zhu, Harrison Zhang, Min Wu, and Quanzeng Wang. Respiratory rate estimation from face videos. InIEEE EMBS International Conference on Biomedical & Health Informatics, 2019

    2019

  17. [17]

    Mediapipe:Aframeworkforperceivingand processingreality

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Yong,JuhyunLee,etal. Mediapipe:Aframeworkforperceivingand processingreality. InThird workshop on Computer Vision for AR/VR at IEEE Computer Vision and Pattern Recognition, 2019

    2019

  18. [18]

    AND-rPPG: A novel denoising- rPPGnetworkforimprovingremoteheartrateestimation.Computers in Biology and Medicine, 141:105146, 2022

    Lokendra Birla and Puneet Gupta. AND-rPPG: A novel denoising- rPPGnetworkforimprovingremoteheartrateestimation.Computers in Biology and Medicine, 141:105146, 2022

    2022

  19. [19]

    Detectingpulse from head motions in video

    GuhaBalakrishnan,FredoDurand,andJohnGuttag. Detectingpulse from head motions in video. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3430–3437, 2013

    2013

  20. [20]

    Nunzia Molinaro, Emiliano Schena, Sergio Silvestri, Fabrizio Bonotti, Damiano Aguzzi, Erika Viola, Fabio Buccolini, and Carlo Massaroni. Contactless vital signs monitoring from videos recorded with digital cameras: An overview.Frontiers in Physiology, 13: :Preprint submitted to Elsevier Page 11 of 12 ELITE-RR 801709, 2022

    2022

  21. [21]

    Serial fusion ofEulerianandLagrangianapproachesforaccurateheart-rateestima- tionusingfacevideos

    Puneet Gupta, Brojeshwar Bhowmick, and Arpan Pal. Serial fusion ofEulerianandLagrangianapproachesforaccurateheart-rateestima- tionusingfacevideos. InIEEE Engineering in Medicine and Biology Society, pages 2834–2837, 2017

    2017

  22. [22]

    A repro- ducible study on remote heart rate measurement.arXiv preprint arXiv:1709.00962, 2017

    Guillaume Heusch, André Anjos, and Sébastien Marcel. A repro- ducible study on remote heart rate measurement.arXiv preprint arXiv:1709.00962, 2017

    Pith/arXiv arXiv 2017

  23. [23]

    Estimation of respiratorysignalsfromremotephotoplethysmographyofRGBfacial videos.Electronics, 14(11):2152, 2025

    Hyunsoo Seo, Seunghyun Kim, and Eui Chul Lee. Estimation of respiratorysignalsfromremotephotoplethysmographyofRGBfacial videos.Electronics, 14(11):2152, 2025. :Preprint submitted to Elsevier Page 12 of 12

    2025