Pith. sign in

REVIEW 3 major objections 5 minor 77 references

CodePhys: Robust Video-based Remote Physiological Measurement through Latent Codebook Querying

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CodePhys recovers heart rate from video by looking up clean PPG codes instead of denoising the signal directly.

desk verdict Solid, well-executed rPPG paper with a genuinely new codebook-query idea; the central mechanism is under-verified, but the empirical case is strong enough to warrant peer review. read the letter →

arxiv 2502.07526 v1 pith:OIZVRHEA submitted 2025-02-11 cs.CV

Shuyang Chu , Menghan Xia , Mengyao Yuan , Xin Liu , Tapio Seppanen , Guoying Zhao , Jingang Shi This is my paper · ORCID
classification cs.CV
keywords remotephotoplethysmographyrPPGcodebookqueryingdiscreterepresentationlearningvectorquantizationheartrateestimationvisualinterferencerobustnessknowledgedistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that accurate remote heart-rate measurement (rPPG) from facial video is best framed not as signal extraction but as retrieval: the system learns a codebook of clean, ground-truth photoplethysmography (PPG) fragments, and any degraded feature pulled from a video is replaced by its nearest clean entry before decoding into a pulse waveform. The claim is that visual interference such as motion blur, camera noise, changing resolution, occlusion, and brightness shifts gets corrected in one uniform mechanism rather than by hand-crafted modules tuned to each nuisance. If this is right, video-based vitals monitoring becomes substantially more resilient: CodePhys reports the lowest heart-rate errors on VIPL-HR, UBFC-rPPG, PURE, and COHFACE in both intra-dataset and cross-dataset settings, and it can be grafted onto existing rPPG backbones with measurable gains. The paper's bets are that a 64-item codebook built from ground-truth PPG signals covers enough pulse variability, and that hard nearest-neighbour lookup in the learned latent space preserves the heart-rate-relevant details.

What carries the argument

The central object is the noise-free PPG codebook $\mathcal{C} = (\mathbf{c}_1,\ldots,\mathbf{c}_N) \in \mathbb{R}^{N\times D}$ with $N=D=64$, learned by reconstructing GT-PPG signals through a signal encoder $\mathbf{E}_s$ and decoder $\mathbf{D}_s$ using a quantization loss. The code query process is a hard nearest-neighbour lookup: for each query feature $\mathbf{z}_i$, a one-hot coordinate $\mathbf{q}_i$ marks the closest codebook item under the Euclidean norm, and the feature is replaced by $\mathbf{q}_i\cdot\mathcal{C}$. The pre-trained decoder then turns the selected clean entries into a pulse signal, which is what converts denoising into a lookup against a noise-free prior.

What would settle it

Train CodePhys on subjects whose heart rates never exceed 80 bpm and test on subjects whose heart rates exceed 120 bpm, then compare MAE against a version with the codebook removed; if the error gap reverses, the codebook coverage assumption fails. A finer check is to perturb query features adversarially and see whether two physiologically distinct waveforms collapse to the same codebook entry, which would show loss of detail.

Watch

Extended reading notes

Core claim

CodePhys treats rPPG measurement as a code query task in a noise-free proxy space. Stage I trains a signal autoencoder on ground-truth PPG signals with a vector-quantization objective, producing a codebook $\mathcal{C}$ of 64 items in $\mathbb{R}^{64}$ whose nearest-neighbour lookup, followed by decoding, reconstructs PPG nearly exactly. Stage II freezes $\mathcal{C}$ and the decoder; a spatial-aware video encoder maps facial video into query features $\mathbf{Z}_{\text{rppg}}$, an auxiliary prior branch aligns their distribution, and the model outputs the decoded signal from the nearest clean codebook items. The central discovery claim is that this single replacement removes general visual interference, not just the specific kinds previous methods target, and yields state-of-the-art heart-rate accuracy: MAE of 4.27 bpm on VIPL-HR, 0.21 bpm on UBFC-rPPG, 0.39 bpm on PURE, and 1.19 bpm on COHFACE, with particular stability when test videos are synthetically degraded by blur, noise, resolution changes, occlusion, or brightness shifts.

Load-bearing premise

The load-bearing premise is that the 64-entry codebook learned from training-set PPG signals covers the pulse variability of unseen subjects well enough that hard nearest-neighbour lookup never throws away heart-rate-relevant detail.

Editorial extensions

If this is right

  • Because the frozen codebook and decoder handle degradation jointly, visual-interference robustness in rPPG no longer needs a bespoke module for each nuisance (motion, blur, compression, occlusion), which is the paper's main bet.
  • Existing end-to-end rPPG networks can be upgraded by replacing only their video feature extractor with the query pipeline, and the paper reports that DeepPhys, EfficientPhys, and PhysFormer all improve in MAE this way.
  • Cross-dataset transfer improves: training on PURE plus COHFACE and testing on UBFC-rPPG reaches MAE 0.58 bpm, and training on VIPL-HR then testing on PURE reaches 4.03 bpm, evidence that the clean-code prior generalizes across domains.
  • The added modules are lightweight at inference, with 5.73 million parameters, 75.79 G MACs, and 0.12 ms per frame on the reported GPU, so the querying step need not block real-time use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors leave implicit is building the Stage I codebook from contact PPG recorded across diverse populations, age groups, and sensor types, which would test whether the clean-code prior covers pulse morphologies outside the benchmark training sets.
  • Because retrieval is a hard nearest-neighbour lookup, a testable variant would replace it with soft or top-$k$ mixtures of code entries; this could preserve waveform morphology for very low or irregular heart rates where a single code may be too coarse.
  • The paper assumes each video query feature is a degraded version of a single clean PPG code, so a face video containing physiological signal from more than one source, such as two faces, might force the lookup to merge distinct pulses; enforcing a sparse spatial mixture of codes per region is a concrete way to address that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. CodePhys is a two-stage framework for camera-based remote photoplethysmography (rPPG). In Stage I, a 1D signal autoencoder is trained to reconstruct GT-PPG waveforms, and a 64-item codebook of latent features is obtained via vector-quantization-style losses. In Stage II, a video feature extractor with spatial attention and a spatio-temporal encoder with an auxiliary prior branch produce query features; a soft feature distillation loss aligns the video features with GT-PPG features, and the query features are replaced by the nearest codebook item before the frozen decoder produces the predicted rPPG signal. The paper reports intra-dataset and cross-dataset heart-rate results on VIPL-HR, UBFC-rPPG, PURE, and COHFACE, component ablations, an efficiency comparison, a plug-and-play experiment with existing backbones, and a robustness study with five synthetic video degradations.

Significance. Conditional on the empirical claims, CodePhys is a novel application of discrete codebook priors to rPPG, and the plug-and-play integration of existing backbones in Table IX is practically useful. The paper is unusually broad in comparisons and includes an ablation for each component (CQP, APB, SFD, SAM, Stage I). However, the central mechanism, that code querying removes visual interference by selecting the correct noise-free code, is not directly evidenced, and the performance claims rest on single-seed point estimates and a one-fold robustness experiment.

major comments (3)
  1. [Section III-B3, Eq. (14), Tables V-VII] The load-bearing claim that Eq. (11) corrects degraded rPPG features by retrieving the correct noise-free codebook item is not directly verified. The CQP ablation in Table V only toggles the query operation as a whole; it cannot distinguish 'correct code selection' from a learned projection that happens to discard noise. Furthermore, Table VII reports near-perfect Stage I reconstruction but does not state whether those reconstructions are obtained on held-out subjects/clips; if they are training-set reconstructions, they do not establish that the 64-item codebook covers test-subject PPG variability. Please report code-selection accuracy (agreement between the argmin in Eq. (2) for video queries and Qgt computed from GT-PPG), per-subject code-usage statistics, and distance-to-selected-code diagnostics on held-out and degraded test clips.
  2. [Section IV-H, Table X] The robustness study applies synthetic degradations only to VIPL-HR Fold-1 and reports only final MAE/RMSE values. It is not specified whether 'trained on the Fold-1' means the test fold is held out, and if the same fold is used for hyperparameter selection the comparison is weakened. At minimum, report robustness across all five VIPL-HR folds or an equivalent held-out protocol, include code-selection accuracy under each degradation type, and provide confidence intervals across degradation realizations so the reader can see that the robustness claim is not specific to one fold.
  3. [Tables I-IV] All results are point estimates from a single training run. Given the small absolute differences on some datasets (e.g., Table II, UBFC-rPPG MAE 0.21 bpm vs. 0.40 bpm for PhysFormer), the state-of-the-art claim is not established without multiple seeds or paired significance tests. Please report mean and standard deviation over at least three seeds for the main intra-dataset and cross-dataset tables, or provide statistical significance tests against the strongest baselines.
minor comments (5)
  1. [Section III-A2] The codebook is initialized randomly and optimized through the 'GLO strategy [50]', but neither the GLO update rule nor its role relative to standard VQ training is described; please add a sentence explaining this choice.
  2. [Eq. (12)] The distance function D(·,·) uses the same symbol as the decoder D_s, which is confusing; please rename one of them.
  3. [Tables I and II] The table footnotes define △, ‡, and ⋆, but the meaning of '--' entries (methods not evaluated on a dataset) is not stated; please clarify.
  4. [Section IV-H] The degradation ranges are given, but the number of random realizations per degradation type and whether the same degraded videos are used for both CodePhys and PhysFormer are not stated; please specify this protocol.
  5. [Figure 9] The hyperparameter ablation figure lacks axis labels; please add them so the sensitivity results are interpretable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CodePhys's codebook is learned from training-set GT-PPG, and held-out/cross-dataset evaluations provide independent support; robustness-test-on-training-fold and absent code-selection-accuracy diagnostics are validation limitations, not circularity.

full rationale

Stage I's Eqs. (2)-(6) define a VQ-VAE-style reconstruction of GT-PPG signals; this is an autoencoder fit, but it is not the paper's predictive claim for rPPG measurement. The predictive chain in Stage II (Eqs. (8)-(11)) maps video features to rPPG features and decodes the nearest codebook item, and it is trained with Lphys, Lcode', Lppg_code', and Ldistill. The label Qgt in Eq. (14) is obtained by encoding the same GT-PPG through the Stage-I codebook; using it as a supervised training target is standard practice, and at inference the code selection is an argmin over the fixed codebook, not a recomputation from test-set GT. Intra-dataset results use subject-exclusive splits (e.g., VIPL-HR 5-fold, UBFC-rPPG train/test split), and cross-dataset results train on two datasets and test on a third, so the reported HR errors are not fitted quantities from the test set. The robustness study in Section IV-H applies synthetic degradations to Fold-1, which is also the training fold, and the paper does not report code-selection accuracy on degraded or held-out data; those are genuine evidence gaps about coverage and correctness, but they are not circular equivalence. The self-citations (e.g., AutoHR [49], PhysFormer [14]) serve as baselines and building blocks, not as a uniqueness theorem or as the sole justification of the central codebook-query premise. No equation in the paper reduces to its own inputs by construction, so there is no significant circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim depends on several hand-tuned hyperparameters (codebook size, feature dimension, loss weights, training schedules) and on empirical distribution-alignment assumptions. No new physical entities are introduced. The codebook itself is a learned representational construct rather than an invented physical entity. The Stage I autoencoder's near-perfect reconstruction of GT-PPG (Table VII) is a sanity check, not evidence that the codebook covers all real-world PPG variability.

free parameters (6)
  • Codebook size N = 64
    Set empirically in Section III-A2; ablation in Fig. 9 shows mild sensitivity, but too few or too many items degrade performance.
  • Feature dimension D = 64
    Set empirically; ablation in Fig. 9.
  • Quantization loss weight delta = 0.25
    Weight in Eq. (5), chosen by hand; no sensitivity study reported.
  • Temporal loss weight lambda = 0.1
    Weight in Eq. (13) balancing negative Pearson and spectral cross-entropy terms.
  • Stage II loss weights alpha and beta = alpha=2.0, beta=0.1
    Weights in Eq. (16); Fig. 9 shows mild sensitivity over a range.
  • Training hyperparameters = lr 5e-3 (Stage I), 1e-4 (Stage II), weight decay 5e-5, batch size 8/4, epochs 15/20
    Section IV-B; chosen by experience, no systematic sweep.
assumptions (4)
  • domain assumption GT-PPG signals are noise-free and time-aligned with the facial video frames.
    The codebook is constructed from GT-PPG in Stage I. If ground-truth pulse data contain motion artifacts or misalignment, the claimed noise-free proxy is polluted. Invoked throughout Section III-A.
  • ad hoc to paper A 64-item codebook learned by a signal autoencoder is a complete basis for PPG variability across unseen subjects.
    No coverage guarantee is given. The ablation in Fig. 9 shows performance depends on N, and cross-dataset tests require the codebook to handle new subject distributions.
  • ad hoc to paper After AdaIN alignment (Eq. 9), video-derived query features and PPG-derived codebook features lie in the same distribution, making Euclidean nearest-neighbor query meaningful.
    This distribution alignment is an empirical assumption. The paper provides no theoretical argument that AdaIN removes the domain gap between video features and PPG-derived features.
  • domain assumption Nearest-neighbor code query in the Euclidean latent space preserves physiologically relevant information while discarding visual interference.
    This is the core mechanism of CQP. The ablation (Tables V and VI) suggests it helps empirically, but the mechanism is not proven and could in principle discard useful waveform detail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CodePhys: Robust Video-based Remote Physiological Measurement through Latent Codebook Querying." pith.science (2026). https://pith.science/paper/OIZVRHEA

@misc{pith2026250207526,
  author       = {Pith},
  title        = {Pith review of: CodePhys: Robust Video-based Remote Physiological Measurement through Latent Codebook Querying},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OIZVRHEA}},
  note         = {Machine review of arXiv:2502.07526}
}
read the original abstract

Remote photoplethysmography (rPPG) aims to measure non-contact physiological signals from facial videos, which has shown great potential in many applications. Most existing methods directly extract video-based rPPG features by designing neural networks for heart rate estimation. Although they can achieve acceptable results, the recovery of rPPG signal faces intractable challenges when interference from real-world scenarios takes place on facial video. Specifically, facial videos are inevitably affected by non-physiological factors (e.g., camera device noise, defocus, and motion blur), leading to the distortion of extracted rPPG signals. Recent rPPG extraction methods are easily affected by interference and degradation, resulting in noisy rPPG signals. In this paper, we propose a novel method named CodePhys, which innovatively treats rPPG measurement as a code query task in a noise-free proxy space (i.e., codebook) constructed by ground-truth PPG signals. We consider noisy rPPG features as queries and generate high-fidelity rPPG features by matching them with noise-free PPG features from the codebook. Our approach also incorporates a spatial-aware encoder network with a spatial attention mechanism to highlight physiologically active areas and uses a distillation loss to reduce the influence of non-periodic visual interference. Experimental results on four benchmark datasets demonstrate that CodePhys outperforms state-of-the-art methods in both intra-dataset and cross-dataset settings.

Figures

Figures reproduced from arXiv: 2502.07526 by the authors.

Figure 1
Figure 1. Conceptual comparison of CodePhys and existing typical rPPG measurement methods. (a) Directly extracting rPPG signals from the video. (b) Designing hand-crafted modules to eliminate specific visual interference, taking the typical method [1] as an example. (c) Learning a codebook to eliminate all types of visual interference. Here the cloud symbolizes the visual interference present in the video or in features, and … view at source ↗
Figure 2
Figure 2. Overview of CodePhys. In Stage I, a codebook consisting of noise-free PPG features is learned by reconstructing GT-PPG signals, which is treated as the prior. In Stage II, the spatial-aware encoder (composed of a video feature extractor and a spatio-temporal encoder with an auxiliary prior branch) queries corresponding PPG features with respect to the input video. Subsequently, the decoder network reconstructs the e… view at source ↗
Figure 3
Figure 3. Architecture of the video feature extractor Ev with a spatial at￾tention mechanism (SAM) attached. The ‘S’ and ‘C’ denote the sigmoid function and concatenation operation, respectively. W refer to channel, sequence length, height and width, while H′ and W′ are H/16 and W/16, respectively. The 3D CNN block starts with an initial 1×5×5 convolution for channel extraction, followed by four stacked 3×3×3 convolutions int… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The Bland-Altman plot (a) and scatter plot (b) show the difference between ground-truth HRs and predicted HRs by CodePhys on the Fold-1 of VIPL-HR dataset. IV. EXPERIMENTAL RESULTS We comprehensively assess the performance of CodePhys on four widely used benchmark data…
Figure 6
Figure 6. Figure 6: Visual comparison of the rPPG signals (left) predicted by CodePhys and their corresponding PSDs (right), alongside the respec￾tive ground-truth. (a) VIPL-HR, (b) UBFC-rPPG, (c) PURE, and (d) COHFACE. C. Intra-dataset Testing We first evaluate the HR estimation on all d…
Figure 5
Figure 5. Figure 5: Visual comparison of the rPPG signals (top) and HRs (bottom) predicted by CodePhys on UBFC-rPPG dataset, alongside the corre￾sponding ground-truth. detector to find the landmarks for each frame, then we crop and align the video frames to 128×128 pixels according to the…
Figure 7
Figure 7. Figure 7: Visual comparison of rPPG signals (top) reconstructed by the signal autoencoder network and their corresponding PSDs (bottom), alongside the corresponding ground-truth. (a) VIPL-HR, (b) UBFC￾rPPG, (c) PURE, and (d) COHFACE datasets. (a) (b) (c) (d) (e) (f) (i) (ii) (ii…
Figure 8
Figure 8. Figure 8: Visualization of spatial attention maps and the corresponding video features. The rows (i), (ii), (iii), and (iv) denote the original facial images, the attention maps Wv predicted by CodePhys with SAM (the brighter, the more attentive), the feature maps Fv predicted b…
Figure 9
Figure 9. Figure 9: Ablation study of the hyperparameters of N, D in the codebook C, and α, β in the loss function Loverall (from left to right) . TABLE IX STUDY OF INTEGRATING EXISTING METHODS INTO CODEPHYS FRAMEWORK. Method CodePhys MAE↓ RMSE↓ r ↑ Framework DeepPhys [60] w/o 11.0 13.8 0…
Figure 10
Figure 10. Figure 10: Visualization of rPPG signals predicted by CodePhys (red) and PhysFormer (green) in case of video quality degradation, alongside with the GT-PPG (blue). the video feature extractor part of these backbone networks, thereby achieving the integration of CodePhys and the …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

77 extracted references · 69 canonical work pages

  1. [1]

    Learning motion-robust remote photoplethys- mography through arbitrary resolution videos,

    J. Li, Z. Yu, and J. Shi, “Learning motion-robust remote photoplethys- mography through arbitrary resolution videos,” in Conference on Artifi- cial Intelligence, AAAI , 2023

    work page 2023

  2. [2]

    Non-contact, automated cardiac pulse measurements using video imaging and blind source separation

    M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation.” Optics express, vol. 18, no. 10, pp. 10 762–10 774, 2010

    2010

  3. [3]

    Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity,

    M. Lewandowska, J. Ruminski, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity,” inFederated Conference on Computer Science and Information Systems, 2011

    work page 2011

  4. [4]

    Robust pulse rate from chrominance-based rppg,

    G. De Haan and V . Jeanne, “Robust pulse rate from chrominance-based rppg,” IEEE Transactions on Biomedical Engineering , vol. 60, no. 10, pp. 2878–2886, 2013

    2013

  5. [5]

    Algorithmic principles of remote PPG,

    W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote PPG,” IEEE Trans. Biomed. Eng. , vol. 64, no. 7, pp. 1479–1491, 2017

    work page 2017

  6. [6]

    Adaptive plane orthogonal to skin-tone vector for non-contact pulse rate monitoring in motion scenarios,

    Q. Zhang, X. Lin, Y . Zhang, X. Liu, and F. Cai, “Adaptive plane orthogonal to skin-tone vector for non-contact pulse rate monitoring in motion scenarios,” Multim. Tools Appl., vol. 83, no. 39, pp. 86 741– 86 755, 2024

    work page 2024

  7. [7]

    Heart rate estimation in driver monitoring system using quality-guided spectrum peak screening,

    Z. Gong, X. Yang, R. Song, X. Han, C. Ren, H. Shi, J. Niu, and W. Li, “Heart rate estimation in driver monitoring system using quality-guided spectrum peak screening,” IEEE Trans. Instrum. Meas. , vol. 73, pp. 1– 14, 2024

    work page 2024

  8. [8]

    Robust and remote photoplethysmography based on smartphone imaging of the human palm,

    C. Lian, Y . Yang, X. Yu, H. Sun, Y . Zhao, G. Zhang, and W. J. Li, “Robust and remote photoplethysmography based on smartphone imaging of the human palm,” IEEE Trans. Instrum. Meas. , vol. 72, pp. 1–11, 2023

    work page 2023

Show all 77 references
  1. [9]

    Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation,

    X. Niu, S. Shan, H. Han, and X. Chen, “Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation,” IEEE Trans. Image Process., vol. 29, pp. 2409–2423, 2020

    2020

  2. [10]

    Video-based remote physiological measurement via cross-verified feature disentangling,

    X. Niu, Z. Yu, H. Han, X. Li, S. Shan, and G. Zhao, “Video-based remote physiological measurement via cross-verified feature disentangling,” in European Conference on Computer Vision (ECCV) , 2020

    2020

  3. [11]

    Dual-gan: Joint BVP and noise modeling for remote physiological measurement,

    H. Lu, H. Han, and S. K. Zhou, “Dual-gan: Joint BVP and noise modeling for remote physiological measurement,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2021

    2021

  4. [12]

    Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks,

    Z. Yu, X. Li, and G. Zhao, “Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks,” in British Machine Vision Conference (BMVC) , 2019

    2019

  5. [13]

    Channel-wise interactive learning for remote heart rate estimation from facial video,

    Q. Li, D. Guo, W. Qian, X. Tian, X. Sun, H. Zhao, and M. Wang, “Channel-wise interactive learning for remote heart rate estimation from facial video,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 6, pp. 4542–4555, 2024

    2024

  6. [14]

    Physformer: Facial video-based physiological measurement with temporal difference transformer,

    Z. Yu, Y . Shen, J. Shi, H. Zhao, P. H. S. Torr, and G. Zhao, “Physformer: Facial video-based physiological measurement with temporal difference transformer,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  7. [15]

    Tranphys: Spatiotemporal masked transformer steered remote photoplethysmogra- phy estimation,

    H. Shao, L. Luo, J. Qian, S. Chen, C. Hu, and J. Yang, “Tranphys: Spatiotemporal masked transformer steered remote photoplethysmogra- phy estimation,” IEEE Transactions on Circuits and Systems for Video Technology, 2023

    2023

  8. [16]

    Efficientphys: Enabling simple, fast and accurate camera-based vitals measurement,

    X. Liu, B. L. Hill, Z. Jiang, S. N. Patel, and D. McDuff, “Efficientphys: Enabling simple, fast and accurate camera-based vitals measurement,” CoRR, vol. abs/2110.04447, 2021

    2021 arXiv

  9. [17]

    NAS-HR: neural architecture search for heart rate estimation from face videos,

    H. Lu and H. Han, “NAS-HR: neural architecture search for heart rate estimation from face videos,” Virtual Real. Intell. Hardw., vol. 3, no. 1, pp. 33–42, 2021

    2021

  10. [18]

    St-phys: Unsupervised spatio-temporal contrastive remote physiological measure- ment,

    M. Cao, X. Cheng, X. Liu, Y . Jiang, H. Yu, and J. Shi, “St-phys: Unsupervised spatio-temporal contrastive remote physiological measure- ment,” IEEE Journal of Biomedical and Health Informatics , vol. 28, pp. 4613–4624, 2024

    2024

  11. [19]

    Condiff-rppg: Robust remote physiological measurement to heterogeneous occlusions,

    J. Wang, X. Wei, H. Lu, Y . Chen, and D. He, “Condiff-rppg: Robust remote physiological measurement to heterogeneous occlusions,” IEEE Journal of Biomedical and Health Informatics , pp. 1–13, 2024

    2024

  12. [20]

    Practical single-image super-resolution using look-up table,

    Y . Jo and S. J. Kim, “Practical single-image super-resolution using look-up table,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021

    2021

  13. [21]

    Neural discrete representation learning,

    A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural discrete representation learning,” in Annual Conference on Neural Information Processing Systems (NeurIPS) , 2017

    2017

  14. [22]

    Taming transformers for high- resolution image synthesis,

    P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high- resolution image synthesis,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2021

    2021

  15. [23]

    Towards robust blind face restoration with codebook lookup transformer,

    S. Zhou, K. C. K. Chan, C. Li, and C. C. Loy, “Towards robust blind face restoration with codebook lookup transformer,” in Annual Conference on Neural Information Processing System (NeurIPS) , 2022

    2022

  16. [24]

    Codetalker: Speech-driven 3d facial animation with discrete motion prior,

    J. Xing, M. Xia, Y . Zhang, X. Cun, J. Wang, and T.-T. Wong, “Codetalker: Speech-driven 3d facial animation with discrete motion prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2023

    2023

  17. [25]

    Remote plethysmo- graphic imaging using ambient light

    W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmo- graphic imaging using ambient light.” Optics express, vol. 16, no. 26, pp. 21 434–21 445, 2008

    2008

  18. [26]

    Advancements in noncontact, multiparameter physiological measurements using a webcam,

    M. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng., vol. 58, no. 1, pp. 7–11, 2011

    2011

  19. [27]

    Camera measurement of physiological vital signs,

    D. McDuff, “Camera measurement of physiological vital signs,” ACM Comput. Surv., vol. 55, pp. 176:1–176:40, 2023

    2023

  20. [28]

    A general remote photoplethysmography es- timator with spatiotemporal convolutional network,

    S.-Q. Liu and P. C. Yuen, “A general remote photoplethysmography es- timator with spatiotemporal convolutional network,” in 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). IEEE, 2020, pp. 481–488

    2020

  21. [29]

    An image is worth 16x16 words: Trans- formers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Trans- formers for image recognition at scale,” in International Conference on Learni...

    2021

  22. [30]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in IEEE/CVF International Conference on Computer Vision (ICCV), 2021

    2021

  23. [31]

    Physformer++: Facial video-based physiological measurement with slowfast temporal difference transformer,

    Z. Yu, Y . Shen, J. Shi, H. Zhao, Y . Cui, J. Zhang, P. H. S. Torr, and G. Zhao, “Physformer++: Facial video-based physiological measurement with slowfast temporal difference transformer,” Int. J. Comput. Vis., vol. 131, no. 6, pp. 1307–1330, 2023

    2023

  24. [32]

    Dual-path tokenlearner for remote photoplethysmography-based phys- iological measurement with facial videos,

    W. Qian, D. Guo, K. Li, X. Zhang, X. Tian, X. Yang, and M. Wang, “Dual-path tokenlearner for remote photoplethysmography-based phys- iological measurement with facial videos,” IEEE Transactions on Com- putational Social Systems , vol. 11, no. 3, pp. 4465–4477, 2024

    2024

  25. [33]

    rppg-mae: Self- supervised pre-training with masked autoencoders for remote physiolog- ical measurement,

    X. Liu, Y . Zhang, Z. Yu, H. Lu, H. Yue, and J. Yang, “rppg-mae: Self- supervised pre-training with masked autoencoders for remote physiolog- ical measurement,” CoRR, vol. abs/2306.02301, 2023

    2023 arXiv

  26. [34]

    Masked autoencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y . Li, P. Doll ´ar, and R. B. Girshick, “Masked autoencoders are scalable vision learners,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022

    2022

  27. [35]

    VIPL-HR: A multi-modal database for pulse estimation from less-constrained face video,

    X. Niu, H. Han, S. Shan, and X. Chen, “VIPL-HR: A multi-modal database for pulse estimation from less-constrained face video,” in Asian Conference on Computer Vision (ACCV) , 2018

    2018

  28. [36]

    Motion matters: Neural motion transfer for better camera physiological measurement,

    A. Paruchuri, X. Liu, Y . Pan, S. Patel, D. McDuff, and S. Sengupta, “Motion matters: Neural motion transfer for better camera physiological measurement,” in 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2024, pp. 5921–5930

    2024

  29. [37]

    Image super-resolution via sparse representation,

    J. Yang, J. Wright, T. S. Huang, and Y . Ma, “Image super-resolution via sparse representation,” IEEE Trans. Image Process. (TIP) , vol. 19, no. 11, pp. 2861–2873, 2010

    2010

  30. [38]

    Anchored neighborhood re- gression for fast example-based super-resolution,

    R. Timofte, V . D. Smet, and L. V . Gool, “Anchored neighborhood re- gression for fast example-based super-resolution,” in IEEE International Conference on Computer Vision (ICCV) , 2013. 14 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS

    2013

  31. [39]

    Convolu- tional sparse coding using wavelets for single image super-resolution,

    A. Ahmed, S. Kun, R. A. Memon, J. Ahmed, and G. Tefera, “Convolu- tional sparse coding using wavelets for single image super-resolution,” IEEE Access, 2019

    2019

  32. [40]

    Benchmarking micro- action recognition: Dataset, methods, and applications,

    D. Guo, K. Li, B. Hu, Y . Zhang, and M. Wang, “Benchmarking micro- action recognition: Dataset, methods, and applications,” IEEE Trans. Cir. and Sys. for Video Technol. , vol. 34, no. 7, p. 6238–6252, 2024

    2024

  33. [41]

    Image denoising via sparse and redundant representations over learned dictionaries,

    M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans. Image Process. (TIP), vol. 15, no. 12, pp. 3736–3745, 2006

    2006

  34. [42]

    Generating diverse high-fidelity images with VQ-V AE-2,

    A. Razavi, A. van den Oord, and O. Vinyals, “Generating diverse high-fidelity images with VQ-V AE-2,” inAnnual Conference on Neural Information Processing Systems (NeurIPS) , 2019

    2019

  35. [43]

    Unicolor: A unified framework for multi-modal colorization with transformer,

    Z. Huang, N. Zhao, and J. Liao, “Unicolor: A unified framework for multi-modal colorization with transformer,” ACM Trans. Graph. (ToG), vol. 41, no. 6, pp. 205:1–205:16, 2022

    2022

  36. [44]

    Generative semantic segmenta- tion,

    J. Chen, J. Lu, X. Zhu, and L. Zhang, “Generative semantic segmenta- tion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2023

    2023

  37. [45]

    Learning to listen: Modeling non-deterministic dyadic facial motion,

    E. Ng, H. Joo, L. Hu, H. Li, T. Darrell, A. Kanazawa, and S. Ginosar, “Learning to listen: Modeling non-deterministic dyadic facial motion,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  38. [46]

    Human pose as compositional tokens,

    Z. Geng, C. Wang, Y . Wei, Z. Liu, H. Li, and H. Hu, “Human pose as compositional tokens,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2023

    2023

  39. [47]

    Learning a model of facial shape and expression from 4d scans,

    T. Li, T. Bolkart, M. J. Black, H. Li, and J. Romero, “Learning a model of facial shape and expression from 4d scans,” ACM Trans. Graph. , vol. 36, no. 6, pp. 194:1–194:17, 2017

    2017

  40. [48]

    Deformable convolutional networks,

    J. Dai, H. Qi, Y . Xiong, Y . Li, G. Zhang, H. Hu, and Y . Wei, “Deformable convolutional networks,” inIEEE International Conference on Computer Vision (ICCV), 2017

    2017

  41. [49]

    Autohr: A strong end-to- end baseline for remote heart rate measurement with neural searching,

    Z. Yu, X. Li, X. Niu, J. Shi, and G. Zhao, “Autohr: A strong end-to- end baseline for remote heart rate measurement with neural searching,” IEEE Signal Process. Lett. , vol. 27, pp. 1245–1249, 2020

    2020

  42. [50]

    Optimizing the latent space of generative networks,

    P. Bojanowski, A. Joulin, D. Lopez-Paz, and A. Szlam, “Optimizing the latent space of generative networks,” arXiv preprint arXiv:1707.05776 , 2017

    2017 arXiv

  43. [51]

    Un- supervised skin tissue segmentation for remote photoplethysmography,

    S. Bobbia, R. Macwan, Y . Benezeth, A. Mansouri, and J. Dubois, “Un- supervised skin tissue segmentation for remote photoplethysmography,” Pattern Recognit. Lett., vol. 124, pp. 82–90, 2019

    2019

  44. [52]

    Non-contact video-based pulse rate measurement on a mobile service robot,

    R. Stricker, S. M ¨uller, and H. Gross, “Non-contact video-based pulse rate measurement on a mobile service robot,” in IEEE International Symposium on Robot and Human Interactive Communication (RO- MAN), 2014

    2014

  45. [53]

    A reproducible study on remote heart rate measurement,

    G. Heusch, A. Anjos, and S. Marcel, “A reproducible study on remote heart rate measurement,” arXiv preprint arXiv:1709.00962 , 2017

    2017 arXiv

  46. [54]

    The impact of video compression on remote cardiac pulse measurement using imaging photo- plethysmography,

    D. J. McDuff, E. B. Blackford, and J. R. Estepp, “The impact of video compression on remote cardiac pulse measurement using imaging photo- plethysmography,” in 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG) , 2017

    2017

  47. [55]

    Meta-rppg: Remote heart rate estimation using a transductive meta-learner,

    E. Lee, E. Chen, and C. Lee, “Meta-rppg: Remote heart rate estimation using a transductive meta-learner,” inEuropean Conference on Computer Vision (ECCV), 2020

    2020

  48. [56]

    Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions,

    S. Tulyakov, X. Alameda-Pineda, E. Ricci, L. Yin, J. F. Cohn, and N. Sebe, “Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR , 2016

    2016

  49. [57]

    Robust remote heart rate estimation from face utilizing spatial-temporal attention,

    X. Niu, X. Zhao, H. Han, A. Das, A. Dantcheva, S. Shan, and X. Chen, “Robust remote heart rate estimation from face utilizing spatial-temporal attention,” in IEEE International Conference on Automatic Face & Gesture Recognition, FG , 2019

    2019

  50. [58]

    Neuron structure modeling for generalizable remote physiological measurement,

    H. Lu, Z. Yu, X. Niu, and Y . Chen, “Neuron structure modeling for generalizable remote physiological measurement,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR , 2023

    2023

  51. [59]

    rppg- hiba: Hierarchical balanced framework for remote physiological mea- surement,

    Y . Wang, H. Lu, Y . Chen, L. Kuang, M. Zhou, and S. Deng, “rppg- hiba: Hierarchical balanced framework for remote physiological mea- surement,” in Proceedings of the 32nd ACM International Conference on Multimedia. ACM, 2024, pp. 2982–2991

    2024

  52. [60]

    Deepphys: Video-based physiological mea- surement using convolutional attention networks,

    W. Chen and D. McDuff, “Deepphys: Video-based physiological mea- surement using convolutional attention networks,” in European Confer- ence on Computer Vision (ECCV) , 2018

    2018

  53. [61]

    Contrast-phys+: Unsupervised and weakly-supervised video-based remote physiological measurement via spatiotemporal con- trast,

    Z. Sun and X. Li, “Contrast-phys+: Unsupervised and weakly-supervised video-based remote physiological measurement via spatiotemporal con- trast,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 8, pp. 5835– 5851, 2024

    2024

  54. [62]

    Rs-rppg: Robust self-supervised learning for rppg,

    M. R. Savic and G. Zhao, “Rs-rppg: Robust self-supervised learning for rppg,” in 18th IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 2024, pp. 1–10

    2024

  55. [63]

    Resolve domain conflicts for generalizable remote physio- logical measurement,

    W. Sun, X. Zhang, H. Lu, Y . Chen, Y . Ge, X. Huang, J. Yuan, and Y . Chen, “Resolve domain conflicts for generalizable remote physio- logical measurement,” in Proceedings of the 31st ACM International Conference on Multimedia . ACM, 2023, pp. 8214–8224

    2023

  56. [64]

    Visual heart rate estimation with convolutional neural network,

    R. Spetl ´ık, V . Franc, J. Cech, and J. Matas, “Visual heart rate estimation with convolutional neural network,” in British Machine Vision Confer- ence (BMVC), 2018

    2018

  57. [65]

    Eta-rppgnet: Effective time-domain attention network for remote heart rate measure- ment,

    M. Hu, F. Qian, D. Guo, X. Wang, L. He, and F. Ren, “Eta-rppgnet: Effective time-domain attention network for remote heart rate measure- ment,” IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1–12, 2021

    2021

  58. [66]

    And-rppg: A novel denoising-rppg network for improving remote heart rate estimation,

    L. Birla and P. Gupta, “And-rppg: A novel denoising-rppg network for improving remote heart rate estimation,” Comput. Biol. Medicine , vol. 141, p. 105146, 2022

    2022

  59. [67]

    The way to my heart is through contrastive learning: Remote photoplethysmography from unlabelled video,

    J. Gideon and S. Stent, “The way to my heart is through contrastive learning: Remote photoplethysmography from unlabelled video,” in IEEE/CVF International Conference on Computer Vision (ICCV) , 2021

    2021

  60. [68]

    Pulsegan: Learning to generate realistic pulse waveforms in remote photoplethys- mography,

    R. Song, H. Chen, J. Cheng, C. Li, Y . Liu, and X. Chen, “Pulsegan: Learning to generate realistic pulse waveforms in remote photoplethys- mography,”IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 5, pp. 1373–1384, 2021

    2021

  61. [69]

    Contrast-phys: Unsupervised video-based remote physiological measurement via spatiotemporal contrast,

    Z. Sun and X. Li, “Contrast-phys: Unsupervised video-based remote physiological measurement via spatiotemporal contrast,” in European Conference on Computer Vision (ECCV) , 2022

    2022

  62. [70]

    Face2ppg: An unsupervised pipeline for blood volume pulse extraction from faces,

    C. A. Casado and M. B. L ´opez, “Face2ppg: An unsupervised pipeline for blood volume pulse extraction from faces,” IEEE Journal of Biomedical and Health Informatics , vol. 27, pp. 5530–5541, 2023

    2023

  63. [71]

    Non-contrastive un- supervised learning of physiological signals from video,

    J. Speth, N. Vance, P. J. Flynn, and A. Czajka, “Non-contrastive un- supervised learning of physiological signals from video,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR. IEEE, 2023, pp. 14 464–14 474

    2023

  64. [72]

    Facial video-based remote physiolog- ical measurement via self-supervised learning,

    Z. Yue, M. Shi, and S. Ding, “Facial video-based remote physiolog- ical measurement via self-supervised learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023

    2023

  65. [73]

    How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230, 000 3d facial landmarks),

    A. Bulat and G. Tzimiropoulos, “How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230, 000 3d facial landmarks),” in IEEE International Conference on Computer Vision (ICCV), 2017

    2017

  66. [74]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR) , 2015

    2015

  67. [75]

    Multi-task learning for simultaneous video generation and remote photoplethysmography estimation,

    Y . Tsou, Y . Lee, and C. Hsu, “Multi-task learning for simultaneous video generation and remote photoplethysmography estimation,” in Asian Conference on Computer Vision (ACCV) , 2020

    2020

  68. [76]

    Domain generalized RPPG network: Disentangled feature learning with domain permutation and do- main augmentation,

    W. Chung, C. Hsieh, S. Liu, and C. Hsu, “Domain generalized RPPG network: Disentangled feature learning with domain permutation and do- main augmentation,” in Asian Conference on Computer Vision (ACCV) , 2022

    2022

  69. [77]

    Multi-task temporal shift attention networks for on-device contactless vitals measurement,

    X. Liu, J. Fromm, S. N. Patel, and D. McDuff, “Multi-task temporal shift attention networks for on-device contactless vitals measurement,” in An- nual Conference on Neural Information Processing Systems (NeurIPS) , 2020

    2020

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.