Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Action Unit Enhance Dynamic Facial Expression Recognition

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a plug-in AU-expression knowledge matrix, used as an auxiliary loss, raises dynamic facial expression recognition by roughly one point on three benchmarks across three models with no increase in inference cost.

desk verdict Plug-in AU loss idea is reasonable and the gains are consistent, but the evaluation has a genuine leakage risk because the AU-knowledge matrix is built on the same datasets where improvements are measured. read the letter →

arxiv 2507.07678 v1 pith:YSH2T2Z7 submitted 2025-07-10 cs.CV

Feng Liu , Lingna Gu , Chen Shi , Xiaolan Fu This is my paper
classification cs.CV
keywords ActionUnitDynamicFacialExpressionRecognitionAUlossknowledgeinjectionAU-expressionmatrixclassimbalancecodingsystem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that dynamic facial expression recognition (DFER) can be improved by injecting quantified knowledge about which facial Action Units (AUs) drive which emotions. The authors compute an 18 by 7 AU-expression weight matrix from OpenFace readings on four video datasets, then add an auxiliary AU loss weighted by that matrix to three existing DFER models. On DFEW, FERV39k, and MAFW, they report that weighted average recall and unweighted average recall rise by roughly one percentage point for M3DFEL, MAE-DFER, and Former-DFER, with no increase in inference FLOPs. The larger practical claim is that any DFER model can be upgraded by this plug-in, and that minority emotion classes, such as disgust, benefit most.

What carries the argument

The machinery is the AU-expression knowledge matrix $K \in \mathbb{R}^{18\times 7}$ and the AU loss that carries it. $K$ encodes, for each expression class, how strongly each of 18 AUs is activated, computed from OpenFace predictions and thresholded M3DFEL pseudo-labels on four datasets, then normalized by subtracting 2.5, applying a sigmoid, and scaling by 5. At training time, the row of $K$ corresponding to each sample's expression label weights the binary cross-entropy terms of an 18-way AU prediction head, so the network is pushed to learn facial features consistent with the psychological prior; at inference the AU head is not used, so compute is unchanged.

What would settle it

A decisive check would be to compute the knowledge matrix exclusively from the training folds of DFEW, retrain M3DFEL with AU-DFER, and compare WAR and UAR on the official test fold with the paper's reported gains; if the gains vanish, the original matrix depended on test-set statistics and the transferable-knowledge claim fails.

Watch

Extended reading notes

Core claim

The central discovery claimed is that a fixed, precomputed matrix encoding the quantitative relationship between 18 Action Units and 7 expressions acts as transferable prior knowledge that improves DFER when used to weight an auxiliary AU classification loss. The matrix is derived by running OpenFace on AffWild2, DFEW, FERV39k, and RAVDESS, keeping only frames whose M3DFEL expression prediction passes a threshold, and taking per-expression medians of AU activation. The resulting matrix is normalized, summed across datasets, and scaled; in training it multiplies a binary cross-entropy AU loss whose positive-class weights can be global, expression-specific, or applied only to minority classes. Across three backbones and three benchmarks, the authors report consistent improvements around 1% in both WAR and UAR, including a rise in M3DFEL's disgust accuracy on DFEW from 0.00% to 10.34%, and they show FLOPs stay unchanged because the AU head is dropped at inference.

Load-bearing premise

The gains rest on the premise that the AU-expression knowledge matrix, derived from OpenFace's AU outputs on the same datasets later used for evaluation, is accurate and transferable knowledge rather than noise or test-set leakage; the authors themselves note in Section 6 that OpenFace's accuracy could be improved and that demographic and cultural bias remains.

Editorial extensions

If this is right

  • M3DFEL, MAE-DFER, and Former-DFER all gain roughly one point in WAR and UAR on DFEW, FERV39k, and MAFW when AU-DFER is added, so the benefit is not tied to one architecture or dataset.
  • Because the AU classifier is removed at inference, the improvement costs no additional FLOPs; only training changes.
  • Minority classes improve disproportionately; for example, M3DFEL's disgust accuracy on DFEW goes from 0.00% to 10.34%, and the choice of positive-class weighting strategy can direct gains toward minor classes.
  • The best ratio between AU loss and expression loss depends on the dataset and model, so AU-DFER requires tuning the mixing weight rather than a single fixed recipe.
  • AU knowledge computed from four datasets transfers to MAFW, on which no knowledge matrix was computed, supporting the claim that the knowledge is universal across datasets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the knowledge matrix is built from the same datasets that are later used for evaluation, part of the reported gain could come from dataset-specific statistics baked into the weights; a leave-one-dataset-out test, where the matrix is built only from held-out training folds or from entirely different datasets, would separate prior knowledge from leakage.
  • The paper's 'no additional arithmetic' claim concerns inference: training still requires an extra AU head and offline OpenFace preprocessing, so the efficiency claim should be read as inference-only.
  • If the mechanism is genuine AU knowledge rather than generic regularization, then corrupting the matrix by shuffling rows or replacing it with a random matrix should erase the gains; a control experiment of this kind would sharpen the causal claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes AU-DFER, a plug-in module for dynamic facial expression recognition (DFER) that quantifies the relationship between Action Units (AUs) and expression categories and injects this knowledge into a base model through an auxiliary AU loss. An 18×7 knowledge matrix K is computed from OpenFace AU outputs and M3DFEL pseudo-labels on four datasets (AffWild2, DFEW, FERV39k, RAVDESS), normalized, and used as the weight in a BCEWithLogits AU loss added to the model's original loss. The method is tested on three base models (M3DFEL, MAE-DFER, Former-DFER) on DFEW, FERV39k, and MAFW. The authors report consistent improvements of roughly 1% in WAR and UAR, with larger gains on minority classes such as Disgust, and claim that AU-DFER can improve any DFER model without extra inference cost.

Significance. If the results hold, the contribution is practically attractive: a simple, model-agnostic loss modification that improves DFER without increasing inference FLOPs. The paper provides code, ablates three weighting strategies, and the MAFW result offers some evidence of cross-dataset behavior since MAFW is not in the K-estimation pool. However, the current evaluation does not establish that K is a transferable 'a priori' prior, because K is estimated on the same datasets (DFEW, FERV39k) later used for training and evaluation, and because the experimental reporting lacks statistical calibration. The central claim of out-of-the-box SOTA performance is also overstated relative to Table 1.

major comments (5)
  1. [Section 3.1 / Section 4.1] The knowledge matrix K is computed from OpenFace AU readings and M3DFEL pseudo-labels on AffWild2, DFEW, FERV39k and RAVDESS, yet the same DFEW and FERV39k data are later used for training and testing in Tables 1–3. The paper never states whether K is estimated on full videos or per training fold. If test-fold frames contribute to K, the weight row k_j in Eq. (9) encodes class-conditional AU statistics of the test videos, so the reported gains (e.g., M3DFEL Disgust from 0.00% to 10.34% in Table 2) may be an artifact of leakage. Please recompute K using only training folds, or using a fully disjoint dataset, and report whether the improvements persist.
  2. [Section 3.1] The threshold θ used to select reliable frames for K estimation is never reported. Since θ controls which M3DFEL pseudo-labels enter the knowledge matrix, tuning θ on the evaluation sets would be an additional leakage pathway. Please disclose the value of θ and show the sensitivity of the main results to this choice.
  3. [Section 5.1, Table 1] The abstract and introduction claim that AU-DFER 'outperforms the state-of-the-art (SOTA) methods.' Table 1 does not support this: S2D has higher UAR than the AU-DFER variants on DFEW (65.45 vs. 61.87 for MAE-DFER+AU-DFER) and on MAFW (57.37 vs. 47.41). The claim should be reformulated to state that AU-DFER improves over its three base models under local replication, and the comparison with published SOTA should be qualified.
  4. [Section 5.1, Tables 1–3] No error bars or multiple runs are reported anywhere. The reported improvements are around 1 WAR/UAR, which is within typical run-to-run variance for DFER. Please provide mean±std over at least three seeds for the main comparisons, or otherwise establish that the gains are statistically significant.
  5. [Section 3.1] The construction of K is under-specified and internally inconsistent. The phrase 'the average of maximum and minimum' is not defined (per row, per matrix, across entries?), and the text states that each dataset matrix is sigmoid-normalized, yet later says the sum of the four matrices 'is till in the range of [0,5]'—since sigmoid outputs are in [0,1], the sum should lie in [0,4]. The constants 2.5 and 5 are introduced without justification. Please give the exact formula for K and ablate the normalization choices to show they are not driving the results.
minor comments (6)
  1. [Section 5.2] The text says the three AU weighting strategies are 'introduced in section 3.4', but the strategies appear in Section 3.3 (Eqs. 6–8).
  2. [Table 2] The FRU-Adapter row is malformed (66.02 76.96 38.65 50.12 61.88 25.52 50.22 74.81 65.45 -), making the per-class accuracies unreadable.
  3. [Table 1] The footnote 'MAFW result based on 7-class' is unexplained. The paper should state MAFW's original label space and describe how the 7-class mapping is obtained.
  4. [Abstract / Section 5.1] The phrase 'without the need for additional arithmetic' is imprecise: the AU loss adds training-time computation, and only the inference FLOPs remain the same as the base model.
  5. [References] References [16] and [50] are duplicates of the same paper, as are [9] and [10].
  6. [Throughout] There are typos, e.g., 'depicited' in Section 3.3, 'till' in Section 3.1, and the sentence in Section 5.1 'the performance suffers from imbalanced data, while adding AU solves this problem to some extent' is grammatically incomplete.

Circularity Check

1 steps flagged · score 6.0 of 10

The AU-expression prior K is estimated on the same datasets (DFEW, FERV39k) that later measure the gain, so part of the reported improvement is a fitted statistic rather than an independent prediction.

  1. fitted input called prediction [Section 1, Section 3.1 (Eqs. 1-3, K), Section 4.1 and Table 1]
    "Specifically, 4 existing mainstream DFER datasets(AffWild2, DFEW, FERV39k and RAVDESS) which are employed to analyze the quantitative relationship between AU and expressions. This enables a better design of the loss function for calculating AU loss, which represents the a priori knowledge of AU. Subsequently, the AU loss is integrated with the loss function of the existing open-source top-3 SOTA model."

    The knowledge matrix K is computed from AU statistics and M3DFEL pseudo-labels on DFEW and FERV39k, and the same two datasets are then used for evaluation. No split is reported between the K-estimation pool and the training/test folds: Section 4.1 lists DFEW and FERV39k as both K sources and evaluation sets. In Eq. (9), the AU loss for each sample is weighted by k_j, the K row of its expression class, so per-class AU statistics of the evaluation videos are encoded in the loss weight. The reported ~1 WAR/UAR gains on these datasets, and the sharp minority-class jumps (e.g., M3DFEL Disgust from 0.00% to 10.34%), are therefore partly fitted rather than transferable predictions. MAFW is not in K and offers partial independence, but the headline DFEW/FERV39k claims do not.

full rationale

The paper's core claim is that a quantitative AU-expression prior, extracted once from four datasets, can be injected into arbitrary DFER models to improve them at no inference cost. The derivation of K in Section 3.1 is not itself circular: OpenFace AU readings and M3DFEL pseudo-labels are external inputs, and the subsequent loss combination (Eqs. 5-10) is a standard auxiliary-loss construction. However, the validation protocol is self-referential for the two principal eval datasets. DFEW and FERV39k appear both in the K-estimation pool (Section 1, Section 3.1) and in the evaluation tables (Table 1), and the paper never states whether K is computed only on training folds. Because Eq. (9) uses the class-specific row of K as a multiplicative loss weight, any dataset-specific AU pattern in the evaluation videos is available to the training objective. This makes the measured gains on DFEW and FERV39k partially a fitted statistic, not an independent confirmation of transferable AU knowledge. The MAFW results, where MAFW is absent from K, provide genuine out-of-pool evidence and prevent a finding of full circularity. The M3DFEL self-citation is not load-bearing in the prohibited sense: M3DFEL is an existing, replaceable base model, and its use as pseudo-label generator is a methodological choice rather than an appeal to an unverified uniqueness theorem. The Section 6 caveat about OpenFace accuracy concerns detector quality, not the leakage issue. Overall, the central DFEW/FERV39k claim reduces in part to a fit, while the MAFW claim retains independent content; hence a partial-circularity score of 6 is appropriate.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities; its additional assumptions are about the validity of OpenFace as an AU oracle, the reliability of M3DFEL pseudo-labels, and transferability of AU-expression statistics. The most consequential free choices are the unstated threshold theta, the per-dataset lambda, and the arbitrary normalization pipeline, all of which materially affect the reported gains.

free parameters (4)
  • AU-expression knowledge matrix K (18x7 entries) = derived from OpenFace and M3DFEL pseudo-labels; exact values in Fig. 4
    All 126 weights used in Eq. 9 are computed from the four datasets, making the 'prior' dataset-dependent.
  • Frame selection threshold theta = not reported
    Section 3.1 keeps frames with predicted expression probability above theta, but theta is never given, so the knowledge matrix depends on an undisclosed choice.
  • AU loss mixing ratio lambda = varies per dataset in Fig. 5 (0.05 to 0.5)
    Selected per dataset/backbone by an ablation, so it is tuned to maximize WAR/UAR on the evaluation sets.
  • Normalization constants (2.5 shift, 5 scale) = 2.5, 5
    Hand-chosen in Section 3.1 to rescale the sigmoid output into OpenFace's range; these change the relative loss weights and are not theoretically motivated.
assumptions (4)
  • domain assumption OpenFace AU outputs are valid proxies for true facial Action Unit activity
    All AU labels (Y_AU) and the knowledge matrix K come from OpenFace 2.2.0; if OpenFace is inaccurate, both the auxiliary labels and the weights are corrupt. The paper concedes this in Section 6.
  • domain assumption M3DFEL pseudo-labels are reliable enough to group frames by expression for knowledge extraction
    Section 3.1 relies on thresholded M3DFEL predictions to assign expression labels; if these are wrong, K reflects the model's mistakes, creating self-training feedback.
  • domain assumption AU-expression statistics are transferable across datasets and models
    The method assumes the aggregate relationship measured on four datasets is beneficial for three other or same datasets and for different backbone models (Section 5.1).
  • ad hoc to paper The sigmoid normalizations produce a meaningful weight distribution
    The transformation (minus mid-range, sigmoid, sum, minus 2.5, sigmoid, times 5) is not derived from any principle and is introduced solely for this method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Action Unit Enhance Dynamic Facial Expression Recognition." pith.science (2026). https://pith.science/paper/YSH2T2Z7

@misc{pith2026250707678,
  author       = {Pith},
  title        = {Pith review of: Action Unit Enhance Dynamic Facial Expression Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YSH2T2Z7}},
  note         = {Machine review of arXiv:2507.07678}
}
read the original abstract

Dynamic Facial Expression Recognition(DFER) is a rapidly evolving field of research that focuses on the recognition of time-series facial expressions. While previous research on DFER has concentrated on feature learning from a deep learning perspective, we put forward an AU-enhanced Dynamic Facial Expression Recognition architecture, namely AU-DFER, that incorporates AU-expression knowledge to enhance the effectiveness of deep learning modeling. In particular, the contribution of the Action Units(AUs) to different expressions is quantified, and a weight matrix is designed to incorporate a priori knowledge. Subsequently, the knowledge is integrated with the learning outcomes of a conventional deep learning network through the introduction of AU loss. The design is incorporated into the existing optimal model for dynamic expression recognition for the purpose of validation. Experiments are conducted on three recent mainstream open-source approaches to DFER on the principal datasets in this field. The results demonstrate that the proposed architecture outperforms the state-of-the-art(SOTA) methods without the need for additional arithmetic and generally produces improved results. Furthermore, we investigate the potential of AU loss function redesign to address data label imbalance issues in established dynamic expression datasets. To the best of our knowledge, this is the first attempt to integrate quantified AU-expression knowledge into various DFER models. We also devise strategies to tackle label imbalance, or minor class problems. Our findings suggest that employing a diverse strategy of loss function design can enhance the effectiveness of DFER. This underscores the criticality of addressing data imbalance challenges in mainstream datasets within this domain. The source code is available at https://github.com/Cross-Innovation-Lab/AU-DFER.

Figures

Figures reproduced from arXiv: 2507.07678 by the authors.

Figure 1
Figure 1. The core architecture design diagram of our AU [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An overview of the proposed AU-DFER architecture.(I) AU-Driven loss design (II) Data-Driven Model Selection (III) [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Design conceptualization of the AU loss function. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Visualization of the AU expression knowledge ac [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Ablation Study of the AU-expression loss ratio using M3DFEL with AU-DFER on 3 datasets. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Heatmap of confusion matrix for M3DFEL in classification tasks, including: whether AU-enhanced, using on DFEW, [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: A comprehensive process for the utilization of AU expression knowledge in the training of M3DFEL on DFEW. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: t-SNE visualization [42] of dynamic facial expres￾sion features obtained by M3DFEL under different conditions: (a) only M3DFEL, (b) simply introduce AU in M3DFEL and (c) use AU in M3DFEL learning process. Different colors repre￾sent different expressions. The figure il…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Explainable graph attention network for stress recognition (StressGAT) via differential action units

    cs.CV 2026-07 conditional novelty 5.0 of 10

    StressGAT reports 88.62% LOSO accuracy for stress-vs-neutral from temporal graph attention over 10-s segments of facial Action Units (58 subjects), plus two claimed expressivity phenotypes — but the differential-AU no...

Reference graph

Works this paper leans on

47 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [2]

    Tadas Baltrušaitis, Marwa Mahmoud, and Peter Robinson. 2015. Cross-dataset learning and person-specific normalisation for automatic Action Unit detection. In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Vol. 06. 1–6. doi:10.1109/FG.2015.7284869

    work page arXiv 2015

  2. [3]

    João Carreira and Andrew Zisserman. 2017. Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 4724–4733. doi:10.1109/CVPR.2017.502

    doi:10.1109/cvpr.2017.502 2017

  3. [4]

    Dongliang Chen, Guihua Wen, Huihui Li, Pei Yang, Chuyun Chen, and Bao Wang

    work page

  4. [5]

    Weiyang Chen and Anrui Wang. 2023. Enhanced Facial Expression Recognition Based on Facial Action Unit Intensity and Region. In 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC) . 1939–1944. doi:10.1109/ SMC53992.2023.10394636

    arXiv 2023

  5. [6]

    Yin Chen, Jia Li, Shiguang Shan, Meng Wang, and Richang Hong. 2024. From static to dynamic: Adapting Landmark-Aware image models for facial expression recognition in videos. IEEE Transactions on Affective Computing (1 2024), 1–15. doi:10.1109/taffc.2024.3453443

    work page arXiv 2024

  6. [8]

    Paul Ekman and Wallace V. Friesen. 1978. Facial action coding system (facs):a technique for the measurement of facial actions.Rivista Di Psichiatria 47, 2 (1978), 126–138

    work page 1978

  7. [9]

    Paul Ekman and Erika Rosenberg. 2005. What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) . What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS)

    work page 2005

  8. [10]

    Paul Ekman and Erika L Rosenberg. 2005. What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) . Oxford University Press

    work page 2005

Show all 47 references
  1. [11]

    Weijun Gong, Yurong Qian, Weihang Zhou, and Hongyong Leng. 2024. Enhanced spatial-temporal learning network for dynamic facial expression recognition. Biomedical Signal Processing and Control 88 (2024), 105316. doi:10.1016/j.bspc. 2023.105316

    2024

  2. [12]

    Shreyank N Gowda, Boyan Gao, and David A. Clifton. 2024. FE-Adapter: Adapting Image-Based Emotion Classifiers to Videos. In 2024 IEEE 18th In- ternational Conference on Automatic Face and Gesture Recognition (FG) . 1–6. doi:10.1109/FG59268.2024.10581905

    2024

  3. [13]

    Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. 2018. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition . 6546–6555

    2018

  4. [14]

    Myungbeom Her, Hamza Ghulam Nabi, and Ji-Hyeong Han. 2025. FRU-Adapter: Frame Recalibration unit adapter for dynamic facial expression recognition. Electronics 14, 5 (2 2025), 978. doi:10.3390/electronics14050978

    2025 doi

  5. [15]

    Jayson Jeganathan, Megan Campbell, Matthew Hyett, Gordon Parker, and Michael Breakspear. 2022. Quantifying dynamic facial expressions under naturalistic conditions. eLife 11 (8 2022). doi:10.7554/elife.79581

    2022 doi

  6. [16]

    Jayson Jeganathan, Megan Campbell, Matthew P Hyett, Gordon Parker, and Michael Breakspear. 2022. Quantifying dynamic facial expressions under natu- ralistic conditions. eLife 11 (2022). https://api.semanticscholar.org/CorpusID: 248754278

    2022

  7. [17]

    Xingxun Jiang, Yuan Zong, Wenming Zheng, Chuangao Tang, Wanchuang Xia, Cheng Lu, and Jiateng Liu. 2020. Dfew: A large-scale database for recognizing dy- namic facial expressions in the wild. In Proceedings of the 28th ACM International Conference on Multimedia. 2881–2889

    2020

  8. [18]

    Rijin Jin, Sirui Zhao, Zhongkai Hao, Yifan Xu, Tong Xu, and Enhong Chen

  9. [20]

    Dimitrios Kollias and Stefanos Zafeiriou. 2018. Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition. (11 2018)

    2018

  10. [21]

    Hanting Li, Hongjing Niu, Zhaoqing Zhu, and Feng Zhao. 2023. Intensity-Aware loss for dynamic facial expression recognition in the wild. Proceedings of the AAAI Conference on Artificial Intelligence 37, 1 (6 2023), 67–75. doi:10.1609/aaai. v37i1.25077 MM ’25, Oct. 27–31, 2025,...

    2023 doi

  11. [22]

    Hanting Li, Mingzhe Sui, Zhaoqing Zhu, et al . 2022. NR-DFERNet: Noise- Robust Network for Dynamic Facial Expression Recognition. arXiv preprint arXiv:2206.04975 (2022)

    2022 arXiv

  12. [24]

    Feng Liu, Hanyang Wang, and Siyuan Shen. 2025. Robust dynamic facial expres- sion recognition. IEEE Transactions on Biometrics Behavior and Identity Science (1 2025), 1. doi:10.1109/tbiom.2025.3546279

    2025

  13. [25]

    Yuanyuan Liu, Wei Dai, Chuanxu Feng, Wenbin Wang, Guanghao Yin, Jiabei Zeng, and Shiguang Shan. 2022. MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wild . ACM, New York, NY, USA. https://doi.org/10.1145/3503161.3548190

    2022

  14. [26]

    Livingstone and Frank A

    Steven R. Livingstone and Frank A. Russo. 2018. The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. PLOS ONE 13, 5 (May 2018), e0196391. doi:10.1371/journal.pone.0196391

    2018 doi

  15. [27]

    Fuyan Ma, Bin Sun, and Shutao Li. 2022. Spatio-Temporal Transformer for Dy- namic Facial Expression Recognition in the Wild. arXiv preprint arXiv:2205.04749 (2022)

    2022 arXiv

  16. [28]

    Shuyi Mao, Xinpeng Li, Qingyang Wu, and Xiaojiang Peng. 2022. AU-Aware Vision Transformers for Biased Facial expression recognition. arXiv (Cornell University) (1 2022). doi:10.48550/arxiv.2211.06609

    2022 doi arXiv

  17. [29]

    Mehrabian

    A. Mehrabian. 2017. Nonverbal communication. doi:10.4324/9781351308724

    2017 doi

  18. [30]

    Nagrani, J

    A. Nagrani, J. S. Chung, and A. Zisserman. 2017. VoxCeleb: a large-scale speaker identification dataset. In INTERSPEECH

    2017

  19. [31]

    Ortony, G

    A. Ortony, G. Clore, and A. Collins. 1988. The cognitive structure of emotions. https://api.semanticscholar.org/CorpusID:15091825

    1988

  20. [32]

    Parr, B.M

    L.A. Parr, B.M. Waller, A.M. Burrows, K.M. Gothard, and S.J. Vick. 2010. Brief communication: MaqFACS: A muscle-based facial movement coding system for the rhesus macaque. American Journal of Physical Anthropology 143, 4 (9 2010), 625–630. doi:10.1002/ajpa.21401

    2010 doi

  21. [33]

    Zhaofan Qiu, Ting Yao, and Tao Mei. 2017. Learning spatio-temporal representa- tion with pseudo-3d residual networks. In proceedings of the IEEE International Conference on Computer Vision . 5533–5541

    2017

  22. [34]

    Partha Pratim Sarangi, Madhumita Panda, Subhashree Mishra, Bhabani Shankar Prasad Mishra, and Banshidhar Majhi. 2022. Machine Learning for Biometrics. Academic Press. https://doi.org/10.1016/C2020-0-02002-5

    2022 doi

  23. [35]

    Ziqiao Shang and Bin Liu. 2024. Facial Action Unit detection based on multi-task learning strategy for unlabeled facial images in the wild. Expert Systems with Applications 253 (5 2024), 124285. doi:10.1016/j.eswa.2024.124285

    2024

  24. [36]

    Licai Sun, Zheng Lian, Bin Liu, and Jianhua Tao. 2023. MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition. In Proceedings of the 31st ACM International Conference on Multimedia (Ottawa ON, Canada) (MM ’23). Association for Computin...

    2023

  25. [38]

    Licai Sun, Zheng Lian, Kexin Wang, Yu He, Mingyu Xu, Haiyang Sun, Bin Liu, and Jianhua Tao. 2024. SVFAP: Self-supervised video facial affect perceiver. IEEE Transactions on Affective Computing (1 2024), 1–18. doi:10.1109/taffc.2024.3436913

    2024

  26. [39]

    Y.-i. Tian, T. Kanade, and J.F. Cohn. 2001. Recognizing action units for facial ex- pression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 2 (1 2001), 97–115. doi:10.1109/34.908962

    2001 doi

  27. [40]

    Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri

  28. [41]

    Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. 2018. A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition . 6450–6459

    2018

  29. [42]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008)

    2008

  30. [43]

    Chu Wang, Jiabei Zeng, Shiguang Shan, and Xilin Chen. 2019. Multi-Task Learn- ing of Emotion Recognition and Facial Action Unit Detection with Adaptively Weights Sharing Network. In 2019 IEEE International Conference on Image Pro- cessing (ICIP). 56–60. doi:10.1109/ICIP.2019.8802914

    2019

  31. [45]

    Yan Wang, Yixuan Sun, Yiwen Huang, Zhongying Liu, Shuyong Gao, Wei Zhang, Weifeng Ge, and Wenqiang Zhang. 2022. FERV39k: A Large-Scale Multi- Scene Dataset for Facial Expression Recognition in Videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2022

  32. [46]

    Yan Wang, Yixuan Sun, Wei Song, Shuyong Gao, Yiwen Huang, Zhaoyu Chen, Weifeng Ge, and Wenqiang Zhang. 2022. DPCNet: Dual Path Multi-Excitation Collaborative Network for Facial Expression Representation Learning in Videos. In Proceedings of the 30th ACM International Conferenc...

    2022

  33. [47]

    Zhijun Zhai, Jianhui Zhao, Chengjiang Long, Wenju Xu, Shuangjiang He, and Huijuan Zhao. 2023. Feature Representation Learning with Adaptive Displace- ment Generation and Transformer Fusion for Micro-Expression Recognition. In 2023 IEEE/CVF Conference on Computer Vision and Pat...

    2023

  34. [48]

    Zengqun Zhao and Qingshan Liu. 2021. Former-DFER: Dynamic Facial Expression Recognition Transformer. In Proceedings of the 29th ACM International Confer- ence on Multimedia (Virtual Event, China) (MM ’21). Association for Computing Machinery, New York, NY, USA, 1553–1561. doi:...

    2021

  35. [49]

    Ruicong Zhi, Caixia Zhou, Tingting Li, Shuai Liu, and Yi Jin. 2020. Action unit analysis enhanced facial expression recognition by deep neural network evolution. Neurocomputing 425 (3 2020), 135–148. doi:10.1016/j.neucom.2020.03.036

    2020 doi

  36. [50]

    Ruicong Zhi, Caixia Zhou, Tingting Li, Shuai Liu, and Yi Jin. 2020. Action unit analysis enhanced facial expression recognition by deep neural network evolution. Neurocomputing 425 (2020), 135–148. https://api.semanticscholar.org/CorpusID: 216183136

    2020

  37. [2015]

    In Proceedings of the IEEE international conference on computer vision

    Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision . 4489–4497

  38. [2022]

    In 2022 IEEE International Conference on Image Processing (ICIP)

    AVT: Au-Assisted Visual Transformer for Facial Expression Recognition. In 2022 IEEE International Conference on Image Processing (ICIP) . 2661–2665. doi:10.1109/ICIP46576.2022.9897960

    2022

  39. [2024]

    Expert Systems with Applications 249 (3 2024), 123635

    Multi-geometry embedded transformer for facial expression recognition in videos. Expert Systems with Applications 249 (3 2024), 123635. doi:10.1016/j.eswa. 2024.123635

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.