Pith. sign in

REVIEW 3 major objections 5 minor 32 references

Scale-interaction transformer: a hybrid cnn-transformer model for facial beauty prediction

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The Scale-Interaction Transformer claims a new state-of-the-art for facial beauty prediction, reaching a Pearson correlation of 0.9187 on SCUT-FBP5500 by explicitly modeling interactions between multi-scale facial features.

desk verdict A clean, honestly-written hybrid architecture whose SOTA claim is under-supported by single-fold results and an incomplete table. read the letter →

arxiv 2509.05078 v1 pith:IWWP5QRE submitted 2025-09-05 cs.CV

Djamel Eddine Boukhari This is my paper · ORCID
classification cs.CV
keywords facialbeautypredictionhybridCNN-transformerscale-interactiontransformermulti-scalefeatureextractionself-attentionSCUT-FBP5500imageregressioncomputationalaesthetics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes the Scale-Interaction Transformer (SIT), a hybrid CNN-transformer built for facial beauty prediction. Its claim is that the missing ingredient in earlier models is not just multi-scale feature extraction, but explicit modeling of how features at different scales interact. On the SCUT-FBP5500 benchmark, SIT reports a Pearson correlation of 0.9187 and an RMSE of 0.2760, which would be a new state of the art, slightly ahead of the region-attentive R3CNN baseline. The paper argues that treating scale-specific features as a short sequence lets self-attention learn the local-to-global relationships that human beauty judgment depends on.

What carries the argument

The key object is the Scale-Interaction Transformer (SIT), which converts multi-scale convolutional feature maps into a 3-token sequence and applies transformer self-attention to model their relationships. The three tokens come from parallel 1x1, 3x3, and 5x5 convolutions applied to the MobileNetV2 backbone output, each pooled by global average pooling concatenated with global max pooling. The two-block, four-head transformer encoder then computes attention over these scale tokens, and the sequence is average-pooled before a dense regression head. The mechanism's work is to replace feature fusion (concatenation) with feature interaction (attention), so the final prediction is conditioned on

What would settle it

Run SIT and R3CNN on the same folds of SCUT-FBP5500 under identical preprocessing and report the per-fold mean and variance of Pearson correlation; the central claim fails if the average PC over five folds is not above 0.9142, or if the reported fold-3 gap does not reproduce.

Watch

Extended reading notes

Core claim

The central discovery the paper tries to establish is that a transformer can serve as a scale-interaction module rather than a spatial-interaction module for facial beauty regression. A pre-trained MobileNetV2 backbone produces a 7x7x1280 feature map; three parallel convolutional branches (1x1, 3x3, 5x5) turn it into three scale-specific maps; global average and max pooling compress each map into a 128-dimensional vector; and a two-block transformer encoder with four attention heads learns the dependencies among these three scale tokens. The ablation shows each component adds measurable gain: adding multi-scale features alone raises Pearson correlation from 0.8995 (backbone baseline) to 0.90

Load-bearing premise

The new-state-of-the-art claim rests on the assumption that the prior results in Table 2 were obtained under exactly the same fold, image preprocessing, and metric conventions; the paper itself states that only a single fold is evaluated, so a small protocol mismatch could erase the 0.0045 advantage.

Editorial extensions

If this is right

  • If SIT's result holds, explicit scale interaction via self-attention is a stronger aggregation strategy than concatenating multi-scale features for facial beauty prediction.
  • The ablation indicates the transformer block is the largest single contributor, implying attention over scale tokens may help other holistic regression tasks, not just facial beauty.
  • The gain from adding global max pooling alongside average pooling suggests the dominant and average response of each scale carry non-overlapping signal.
  • The reported numbers (PC 0.9187, RMSE 0.2760) on fold 3 of SCUT-FBP5500 become the benchmark point that future facial beauty prediction methods would need to beat.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same scale-token design could be adapted to apparent age estimation, where global structure and local skin texture interact; this is an extension, not a claim tested in the paper.
  • If the self-attention weights over the three scales were visualized per image, the model could reveal whether beauty judgments lean on global structure or local texture depending on the face; the paper lists explainability as future work.
  • Because the paper reports a single fold, averaging all five folds of SCUT-FBP5500 is the natural confirmation step; the stated SOTA margin of 0.0045 in Pearson correlation may shrink or vanish under fold averaging.
  • The 3-token sequence is far shorter than patch-based ViT sequences, so the architecture offers a computationally cheap way to add global relational reasoning to a CNN for regression, if the single-fold result holds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Scale-Interaction Transformer (SIT), a hybrid CNN-transformer for facial beauty prediction on SCUT-FBP5500. The architecture extracts a MobileNetV2 feature map, applies three parallel convolutions (1×1, 3×3, 5×5), pools each with GAP and GMP, stacks the resulting three 128-dimensional vectors into a sequence, and processes this sequence with a two-block transformer encoder before a regression head. The reported results are PC 0.9187, MAE 0.2180, RMSE 0.2760 on fold 3, which the authors claim is a new state-of-the-art, surpassing R3CNN (PC 0.9142, RMSE 0.2800) and prior methods. Ablations show progressive improvement from the CNN baseline to the full model, supporting the contribution of the multi-scale and transformer modules. The paper also includes a limitations section acknowledging single-fold evaluation.

Significance. If the reported result is robust, the paper would make a modest empirical contribution: it demonstrates that a small transformer operating on pooled multi-scale CNN features can improve correlation on a standard FBP benchmark, and the ablation study clearly attributes gains to the proposed components. The architecture is simple, clearly described, and reproducible in principle. The authors also explicitly admit the single-fold limitation, which is a positive sign of scientific transparency. However, the headline 'new state-of-the-art' claim is not currently supported by the evidence: the comparison table stops at R3CNN (2019) and omits recent transformer-based FBP methods, the margin over R3CNN is small relative to likely run-to-run variance, and the abstract's claim of outperforming on 'both correlation and error metrics' is contradicted by the paper's own Table 2, where R3CNN has a lower MAE.

major comments (3)
  1. [Section 4.2, Table 2] The central state-of-the-art claim is supported only by comparisons against a small, largely pre-2020 baseline set. The reference list includes several recent transformer- or attention-based FBP methods, e.g., [13] (Global Context Vision Transformer), [25] (MobileViT), [26] (ViT), and [30] (Mamba-CNN), yet none appear in Table 2. Without protocol-matched results for these methods on the same fold, the statement that SIT 'establishes a new state-of-the-art' is not substantiated. Please either include these baselines under the same evaluation protocol or explicitly justify their exclusion.
  2. [Section 5.1 and Table 2] The results are reported on a single fold (fold 3) with no standard deviation, no multiple seeds, and no statistical significance test. The paper's own limitations section admits that averaging across all five folds would be more robust. The advantage over R3CNN is PC +0.0045 and RMSE -0.004, which is within the range one would expect from initialization and training stochasticity. To support the stated ranking, the authors should report mean±std over the five folds (and preferably over multiple runs), and/or provide per-fold results demonstrating that the advantage is consistent.
  3. [Abstract, Section 1 (contribution 4), and Section 4.2] The manuscript repeatedly claims that SIT outperforms prior methods 'in both correlation and error metrics,' but Table 2 shows R3CNN achieves a lower MAE (0.2120 vs. 0.2180). The text in Section 4.2 acknowledges this trade-off, but the abstract and introduction do not. This inconsistency should be corrected, and the authors should state explicitly that SIT improves PC and RMSE at the cost of a slightly higher MAE, or provide evidence that the MAE difference is not material.
minor comments (5)
  1. [Section 3.2.1, Eqs. (3)–(5)] The spatial dimensions of the 3×3 and 5×5 convolutions are stated as 7×7, but the padding and stride are not specified. For reproducibility, please state these hyperparameters (e.g., same padding, stride 1).
  2. [Table 2] The table caption does not specify the fold/split for each baseline. Since fold 3 is used for SIT, please verify and report that all cited baseline numbers correspond to the same evaluation protocol, or note the source of each baseline value.
  3. [Table 2, 'CNN + LDL' row] The MAE and RMSE entries are missing. If these values are unavailable in the original paper, please state that explicitly; otherwise, the dash is ambiguous.
  4. [References] Several methods cited in Related Work ([13], [25], [26], [30]) are closely related to the proposed approach; adding them to the comparison table (or explaining their omission) would also help align the narrative with the reference list.
  5. [Throughout] There are minor typographical issues: the running header 'APREPRINT', the title/header spacing in 'CNN-TRANSFORMERMODEL', and 'de factostandard' should be fixed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the SOTA claim is an empirical measurement, not a derivation from its own outputs.

full rationale

This paper is an empirical architecture study. The headline result (PC 0.9187, RMSE 0.2760) is a measured test-fold statistic, not a quantity solved for from the model equations. The forward chain—MobileNetV2 features (Eq. 2), multi-scale convolutions (Eqs. 3–5), pooling and stacking (Eqs. 6–8), transformer blocks (Eqs. 9–10), and regression head (Eq. 11)—constructs a prediction from the input image; no equation is defined in terms of the target score or the reported benchmark result. The claim that modeling scale interactions helps is supported by an ablation study (Table 3) rather than by construction, and removing the transformer or GMP changes the architecture in a way that is not tautologically tied to the final score. The numerous self-citations ([3,6,10,13,23,25,26,29,30]) appear in related-work, context, or prior-art roles and are not used to justify the numerical outcome; the decisive comparison is against the external R3CNN baseline [19]. The paper's own limitation section (Section 5.1) explicitly admits that results are reported on a single fold, and the comparison set is small, but that is a robustness and completeness concern, not circularity. No fitted parameter is renamed as a prediction, and no uniqueness theorem or prior author result is imported to force the architecture. The central claim retains independent empirical content, so the appropriate finding is no significant circularity.

Assumptions & free parameters 12 free parameters · 7 assumptions · 1 invented entities

The central claim rests on hand-chosen architecture hyperparameters, the single-fold protocol, and comparability of prior published numbers. No new physical entities are introduced; the only invented item is the model architecture itself.

free parameters (12)
  • Projection dimension D_proj = 128
    Sets the transformer width; chosen by hand.
  • Number of transformer blocks L = 2
    Depth chosen by hand; ablation only tests presence versus absence, not depth.
  • Number of attention heads = 4
    Chosen by hand.
  • FFN dimension = 512
    Chosen by hand.
  • Dropout rate = 0.1
    Chosen by hand for both attention and regression head.
  • Initial learning rate = 1e-4
    Adam step size, chosen by hand.
  • Batch size = 32
    Chosen by hand.
  • Max epochs, early stopping patience, LR patience, LR factor = 300, 10, 5, 0.5
    Training schedule choices.
  • Multi-scale kernel sizes = 1x1, 3x3, 5x5
    Choice of scales is a free design decision.
  • Pooling strategy = GAP+GMP
    Dual pooling chosen by hand and validated in the ablation, but only on one fold.
  • Backbone feature depth = 7x7x1280 to 3 branches of 7x7x64
    Architecture width choices.
  • Selected fold = fold 3
    Single split used for all reported results; not averaged over folds.
assumptions (7)
  • domain assumption SCUT-FBP5500 beauty scores (mean of 60 raters) are treated as ground truth for human beauty perception.
    Underlies all training and evaluation; used in Section 3.1.
  • domain assumption The third fold of the 5-fold protocol is representative of the full dataset and directly comparable to prior published fold-3 numbers.
    Used in Sections 3.1 and 4.1; the paper's own limitation section says single-fold results may be split-sensitive.
  • domain assumption MobileNetV2 features pretrained on ImageNet transfer to facial beauty.
    Backbone initialization described in Section 3.5.
  • ad hoc to paper A 3-token transformer with self-attention over pooled scale vectors can capture scale interactions.
    Core design hypothesis in Section 3.2.2; no independent evidence beyond the ablation on one fold.
  • domain assumption Table 2 prior results were obtained under the same protocol (fold, preprocessing, metrics).
    Needed for the SOTA claim; not verifiable from the paper.
  • domain assumption MSE loss is a suitable objective for score regression.
    Standard choice but assumes roughly symmetric error; used in Eq. 12.
  • standard math Standard transformer block definitions (LayerNorm, residual, FFN) hold as described.
    Algorithm 1 relies on standard operations.
invented entities (1)
  • Scale-Interaction Transformer (SIT) architecture
    purpose: To model inter-dependencies between multi-scale CNN features for facial beauty regression
    The paper introduces SIT as a new model; its only evidence is a single dataset/fold, so no external falsifiable handle is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scale-interaction transformer: a hybrid cnn-transformer model for facial beauty prediction." pith.science (2026). https://pith.science/paper/IWWP5QRE

@misc{pith2026250905078,
  author       = {Pith},
  title        = {Pith review of: Scale-interaction transformer: a hybrid cnn-transformer model for facial beauty prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IWWP5QRE}},
  note         = {Machine review of arXiv:2509.05078}
}
read the original abstract

Automated Facial Beauty Prediction (FBP) is a challenging computer vision task due to the complex interplay of local and global facial features that influence human perception. While Convolutional Neural Networks (CNNs) excel at feature extraction, they often process information at a fixed scale, potentially overlooking the critical inter-dependencies between features at different levels of granularity. To address this limitation, we introduce the Scale-Interaction Transformer (SIT), a novel hybrid deep learning architecture that synergizes the feature extraction power of CNNs with the relational modeling capabilities of Transformers. The SIT first employs a multi-scale module with parallel convolutions to capture facial characteristics at varying receptive fields. These multi-scale representations are then framed as a sequence and processed by a Transformer encoder, which explicitly models their interactions and contextual relationships via a self-attention mechanism. We conduct extensive experiments on the widely-used SCUT-FBP5500 benchmark dataset, where the proposed SIT model establishes a new state-of-the-art. It achieves a Pearson Correlation of 0.9187, outperforming previous methods. Our findings demonstrate that explicitly modeling the interplay between multi-scale visual cues is crucial for high-performance FBP. The success of the SIT architecture highlights the potential of hybrid CNN-Transformer models for complex image regression tasks that demand a holistic, context-aware understanding.

Figures

Figures reproduced from arXiv: 2509.05078 by the authors.

Figure 1
Figure 1. Input Image R 224×224×3 MobileNetV2 Backbone (Pre-trained) Conv 1x1 Conv 3x3 Conv 5x5 R 7×7×64 R 7×7×64 R 7×7×64 GAP ⊕ GMP GAP ⊕ GMP GAP ⊕ GMP R 128 R 128 R 128 Stacking & Projection R 3×128 → R 3×Dproj Transformer Encoder (L=2 Blocks) Transformer Block Multi-Head Attention + Feed-Forward Network + Global Avg Pooling (Sequence) R Dproj Regression Head (Dropout + Dense) Predicted Score yˆ ∈ R Inorm Fbase ∈ R 7×7×1280… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 30 canonical work pages

  1. [13]

    Facial Beauty Prediction Using Global Context Vision Transformer,

    D. Eddine Boukhari, A. Chemsa and Z. -E. Baarir, "Facial Beauty Prediction Using Global Context Vision Transformer," 2025 International Symposium on iNnovative Informatics of Biskra (ISNIB), Biskra, Algeria, 2025

    work page 2025

  2. [25]

    MobileViT architecture for Facial Beauty Prediction

    Djamel Eddine Boukhari, Ali Chemsa, and Zine-Eddine Baarir. "MobileViT architecture for Facial Beauty Prediction." 2024 International Conference on Telecommunications and Intelligent Systems (ICTIS). IEEE, 2024

    work page 2024

  3. [26]

    Facial Beauty Prediction Based on Vision Transformer

    Djamel Eddine Boukhari, Ali Chemsa, and Riadh Ajgou. "Facial Beauty Prediction Based on Vision Transformer." International Journal of Electrical and Electronic Engineering and Telecommunications, ISSN (2023): 2319-2518

    work page 2023

  4. [30]

    Mamba-CNN: A Hybrid Architecture for Efficient and Accurate Facial Beauty Prediction

    Boukhari, Djamel Eddine. "Mamba-CNN: A Hybrid Architecture for Efficient and Accurate Facial Beauty Prediction." arXiv preprint arXiv:2509.01431 (2025)

    work page Pith review arXiv 2025

  5. [1]

    Zhang, F

    D. Zhang, F. Chen, and Y . Xu, Computer Models for Facial Beauty Analysis, Switzerland: Springer International Publishing, 2016

    work page 2016

  6. [2]

    Ranking facial attractiveness,

    H. Knight and O. Keith, “Ranking facial attractiveness,” The European Journal of Orthodontics, vol. 27, no. 4 pp. 340-348, 2005

    work page 2005

  7. [3]

    A comprehensive review of facial beauty prediction using deep learning techniques

    Djamel Eddine Boukhari,et al. "A comprehensive review of facial beauty prediction using deep learning techniques." Engineering Applications of Artificial Intelligence 161 (2025): 112009

    work page 2025

  8. [4]

    The role of the golden proportion in the evaluation of facial esthetics

    Rossetti, Alberto, et al. "The role of the golden proportion in the evaluation of facial esthetics." The Angle Orthodontist 83.5 (2013): 801-808

    work page 2013

Show all 32 references
  1. [5]

    Computation of a face attractiveness index based on neoclassical canons, symmetry, and golden ratios

    Schmid, Kendra, David Marx, and Ashok Samal. "Computation of a face attractiveness index based on neoclassical canons, symmetry, and golden ratios." Pattern Recognition 41.8 (2008): 2710-2717

    2008

  2. [6]

    D. E. Boukhari, A. Chemsa, R. Ajgou, et al., An Ensemble of Deep Convolutional Neural Networks Models for Facial Beauty Prediction, Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 27 no

  3. [7]

    O’shea, K., and Nash, R. (2015). An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458

    2015 arXiv

  4. [8]

    Imagenet classification with deep convolutional neural networks

    Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems 25 (2012)

    2012

  5. [9]

    K. He, X. Zhang, S. Ren et al., Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV , USA, pp. 770-778, 2016

    2016

  6. [10]

    Facial Beauty Prediction Using an Ensemble of Deep Convolutional Neural Networks

    Djamel Eddine Boukhari, et al. "Facial Beauty Prediction Using an Ensemble of Deep Convolutional Neural Networks." Engineering Proceedings 56.1 (2023): 125

    2023

  7. [11]

    Natural language processing

    Chowdhary, KR1442. "Natural language processing." Fundamentals of artificial intelligence (2020): 603-649

    2020

  8. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, Alexey, et al. "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929 (2020)

    2010 arXiv

  9. [14]

    Vision research, vol 48, no 2, pp

    A Kagian, G Dror, T Leyvand, et al., A machine learning predictor of facial attractiveness revealing human-like psychophysical biases. Vision research, vol 48, no 2, pp. 235-243, 2008

    2008

  10. [15]

    Transfer learning adaptive facial attractiveness assessment

    I Lebedeva,Y Guo and F Ying. Transfer learning adaptive facial attractiveness assessment. Journal of Physics: Conference Series. vol. 1922, no. 1, 2021

    1922

  11. [16]

    Cao, K Choi, H Jung et al., Deep learning for facial beauty prediction

    K. Cao, K Choi, H Jung et al., Deep learning for facial beauty prediction. Information, vol. 11, no. 8, 2020

    2020

  12. [17]

    Label distribution-based facial attractiveness computation by deep residual learning

    Fan, Yang-Yu, et al. "Label distribution-based facial attractiveness computation by deep residual learning." IEEE Transactions on Multimedia 20.8 (2017): 2196-2208

    2017

  13. [18]

    Dynamic attentive convolution for facial beauty prediction

    Sun, Zhishu, et al. "Dynamic attentive convolution for facial beauty prediction." IEICE TRANSACTIONS on Information and Systems 107.2 (2024): 239-243

    2024

  14. [19]

    Regression Guided by Relative Ranking Using Convolutional Neural Network (R3CNN) for Facial Beauty Prediction

    Lin, L.; Liang, L.; Jin, L. Regression Guided by Relative Ranking Using Convolutional Neural Network (R3CNN) for Facial Beauty Prediction. IEEE Trans. Affect. Comput. 2019, 1

    2019

  15. [20]

    EEE Access, vol

    J Gan, L Xiang, Y Zhai, et al., 2M BeautyNet: Facial beauty prediction based on multi-task transfer learning. EEE Access, vol. 8, pp. 20245-20256, 2020. 10 APREPRINT- SEPTEMBER8, 2025

    2020

  16. [21]

    Rethinking the inception architecture for computer vision

    Szegedy, Christian, et al. "Rethinking the inception architecture for computer vision." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016

    2016

  17. [22]

    Feature pyramid networks for object detection

    Lin, Tsung-Yi, et al. "Feature pyramid networks for object detection." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017

    2017

  18. [23]

    Generative Pre-training for Subjective Tasks: A Diffusion Transformer-Based Frame- work for Facial Beauty Prediction

    Boukhari, Djamel Eddine. "Generative Pre-training for Subjective Tasks: A Diffusion Transformer-Based Frame- work for Facial Beauty Prediction." arXiv preprint arXiv:2507.20363 (2025)

    2025 arXiv

  19. [24]

    Vision transformers for image classification: A comparative survey

    Wang, Yaoli, et al. "Vision transformers for image classification: A comparative survey." Technologies 13.1 (2025): 32

    2025

  20. [27]

    Liang, L

    L. Liang, L. Lin, L. Jin et al., SCUT-FBP5500: A diverse benchmark dataset for multi-paradigm facial beauty prediction. 24th International Conference on Pattern Recognition (ICPR), Beijing, China, pp. 1598-1603, 2018

    2018

  21. [28]

    Geometric prior guided hybrid deep neural network for facial beauty analysis

    T. Peng, M. Li, F. Chen, et al., "Geometric prior guided hybrid deep neural network for facial beauty analysis." CAAI Transactions on Intelligence Technology, pp. 1–14, 2023

    2023

  22. [29]

    An Uncertainty-Aware and Explainable Deep Learning Model for Facial Beauty Prediction

    Boukhari, Djamel Eddine, and Ali Chemsa. "An Uncertainty-Aware and Explainable Deep Learning Model for Facial Beauty Prediction." (2025)

    2025

  23. [31]

    Personalized facial beauty assessment: a meta-learning approach

    I Lebedeva, F Ying, and Y Guo. Personalized facial beauty assessment: a meta-learning approach. The Visual Computer: International Journal of Computer Graphics, V ol. 39, no. 3,pp. 1095–1107, 2023

    2023

  24. [32]

    Improving Facial Attractiveness Prediction via Co-attention Learning,

    S. Shi, F. Gao, X. Meng, et al., "Improving Facial Attractiveness Prediction via Co-attention Learning," IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, pp. 4045-4049, 2019. . 11

    2019

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.