REVIEW 4 major objections 7 minor 41 references
Achieving 3D Attention via Triplet Squeeze and Excitation Block
T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A new attention block combining Triplet Attention and Squeeze-and-Excitation reports 78.27% on FER2013.
desk verdict An honest engineering paper whose controlled comparison supports the TripSE block, but the headline 78.27% FER2013 SOTA is a single selected run and needs seeds and a clear test/validation statement before it can be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the TripSE attention block. It takes TA's three rotational branches, operating over the $C \times W$, $C \times H$, and $W \times H$ faces of the tensor, each producing a 2D attention map through z-pooling, convolution, batch normalisation, and sigmoid, and fuses each map with the output of an SE module that computes global channel weights from the permuted tensor. Depending on the variant, the SE sits before, after, or parallel to the TA branch; TripSE3 and TripSE4 run SE in parallel with each branch, and TripSE4 adds a shift before the sigmoid and a final SE at the end, yielding a full 3D weighting tensor used to scale the branch input. The block's work is to let inter-dimensional attention and global channel importance modulate each other before the three branches are combined.
What would settle it
Re-run ConvNeXt-S plus TripSE4 ($r=1$) on FER2013 under the standard protocol: fix the configuration before evaluation, report the split, run multiple seeds, and compare the mean and variance against the 76.82% baseline. If the mean is below 78.27% or within noise of the plain ConvNeXt-S score of 77.19%, the state-of-the-art claim is not sustained.
Extended reading notes
Core claim
On its own terms, the paper discovers that the gap between TA's cross-dimension attention and SE's global channel weighting can be closed by a multiplicative, and in TripSE4 affine, fusion of the two, producing a 3D attention tensor. In a controlled comparison on FER2013, ConvNeXt-T with TripSE1 reaches 77.86%, above the reported reproducible baseline of 76.82% from a residual masking network ensemble, and ConvNeXt-S with TripSE4 ($r=1$) reaches 78.27%. The authors attribute the success of TripSE4 to its parallel SE branches with a shift-and-scale transformation per rotational channel and a final SE at branch unification, which they say preserves information better than simple averaging. They also report that adding SE alone or TA alone to ConvNeXt lowers FER2013 accuracy relative to the plain backbone, whereas TripSE raises it.
Load-bearing premise
The headline 78.27% result rests on the assumption that a single run, whose split is not stated and whose configuration was chosen after seeing all variant results, is a trustworthy estimate of generalization on FER2013.
Editorial extensions
If this is right
- If the FER2013 result holds, a simple plug-in attention block can lift existing CNN backbones above published facial-expression state of the art without an architectural redesign.
- The controlled comparisons imply that TA alone and SE alone are not enough on this task; their fusion is what produces the reported gain.
- The near-zero parameter overhead (28.6 to 28.7 million on ConvNeXt-T) means the block is cheap enough to add to deployed models.
- The AffectNet improvement, from 62.87% to 64.16%, suggests the benefit is not specific to a single dataset.
- The underperformance of TripSE2 and TripSE3 relative to TripSE1 and TripSE4 is presented as evidence that 3D scaling and a final SE at unification are load-bearing design choices.
Reading between the lines
- The reported 78.27% comes from a configuration selected after seeing multiple variant and ratio results, so independent multi-seed reproduction with a pre-registered configuration is needed to know how much of the gap over baselines is real.
- The same fusion idea may transfer to other small-sample vision tasks, such as medical image classification, where global channel statistics are informative and overfitting risk is high.
- A natural testable extension is to vary the reduction ratio and branch placement systematically across more backbones, and to measure whether TripSE's benefit grows with the number of inserted blocks.
- Because the paper does not state whether the FER2013 number is measured on the test or validation split, any comparison against published leaderboard results is provisional until the protocol is clarified.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a family of attention blocks, TripSE1-4, that combine Triplet Attention (TA) and Squeeze-and-Excitation (SE) to produce what the authors call 3D attention maps. The blocks are inserted after each group of blocks in ResNet18, DenseNet, and ConvNeXt, and the resulting models are evaluated on CIFAR100, ImageNet, FER2013, and AffectNet. The central claim is that ConvNeXt-S with TripSE4 (r=1) achieves 78.27% accuracy on FER2013, described as a new state-of-the-art and 'an unprecedented feat' (Section V.C, Table III). The paper also reports a controlled comparison on FER2013 in which TripSE1 outperforms both TA-only and SE-only variants under the same finetuning protocol.
Significance. If the central claim holds, the work offers a simple, parameter-efficient way to improve CNN-based facial expression recognition: the TripSE block adds only about 0.1M parameters to ConvNeXt-T (28.6M vs 28.7M) while reportedly gaining over one point on FER2013 relative to the ConvNeXt-S baseline and 1.45 points over the stated reproducible baseline of 76.82% (Section V.C, Table III). The controlled comparison on FER2013 is a genuine strength: under the same finetuning protocol, TripSE1 reaches 78.08% against 76.51% for TA and 76.93% for SE, which supports the claim that the proposed combination, rather than a generic attention module, is responsible for the gain. The significance is conditional, however, on the 78.27% headline number being a reproducible estimate of generalization; the paper provides no seeds, no error bars, and no explicit statement of whether the result is measured on the official test split. The paper does not include code or any machine-checked derivations, so the contribution rests entirely on the empirical results.
major comments (4)
- [Section V.C, Table III] The state-of-the-art claim of 78.27% rests on a single run with no seeds, no standard deviations, and no explicit statement of whether FER2013 accuracy is measured on the official test split (3,589 images) or the validation split. Because the headline number is the maximum over four variants and two reduction ratios (eight configurations reported in Table III), the result may reflect selection on the test set rather than a genuine improvement. Please report mean and standard deviation over at least three seeds, state clearly which split is used, and either preselect the variant and r before evaluation or apply an appropriate multiple-comparison correction.
- [Table III and Section V.C] The internal evidence for the robustness of the proposal is mixed. ConvNeXt-S + TripSE1 (r=1) scores 76.61%, below the 77.19% baseline, and ConvNeXt-S + TripSE4 (r=16) scores 76.71%, also below baseline, while the r=16/r=1 counterparts achieve 78.08% and 78.27%. The text states that TripSE1 'consistently demonstrated the effectiveness' of the block and that the original ConvNeXt-S failed to gain when scaled up, but the table shows the improvement disappears or reverses for some configurations. This instability requires explanation; at minimum, all runs, the selection procedure, and the sensitivity to r should be reported and discussed honestly.
- [Section III.C and III.D] The method is specified only verbally and through figures. Terms such as 'rotational channels', 'rotational maps', 'multiplicative or translational broadcasting', and the exact position of the SE block in each variant are not defined with equations or tensor-shape diagrams. This makes the four variants impossible to reproduce or formally check. Please provide precise mathematical definitions of TripSE1-4, including where the SE block is applied, how the 1D SE vector is expanded and combined with the 2D TA attention map, and what operation occurs at branch unification.
- [Section IV.B] The implementation details omit key protocol elements needed to interpret the FER2013 numbers: the evaluation split (validation vs test), any normalization used for the resized 224x224 images, and whether the learning-rate scheduler is driven by validation accuracy and then followed by evaluation on the test set. Given that the FER2013 test labels are public and that validation-set evaluation is common in this literature, the paper must state explicitly which split produced the reported numbers and how early stopping was applied.
minor comments (7)
- [Abstract] The abstract says 'a new feet for this dataset' and should read 'a new feat'.
- [Table II footnote] The footnote contains a typo: 'Trained from scrtach' should be 'Trained from scratch'.
- [Section V.C] The text reports the ConvNeXt-T baseline as 77.20%, while Table III reports 77.19% for the same model; the inconsistency should be corrected.
- [Table III] The rows for ConvNeXt-T + SE and ConvNeXt-T + TA are labeled inconsistently; the SE row has no reduction ratio, and the TA row is listed as ConvNeXt-T + TA1 but discussed as the TA block. Please add the exact hyperparameters used for these baselines.
- [III.D] Figure 2's caption contains 'visa-versa', which should be 'vice versa'.
- [III.C] The sentence 'we consolidate the resultant 3 3D rotational attention maps' appears to contain a duplicated '3'; it should read 'three 3D'.
- [References] Reference [33] is malformed: the author list contains a garbled fragment ('and D. H., "Lee,"Y. Zhou, ...') that should be cleaned up.
Circularity Check
No significant circularity: TripSE is an empirical architecture combination and the FER2013 SOTA claim is a reported measurement, not a quantity derived from its own inputs.
full rationale
The paper does not contain a derivation chain that could reduce to its own inputs. TripSE is defined in Section III.C as an explicit combination of the external Triplet Attention module [12] and Squeeze-and-Excitation [9]; the four variants in Section III.D are architectural compositions described by forward-pass operations, and the reported accuracies in Table III are empirical outcomes of finetuning on FER2013. No equation in the paper defines TripSE's expected accuracy as a function of baseline accuracy, and no fitted parameter is renamed as a prediction. The winning configuration (TripSE4, r=1) is selected after observing evaluation results across several variants, which is a model-selection and statistical-robustness concern (single run, unspecified test/validation split), not a circularity reduction. The only self-citation is [30] (Alhazmi and Altahhan), used in Section II.B to report that Triplet Attention performed best among CBAM/BAM/TA in prior work; this background claim is not load-bearing for the central empirical result, which is compared against many external methods in Table III. There is no uniqueness theorem, no imported ansatz, and no renaming of a known result: the block outputs are new combinations, and the SOTA claim is externally falsifiable by reproducing the stated training procedure. Therefore no circular step is exhibited.
Assumptions & free parameters
free parameters (2)
- SE reduction ratio r =
1 (best FER2013), also 16
- TripSE variant selection =
TripSE4 for headline; TripSE1 for most claims
assumptions (3)
- domain assumption Applying learned attention maps via element-wise multiplication improves feature discrimination in CNNs.
- ad hoc to paper Rotating feature tensors along W and H channels creates meaningful 'rotational channels' that can be Squeeze-and-Excitation weighted.
- domain assumption Pretrained ImageNet weights remain effective when four new attention blocks are inserted after each group of blocks.
Cite this review
Pith. "Pith review of Achieving 3D Attention via Triplet Squeeze and Excitation Block." pith.science (2026). https://pith.science/paper/WFG6KYX2
@misc{pith2026250505943,
author = {Pith},
title = {Pith review of: Achieving 3D Attention via Triplet Squeeze and Excitation Block},
year = {2026},
howpublished = {\url{https://pith.science/paper/WFG6KYX2}},
note = {Machine review of arXiv:2505.05943}
}
read the original abstract
The emergence of ConvNeXt and its variants has reaffirmed the conceptual and structural suitability of CNN-based models for vision tasks, re-establishing them as key players in image classification in general, and in facial expression recognition (FER) in particular. In this paper, we propose a new set of models that build on these advancements by incorporating a new set of attention mechanisms that combines Triplet attention with Squeeze-and-Excitation (TripSE) in four different variants. We demonstrate the effectiveness of these variants by applying them to the ResNet18, DenseNet and ConvNext architectures to validate their versatility and impact. Our study shows that incorporating a TripSE block in these CNN models boosts their performances, particularly for the ConvNeXt architecture, indicating its utility. We evaluate the proposed mechanisms and associated models across four datasets, namely CIFAR100, ImageNet, FER2013 and AffectNet datasets, where ConvNext with TripSE achieves state-of-the-art results with an accuracy of \textbf{78.27\%} on the popular FER2013 dataset, a new feat for this dataset.
Figures
Reference graph
Works this paper leans on
-
[1]
Gradient-based learning applied to document recognition,
Y . Lecun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998
1998
-
[2]
Imagenet classifica- tion with deep convolutional neural networks,
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classifica- tion with deep convolutional neural networks,” in Advances in Neural Information Processing Systems , F. Pereira, C. Burges, L. Bottou, and K. Weinberger, Eds., vol. 25. Curran Associates, Inc., 2012
2012
-
[3]
Going deeper with convolutions,
C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V . Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” 2014. [Online]. Available: https://arxiv.org/abs/1409.4842
arXiv 2014
-
[4]
Very deep convolutional networks for large-scale image recognition,
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014. [Online]. Available: https: //arxiv.org/abs/1409.1556
arXiv 2014
-
[5]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015. [Online]. Available: https://arxiv.org/abs/1512.03385
arXiv 2015
-
[6]
Mobilenetv2: Inverted residuals and linear bottlenecks,
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” 2018. [Online]. Available: https://arxiv.org/abs/1801.04381
arXiv 2018
-
[7]
A. V . Savchenko, “Facial expression and attributes recognition based on multi-task learning of lightweight neural networks,” in 2021 IEEE 19th International Symposium on Intelligent Systems and Informatics (SISY) , 2021, pp. 119–124
work page 2021
-
[8]
Densely connected convolutional networks,
G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” 2016. [Online]. Available: https://arxiv.org/abs/1608.06993
arXiv 2016
Show all 41 references
-
[9]
Squeeze-and-excitation networks,
J. Hu, L. Shen, S. Albanie, G. Sun, and E. Wu, “Squeeze-and-excitation networks,” 2017. [Online]. Available: https://arxiv.org/abs/1709.01507
2017 arXiv
-
[10]
Cbam: Convolutional block attention module,
S. Woo, J. Park, J.-Y . Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” 2018. [Online]. Available: https://arxiv.org/ abs/1807.06521
2018 arXiv
-
[11]
Bam: Bottleneck attention module,
J. Park, S. Woo, J.-Y . Lee, and I. S. Kweon, “Bam: Bottleneck attention module,” 2018. [Online]. Available: https://arxiv.org/abs/1807.06514
2018 arXiv
-
[12]
Rotate to attend: Convolutional triplet attention module,
D. Misra, T. Nalamada, A. U. Arasanipalai, and Q. Hou, “Rotate to attend: Convolutional triplet attention module,” 2020. [Online]. Available: https://arxiv.org/abs/2010.03045
2020 arXiv
-
[13]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2017. [Online]. Available: https://arxiv.org/abs/1706.03762
2017 arXiv
-
[14]
An image is worth 16x16 words: Transformers for image recognition at scale,
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” 2020. [Online]. Available: https://arxi...
2020 arXiv
-
[15]
Training data-efficient image transformers & distillation through attention,
H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J ´egou, “Training data-efficient image transformers & distillation through attention,” 2021
2021
-
[16]
A convnet for the 2020s,
Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022
2022
-
[17]
Emonext: an adapted convnext for facial emotion recognition,
Y . El Boudouri and A. Bohi, “Emonext: an adapted convnext for facial emotion recognition,” in 2023 IEEE 25th International Workshop on Multimedia Signal Processing (MMSP) , 2023, pp. 1–6
2023
-
[18]
Spatial transformer networks,
M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu, “Spatial transformer networks,” 2015. [Online]. Available: https: //arxiv.org/abs/1506.02025
2015 arXiv
-
[19]
An attentive survey of attention models,
S. Chaudhari, V . Mithal, G. Polatkan, and R. Ramanath, “An attentive survey of attention models,” 2019. [Online]. Available: https://arxiv.org/abs/1904.02874
2019 arXiv
-
[20]
Attention mechanisms in computer vision: A survey,
M.-H. Guo, T.-X. Xu, J.-J. Liu, Z.-N. Liu, P.-T. Jiang, T.-J. Mu, S.-H. Zhang, R. R. Martin, M.-M. Cheng, and S.-M. Hu, “Attention mechanisms in computer vision: A survey,” Computational Visual Media, vol. 8, no. 3, pp. 331–368, mar 2022. [Online]. Available: https://doi.org/10.1007
2022
-
[21]
Global second-order pooling convolutional networks,
Z. Gao, J. Xie, Q. Wang, and P. Li, “Global second-order pooling convolutional networks,” 2018. [Online]. Available: https: //arxiv.org/abs/1811.12006
2018 arXiv
-
[22]
Srm : A style-based recalibration module for convolutional neural networks,
H. Lee, H.-E. Kim, and H. Nam, “Srm : A style-based recalibration module for convolutional neural networks,” 2019. [Online]. Available: https://arxiv.org/abs/1903.10829
2019 arXiv
-
[23]
Gated channel transformation for visual recognition,
Z. Yang, L. Zhu, Y . Wu, and Y . Yang, “Gated channel transformation for visual recognition,” 2019. [Online]. Available: https://arxiv.org/abs/ 1909.11519
2019 arXiv
-
[24]
Eca-net: Efficient channel attention for deep convolutional neural networks,
Q. Wang, B. Wu, P. Zhu, P. Li, W. Zuo, and Q. Hu, “Eca-net: Efficient channel attention for deep convolutional neural networks,”
-
[25]
Fcanet: Frequency channel attention networks,
Z. Qin, P. Zhang, F. Wu, and X. Li, “Fcanet: Frequency channel attention networks,” 2020. [Online]. Available: https://arxiv.org/abs/2012.11879
2020 arXiv
-
[26]
Recurrent models of visual attention,
V . Mnih, N. Heess, A. Graves, and K. Kavukcuoglu, “Recurrent models of visual attention,” 2014. [Online]. Available: https://arxiv.org/ abs/1406.6247
2014 arXiv
-
[27]
Show, attend and tell: Neural image caption generation with visual attention,
K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y . Bengio, “Show, attend and tell: Neural image caption generation with visual attention,” in Proceedings of the 32nd International Conference on Machine Learning , ser. Proceedings of Machine Learni...
2015
-
[28]
Non-local neural networks,
X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” 2017. [Online]. Available: https://arxiv.org/abs/1711.07971
2017 arXiv
-
[29]
Residual attention network for image classification,
F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang, “Residual attention network for image classification,” 2017. [Online]. Available: https://arxiv.org/abs/1704.06904
2017 arXiv
-
[30]
Best fit activation functions for attention mechanism: Comparison and enhancement,
M. Alhazmi and A. Altahhan, “Best fit activation functions for attention mechanism: Comparison and enhancement,” in 2023 International Joint Conference on Neural Networks (IJCNN) , 2023, pp. 1–8
2023
-
[31]
Facial expression recognition using residual masking network,
L. Pham, T. H. Vu, and T. A. Tran, “Facial expression recognition using residual masking network,” in 2020 25th International Conference on Pattern Recognition (ICPR), 2021, pp. 4513–4519
2020
-
[32]
A novel facial emotion recognition model using segmentation vgg- 19 architecture - international journal of information technology,
S. Vignesh, M. Savithadevi, M. Sridevi, and R. Sridhar, “A novel facial emotion recognition model using segmentation vgg- 19 architecture - international journal of information technology,” Mar 2023. [Online]. Available: https://link.springer.com/article/10.1007/ s41870-023-01...
2023
-
[33]
I. J. Goodfellow, D. Erhan, P. L. Carrier, A. Courville, M. Mirza, B. Hamner, W. Cukierski, Y . Tang, D. Thaler, and D. H., “Lee,”Y. Zhou, C. Ramaiah, F. Feng, R. Li, X. Wang, D. Athanasakis, J. Shawe-Taylor, M. Milakov, J. Park, R. Ionescu, M. Popescu, C. Grozea, J. Bergstra,...
2015
-
[34]
Deep-emotion: Facial expression recognition using attentional convolutional network,
S. Minaee, M. Minaei, and A. Abdolrashidi, “Deep-emotion: Facial expression recognition using attentional convolutional network,” Sensors, vol. 21, no. 9, 2021. [Online]. Available: https://www.mdpi. com/1424-8220/21/9/3046
2021
-
[35]
Facial emotion recognition: State of the art performance on fer2013,
Y . Khaireddin and Z. Chen, “Facial emotion recognition: State of the art performance on fer2013,” 2021
2021
-
[36]
Ad-corre: Adaptive correlation-based loss for facial expression recognition in the wild,
A. P. Fard and M. H. Mahoor, “Ad-corre: Adaptive correlation-based loss for facial expression recognition in the wild,” IEEE Access, vol. 10, pp. 26 756–26 768, 2022
2022
-
[37]
Facial expression recogni- tion with deep learning,
A. Khanzada, C. Bai, and F. T. Celepcikay, “Facial expression recogni- tion with deep learning,” 2020
2020
-
[38]
A novel facial emotion recognition model using segmentation vgg- 19 architecture,
S. Vignesh, M. Savithadevi, M. Sridevi, and R. Sridhar, “A novel facial emotion recognition model using segmentation vgg- 19 architecture,” International Journal of Information Technology , vol. 15, no. 4, pp. 1777–1787, Apr 2023. [Online]. Available: https://doi.org/10.1007/s...
2023 doi
-
[39]
Local multi-head channel self- attention for facial expression recognition,
R. Pecoraro, V . Basile, and V . Bono, “Local multi-head channel self- attention for facial expression recognition,” Information, vol. 13, no. 9,
-
[2019]
Available: https://arxiv.org/abs/1910.03151
[Online]. Available: https://arxiv.org/abs/1910.03151
1910 arXiv
-
[2022]
Available: https://www.mdpi.com/2078-2489/13/9/419
[Online]. Available: https://www.mdpi.com/2078-2489/13/9/419
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.