{"id":"eb977062-99b1-4e42-8913-713ce0f51971","arxiv_id":"2412.18988","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"MTCAE-DFER, a cascaded multi-task transformer with a VideoMAE encoder, reports state-of-the-art dynamic facial expression recognition results on RAVDESS, CREMA-D, and MEAD.","lead":"A new model for reading emotions from video, MTCAE-DFER, cascades face detection and landmark tasks before expression classification, reporting top scores on three public video datasets. The comparison design leaves the true size of the improvement uncertain.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MTCAE-DFER's SOTA claim rests on Tables 2-4 quoting external baselines without matching splits, frame sampling, or preprocessing; if protocols differ, the 3.7-4.3 WAR margins over MTFormer may be experimental artifacts.","rationale":"Reader's weakest assumption identifies protocol mismatch in external SOTA comparisons; my read agrees. The paper's internal ablation is well designed (six architectures) and the cascade vs non-fully-shared ViT decoder comparison (Table 1) isolates the cascade mechanism from decoder type. But the strongest claim in Section 4.5 depends on quoted numbers whose protocol provenance is unstated. MAE-DFER and MTFormer were originally designed for different benchmarks, and the datasets used here (lab-recorded RAVDESS, CREMA-D, MEAD) are not the standard in-the-wild DFER testbeds; without re-running under the paper's protocol, the reported gains are not attributable to the architecture. I also note a smaller internal inconsistency in the STL baseline definition, which affects the size but not the existence of the cascade benefit. Since the reader already issued a conditional verdict requiring protocol verification, my stress-test leaves the verdict unchanged.","tokens_in":11491,"tokens_out":8533,"duration_ms":84431,"concrete_test":"Obtain or rerun the three strongest baselines (MAE-DFER, MTFormer, and a ViT-D decoder MTL variant) under the exact protocol of Sections 4.1-4.2 (same 5-fold subject-independent splits, same 16-frame uniform downsampling, same MTCNN auxiliary labels, same VideoMAE-L checkpoint and optimization schedule) and compare WAR on all three datasets. If the gap to MTFormer shrinks to below ~2 points on any dataset, or if the cascade's gain over the corrected STL baseline (Norm+Pool+FC) falls below ~3 points, the paper's headline comparisons are not reliable. A released code/checkpoint reproduction of Tables 2-4 would settle the same question.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Primary concern: the central claim in Section 4.5 ('consistently surpasses current SOTA models across all datasets') is supported only by Tables 2-4, whose baseline numbers are quoted from other papers. The paper does not state that MAE-DFER, SVFAP, MTFormer, or MTL-ER were rerun under the paper's exact protocol (5-fold subject-independent splits; 16-frame average downsampling; MTCNN-generated face/landmark labels; VideoMAE-L pretrained initialization; AdamW lr=1e-3; specific loss weights). Published DFER results often use different datasets (DFEW, FERV39k, MAFW) or different split/sampling schemes, so a WAR gap of 3.7-4.3 points over MTFormer is within the range that protocol differences can produce. If the baselines were not re-evaluated in this pipeline, the SOTA comparison does not isolate the proposed architecture. Secondary concern: the STL ablation head is described in Section 4.3 as 'only Fully Connected layer,' while Section 4.2 defines task-specific heads as Norm+Pool+FC; this ambiguity could understate the STL baseline and inflate the 8.8-9.25 WAR gains attributed to the cascade. Both issues are addressable without changing the method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes MTCAE-DFER, a multi-task cascaded autoencoder framework for dynamic facial expression recognition. It uses a shared VideoMAE encoder and three cascaded ViT decoders for face detection, face landmark, and expression recognition, where the decoder output of each previous task serves as the query for the next task's cross-attention with the shared encoder's global features. The paper reports ablation studies over six architecture variants on RAVDESS, CREMA-D, and MEAD, and compares against SOTA methods, claiming consistent improvements. The central claims are that cascading related facial tasks improves DFER accuracy and that the proposed cascaded ViT decoder enables global-local feature interaction.","tokens_in":11786,"tokens_out":7781,"duration_ms":63840,"significance":"If the reported results hold under controlled comparison, the paper makes a useful contribution: it introduces a plug-and-play cascaded ViT decoder for multi-task learning in DFER and provides a self-consistent internal ablation showing that cascaded ViT decoding outperforms non-cascaded variants. The method is clearly described and the ablation ordering is coherent. The main limitations are the uncontrolled baseline comparisons and the absence of statistical detail, which currently leave the core SOTA claim unverified.","major_comments":[{"comment":"The state-of-the-art comparison is the central evidence for the claim in §4.5 that MTCAE-DFER 'consistently surpasses the current SOTA models across all datasets.' However, Tables 2–4 quote WAR/UAR values from prior publications without any statement that those baselines were evaluated under the same protocol as the proposed model. The paper's own protocol includes 5-fold subject-independent cross-validation, 16-frame average downsampling to 224×224, MTCNN-generated face/landmark labels, VideoMAE-L pretrained initialization, AdamW with lr=1e-3, and specific loss weights (0.5/0.5/1.5). Published results for MAE-DFER, MTFormer, SVFAP, and MTL-ER may use different frame sampling, input resolutions, subject splits, or training budgets, and protocol differences of this kind can easily account for the 3.7–4.3 percentage-point WAR gaps over MTFormer reported here. The authors should either re-run the baselines in their own pipeline or provide explicit, verifiable documentation that the quoted numbers were produced under identical protocols; without this, the SOTA claim is not established.","section":"§4.4.2, Tables 2–4"},{"comment":"The STL ablation baseline is described in §4.3 as 'the VideoMAE encoder and the task-specific head with only Fully Connected layer,' whereas the task-specific heads used in the MTL variants are defined in §4.2 as including Normalization, Pooling, and Fully Connected layers. If the STL head lacks pooling and normalization, it is a weaker predictor, and the large STL-to-MTL gains in Table 1 (for example, 9.25 WAR points on RAVDESS) may partly reflect a head-architecture mismatch rather than the effect of MTL or cascading. The STL baseline should use the same Norm+Pool+FC head as the MTL variants to make the comparison controlled.","section":"§4.3 vs §4.2"},{"comment":"The ablation study only compares the full three-task cascade against STL and non-cascaded MTL variants; it does not include configurations with only face detection or only face landmark as auxiliary tasks, nor does it test alternative task orderings. Consequently, the paper's stated goal of exploring the 'impact of dynamic face detection and dynamic face landmark on dynamic facial expression recognition' is not directly supported: one cannot tell from Table 1 whether both auxiliary tasks contribute, whether one dominates, or whether the specific order detection→landmark→expression is important. Adding these ablations would substantially strengthen the causal claims in §4.5.","section":"Table 1, §4.5"},{"comment":"All experiments appear to be single runs, with no error bars, per-fold results, or significance tests. Several of the key differences are small—for example, the cascade over non-fully-shared ViT decoder improves WAR by 1.52 points on RAVDESS, 1.36 on CREMA-D, and 1.87 on MEAD in Table 1. Without an estimate of run-to-run variance (at least 3–5 seeds with mean and standard deviation, or paired per-fold comparisons), these differences, and the corresponding SOTA margins, cannot be distinguished from noise. The authors should report variance or per-fold breakdowns.","section":"Tables 1–4"}],"minor_comments":[{"comment":"The abstract states that the model 'alleviates overfitting of complex large model,' but the paper provides no overfitting analysis (e.g., train/validation gap or regularization comparison). Either provide evidence or remove the claim.","section":"Abstract and §4.5"},{"comment":"The dimension of the Dense layer that produces K and V is not specified in the algorithm; §3.3 later gives the ViT Decoder feature dimension as 512. Please state this dimension in the algorithm description.","section":"§4.2, Algorithm 1"},{"comment":"The text reports differences in percentage points (e.g., '20.70%') but should say 'percentage points' to avoid confusion with relative improvement.","section":"§4.4.2"},{"comment":"Reference [30] is a millimeter-wave radar gesture recognition paper and does not support the sentence in the Introduction citing 'ResNet18 + LSTM' for dynamic facial expression recognition. Please replace it with a relevant reference.","section":"References"},{"comment":"The dataset name is rendered as 'RA VDESS' and the metric as 'W AR' with an extra space in several places (for example, §1, §4.1, Table 2). Please fix the spacing.","section":"Throughout"},{"comment":"The hyperparameters (loss weights 0.5/0.5/1.5, decoder depth 5, heads 3) are chosen without a sensitivity study; a small analysis of the loss-weight ratio would help, though this is not essential.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper would benefit from the authors making code and trained models available to allow verification of the baseline comparisons. The citation mismatch for [30] and the lack of any discussion of protocol differences for quoted baselines suggest the manuscript needs careful revision of its experimental reporting. The fit with the journal's scope is acceptable if the experimental concerns are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The internal ablation is clean and the cascade gives a consistent small gain, but the SOTA claim does not hold up as written because Tables 2-4 are quoted numbers from other papers, not protocol-matched reruns. That's the one thing you should remember.\n\nWhat's actually new: MTCAE-DFER chains three ViT decoders over a shared VideoMAE encoder—face detection output becomes the Q for the landmark decoder, whose output becomes the Q for the expression decoder, while K and V stay fixed from the encoder. That cascaded global-local attention is a specific arrangement I haven't seen in prior DFER work, and the paper describes it clearly. Credit: the ablation (Table 1) compares six structures and the ordering is consistent across RAVDESS, CREMA-D, MEAD. The cascade beats the best non-cascaded ViT decoder by 1.5-2.3 WAR points, and the ViT decoder consistently beats MLP decoders by 3.8-4.9 points. Those are believable, internally consistent results.\n\nSoft spots, in proportion. First, the headline 'consistently surpasses current SOTA models across all datasets' is only supported by Tables 2-4, where MAE-DFER, MTFormer, SVFAP etc. are quoted from their own papers. Nothing says those models were run under the same 5-fold subject-independent splits, 16-frame average downsampling, MTCNN-generated labels, VideoMAE-L initialization, and AdamW settings. Differences of 3.7-4.3 WAR over MTFormer are within reach of protocol changes. That claim needs reruns or a star in the table saying 'as reported in original paper,' with a protocol table to show comparability. Second, Section 4.3 describes the STL head as 'only Fully Connected layer,' while Section 4.2 defines task-specific heads as Norm+Pool+FC. If the STL baseline really was a plain FC, the 8.8-9.25 WAR gain attributed to the cascade is partly a head upgrade. That ambiguity should be fixed—ideally by rerunning STL with the same head. Minor: no error bars, no code, and no cascaded-MLP ablation, though the cascade effect is already isolated against non-cascaded ViT decoders; the missing row would be informative, not load-bearing.\n\nOverall, the paper is honest and methodologically clear, with limitations acknowledged. It's an incremental architecture change, not a breakthrough. I'd trust the internal comparison for direction, but I wouldn't put it on a reference list until the SOTA claim is verified. If the authors supply protocol-matched baselines and clarify the STL head, it's a solid contribution to affective computing. I'd accept this for peer review, with a request for those experiments. Reading group? Maybe, as a case study of why internal ablations can't carry an external SOTA claim.","headline":"Clean internal ablation, but the SOTA claim rests on unverified quoted numbers—worth a serious referee if the protocol gap is fixed.","tokens_in":12238,"tokens_out":4177,"would_cite":false,"duration_ms":36128,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that cascading dynamic face detection, landmark localization, and expression recognition through a shared VideoMAE encoder with task-level ViT decoders beats single-task and non-cascaded multi-task baselines on RAVDESS…","keywords":["dynamic facial expression recognition","multi-task learning","cascaded autoencoder","Vision Transformer decoder","VideoMAE","global-local feature interaction","face detection","face landmark"],"falsifier":"Retrain MAE-DFER and MTFormer under the exact protocol used for MTCAE-DFER — 16 frames per clip via average downsampling, MTCNN-generated auxiliary labels, 5-fold subject-independent cross-validation, and matched training budgets — and compare WAR; if the gaps in Tables 2-4 shrink to noise, the central claim fails. A second check: replace the cascaded Q input of the DFER decoder with a fixed non-cascaded local feature of the same dimension; if WAR does not drop, the cascade order is not the cause.","tokens_in":11310,"feed_emoji":"😊","tokens_out":7322,"duration_ms":62712,"temperature":0.7,"pith_summary":"The paper argues that dynamic facial expression recognition is best treated as the last stage of a cascade of related face tasks. Its MTCAE-DFER model shares one VideoMAE encoder across dynamic face detection, dynamic face landmark, and expression recognition, and connects the tasks through a plug-and-play Cascaded ViT Decoder in which the shared encoder supplies K and V while the previous task's decoder output supplies Q. On RAVDESS, CREMA-D, and MEAD, the paper reports weighted average recall of 83.69, 85.03, and 88.44, beating the prior multi-task self-supervised baseline MTFormer by about 4 points on each dataset and the single-task baseline by 7 to 9 points. If the comparison holds, the value is a way to inject fine-grained facial geometry into video emotion recognition without a separate offline pipeline.","feed_headline":"Cascading face tasks lifts emotion recognition 7-9 points","feed_subtitle":"The plug-in decoder feeds face detection and landmark cues into expression recognition, adding 7-9 points in weighted average recall.","key_machinery":"The Cascaded ViT Decoder is the load-bearing module. It is a compact Transformer decoder, configured here with 5 layers and 3 attention heads at feature dimension 512, whose multi-head attention is re-pointed so that K and V come from the normalized dense output of the shared VideoMAE encoder and Q comes from the previous task's decoder output (or from the encoder itself for the first task). Q updates through recursion while K and V stay fixed, so each stage mixes global video features with local task features. The cascade is written as Y1 = ViTDecoder(F,F), Y2 = ViTDecoder(Y1,F), and Y3 = ViTDecoder(Y2,F), where F is the encoder output; this chaining is what turns parallel multi-task decoders into a task cascade.","core_discovery":"The central claim is that cascading three face-related tasks through a shared encoder and task-level ViT decoders consistently outperforms both single-task learning and non-cascaded multi-task learning across all three datasets. The mechanism is global-local feature interaction: at each stage, the global dynamic representation from the VideoMAE encoder is entered as K and V, and the local output of the previous stage is entered as Q, so the expression head receives features already filtered by face detection and landmark localization. In the paper's reported numbers, the cascaded model reaches WAR 83.69 on RAVDESS (7-class), 85.03 on CREMA-D (6-class), and 88.44 on MEAD (8-class), improving on the previous best multi-task self-supervised model MTFormer by 4.01, 4.29, and 3.70 points respectively.","pith_inferences":["The cascade's success may depend on the proxy labels: because MTCNN generates the detection and landmark supervision, a clean test would reorder or randomly relabel the auxiliary tasks to see whether the gain comes from task order or simply from added supervision.","The reported margin over prior models is measured against numbers quoted from other papers; re-running those baselines under identical 16-frame downsampling, subject-independent 5-fold splits, and training budgets could yield smaller margins than the tables show.","The same global-K/V plus local-Q design could apply to other video-understanding chains, such as person detection to pose estimation to action recognition, where the shared encoder provides global tokens and the earlier task provides local keypoints."],"forward_implications":["Expression recognition can be improved by consuming features already shaped by face detection and landmark localization, not only raw global video tokens.","The Cascaded ViT Decoder is plug-and-play: attaching it to a shared VideoMAE encoder and cascading the three tasks adds 1.52 to 2.26 WAR points over the non-cascaded ViT decoder MTL variant on the three datasets.","Even with MLP decoders, multi-task learning with the shared encoder beats single-task fine-tuning by at least 1.32 to 2.40 WAR points, so the MTL formulation itself contributes to generalization.","The same task-chain recipe can be transferred to other related-task video problems, and the authors propose combining it with emotion-based text generation and multi-modal encoders."],"supporting_citations":[{"why":"Supplies the VideoMAE self-supervised encoder used as the shared backbone for all three tasks.","marker":"[23]"},{"why":"Provides the pre-trained ViT-L weights inherited for fine-tuning and is the strongest single-task self-supervised baseline.","marker":"[21]"},{"why":"Provides the multi-task self-supervised MTFormer baseline that MTCAE-DFER is compared against, and motivates cross-task reasoning.","marker":"[27]"},{"why":"Supplies the multi-task cascade concept of using cascaded decoders to relate tasks.","marker":"[5]"},{"why":"Provides the Transformer decoder attention concept where Q comes from a different source than K and V.","marker":"[25]"},{"why":"Provides the Vision Transformer architecture that the Cascaded ViT Decoder is built on.","marker":"[6]"},{"why":"Generates the frame-level face bounding-box and five-point landmark labels used to supervise the first two tasks.","marker":"[28]"},{"why":"Is one of the three evaluation datasets (RAVDESS) used for the ablation and SOTA comparisons.","marker":"[14]"},{"why":"Provides the CREMA-D evaluation dataset used for the second benchmark.","marker":"[2]"},{"why":"Provides the MEAD evaluation dataset used for the third benchmark.","marker":"[26]"}],"fun_headline_variants":["Cascaded face tasks add 7–9 points to emotion recognition","Face cascade sharpens emotion recognition by up to 9 points","Multi-task cascade lifts expression recognition 7–9 points","Cascading face detection and landmarks boosts emotion scores","Task cascade with ViT decoders improves facial expression accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the comparison numbers quoted from other papers were produced under protocols compatible enough with the new model's subject splits, frame sampling, and training setup; if those protocols differ, the reported gains could be artifacts of experimental setup rather than the cascaded architecture.","fun_headline_variants_meta":{"raw":{"variants":["Cascaded face tasks add 7–9 points to emotion recognition","Face cascade sharpens emotion recognition by up to 9 points","Multi-task cascade lifts expression recognition 7–9 points","Cascading face detection and landmarks boosts emotion scores","Task cascade with ViT decoders improves facial expression accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000352,"raw_usage":{"total_tokens":1927,"prompt_tokens":965,"completion_tokens":962,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":878}},"tokens_in":581,"tokens_out":962,"duration_ms":9909,"temperature":1.0,"reasoning_tokens":878,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:57:21.994779+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain MAE-DFER and MTFormer under the exact protocol used for MTCAE-DFER — 16 frames per clip via average downsampling, MTCNN-generated auxiliary labels, 5-fold subject-independent cross-validation, and matched training budgets — and compare WAR; if the gaps in Tables 2-4 shrink to noise, the central claim fails. A second check: replace the cascaded Q input of the DFER decoder with a fixed non-cascaded local feature of the same dimension; if WAR does not drop, the cascade order is not the cause.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the VideoMAE self-supervised encoder used as the shared backbone for all three tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pre-trained ViT-L weights inherited for fine-tuning and is the strongest single-task self-supervised baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the multi-task self-supervised MTFormer baseline that MTCAE-DFER is compared against, and motivates cross-task reasoning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the multi-task cascade concept of using cascaded decoders to relate tasks."},{"cited_title":"Zhang, Z","cited_arxiv_id":null,"evidence_quote":"Generates the frame-level face bounding-box and five-point landmark labels used to supervise the first two tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is one of the three evaluation datasets (RAVDESS) used for the ablation and SOTA comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CREMA-D evaluation dataset used for the second benchmark."}],"review_version":1}