{"id":"41e3d5b0-bf4f-4ad6-a20f-5bb1f49b163e","arxiv_id":"2505.16372","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"TSFmicro, a dual-branch RetNet and vision transformer framework, reports state-of-the-art results on CASME II, SAMM, and CAS(ME)3.","lead":"TSFmicro is a two-part AI system for spotting micro-expressions: one branch learns how the face changes over time, the other learns where on the face the change happens. The authors report top accuracy on three standard micro-expression datasets, but they provide no code and no uncertainty estimates.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim is undermined by selecting the T-S(late) fusion variant on the same test folds used to report results, with no error bars or significance testing.","rationale":"The reader's verdict is CONDITIONAL and identifies the overstated temporal modeling as the weakest assumption. I agree that the 'temporal' branch is effectively a two-frame difference image, which weakens the paper's dynamic-modeling narrative. However, the most load-bearing concern for the central SOTA claim is the evaluation protocol: the fusion mode is selected on the same test folds that produce the headline numbers, and no error bars or significance tests are reported. On small datasets like SAMM (136 samples) and CASME II (248 samples), the margin between fusion modes is a handful of samples, so the reported best result could be a selection artifact. This concern is directly testable with per-fold predictions and a paired test, whereas the temporal-modeling concern affects interpretation but not necessarily the reported accuracy. The paper does have strengths: a systematic fusion study, ablations on three datasets, and a clear architecture description; those justify keeping the verdict at CONDITIONAL rather than REJECT, but the missing statistical validation and the test-set selection are precisely what must be supplied before the SOTA claim can be accepted.","tokens_in":18796,"tokens_out":2935,"duration_ms":27205,"concrete_test":"Obtain the per-subject (LOSO fold) predictions for all four fusion modes on CASME II and SAMM, then run a paired McNemar test between T-S(late) and the next-best fusion mode, and also between T-S(late) and the best external baseline on a per-fold basis. If p>0.05 on either dataset, the headline SOTA claim is not statistically supported. As a second check, repeat the fusion-mode selection using nested cross-validation (choose the fusion on inner folds only and evaluate on held-out outer folds); if the selected fusion mode varies across outer folds, the reported fixed choice is overfit to the test set.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is state-of-the-art accuracy on CASME II, SAMM, and CAS(ME)3. For the two datasets where the fusion strategy is studied, Section 4.4 and Table 7 evaluate four fusion modes (T-S early, T to S, S to T, T-S late) and then adopt T-S(late) as the final model. Because the same LOSO test folds are used both to select the fusion mode and to produce the headline numbers, the reported accuracy is the maximum of four correlated estimates. On SAMM 5-class, T-S(late) at 80.88% beats S-to-T at 79.41% by 1.47 points, i.e., 2 samples out of 136; on CASME II 5-class, the margin over T-to-S is 1.62 points, i.e., 4 samples out of 248. Without per-fold results, confidence intervals, or a paired significance test, the claimed improvement over the second-best fusion mode and over external SOTA methods (e.g., +1.41 over MMNet on CASME II) is within plausible random variation. The reader's concern about the temporal branch is valid but secondary: even if the RetNet only processes a difference image, the ablation shows that difference image carries substantial signal; the primary threat to the SOTA claim is the unreliable, test-set-selected comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes TSFmicro, a dual-stream framework for micro-expression recognition. The temporal branch computes a difference image between the apex and onset frames, processes it through a convolutional stem and a RetNet block, and the spatial branch tokenizes the onset frame with position embeddings through a two-layer transformer. The paper evaluates four fusion strategies, selects 'T-S late' as the final model, and reports state-of-the-art accuracy on CASME II (87.50% 5-class ACC), SAMM (80.88% 5-class ACC), and CAS(ME)3 (53.29% UF1, 62.73% UAR for 7-class), together with ablation studies and visualizations. The central claim is that this architecture and fusion method outperform contemporary methods on three widely used micro-expression datasets.","tokens_in":19087,"tokens_out":7311,"duration_ms":55454,"significance":"If the reported results are reliable, TSFmicro would be a competitive approach for micro-expression recognition, and the paper would provide a useful comparison of temporal-spatial fusion strategies. The ablation clearly shows that the difference-image temporal branch carries most of the classification signal, and the fusion study explores a relevant design space. However, the evidence is currently not strong enough to support the state-of-the-art claim: no error bars or significance tests are reported, the final fusion strategy is selected on the same test folds that produce the headline numbers, and on SAMM the margin over the second-best fusion mode is one to two samples. The paper also does not release code or per-fold details, which limits reproducibility and independent verification.","major_comments":[{"comment":"The final model is selected by evaluating four fusion strategies (Table 7) on the same LOSO folds that are then used for the headline comparisons in Tables 4 and 5. Since T-S(late) is the empirical maximum among four options on those folds, the reported SOTA margins are optimistically biased. Concretely, on SAMM 5-class the gap between T-S(late) and S-to-T is 1.47 percentage points (2 samples out of 136), and the gap to MMNet is 0.74 percentage points (1 sample out of 136); on CASME II 5-class the gap to T-to-S is 1.62 points (4 samples out of 248) and to MMNet is 1.41 points (4 samples out of 248). These margins are within the range of random variation for a paired design at these sample sizes. Please report per-fold results, confidence intervals, and a paired significance test (e.g., Wilcoxon signed-rank or McNemar), and use a nested or validation-based selection procedure for the fusion mode.","section":"Section 4.4, Tables 4, 5, 7"},{"comment":"The temporal branch is defined by Eq. (1) as a single difference image f_t = f_apex - f_onset. After the Conv Stem, this one image is passed through the RetNet-Block, and Eq. (2) is written as a retention sum over sequence positions n and m. With a single input image, the sum collapses to a single term; no frame sequence is processed. The 'temporal' modeling is therefore a two-frame subtraction followed by static feature extraction, not a temporal sequence model. This weakens the paper's claims of capturing 'temporal relations' and 'dynamic micro-expression recognition.' Either feed an actual sequence of frames (e.g., multiple difference images or a short clip) into the retention mechanism, or revise the text to describe the temporal branch as a motion-difference representation.","section":"Section 3.1, Eqs. (1)-(2)"},{"comment":"The fair comparison with baselines is not established. Section 4.1.2 states that baseline source codes were used and tuned, but only the MMNet replication is described in Section 4.2. It is unclear whether the other baseline numbers in Tables 3-5 were reproduced with the same face cropping, resizing, augmentation, and LOSO splits as TSFmicro, or taken from the original papers. Since the SOTA margins are a few samples on CASME II and SAMM, small preprocessing differences could change the ranking. Please provide the exact evaluation protocol for each baseline, including the split identity (e.g., subject IDs) and preprocessing pipeline, and ideally release the evaluation code.","section":"Section 4.2, Tables 3-5"},{"comment":"The ablation in Table 6 reports single runs without variance estimates. The main fusion gain on CASME II 5-class is +3.63 ACC (from 83.87 to 87.50); on SAMM 5-class it is +5.88 ACC (from 75.00 to 80.88). These gains are presented as definitive, but on 248 and 136 samples respectively they may not be statistically significant. Please add per-fold breakdowns and significance tests for the ablation, or at least standard deviations across LOSO folds.","section":"Table 6 ablation"}],"minor_comments":[{"comment":"The sentence 'The total data volume of the 12 published spontaneous micro-expressions datasets ... is about 10,000, which is a typical classification and recognition task compared to other small-sample problem compared to other classification recognition tasks' is garbled and should be rewritten.","section":"Section 1"},{"comment":"Eq. (2) uses Q_n, K_m, and V_m without defining these quantities; please add a sentence explaining that they are linear projections of the input token sequence, and clarify how the retention mechanism is applied when the sequence length is one.","section":"Section 3.1"},{"comment":"The paper states 'a five-layer RetNet network was employed' and 'a ViT was used' but does not specify the number of heads, hidden dimensions, or the patch size P used in the spatial branch; please provide the full architecture hyperparameters.","section":"Section 4.1.3"},{"comment":"The phrase 'curve-fitted the scores' is unclear; if these are just training curves or smoothed validation curves, please state that explicitly and describe the smoothing method.","section":"Figures 3 and 5"},{"comment":"The cross-cultural explanation for the difference between T-to-S and S-to-T on CASME II vs SAMM is speculative; the paper does not provide per-subject cultural subgroup analysis or statistical evidence. Please temper the claim or add supporting analysis.","section":"Section 5 Discussion"},{"comment":"The columns for CASME II include 'Repression' while Table 1 uses 'Rep'; please standardize the label names.","section":"Table 8"},{"comment":"Reference [70] is co-authored by one of the present authors (X. Fu) and is cited only for a general statement about cultural differences; consider removing or citing a non-self reference, or explicitly noting the overlap.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an arXiv preprint with no associated code or data release. The headline SOTA margins are very small, especially on SAMM (0.74 points over MMNet, i.e., roughly one sample), so I would require the per-fold statistics and protocol sharing before considering acceptance. The temporal-branch concern is conceptual but fixable by either using real sequences or reframing the claim. I also note that the 'curve fitting' of training scores is nonstandard and should be clarified or removed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: TSFmicro is a straightforward dual-stream fusion of a RetNet temporal branch operating on apex-minus-onset difference images and a shallow two-layer ViT spatial branch on onset frames. The architecture is not groundbreaking, but the paper does something useful: it systematically compares four temporal-spatial fusion strategies, reports clean ablations on three standard datasets, and the numbers are internally consistent. The CAS(ME)3 gains are large enough to matter if they hold up.\n\nWhat is genuinely new is applying RetNet to micro-expression recognition and the systematic fusion comparison. The ablation shows the temporal branch carries most of the signal, which is plausible and consistent with prior work. The authors also reproduce several baselines, including MMNet, which is more than many MER papers do.\n\nThe main problem is the central SOTA claim. The winning fusion mode was selected after evaluating all four modes on the same LOSO folds used to report the headline numbers. On SAMM 5-class the margin between T-S(late) and S-to-T is two samples; on CASME II it is four samples. Without per-fold results, confidence intervals, or a paired significance test, that margin is within random variation, and it weakens the comparison against external baselines like MMNet. This is the load-bearing issue, not the temporal branch. The temporal branch does only see a single difference image rather than a sequence, so calling it 'dynamic' is a stretch, but the ablation shows that difference image carries substantial signal, so that point is secondary.\n\nThere is no code released, and some implementation details are missing—patch size, RetNet depth, training schedule specifics—which makes independent verification harder. The writing is rough in places, and the 'where-how' interpretation is qualitative, but those are minor.\n\nWho is this for? Someone working in micro-expression recognition who wants a benchmark comparison and a reasonable dual-stream baseline to build on. It deserves a serious referee, but the SOTA claim needs to be qualified or supported with proper statistics. I would recommend sending it to review with the expectation that the authors add confidence intervals, justify or pre-specify the fusion selection, and release code.","headline":"Solid empirical architecture paper for micro-expression recognition, but the SOTA claim rests on a test-set-selected fusion choice without error bars—needs independent confirmation before I'd trust the numbers.","tokens_in":19588,"tokens_out":1535,"would_cite":true,"duration_ms":13900,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TSFmicro, a dual-stream network that treats the apex-minus-onset frame difference as its temporal signal and fuses it with a shallow positional transformer on the onset frame, reports the best published accuracy on CASME II (87.50% ACC)…","keywords":["micro-expression recognition","temporal-spatial feature fusion","RetNet","difference frame","multi-modal fusion","CASME II","SAMM","CAS(ME)3"],"falsifier":"Feed the same architecture a two-channel input of successive frame differences (onset-to-middle and middle-to-apex) and check whether accuracy improves; also randomize the order of the 196 spatial tokens before the RetNet block and see whether performance drops, which would distinguish real temporal modeling from spatial-layout modeling. Re-running all baselines under identical face cropping and augmentation would confirm the reported margins.","tokens_in":18577,"feed_emoji":"😶","tokens_out":5830,"duration_ms":45041,"temperature":0.7,"pith_summary":"This paper claims that combining a temporal branch, which extracts motion from a single apex-minus-onset difference image, with a spatial branch that locates facial action via positional embeddings, yields the best published micro-expression recognition results on three standard datasets. The reported scores top all contemporary state-of-the-art methods: 87.50% accuracy on CASME II, 80.88% on SAMM, and 53.29% UF1 with 62.73% UAR on CAS(ME)3. The paper argues that late fusion of the two feature streams, rather than early or sequential fusion, best preserves the complementary 'where-how' information each branch carries.","feed_headline":"Dual-stream model hits 87.5% on CASME II micro-expressions","feed_subtitle":"The apex-minus-onset difference plus a spatial transformer beats prior SOTA on CASME II, SAMM, and CAS(ME)3.","key_machinery":"The central object is the dual-stream TSFmicro architecture with late element-wise summation of the two feature maps. The temporal stream computes the difference image $f_t = f_{\\text{apex}} - f_{\\text{onset}}$, embeds it with a Conv Stem, and applies a RetNet block whose retention mechanism $o_n = \\sum_{m=1}^{n} \\gamma^{n-m}(Q_n e^{in\\theta})(K_m e^{im\\theta})^\\dagger v_m$ imposes an exponential decay over a sequence of tokens, treating the spatial grid as a sequence to model order-dependent dynamics. The spatial stream slices the onset frame into patches, adds learnable position embeddings, and processes them through only two transformer layers to avoid extracting identity information unrelated to motion.","core_discovery":"The paper's central claim is that TSFmicro achieves superior performance over contemporary state-of-the-art methods on CASME II, SAMM, and CAS(ME)3. The temporal branch, fed only the difference between onset and apex frames, contributes most of the recognition accuracy, while the spatial branch adds geographic localization that improves accuracy particularly for under-represented emotion classes; the ablation shows the full model raises 5-class accuracy on CASME II by 24.20 points over the spatial branch alone. Late fusion by element-wise summation in high-dimensional feature space outperforms early and sequential fusion strategies, which the paper attributes to avoiding interference between temporal and spatial information.","pith_inferences":["Because the temporal input is a single image, the word 'dynamic' overstates the modeling; the gain likely comes from the difference map acting as a motion prior, and a testable extension would replace RetNet with a plain ResNet on the same difference image to compare.","The retention mechanism's decay may operate on spatial patch order rather than true time; randomizing the 196 token positions before the RetNet block would reveal whether the ordering is load-bearing.","The paper attributes fusion-strategy differences between CASME II and SAMM to culture, but class imbalance and face-alignment difficulty are not controlled; a balanced resampling experiment could separate those factors."],"forward_implications":["A two-frame difference (apex minus onset) together with a spatial branch is sufficient to beat full-video and optical-flow methods, suggesting that complete frame sequences may be unnecessary for micro-expression recognition.","Late fusion, summing features in a high-dimensional space, is the best of the four fusion modes tested, implying that delaying cross-modal interaction until the final classifier helps this task.","The deliberate use of a shallow two-layer spatial branch indicates that deeper spatial backbones are not beneficial and may introduce identity-specific artifacts.","The same framework reaches 94.84% ACC on CASME II's 3-class task and 70.75% UF1 on CAS(ME)3's 4-class task, substantially reducing the gap to practical deployment."],"supporting_citations":[{"why":"Supplies the CASME II benchmark and its 5/3-class labels used for the main comparison.","marker":"[20]"},{"why":"Supplies the CAS(ME)3 dataset and 7/4-class labels where the largest reported gains occur.","marker":"[22]"},{"why":"Supplies the SAMM dataset with cross-cultural samples used for the second comparison.","marker":"[26]"},{"why":"Justifies using apex frames instead of full sequences, the basis of the difference-frame input.","marker":"[36]"},{"why":"Provides the RetNet architecture and retention mechanism that forms the temporal branch.","marker":"[53]"},{"why":"MMNet, the strongest prior baseline on CASME II and SAMM that TSFmicro must beat.","marker":"[57]"},{"why":"DecFlow, a 2024 optical-flow baseline previously leading some metrics on these datasets.","marker":"[59]"},{"why":"MPFNet, a recent video-based state-of-the-art baseline compared across datasets.","marker":"[60]"}],"fun_headline_variants":["Temporal-spatial fusion hits 87.5% on CASME II micro-expressions","Element-wise fusion of spatial-temporal features hits 87.5% on CASME II","RetNet and transformer fusion recognize micro-expressions at 87.5%","Beats prior best on CASME II with 87.5% micro-expression accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All temporal modeling rests on a single image formed by subtracting the onset frame from the apex frame; if that difference does not capture the motion, the temporal branch is just a static classifier on a transformed image.","fun_headline_variants_meta":{"raw":{"variants":["Temporal-spatial fusion hits 87.5% on CASME II micro-expressions","Element-wise fusion of spatial-temporal features hits 87.5% on CASME II","RetNet and transformer fusion recognize micro-expressions at 87.5%","Beats prior best on CASME II with 87.5% micro-expression accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00127,"raw_usage":{"total_tokens":5186,"prompt_tokens":926,"completion_tokens":4260,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":4170}},"tokens_in":542,"tokens_out":4260,"duration_ms":24150,"temperature":1.0,"reasoning_tokens":4170,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:02:30.068258+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Feed the same architecture a two-channel input of successive frame differences (onset-to-middle and middle-to-apex) and check whether accuracy improves; also randomize the order of the 196 spatial tokens before the RetNet block and see whether performance drops, which would distinguish real temporal modeling from spatial-layout modeling. Re-running all baselines under identical face cropping and augmentation would confirm the reported margins.","supporting_citations":[{"cited_title":"Fa- cialflownet: Advancing facial optical flow estimation with a diverse dataset and a decomposed model","cited_arxiv_id":null,"evidence_quote":"DecFlow, a 2024 optical-flow baseline previously leading some metrics on these datasets."},{"cited_title":"Casme ii: An improved spontaneous micro- expression database and the baseline evaluation.PLOS ONE, 9(1):1– 8, 01 2014","cited_arxiv_id":null,"evidence_quote":"Supplies the CASME II benchmark and its 5/3-class labels used for the main comparison."},{"cited_title":"Cas(me)3: A third generation facial spontaneous micro-expression database with depth information and high ecological validity","cited_arxiv_id":null,"evidence_quote":"Supplies the CAS(ME)3 dataset and 7/4-class labels where the largest reported gains occur."},{"cited_title":"Davison, Cliff Lansley, Nicholas Costen, Kevin Tan, and Moi Hoon Yap","cited_arxiv_id":null,"evidence_quote":"Supplies the SAMM dataset with cross-cultural samples used for the second comparison."},{"cited_title":"Automatic micro-expression recognition from long video using a single spotted apex","cited_arxiv_id":null,"evidence_quote":"Justifies using apex frames instead of full sequences, the basis of the difference-frame input."},{"cited_title":"Rmt: Retentive networks meet vision transformers","cited_arxiv_id":null,"evidence_quote":"Provides the RetNet architecture and retention mechanism that forms the temporal branch."},{"cited_title":"Mmnet: Muscle motion-guided network for micro-expression recognition","cited_arxiv_id":null,"evidence_quote":"MMNet, the strongest prior baseline on CASME II and SAMM that TSFmicro must beat."},{"cited_title":"A multi-prior fusion network for video-based micro- expression recognition","cited_arxiv_id":null,"evidence_quote":"MPFNet, a recent video-based state-of-the-art baseline compared across datasets."}],"review_version":1}