{"id":"32d7ed2b-1a40-4b44-84c6-87cfc03dd62e","arxiv_id":"2507.10209","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Ethnicity labels derived from FaceXFormer and heuristic correction are added to CASME2 and SAMM, and an ethnicity-aware fusion model improves micro-expression recognition over a motion-only baseline.","lead":"The authors combine two existing micro-expression datasets, add ethnicity labels by algorithm plus human correction, and report that mixing ethnicities in training data lowers recognition performance. They then show a model that uses ethnicity as an extra learning signal improves three-class emotion recognition over a motion-only baseline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ethnicity is never isolated as the causal variable: the reported gains are confounded with model capacity, auxiliary losses, and dataset identity.","rationale":"The reader's weakest assumption is label reliability. I agree that noisy labels would weaken the results, but the paper's own central claim fails even under perfect labels. Table V's design cannot separate ethnicity from general auxiliary learning, and Table IV's design cannot separate ethnicity from dataset shift. Since these confounds attack the causal interpretation directly, they are more load-bearing than label noise: fixing annotation quality alone would not make the current experiments valid. The code is public and the paper is appropriately framed as preliminary, so the issue is correctable; a conditional acceptance with the proposed control experiment is the right level. If the controls fail, the central claim should be downgraded to 'multi-task training helps,' not 'ethnicity helps.'","tokens_in":7961,"tokens_out":5981,"duration_ms":75891,"concrete_test":"Run the dual-backbone late-fusion model from Table V row 2 with three control auxiliary labels—(i) randomly permuted ethnicity labels, (ii) gender labels, (iii) a duplicate emotion label—otherwise identical hyperparameters (15 epochs, LR 0.001, LOSO/CDE). If any control reaches macro-F1 statistically indistinguishable from 0.6639 over 5 seeds, the ethnicity-specific attribution fails. Separately, for Table IV, repeat the mixed condition while balancing source dataset (e.g., 8 CASME2 + 8 SAMM) and report per-seed macro-F1 with a permutation test.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Table V, the claimed benefit of 'ethnic context' rests on comparing a single ResNet-18 (0.6210) with a dual-backbone late-fusion model (0.6639) that adds an ethnic branch and an extra CCE loss (Eqs. 1-4). The improvement could therefore come from extra parameters, multi-task regularization, or any auxiliary supervision, not from the semantic content of the ethnicity labels. No ablation replaces the ethnicity labels with shuffled labels, gender, or a second emotion task while keeping the architecture fixed, so the specific contribution of ethnicity is unidentified. The RGB-texture comparison is likewise confounded: the architecture changes from ResNet-18 alone to ResNet-18 + TinyViT at the same time as the input modality, so the observed failure/success cannot be attributed to the ethnicity representation. The prima facie study in Table IV has a parallel problem: in the CASME2-SAMM joint set, non-Asian subjects come from SAMM while Asian subjects come predominantly from CASME2, so the mixed-ethnicity condition is also a mixed-dataset condition. The average drop in Table IV is small (0.017 vs Asian-only, 0.041 vs non-Asian-only) and no variance or significance test is reported. Thus, even granting perfect ethnicity labels, the experiments do not establish that ethnicity, rather than domain shift or generic auxiliary supervision, drives the observed differences.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether ethnicity influences micro-expression recognition. The authors construct an ethnically annotated joint CASME2-SAMM dataset using FaceXFormer plus heuristic correction, run a \"prima facie\" experiment comparing mono-ethnic and mixed-ethnic groups under a leave-one-subject-out protocol, and propose a dual-branch framework that adds an ethnicity-classification branch to a motion-based emotion classifier. They report that the mixed-ethnicity condition yields lower macro-F1 than mono-ethnic conditions, and that adding an ethnicity branch improves average macro-F1 from 0.6210 to 0.6639, with a further improvement to 0.6826 when a transformer is used for RGB texture. Qualitative activation maps are presented as evidence for distinct expression patterns across ethnic groups. The paper argues that these results challenge the universality hypothesis and motivate ethnicity-aware micro-expression recognition.","tokens_in":8255,"tokens_out":3629,"duration_ms":43159,"significance":"The question of whether ethnicity affects micro-expression recognition is timely and underexplored; if established convincingly, it would have implications for both affective computing and cross-cultural psychology. The paper also contributes a public code release and an ethnicity-annotated subset of two standard datasets. However, the evidence presented is currently too confounded to support the central claims. The main value of the submission at this stage is as a preliminary investigation that could become publishable after substantial re-analysis, including controlled ablations, significance testing, and label-quality validation.","major_comments":[{"comment":"The central benchmarking claim that \"ethnic context\" improves micro-expression recognition is confounded by architecture and loss-function changes. The baseline is a single ResNet-18 trained with one categorical cross-entropy loss, while the proposed method uses two ResNet-18 backbones with late fusion and an additional ethnicity loss Lethnic. The observed improvement from 0.6210 to 0.6639 could therefore be due to extra model capacity, multi-task regularization, or any auxiliary supervision, rather than the semantic content of the ethnicity labels. The same confound affects the RGB-texture comparison: the architecture changes from ResNet-18 alone to ResNet-18 plus TinyViT at the same time as the input modality changes, so the failure/success cannot be attributed to the ethnicity representation. To support the claim, the authors should add ablations that keep the architecture fixed and replace the ethnicity labels with shuffled labels, gender labels, or a second emotion task.","section":"Section IV-B, Table V, Equations (1)-(4)"},{"comment":"The prima facie study is confounded by dataset identity. Given the subject distributions in Table I, the Asian-only group is drawn predominantly from CASME2 and the non-Asian-only group from SAMM, so the mixed-ethnicity condition is also a mixed-dataset condition. The observed average macro-F1 drop (0.4546/0.4785 to 0.4375) is small, and no variance or significance testing is reported for this 16-subject-per-condition experiment. This design cannot distinguish the effect of ethnicity from domain shift between datasets. The authors should report per-dataset performance, add error bars over multiple runs or bootstraps, and ideally include a condition that mixes subjects from both datasets while keeping the ethnic composition constant.","section":"Section III-B and Table IV"},{"comment":"The reliability of the ethnicity labels is not established. The labels come from FaceXFormer with \"heuristic-driven screening to correct some annotations\" by human eyes, but no inter-annotator agreement, validation against known subject metadata, or per-sample confidence measures are reported. The mapping of Indian and \"Others\" subjects to the Asian group is ad hoc and may introduce label noise. If FaceXFormer's errors are correlated with the source database (e.g., it performs differently on CASME2 versus SAMM images), then both the prima facie study and the benchmarking results could reflect label noise rather than ethnicity. The authors should measure annotation accuracy, report confusion on a held-out subset, or at least conduct a sensitivity analysis with alternative label groupings.","section":"Section III-A"},{"comment":"The qualitative activation-map analysis is based on a single illustrative example per emotion class and per ethnicity group. The text draws strong conclusions, such as \"Asians demonstrate activated maps on both sides of the face instead of one\" and that different cultures express surprise with different facial muscle regions, but no quantification, subject-level aggregation, or statistical testing is provided. These observations are anecdotal and do not provide reliable evidence for the paper's challenge to the universality hypothesis. At minimum, the authors should show activation maps for multiple subjects and report averaged activation statistics with error bars.","section":"Section IV-C and Figures 3-4"}],"minor_comments":[{"comment":"The abstract uses \"stereo-ethnicity\" while the body consistently uses \"mixed ethnicity\"; please unify the terminology.","section":"Abstract"},{"comment":"The mapping of \"Others\" to Asian is described as \"manually heuristic\" in Table II and as part of \"mapping is done for data distribution balancing\" in Table I; the two descriptions are confusingly redundant and should be reconciled.","section":"Tables I and II"},{"comment":"The text refers to \"MEGC 2023 [6]\" but the cited reference is titled \"Megc2024\"; please correct the year or the citation.","section":"Related Work and References"},{"comment":"The prima facie experiments are said to use \"ResNet-18 off-the-shelf features\" with a random forest classifier, but this detail appears only in the prose and not in Table IV or its caption; please state the exact feature-extraction and classification protocol in the table caption for reproducibility.","section":"Section IV-A"},{"comment":"The abbreviation CDE is used before it is defined; the full phrase \"composite database evaluation\" appears only later in Section IV. Please define it at first use.","section":"Section III-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads more like a workshop paper than a journal article in its current form: the claims are broad relative to the evidence, and the experimental design does not yet separate ethnicity from dataset identity or architectural capacity. However, the topic is important and the code release is a positive element. I believe the confounds can be partially addressed with new experiments (ablation with shuffled labels, cross-dataset controls, significance tests), so major revision rather than rejection seems appropriate. The authors should also be encouraged to either soften the strong anti-universality framing or support it with more rigorous evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the paper is genuinely preliminary and says so: it builds an ethnicity-annotated CASME2+SAMM joint dataset and tests whether adding an ethnicity branch helps micro-expression recognition. Second, the headline claim—that ethnicity clearly influences micro-expression recognition—is not supported by the evidence as it stands. The best reading is that the paper opens a question and provides a resource, not that it answers the question.\n\nWhat is new: the joint dataset with heuristic ethnicity labels (Asian/non-Asian), the prima facie design, and the fusion framework. The code link is public. The authors also cite the relevant psychology literature (Chen et al., Cowen et al.) and position the work against MEGC cross-cultural spotting. Credit is due for the honest framing and for making the resource available.\n\nThe soft spots are in the experiments. The prima facie study uses 16 subjects per condition, reports no variance or significance, and the 0.04 average drop is within noise. Worse, the mixed condition is confounded with dataset identity: non-Asian subjects come from SAMM and Asian subjects mostly from CASME2, so the drop could be domain shift. That is the load-bearing comparison, so the central claim is fragile.\n\nThe benchmarking has the same structure. Table V compares a single ResNet-18 (0.6210) with a dual-backbone late-fusion model with extra parameters and an extra CCE loss (0.6639). No ablation replaces the ethnicity labels with shuffled labels, gender, or a second emotion task while keeping the architecture fixed, so the specific contribution of ethnicity is unidentified. The RGB comparison changes architecture at the same time as modality. The paper's own conclusion admits a causality study is needed, which is honest, but the abstract and conclusion still claim to have 'validated' the influence. That is too strong.\n\nThe ethnicity labels themselves are the other soft spot. FaceXFormer plus heuristic eye-correction without inter-annotator agreement or validation could inject label noise, and the Asian/non-Asian dichotomy lumps Indians with Asians and 'Others' into Asian with no data support.\n\nWho is this for: people working on cross-cultural micro-expression recognition, or anyone who wants a cautionary example of confounded experimental design. It is not the last word on universality, and it doesn't need to be. It deserves a serious referee, because the dataset and question are worth someone's time and the authors are transparent about their limitations, but the revision bar should be high: error bars or significance tests, a fixed-architecture ablation, and a dataset confound analysis.\n\nMy recommendation: get it a competent reviewer, but prepare a clear list of required experiments.","headline":"An honest, under-powered preliminary study; the dataset is worth having, but the experiments do not isolate ethnicity as the driver.","tokens_in":8722,"tokens_out":2214,"would_cite":true,"duration_ms":24038,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that ethnicity affects micro-expression recognition and that adding an ethnicity branch to a motion classifier improves emotion prediction.","keywords":["ethnicity","micro-expression recognition","cross-cultural facial analysis","optical flow image","emotion universality","action units","auxiliary learning","vision transformer"],"falsifier":"Annotate the same CASME II/SAMM subjects with multiple independent human raters; if the mono-ethnic versus mixed-ethnic performance gap disappears or reverses when only consensus labels are used, the observed ethnic influence is annotation noise rather than ethnicity.","tokens_in":7792,"feed_emoji":"😮","tokens_out":7237,"duration_ms":70955,"temperature":0.7,"pith_summary":"The paper sets out to test whether ethnicity affects automatic micro-expression recognition, a question usually bypassed by the field's default assumption that emotional expressions are universal. It combines CASME II and SAMM into one dataset, labels each subject as Asian or non-Asian via an automated face-analysis model plus manual correction, and compares models trained on a single ethnic group with models trained on a mixed group. The mixed group loses accuracy, which the authors read as evidence of ethnic influence. They then add an ethnicity-aware branch to a motion-based classifier and report that macro-F1 rises from 0.6210 to 0.6639 with optical flow features, and to 0.6826 when RGB texture features from a vision transformer are included. The implication the paper argues for is that emotional universality is an overgeneralization, at least for micro-expressions.","feed_headline":"Ethnicity shifts micro-expression recognition","feed_subtitle":"Mixing Asian and non-Asian faces cuts accuracy, and adding an ethnicity branch restores it.","key_machinery":"The carrying mechanism is a dual-branch late-fusion architecture fed by an optical flow image. The optical flow image stacks horizontal and vertical motion displacement ($f_x$, $f_y$) with optical strain ($\\epsilon$), computed between onset and apex frames, so the same motion input drives two separate backbones: one fine-tuned with emotion labels (Positive/Negative/Surprise) and one with ethnicity labels (Asian/Non-Asian). Their features are concatenated and passed through a fully connected layer so the final emotion prediction can use ethnic context without weight sharing. Ethnic labels come from a semi-automatic pipeline (FaceXFormer with heuristic human-eye correction), and evaluation is leave-one-subject-out with composite database evaluation, reported as macro-F1.","core_discovery":"The central claim is that ethnicity is not a nuisance variable but a source of signal in micro-expression emotion classification. The paper establishes this in two steps. First, in a controlled prima facie design with 16 subjects per group and balanced emotion classes, the Asian-only and non-Asian-only setups score macro-F1 0.4546 and 0.4785 while the mixed-ethnicity setup scores 0.4375, with the mixed setup effectively losing the smaller positive and surprise classes. Second, in the main benchmark on the joint dataset, the plain motion-based ResNet-18 reaches 0.6210 average macro-F1, while the same motion backbone with an added ethnicity branch reaches 0.6639, and fusing a patch-based TinyViT trained on RGB face texture reaches 0.6826. Qualitative activation maps show different action-unit emphasis between Asian and non-Asian samples for happiness and surprise, which the paper presents as evidence against the emotion universality hypothesis.","pith_inferences":["Permuting the ethnicity labels during training would test whether the accuracy gain comes from ethnicity information or simply from adding a second task; if shuffled labels give the same gain, the extra capacity, not ethnicity, is responsible.","The Asian/non-Asian dichotomy pools small groups such as Indian and Others into the Asian side, so the measured gap likely underestimates the cultural variation present in the data.","The reported asymmetric AU activation for happiness and surprise implies a testable prediction: an expression classifier trained mostly on non-Asian faces should be systematically less accurate at detecting surprise and positive emotion on Asian faces."],"forward_implications":["A micro-expression model trained on one ethnic group may not transfer to another, so benchmarks should report or control the ethnic mix of training and test subjects.","Adding an ethnicity-aware branch is a way to recover part of the lost accuracy on this joint dataset, raising macro-F1 from 0.6210 to 0.6639 with motion features alone.","Ethnicity is learnable from optical flow motion, not only from static RGB face texture, since the motion-only dual-branch model already improves over the baseline.","The qualitative activation maps suggest that cross-cultural micro-expression analysis should attend to which facial regions are activated, not just which emotion label is produced.","The combined annotated dataset, while small and imbalanced, is a reusable testbed for further cross-cultural micro-expression studies."],"supporting_citations":[{"why":"supplies the automated ethnicity annotations that define the Asian/non-Asian split","marker":"[12]"},{"why":"provides the optical strain channel of the motion feature image","marker":"[13]"},{"why":"provides the optical flow estimation used to build the motion input","marker":"[14]"},{"why":"states the emotion universality hypothesis that the paper argues is overgeneralized","marker":"[5]"},{"why":"prior evidence that cultural groups differ in macro-expression dynamics, which the paper extends to micro-expressions","marker":"[7]"},{"why":"earlier cross-cultural micro-expression challenge whose setting is extended with explicit ethnic labels","marker":"[6]"}],"fun_headline_variants":["Ethnicity skews micro-expression recognition","Ethnic branch lifts micro-expression accuracy","Mixed-ethnicity faces reduce expression F1","Ethnic-aware model beats emotion-universal baseline"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the automatically assigned Asian/non-Asian labels are correct, because the mono-ethnic versus mixed-ethnic performance gap is measured on those labels and no inter-annotator agreement or ground-truth validation is reported.","fun_headline_variants_meta":{"raw":{"variants":["Ethnicity skews micro-expression recognition","Ethnic branch lifts micro-expression accuracy","Mixed-ethnicity faces reduce expression F1","Ethnic-aware model beats emotion-universal baseline"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000279,"raw_usage":{"total_tokens":1676,"prompt_tokens":985,"completion_tokens":691,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":634}},"tokens_in":601,"tokens_out":691,"duration_ms":8225,"temperature":1.0,"reasoning_tokens":634,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:35:35.268110+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Annotate the same CASME II/SAMM subjects with multiple independent human raters; if the mono-ethnic versus mixed-ethnic performance gap disappears or reverses when only consensus labels are used, the observed ethnic influence is annotation noise rather than ethnicity.","supporting_citations":[{"cited_title":"Towards macro-and micro-expression spotting in video using strain patterns,","cited_arxiv_id":null,"evidence_quote":"provides the optical strain channel of the motion feature image"},{"cited_title":"Secrets of optical flow estimation and their principles,","cited_arxiv_id":null,"evidence_quote":"provides the optical flow estimation used to build the motion input"},{"cited_title":"Universals and cultural differences in the judgments of facial expressions of emotion.,","cited_arxiv_id":null,"evidence_quote":"states the emotion universality hypothesis that the paper argues is overgeneralized"},{"cited_title":"Cultural facial expressions dynamically convey emotion category and intensity information,","cited_arxiv_id":null,"evidence_quote":"prior evidence that cultural groups differ in macro-expression dynamics, which the paper extends to micro-expressions"},{"cited_title":"Megc2024: Acm multimedia 2024 facial micro-expression grand challenge,","cited_arxiv_id":null,"evidence_quote":"earlier cross-cultural micro-expression challenge whose setting is extended with explicit ethnic labels"}],"review_version":1}