{"id":"21b08077-9edb-43c2-8956-d14953693b22","arxiv_id":"2505.07901","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A latent diffusion model with a context-aware autoencoder generates more diverse and synchronized listener facial reactions on REACT2024 than four comparison methods, at the cost of a slightly worse reaction-distance score.","lead":"This paper trains a two-stage model that compresses listener facial reactions into a compact latent space and then uses diffusion to generate multiple plausible reactions from a speaker's behavior. It reports better diversity and synchrony scores than four recent baselines on the REACT2024 benchmark, though one appropriateness metric remains worse than a competitor.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported FRDist contradicts SOTA-appropriateness claim: Dam et al. 84.94 vs Ours 89.40 on a lower-is-better metric, so the abstract's 'significantly outperforms' is not supported by Table 1.","rationale":"The reader's weakest assumption was the fairness of the comparison setup, which is relevant but not the most immediate problem. The more direct issue is that the paper's own Table 1 already undercuts the 'state-of-the-art in appropriateness' claim: on FRDist, a lower-is-better appropriateness metric, the proposed method is worse than Dam et al. and Liu et al. The reader's concern about protocol details matters for whether the baseline numbers are trustworthy, but even taking the numbers at face value, the headline claim is too strong. The proposed concrete check—running the official FRDist evaluation on both models under identical conditions—would settle whether the gap is real. If it is real, the contribution remains useful but the abstract and Section 3 should be weakened; if it disappears, the concern is resolved. The ablation study over denoising steps, PLMS orders, and sampler choices is a positive feature of the paper and gives some support to the methodological claims. The lack of released code and error bars remains an obstacle, so the reader's CONDITIONAL verdict is appropriate and I recommend keeping it unchanged.","tokens_in":10405,"tokens_out":7905,"duration_ms":78854,"concrete_test":"Take the official REACT2024 evaluation code for FRDist and recompute it for the proposed model and for Dam et al.'s released model, or retrain Dam et al. under the same train/test split, feature normalization, and number of generated sequences per speaker segment. Report bootstrap 95% confidence intervals across at least 5 random seeds. If the FRDist gap between Ours and Dam remains greater than 4 points with non-overlapping intervals, the abstract and Section 3 should be revised to claim improved diversity and synchrony with only competitive appropriateness. If the gap is within noise or reverses under the official protocol, the current SOTA-appropriateness claim can be retained.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 1 is the core evidence for the central claim. Appropriateness is measured by FRCorr (higher is better) and FRDist (lower is better). Under the paper's own numbers, the proposed model has FRCorr 0.37 (best) but FRDist 89.40, while Dam et al. [25] has 84.94 and Liu et al. [26] has 88.32. Since FRDist is an appropriateness metric, claiming state-of-the-art performance in appropriateness (Section 3) is not supported by the reported comparison: the model is third of five on one of the two appropriateness measures. The text acknowledges a 'trade-off' and calls the gap 'within a fair margin,' but no error bars, number of generated samples, metric-code version, or statistical test are reported, so the margin cannot be assessed. This is not an external-consensus issue; it is an internal mismatch between the headline and the results. Even if the model is best on diversity and synchrony, the strongest claim of overall superiority needs a defensible reading of FRDist.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage generative model for dyadic facial reaction generation (fMARG). A context-aware autoencoder compresses listener reaction sequences into a latent space, and a conditional latent diffusion model with a PLMS sampler generates multiple appropriate reactions from speaker behavior. The method is evaluated on the REACT2024 dataset against four baselines (Trans-VAE, BeLFusion, Dam et al., Liu et al.) using six reaction-level metrics plus FID. The authors report the best results on FRCorr, FRDiv, FRVar, FRDvs, and FRSyn, and on FID, but their FRDist value (89.40) is worse than Dam et al. (84.94) and Liu et al. (88.32). The paper claims in the abstract and in Section 3 that the method 'significantly outperforms recent methods' and achieves state-of-the-art appropriateness, diversity, and synchrony.","tokens_in":10634,"tokens_out":2781,"duration_ms":28542,"significance":"If the empirical claims were fully supported, the work would be a useful contribution to the fMARG literature: it demonstrates that a non-autoregressive latent diffusion approach with a transformer-based autoencoder prior and PLMS sampling can produce diverse and synchronized reactions, and it includes ablations over denoising steps, sampler types, and PLMS convergence orders. The authors are also to be credited for evaluating against two challenge baselines and for reporting FID in addition to the official metrics. However, the headline claim of state-of-the-art appropriateness is not supported by the reported numbers, and the absence of error bars, statistical tests, a precise evaluation protocol, and code makes the significance claim difficult to assess reproducibly. The central idea is plausible, but the paper currently overstates what its own Table 1 shows.","major_comments":[{"comment":"The claim that the method achieves state-of-the-art performance in appropriateness is internally contradicted by the paper's own results. FRDist is a lower-is-better appropriateness metric, and Table 1 reports Ours = 89.40, Dam et al. = 84.94, and Liu et al. = 88.32. Thus on one of the two appropriateness metrics the proposed model is third among the four compared methods, and the abstract's 'significantly outperforms recent methods' is not supported. The text in Section 4.3 acknowledges a 'trade-off' and calls the gap 'within a fair margin,' but without error bars or statistical tests there is no basis for calling the margin fair. Please either provide a corrected or extended comparison (e.g., rerunning baselines under identical conditions with multiple seeds) or substantially soften the appropriateness claim.","section":"Abstract, Section 3, and Table 1"},{"comment":"No error bars, confidence intervals, statistical significance tests, or number of random seeds are reported anywhere, so the word 'significantly' in the abstract is not justified by the evidence. This matters particularly for the FRDist comparison, where the reported gap between Ours and Dam et al. is large (89.40 vs. 84.94), and for the ablation table, where several configurations differ by only 0.001-0.002 in FRVar or FRDiv. Please report mean +/- standard deviation over at least three runs and apply a paired test (e.g., Wilcoxon or paired bootstrap) for the headline comparisons.","section":"Section 4.1 and Section 4.3, Tables 1-3"},{"comment":"The comparison omits the methods in references [27] and [28], which are highly relevant prior fMARG models, and [28] is from the same group as the current paper. Since the paper's contribution is framed as superiority over 'recent methods' and both references are cited, the omission needs explicit justification. If those methods are excluded for a specific reason (e.g., different input features or evaluation protocol), state the reason; otherwise include them in Table 1 or clearly position the current work as an incremental study relative to [28].","section":"Section 4.1, Comparison Methods"},{"comment":"The description of the autoencoder is ambiguous about whether the latent space is discrete or continuous. The text mentions a Vector Quantized technique and a codebook, then states the model avoids a standard Gaussian or a 1-D codebook, and later the diffusion operates on a continuous latent. Please clarify the exact training objective for the codebook (including the codebook size and commitment coefficient) and specify how the discrete and continuous branches interact, since this affects the reproducibility of the whole method.","section":"Section 3.2, Eq. (4)"}],"minor_comments":[{"comment":"There is a typo in the caption: 'Comparision' should be 'Comparison', and the table is preceded by 'T able 1'.","section":"Table 1 caption"},{"comment":"References [25] and [26] do not include publication venues or years; please complete the bibliographic information.","section":"References"},{"comment":"The notation in Eq. (2) mixes a set definition with inequality symbols; please rewrite it to make clear that the M reactions are distinct sampled outputs rather than a set of pairwise-inequivalent elements.","section":"Section 3.1, Eq. (2)"},{"comment":"The loss notation uses a transposed summation symbol 'TX' and tildes in inconsistent places; please clean up the mathematical typesetting for readability.","section":"Section 3.3, Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central claim is stronger than its evidence. The FRDist result alone would justify a major revision even if every other metric favored the method. I also note that reference [28] is a same-group prior method that is not compared; the editor may wish to ask the authors to clarify the incremental contribution relative to that work. The paper could become publishable after a careful re-evaluation with error bars, a clarified protocol, and either added baselines or qualified claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read on arXiv:2505.07901. The core is straightforward: a VQ-VAE compresses listener reaction sequences into a discrete latent space, then a conditional latent diffusion model (conditioned with an encoder reused from the first stage) generates reactions non-autoregressively with a PLMS sampler. That's a sensible composition of known parts—BeLFusion already did latent diffusion for dyadic motion, and the authors' own prior VQ-diffusion paper applied diffusion to this exact fMARG task. The genuinely new bit is the context-aware autoencoder prior and the way the semantic encoder is reused as the diffusion condition. That's a reasonable extension, not a revolution.\n\nWhat the paper does well: the ablation study over denoising steps and PLMS orders is useful and honest about trade-offs. The reported gains on diversity (FRDiv, FRVar, FRDvs) and synchrony (FRSyn) over Trans-VAE, BeLFusion, Dam, and Liu are consistent across the table, and the FID improvement (50.95 vs 54.00) is another point in its favor. If those numbers hold, the method is competitive and likely useful for virtual agent work.\n\nWhere it's soft: the central claim doesn't match its own table. FRDist is an appropriateness metric (lower is better), and Ours is 89.40, worse than Dam (84.94) and Liu (88.32). The text acknowledges a 'trade-off' and calls it 'within a fair margin,' but with no error bars, no statistical tests, and no release of code or metric implementation, there's no way to evaluate that margin. The abstract's 'significantly outperforms' is simply not supported. Also, the authors omit their own prior VQ-diffusion model (ref 28) from Table 1, which is the most important baseline for an incremental claim. That looks like cherry-picking, even if unintentional.\n\nThe reader's concern about evaluation setup is fair: no split details, no metric code version, and no indication whether baselines were rerun under identical conditions. Those details matter for a single-dataset benchmark.\n\nBottom line: this is a competent, incremental paper with some real empirical gains hidden behind an overclaimed headline. If the authors add error bars, include their previous model, and soften the SOTA language, it could be a solid workshop or short conference paper. I'd send it to review, but I'd expect heavy revision.","headline":"Incremental but competently built latent diffusion for fMARG; the headline 'SOTA appropriateness' is contradicted by its own FRDist column, but the diversity/synchrony gains look real.","tokens_in":11194,"tokens_out":2556,"would_cite":false,"duration_ms":20339,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A latent diffusion model generates more diverse, synchronized listener reactions in dyadic conversations, beating prior methods on the REACT2024 benchmark.","keywords":["latent diffusion","dyadic interaction","facial reaction generation","multiple appropriate reactions","non-autoregressive generation","action units","REACT2024","diversity and synchrony"],"falsifier":"Re-running Trans-VAE, BeLFusion, Dam et al., and Liu et al. under exactly the same splits, metric code, and compute budget and finding that one of them matches or beats the reported FRDiv, FRDvs, FRSyn, or FID values would directly falsify the claim of state-of-the-art performance.","tokens_in":10205,"feed_emoji":"🎭","tokens_out":5186,"duration_ms":45930,"temperature":0.7,"pith_summary":"This paper tries to establish that a two-stage latent diffusion model—a context-aware autoencoder that compresses listener reaction sequences into a compact latent code, plus a diffusion generator that samples in that latent space—can produce multiple appropriate facial reactions to a speaker's behavior. The authors argue that existing deterministic and Gaussian-prior models either collapse to a single reaction or suffer posterior collapse, while their latent diffusion with a Transformer-based autoencoder prior captures the one-to-many mapping between speaker behavior and plausible listener reactions. If true, the reported REACT2024 results (best FRCorr, FRDiv, FRVar, FRDvs, FRSyn, and FID among compared methods) would mean that latent-space diffusion is a practical recipe for generating diverse, synchronized, and contextually appropriate non-verbal reactions.","feed_headline":"Latent diffusion beats prior methods on facial-reaction benchmark","feed_subtitle":"Two-stage autoencoder plus diffusion yields more diverse, synchronized listener reactions in dyadic conversations.","key_machinery":"The central mechanism is the Latent Behavior Diffusion Model, made of two stages: a context-aware autoencoder trained with VQ-VAE-style losses to map fixed-length listener reaction sequences into low-dimensional latent codes, and a conditional latent diffusion generator whose denoising network is a residual MLP conditioned on a semantic code extracted from the speaker behavior by the same encoder. The generator learns to denoise in latent space with a composite loss combining MAE in latent space and MSE in reconstructed space, and is sampled non-autoregressively using a fourth-order PLMS (Pseudo Linear Multi-Step) sampler.","core_discovery":"The paper claims its Latent Behavior Diffusion Model outperforms recent methods in appropriateness, diversity, and synchrony of generated facial reactions on REACT2024, with reported values of FRCorr 0.37, FRDist 89.40, FRDiv 0.1211, FRVar 0.0653, FRDvs 0.1505, FRSyn 43.48, and FID 50.95. The key discovery is that running the diffusion process in the low-dimensional latent space of a context-aware autoencoder avoids the posterior collapse seen in Gaussian-prior VAE approaches and the diversity limits of one-dimensional discrete codebooks, while the non-autoregressive reverse sampler (fourth-order PLMS with 50 steps) keeps the generated reactions both diverse and synchronized with the speaker.","pith_inferences":["An implication the authors leave implicit is that their results, if the comparison is fair, weaken the case for discrete tokenization in this task: continuous latent diffusion can match or beat finite-scalar and discrete-latent baselines on diversity and synchrony.","A testable extension is whether the model generalizes to unseen listeners without fine-tuning; the paper does not measure identity preservation beyond image-level FID on rendered faces.","The two-stage design implies that the autoencoder's reconstruction fidelity bounds the final reaction quality; one could probe this by varying the latent compression ratio and tracking FRDist and FRSyn.","If the claimed performance holds, the 50-step PLMS sampler is short enough to suggest near-real-time interactive agents that generate varied listener feedback in video calls or social robots, though runtime is not reported."],"forward_implications":["The two-stage latent diffusion recipe can be adapted to other one-to-many sequence generation tasks, such as gesture or body-motion synthesis, where diverse plausible outputs are required from the same context.","The improved FRVar and FRDiv indicate the model produces a wider spread of distinct reactions from the same speaker input, directly addressing the one-to-many mapping problem.","The lower FRSyn suggests the generated reactions stay better synchronized with the speaker's behavior across time, which matters for natural human-like interaction.","The lower FID relative to ground-truth rendering implies the predicted 3DMM parameters, when rendered to images, look more realistic than the ground-truth renderings used in the comparison.","The ablation showing 50 denoising steps as the best balance implies the method is computationally practical and does not need very long sampling chains."],"supporting_citations":[{"why":"Defines the fMARG task and the appropriateness, diversity, and synchrony metrics that all comparisons rely on.","marker":"[11]"},{"why":"Provides the REACT2024 dataset and challenge protocol, including the baseline methods and evaluation setup.","marker":"[13]"},{"why":"Supplies the latent diffusion modeling idea that the generator adapts to reaction sequences.","marker":"[12]"},{"why":"Introduces diffusion autoencoders, the basis for the conditional behavior decoder's residual MLP and semantic conditioning.","marker":"[29]"},{"why":"Provides the PLMS sampler whose fourth-order variant gives the best reported results.","marker":"[30]"},{"why":"Establishes the denoising diffusion probabilistic model formalism used for the forward/reverse Markov chain.","marker":"[31]"},{"why":"A discrete-latent baseline that the model claims to outperform on diversity and synchrony.","marker":"[25]"},{"why":"The BeLFusion baseline, a Gaussian-prior latent diffusion method that the model is compared against and outperforms.","marker":"[38]"}],"fun_headline_variants":["Latent diffusion yields diverse, synchronized dyadic reactions","Latent-space diffusion sidesteps posterior collapse in reactions","Diffusion on latent codes beats VAE posterior collapse for reactions","Non-autoregressive latent diffusion improves dyadic facial reactions","Latent diffusion for dyadic reactions: diverse and synchronized"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed gains rest on the comparisons being fair: identical REACT2024 train and test splits, identical implementations of the metrics, and comparable training budgets across every method, none of which the paper documents.","fun_headline_variants_meta":{"raw":{"variants":["Latent diffusion yields diverse, synchronized dyadic reactions","Latent-space diffusion sidesteps posterior collapse in reactions","Diffusion on latent codes beats VAE posterior collapse for reactions","Non-autoregressive latent diffusion improves dyadic facial reactions","Latent diffusion for dyadic reactions: diverse and synchronized"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001132,"raw_usage":{"total_tokens":4661,"prompt_tokens":858,"completion_tokens":3803,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":3722}},"tokens_in":474,"tokens_out":3803,"duration_ms":24909,"temperature":1.0,"reasoning_tokens":3722,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:17:46.645969+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-running Trans-VAE, BeLFusion, Dam et al., and Liu et al. under exactly the same splits, metric code, and compute budget and finding that one of them matches or beats the reported FRDiv, FRDvs, FRSyn, or FID values would directly falsify the claim of state-of-the-art performance.","supporting_citations":[{"cited_title":"Diffusion autoencoders: Toward a meaningful and decodable rep- resentation","cited_arxiv_id":null,"evidence_quote":"Introduces diffusion autoencoders, the basis for the conditional behavior decoder's residual MLP and semantic conditioning."},{"cited_title":"Finite scalar quantization as facial tokenizer for dyadic reaction generation","cited_arxiv_id":null,"evidence_quote":"A discrete-latent baseline that the model claims to outperform on diversity and synchrony."},{"cited_title":"Belfusion: Latent diffusion for behavior-driven human motion prediction","cited_arxiv_id":null,"evidence_quote":"The BeLFusion baseline, a Gaussian-prior latent diffusion method that the model is compared against and outperforms."}],"review_version":1}