{"id":"3583b554-957f-4d48-9ba3-c1680e63d62a","arxiv_id":"2505.04996","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"An inter-diffusion model generates speaker and listener full-body gestures from speech audio by coupling dual-branch denoisers with role-specific condition scaling.","lead":"This paper trains a diffusion model to generate full-body gestures for both a speaker and a listener in conversation, using audio as the main input. It claims a new inter-diffusion mechanism and a GAN speedup, but the GAN is absent from the method and the listener training data is stitched together from two datasets with interpolation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The model's central interaction claim cannot be supported by its training data: Section 4.1 pairs ZeroEGG speaker audio with slerp-resynchronized TWH listener clips that were never recorded together, so the model never observes a real speaker-listener coupling.","rationale":"The paper proposes a dyadic gesture-generation model whose central contribution is that listener full-body gestures are generated in response to speaker audio and listener feedback. The claim can hold only if the training data contains genuine speaker-listener couplings. Section 4.1 shows the couplings are constructed by slerp-resynchronizing TWH listener clips to ZeroEGG audio; the two datasets were not recorded together. This is the same load-bearing weakness the reader identified, and I agree it is fatal to the stated novelty claim. A random-permutation retraining test would settle whether the listener branch depends on the audio condition at all; absent such a control, the reported user-study scores could reflect generic listener-motion plausibility rather than interaction quality. I do not count the absence of code or formal verification as an independent failure, but I do note the paper's own limitation section contradicts the abstract's realtime/GAN claim: Section 6 says Markov-chain DDPM is too slow for realtime use, and no adversarial loss appears in Eq. (11). I selected the data-pairing concern as most load-bearing because it targets the core 'interaction' claim, not just the speed-up. The reader's REJECT verdict remains appropriate.","tokens_in":13453,"tokens_out":8438,"duration_ms":88052,"concrete_test":"Permutation control: retrain the identical model with the same hyperparameters and loss on the same ZeroEGG audio clips but with TWH listener clips randomly reassigned across audios, preserving the slerp resynchronization. Then rerun the Table 2 user study with a fresh participant pool under the same protocol, plus FGD/BA/DIV on the speaker side, for both models. If the shuffled-control listener-coherence and naturalness scores are statistically indistinguishable from the reported 'Ours' values (4.17, 4.20, 3.88, 4.01), the listener branch is not learning audio-conditioned interaction. If the shuffled control does degrade, the authors should still validate on a genuinely co-occurring dyadic dataset before claiming 'synergistic interaction.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 (Dataset Design) is the load-bearing point. The method extracts listener motion from TWH, applies slerp interpolation so the motion length matches ZeroEGG audio, and then trains on these paired clips. No evidence is given that any TWH listener clip is a natural response to the ZeroEGG audio it is paired with; the two corpora are independent and joined only by length. Because the diffusion training target in Eq. (10) reconstructs a motion sequence that was never actually produced in response to the conditioning audio, the dual-branch denoiser can at best learn statistics of the forced alignment, not conversational interaction between two people. This is an internal-evidence problem, not an external-consensus objection: the paper itself describes the synthetic alignment. The listener branch also receives no objective validation: Table 1 reports speaker-side FGD/BA/DIV, and the 'Listener-coherence' user score in Table 2 lacks any control condition or comparison against real dyadic recordings. Separately, the abstract's claim of GAN-accelerated realtime generation is not present in the method: Eq. (11) has no adversarial term, no discriminator or GAN loss is defined, and Section 6 admits the Markov-chain DDPM is too slow for realtime use. That is a serious internal inconsistency, but the synthetic pairing is more fundamental because it undermines the claimed listener-interaction contribution itself.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Inter-Diffusion, a dual-branch diffusion architecture for jointly generating speaker and listener full-body gestures conditioned on the same speech audio. The speaker branch receives a strongly weighted audio condition (λC), the listener branch a weakly weighted one ((1−λ)C), and the two branches are coupled through cross-local attention and cross-attention modules. The model is trained on a paired dataset built by slerp-interpolating listener motion clips from the TWH dataset to match the length of speech clips from the ZeroEGG dataset. Experiments compare the speaker-side outputs to several baselines using FGD, BA, and DIV, and a user study rates human-likeness, gesture appropriateness, emotion compatibility, and listener coherence. The paper claims, for the first time, to generate listener gestures and to achieve GAN-accelerated realtime generation. However, the method section does not include any GAN component, and the training data are not real co-occurring speaker-listener exchanges.","tokens_in":13756,"tokens_out":6423,"duration_ms":57141,"significance":"If the method were trained and validated on real dyadic interaction data, the idea of modeling speaker and listener jointly with role-scaled conditioning would be a worthwhile contribution to co-speech gesture generation. The paper is also transparent in its limitations section about the speed issue. However, the two most consequential claims—capturing genuine speaker-listener interaction and GAN-accelerated realtime generation—are not supported by the evidence as presented: the training data are fabricated by alignment, the method has no adversarial component, and the evaluation omits objective listener metrics and meaningful baselines. The paper does not release code or data, and the user-study numbers are reported without statistical tests.","major_comments":[{"comment":"The dataset construction in Section 4.1 pairs ZeroEGG speaker audio with TWH listener motion clips by slerp interpolation to match sequence lengths, but the paper provides no evidence that any TWH listener clip is a natural response to the ZeroEGG audio with which it is paired. Because the diffusion training target in Eq. (10) reconstructs these synthetically aligned pairs, the dual-branch denoiser can at best learn statistics of the forced alignment rather than genuine speaker-listener interaction, which is the central contribution claimed in the abstract and Section 1. This is an internal-evidence problem: the paper itself describes the pairing as a length-matching procedure, not as semantically synchronized dyadic data.","section":"Section 4.1 (Dataset Design)"},{"comment":"No adversarial loss or discriminator appears anywhere in the method: the training objective in Eq. (11) consists only of the simple diffusion loss L_simple (Eq. 10) and the foot-geometry loss L_foot. Yet the abstract claims that 'the GAN model' is introduced to increase the denoising step size and enable realtime response, and the Implementation Details paragraph in Section 4.1 mentions a discriminator learning rate and monitoring of generator and discriminator losses. Section 6 explicitly concedes that the Markov-chain DDPM is too slow for real-time applications. The GAN/realtime claim is therefore internally inconsistent with the method and the authors' own limitation statement.","section":"Section 3.3 (Loss Function) and Section 6"},{"comment":"Table 1 reports FGD, BA, and DIV only for the speaker branch, so the listener branch receives no objective validation. In Table 2, the 'Listener-coherence' score for Ours (4.01±0.17) overlaps with Ground Truth (4.12±0.31) and is not accompanied by any statistical significance test; moreover, the two baselines (DSG and DiffGesture) are speaker-only methods, so a listener-coherence comparison does not demonstrate that the proposed model produces better listener gestures than any existing listener-aware alternative (of which none is evaluated). The claims of superiority in listener coherence are therefore not established.","section":"Tables 1 and 2"}],"minor_comments":[{"comment":"The phrase 'reverse-connect the listener's actions' is undefined; the description of the slerp alignment only explains length matching, not any semantic synchronization.","section":"Section 4.1"},{"comment":"The citation formatting is inconsistent: the text reads 'Tri-modal's [28] 22.4 [16], and HA2G's [15] 18.8', and Table 1 lists 'CAMN [15]' while reference [15] is the BEAT dataset paper; please correct the citation keys.","section":"Table 1 and surrounding text"},{"comment":"The 'Ground Truth' condition in the user study is not described; it is unclear whether it consists of real TWH listener clips and how those are synchronized with the ZeroEGG test audio.","section":"Section 4.3"},{"comment":"The notation '⊕' is used for both concatenation in Eq. (3) and for conditioning additions in Eqs. (4)–(9); please distinguish the two operations or define the symbols explicitly.","section":"Section 3.3"},{"comment":"The phrase 'increase the denoising step size' is ambiguous; if the intent is fewer denoising steps, please state that, and align the abstract and Section 6 (which says the model is too slow for real-time) to avoid direct contradiction.","section":"Abstract and Section 6"},{"comment":"The paper does not provide code, data, or a link to a project page, which limits reproducibility of the reported results.","section":"General"}],"recommendation":"reject","confidential_remarks":"To the editor: The manuscript appears to have been formatted for a conference (ICMR '25), and the claims in the abstract are stronger than what the method and experiments support. The two most serious issues—the synthetic dataset pairing and the missing GAN component—are not fixable by local edits; they require either new real dyadic data and re-training or a substantial rewrite of the claims. The heavy reliance on the authors' own prior work (refs [3,4,6]) without explaining their role in the method also warrants attention."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper is the first to treat full-body listener gesture generation as a joint task with speaker gestures, and the architecture is a plausible way to model two roles with different condition strengths. But the central interaction claim is undercut by the training data, and the abstract overstates the method. Not ready for peer review as is.\n\nWhat's new: the task framing (joint speaker/listener generation with role-dependent conditioning) and the dual-branch denoiser with cross-attention between branches. Equations 8-9, scaling the same audio condition differently for speaker and listener, is a simple but reasonable idea. The user study at least attempts to evaluate listener coherence, which no one else has done.\n\nWhere it falls down. Section 4.1 pairs TWH listener clips with ZeroEGG audio by slerp interpolation to match lengths. Those clips were never recorded with that audio, so the model never observes a real speaker-listener exchange. It can at best learn statistics of a forced alignment, not interaction. That's not an external critique; the paper describes the alignment itself. The listener branch also has no objective validation: Table 2's 'Listener-coherence' is a subjective score with no comparison to real dyadic recordings or a no-interaction control.\n\nSecond, the abstract promises a GAN that increases the denoising step size and enables realtime response, but Section 3 has no discriminator or adversarial loss; Equation 11 is just L_simple plus a foot-geometry term. Section 6 admits the Markov-chain DDPM is too slow for realtime. That's a direct internal contradiction.\n\nThird, evaluation details: no error bars on FGD/BA/DIV, and Table 3 lists DSG FGD as 25.7 while Table 1 gives 15.67 for the same method. Could be a typo, but it makes the ablation table hard to trust. No code or data release, so nothing is independently checkable.\n\nWho this is for: researchers working on co-speech gesture generation might find the task framing and the role-condition trick worth a skim. But as a research contribution, the paper doesn't support its claims. I would desk-reject in current form. The data pairing problem alone is disqualifying, and the method/abstract mismatch would need to be fixed before any serious review.","headline":"Interesting task framing and a plausible dual-branch architecture, but the synthetic speaker-listener data pairing and the missing GAN make the central claims unsupported.","tokens_in":14276,"tokens_out":2910,"would_cite":false,"duration_ms":27735,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes an inter-diffusion model that generates full-body gestures for both speaker and listener from one speech signal, coupling their denoising paths with role-specific audio conditions.","keywords":["gesture generation","diffusion model","listener gestures","speaker-listener interaction","inter-diffusion","co-speech gesture","beat alignment","virtual human animation"],"falsifier":"A decisive test would be to retrain the identical Inter-Diffusion model on motion-captured dyads in which a real speaker and real listener move simultaneously, then compare listener-coherence against the current model in a blinded user study. A cheaper, still decisive check is to compute whether the TWH listener motions, after slerp alignment, carry any measurable correlation with the ZeroEGG audio; near-chance correlation would mean the training signal for listener responsiveness is absent.","tokens_in":13232,"feed_emoji":"🤝","tokens_out":9505,"duration_ms":90658,"temperature":0.7,"pith_summary":"This paper tries to close a gap in co-speech gesture generation: until now, models animate the speaker but leave the listener static. It claims that a single diffusion-based model, called Inter-Diffusion, can generate full-body gestures for both people at once, with the listener's nodding, posture, and head movements shaped by the same speech signal but with a looser condition weight. The payoff, if the claim holds, is that virtual conversations look like exchanges rather than monologues, with listener reactions that feel contingent on what is being said. The paper reports objective gains in gesture quality and beat alignment on the ZeroEGG dataset and higher user-study ratings for human-likeness and listener coherence.","feed_headline":"One diffusion model generates speaker and listener gestures together","feed_subtitle":"Splitting the audio condition between roles lets the listener's body react while the speaker's hands talk.","key_machinery":"The named central object is the inter-diffusion mechanism: a conditional denoising process that treats the speaker and listener as two separate latent streams, $x^s_t$ and $x^l_t$, concatenated with a role encoding that tells the network which body it is denoising. Each stream passes through a shared Cross Local Attention module conditioned on role-scaled audio ($\\lambda C$ for the speaker, $(1-\\lambda) C$ for the listener), the streams are then fused by self-attention, and cross-attention re-splits the fused representation to predict the clean motions $x^s_0$ and $x^l_0$. The mechanism's job is to let the two roles share a learned interaction context without letting one role's features drown out the other; the role encoding and asymmetric condition weights are what carry the role distinction, and the loss is the standard diffusion variational bound plus a foot-contact geometry term.","core_discovery":"The central claim the paper is trying to establish is that listener gestures can be brought inside a gesture-diffusion framework rather than appended afterward. The model represents the momentary motion of both people as two noisy trajectories, runs them through separate branch modules with a shared local attention mechanism, then concatenates their latent representations and lets self- and cross-attention couple them. The audio condition is split by a weight λ: the speaker's branch receives λC and the listener's branch receives (1−λ)C, so the speech constrains the speaker tightly while leaving the listener freer to produce plausible listening behavior. The paper's strongest formulation is that this is the first integration of full-body listener gestures into a generation framework, and that the resulting two-party motion is synchronized and mutually responsive rather than a spliced pair of independent animations. An adversarial component is added with the stated purpose of allowing larger denoising steps, though the paper's own limitation section concedes that the Markov-chain denoising still limits generation speed.","pith_inferences":["Editorial inference: the λ-split conditioning is a general recipe for any asymmetric two-party generation task—turn-taking dialogue, teacher-student scenes, interviewer-interviewee—where one party is constrained by the stimulus and the other is freer.","Editorial inference: the paper does not test whether removing the listener branch changes the speaker's generated motion; a model that is truly interactive should show such coupling, so an ablation that varies listener context while holding audio fixed would sharpen the claim.","Editorial inference: the paper's own Limitation section concedes that Markov-chain denoising is slow, which sits in tension with the abstract's 'real-time' wording; the reported experiments do not quantify generation latency, so the speed advantage of the adversarial component remains unverified."],"forward_implications":["Virtual characters in games, films, and conferencing could show coordinated listening behavior—nods, head tilts, posture shifts—rather than standing still while the speaker gestures.","The role-split weight λ becomes a practical control: raising it makes the listener's motion track the speech more tightly, lowering it makes the listener react more freely.","Because both roles are produced by one denoising pass, generating a two-character conversation no longer requires stitching together separately trained speaker and listener models, which should reduce inconsistency between the two bodies.","If the adversarial module does enlarge denoising steps, the same architecture becomes a candidate for interactive avatars that update gestures during a conversation."],"supporting_citations":[{"why":"Supplies the human-motion diffusion backbone and the foot-geometry loss that the Inter-Diffusion model inherits.","marker":"[20]"},{"why":"Supplies the classifier-free guidance combination in Eq. (2) used at sampling.","marker":"[12]"},{"why":"Defines the forward noising process and variational-bound objective that the denoiser optimizes.","marker":"[11]"},{"why":"Provides the cross-local attention module reused for both role branches and serves as a styled-gesture baseline.","marker":"[24]"},{"why":"Provides a contemporary diffusion-based baseline whose objective metrics set the comparison level.","marker":"[25]"},{"why":"Provides the DiffGesture baseline that is re-trained and compared on the same data.","marker":"[32]"},{"why":"Introduces the Fréchet Gesture Distance objective metric used to judge gesture quality.","marker":"[27]"},{"why":"Provides the beat-alignment metric used to judge speech-gesture synchronization.","marker":"[13]"}],"fun_headline_variants":["First model to generate speaker and listener gestures in one diffusion pass","Inter-diffusion couples speaker and listener gestures with audio split","Listener gestures generated inside diffusion model via inter-diffusion","Unified diffusion for responsive speaker-listener gesture pairs","Audio-conditioned inter-diffusion lets listeners react in real time"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the listener motions borrowed from the TWH dataset, after being time-stretched to fit ZeroEGG audio, are genuine and naturally synchronized listener reactions to that audio; if they are not genuine pairs, the model never sees a real two-person interaction and cannot learn the contingency it claims.","fun_headline_variants_meta":{"raw":{"variants":["First model to generate speaker and listener gestures in one diffusion pass","Inter-diffusion couples speaker and listener gestures with audio split","Listener gestures generated inside diffusion model via inter-diffusion","Unified diffusion for responsive speaker-listener gesture pairs","Audio-conditioned inter-diffusion lets listeners react in real time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000657,"raw_usage":{"total_tokens":3035,"prompt_tokens":1002,"completion_tokens":2033,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":1949}},"tokens_in":618,"tokens_out":2033,"duration_ms":15110,"temperature":1.0,"reasoning_tokens":1949,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:15:49.928142+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test would be to retrain the identical Inter-Diffusion model on motion-captured dyads in which a real speaker and real listener move simultaneously, then compare listener-coherence against the current model in a blinded user study. A cheaper, still decisive check is to compute whether the TWH listener motions, after slerp alignment, carry any measurable correlation with the ZeroEGG audio; near-chance correlation would mean the training signal for listener responsiveness is absent.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the forward noising process and variational-bound objective that the denoiser optimizes."},{"cited_title":"G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Model","cited_arxiv_id":"2208.09141","evidence_quote":"Provides the cross-local attention module reused for both role branches and serves as a styled-gesture baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides a contemporary diffusion-based baseline whose objective metrics set the comparison level."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the DiffGesture baseline that is re-trained and compared on the same data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Fréchet Gesture Distance objective metric used to judge gesture quality."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the beat-alignment metric used to judge speech-gesture synchronization."}],"review_version":1}