Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communication

Gaoge Han; Jinewei Li; Jing Zhang; Jinhe Huang; Xingjian Gu; Yongkang Cheng; Yuming Hang

REVIEW 3 major objections 6 minor 35 references

Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communication

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes an inter-diffusion model that generates full-body gestures for both speaker and listener from one speech signal, coupling their denoising paths with role-specific audio conditions.

desk verdict Interesting task framing and a plausible dual-branch architecture, but the synthetic speaker-listener data pairing and the missing GAN make the central claims unsupported. read the letter →

arxiv 2505.04996 v1 pith:GGKVHGSU submitted 2025-05-08 cs.GR cs.CVcs.SDeess.AS

Jinhe Huang , Yongkang Cheng , Yuming Hang , Gaoge Han , Jinewei Li , Jing Zhang , Xingjian Gu This is my paper · ORCID

classification cs.GRcs.CVcs.SDeess.AS

keywords gesturegenerationdiffusionmodellistenergesturesspeaker-listenerinteractioninter-diffusionco-speechbeatalignmentvirtualhumananimation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to close a gap in co-speech gesture generation: until now, models animate the speaker but leave the listener static. It claims that a single diffusion-based model, called Inter-Diffusion, can generate full-body gestures for both people at once, with the listener's nodding, posture, and head movements shaped by the same speech signal but with a looser condition weight. The payoff, if the claim holds, is that virtual conversations look like exchanges rather than monologues, with listener reactions that feel contingent on what is being said. The paper reports objective gains in gesture quality and beat alignment on the ZeroEGG dataset and higher user-study ratings for human-likeness and listener coherence.

What carries the argument

The named central object is the inter-diffusion mechanism: a conditional denoising process that treats the speaker and listener as two separate latent streams, $x^s_t$ and $x^l_t$, concatenated with a role encoding that tells the network which body it is denoising. Each stream passes through a shared Cross Local Attention module conditioned on role-scaled audio ($\lambda C$ for the speaker, $(1-\lambda) C$ for the listener), the streams are then fused by self-attention, and cross-attention re-splits the fused representation to predict the clean motions $x^s_0$ and $x^l_0$. The mechanism's job is to let the two roles share a learned interaction context without letting one role's features drown out the other; the role encoding and asymmetric condition weights are what carry the role distinction, and the loss is the standard diffusion variational bound plus a foot-contact geometry term.

What would settle it

A decisive test would be to retrain the identical Inter-Diffusion model on motion-captured dyads in which a real speaker and real listener move simultaneously, then compare listener-coherence against the current model in a blinded user study. A cheaper, still decisive check is to compute whether the TWH listener motions, after slerp alignment, carry any measurable correlation with the ZeroEGG audio; near-chance correlation would mean the training signal for listener responsiveness is absent.

Watch

Extended reading notes

Core claim

The central claim the paper is trying to establish is that listener gestures can be brought inside a gesture-diffusion framework rather than appended afterward. The model represents the momentary motion of both people as two noisy trajectories, runs them through separate branch modules with a shared local attention mechanism, then concatenates their latent representations and lets self- and cross-attention couple them. The audio condition is split by a weight λ: the speaker's branch receives λC and the listener's branch receives (1−λ)C, so the speech constrains the speaker tightly while leaving the listener freer to produce plausible listening behavior. The paper's strongest formulation is that this is the first integration of full-body listener gestures into a generation framework, and that the resulting two-party motion is synchronized and mutually responsive rather than a spliced pair of independent animations. An adversarial component is added with the stated purpose of allowing larger denoising steps, though the paper's own limitation section concedes that the Markov-chain denoising still limits generation speed.

Load-bearing premise

The load-bearing premise is that the listener motions borrowed from the TWH dataset, after being time-stretched to fit ZeroEGG audio, are genuine and naturally synchronized listener reactions to that audio; if they are not genuine pairs, the model never sees a real two-person interaction and cannot learn the contingency it claims.

Editorial extensions

If this is right

Virtual characters in games, films, and conferencing could show coordinated listening behavior—nods, head tilts, posture shifts—rather than standing still while the speaker gestures.
The role-split weight λ becomes a practical control: raising it makes the listener's motion track the speech more tightly, lowering it makes the listener react more freely.
Because both roles are produced by one denoising pass, generating a two-character conversation no longer requires stitching together separately trained speaker and listener models, which should reduce inconsistency between the two bodies.
If the adversarial module does enlarge denoising steps, the same architecture becomes a candidate for interactive avatars that update gestures during a conversation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the λ-split conditioning is a general recipe for any asymmetric two-party generation task—turn-taking dialogue, teacher-student scenes, interviewer-interviewee—where one party is constrained by the stimulus and the other is freer.
Editorial inference: the paper does not test whether removing the listener branch changes the speaker's generated motion; a model that is truly interactive should show such coupling, so an ablation that varies listener context while holding audio fixed would sharpen the claim.
Editorial inference: the paper's own Limitation section concedes that Markov-chain denoising is slow, which sits in tension with the abstract's 'real-time' wording; the reported experiments do not quantify generation latency, so the speed advantage of the adversarial component remains unverified.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this paper is the first to treat full-body listener gesture generation as a joint task with speaker gestures, and the architecture is a plausible way to model two roles with different condition strengths. But the central interaction claim is undercut by the training data, and the abstract overstates the method. Not ready for peer review as is.

What's new: the task framing (joint speaker/listener generation with role-dependent conditioning) and the dual-branch denoiser with cross-attention between branches. Equations 8-9, scaling the same audio condition differently for speaker and listener, is a simple but reasonable idea. The user study at least attempts to evaluate listener coherence, which no one else has done.

Where it falls down. Section 4.1 pairs TWH listener clips with ZeroEGG audio by slerp interpolation to match lengths. Those clips were never recorded with that audio, so the model never observes a real speaker-listener exchange. It can at best learn statistics of a forced alignment, not interaction. That's not an external critique; the paper describes the alignment itself. The listener branch also has no objective validation: Table 2's 'Listener-coherence' is a subjective score with no comparison to real dyadic recordings or a no-interaction control.

Second, the abstract promises a GAN that increases the denoising step size and enables realtime response, but Section 3 has no discriminator or adversarial loss; Equation 11 is just L_simple plus a foot-geometry term. Section 6 admits the Markov-chain DDPM is too slow for realtime. That's a direct internal contradiction.

Third, evaluation details: no error bars on FGD/BA/DIV, and Table 3 lists DSG FGD as 25.7 while Table 1 gives 15.67 for the same method. Could be a typo, but it makes the ablation table hard to trust. No code or data release, so nothing is independently checkable.

Who this is for: researchers working on co-speech gesture generation might find the task framing and the role-condition trick worth a skim. But as a research contribution, the paper doesn't support its claims. I would desk-reject in current form. The data pairing problem alone is disqualifying, and the method/abstract mismatch would need to be fixed before any serious review.

Referee Report

3 major / 6 minor

Summary. This paper proposes Inter-Diffusion, a dual-branch diffusion architecture for jointly generating speaker and listener full-body gestures conditioned on the same speech audio. The speaker branch receives a strongly weighted audio condition (λC), the listener branch a weakly weighted one ((1−λ)C), and the two branches are coupled through cross-local attention and cross-attention modules. The model is trained on a paired dataset built by slerp-interpolating listener motion clips from the TWH dataset to match the length of speech clips from the ZeroEGG dataset. Experiments compare the speaker-side outputs to several baselines using FGD, BA, and DIV, and a user study rates human-likeness, gesture appropriateness, emotion compatibility, and listener coherence. The paper claims, for the first time, to generate listener gestures and to achieve GAN-accelerated realtime generation. However, the method section does not include any GAN component, and the training data are not real co-occurring speaker-listener exchanges.

Significance. If the method were trained and validated on real dyadic interaction data, the idea of modeling speaker and listener jointly with role-scaled conditioning would be a worthwhile contribution to co-speech gesture generation. The paper is also transparent in its limitations section about the speed issue. However, the two most consequential claims—capturing genuine speaker-listener interaction and GAN-accelerated realtime generation—are not supported by the evidence as presented: the training data are fabricated by alignment, the method has no adversarial component, and the evaluation omits objective listener metrics and meaningful baselines. The paper does not release code or data, and the user-study numbers are reported without statistical tests.

major comments (3)

[Section 4.1 (Dataset Design)] The dataset construction in Section 4.1 pairs ZeroEGG speaker audio with TWH listener motion clips by slerp interpolation to match sequence lengths, but the paper provides no evidence that any TWH listener clip is a natural response to the ZeroEGG audio with which it is paired. Because the diffusion training target in Eq. (10) reconstructs these synthetically aligned pairs, the dual-branch denoiser can at best learn statistics of the forced alignment rather than genuine speaker-listener interaction, which is the central contribution claimed in the abstract and Section 1. This is an internal-evidence problem: the paper itself describes the pairing as a length-matching procedure, not as semantically synchronized dyadic data.
[Section 3.3 (Loss Function) and Section 6] No adversarial loss or discriminator appears anywhere in the method: the training objective in Eq. (11) consists only of the simple diffusion loss L_simple (Eq. 10) and the foot-geometry loss L_foot. Yet the abstract claims that 'the GAN model' is introduced to increase the denoising step size and enable realtime response, and the Implementation Details paragraph in Section 4.1 mentions a discriminator learning rate and monitoring of generator and discriminator losses. Section 6 explicitly concedes that the Markov-chain DDPM is too slow for real-time applications. The GAN/realtime claim is therefore internally inconsistent with the method and the authors' own limitation statement.
[Tables 1 and 2] Table 1 reports FGD, BA, and DIV only for the speaker branch, so the listener branch receives no objective validation. In Table 2, the 'Listener-coherence' score for Ours (4.01±0.17) overlaps with Ground Truth (4.12±0.31) and is not accompanied by any statistical significance test; moreover, the two baselines (DSG and DiffGesture) are speaker-only methods, so a listener-coherence comparison does not demonstrate that the proposed model produces better listener gestures than any existing listener-aware alternative (of which none is evaluated). The claims of superiority in listener coherence are therefore not established.

minor comments (6)

[Section 4.1] The phrase 'reverse-connect the listener's actions' is undefined; the description of the slerp alignment only explains length matching, not any semantic synchronization.
[Table 1 and surrounding text] The citation formatting is inconsistent: the text reads 'Tri-modal's [28] 22.4 [16], and HA2G's [15] 18.8', and Table 1 lists 'CAMN [15]' while reference [15] is the BEAT dataset paper; please correct the citation keys.
[Section 4.3] The 'Ground Truth' condition in the user study is not described; it is unclear whether it consists of real TWH listener clips and how those are synchronized with the ZeroEGG test audio.
[Section 3.3] The notation '⊕' is used for both concatenation in Eq. (3) and for conditioning additions in Eqs. (4)–(9); please distinguish the two operations or define the symbols explicitly.
[Abstract and Section 6] The phrase 'increase the denoising step size' is ambiguous; if the intent is fewer denoising steps, please state that, and align the abstract and Section 6 (which says the model is too slow for real-time) to avoid direct contradiction.
[General] The paper does not provide code, data, or a link to a project page, which limits reproducibility of the reported results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the diffusion derivation is standard and externally benchmarked; data-pairing and GAN/realtime concerns are validity issues, not circular reductions.

full rationale

The derivation chain is not circular. The Inter-Diffusion denoiser is a standard MDM/DDPM architecture with a dual-branch CLA/CA design; its loss in Eq. (10) is the usual epsilon-prediction objective plus an MDM foot-geometry regularizer, and the conditioning weights in Eqs. (8)-(9) are explicit hyperparameters rather than fitted outputs renamed as predictions. Speaker-side results are evaluated on held-out ZeroEGG audio against external baselines, so the reported FGD, BA, and DIV values are not forced by construction. The only self-citation of note, reference [6], is used in Section 6 merely as an example that Markov-chain DDPM is effective for high-quality gesture generation; no adversarial loss or discriminator appears in Eq. (11), so the abstract's GAN-acceleration claim is unsupported rather than circularly derived. Similarly, the TWH/ZeroEGG slerp pairing in Section 4.1 raises a legitimate validity question about whether the listener branch learns true interactivity, but this is not a circular reduction: the training target is an externally constructed motion sequence, not a quantity equal to the model's input or to the claimed output by definition. The paper's internal inconsistencies are correctness risks, not evidence that the derivation reduces to its own inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two unvalidated assumptions: (1) that synthetic alignment of listener motion to arbitrary audio yields a valid interaction signal, and (2) that a simple linear scaling of the speaker audio condition is a sufficient interaction condition for the listener. The model has at least three unreported hyperparameters (lambda, alpha, role encoding dimension), and the GAN component promised in the abstract is absent from the method, adding an undisclosed reliance on the authors' prior work.

free parameters (3)

lambda (condition weight) = not reported
Controls the scaling of audio condition for speaker (lambda) vs. listener (1-lambda) in Equations 8-9. The paper never reports its value or how it was chosen, and this scaling is central to the claimed interaction modeling.
alpha (foot geometry loss weight) = not reported
Weight of the MDM foot-geometry loss in Equation 11. Inherited from MDM but the specific value used is not given.
role encoding dimension r = not reported
Dimension of the role encoding appended to latent variables in Equation 3 context; not specified in the paper, and it is an architectural choice that affects the representation.

assumptions (4)

ad hoc to paper slerp interpolation of TWH listener motion to ZeroEGG audio length preserves valid listener-response semantics
Section 4.1: The paper reverse-connects listener actions with slerp to match audio, but there is no evidence that the resynchronized motion is a natural response to the corresponding speech.
ad hoc to paper scaling the audio condition by (1-lambda) for the listener yields context-appropriate listener gestures
Equations 8-9 postulate that a linearly scaled version of the speaker's audio acts as a valid listener condition. No psychophysical or empirical support is provided.
standard math MDM/DDPM and classifier-free guidance provide an adequate generative backbone for two-person gestures
The paper builds on MDM [20] and DDPM [11], which are standard diffusion frameworks; these are not the paper's contribution but are assumed valid.
domain assumption The two datasets (TWH and ZeroEGG) can be combined into one coherent speaker-listener corpus
The paper assumes that isolating listener data from TWH and pairing it with ZeroEGG audio creates a training set representative of dyadic communication; this is not validated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communication." pith.science (2026). https://pith.science/paper/GGKVHGSU

@misc{pith2026250504996,
  author       = {Pith},
  title        = {Pith review of: Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communication},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GGKVHGSU}},
  note         = {Machine review of arXiv:2505.04996}
}

read the original abstract

Full-body gestures play a pivotal role in natural interactions and are crucial for achieving effective communication. Nevertheless, most existing studies primarily focus on the gesture generation of speakers, overlooking the vital role of listeners in the interaction process and failing to fully explore the dynamic interaction between them. This paper innovatively proposes an Inter-Diffusion Generation Model of Speakers and Listeners for Effective Communication. For the first time, we integrate the full-body gestures of listeners into the generation framework. By devising a novel inter-diffusion mechanism, this model can accurately capture the complex interaction patterns between speakers and listeners during communication. In the model construction process, based on the advanced diffusion model architecture, we innovatively introduce interaction conditions and the GAN model to increase the denoising step size. As a result, when generating gesture sequences, the model can not only dynamically generate based on the speaker's speech information but also respond in realtime to the listener's feedback, enabling synergistic interaction between the two. Abundant experimental results demonstrate that compared with the current state-of-the-art gesture generation methods, the model we proposed has achieved remarkable improvements in the naturalness, coherence, and speech-gesture synchronization of the generated gestures. In the subjective evaluation experiments, users highly praised the generated interaction scenarios, believing that they are closer to real life human communication situations. Objective index evaluations also show that our model outperforms the baseline methods in multiple key indicators, providing more powerful support for effective communication.

Figures

Figures reproduced from arXiv: 2505.04996 by the authors.

**Figure 1.** Overview training process of our Inter-Diffusion. language can influence the speaker’s subsequent speech and actions, creating a dynamic interaction loop. Our work is groundbreaking as we introduce the generation of listener’s global gestures for the first time. By incorporating listeners into the model, we can create a more realistic and interactive communication environment, filling a significant gap in the curren… view at source ↗

**Figure 2.** In comparison to contemporary diffusion-based methods such as DSG and FreeTalker, our approach achieves the best result. our generated gestures are more closely aligned with the original motion distribution. Regarding BA, which measures the synchronization between audio and gesture beats, our method attains a value of 0.82. This figure is higher than most of the compared methods, such as DSG’s 0.81, FreeTalker’s 0.… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 25 canonical work pages

[1]

Chaitanya Ahuja, Dong Won Lee, Yukiko I Nakano, and Louis-Philippe Morency. 2020. Style transfer for co-speech gesture animation: A multi- speaker conditional-mixture approach. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16. Springer, 248–265

work page 2020
[2]

Tenglong Ao, Zeyi Zhang, and Libin Liu. 2023. GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents. ACM Trans. Graph. 42, 4 (2023), 42:1–42:18. doi: 10.1145/3592097

doi:10.1145/3592097 2023
[3]

Yongkang Cheng and Shaoli Huang. 2025. HoloGest: Decoupled Diffusion and Motion Priors for Generating Holisticly Expressive Co- speech Gestures. arXiv preprint arXiv:2503.13229 (2025)

arXiv 2025
[4]

Yongkang Cheng, Shaoli Huang, Xuelin Chen, Jifeng Ning, and Ming- ming Gong. 2025. DIDiffGes: Decoupled Semi-Implicit Diffusion Models for Real-time Gesture Generation from Speech. In Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 39. 2464–2472

work page 2025
[5]

Yongkang Cheng, Shaoli Huang, Jifeng Ning, and Ying Shan. 2023. Bopr: Body-aware part regressor for human shape and pose estimation. arXiv preprint arXiv:2303.11675 (2023)

work page Pith review arXiv 2023
[6]

Yongkang Cheng, Mingjiang Liang, Shaoli Huang, Gaoge Han, Jifeng Ning, and Wei Liu. 2025. Conditional gan for enhancing diffusion models in efficient and authentic global gesture generation from audios. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2164–2173

work page 2025
[7]

Yongkang Cheng, Mingjiang Liang, Shaoli Huang, Jifeng Ning, and Wei Liu. 2024. Expgest: Expressive speaker generation using diffusion model and hybrid audio-text guidance. In 2024 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6

work page 2024
[8]

Ikhsanul Habibie, Mohamed Elgharib, Kripasindhu Sarkar, Ahsan Ab- dullah, Simbarashe Nyatsanga, Michael Neff, and Christian Theobalt

work page

Show all 35 references

[9]

Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta, Lingjie Liu, Hans- Peter Seidel, Gerard Pons-Moll, Mohamed Elgharib, and Christian Theobalt. 2021. Learning speech-driven 3d conversational gestures from video. In Proceedings of the 21st ACM International Conference on Intelligent...

2021
[10]

Gaoge Han, Mingjiang Liang, Jinglei Tang, Yongkang Cheng, Wei Liu, and Shaoli Huang. 2025. Reindiffuse: Crafting physically plausible motions with reinforced diffusion model. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2218– 2227

2025
[11]

Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffu- sion probabilistic models. Advances in neural information processing systems 33, 6840–6851

2020
[12]

Jonathan Ho and Tim Salimans. 2022. Classifier-free diffusion guid- ance. arXiv preprint arXiv:2207.12598 (2022)

2022 arXiv
[13]

Ruilong Li, Shan Yang, David A Ross, and Angjoo Kanazawa. 2021. Ai choreographer: Music conditioned 3d dance generation with aist++. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 13401–13412

2021
[14]

Mingjiang Liang, Yongkang Cheng, Hualin Liang, Shaoli Huang, and Wei Liu. 2025. Ropetp: Global human motion recovery via integrat- ing robust pose estimation with diffusion trajectory prior. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2973–2982

2025
[15]

Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, and Bo Zheng. 2022. Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis. In European Conference on Computer Vision. Springer, 612– 630

2022
[16]

Xian Liu, Qianyi Wu, Hang Zhou, Yuanqi Du, Wayne Wu, Dahua Lin, and Ziwei Liu. 2022. Audio-Driven Co-Speech Gesture Video Generation. Advances in Neural Information Processing Systems 35, 21386–21399

2022
[17]

Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. 2022. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems 35 (2022), 5775–5787

2022
[18]

Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)

2020 arXiv
[19]

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Ku- mar, Stefano Ermon, and Ben Poole. 2020. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 (2020)

2020 arXiv
[20]

Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen- Or, and Amit H Bermano. 2022. Human motion diffusion model. arXiv preprint arXiv:2209.14916

2022 arXiv
[21]

Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. 2022. Diffusion-gan: Training gans with diffusion. arXiv preprint arXiv:2206.02262 (2022)

2022 arXiv
[22]

Pan Xie, Qipeng Zhang, Zexian Li, Hao Tang, Yao Du, and Xiaohui Hu
[23]

Sicheng Yang, Zilin Wang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Qiaochu Huang, Lei Hao, Songcen Xu, Xiaofei Wu, Zonghong Dai, et al. 2023. UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons. arXiv preprint arXiv:2309.07051

2023 arXiv
[24]

arXiv preprint arXiv:2208.09141 (2022)

Vector quantized diffusion model with codeunet for text-to-sign pose sequences generation. arXiv preprint arXiv:2208.09141 (2022)

2022 arXiv
[25]

Sicheng Yang, Zunnan Xu, Haiwei Xue, Yongkang Cheng, Shaoli Huang, Mingming Gong, and Zhiyong Wu. 2024. Freetalker: Con- trollable speech and text-driven gesture generation based on diffusion models for enhanced speaker naturalness. In ICASSP 2024-2024 IEEE International Confe...

2024
[26]

Sicheng Yang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Lei Hao, Weihong Bao, Ming Cheng, and Long Xiao. 2023. DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models. arXiv preprint arXiv:2305.04919

2023 arXiv
[27]

Youngwoo Yoon, Bok Cha, Joo-Haeng Lee, Minsu Jang, Jaeyeon Lee, Jaehong Kim, and Geehyuk Lee. 2020. Speech gesture generation from the trimodal context of text, audio, and speaker identity. ACM Transactions on Graphics (TOG) 39, 6, 1–16

2020
[28]

Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, and Michael J Black. 2023. Generating holis- tic 3d human motion from speech. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 469–480

2023
[29]

Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xiny- ing Guo, Lei Yang, and Ziwei Liu. 2022. Motiondiffuse: Text- driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001

2022 arXiv
[30]

Youngwoo Yoon, Pieter Wolfert, Taras Kucherenko, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the 2022 International Confer- ence on Multimo...

2022
[31]

Chi Zhou, Tengyue Bian, and Kang Chen. 2022. Gesturemaster: Graph- based speech-driven gesture generation. In Proceedings of the 2022 International Conference on Multimodal Interaction. 764–770

2022
[32]

Mengyi Zhao, Mengyuan Liu, Bin Ren, Shuling Dai, and Nicu Sebe
[35]

Lingting Zhu, Xian Liu, Xuanyu Liu, Rui Qian, Ziwei Liu, and Lequan Yu. 2023. Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10544–10553

2023
[2022]

In ACM SIGGRAPH 2022 Conference Proceedings

A Motion Matching-based Framework for Controllable Ges- ture Synthesis from Speech. In ACM SIGGRAPH 2022 Conference Proceedings. 1–9

2022
[2023]

arXiv preprint arXiv:2301.03949 (2023)

Modiff: Action-Conditioned 3D Motion Generation with Denois- ing Diffusion Probabilistic Models. arXiv preprint arXiv:2301.03949 (2023)

2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Chaitanya Ahuja, Dong Won Lee, Yukiko I Nakano, and Louis-Philippe Morency. 2020. Style transfer for co-speech gesture animation: A multi- speaker conditional-mixture approach. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16. Springer, 248–265

work page 2020

[2] [2]

Tenglong Ao, Zeyi Zhang, and Libin Liu. 2023. GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents. ACM Trans. Graph. 42, 4 (2023), 42:1–42:18. doi: 10.1145/3592097

doi:10.1145/3592097 2023

[3] [3]

Yongkang Cheng and Shaoli Huang. 2025. HoloGest: Decoupled Diffusion and Motion Priors for Generating Holisticly Expressive Co- speech Gestures. arXiv preprint arXiv:2503.13229 (2025)

arXiv 2025

[4] [4]

Yongkang Cheng, Shaoli Huang, Xuelin Chen, Jifeng Ning, and Ming- ming Gong. 2025. DIDiffGes: Decoupled Semi-Implicit Diffusion Models for Real-time Gesture Generation from Speech. In Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 39. 2464–2472

work page 2025

[5] [5]

Yongkang Cheng, Shaoli Huang, Jifeng Ning, and Ying Shan. 2023. Bopr: Body-aware part regressor for human shape and pose estimation. arXiv preprint arXiv:2303.11675 (2023)

work page Pith review arXiv 2023

[6] [6]

Yongkang Cheng, Mingjiang Liang, Shaoli Huang, Gaoge Han, Jifeng Ning, and Wei Liu. 2025. Conditional gan for enhancing diffusion models in efficient and authentic global gesture generation from audios. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2164–2173

work page 2025

[7] [7]

Yongkang Cheng, Mingjiang Liang, Shaoli Huang, Jifeng Ning, and Wei Liu. 2024. Expgest: Expressive speaker generation using diffusion model and hybrid audio-text guidance. In 2024 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6

work page 2024

[8] [8]

Ikhsanul Habibie, Mohamed Elgharib, Kripasindhu Sarkar, Ahsan Ab- dullah, Simbarashe Nyatsanga, Michael Neff, and Christian Theobalt

work page