Pith. sign in

REVIEW 3 major objections 6 minor 31 references

M3G: Multi-Granular Gesture Generator for Audio-Driven Full-Body Human Motion Synthesis

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read M3G claims that audio-driven gesture models improve when motion is tokenized at multiple temporal scales instead of one frame at a time.

desk verdict A genuine architectural contribution with clean ablations, but the undisclosed body-hint inference protocol makes the headline comparison untrustworthy until clarified. read the letter →

arxiv 2505.08293 v2 pith:EO6SR6QN submitted 2025-05-13 cs.GR cs.AIcs.CVcs.SDeess.AS

Zhizhuo Yin , Yuk Hang Tsui , Pan Hui This is my paper · ORCID
classification cs.GRcs.AIcs.CVcs.SDeess.AS
keywords audio-drivengesturegenerationfull-bodyhumanmotionsynthesisVQ-VAEmulti-granulartokenizationdiscretelatentspaceco-speechBEAT2temporalcross-attention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the reason audio-driven gesture models plateau is that they tokenize motion one frame at a time, even though a complete expressive gesture can last anywhere from a few frames to a longer phrase. It proposes M3G, in which a Multi-Granular VQ-VAE encodes each motion into several token streams at different temporal resolutions that share one codebook, and a token predictor reconstructs those streams from audio, text, and speaker identity. On BEAT2, M3G reports the lowest FGD among compared methods (4.784 versus EMAGE's 5.643), lower facial error, and higher diversity, and its reconstruction study finds MGVQ-VAE beats a vanilla VQ-VAE (FGD 1.497 versus 3.302). The intended significance is that granularity is a property of gesture itself, so gesture tokens should be multi-scale rather than frame-wise.

What carries the argument

The load-bearing object is the Multi-Granular VQ-VAE (MGVQ-VAE): a VQ-VAE whose encoder is $n$ parallel TCN streams with downsampling factors $2^i$, all mapping into one shared codebook, with the decoder summing Transpose-TCN reconstructions from every granularity. It turns 'grain size' into a first-class modeling choice instead of a fixed frame rate. The second mechanism is the multi-granular token predictor, which generates the token streams from audio and text by combining mean-pooled and TCN-derived latents per body part with temporal cross-attention and quantizing coarse-to-fine. Together these allow motion patterns with different durations to be encoded, predicted, and summed back into a full gesture.

What would settle it

Run the trained M3G with zero vectors in place of the first eight ground-truth body frames at inference, then recompute FGD on BEAT2; if the score rises to or above EMAGE's 5.643, the reported advantage comes from the body-hint prefix rather than from multi-granularity.

Watch

Extended reading notes

Core claim

The central discovery is that a single gesture is better represented by several token streams at different temporal scales than by one token per frame. MGVQ-VAE replaces the frame-wise encoder with $n$ temporal convolutional encoders that downsample the motion by factors $2^i$, quantizes all resulting embeddings against one shared codebook, and reconstructs by summing $n$ transposed-TCN decoder outputs; the scale normalization in Eq. (6) keeps tokens from different granularities in one semantic space. The multi-granular token predictor fuses rhythmic audio features with text embeddings, separates face and body, builds two multi-granular latent views per body part (mean-pooled and TCN-derived), correlates them with temporal cross-attention, and predicts tokens coarse-to-fine. The paper reports that this pipeline outperforms compared methods on BEAT2, with FGD 4.784 versus 5.643 for EMAGE, better facial MSE and LVD, and that MGVQ-VAE reconstruction preserves motion far better than a single-granularity VQ-VAE.

Load-bearing premise

The token predictor is trained with the first eight ground-truth body frames as a body hint, and the paper does not state whether those frames are also fed at inference; if they are, the comparison against baselines that generate from scratch is not apples-to-apples, and if they are not, there is an unstudied training/inference mismatch.

Editorial extensions

If this is right

  • Gesture tokenization for co-speech animation should treat temporal granularity as a variable rather than a fixed frame rate, if the reported BEAT2 gains hold.
  • A multi-granular VQ-VAE can serve as a standalone motion autoencoder, reconstructing BEAT2 motion with FGD 1.497 versus 3.302 for frame-wise VQ-VAE, which suggests richer discrete motion priors.
  • The four-granularity configuration with sizes 1, 2, 4, and 8 is the reported sweet spot; adding more granularities degrades results, so scale count and sizing are important design choices.
  • Facial expression, hands, upper body, lower body, and global translation can share one multi-scale token vocabulary, which is what makes holistic full-body generation tractable in this design.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is predicting the granularity itself per gesture or per body part, rather than fixing it to powers of two, which could adapt to fast versus slow speakers.
  • The reported comparison may depend on the 'body hint' protocol: the token predictor is trained with the first eight ground-truth frames, and the paper does not state whether those frames are also provided at inference; a fair head-to-head should run both settings.
  • If multi-granular tokens are a better motion language, they should also benefit motion editing, interpolation, and retrieval, because one codebook spans short beats and longer phrase-level movements; that is my inference, not a claim tested here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript introduces M3G, a two-stage framework for audio-driven full-body gesture synthesis. In the first stage, a Multi-Granular VQ-VAE (MGVQ-VAE) encodes motion sequences into token series at n temporal granularities using TCN encoders and a shared codebook, and reconstructs the motion by summing upsampled TransTCN outputs from all granularities. In the second stage, a multi-granular token predictor fuses audio and text features (with a body-hint input) to predict the token series, which are decoded by the fixed MGVQ-VAE. Experiments on BEAT2 compare M3G with S2G, Trimodal, HA2G, DisCo, CaMN, DiffStyleGesture, Habibie et al., TalkSHOW, and EMAGE; M3G reports the best FGD, MSE, LVD, and Diversity among full-body methods, while its Beat Align is lower than some baselines. A perceptual A/B study finds participants prefer M3G over CaMN and EMAGE. The paper also ablates the number of granularities and various components.

Significance. If the reported results hold under a fair protocol, the paper would make a solid contribution: the core idea that gestures span multiple temporal granularities and that a shared codebook plus multi-granular tokenizer can improve reconstruction and generation is clearly stated, well-motivated, and supported by a substantial set of ablations. The authors also report statistical significance across five runs, which is commendable. The main technical uncertainties—the undisclosed inference protocol for the body-hint conditioning and the uncontrolled token budget in the reconstruction comparison—must be resolved before the central claims can be accepted.

major comments (3)
  1. [Section 3.4, Eq. (9)] The body-hint input is not specified at inference. The text defines \bar g as a masked gesture sequence where the first 8 frames are ground truth and the remaining frames are zero vectors, and uses h_hints as a conditioning signal. The paper never states what is used for \bar g at inference time, where ground-truth frames are unavailable. If the same masking is used at inference, M3G is not a from-scratch generator and Table 1's comparison with EMAGE and other baselines is not under an equivalent protocol; if the masking is not used, there is a train/inference mismatch whose effect on FGD, Diversity, and the perceptual results is unquantified. The authors must report the inference protocol explicitly, and either remove the ground-truth conditioning or re-evaluate the baselines under the same conditioning; alternatively, they should ablate the body-hint at training to measure the mismatch.
  2. [Section 4.6, Table 4] The reconstruction comparison between MGVQ-VAE and vanilla VQ-VAE is confounded by token count. MGVQ-VAE uses n token series with lengths T, T/2, ..., T/2^(n-1), so it emits roughly 2T tokens per sequence when n=4, whereas vanilla VQ-VAE emits T tokens. The observed improvements in JRMSE and FGD could reflect this increased token budget rather than a superior multi-granular representation. To support the claim that MGVQ-VAE surpasses VQ-VAE in tokenization quality, the authors should compare against a VQ-VAE with a matched total number of tokens (e.g., residual VQ with multiple levels) or report rate-distortion curves.
  3. [Section 4.5, Table 2] The number of granularities is selected on the test set without mention of a held-out validation split. The text says the experiment adjusts the number of granularities to find the best one, and then the best configuration (4 granularities) is used in all subsequent comparisons. This selection on the test set can overstate the benefits of the chosen hyperparameter. Please state whether the selection was made on a validation split or, if not, treat the reported numbers as partially selected.
minor comments (6)
  1. [Section 1] The phrase "corase-to-fine-grained" contains a spelling error and should be "coarse-to-fine-grained".
  2. [Section 4.5] The description of the "w/o TransTCN" variant references "equation (4)" and "equation (2)", but TransTCN appears in Eq. (5) and the encoder TCN in Eq. (2); the cross-references are incorrect.
  3. [Section 4.4, Table 1] The p-value row does not state which statistical test was used; please specify the test (e.g., paired t-test or Wilcoxon) and the null hypothesis for each column.
  4. [Section 5, Figure 3] The text contains "Ground Turth" in the figure caption and "EMage" with inconsistent capitalization; these typos should be corrected.
  5. [Section 3.3 and Section 4.6] The model name is inconsistent: the paper uses "MGVQ-VAE" in most places but "MG-VQV AE" in Section 4.6; please unify the notation.
  6. [Section 3.3, Eq. (6)] The normalization by kernel size in Eq. (6) is introduced without explanation of why kernel size affects scale; a brief justification or reference would help readers understand the motivation.

Circularity Check

1 steps flagged · score 4.0 of 10

Target-prefix conditioning in Eq. (9) makes the headline FGD comparison partly target-assisted; the rest of the pipeline is standard two-stage VQ generation and is not otherwise circular.

  1. other [Section 3.4 (Multi-Granular Token Predictor), Eq. (9); end-to-end evaluation in Section 4.4, Table 1]
    "Body hints h_hints∈ R^{T×h} are encoded from the masked gesture sequence ¯g, where the first 8 frames are set to be the ground truth and the left frames are set to be 0 vectors. ... h_body = h_hints + TCAT(h_hints ⊕ p_t, f_body)"

    By construction, Eq. (9) injects a latent encoding of the ground-truth first 8 frames into the body-predictor state, and the paper gives no inference-time procedure that replaces these ground-truth frames with generated or zero frames. Figure 2 also shows 'Body Hint' as a standing input to the M3G pipeline. As written, the model's prediction is therefore not a pure function of audio: the target prefix is available to the predictor, so the early portion of the 'generated' motion is target-conditioned rather than audio-predicted. Since Table 1's headline FGD is computed over full motion sequences and most listed baselines generate from scratch, part of the reported FGD gap can be forced by this target leakage.

full rationale

The central derivation is conventional: train MGVQ-VAE on ground-truth motion, then train a token predictor to output those discrete tokens from audio, then decode the tokens back to motion. Training the predictor to match MGVQ-VAE's own latents (Eqs. 14 and 17) is not by itself circular, because the final metrics—FGD, Diversity, Beat Align, MSE, and the perceptual study—are computed on decoded motion against ground-truth motion and against external baselines. The perceptual study even shows that M3G is preferred over its own MGVQ-VAE reconstruction only 30.3% of the time, which confirms the predicted tokens are not a trivial copy of the tokenizer's ground-truth tokens. The one real circularity-adjacent issue is the body-hint input in Eq. (9): the predictor is conditioned on a masked gesture whose first 8 frames are ground truth, and no inference-time protocol is stated. If those frames are used at inference, the model is not a from-scratch audio-driven generator, and the Table 1 comparison against from-scratch baselines is partially forced by target leakage. Self-citation is not load-bearing: the only author self-citation (Lee et al. 2021) is background motivation. Score 4 reflects this partial, protocol-level target conditioning rather than a fully circular derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The ledger shows that the core contribution is an architectural stack whose main free choices, the granularity count and the scale schedule, are hand-tuned and selected on the evaluation set. The body-hint conditioning is an unverified input that can materially change comparisons, and the missing codebook and dimension values make the configuration incomplete as reported.

free parameters (4)
  • Number of granularity levels n = 4
    Chosen as the best setting in the granularity-selection experiment (Section 4.5, Table 2); all headline results use this value.
  • Granularity scale schedule = [1, 2, 4, 8] (powers of two)
    Hand-chosen 'to prevent information overlap among different sizes' (Section 4.5); no mechanism measures true gesture durations.
  • Codebook size and latent dimension
    C and d appear in Eqs. (1)-(4) and determine the predictor's token space, but their values are not reported anywhere in the paper.
  • Body-hint window length = 8 frames
    Section 3.4 sets the first 8 frames of the masked gesture sequence to ground truth; this is an arbitrary conditioning choice that can affect generation quality and fair comparison.
assumptions (5)
  • ad hoc to paper A token at granularity i represents a complete expressive gesture spanning 2^i frames.
    Section 3.3 defines granularity as temporal downsampling by powers of two; this operationalizes the motivating idea but is not validated against actual gesture boundaries.
  • ad hoc to paper Summing the upsampled reconstructions from all granularity levels yields a valid motion reconstruction.
    Eq. (5) reconstructs motion by simple summation of TransTCN outputs with no proof or analysis of interference or cancellation across scales.
  • domain assumption The first 8 ground-truth body frames are available as body hints under the same protocol in training and evaluation.
    Eq. (9) uses a masked gesture sequence with the first 8 frames set to ground truth; the paper does not specify whether this conditioning is used at inference.
  • standard math VQ-VAE commitment and stop-gradient training produce a codebook that preserves motion information across all granularities.
    The loss in Eq. (7) follows Van Den Oord et al.; this is a standard assumption, not re-derived here.
  • domain assumption Audio features (onset, amplitude) and FastText transcript features are sufficient, frame-aligned conditioning for gesture generation.
    Section 3.4 fuses these features via attention; no analysis of alignment or information sufficiency is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of M3G: Multi-Granular Gesture Generator for Audio-Driven Full-Body Human Motion Synthesis." pith.science (2026). https://pith.science/paper/EO6SR6QN

@misc{pith2026250508293,
  author       = {Pith},
  title        = {Pith review of: M3G: Multi-Granular Gesture Generator for Audio-Driven Full-Body Human Motion Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EO6SR6QN}},
  note         = {Machine review of arXiv:2505.08293}
}
read the original abstract

Generating full-body human gestures encompassing face, body, hands, and global movements from audio is a valuable yet challenging task in virtual avatar creation. Previous systems focused on tokenizing the human gestures framewisely and predicting the tokens of each frame from the input audio. However, one observation is that the number of frames required for a complete expressive human gesture, defined as granularity, varies among different human gesture patterns. Existing systems fail to model these gesture patterns due to the fixed granularity of their gesture tokens. To solve this problem, we propose a novel framework named Multi-Granular Gesture Generator (M3G) for audio-driven holistic gesture generation. In M3G, we propose a novel Multi-Granular VQ-VAE (MGVQ-VAE) to tokenize motion patterns and reconstruct motion sequences from different temporal granularities. Subsequently, we proposed a multi-granular token predictor that extracts multi-granular information from audio and predicts the corresponding motion tokens. Then M3G reconstructs the human gestures from the predicted tokens using the MGVQ-VAE. Both objective and subjective experiments demonstrate that our proposed M3G framework outperforms the state-of-the-art methods in terms of generating natural and expressive full-body human gestures.

Figures

Figures reproduced from arXiv: 2505.08293 by the authors.

Figure 1
Figure 1. Illustration of gesture sequences with expressive motion patterns in different granularities. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The Overall Workflow of M3G. We illustrate the workflow in two stages: 1. Training MGVQ-VAE to learn discrete latent [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Perceptual study results on motion generation. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative results. think is more natural? [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 24 canonical work pages

  1. [1]

    Language2pose: Natural language grounded pose forecasting

    [Ahuja and Morency, 2019] Chaitanya Ahuja and Louis- Philippe Morency. Language2pose: Natural language grounded pose forecasting. In 2019 International Confer- ence on 3D Vision (3DV), pages 719–728. IEEE,

    work page 2019

  2. [10]

    Acting the part: the role of gesture on avatar identity

    [Feng et al., 2014] Andrew Feng, Gale Lucas, Stacy Marsella, Evan Suma, Chung-Cheng Chiu, Dan Casas, and Ari Shapiro. Acting the part: the role of gesture on avatar identity. In Proceedings of the 7th International Conference on Motion in Games , MIG ’14, page 49–54, New York, NY , USA,

    work page 2014

  3. [12]

    Gen- erative adversarial nets

    [Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. Advances in neural information processing systems, 27,

    2014

  4. [13]

    Learning speech-driven 3d conversational gestures from video

    [Habibie et al., 2021] Ikhsanul Habibie, Weipeng Xu, Dushyant Mehta, Lingjie Liu, Hans-Peter Seidel, Gerard Pons-Moll, Mohamed Elgharib, and Christian Theobalt. Learning speech-driven 3d conversational gestures from video. In Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents , pages 101–108,

    work page 2021

  5. [14]

    Robot behavior toolkit: generating effective social behaviors for robots

    [Huang and Mutlu, 2012] Chien-Ming Huang and Bilge Mutlu. Robot behavior toolkit: generating effective social behaviors for robots. In Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot In- teraction, pages 25–32,

    work page 2012

  6. [17]

    Emage: Towards unified holistic co-speech gesture generation via masked audio gesture modeling

    [Liu et al., 2023] Haiyang Liu, Zihao Zhu, Giorgio Becherini, Yichen Peng, Mingyang Su, You Zhou, Naoya Iwamoto, Bo Zheng, and Michael J Black. Emage: Towards unified holistic co-speech gesture generation via masked audio gesture modeling. arXiv preprint arXiv:2401.00374,

    arXiv 2023

  7. [18]

    Studies on gesture expressivity for a virtual agent

    [Pelachaud, 2009] Catherine Pelachaud. Studies on gesture expressivity for a virtual agent. Speech Communication, 51(7):630–639,

    work page 2009

  8. [20]

    Bailando: 3d dance generation by actor- critic gpt with choreographic memory

    [Siyao et al., 2022] Li Siyao, Weijiang Yu, Tianpei Gu, Chunze Lin, Quan Wang, Chen Qian, Chen Change Loy, and Ziwei Liu. Bailando: 3d dance generation by actor- critic gpt with choreographic memory. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 11050–11059,

    work page 2022

Show all 31 references
  1. [21]

    Body language in the brain: con- structing meaning from expressive movement

    [Tipper et al., 2015] Christine M Tipper, Giulia Signorini, and Scott T Grafton. Body language in the brain: con- structing meaning from expressive movement. Frontiers in human neuroscience, 9:450,

    2015

  2. [24]

    The persona effect: how sub- stantial is it? In People and computers XIII: Proceedings of HCI’98, pages 53–66

    [Van Mulken et al., 1998] Susanne Van Mulken, Elisabeth Andre, and Jochen M ¨uller. The persona effect: how sub- stantial is it? In People and computers XIII: Proceedings of HCI’98, pages 53–66. Springer,

    1998

  3. [26]

    Codetalker: Speech-driven 3d facial animation with dis- crete motion prior

    [Xing et al., 2023] Jinbo Xing, Menghan Xia, Yuechen Zhang, Xiaodong Cun, Jue Wang, and Tien-Tsin Wong. Codetalker: Speech-driven 3d facial animation with dis- crete motion prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12780–12790,

    2023

  4. [27]

    Diffusestylegesture: Stylized audio-driven co-speech gesture generation with diffusion models

    [Yang et al., 2023] Sicheng Yang, Zhiyong Wu, Minglei Li, Zhensong Zhang, Lei Hao, Weihong Bao, Ming Cheng, and Long Xiao. Diffusestylegesture: Stylized audio-driven co-speech gesture generation with diffusion models. arXiv preprint arXiv:2305.04919,

    2023 arXiv

  5. [28]

    Gesture2vec: Clustering gestures us- ing representation learning methods for co-speech gesture generation

    [Yazdian et al., 2022] Payam Jome Yazdian, Mo Chen, and Angelica Lim. Gesture2vec: Clustering gestures us- ing representation learning methods for co-speech gesture generation. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 3100–

    2022

  6. [29]

    Generating holistic 3d human motion from speech

    [Yi et al., 2023] Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, Dacheng Tao, and Michael J Black. Generating holistic 3d human motion from speech. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 469– 480,

    2023

  7. [30]

    Speech gesture generation from the trimodal context of text, audio, and speaker identity.ACM Transac- tions on Graphics (TOG), 39(6):1–16,

    [Yoon et al., 2020] Youngwoo Yoon, Bok Cha, Joo-Haeng Lee, Minsu Jang, Jaeyeon Lee, Jaehong Kim, and Gee- hyuk Lee. Speech gesture generation from the trimodal context of text, audio, and speaker identity.ACM Transac- tions on Graphics (TOG), 39(6):1–16,

    2020

  8. [31]

    Audio-driven neural gesture reenactment with video mo- tion graphs

    [Zhou et al., 2022] Yang Zhou, Jimei Yang, Dingzeyu Li, Jun Saito, Deepali Aneja, and Evangelos Kalogerakis. Audio-driven neural gesture reenactment with video mo- tion graphs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3418– 3...

    2022

  9. [1998]

    Attention is all you need

    [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30,

    2017

  10. [2001]

    Uni- versal transformers

    [Dehghani et al., 2018] Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Uni- versal transformers. arXiv preprint arXiv:1807.03819 ,

    2018 arXiv

  11. [2009]

    Blendface: Re-designing iden- tity encoders for face-swapping

    [Shiohara et al., 2023] Kaede Shiohara, Xingchao Yang, and Takafumi Taketomi. Blendface: Re-designing iden- tity encoders for face-swapping. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 7634–7644,

    2023

  12. [2011]

    Neural discrete representation learning

    [Van Den Oord et al., 2017] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems , 30,

    2017

  13. [2012]

    Auto-encoding variational bayes

    [Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114,

    2013 arXiv

  14. [2013]

    All one needs to know about metaverse: A complete survey on technological singular- ity, virtual ecosystem, and research agenda

    [Lee et al., 2021] Lik-Hang Lee, Tristan Braud, Pengyuan Zhou, Lin Wang, Dianlei Xu, Zijun Lin, Abhishek Kumar, Carlos Bermejo, and Pan Hui. All one needs to know about metaverse: A complete survey on technological singular- ity, virtual ecosystem, and research agenda. arXiv p...

    2021 arXiv

  15. [2014]

    [Ginosar et al., 2019] Shiry Ginosar, Amir Bar, Gefen Ko- havi, Caroline Chan, Andrew Owens, and Jitendra Malik

    Association for Computing Machinery. [Ginosar et al., 2019] Shiry Ginosar, Amir Bar, Gefen Ko- havi, Caroline Chan, Andrew Owens, and Jitendra Malik. Learning individual styles of conversational gesture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2019

  16. [2015]

    Direct iterative closest point for real- time visual odometry

    [Tykk¨al¨a et al., 2011] Tommi Tykk¨al¨a, C ´edric Audras, and Andrew I Comport. Direct iterative closest point for real- time visual odometry. In 2011 IEEE International Confer- ence on Computer Vision Workshops (ICCV Workshops) , pages 2050–2056. IEEE,

    2011

  17. [2017]

    Beat: the behavior expression animation toolkit

    [Cassell et al., 2001] Justine Cassell, Hannes H ¨ogni Vilhj´almsson, and Timothy Bickmore. Beat: the behavior expression animation toolkit. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 477–486,

    2001

  18. [2018]

    Faceformer: Speech- driven 3d facial animation with transformers

    [Fan et al., 2022] Yingruo Fan, Zhaojiang Lin, Jun Saito, Wenping Wang, and Taku Komura. Faceformer: Speech- driven 3d facial animation with transformers. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18770–18780,

    2022

  19. [2019]

    No gestures left be- hind: Learning relationships between spoken language and freeform gestures

    [Ahuja et al., 2020] Chaitanya Ahuja, Dong Won Lee, Ryo Ishii, and Louis-Philippe Morency. No gestures left be- hind: Learning relationships between spoken language and freeform gestures. In Findings of the Association for Com- putational Linguistics: EMNLP 2020 , pages 1884–1895,

    2020

  20. [2020]

    Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchi- cal neural embeddings

    [Ao et al., 2022] Tenglong Ao, Qingzhe Gao, Yuke Lou, Baoquan Chen, and Libin Liu. Rhythmic gesticulator: Rhythm-aware co-speech gesture synthesis with hierarchi- cal neural embeddings. ACM Transactions on Graphics (TOG), 41(6):1–19,

    2022

  21. [2021]

    Enriching word vec- tors with subword information

    [Bojanowski et al., 2017] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vec- tors with subword information. Transactions of the asso- ciation for computational linguistics , 5:135–146,

    2017

  22. [2022]

    Gesturediffuclip: Gesture diffusion model with clip la- tents

    [Ao et al., 2023] Tenglong Ao, Zeyi Zhang, and Libin Liu. Gesturediffuclip: Gesture diffusion model with clip la- tents. ACM Transactions on Graphics (TOG), 42(4):1–18,

    2023

  23. [2023]

    Text2gestures: A transformer- based network for generating emotive body gestures for virtual agents

    [Bhattacharya et al., 2021] Uttaran Bhattacharya, Nicholas Rewkowski, Abhishek Banerjee, Pooja Guhan, Aniket Bera, and Dinesh Manocha. Text2gestures: A transformer- based network for generating emotive body gestures for virtual agents. In 2021 IEEE virtual reality and 3D user ...

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.