Pith. sign in

REVIEW 3 major objections 7 minor 2 cited by

Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human Motion Generation

T0 review · 3 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mogo proposes a single hierarchical causal transformer over a residual-quantized motion tokenizer that reaches HumanML3D FID 0.079, matching or slightly beating masked-model quality while keeping GPT-style streaming output.

desk verdict A plausible single-pass hierarchical causal transformer with a clever RVQ-layer conditioning scheme, undermined by test-set model selection and an overstated OOD claim. read the letter →

arxiv 2412.07797 v1 pith:RXFMN7L3 submitted 2024-12-05 cs.CV

Dongjie Fu This is my paper · ORCID
classification cs.CV
keywords text-to-motiongeneration3Dhumanmotionresidualvectorquantizationhierarchicalcausaltransformerautoregressivezero-shotgeneralizationML3Dstreamingoutput
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a GPT-style autoregressive transformer can match or slightly exceed the generation quality of masked BERT-style models for text-to-3D human motion, without any second refinement model. The proposed system, Mogo, pairs an RVQ-VAE tokenizer with a hierarchical causal transformer: one small autoregressive sub-model per residual quantization layer, all trained together in a single pass. On the HumanML3D benchmark the model reports FID 0.079, below the related BERT-type MMM's 0.080 and well below GPT-type T2M-GPT's 0.116, and it reports the strongest zero-shot results on an out-of-distribution combat-motion set. If the comparisons hold, the result matters because it gives game and multimedia applications the streaming, token-by-token output of autoregressive models without giving up the quality that masked models currently provide.

What carries the argument

The load-bearing mechanism is the cumulative cross-layer input defined in Eq. (3): at quantization layer $v$, the sub-transformer's motion input is the sum of learned token embeddings from all previous RVQ layers, prefixed by text and layer-index embeddings. This single summed sequence lets one autoregressive model produce base and residual tokens in a single pass, with Transformer-XL-style relative positional encoding used so the longer token sequences caused by stride-1 convolution remain tractable. The RVQ-VAE tokenizer uses six quantization layers with an $8192 \times 128$ codebook, and the hierarchical transformer assigns progressively smaller sub-models to the higher residual layers.

What would settle it

Take two motion sequences whose base and residual code histories differ but whose cumulative embeddings at some layer $v$ are nearly identical, for example with cosine similarity above 0.99. If the layer-$v$ sub-transformer produces nearly identical next-token predictions for both, the model is demonstrably blind to the collapsed code identity; equivalently, an ablation that replaces the cumulative sum with concatenated per-layer embeddings and shows a large FID change would confirm the sum is doing real work.

Watch

Extended reading notes

Core claim

Mogo's central claim is that residual vector quantization and a single hierarchical causal transformer suffice for high-quality text-driven 3D motion generation. Whereas MoMask and MMM use a masked transformer plus a separate residual transformer or refinement stage to reconstruct high-fidelity motion, Mogo generates the base motion tokens and all residual-layer tokens with one transformer whose sub-modules are arranged by RVQ layer, each sub-module receiving as its motion input the sum of embeddings from all earlier layers, as defined in Eq. (3). The paper reports that this architecture achieves state-of-the-art FID among GPT-type models on HumanML3D and KIT-ML, outperforms the BERT-type MMM on HumanML3D FID (0.079 vs 0.080), achieves the best zero-shot metrics on the CMP combat-motion set, and produces continuous cyclic sequences up to 260 frames.

Load-bearing premise

The load-bearing premise is that adding together the token embeddings of all earlier quantization layers, as in Eq. (3), gives the next layer's sub-model an unambiguous description of what codes came before; if two different code histories can produce nearly the same summed embedding, the single-pass prediction loses information, and the paper offers no proof or dedicated ablation showing this cannot happen.

Editorial extensions

If this is right

  • GPT-type motion generators can match masked-model quality, so streaming token-by-token output no longer forces a quality penalty in game and multimedia settings.
  • Continuous cyclic motion sequences can be generated beyond the training length ceiling, reaching 260 frames (13 seconds) versus the 196-frame HumanML3D limit, with the user's requested length replacing an END token.
  • Zero-shot out-of-distribution generation, measured on combat-style CMP motions, is improved relative to both GPT-type and BERT-type transformer baselines.
  • Because the single transformer replaces MoMask's separate residual transformer, scaling to larger mixed datasets is simpler, and the paper reports FID improving as more CMP data is added.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cumulative-sum trick is a general pattern: any hierarchical residual quantizer for audio, video, or point clouds could use a single causal transformer over summed layer embeddings, removing the separate refinement model, provided the unambiguity of the sum is established.
  • A decisive experiment would be to ablate Eq. (3) against a concatenated-per-layer input; if FID changes little, the architecture's gain comes from added parameters or the tokenizer rather than from the cumulative condition itself.
  • The stated inference speed of about 0.002 seconds per frame suggests real-time streaming is plausible, but a stress test extending generation to thousands of frames would show whether cross-layer errors accumulate during open-loop decoding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This manuscript introduces Mogo, a text-to-3D human motion generator built from an RVQ-VAE tokenizer and a hierarchical causal Transformer with six sub-transformers that predict base and residual code layers in a single autoregressive pass. The paper reports HumanML3D FID 0.079±0.002, claiming to surpass the BERT-type MMM (0.080±0.004) and all GPT-type baselines, and asserts state-of-the-art zero-shot performance on the CMP dataset, together with the ability to generate cyclic sequences up to 260 frames. The method also includes an LLM-based prompt rewriting framework for open-vocabulary use.

Significance. The architectural idea of folding all RVQ layers into one hierarchical causal model with cumulative cross-layer inputs is a reasonable and interesting direction, and the paper includes a fairly extensive ablation of codebook size, parameter depth, and input conditioning. The reported reconstruction FID (0.016) is strong, and the paper is honest in the limitations section about editing and length constraints. However, the headline claims are not supported as stated: the main HumanML3D comparison is statistically indistinguishable from MMM, the configuration was selected on the test set, and the abstract's out-of-distribution superiority is contradicted by Table 1. The paper would be a useful contribution if the evaluation were corrected and the claims calibrated; as it stands, the central results are not established.

major comments (3)
  1. [Table 1; Tables 3 and 7] The claim that Mogo outperforms MMM on HumanML3D is not supported. In Table 1, Mogo's FID is 0.079±0.002 versus MMM's 0.080±0.004; the difference is far smaller than either 95% confidence interval, so no significant improvement is demonstrated. Moreover, the reported 0.079 was selected by scanning codebook sizes in Table 3 and parameter settings (Parameters A–D) in Table 7, both evaluated on the HumanML3D test split. This is test-set model selection; the headline number is the minimum of a set of test-set FIDs rather than the performance of a configuration selected on validation, which biases the comparison in Mogo's favor. The authors should perform model selection on the validation split and report the test performance of the single chosen configuration, along with a paired significance test against MMM.
  2. [Abstract and Table 1 (CMP rows)] The abstract's statement that 'our model achieves the best quantitative performance in out-of-distribution generation' is contradicted by the zero-shot CMP rows of Table 1: MotionGPT has FID 10.654±0.183, which is better than Mogo's 14.724±0.171, and MMM has better Top-1, Top-2, and Top-3 R-Precision than Mogo. Mogo does achieve the best Top-1/2/3? Wait check: Mogo Top1 0.069, Top2 0.119, Top3 0.166; MMM Top1 0.067, Top2 0.116, Top3 0.154. So Mogo is slightly better than MMM on R-Precision. But MotionGPT FID is better. The claim of 'best quantitative performance' is false because FID is a key metric and MotionGPT beats Mogo. Need to rephrase: The claim as written is contradicted because FID is part of quantitative performance. The authors should either restrict the claim to specific metrics or present the CMP results honestly.
  3. [Eq. (3)] The cumulative cross-layer input defined in Eq. (3) is a load-bearing design choice: each sub-transformer v receives the sum t^{1:n}_v = Σ_{i=0}^v t^{1:n}_i of code embeddings from all previous layers, rather than the individual layer tokens. Two different histories of base and residual codes can in principle produce similar or identical summed embeddings, making the prediction task for layer v ambiguous. The paper provides no analysis, probe, or ablation demonstrating that this representation preserves the necessary code identity (e.g., by checking codebook collision rates or comparing against a model with per-layer token inputs). Since the central architecture claim is that all RVQ layers can be generated in a single pass with this condition, this needs at least an empirical validation.
minor comments (7)
  1. [Section 3.2, Eq. (3)] The notation t^{1:n}_v is used for both the actual token sequence of layer v and the cumulative sum of embeddings from previous layers; please define separate symbols to avoid ambiguity.
  2. [Eq. (6)] The loss sums over v=1 to V, but the RVQ-VAE has V+1 quantization layers (0 through V); clarify whether the base layer (v=0) is included in the maximum-likelihood objective and, if not, why.
  3. [Section 4.2] The implementation details state 'with 6 quantization layers' while the method defines V+1 layers; please state the exact value of V and the total number of code layers for reproducibility.
  4. [Section 4.4] The text refers to 'Tab. 6' for reconstruction comparison, but the reconstruction table is labeled Table 2 in the main text; correct the cross-reference.
  5. [Table 4 and Section 4.6] The table titled 'Impact of dataset size' actually shows FID at different training epochs for different CMP mixtures; clarify in the caption and text that the reported values are on the HumanML3D test set and how the mixing is performed.
  6. [Figure 2 caption] The caption contains 'a Rq Hierarchical Causal Transformer' and inconsistent spacing in 'VQ-V AE'; please copyedit the figure captions.
  7. [Abstract and Section 5] The abstract states the model surpasses the 196-frame limit, but Section 5 limits this to 'continuous, cyclic motion sequences'; the abstract should state this qualification.

Circularity Check

1 steps flagged · score 4.0 of 10

Reported 0.079 FID is the best of a test-set hyperparameter search, so the headline comparison with MMM is not an unbiased prediction.

  1. fitted input called prediction [Section 4.6 (Ablation Study), Tables 3 and 7; headline result in Abstract and Table 1]
    "We used the FID of the generated results as the core reference metric, ultimately selecting a codebook size of 8192 × 128. ... Parameter D represents heads: [16, 12, 6, 2, 2, 2], layers: [18, 16, 8, 4, 2, 2]. ... Parameter D 0.079±0.002 ... Bold face indicates the best result."

    The headline FID 0.079 is not an out-of-sample prediction: Tables 3 and 7 select the codebook size and transformer layer/head counts by taking the configuration with the lowest generation FID measured on the HumanML3D test set. The abstract then reports that same selected-best number as Mogo's result and compares it with MMM's 0.080. Because the test split was used as the model-selection criterion, the reported value is optimistically biased and is the minimum of the searched configurations rather than an independent estimate; the claimed superiority over MMM is therefore forced by the selection procedure, not demonstrated by a fair comparison.

full rationale

The derivation of the Mogo architecture is self-contained: RVQ-VAE residual coding (Eqs. 1-2), the hierarchical causal transformer loss (Eq. 6), and relative positional attention (Eqs. 4-5) are standard components assembled without importing a uniqueness theorem or ansatz from the authors' prior work, and the paper contains no load-bearing self-citations. The one genuinely self-referential element is the headline benchmark: the codebook size and layer/head configuration were chosen by minimizing FID on the HumanML3D test split (Tables 3 and 7), and the same selected value is reported as Mogo's Table 1 FID. This makes the '0.079 beats 0.080' claim the best of a test-set search rather than an unbiased prediction. The lack of statistical significance (the 95% confidence intervals overlap) and the inconsistency between the abstract's OOD claim and Table 1's CMP FID (MotionGPT 10.654 vs Mogo 14.724) are correctness concerns, not circularity. The cumulative-sum input concern in Eq. (3) is an unproven architectural assumption, not a circular step.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical architecture study, not a derivation, so the ledger records the architecture hyperparameters chosen by hand or by test-set FID, plus the key modeling assumptions the pipeline depends on. No new physical or conceptual entities are introduced.

free parameters (6)
  • Codebook size = 8192 x 128
    Selected by comparing generation FID on the HumanML3D test set (Table 3); the reported result uses this best configuration.
  • Number of RVQ-VAE layers = 6
    Chosen after reconstruction-FID comparisons in Appendix Table 6; more layers improved reconstruction from 0.070 (1 layer) to 0.016 (6 layers).
  • Transformer stacks per quantization layer = heads [16,12,6,2,2,2], layers [18,16,8,4,2,2]
    Parameter D was selected as best on HumanML3D test FID (Table 7), so the headline FID is the result of test-set model selection.
  • Masking ratio tau = 0.5
    Data augmentation replaces tau x 100% of real code indices with random ones; set to 0.5 in Sec. 3.2.
  • Commitment loss weight beta = not reported
    Defined in Eq. (2) but no value is given anywhere in the paper; it affects RVQ-VAE training and therefore generation quality.
  • RVQ-VAE convolution stride = 1
    Changed from stride 2 used in prior works [16,43] to stride 1; this quadruples token sequence length and is a design choice central to the long-sequence claims.
assumptions (4)
  • domain assumption The cumulative sum of code embeddings from all previous RVQ layers is a sufficient and unambiguous condition for predicting the current layer's tokens.
    Eq. (3) in Sec. 3.2 defines s_v using this sum; the paper gives no proof or ablation showing that summed vectors preserve enough information to distinguish different code-token histories.
  • domain assumption Residual quantization with 6 layers and an 8192-entry codebook captures motion detail well enough that generation quality is limited by the transformer, not the tokenizer.
    Table 2 reports low reconstruction FID, but reconstruction FID is computed on the same benchmark used for generation and does not by itself guarantee that the base-layer code is informative enough for autoregressive generation.
  • domain assumption Relative positional encoding from Transformer-XL generalizes beyond the 196-frame training horizon for cyclic motions.
    Sec. 3.2 and Sec. 5; the paper admits non-continuous motions cannot exceed 196 frames, so the long-sequence claim is only asserted for cyclic cases and is not quantitatively evaluated.
  • domain assumption The FID and R-Precision evaluation protocol is comparable across methods that use different tokenizers, output lengths, and text encoders.
    Sec. 4.1; the paper focuses on FID without controlling for sequence length or reporting per-capability metrics, and the same test set is used for model selection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human Motion Generation." pith.science (2026). https://pith.science/paper/RXFMN7L3

@misc{pith2026241207797,
  author       = {Pith},
  title        = {Pith review of: Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human Motion Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RXFMN7L3}},
  note         = {Machine review of arXiv:2412.07797}
}
read the original abstract

In the field of text-to-motion generation, Bert-type Masked Models (MoMask, MMM) currently produce higher-quality outputs compared to GPT-type autoregressive models (T2M-GPT). However, these Bert-type models often lack the streaming output capability required for applications in video game and multimedia environments, a feature inherent to GPT-type models. Additionally, they demonstrate weaker performance in out-of-distribution generation. To surpass the quality of BERT-type models while leveraging a GPT-type structure, without adding extra refinement models that complicate scaling data, we propose a novel architecture, Mogo (Motion Only Generate Once), which generates high-quality lifelike 3D human motions by training a single transformer model. Mogo consists of only two main components: 1) RVQ-VAE, a hierarchical residual vector quantization variational autoencoder, which discretizes continuous motion sequences with high precision; 2) Hierarchical Causal Transformer, responsible for generating the base motion sequences in an autoregressive manner while simultaneously inferring residuals across different layers. Experimental results demonstrate that Mogo can generate continuous and cyclic motion sequences up to 260 frames (13 seconds), surpassing the 196 frames (10 seconds) length limitation of existing datasets like HumanML3D. On the HumanML3D test set, Mogo achieves a FID score of 0.079, outperforming both the GPT-type model T2M-GPT (FID = 0.116), AttT2M (FID = 0.112) and the BERT-type model MMM (FID = 0.080). Furthermore, our model achieves the best quantitative performance in out-of-distribution generation.

Figures

Figures reproduced from arXiv: 2412.07797 by the authors.

Figure 1
Figure 1. Mogo is a novel GPT-type model for high-quality text-to-motion generation, capable of producing longer sequences and handling [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The architecture of the model:(a) RVQ-VAE is a hierarchical residual vector quantization variational autoencoder that discretizes continuous motion sequences with high precision; (b) a Rq Hierarchical Causal Transformer generates base motion sequences autore￾gressively while inferring residuals across layers. to reveal greater potential. Experiments by [43] confirm that GPT-type models continue to benefit from scali… view at source ↗
Figure 3
Figure 3. The architecture of prompt engineering for enhanced model inference [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparison of the generation quality between our model and the latest SOTA motion generation models[ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: HumanML3D User Study alignment and Generation quality. For Prompt align￾ment, full motion alignment with the prompt scores 5 points; partial alignment results in a deduction of 1 to 3 points based on motion proportion and order, and no align￾ment leads to a full 5-poin…
Figure 6
Figure 6. Figure 6: Demonstration of Transformer attention masks with dif [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: illustrate the statistical results of our user study, clas￾sified according to the dimensions of experimental evalua￾tion [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Impact of different codebook sizes on training results in [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Demonstration of generative capabilities for open vocabulary and ultra-long sequences. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation

    cs.CV 2026-07 conditional novelty 7.0 of 10

    Distilling frozen Motion-JEPA features into a compact 32-D latent whose geometry is coupled to the decoder lets a standard non-autoregressive flow-matching DiT reach state-of-the-art text-to-motion quality on HumanML3...

  2. Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregression

    cs.CV 2024-11 conditional novelty 7.0 of 10

    A masked-autoregressive diffusion model trained on a compact essential-feature latent space claims state-of-the-art text-to-motion generation under a new essential-dimension evaluation protocol.

Reference graph

Works this paper leans on

47 extracted references · 34 canonical work pages · cited by 2 Pith papers

  1. [1]

    Ahuja and L

    C. Ahuja and L. Morency. Language2pose: Natural language grounded pose forecasting. In 2019 International Confer- ence on 3D Vision (3DV), pages 719–728, Los Alamitos, CA, USA, 2019. IEEE Computer Society. 1, 2

    work page 2019

  2. [2]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz...

    2020

  3. [3]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Ma teusz L...

    arXiv 2005

  4. [4]

    Deep video generation, prediction and completion of human action sequences

    Haoye Cai, Chunyan Bai, Yu-Wing Tai, and Chi-Keung Tang. Deep video generation, prediction and completion of human action sequences. In Proceedings of the European Conference on Computer Vision (ECCV), 2018. 2

    work page 2018

  5. [5]

    Murphy, William T

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jos´e Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin P. Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative transformers. 2023. 2

    work page 2023

  6. [6]

    Chen, Chengkuan Chen, Yicong Li, Tiffany Y

    Richard J. Chen, Chengkuan Chen, Yicong Li, Tiffany Y . Chen, Andrew D. Trister, Rahul G. Krishnan, and Faisal Mahmood. Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16144–16155, 2022. 3

    work page 2022

  7. [7]

    Executing your commands via motion diffusion in latent space

    Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion in latent space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18000–18010, 2023. 2

    work page 2023

  8. [8]

    Transformer-XL: At- tentive language models beyond a fixed-length context

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov. Transformer-XL: At- tentive language models beyond a fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2978–2988, Florence, Italy, 2019. Association for Computational Linguistics. 4, 5, 8

    work page 2019

Show all 47 references
  1. [9]

    Cogview2: Faster and better text-to-image generation via hi- erarchical transformers

    Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hi- erarchical transformers. arXiv preprint arXiv:2204.14217 ,

    arXiv

  2. [10]

    Synthesis of compositional animations from textual descriptions

    Anindita Ghosh, Noshaba Cheema, Cennet Oguz, Christian Theobalt, and Philipp Slusallek. Synthesis of compositional animations from textual descriptions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1396–1406, 2021. 2

    2021

  3. [11]

    Chatglm: A family of large language mod- els from glm-130b to glm-4 all tools, 2024

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie H...

    2024

  4. [12]

    Ac- tion2motion: Conditioned generation of 3d human motions

    Chuan Guo, Xinxin Zuo, Sen Wang, Shihao Zou, Qingyao Sun, Annan Deng, Minglun Gong, and Li Cheng. Ac- tion2motion: Conditioned generation of 3d human motions. Proceedings of the 28th ACM International Conference on Multimedia, 2020. 5

    2020

  5. [13]

    Generating diverse and natural 3d human motions from text

    Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5152–5161, 2022. 2, 3, 5, 6, 7, 8

    2022

  6. [14]

    Tm2t: Stochastic and tokenized modeling for the reciprocal gener- ation of 3d human motions and texts

    Chuan Guo, Xinxin Zuo, Sen Wang, and Li Cheng. Tm2t: Stochastic and tokenized modeling for the reciprocal gener- ation of 3d human motions and texts. In ECCV, 2022. 3, 7

    2022

  7. [15]

    Action2video: Gener- ating videos of human 3d actions

    Chuan Guo, Xinxin Zuo, Sen Wang, Xinshuang Liu, Shihao Zou, Minglun Gong, and Li Cheng. Action2video: Gener- ating videos of human 3d actions. International Journal of Computer Vision, 130(2):285–315, 2022. 2

    2022

  8. [16]

    Momask: Generative masked mod- eling of 3d human motions

    Chuan Guo, Yuxuan Mu, Muhammad Gohar Javed, Sen Wang, and Li Cheng. Momask: Generative masked mod- eling of 3d human motions. 2023. 1, 2, 3, 4, 6, 7, 8

    2023

  9. [17]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. In Proceedings of the 31st International Conference on Neural Information Processing Systems , page 6...

    2017

  10. [18]

    Motiongpt: Human motion as a foreign language

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. Motiongpt: Human motion as a foreign language. Ad- vances in Neural Information Processing Systems, 36, 2024. 2, 3, 6, 7

    2024

  11. [19]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020. 2

    2020

  12. [20]

    Flame: Free- 9 form language-based motion synthesis & editing

    Jihoon Kim, Jiseob Kim, and Sungjoon Choi. Flame: Free- 9 form language-based motion synthesis & editing. arXiv preprint arXiv:2209.00349, 2022. 2

    2022 arXiv

  13. [21]

    Priority-centric human motion genera- tion in discrete latent space, 2023

    Hanyang Kong, Kehong Gong, Dongze Lian, Michael Bi Mi, and Xinchao Wang. Priority-centric human motion genera- tion in discrete latent space, 2023. 2, 7

    2023

  14. [22]

    Omg: Towards open-vocabulary motion generation via mix- ture of controllers

    Han Liang, Jiacheng Bao, Ruichi Zhang, Sihan Ren, Yuecheng Xu, Sibei Yang, Xin Chen, Jingyi Yu, and Lan Xu. Omg: Towards open-vocabulary motion generation via mix- ture of controllers. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, page...

    2024

  15. [23]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 3

    2021

  16. [24]

    SGDR: Stochastic gradi- ent descent with warm restarts

    Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradi- ent descent with warm restarts. In International Conference on Learning Representations, 2017. 6

    2017

  17. [25]

    Troje, Ger- ard Pons-Moll, and Michael J

    Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Ger- ard Pons-Moll, and Michael J. Black. Amass: Archive of motion capture as surface shapes. 2019 IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 5441– 5450, 2019. 5

    2019

  18. [26]

    Hierarchical transformers are more efficient language models

    Piotr Nawrot, Szymon Tworkowski, Michał Tyrolski, Lukasz Kaiser, Yuhuai Wu, Christian Szegedy, and Henryk Michalewski. Hierarchical transformers are more efficient language models. In Findings of the Association for Compu- tational Linguistics: NAACL 2022 , pages 1559–1571, Se...

    2022

  19. [27]

    Pappagari, Piotr ˙Zelasko, Jes´us Villalba, Yishay Carmiel, and Najim Dehak

    R. Pappagari, Piotr ˙Zelasko, Jes´us Villalba, Yishay Carmiel, and Najim Dehak. Hierarchical transformers for long docu- ment classification. 2019 IEEE Automatic Speech Recogni- tion and Understanding Workshop (ASRU), pages 838–844,

    2019

  20. [28]

    Black, and G ¨ul Varol

    Mathis Petrovich, Michael J. Black, and G ¨ul Varol. Action- conditioned 3D human motion synthesis with transformer V AE. In International Conference on Computer Vision (ICCV), 2021. 2

    2021

  21. [29]

    Mmm: Generative masked motion model, 2024

    Ekkasit Pinyoanuntapong, Pu Wang, Minwoo Lee, and Chen Chen. Mmm: Generative masked motion model, 2024. 1, 2, 6, 7, 8

    2024

  22. [30]

    The KIT motion-language dataset

    Matthias Plappert, Christian Mandery, and Tamim Asfour. The KIT motion-language dataset. Big Data, 4(4):236–252,

  23. [31]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Con...

    2021

  24. [32]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learn- ing Research, 21(140):1–67, 2020. 3

    2020

  25. [33]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 2

    2021

  26. [34]

    Human motion diffu- sion model

    Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-or, and Amit Haim Bermano. Human motion diffu- sion model. In The Eleventh International Conference on Learning Representations, 2023. 2

    2023

  27. [35]

    Neural discrete representation learning

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems , page 6309–6318, Red Hook, NY , USA, 2017. Curran Associates Inc. 4

    2017

  28. [36]

    Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N

    Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural Infor- mation Processing Systems, 2017. 1, 2

    2017

  29. [37]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, Yingli Zhao, Yulong Ao, Xuebin Min, Tao Li, Boya Wu, Bo Zhao, Bowen Zhang, Lian zi Wang, Guang Liu, Zheqi He, Xi Yang, Jingjing Liu, Yonghua Lin, Tieju...

    2024

  30. [38]

    Yin Wang, Zhiying Leng, Frederick W. B. Li, Shun-Cheng Wu, and Xiaohui Liang. Fg-t2m: Fine-grained text-driven human motion generation via diffusion model, 2023. 6

    2023

  31. [39]

    Learning diverse stochastic human-action generators by learning smooth la- tent transitions, 2019

    Zhenyi Wang, Ping Yu, Yang Zhao, Ruiyi Zhang, Yufan Zhou, Junsong Yuan, and Changyou Chen. Learning diverse stochastic human-action generators by learning smooth la- tent transitions, 2019. 1, 2

    2019

  32. [40]

    Peng Xu, Xiatian Zhu, and David A. Clifton. Multimodal learning with transformers: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45:12113– 12132, 2022. 2

    2022

  33. [41]

    Prompt engineering a prompt engineer

    Qinyuan Ye, Mohamed Ahmed, Reid Pryzant, and Fereshte Khani. Prompt engineering a prompt engineer. In Find- ings of the Association for Computational Linguistics ACL 2024, pages 355–385, Bangkok, Thailand and virtual meet- ing, 2024. Association for Computational Linguistics. 5

    2024

  34. [42]

    Animationgpt:an aigc tool for generating game combat mo- tion assets

    Ziming Cheng Yihao Liao, Yiyu Fu and Jiangfeiyang Wang. Animationgpt:an aigc tool for generating game combat mo- tion assets. https : / / github . com / fyyakaxyy / AnimationGPT, 2024. 2, 5, 7, 8

    2024

  35. [43]

    T2m-gpt: Generating human motion from textual de- scriptions with discrete representations

    Jianrong Zhang, Yangsong Zhang, Xiaodong Cun, Shaoli Huang, Yong Zhang, Hongwei Zhao, Hongtao Lu, and Xi Shen. T2m-gpt: Generating human motion from textual de- scriptions with discrete representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2023

  36. [44]

    Motiondif- fuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. arXiv preprint arXiv:2208.15001, 2022. 2, 6

    2022 arXiv

  37. [45]

    Remodiffuse: Retrieval-augmented motion diffusion model

    Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, and Ziwei Liu. Remodiffuse: Retrieval-augmented motion diffusion model. arXiv preprint arXiv:2304.01116, 2023. 1, 2 10

    2023 arXiv

  38. [46]

    Attt2m: Text-driven human motion generation with multi- perspective attention mechanism

    Chongyang Zhong, Lei Hu, Zihao Zhang, and Shihong Xia. Attt2m: Text-driven human motion generation with multi- perspective attention mechanism. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 509–519, 2023. 1, 2, 3, 6

    2023

  39. [47]

    walk like a monkey

    Wentao Zhu, Xiaoxuan Ma, Dongwoo Ro, Hai Ci, Jinlu Zhang, Jiaxin Shi, Feng Gao, Qi Tian, and Yizhou Wang. Human motion generation: A survey, 2023. 2 11 Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human Motion Generation Supplementary Material A. Overview In th...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.