REVIEW 32 cited by
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Human motion modeling is important for many modern graphics applications, which typically require professional skills. In order to remove the skill barriers for laymen, recent motion generation methods can directly generate human motions conditioned on natural languages. However, it remains challenging to achieve diverse and fine-grained motion generation with various text inputs. To address this problem, we propose MotionDiffuse, the first diffusion model-based text-driven motion generation framework, which demonstrates several desired properties over existing methods. 1) Probabilistic Mapping. Instead of a deterministic language-motion mapping, MotionDiffuse generates motions through a series of denoising steps in which variations are injected. 2) Realistic Synthesis. MotionDiffuse excels at modeling complicated data distribution and generating vivid motion sequences. 3) Multi-Level Manipulation. MotionDiffuse responds to fine-grained instructions on body parts, and arbitrary-length motion synthesis with time-varied text prompts. Our experiments show MotionDiffuse outperforms existing SoTA methods by convincing margins on text-driven motion generation and action-conditioned motion generation. A qualitative analysis further demonstrates MotionDiffuse's controllability for comprehensive motion generation. Homepage: https://mingyuan-zhang.github.io/projects/MotionDiffuse.html
Forward citations
Cited by 32 Pith papers
-
ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes
ViPS distills a compact, controllable distribution of valid joint configurations for any auto-rigged mesh from video diffusion priors, matching 4D-trained methods in plausibility while generalizing zero-shot to unseen...
-
MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation
Distilling frozen Motion-JEPA features into a compact 32-D latent whose geometry is coupled to the decoder lets a standard non-autoregressive flow-matching DiT reach state-of-the-art text-to-motion quality on HumanML3...
-
DrawMotion: Generating 3D Human Motions by Freehand Drawing
DrawMotion is a diffusion-based framework that fuses text and hand-drawn stickman conditions via a Multi-Condition Module and training-free guidance to generate 3D human motions.
-
ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation
ScaleMoGen introduces a scale-wise autoregressive framework that quantizes motions into hierarchical discrete tokens and predicts next-scale maps to achieve SOTA FID 0.030 on HumanML3D and text-guided editing.
-
ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation
ScaleMoGen applies next-scale autoregressive prediction to human motion generation with multi-scale skeletal-temporal bitwise token maps, reporting SOTA FID on HumanML3D.
-
Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllers
A two-stage framework augments HOI data with dynamic priors and blends pre-trained dynamic motion and static interaction agents via a composer network to enable long-term dynamic human-object interactions with higher ...
-
DanceCrafter: Fine-Grained Text-Driven Controllable Dance Generation via Choreographic Syntax
DanceCrafter generates high-fidelity, text-controlled dance sequences using a new Choreographic Syntax framework and a large fine-grained motion dataset.
-
ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes
ViPS learns a universal, controllable pose space for auto-rigged meshes by transferring motion priors from video diffusion models, matching SOTA performance on plausibility and diversity while enabling zero-shot gener...
-
ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos
ExpertEdit edits novice motions to expert skill levels by learning a motion prior from unpaired videos and infilling masked skill-critical spans.
-
Human Motion Diffusion Model
MDM is a classifier-free diffusion model that generates expressive human motions by predicting clean samples rather than noise, supporting text and action conditioning and outperforming prior methods on standard benchmarks.
-
GIRAF: Towards Generalizable Human Interactions with Articulated Objects
A text-conditioned diffusion model using dynamic object-centric BPS, mixed-domain training, and contact augmentation produces generalizable full-body locomotion-to-articulated-object interaction sequences that beat ad...
-
In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physics
ICMPG combines LLM-based candidate generation with MPC-style physical simulation and semantic scoring to produce text-driven human motions that are both plausible and faithful.
-
Feed-forward Motion In-betweening for Any 4D
Proposes a feed-forward keyframe-conditioned in-betweening method for arbitrary 4D meshes using a topology-agnostic VAE and MMDiT-based rectified flow model.
-
EgoPriMo: Egocentric Motion Generation for Interactive Humanoid Control
EgoPriMo learns a unified egocentric motion prior with a Triple-stream DiT model that supports reconstruction, generation, and forecasting of SMPL motions from egocentric views and text, outperforming prior methods an...
-
Bionic Human-Motion Style Transfer for Physically Executable Whole-Body Control of Humanoid Robots
A multi-condition latent diffusion model transfers human motion styles to diverse humanoid robot contents with physics regularizations, achieving 96% success in real-robot trials on Unitree G1.
-
IAM: Identity-Aware Human Motion and Shape Joint Generation
IAM jointly synthesizes motion sequences and body shape parameters conditioned on multimodal identity signals to achieve more realistic and identity-consistent human motions.
-
HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching
HO-Flow synthesizes realistic hand-object motions from text and canonical 3D objects via an interaction-aware VAE and masked flow matching, reporting SOTA physical plausibility and diversity on GRAB, OakInk, and DexYCB.
-
Language-Guided Transformer Tokenizer for Human Motion Generation
Injecting language into the motion tokenizer yields more compact semantic tokens and state-of-the-art generation scores on HumanML3D and Motion-X.
-
Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation
RAM couples motion reconstruction with text-to-motion diffusion and adds reconstruction-anchored error guidance, reporting FID 0.032 on HumanML3D with 20 inference steps.
-
MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
A semantically aligned latent space plus multi-token cross-attention conditioning sets a new state of the art in text-to-human-motion generation on HumanML3D.
-
A Scalable Whole-body Motion Transfer via Implicit Kinodynamic Motion Retargeting
A neural retargeting pipeline maps human motion to humanoid robot motion at 5000+ frames per second using a shared latent space and physics-based fine-tuning, filtering noise and producing physically feasible trajectories.
-
Being-M0.5: A Real-Time Controllable Vision-Language-Motion Model
Being-M0.5 combines part-aware residual quantization with a 5M-sequence web-video dataset to reach real-time, part-controllable 3D motion generation, though its state-of-the-art claim does not hold on every standard b...
-
Integration Matters: Rollout-Based Training for Constrained Diffusion Models
Rollout-based fine-tuning with a learned adaptive guidance scaling yields near-zero constraint violations while preserving sample fidelity in constrained diffusion models.
-
InterCMDM: Block-Causal Diffusion for Autoregressive Human Interaction Generation
InterCMDM proposes a block-causal latent diffusion framework with dual-stream causal transformers and multi-task attention masks for autoregressive text-conditioned two-person interaction generation and reports SOTA r...
-
Zero-Gated Language-conditioned Human Motion Prediction
ZGL injects frozen CLIP text embeddings of VLM-generated motion captions into a DCT Transformer via zero-gated adapters and reports lower MPJPE than pose-only baselines on Human3.6M with transfer to CMUMocap.
-
GPC: Large-Scale Generative Pretraining for Transferable Motor Control
GPC learns a motion vocabulary via Finite Scalar Quantization and end-to-end RL, then trains an autoregressive transformer for next-token control generation, achieving 99.98% motion reproduction success with emergent ...
-
OMG: Omni-Modal Motion Generation for Generalist Humanoid Control
OMG is a diffusion model for omni-modal whole-body humanoid motion generation that uses language, audio, and reference motions after large-scale data curation to achieve state-of-the-art performance and adaptation.
-
Coordinate-Based Dual-Constrained Autoregressive Motion Generation
CDAMD is a new autoregressive text-to-motion framework operating on continuous motion coordinates with dual constraints and diffusion-inspired components, establishing new benchmarks and claiming SOTA fidelity plus se...
-
E-React: Towards Emotionally Controlled Synthesis of Human Reactions
E-React generates diverse two-person reaction motions conditioned on an actor's movement and a specified emotion using a learned Gaussian emotion prior inside a symmetrical actor-reactor diffusion model.
-
Evaluating Idle Animation Believability: a User Perspective
Users cannot distinguish genuine from acted idle animations but perceive handmade and recorded ones differently; ReActIdle dataset released to simplify future recording.
-
A Spatio-temporal Continuous Network for Stochastic 3D Human Motion Prediction
STCN predicts stochastic 3D human futures by combining a spatio-temporal continuous network with anchor-based Gaussian mixture sampling.
-
Agent AI: Surveying the Horizons of Multimodal Interaction
The paper defines Agent AI as interactive multimodal systems that perceive grounded data and generate embodied actions, arguing this approach can mitigate hallucinations in foundation models.
Discussion (0). Sign in to comment.