REVIEW 6 cited by
MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Motions in a video primarily consist of camera motion, induced by camera movement, and object motion, resulting from object movement. Accurate control of both camera and object motion is essential for video generation. However, existing works either mainly focus on one type of motion or do not clearly distinguish between the two, limiting their control capabilities and diversity. Therefore, this paper presents MotionCtrl, a unified and flexible motion controller for video generation designed to effectively and independently control camera and object motion. The architecture and training strategy of MotionCtrl are carefully devised, taking into account the inherent properties of camera motion, object motion, and imperfect training data. Compared to previous methods, MotionCtrl offers three main advantages: 1) It effectively and independently controls camera motion and object motion, enabling more fine-grained motion control and facilitating flexible and diverse combinations of both types of motion. 2) Its motion conditions are determined by camera poses and trajectories, which are appearance-free and minimally impact the appearance or shape of objects in generated videos. 3) It is a relatively generalizable model that can adapt to a wide array of camera poses and trajectories once trained. Extensive qualitative and quantitative experiments have been conducted to demonstrate the superiority of MotionCtrl over existing methods. Project Page: https://wzhouxiff.github.io/projects/MotionCtrl/
Forward citations
Cited by 6 Pith papers
-
Rays as Pixels: Learning A Joint Distribution of Videos and Camera Trajectories
Encoding cameras as pixel-aligned raxels lets one video diffusion model jointly denoise video and trajectories, supporting pose estimation, controlled generation, and joint synthesis.
-
CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation
Users control a text-to-video model by arranging 3D bounding boxes and camera motion, and the system renders depth maps that steer the diffusion model to generate matching object and camera motion.
-
GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control
GeoDrive conditions a frozen video diffusion model on a 3D-rendered version of the requested ego trajectory, cutting trajectory-following error by 42% versus Vista while using 99.7% less training data.
-
DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation
A diffusion model fuses per-person masks with pose keypoints to generate identity-preserving, two-person interactive videos from a single reference image, outperforming prior single-person-animation pipelines.
-
VIGOR: VIdeo Geometry-Oriented Reward for Temporal Generative Alignment
A VGGT-based pointwise reprojection reward with geometry-aware sampling improves video geometric consistency via SFT/DPO and causal test-time search.
-
Tora2: Motion and Appearance Customized Diffusion Transformer for Multi-Entity Video Generation
Tora2 adds decoupled personalization embeddings, gated self-attention binding, and contrastive learning to Tora, enabling simultaneous appearance and trajectory customization for multiple entities in generated video.
Discussion (0). Continue with ORCID to comment.