REVIEW 5 cited by
Motion Inversion for Video Customization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this work, we present a novel approach for motion customization in video generation, addressing the widespread gap in the exploration of motion representation within video generative models. Recognizing the unique challenges posed by the spatiotemporal nature of video, our method introduces Motion Embeddings, a set of explicit, temporally coherent embeddings derived from a given video. These embeddings are designed to integrate seamlessly with the temporal transformer modules of video diffusion models, modulating self-attention computations across frames without compromising spatial integrity. Our approach provides a compact and efficient solution to motion representation, utilizing two types of embeddings: a Motion Query-Key Embedding to modulate the temporal attention map and a Motion Value Embedding to modulate the attention values. Additionally, we introduce an inference strategy that excludes spatial dimensions from the Motion Query-Key Embedding and applies a differential operation to the Motion Value Embedding, both designed to debias appearance and ensure the embeddings focus solely on motion. Our contributions include the introduction of a tailored motion embedding for customization tasks and a demonstration of the practical advantages and effectiveness of our method through extensive experiments.
Forward citations
Cited by 5 Pith papers
-
Precise Action-to-Video Generation Through Visual Action Prompts
Skeleton-based visual action prompts give precise, cross-domain action control for video generation of human and robot interactions.
-
MotionShot: Adaptive Motion Transfer across Arbitrary Objects for Text-to-Video Generation
MotionShot transfers motion from a reference video to an unseen target object in text-to-video generation by combining semantic and morphological alignment in a training-free pipeline.
-
When Distillation Breaks Motion Control: Restoring Generative Trajectories for Fast Video Generators
MotionEcho adaptively re-injects teacher-model guidance into few-step distilled video generators so reference motion can be copied at test time without training.
-
LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer
LMP transfers motion from a reference video to newly generated videos in text-to-video and image-to-video settings without training, using attention maps in a frozen diffusion transformer.
-
Follow-Your-Creation: Empowering 4D Creation through Video Inpainting
Follow-Your-Creation fine-tunes the Wan2.1 video inpainting model on composite point-cloud and editing masks so a single monocular video can be converted into editable 4D video with new camera motion.
Discussion (0). Continue with ORCID to comment.