REVIEW 8 cited by
DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Animating a still image offers an engaging visual experience. Traditional image animation techniques mainly focus on animating natural scenes with stochastic dynamics (e.g. clouds and fluid) or domain-specific motions (e.g. human hair or body motions), and thus limits their applicability to more general visual content. To overcome this limitation, we explore the synthesis of dynamic content for open-domain images, converting them into animated videos. The key idea is to utilize the motion prior of text-to-video diffusion models by incorporating the image into the generative process as guidance. Given an image, we first project it into a text-aligned rich context representation space using a query transformer, which facilitates the video model to digest the image content in a compatible fashion. However, some visual details still struggle to be preserved in the resultant videos. To supplement with more precise image information, we further feed the full image to the diffusion model by concatenating it with the initial noises. Experimental results show that our proposed method can produce visually convincing and more logical & natural motions, as well as higher conformity to the input image. Comparative evaluation demonstrates the notable superiority of our approach over existing competitors.
Forward citations
Cited by 8 Pith papers
-
MVGBench: Comprehensive Benchmark for Multi-view Generation Models
MVGBench evaluates multi-view generators through self-consistency of 3D reconstructions and uses this protocol to rank 12 models and build a better one.
-
UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models
UniWorld-View couples an occlusion-aware point cloud renderer with a dual-stream video diffusion model to synthesize large-baseline novel views from monocular video.
-
LINR Bridge: Vector Graphic Animation via Neural Implicits and Video Diffusion Priors
LINR Bridge turns an SVG into layered implicit neural fields, animates those fields with video score distillation from a pretrained text-to-video model, then warps the original SVG to match.
-
SketchAnimator: Animate Sketch via Motion Customization of Text-to-Video Diffusion Models
A three-stage method (appearance LoRA, motion LoRA, SDS stroke optimization) animates a user sketch with the motion of a reference video in a one-shot setting.
-
Scaling Up Audio-Synchronized Visual Animation: An Efficient Training Paradigm
An audio-conditioned video animation model is pretrained on noisy auto-curated videos and fine-tuned on a few clean examples, achieving top synchronization scores on a new 48-class benchmark with only 1.9% additional ...
-
TokensGen: Harnessing Condensed Tokens for Long Video Generation
TokensGen generates consistent long videos by representing each clip as condensed semantic tokens, generating all tokens jointly from text, and stitching clips with adaptive FIFO denoising.
-
Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval
Context-as-Memory conditions video generation on selected historical frames chosen by camera FOV overlap, improving scene consistency in long generated videos.
-
GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control
GeoDrive conditions a frozen video diffusion model on a 3D-rendered version of the requested ego trajectory, cutting trajectory-following error by 42% versus Vista while using 99.7% less training data.
Discussion (0). Sign in to comment.