Pith. sign in

REVIEW 4 major objections 6 minor 5 cited by

RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RealCam-I2V claims that anchoring camera trajectories to metric-scale depth makes complex camera control precise and practical for real-world images, reporting over 30% gains on camera-control metrics and over 10% gains on FVD compared…

desk verdict RealCam-I2V is a credible, well-ablated extension that unifies camera scale via monocular metric depth, but the unspecified intrinsics for arbitrary images and missing 4DiM/AC3D baselines keep the real-world claim from being fully proven. read the letter →

arxiv 2502.10059 v2 pith:JOPY3MVR submitted 2025-02-14 cs.CV

Teng Li , Guangcong Zheng , Rui Jiang , Shuigen Zhan , Tao Wu , Yehao Lu , Yining Lin , Chuanyun Deng
show 4 more authors
Yepan Xiong Min Chen Lin Cheng Xi Li
This is my paper · ORCID
classification cs.CV
keywords image-to-videogenerationcameracontrolmetricdepthestimationdiffusionmodel3DsceneinteractionnoiseshapingtrajectoryRealEstate10K
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the main obstacle to practically useful camera-controlled image-to-video generation is scene scale: existing models are trained on relative-scale camera trajectories, so the camera motion they learn is ambiguous for real images of unknown size. RealCam-I2V inserts a monocular metric-depth step that reconstructs the input image as a metric-scale 3D point cloud, and uses that reconstruction in two places. During training, it rescales each video clip's structure-from-motion camera poses to metric units by aligning them with the depth-based point cloud; during inference, the same metric scene lets a user draw a camera trajectory directly in 3D and preview it as a static rendering in under a second. A scene-constrained noise-shaping step then pastes visible pixels of that preview into the diffusion model's high-noise stages, pinning the early layout and camera motion while leaving later stages free for dynamics. On RealEstate10K the method reports camera-control improvements over 30% on metric-scale measurements and FVD improvements over 10% relative to the prior state of the art, as well as generalization to out-of-domain images.

What carries the argument

The load-bearing object is metric scene-scale alignment: camera translations are rescaled by a per-scene factor obtained by aligning the monocular metric-depth point cloud with the structure-from-motion point cloud, which makes camera-conditioned diffusion training operate on physically meaningful units. The interactive 3D scene reprojects the depth map into a point cloud that the user can drag a trajectory through, and a rendered static preview gives immediate feedback. Scene-constrained noise shaping masks that preview and pastes visible, non-edge pixels into the predicted clean latent at noise levels $t>0.9$, constraining early layout while leaving later denoising steps free to synthesize dynamics.

What would settle it

Take a real indoor scene with ground-truth depth from a depth sensor, run the full RealCam-I2V pipeline using the sensor depth instead of the monocular prediction, and compare the camera-control metrics; if metric-scale alignment only helps when the depth comes from the learned predictor, or if the sensor-depth version does not match the monocular one, then the claimed benefit is not the metric scale itself.

Watch

Extended reading notes

Core claim

The paper's central claim is that camera-trajectory-guided image-to-video generation becomes precise and practical once the trajectory is expressed in metric scale, because the model no longer has to guess the scene's true size. It establishes this by using a monocular metric-depth network to reconstruct the reference image as a metric-scale 3D point cloud, rescaling every training video's structure-from-motion camera poses to match that metric scale, and letting users draw trajectories directly in the reconstructed scene while feeding a static preview into the diffusion model's high-noise stages via scene-constrained noise shaping. RealCam-I2V reports that this combination lowers camera-control errors by over 30% on metric-scale measurements and improves FVD by over 10% relative to the strongest prior method, while preserving the base model's ability to generate dynamic content and generalizing to out-of-domain images.

Load-bearing premise

Everything depends on the monocular depth network's prediction being the true real-world scale of the scene; if that scale is wrong for an image, the training alignment, the 3D drawing interface, and the noise-shaping preview all inherit the same error.

Editorial extensions

If this is right

  • With metric-scale conditioning, camera-control error measures such as RotErr, TransErr, and CamMC drop sharply, and training the same base model on aligned trajectories improves even existing camera-conditioned baselines.
  • Users can produce a precise trajectory in one round: draw in the 3D scene, see a static preview immediately, then generate; the slow error-then-adjust loop of text or 2D input disappears.
  • Scene-constrained noise shaping at high noise levels keeps camera adherence strong without freezing the scene; lowering the shaping threshold to 600 or 800 trades dynamics for stricter layout matching.
  • The same model supports camera-controlled loops, frame interpolation, and continuation through simple changes to the condition-frame concatenation, enabling smooth transitions and long videos.
  • The method generalizes outside its training domain to pets, landscapes, anime, and food, while preserving the base model's motion dynamics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If monocular depth is trustworthy, the core idea turns camera control into a 3D drag-and-preview interaction, which could lower the entry barrier for camera-controlled generation enough for consumer editing tools.
  • The alignment recipe is not tied to a particular base model: because it only rescales the conditioning signal, it could be applied to other diffusion backbones (the paper reports one transformer-based variant) with the same expected benefit.
  • The noise-shaping step is a general recipe: any cheap geometric proxy of the desired video, such as a depth map, point-cloud render, or coarse layout, could be injected at high noise levels to steer early structure, suggesting a testable extension to object motion or lighting control.
  • A direct test of the paper's load-bearing premise would be to swap the monocular depth for sensor depth on a few real indoor scenes; if the gains persist with imperfect monocular depth but not with perfect depth, the claimed benefit is not actually caused by metric scale.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes RealCam-I2V, an image-to-video generation framework that adds interactive, precise camera control by integrating monocular metric depth as a preprocessing step. During training, camera poses from RealEstate10K are rescaled from relative to metric units by aligning COLMAP point clouds with Depth Anything V2 metric depth. During inference, the metric depth is used to build an interactive 3D point-cloud scene where users draw camera trajectories and receive preview feedback, and a scene-constrained noise-shaping mechanism modifies early denoising steps using rendered preview frames. Experiments on RealEstate10K and out-of-domain images, with ablations of the two main components, report improved camera-control metrics and FVD relative to reproduced MotionCtrl, CameraCtrl, and CamI2V baselines, and the paper additionally demonstrates applications such as looping, interpolation, and transitions.

Significance. If the results hold, the paper addresses a genuine usability and consistency problem in camera-controlled video generation: replacing relative-scale trajectory conditioning with a single, depth-derived metric scale and giving users an interactive 3D preview is a practical and plausible design. The paper is also commendable for including a component-wise ablation of metric-scale alignment and noise shaping, sensitivity analysis for the noise-shaping threshold (Table 4), generalization tests on an additional dynamic dataset (RealCam-Vid), and applications beyond basic generation. The core conceptual contribution—unifying the scale of training data and user input through one metric-depth model—is clearly stated and internally consistent. However, the verification of the central claim is weakened by (i) an underspecified inference procedure for camera intrinsics, (ii) omission of the closest concurrent metric-depth-based methods from the comparison, (iii) an evaluation protocol that filters difficult test clips and lacks error bars, and (iv) a depth backbone whose stated indoor range conflicts with the claimed out-of-domain generality.

major comments (4)
  1. [Section 3.3, Eq. (pc = D(u,v)K^{-1}[u v 1])] The inference pipeline is underspecified for arbitrary user-supplied images because the camera intrinsics matrix K is never defined for such inputs. On RealEstate10K, K is known from the dataset; for the out-of-domain images in Figure 9 and the claimed real-world use case, the paper does not state whether K is user-provided, estimated, or set to a heuristic default. Since the lateral 3D coordinates and the resulting camera translation scale depend on K, a wrong or arbitrary K breaks the claimed metric-scale consistency between the interactive scene and the metric-aligned training data. The consistency argument in Appendix E covers depth-predictor error but assumes the same K mapping. Please specify how K is obtained for arbitrary images, or explicitly state the assumption and its consequences for the metric-scale guarantee.
  2. [Section 4.3, Table 1] The quantitative comparison omits 4DiM [70] and AC3D [1], both of which are named in Section 2 as methods that also use monocular metric depth to address scale inconsistency. Because these are the closest prior approaches to RealCam-I2V, the paper's claim of establishing 'a new sota' is not supported without comparing against them. Please add these baselines under the same base model and evaluation protocol, or justify their exclusion.
  3. [Sections 4.1 and 4.2] The evaluation protocol filters the dataset by removing the top and bottom 2% of maximum and minimum frame-level scaling factors before splitting into training and test sets. This removes clips with unstable scale alignment and likely inflates the measured performance, especially for metrics that depend on scale. Additionally, although five SfM trials are run per clip, the paper does not report standard deviations or confidence intervals for RotErr, TransErr, CamMC, or FVD. The headline 'over 30% improvement' in Table 1 is therefore not accompanied by evidence of statistical robustness. Please report error bars for the main results and, if possible, evaluate on the unfiltered test set.
  4. [Section 4.1, Appendix C, Appendix E, Figure 9] The paper uses Depth Anything V2 Large Indoor as the metric depth predictor, which is stated to estimate metric depth only up to 20 m and is fine-tuned on indoor data. The method is nevertheless presented as a general real-world solution and demonstrated on outdoor and diverse-domain images (e.g., landscape, food, pets in Figure 9). The consistency argument in Appendix E assumes that the depth model's errors are propagated consistently between training and inference, but for outdoor images beyond the 20 m range, truncation or domain-dependent scale errors can break that consistency. The paper should validate the metric depth against ground-truth metric scale for the out-of-domain images, or restrict the claims to scenes within the depth model's reliable range.
minor comments (6)
  1. [Abstract] "We further enables applications" should read "We further enable applications."
  2. [Section 3.2] "an metric-scale camera-to-world transformation" should be "a metric-scale camera-to-world transformation."
  3. [Figure 1] The figure contains a garbled fragment "RelativScalefailson" that appears to be a placeholder; please clean up the figure text.
  4. [Table 2] The asterisk notation is confusing: the caption says "* denotes our reproduced results on DynamiCrafter" but the table lists DynamiCrafter itself as the first row; clarify which rows are reproduced and which are original.
  5. [Table 4 / Appendix D] The noise-shaping threshold tNS is described as a value in the range [0,1000], but the noise level t in the main text is normalized to [0,1] (Section 3.1). Please make the scale consistent or explicitly explain the conversion.
  6. [References] Metric3D v2 [28] and MonST3R [90] are cited only in the appendix but are relevant to the main text's discussion of depth predictors; consider citing them in Section 2 or Section 3.2.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the metric-scale alignment is an internal-consistency construction, while the evaluated camera and video metrics are measured independently.

full rationale

RealCam-I2V's derivation is not circular. The training-time scale alignment (Sec. 3.2) fits a per-clip scale factor by aligning COLMAP point clouds to metric depth from Depth Anything V2; the inference-time 3D scene (Sec. 3.3) is built from the same metric depth predictor, so the user-drawn trajectory is ensured to be in the same coordinate convention as the training camera conditions. This is an internal-consistency construction, not an evaluation target. Camera-control metrics are computed by running COLMAP/GLOMAP on generated videos and comparing the recovered poses to ground-truth RealEstate10K poses; FVD compares generated videos to ground-truth videos. Neither measurement is a function of the fitted scale factor or of the noise-shaping mask, so the reported improvements in Table 1 are experimental outcomes rather than consequences of the construction. The paper cites its own CamI2V and RealCam-Vid, but only as a comparison baseline and an additional training/evaluation dataset; the load-bearing components (DynamiCrafter as base model, Depth Anything V2 as depth prior, COLMAP for SfM, RealEstate10K as benchmark) are external. The open concerns in the manuscript, such as the assumed metric correctness of Depth Anything and the unspecified handling of camera intrinsics K for arbitrary user images, are robustness or underspecification issues and do not amount to a circular reduction of any claimed result to its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method rests on an external monocular depth model (Depth Anything V2) whose metric scale is taken as ground truth for both training alignment and inference scene construction, and on COLMAP poses as ground truth for evaluation. It also adds several hand-tuned hyperparameters (noise-shaping threshold, visibility kernel size, outlier quantiles). No new physical entities are introduced.

free parameters (3)
  • Noise shaping threshold tNS = 900 (timestep, apply when t > 0.9)
    Controls the trade-off between camera-control fidelity and video dynamics; selected from {800, 900, 1000} in Appendix D, Table 4.
  • Visibility filter kernel size k = 5 (Fig. 6; text notes k >= 3)
    Excludes edge pixels near depth holes from noise shaping; chosen by the authors to stabilize depth-prediction outliers.
  • Outlier quantile for scale alignment = top and bottom 2% of frame-level scaling factors
    Clips with extreme scale factors are removed from training and test sets in Section 4.1; this is a post-hoc data selection choice.
assumptions (4)
  • domain assumption Depth Anything V2 metric depth estimates are accurate enough to define a unified real-world scale across training and inference images.
    Section 3.2 uses fdepth output D(u,v) to construct the 3D scene and to align COLMAP trajectories; Appendix C notes the indoor model tops out at 20m.
  • domain assumption COLMAP camera poses from RealEstate10K are reliable relative-scale ground truth for alignment and evaluation.
    Section 4.1 reconstructs point clouds from RealEstate10K with COLMAP; metrics also rely on COLMAP/GLOMAP pose estimation on generated videos.
  • ad hoc to paper The static 3D point cloud of the reference frame is a sufficient proxy for the generated video's layout during high-noise stages.
    Scene-constrained noise shaping in Section 3.3 overwrites z0 with preview pixels; this assumes geometry holes and depth errors do not corrupt the layout.
  • standard math Standard latent diffusion training objective (Eq. 1) and noise schedule are valid.
    Uses the conventional epsilon/v-prediction loss from DynamiCrafter, with no new derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control." pith.science (2026). https://pith.science/paper/JOPY3MVR

@misc{pith2026250210059,
  author       = {Pith},
  title        = {Pith review of: RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JOPY3MVR}},
  note         = {Machine review of arXiv:2502.10059}
}
read the original abstract

Recent advancements in camera-trajectory-guided image-to-video generation offer higher precision and better support for complex camera control compared to text-based approaches. However, they also introduce significant usability challenges, as users often struggle to provide precise camera parameters when working with arbitrary real-world images without knowledge of their depth nor scene scale. To address these real-world application issues, we propose RealCam-I2V, a novel diffusion-based video generation framework that integrates monocular metric depth estimation to establish 3D scene reconstruction in a preprocessing step. During training, the reconstructed 3D scene enables scaling camera parameters from relative to metric scales, ensuring compatibility and scale consistency across diverse real-world images. In inference, RealCam-I2V offers an intuitive interface where users can precisely draw camera trajectories by dragging within the 3D scene. To further enhance precise camera control and scene consistency, we propose scene-constrained noise shaping, which shapes high-level noise and also allows the framework to maintain dynamic and coherent video generation in lower noise stages. RealCam-I2V achieves significant improvements in controllability and video quality on the RealEstate10K and out-of-domain images. We further enables applications like camera-controlled looping video generation and generative frame interpolation. Project page: https://zgctroy.github.io/RealCam-I2V.

Figures

Figures reproduced from arXiv: 2502.10059 by the authors.

Figure 1
Figure 1. We propose RealCam-I2V, a camera controllable image-to-video generation framework for complex real-world camera control [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. RealCam-I2V pipeline. For training, we align camera parameters from relative scale to metric scale. For inference, we use metric depth estimation to construct the point cloud for users to interactively draw the camera trajectory. Due to the metric scale alignment, the user-given camera trajectory in the 3D scene shares the same scene scale as those in real world. elScope [62], LAVIE [67], and VideoFactory [65] have … view at source ↗
Figure 3
Figure 3. Scene scale mismatch. Point clouds reconstructed from metric depth estimation (RGB) are robust and unified, whereas SfM reconstructions (yellow) are relative-scale that may vary across frames. Our alignment enables relative-to-metric conver￾sion of scene scale for real-world applications [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Camera trajectory ambiguity. The relative scene-scale measurement fundamentally hinders models from learning physi￾cally consistent camera motion. learns to estimate the noise ϵ added at each timestep t, con￾ditioned on both a text prompt ctxt, a reference image cimg, …
Figure 7
Figure 7. Figure 7: Preview video rendering for user-expected camera trajectory. It also serves as the reference video in our proposed scene-constrained noise shaping. Basic Mode Interpolation Mode … !! " !! # !! $ !! %&" !! % … !' " !' " !' " !' " !' " … !! " !! # !! $ !! %&" !! % … !' "…
Figure 8
Figure 8. Figure 8: Concatenation strategies for different tasks. Basic mode, interpolation mode, and continuation mode can be sup￾ported with only minor modification. contains any invisible pixels, it is considered to lie on an object’s edge and potentially affected by depth prediction e…
Figure 9
Figure 9. Figure 9: Visualization on various domains in real life scenarios. Despite training on RealEstate10K [101], our method can generalize naturally to out-of-domain images, including pets, landscape, anime, food and etc [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Visualization on complex trajectory, large move￾ment and video dynamism. Our framework achieves precise tra￾jectory adherence for complex camera motion paths while preserv￾ing high-fidelity video generation with dynamics. ter dynamics compared to using noise shaping a…
Figure 11
Figure 11. Figure 11: Outlier filtering for noise shaping. Undesired pixels will not be pasted in mistake for filtering kernel size k ≥3. with mixed-precision fp16 and ZeRO-1. For MotionCtrl [69] and CameraCtrl [22], we repro￾duce all results on DynamiCrafter for fair comparison. For CamI2…
Figure 12
Figure 12. Figure 12: Camera Trajectory Interpolation. We interpolate camera keyframes given by user to dense trajectories. catering to different preferences. Our interactive preview is designed for nearly real-time feedback. Monocular depth estimation is efficient. For preview videos (e.g…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. InverseCrafter: Efficient Video ReCapture as a Latent Domain Inverse Problem

    cs.CV 2025-12 conditional novelty 6.0 of 10

    A training-free, near-zero-overhead inverse solver for novel-view video generation and inpainting that projects masks into continuous multi-channel latent masks and applies DDS with conjugate gradient in latent space.

  2. PostCam: Camera-Controllable Novel-View Video Generation with Query-Shared Cross-Attention

    cs.CV 2025-11 conditional novelty 6.0 of 10

    PostCam generates new videos from a reference video along user-specified camera trajectories using a query-shared cross-attention that fuses pose data and rendered frames, improving control precision and detail preservation.

  3. EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidance

    cs.CV 2025-05 conditional novelty 6.0 of 10

    EPiC trains a 30M-parameter visibility-aware ControlNet on mask-based anchor videos from 5,000 in-the-wild videos and 500 steps, reaching SOTA camera accuracy on RealEstate10K and MiraData.

  4. LiON-LoRA: Rethinking LoRA Fusion to Unify Controllable Spatial and Temporal Generation for Video Diffusion

    cs.CV 2025-07 conditional novelty 5.0 of 10

    LiON-LoRA adds a learned scaling token to video-diffusion LoRA adapters, enabling linear and independent control of camera trajectory and object motion strength.

  5. Follow-Your-Creation: Empowering 4D Creation through Video Inpainting

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Follow-Your-Creation fine-tunes the Wan2.1 video inpainting model on composite point-cloud and editing masks so a single monocular video can be converted into editable 4D video with new camera motion.

Reference graph

Works this paper leans on

100 extracted references · 13 canonical work pages · cited by 5 Pith papers

  1. [70]

    Controlling space and time with diffusion models

    Daniel Watson, Saurabh Saxena, Lala Li, Andrea Tagliasac- chi, and David J Fleet. Controlling space and time with diffusion models. arXiv preprint arXiv:2407.07860, 2024. 3

    arXiv 2024

  2. [1]

    Ac3d: Analyzing and improving 3d camera control in video diffusion trans- formers

    Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Ali- aksandr Siarohin, Willi Menapace, Andrea Tagliasacchi, David B Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion trans- formers. arXiv preprint arXiv:2411.18673, 2024. 3

    arXiv 2024

  3. [2]

    Vd3d: Taming large video diffu- sion transformers for 3d camera control

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. Vd3d: Taming large video diffu- sion transformers for 3d camera control. arXiv preprint arXiv:2407.12781, 2024. 3

    arXiv 2024

  4. [3]

    Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models

    Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models. arXiv preprint arXiv:2405.04233, 2024. 3

    arXiv 2024

  5. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3

    arXiv 2023

  6. [5]

    Align your latents: High-resolution video synthe- sis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthe- sis with latent diffusion models. In CVPR, pages 22563– 22575, 2023. 2

    2023

  7. [6]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators

  8. [7]

    Still-moving: Customized video generation without customized video data

    Hila Chefer, Shiran Zada, Roni Paiss, Ariel Ephrat, Omer Tov, Michael Rubinstein, Lior Wolf, Tali Dekel, Tomer Michaeli, and Inbar Mosseri. Still-moving: Customized video generation without customized video data. arXiv preprint arXiv:2407.08674, 2024. 3

    arXiv 2024

Show all 100 references
  1. [8]

    Motion-zero: Zero-shot mov- ing object control framework for diffusion-based video gen- eration

    Changgu Chen, Junwei Shu, Lianggangxu Chen, Gaoqi He, Changbo Wang, and Yang Li. Motion-zero: Zero-shot mov- ing object control framework for diffusion-based video gen- eration. arXiv preprint arXiv:2401.10150, 2024. 3

    2024 arXiv

  2. [9]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023. 2

    arXiv

  3. [10]

    Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models. In CVPR, pages 7310–7320, 2024. 2

    2024

  4. [11]

    Goku: Flow based video genera- tive foundation models

    Shoufa Chen, Chongjian Ge, Yuqi Zhang, Yida Zhang, Fengda Zhu, Hao Yang, Hongxiang Hao, Hui Wu, Zhichao Lai, Yifei Hu, et al. Goku: Flow based video genera- tive foundation models. arXiv preprint arXiv:2502.04896,

    arXiv

  5. [12]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In Proceedings of the IEEE/CVF Confere...

    2024

  6. [13]

    Seine: Short-to-long video diffu- sion model for generative transition and prediction

    Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffu- sion model for generative transition and prediction. In The Twelfth International Conference on Learning Representa- ti...

    2023

  7. [14]

    Vchitect-2.0: Parallel transformer for scaling up video diffusion models

    Weichen Fan, Chenyang Si, Junhao Song, Zhenyu Yang, Yinan He, Long Zhuo, Ziqi Huang, Ziyue Dong, Jingwen He, Dongwei Pan, et al. Vchitect-2.0: Parallel transformer for scaling up video diffusion models. arXiv preprint arXiv:2501.08453, 2025. 3

    2025 arXiv

  8. [15]

    I2vcontrol-camera: Precise video camera control with adjustable motion strength

    Wanquan Feng, Jiawei Liu, Pengqi Tu, Tianhao Qi, Mingzhen Sun, Tianxiang Ma, Songtao Zhao, Siyu Zhou, and Qian He. I2vcontrol-camera: Precise video camera control with adjustable motion strength. 2024. 3

    2024

  9. [16]

    Lumina-t2x: Transforming text into any modality, resolution, and duration via flow-based large diffusion transformers

    Peng Gao, Le Zhuo, Dongyang Liu, Ruoyi Du, Xu Luo, Longtian Qiu, Yuhang Zhang, Chen Lin, Rongjie Huang, Shijie Geng, et al. Lumina-t2x: Transforming text into any modality, resolution, and duration via flow-based large diffusion transformers. arXiv preprint arXiv:2405.05945 ,

    arXiv

  10. [17]

    Preserve your own cor- relation: A noise prior for video diffusion models

    Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, An- drew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji. Preserve your own cor- relation: A noise prior for video diffusion models. In Proceedings of the IEEE/CVF International Conference on C...

    2023

  11. [18]

    Emu video: Factoriz- ing text-to-video generation by explicit image conditioning

    Rohit Girdhar, Mannat Singh, Andrew Brown, Quentin Du- val, Samaneh Azadi, Sai Saketh Rambhatla, Akbar Shah, Xi Yin, Devi Parikh, and Ishan Misra. Emu video: Factoriz- ing text-to-video generation by explicit image conditioning. arXiv preprint arXiv:2311.10709, 2023. 2

    2023 arXiv

  12. [19]

    Atomovideo: High fidelity image-to-video generation

    Litong Gong, Yiran Zhu, Weijie Li, Xiaoyang Kang, Biao Wang, Tiezheng Ge, and Bo Zheng. Atomovideo: High fidelity image-to-video generation. arXiv preprint arXiv:2403.01800, 2024. 3

    2024 arXiv

  13. [20]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 2, 3

    2023 arXiv

  14. [21]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In European Conference on Computer Vision, pages 330–348. Springer, 2025. 3

    2025

  15. [22]

    Cameractrl: En- abling camera control for text-to-video generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: En- abling camera control for text-to-video generation. arXiv preprint arXiv:2404.02101, 2024. 2, 3, 6, 7, 1

    2024 arXiv

  16. [23]

    Co-speech gesture video gen- eration via motion-decoupled diffusion model

    Xu He, Qiaochu Huang, Zhensong Zhang, Zhiwei Lin, Zhiyong Wu, Sicheng Yang, Minglei Li, Zhiyi Chen, Song- cen Xu, and Xiaofei Wu. Co-speech gesture video gen- eration via motion-decoupled diffusion model. In CVPR, pages 2263–2273, 2024. 3

    2024

  17. [24]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221, 2022. 2

    2022 arXiv

  18. [25]

    Training-free camera control for video generation

    Chen Hou, Guoqiang Wei, Yan Zeng, and Zhibo Chen. Training-free camera control for video generation. arXiv preprint arXiv:2406.10126, 2024. 3

    2024 arXiv

  19. [26]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. 3

    2021 arXiv

  20. [27]

    Animate anyone: Consistent and controllable image-to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. InCVPR, pages 8153–8163, 2024. 3

    2024

  21. [28]

    Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Pattern Analys...

    2024

  22. [29]

    Mo- tionmaster: Training-free camera motion transfer for video generation

    Teng Hu, Jiangning Zhang, Ran Yi, Yating Wang, Hongrui Huang, Jieyu Weng, Yabiao Wang, and Lizhuang Ma. Mo- tionmaster: Training-free camera motion transfer for video generation. arXiv preprint arXiv:2404.15789, 2024. 2, 3

    2024 arXiv

  23. [30]

    Vbench++: Comprehensive and versatile bench- mark suite for video generative models

    Ziqi Huang, Fan Zhang, Xiaojie Xu, Yinan He, Jiashuo Yu, Ziyue Dong, Qianli Ma, Nattapol Chanpaisit, Chenyang Si, Yuming Jiang, Yaohui Wang, Xinyuan Chen, Ying- Cong Chen, Limin Wang, Dahua Lin, Yu Qiao, and Zi- wei Liu. Vbench++: Comprehensive and versatile bench- mark suite ...

    2024 arXiv

  24. [31]

    Peekaboo: Interactive video generation via masked- diffusion

    Yash Jain, Anshul Nasery, Vibhav Vineet, and Harkirat Behl. Peekaboo: Interactive video generation via masked- diffusion. In CVPR, pages 8079–8088, 2024. 2, 3

    2024

  25. [32]

    A survey of multimodal con- trollable diffusion models

    Rui Jiang, Guang-Cong Zheng, Teng Li, Tian-Rui Yang, Jing-Dong Wang, and Xi Li. A survey of multimodal con- trollable diffusion models. Journal of Computer Science and Technology, 39(3):509–541, 2024. 3

    2024

  26. [33]

    Videobooth: Diffusion-based video generation with image prompts

    Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, and Ziwei Liu. Videobooth: Diffusion-based video generation with image prompts. In CVPR, pages 6689–6700, 2024. 3

    2024

  27. [34]

    Miradata: A large-scale video dataset with long du- rations and structured captions

    Xuan Ju, Yiming Gao, Zhaoyang Zhang, Ziyang Yuan, Xin- tao Wang, Ailing Zeng, Yu Xiong, Qiang Xu, and Ying Shan. Miradata: A large-scale video dataset with long du- rations and structured captions. Advances in Neural Infor- mation Processing Systems, 37:48955–48970, 2024. 3

    2024

  28. [35]

    Hunyuanvideo: A systematic frame- work for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jian- wei Zhang, et al. Hunyuanvideo: A systematic frame- work for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 3

    2024 arXiv

  29. [36]

    Collaborative video diffusion: Consistent multi- video generation with camera control

    Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hongsheng Li, Leonidas Guibas, and Gordon Wet- zstein. Collaborative video diffusion: Consistent multi- video generation with camera control. arXiv preprint arXiv:2405.17414, 2024. 3

    2024 arXiv

  30. [37]

    Personalvideo: High id-fidelity video customization without dynamic and semantic degradation

    Hengjia Li, Haonan Qiu, Shiwei Zhang, Xiang Wang, Yu- jie Wei, Zekun Li, Yingya Zhang, Boxi Wu, and Deng Cai. Personalvideo: High id-fidelity video customization without dynamic and semantic degradation. arXiv preprint arXiv:2411.17048, 2024. 3

    2024 arXiv

  31. [38]

    Image conductor: Precision control for interactive video synthesis

    Yaowei Li, Xintao Wang, Zhaoyang Zhang, Zhouxia Wang, Ziyang Yuan, Liangbin Xie, Yuexian Zou, and Ying Shan. Image conductor: Precision control for interactive video synthesis. arXiv preprint arXiv:2406.15339, 2024. 2, 3

    2024 arXiv

  32. [39]

    Generative image dynamics

    Zhengqi Li, Richard Tucker, Noah Snavely, and Aleksander Holynski. Generative image dynamics. In CVPR, pages 24142–24153, 2024. 3

    2024

  33. [40]

    Open-sora plan: Open-source large video generation model

    Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang, Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, et al. Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131,

    arXiv

  34. [41]

    Step-video-t2v technical report: The practice, challenges, and future of video foun- dation model

    Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, et al. Step-video-t2v technical report: The practice, challenges, and future of video foun- dation model. arXiv preprint arXiv:2502.10248, 2025. 3

    2025 arXiv

  35. [42]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Zi- wei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024. 3

    2024 arXiv

  36. [43]

    Follow your pose: Pose- guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose- guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 4117–4125, 2024. 3

    2024

  37. [44]

    Sdedit: Guided image synthesis and editing with stochastic differential equations

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073, 2021. 5

    2021 arXiv

  38. [45]

    T2i-adapter: Learn- ing adapters to dig out more controllable ability for text-to- image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learn- ing adapters to dig out more controllable ability for text-to- image diffusion models. In Proceedings of the AAAI Con- ference on Artificial Intelligence , pages 4296–43...

    2024

  39. [46]

    Sg-i2v: Self- guided trajectory control in image-to-video generation

    Koichi Namekata, Sherwin Bahmani, Ziyi Wu, Yash Kant, Igor Gilitschenski, and David B Lindell. Sg-i2v: Self- guided trajectory control in image-to-video generation. arXiv preprint arXiv:2411.04989, 2024. 3

    2024 arXiv

  40. [47]

    Openvid-1m: A large-scale high-quality dataset for text-to- video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhen- heng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to- video generation. arXiv preprint arXiv:2407.02371, 2024. 3

    2024 arXiv

  41. [48]

    Global Structure-from-Motion Revisited

    Linfei Pan, Daniel Barath, Marc Pollefeys, and Jo- hannes Lutz Sch ¨onberger. Global Structure-from-Motion Revisited. In European Conference on Computer Vision (ECCV), 2024. 6, 2

    2024

  42. [49]

    Scalable diffusion mod- els with transformers

    William Peebles and Saining Xie. Scalable diffusion mod- els with transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 4195– 4205, 2023. 3

    2023

  43. [50]

    Controlnext: Powerful and effi- cient control for image and video generation.arXiv preprint arXiv:2408.06070, 2024

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation.arXiv preprint arXiv:2408.06070, 2024. 3

    2024 arXiv

  44. [51]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 3

    2023 arXiv

  45. [52]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022. 2

    2022 arXiv

  46. [53]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022. 2, 3

    2022

  47. [54]

    Structure-from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4104–4113, 2016. 2, 6

    2016

  48. [55]

    Repvideo: Rethinking cross- layer representation for video generation

    Chenyang Si, Weichen Fan, Zhengyao Lv, Ziqi Huang, Yu Qiao, and Ziwei Liu. Repvideo: Rethinking cross- layer representation for video generation. arXiv preprint arXiv:2501.08994, 2025. 3

    2025 arXiv

  49. [56]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 5

    2010 arXiv

  50. [57]

    Moma: Multimodal llm adapter for fast personalized image generation

    Kunpeng Song, Yizhe Zhu, Bingchen Liu, Qing Yan, Ahmed Elgammal, and Xiao Yang. Moma: Multimodal llm adapter for fast personalized image generation. arXiv preprint arXiv:2404.05674, 2024. 3

    2024 arXiv

  51. [58]

    Any-to-any generation via composable diffusion

    Zineng Tang, Ziyi Yang, Chenguang Zhu, Michael Zeng, and Mohit Bansal. Any-to-any generation via composable diffusion. In Thirty-seventh Conference on Neural Informa- tion Processing Systems, 2023. 3

    2023

  52. [59]

    Emo: Emote portrait alive-generating expressive portrait videos with audio2video diffusion model under weak conditions

    Linrui Tian, Qi Wang, Bang Zhang, and Liefeng Bo. Emo: Emote portrait alive-generating expressive portrait videos with audio2video diffusion model under weak conditions. arXiv preprint arXiv:2402.17485, 2024. 3

    2024 arXiv

  53. [60]

    Consistent view syn- thesis with pose-guided diffusion models

    Hung-Yu Tseng, Qinbo Li, Changil Kim, Suhib Alsisan, Jia-Bin Huang, and Johannes Kopf. Consistent view syn- thesis with pose-guided diffusion models. In CVPR, pages 16773–16783, 2023. 3

    2023

  54. [61]

    Towards accurate generative models of video: A new met- ric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new met- ric & challenges. arXiv preprint arXiv:1812.01717, 2018. 7

    2018 arXiv

  55. [62]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 3

    2023 arXiv

  56. [63]

    Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content

    Qiuheng Wang, Yukai Shi, Jiarong Ou, Rui Chen, Ke Lin, Jiahao Wang, Boyuan Jiang, Haotian Yang, Mingwu Zheng, Xin Tao, et al. Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content. arXiv preprint arXiv:2410.08260, 2024. 3

    arXiv

  57. [64]

    Disco: Disentangled control for referring hu- man dance generation in real world

    Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Li- juan Wang. Disco: Disentangled control for referring hu- man dance generation in real world. arXiv e-prints, pages arXiv–2307, 2023. 3

    2023

  58. [65]

    Videofactory: Swap attention in spatiotemporal diffusions for text-to-video gen- eration, 2024

    Wenjing Wang, Huan Yang, Zixi Tuo, Huiguo He, Junchen Zhu, Jianlong Fu, and Jiaying Liu. Videofactory: Swap attention in spatiotemporal diffusions for text-to-video gen- eration, 2024. 3

    2024

  59. [66]

    Videocomposer: Compositional video syn- thesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. Videocomposer: Compositional video syn- thesis with motion controllability. Advances in Neural In- formation Processing Systems, 36, 2024. 2, 3

    2024

  60. [67]

    Lavie: High-quality video genera- tion with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video genera- tion with cascaded latent diffusion models. arXiv preprint arXiv:2309.15103, 2023. 3

    2023 arXiv

  61. [68]

    Customvideo: Customizing text- to-video generation with multiple subjects

    Zhao Wang, Aoxue Li, Enze Xie, Lingting Zhu, Yong Guo, Qi Dou, and Zhenguo Li. Customvideo: Customizing text- to-video generation with multiple subjects. arXiv preprint arXiv:2401.09962, 2024. 3

    2024

  62. [69]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 2, 3, 6, 7, 1

    2024

  63. [71]

    Motionbooth: Motion-aware customized text-to- video generation

    Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Motionbooth: Motion-aware customized text-to- video generation. arXiv preprint arXiv:2406.17758, 2024. 3

    2024 arXiv

  64. [72]

    Spherediffusion: Spherical geometry-aware distortion resilient diffusion model

    Tao Wu, Xuewei Li, Zhongang Qi, Di Hu, Xintao Wang, Ying Shan, and Xi Li. Spherediffusion: Spherical geometry-aware distortion resilient diffusion model. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 6126–6134, 2024. 3

    2024

  65. [73]

    Videomaker: Zero-shot customized video generation with the inherent force of video diffusion models

    Tao Wu, Yong Zhang, Xiaodong Cun, Zhongang Qi, Junfu Pu, Huanzhang Dou, Guangcong Zheng, Ying Shan, and Xi Li. Videomaker: Zero-shot customized video generation with the inherent force of video diffusion models. arXiv preprint arXiv:2412.19645, 2024. 3

    2024 arXiv

  66. [74]

    Customcrafter: Customized video generation with pre- serving motion and concept composition abilities

    Tao Wu, Yong Zhang, Xintao Wang, Xianpan Zhou, Guangcong Zheng, Zhongang Qi, Ying Shan, and Xi Li. Customcrafter: Customized video generation with pre- serving motion and concept composition abilities. arXiv preprint arXiv:2408.13239, 2024. 3

    2024 arXiv

  67. [75]

    Ifadapter: Instance feature con- trol for grounded text-to-image generation

    Yinwei Wu, Xianpan Zhou, Bing Ma, Xuefeng Su, Kai Ma, and Xinchao Wang. Ifadapter: Instance feature con- trol for grounded text-to-image generation. arXiv preprint arXiv:2409.08240, 2024. 3

    2024 arXiv

  68. [76]

    Dynamicrafter: Animating open-domain images with video diffusion pri- ors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Xin- tao Wang, Tien-Tsin Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion pri- ors. arXiv preprint arXiv:2310.12190, 2023. 3, 6, 7, 1

    2023 arXiv

  69. [77]

    Cavia: Camera-controllable multi-view video diffusion with view-integrated attention

    Dejia Xu, Yifan Jiang, Chen Huang, Liangchen Song, Thorsten Gernoth, Liangliang Cao, Zhangyang Wang, and Hao Tang. Cavia: Camera-controllable multi-view video diffusion with view-integrated attention. arXiv preprint arXiv:2410.10774, 2024. 3

    arXiv

  70. [79]

    Camco: Camera-controllable 3d-consistent image-to-video genera- tion

    Dejia Xu, Weili Nie, Chao Liu, Sifei Liu, Jan Kautz, Zhangyang Wang, and Arash Vahdat. Camco: Camera-controllable 3d-consistent image-to-video genera- tion. arXiv preprint arXiv:2406.02509, 2024. 2, 6

    2024 arXiv

  71. [80]

    Magicanimate: Temporally consistent human image animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Han- shu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human image animation using diffusion model. In CVPR, pages 1481–1490, 2024. 3

    2024

  72. [81]

    Depth any- thing v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2. arXiv:2406.09414, 2024. 2, 6, 1

    2024 arXiv

  73. [82]

    Direct-a-video: Customized video generation with user- directed camera movement and object motion

    Shiyuan Yang, Liang Hou, Haibin Huang, Chongyang Ma, Pengfei Wan, Di Zhang, Xiaodong Chen, and Jing Liao. Direct-a-video: Customized video generation with user- directed camera movement and object motion. In ACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024. 3

    2024

  74. [83]

    Cogvideox: Text-to- video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to- video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 2, 3

    2024 arXiv

  75. [84]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  76. [85]

    Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089, 2023. 3

    2023 arXiv

  77. [86]

    Scaling autore- gressive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autore- gressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2(3):5, 2022. 3

    2022 arXiv

  78. [87]

    Efficient video diffusion mod- els via content-frame motion-latent decomposition

    Sihyun Yu, Weili Nie, De-An Huang, Boyi Li, Jinwoo Shin, and Anima Anandkumar. Efficient video diffusion mod- els via content-frame motion-latent decomposition. arXiv preprint arXiv:2403.14148, 2024. 3

    2024 arXiv

  79. [88]

    Make pixels dance: High-dynamic video generation

    Yan Zeng, Guoqiang Wei, Jiani Zheng, Jiaxin Zou, Yang Wei, Yuchen Zhang, and Hang Li. Make pixels dance: High-dynamic video generation. In CVPR, pages 8850– 8860, 2024. 3

    2024

  80. [89]

    Moonshot: To- wards controllable video generation and editing with multi- modal conditions

    David Junhao Zhang, Dongxu Li, Hung Le, Mike Zheng Shou, Caiming Xiong, and Doyen Sahoo. Moonshot: To- wards controllable video generation and editing with multi- modal conditions. arXiv preprint arXiv:2401.01827, 2024. 3

    2024 arXiv

  81. [90]

    Monst3r: A simple approach for esti- mating geometry in the presence of motion

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jam- pani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for esti- mating geometry in the presence of motion. arXiv preprint arxiv:2410.03825, 2024. 2

    arXiv

  82. [91]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 3

    2023

  83. [92]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023. 3

    2023 arXiv

  84. [93]

    Pia: Your personalized image ani- mator via plug-and-play modules in text-to-image models

    Yiming Zhang, Zhening Xing, Yanhong Zeng, Youqing Fang, and Kai Chen. Pia: Your personalized image ani- mator via plug-and-play modules in text-to-image models. In CVPR, pages 7747–7756, 2024. 3

    2024

  85. [94]

    3d object manipulation in a single image using generative models

    Ruisi Zhao, Zechuan Zhang, Zongxin Yang, and Yi Yang. 3d object manipulation in a single image using generative models. arXiv preprint arXiv:2501.12935, 2025. 3

    2025 arXiv

  86. [95]

    Entropy-driven sampling and training scheme for conditional diffusion gen- eration

    Guangcong Zheng, Shengming Li, Hui Wang, Taiping Yao, Yang Chen, Shouhong Ding, and Xi Li. Entropy-driven sampling and training scheme for conditional diffusion gen- eration. In European Conference on Computer Vision , pages 754–769. Springer, 2022. 2

    2022

  87. [96]

    Layoutdiffusion: Controllable diffusion model for layout-to-image generation

    Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Controllable diffusion model for layout-to-image generation. In CVPR, pages 22490–22499, 2023. 3

    2023

  88. [97]

    Cami2v: Camera-controlled image-to-video dif- fusion model

    Guangcong Zheng, Teng Li, Rui Jiang, Yehao Lu, Tao Wu, and Xi Li. Cami2v: Camera-controlled image-to-video dif- fusion model. arXiv preprint arXiv:2410.15957, 2024. 2, 3, 6, 7, 1

    2024 arXiv

  89. [98]

    Realcam-vid: High-resolution video dataset with dynamic scenes and metric-scale camera movements

    Guangcong Zheng, Teng Li, Xianpan Zhou, and Xi Li. Realcam-vid: High-resolution video dataset with dynamic scenes and metric-scale camera movements. arXiv preprint arXiv:2504.08212, 2025. 6, 2

    2025 arXiv

  90. [99]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024. 3

    2024 arXiv

  91. [100]

    Open3D: A modern library for 3D data processing

    Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3D: A modern library for 3D data processing. arXiv:1801.09847, 2018. 2

    2018 arXiv

  92. [101]

    Stereo magnification: Learning view synthesis using multiplane images

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. In SIGGRAPH, 2018. 2, 6, 7, 8, 1 RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control Appendix ...

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.