Pith. sign in

REVIEW 2 major objections 5 minor 4 cited by

Light-A-Video: Training-free Video Relighting via Progressive Light Fusion

T0 review · 2 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Light-A-Video makes video relighting training-free by blending frames during denoising.

desk verdict A practical training-free relighting pipeline whose quality claim rests on a self-referential FID; the method itself is plausible and worth refereeing. read the letter →

arxiv 2502.08590 v2 pith:RUXYLZON submitted 2025-02-12 cs.CV

Yujie Zhou , Jiazi Bu , Pengyang Ling , Pan Zhang , Tong Wu , Qidong Huang , Jinsong Li , Xiaoyi Dong
show 5 more authors
Yuhang Zang Yuhang Cao Anyi Rao Jiaqi Wang Li Niu
This is my paper · ORCID
classification cs.CV
keywords videorelightingtraining-freediffusionmodelsconsistentlightattentionprogressivefusiontemporalconsistencytransportIC-Light
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Light-A-Video sets out to prove that temporally smooth relighting of arbitrary videos can be achieved without any training or optimization, by reusing a frozen image relighting model in tandem with a frozen video diffusion model. The authors show that a straightforward per-frame application of an image relighter flickers because the generated light source and the relighted appearance are unconstrained across time. They counter this with two interventions: consistent light attention, which injects time-averaged features into the relighter's self-attention to stabilize the background light, and progressive light fusion, which replaces the video model's denoising target with a linearly blended version of the source and relighted frames, with the blend weight shrinking as denoising progresses. If the claim holds, relighting becomes an inference-only operation that works with off-the-shelf components, and it supports both whole-video relighting and text-guided relighting of foreground sequences with generated backgrounds.

What carries the argument

The argument runs on two modules placed in the inference loop. Consistent Light Attention (CLA) alters each self-attention layer of the frozen IC-Light relighter so that the frame's features are mixed with their temporally averaged counterpart, $h'_o = (1-\gamma) h'_1 + \gamma \bar{h}'_2$, giving the generated light source a stable cross-frame anchor. Progressive Light Fusion (PLF) is the deeper mechanism: for each denoising step the video model's predicted ``consistent target'' $z^v_{0\leftarrow t}$ is relighted to $z^r_{0\leftarrow t}$, both are decoded and blended by the light-transport-motivated formula $I_p^t = I_v^t + \lambda_t(I_r^t - I_v^t)$ with $\lambda_t = 1 - t/T_m$, and the re-encoded blend $E(I_p^t)$ replaces the original denoising target so that the step direction is bent toward relighting without leaving the video model's refinement envelope. A detail compensation term $\Delta d_m$ captured in the first denoising step is added back so that source-video details are not lost in the consistent target.

What would settle it

Use a synthetic scene with known geometry, albedo, and two environment lights, so ground-truth renders exist for every convex combination of the lights. Compare $I_v^t + \lambda_t(I_r^t - I_v^t)$ against a true render under the blended light for a sweep of $\lambda_t$; if the frames diverge substantially in shadow or specular regions, the light-transport interpolation behind Eq. 10 is empirically false. A second check is to weaken the VDM's refinement by using a near-zero-strength denoiser and ask whether the fused frames alone form a temporally smooth relighted video.

Watch

Extended reading notes

Core claim

The central discovery is that the illumination change imposed by an image relighting model can be transferred onto a video simply by steering the denoising trajectory of a video diffusion model; no video-specific relighting network or dataset is required. At each denoising step the video model's predicted clean frame is relit by the image model, and then a convex combination of the original and relighted appearances, $I_p^t = I_v^t + \lambda_t (I_r^t - I_v^t)$, is re-encoded and used as the new denoising target. The weight $\lambda_t$ decreases from near 1 to 0 across the schedule, so early steps push the output toward the relighted appearance while later steps let the video model's motion priors snap the result back to a coherent video. The paper presents this as a practical application of the linearity of light transport: since a scene's appearance under combined illumination is the sum of appearances under each illumination, the authors treat appearance blending as a proxy for lighting interpolation, and report that a constant blend target degrades quality while the progressive schedule preserves both relight fidelity and temporal consistency.

Load-bearing premise

The load-bearing premise is that a pixel-space blend of the source frame and the relighted frame, with a weight that shrinks during denoising, behaves like an intermediate illumination state that the video model can refine; if that blend is not a valid middle state, the denoising trajectory is steered toward an appearance that the model must then conceal rather than complete.

Editorial extensions

If this is right

  • Per-frame image relighting becomes a serviceable video relighter: any user with IC-Light and a public video diffusion model can relight a video or a foreground sequence with a text prompt, with no training run.
  • The method is backbone-agnostic: the same progressive-fusion recipe operates on UNet-based models such as AnimateDiff and DiT-based models such as CogVideoX, so improvements in either component should transfer directly.
  • Because the fusion target is a linear blend, the schedule $\lambda_t = 1 - t/T_m$ is a single hyperparameter; the supplementary experiments show that a constant blend target degrades motion and quality, implying the progressive schedule is what keeps the denoising trajectory inside the video model's refinement capability.
  • The approach inherits the limitations of its frozen components: output quality tracks the image relighter and the video diffusion model, and the authors note that the CLA module does not handle dynamic lighting changes well.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same progressive-fusion recipe could generalize to other per-frame image edits—color grading, exposure adjustment, deflickering—by using a different base editor in place of the relighter, since nothing in PLF is lighting-specific beyond the motivation.
  • The paper's identification of appearance interpolation with lighting interpolation suggests a concrete failure mode the authors do not test: cast shadows and specular highlights move nonlinearly with light direction, so at intermediate $\lambda_t$ these regions may ghost or blur, and a region-adaptive schedule weighting shadow and specular areas differently could address it.
  • A testable extension is to modulate $\lambda_t$ per-pixel or per-region based on albedo: stable regions can be relighted quickly while high-frequency illumination effects are introduced later, which would preserve identity and motion more faithfully.
  • CLA's temporal average is a fixed blur over the whole clip; replacing it with a temporally shifted or tracked average could address the acknowledged dynamic-lighting limitation and enable relighting under moving light sources.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes Light-A-Video, a training-free pipeline for video relighting that combines a per-frame image relighting model (IC-Light) with a video diffusion model (AnimateDiff or CogVideoX). Two components are introduced: a Consistent Light Attention (CLA) module that stabilizes the generated light source via cross-frame attention, and a Progressive Light Fusion (PLF) strategy that blends the source video appearance with the relighted appearance during denoising under a linear-interpolation schedule. The paper reports improved temporal CLIP scores, lower motion preservation error, and better user-study rankings relative to baselines, and it includes qualitative results on video and foreground-sequence relighting.

Significance. If validated, the method would be a practical, training-free way to extend image relighting models to video, with easy integration into existing video diffusion backbones. The paper is honest about limitations (dynamic lighting changes are not handled) and provides qualitative evidence, a user study, and ablations over the main hyperparameters. However, the central quantitative claim of maintained relighted image quality rests on a self-referential FID metric, and the physical justification for the fusion equation is improperly applied. These issues leave the core contribution plausible but not yet established by the reported experiments.

major comments (2)
  1. [Section 5.1, Table 1]
  2. [Section 4.3, Eqs. (5)-(10)]
minor comments (5)
  1. [Eq. (2)]
  2. [Section 5.1]
  3. [Section 4.2, Eq. (8)]
  4. [Table 1]
  5. [Section 5.6]

Circularity Check

1 steps flagged · score 6.0 of 10

Relight quality is quantified as FID to the same per-frame IC-Light outputs that define the method's fusion target, making the central quality claim self-referential.

  1. self definitional [Section 5.1 (Evaluation metrics), used in Table 1 and Section 5.4]
    "The FID [42] score is then calculated between the results of each method and the frame-by-frame IC-Light results, serving as the metric for relight quality evaluation."

    The 'relighted image quality' claim is defined as FID distance to per-frame IC-Light outputs. But the proposed pipeline's relight target is itself produced by IC-Light: Eq. 10 builds the fusion target from I_r^t, the CLA-processed IC-Light relight appearance, and the denoising trajectory is progressively steered toward that target. Thus the metric measures how closely the output stays near the method's own input guidance. A trivial baseline that returns the unmodified per-frame IC-Light frames would score essentially zero FID, even though that is exactly the temporally flickery failure mode the paper sets out to remove; conversely, genuine temporal smoothing that moves appearance away from IC-Light's independent per-frame outputs is penalized.

full rationale

The CLA and PLF modules are empirical engineering designs rather than derivations from fitted parameters, and the paper does not invoke a uniqueness theorem to forbid alternatives. The PLF interpolation in Eq. 10 is not itself circular: Eq. 6 justifies additive appearance combination in linear radiance, while Eq. 10 is presented as an observed/heuristic modification after direct fusion 'results in suboptimal performance.' The weak point there is that I_v^t and I_r^t are decoded, non-linear, likely tone-mapped appearances rather than linear radiance, so the light-transport derivation does not directly apply; that is a correctness gap, not a self-referential reduction. The genuinely circular step is the evaluation design: 'Relighted Image Quality' is operationalized as FID to frame-by-frame IC-Light outputs, which are precisely the outputs the method's fusion target is constructed from and guided toward. This makes the reported FID (29.63 in Table 1) an expected consequence of the method's construction rather than independent evidence of quality. Additionally, hyperparameters such as gamma and the PLF decay schedule are selected using the same collected test set through the supplementary ablation, adding mild evaluation circularity. Because the temporal-consistency half of the central claim is supported by separate CLIP and motion-preservation metrics, the circularity is partial rather than total, so the score is 6 rather than higher.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the linearity of light transport, the quality of pretrained IC-Light and video diffusion models, and several hand-set hyperparameters that are tuned on the test set. No new physical entities are introduced.

free parameters (3)
  • gamma (CLA trade-off) = 0.5
    Controls the balance between per-frame self-attention and temporally averaged attention in Eq. 8. Selected via ablations on the test set (Supplementary Fig. 10a).
  • lambda_t schedule exponent k = 1 (lambda_t = 1 - t/Tm)
    Sets the decay of the relight target influence over denoising steps (Eq. 14). Chosen from comparisons of k=0.5,1,2 on the test set (Supplementary Fig. 10).
  • initial noise level / Tm = 50% noise, Tm=25 steps
    Source video is noised and denoised over 25 steps; this amount of noise is chosen empirically and no sensitivity analysis is reported.
assumptions (4)
  • domain assumption Light transport is linear: an image appearance equals a light transport matrix times the environment illumination, so superposed illuminations yield added appearances (Eq. 5-6).
    Invoked in Sections 3.2 and 4.3 to justify fusing source and relighted appearances; it is external physics (Debevec et al.) but the paper's use of convex interpolation goes beyond the stated additivity.
  • domain assumption IC-Light produces high-quality per-frame relighting with albedo preserved, and its quality limits the final output.
    The entire pipeline inherits IC-Light's behavior; the paper notes this in Section 5.6, and it is not re-validated for the VDM's predicted denoising targets.
  • domain assumption The video diffusion model's denoising trajectory can be steered by replacing the predicted noise-free target each step without destroying temporal coherence.
    Assumed in Section 4.3; this is the basis for PLF and is not proven for arbitrary blending weights.
  • ad hoc to paper The noise-free prediction at step t, plus the details compensation Delta_dm derived from the first step, is a faithful representation of the source video's content and motion.
    Details compensation is defined in Fig. 5 and Section 4.3 as the difference between the first-step prediction and the source video, assuming this difference captures only high-frequency detail loss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Light-A-Video: Training-free Video Relighting via Progressive Light Fusion." pith.science (2026). https://pith.science/paper/RUXYLZON

@misc{pith2026250208590,
  author       = {Pith},
  title        = {Pith review of: Light-A-Video: Training-free Video Relighting via Progressive Light Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RUXYLZON}},
  note         = {Machine review of arXiv:2502.08590}
}
read the original abstract

Recent advancements in image relighting models, driven by large-scale datasets and pre-trained diffusion models, have enabled the imposition of consistent lighting. However, video relighting still lags, primarily due to the excessive training costs and the scarcity of diverse, high-quality video relighting datasets. A simple application of image relighting models on a frame-by-frame basis leads to several issues: lighting source inconsistency and relighted appearance inconsistency, resulting in flickers in the generated videos. In this work, we propose Light-A-Video, a training-free approach to achieve temporally smooth video relighting. Adapted from image relighting models, Light-A-Video introduces two key techniques to enhance lighting consistency. First, we design a Consistent Light Attention (CLA) module, which enhances cross-frame interactions within the self-attention layers of the image relight model to stabilize the generation of the background lighting source. Second, leveraging the physical principle of light transport independence, we apply linear blending between the source video's appearance and the relighted appearance, using a Progressive Light Fusion (PLF) strategy to ensure smooth temporal transitions in illumination. Experiments show that Light-A-Video improves the temporal consistency of relighted video while maintaining the relighted image quality, ensuring coherent lighting transitions across frames. Project page: https://bujiazi.github.io/light-a-video.github.io/.

Figures

Figures reproduced from arXiv: 2502.08590 by the authors.

Figure 1
Figure 1. Training-free video relighting. Equipped with an image relighting model (e.g., IC-Light [62]) and a video diffusion model (e.g., CogVideoX [59] and AnimateDiff [13]), Light-A-Video enables training-free video relighting for given video sequences or foreground sequences. Abstract Recent advancements in image relighting models, driven by large-scale datasets and pre-trained diffusion models, have enabled the impositio… view at source ↗
Figure 2
Figure 2. Relighted frames of vanilla IC-Light and “IC-Light + CLA” . The line chart depicts the average optical flow inten￾sity between adjacent frames. Since IC-Light performs image re￾lighting based on each independent frame, its results show a no￾ticeable jitter between frames, especially in the generated back￾ground lighting. Conversely, the proposed CLA facilitates consis￾tent lighting generation by forcing interaction … view at source ↗
Figure 3
Figure 3. The pipeline of Light-A-Video. A source video is first noised and processed through the VDM for denoising across Tm steps. At each step, the predicted noise-free component with details compensation serves as the Consistent Target z v 0←t , inherently representing the VDM’s denoising direction. Consistent Light Attention infuses z v 0←t with unique lighting information, transforming it into the Relight Target z r 0←t… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Visualization of the PLF strategy. During the denois￾ing process of the VDM, the PLF strategy progressively replaces the original Consistent Target z v 0←t with the Fusion Target z˜0←t, guiding the denoising direction from vt to v˜t. Predicted ← Result without Detail C…
Figure 5
Figure 5. Figure 5: Visualization of the detail compensation. ∆dm records the difference between zˆ0←m and the source video in the first denoising step, which is used as a detail compensation com￾ponent for detail preservation in the consistent target. still exist between the predicted zˆ…
Figure 6
Figure 6. Figure 6: Qualitative comparison of baseline methods. Given a source video and guidance text prompt, Light-A-Video achieves high temporal consistency and fidelity to the light condition, outperforming other methods in avoiding flickering, jitter, and identity shifts. VDM used: A…
Figure 7
Figure 7. Figure 7: Text-conditioned video illumination modifying with background generation. Given a video foreground sequence and a text description of the target illumination, our method synthe￾sizes suitable backgrounds and harmonious illumination. 5.4. Quantitative Evaluation The qua…
Figure 9
Figure 9. Figure 9: Evolution of λt over time steps t for different PLF strategies. λt determines the proportion of the relight target mixed into the fusion target. A. Comprehensive Ablation Studies In this section, we conduct comprehensive ablation stud￾ies to explore the effects of the …
Figure 10
Figure 10. Figure 10: The relative effectiveness of different PLF strategy on Light-A-Video performance. (a) FID scores, (b) Temporal CLIP scores, and (c) Motion Preservation scores are shown for four strategies: PLF with constant λ (λt ≡ 1), and PLF with k = 0.5, 1, 2. Lower FID/Motion Pr…
Figure 11
Figure 11. Figure 11: More results of Light-A-Video in foreground sequences relighting with background generation [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: More results of Light-A-Video in foreground sequences relighting with background generation [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: More results of Light-A-Video in video sequences relighting [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: More results of Light-A-Video in video sequences relighting [PITH_FULL_IMAGE:figures/full_fig_p017_14.png]
Figure 15
Figure 15. Figure 15: More results of Light-A-Video in video sequences relighting on CogVideoX [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LiveLight: Real-time Streaming Video Relighting with Interactive Control

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A diffusion-based system performs real-time, interactive video relighting by injecting multi-plane light irradiance conditions and streaming latent chunks.

  2. ANYPORTAL: Zero-Shot Consistent Video Background Replacement

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A training-free video background replacement pipeline that keeps the foreground pixel-consistent by projecting refined latents through a deterministic reparameterization.

  3. IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation

    cs.CV 2025-06 reject novelty 6.0 of 10

    A diffusion video model that jointly uses HDR lighting, relit frames, and 3D point tracks to relight videos from text prompts.

  4. VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation

    cs.CV 2025-02 conditional novelty 6.0 of 10

    VidCRAFT3 is a single image-to-video diffusion system that accepts camera, object, and lighting direction controls separately or jointly, trained in three stages with a new synthetic lighting dataset.

Reference graph

Works this paper leans on

66 extracted references · 28 canonical work pages · cited by 4 Pith papers

  1. [1]

    Shape, illumination, and reflectance from shading

    Jonathan T Barron and Jitendra Malik. Shape, illumination, and reflectance from shading. IEEE transactions on pattern analysis and machine intelligence , 37(8):1670–1687, 2014. 3

    work page 2014

  2. [2]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

    arXiv 2023

  3. [3]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22563–22575, 2023. 3

    2023

  4. [4]

    Broadway: Boost your text-to-video gen- eration model in a training-free way

    Jiazi Bu, Pengyang Ling, Pan Zhang, Tong Wu, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Dahua Lin, and Ji- aqi Wang. Broadway: Boost your text-to-video gen- eration model in a training-free way. arXiv preprint arXiv:2410.06241, 2024

    arXiv

  5. [5]

    Videocrafter1: Open diffusion models for high-quality video generation

    Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafter1: Open diffusion models for high-quality video generation. arXiv preprint arXiv:2310.19512, 2023. 3

    arXiv

  6. [6]

    Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models

    Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffu- sion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7310– 7320, 2024. 3

    2024

  7. [7]

    Livephoto: Real image animation with text-guided motion control

    Xi Chen, Zhiheng Liu, Mengting Chen, Yutong Feng, Yu Liu, Yujun Shen, and Hengshuang Zhao. Livephoto: Real image animation with text-guided motion control. In Eu- ropean Conference on Computer Vision , pages 475–491. Springer, 2025. 3

    work page 2025

  8. [8]

    Flatten: optical flow- guided attention for consistent text-to-video editing

    Yuren Cong, Mengmeng Xu, Christian Simon, Shoufa Chen, Jiawei Ren, Yanping Xie, Juan-Manuel Perez-Rua, Bodo Rosenhahn, Tao Xiang, and Sen He. Flatten: optical flow- guided attention for consistent text-to-video editing. arXiv preprint arXiv:2310.05922, 2023. 3

    arXiv

Show all 66 references
  1. [9]

    Acquiring the reflectance field of a human face

    Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. Acquiring the reflectance field of a human face. In Proceedings of the 27th annual conference on Computer graphics and interac- tive techniques, pages 145–156, 2000. 4

    2000

  2. [10]

    Flashtex: Fast relightable mesh texturing with lightcontrolnet

    Kangle Deng, Timothy Omernick, Alexander Weiss, Deva Ramanan, Jun-Yan Zhu, Tinghui Zhou, and Maneesh Agrawala. Flashtex: Fast relightable mesh texturing with lightcontrolnet. In European Conference on Computer Vi- sion, pages 90–107. Springer, 2025. 2, 3

    2025

  3. [11]

    Tokenflow: Consistent diffusion features for consistent video editing

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. arXiv preprint arXiv:2307.10373, 2023. 3

    2023 arXiv

  4. [12]

    I2v-adapter: A general image- to-video adapter for video diffusion models

    Xun Guo, Mingwu Zheng, Liang Hou, Yuan Gao, Yufan Deng, Chongyang Ma, Weiming Hu, Zhengjun Zha, Haibin Huang, Pengfei Wan, et al. I2v-adapter: A general image- to-video adapter for video diffusion models. arXiv preprint arXiv:2312.16693, 2023. 3

    2023 arXiv

  5. [13]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 1, 3, 6

    2023 arXiv

  6. [14]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In European Conference on Computer Vision, pages 330–348. Springer, 2025. 3

    2025

  7. [15]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  8. [16]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022. 3

    2022 arXiv

  9. [17]

    Towards high fidelity face relight- ing with realistic shadows

    Andrew Hou, Ze Zhang, Michel Sarkis, Ning Bi, Yiying Tong, and Xiaoming Liu. Towards high fidelity face relight- ing with realistic shadows. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14719–14728, 2021. 2, 3

    2021

  10. [18]

    Videocontrolnet: A motion-guided video-to-video translation framework by using diffusion model with controlnet

    Zhihao Hu and Dong Xu. Videocontrolnet: A motion-guided video-to-video translation framework by using diffusion model with controlnet. arXiv preprint arXiv:2307.14073 ,

    arXiv

  11. [19]

    Neural gaffer: Relighting any object via diffusion

    Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neural gaffer: Relighting any object via diffusion. arXiv preprint arXiv:2406.07520, 2024. 2, 3

    2024 arXiv

  12. [20]

    Rave: Randomized noise shuf- fling for fast and consistent video editing with diffusion mod- els

    Ozgur Kara, Bariscan Kurtkaya, Hidir Yesiltepe, James M Rehg, and Pinar Yanardag. Rave: Randomized noise shuf- fling for fast and consistent video editing with diffusion mod- els. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 6507–6516,

  13. [21]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565–26577, 2022. 6

    2022

  14. [22]

    Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting

    Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switchlight: Co-design of physics- driven architecture and pre-training framework for human portrait relighting. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition,...

    2024

  15. [23]

    Diffusehigh: Training-free progressive high- resolution image synthesis through structure guidance.arXiv preprint arXiv:2406.18459, 2024

    Younghyun Kim, Geunmin Hwang, Junyu Zhang, and Eun- byung Park. Diffusehigh: Training-free progressive high- resolution image synthesis through structure guidance.arXiv preprint arXiv:2406.18459, 2024. 5

    2024 arXiv

  16. [24]

    Lightit: Illumination modeling and control for diffusion models

    Peter Kocsis, Julien Philip, Kalyan Sunkavalli, Matthias Nießner, and Yannick Hold-Geoffroy. Lightit: Illumination modeling and control for diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9359–9369, 2024. 2

    2024

  17. [25]

    Anyv2v: A tuning-free framework for any video-to- video editing tasks

    Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. Anyv2v: A tuning-free framework for any video-to- video editing tasks. arXiv preprint arXiv:2403.14468, 2024. 6, 7

    2024 arXiv

  18. [26]

    Motionclone: Training-free motion cloning for controllable video generation

    Pengyang Ling, Jiazi Bu, Pan Zhang, Xiaoyi Dong, Yuhang Zang, Tong Wu, Huaian Chen, Jiaqi Wang, and Yi Jin. Motionclone: Training-free motion cloning for controllable video generation. arXiv preprint arXiv:2406.05338, 2024. 3

    2024 arXiv

  19. [27]

    Flow matching for generative mod- eling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling. arXiv preprint arXiv:2210.02747, 2022. 6

    2022 arXiv

  20. [28]

    Video-p2p: Video editing with cross-attention control

    Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8599–8608, 2024. 3

    2024

  21. [29]

    Follow-your-click: Open- domain regional image animation via short prompts

    Yue Ma, Yingqing He, Hongfa Wang, Andong Wang, Chenyang Qi, Chengfei Cai, Xiu Li, Zhifeng Li, Heung- Yeung Shum, Wei Liu, et al. Follow-your-click: Open- domain regional image animation via short prompts. arXiv preprint arXiv:2403.08268, 2024. 3

    2024 arXiv

  22. [30]

    Sdedit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equa- tions. arXiv preprint arXiv:2108.01073, 2021. 3, 6

    2021 arXiv

  23. [31]

    Null-text inversion for editing real im- ages using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real im- ages using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6038–6047, 2023. 3

    2023

  24. [32]

    Learning physics-guided face re- lighting under directional light

    Thomas Nestmeyer, Jean-Franc ¸ois Lalonde, Iain Matthews, and Andreas Lehrmann. Learning physics-guided face re- lighting under directional light. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5124–5133, 2020. 2, 3

    2020

  25. [33]

    Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model

    Muyao Niu, Xiaodong Cun, Xintao Wang, Yong Zhang, Ying Shan, and Yinqiang Zheng. Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model. In European Con- ference on Computer Vision, pages 111–128. Springer, 2025. 3

    2025

  26. [34]

    Total relighting: learning to relight portraits for background replacement

    Rohit Pandey, Sergio Orts-Escolano, Chloe Legendre, Chris- tian Haene, Sofien Bouaziz, Christoph Rhemann, Paul E De- bevec, and Sean Ryan Fanello. Total relighting: learning to relight portraits for background replacement. ACM Trans. Graph., 40(4):43–1, 2021. 2, 3

    2021

  27. [35]

    pixabay. pixabay. https://pixabay.com/videos/,

  28. [36]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017. 6

    2017 arXiv

  29. [37]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15932–15942, 2023. 3

    2023

  30. [38]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  31. [39]

    Relightful harmonization: Lighting-aware portrait background replacement

    Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, and He Zhang. Relightful harmonization: Lighting-aware portrait background replacement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6...

    2024

  32. [40]

    Semantic im- age inversion and editing using rectified stochastic differen- tial equations

    Litu Rout, Yujia Chen, Nataniel Ruiz, Constantine Carama- nis, Sanjay Shakkottai, and Wen-Sheng Chu. Semantic im- age inversion and editing using rectified stochastic differen- tial equations. arXiv preprint arXiv:2410.10792, 2024. 5

    arXiv

  33. [41]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512, 2022. 3

    2022 arXiv

  34. [42]

    pytorch-fid: FID Score for PyTorch

    Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. https://github.com/mseitzer/pytorch-fid ,

  35. [43]

    Sfsnet: Learning shape, reflectance and illuminance of facesin the wild’

    Soumyadip Sengupta, Angjoo Kanazawa, Carlos D Castillo, and David W Jacobs. Sfsnet: Learning shape, reflectance and illuminance of facesin the wild’. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 6296–6305, 2018. 3

    2018

  36. [44]

    A light stage on every desk

    Soumyadip Sengupta, Brian Curless, Ira Kemelmacher- Shlizerman, and Steven M Seitz. A light stage on every desk. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2420–2429, 2021. 2, 3

    2021

  37. [45]

    Style transfer for headshot por- traits

    YiChang Shih, Sylvain Paris, Connelly Barnes, William T Freeman, and Fr´edo Durand. Style transfer for headshot por- traits. 2014. 3

    2014

  38. [46]

    Portrait lighting trans- fer using a mass transport approach

    Zhixin Shu, Sunil Hadap, Eli Shechtman, Kalyan Sunkavalli, Sylvain Paris, and Dimitris Samaras. Portrait lighting trans- fer using a mass transport approach. ACM Transactions on Graphics (TOG), 36(4):1, 2017. 3

    2017

  39. [47]

    Single image portrait relighting

    Tiancheng Sun, Jonathan T Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul Debevec, and Ravi Ramamoorthi. Single image portrait relighting. ACM Transactions on Graphics (TOG) , 38(4):1–12, 2019. 2, 3

    2019

  40. [48]

    Emergent correspondence from image diffusion

    Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. Advances in Neural Information Pro- cessing Systems, 36:1363–1389, 2023. 3

    2023

  41. [49]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part II 16, pages 402–419. Springer,

    2020

  42. [50]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 3

    2023 arXiv

  43. [51]

    Cove: Unleashing the diffusion feature correspondence for consistent video editing

    Jiangshan Wang, Yue Ma, Jiayi Guo, Yicheng Xiao, Gao Huang, and Xiu Li. Cove: Unleashing the diffusion feature correspondence for consistent video editing. arXiv preprint arXiv:2406.08850, 2024. 3

    2024 arXiv

  44. [52]

    Zero-shot video editing using off-the-shelf image diffusion models

    Wen Wang, Yan Jiang, Kangyang Xie, Zide Liu, Hao Chen, Yue Cao, Xinlong Wang, and Chunhua Shen. Zero-shot video editing using off-the-shelf image diffusion models. arXiv preprint arXiv:2303.17599, 2023. 3

    2023 arXiv

  45. [53]

    Lavie: High-quality video gener- ation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video gener- ation with cascaded latent diffusion models. arXiv preprint arXiv:2309.15103, 2023. 3

    2023 arXiv

  46. [54]

    Sunstage: Portrait reconstruction and re- lighting using the sun as a light stage

    Yifan Wang, Aleksander Holynski, Xiuming Zhang, and Xuaner Zhang. Sunstage: Portrait reconstruction and re- lighting using the sun as a light stage. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20792–20802, 2023. 2, 3

    2023

  47. [55]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  48. [56]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In Eu- ropean Conference on Computer Vision , pages 399–417. Springer, 2025. 3

    2025

  49. [57]

    Rerender a video: Zero-shot text-guided video-to-video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender a video: Zero-shot text-guided video-to-video translation. In SIGGRAPH Asia 2023 Conference Papers , pages 1–11, 2023. 3

    2023

  50. [58]

    Fresco: Spatial-temporal correspondence for zero-shot video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Fresco: Spatial-temporal correspondence for zero-shot video translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8703– 8712, 2024. 3

    2024

  51. [59]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 1, 3

    2024 arXiv

  52. [60]

    Dilightnet: Fine-grained light- ing control for diffusion-based image generation

    Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained light- ing control for diffusion-based image generation. In ACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024. 2, 3

    2024

  53. [61]

    Show-1: Marrying pixel and latent diffusion models for text-to-video generation

    David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. Show-1: Marrying pixel and latent diffusion models for text-to-video generation. International Journal of Com- puter Vision, pages 1–15, 2024. 3

    2024

  54. [62]

    Scal- ing in-the-wild training for diffusion-based illumination har- monization and editing by imposing consistent light trans- port

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scal- ing in-the-wild training for diffusion-based illumination har- monization and editing by imposing consistent light trans- port. In The Thirteenth International Conference on Learn- ing Representations, 2025. 1, 2, 3, 4, 6, 7

    2025

  55. [63]

    Pia: Your personalized image animator via plug-and-play modules in text-to-image models

    Yiming Zhang, Zhening Xing, Yanhong Zeng, Youqing Fang, and Kai Chen. Pia: Your personalized image animator via plug-and-play modules in text-to-image models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7747–7756, 2024. 3

    2024

  56. [64]

    Lumis- culpt: A consistency lighting control network for video gen- eration

    Yuxin Zhang, Dandan Zheng, Biao Gong, Jingdong Chen, Ming Yang, Weiming Dong, and Changsheng Xu. Lumis- culpt: A consistency lighting control network for video gen- eration. arXiv preprint arXiv:2410.22979, 2024. 2, 3

    2024 arXiv

  57. [65]

    Deep single-image portrait relighting

    Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, and David W Jacobs. Deep single-image portrait relighting. In Proceed- ings of the IEEE/CVF international conference on computer vision, pages 7194–7202, 2019. 2, 3

    2019

  58. [66]

    Re- lightable neural human assets from multi-view gradient il- luminations

    Taotao Zhou, Kai He, Di Wu, Teng Xu, Qixuan Zhang, Kuix- iang Shao, Wenzheng Chen, Lan Xu, and Jingyi Yu. Re- lightable neural human assets from multi-view gradient il- luminations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 43...

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.