Pith. sign in

REVIEW 4 major objections 5 minor 74 references

DreamColour: Controllable Video Colour Editing without Training

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training-free video colour editor propagates point-and-click recolouring across all frames using a pre-trained image-to-video diffusion model.

desk verdict A credible training-free video colour editing system whose 'any-frame' claim needs a junction-consistency check and a more rigorous quantitative evaluation. read the letter →

arxiv 2412.05180 v1 pith:ESCEVJZE submitted 2024-12-06 cs.CV

Chaitat Utintu , Pinaki Nath Chowdhury , Aneeshan Sain , Subhadeep Koley , Ayan Kumar Bhunia , Yi-Zhe Song This is my paper · ORCID
classification cs.CV
keywords videocoloureditingtraining-freediffusionmodelsDDIMinversionspatio-temporalfeatureinjectionbidirectionalpropagationinstancesegmentationimage-to-videogeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DreamColour argues that precise video colour editing does not need training or per-video fine-tuning. The paper splits the task into a spatial stage, where users click colours on a $16\times16$ grid and automatic instance segmentation keeps the colour inside the chosen object, and a temporal stage, where a pre-trained image-to-video diffusion model carries the new colour through every frame. Bidirectional propagation, built on deterministic diffusion inversion, injected spatio-temporal features, and motion-aware blending, is claimed to match or exceed existing text-based editing methods while preserving sharp boundaries and temporal stability. The upshot is that professional-quality recolouring becomes a point-and-click operation that runs on ordinary hardware with no training delays.

What carries the argument

The load-bearing mechanism is bidirectional colour propagation through a pre-trained image-to-video (I2V) diffusion model. Deterministic inversion (DDIM inversion) turns the source video into latent noise; a parallel sampling pass, conditioned on the recoloured frame, receives convolutional and attention features from the inverted pass so that the regenerated video keeps the original motion and structure. Spatial containment is provided by a $16\times16$ colour-hint grid paired with automatic instance-segmentation masks that refine the hints sent to a colourisation transformer, and automatically generated text descriptions of object colour and scene act as conditioning prompts during resampling. A linear blend operator merges forward- and backward-propagated results when several frames are edited, and motion-aware feature blending adapts to fast versus subtle movement.

What would settle it

Edit a frame in several videos containing thin, fast-moving objects such as a swinging rope or a fluttering ribbon, then measure whether the chosen colour stays inside the object's mask across all frames; if colour consistently bleeds outside the mask or flickers in frames far from the edited frame, the propagation assumption is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the motion prior already stored in a pre-trained image-to-video diffusion model can be reused as-is for colour propagation. The user's interaction is confined to one key frame: choose RGB colours on a grid, let automatic instance-segmentation masks stop colours from bleeding into neighbouring regions, and let a colourisation transformer paint the masked area. The source video is then inverted into latent noise that encodes its motion, and a second sampling pass conditioned on the recoloured frame receives the source video's convolutional, spatial-attention, and temporal-attention features, so the model re-generates the same dynamics with the new colours. Because the same mechanism works when the video is reversed, any frame can serve as the edit anchor and edits propagate both forwards and backwards; two anchors can be blended with a weighted sum and re-sampled with a text prompt to produce gradual colour transitions. The paper's main point is that all of this happens without updating any model parameters.

Load-bearing premise

The method assumes that a pre-trained video-generation model can faithfully re-create the original video's motion and appearance from its compressed internal representations and injected features even when the conditioning frame has been recoloured, and that this works in both forward and backward time directions.

Editorial extensions

If this is right

  • A user can recolour any object in a video by clicking a few points on one frame, and the change propagates automatically to all other frames.
  • Edits can be anchored at intermediate frames, not just the first, because the video is split into two subsequences, recoloured in each direction, and reassembled.
  • Multiple frames can receive different colours and the model blends them into a smooth transition across the video using a weighted sum and a text prompt.
  • Because the pipeline uses only pre-trained modules and classical operations, it requires no training, no fine-tuning per video, and no specialised hardware.
  • Reported numbers on a standard video benchmark place the method above the text-based baselines it is compared with on fidelity, structural similarity, colourfulness, and temporal consistency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that backward propagation quality may limit usable clip length; a natural stress-test is measuring colour drift in frames far from the anchor on longer sequences.
  • The same spatio-temporal injection recipe would plausibly transfer to other appearance edits, such as texture, lighting, or object identity, since the mechanism propagates any first-frame change rather than colour specifically.
  • The interface suggests an accessible consumer workflow in which the segmentation model and the image-to-video model are swapped for newer off-the-shelf versions without retraining the editing logic.
  • A testable extension is to quantify how often colour bleeds outside the intended object on thin or motion-blurred regions, since the paper reports these as failure cases.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. DreamColour proposes a training-free pipeline for video colour editing. In the spatial stage, user colour hints on a 16x16 grid are combined with SAM2 instance masks and the UniColor colourisation model to produce an edited key frame; in the temporal stage, DDIM inversion of the source video and spatio-temporal feature injection into the pre-trained I2VGen-XL model propagate the edit forward and backward, with BLIP-2 visual-question-answering prompts used as semantic guidance. The paper claims that this modular, training-free framework matches or exceeds state-of-the-art video editing methods while supporting single-region, multi-region, intermediate-frame, and multi-frame colour-blending scenarios.

Significance. If the claims are substantiated, the work would offer a practical and accessible interface for video recolouring, and its composition of pre-trained components (SAM2, UniColor, BLIP-2, I2VGen-XL) is a useful demonstration of training-free editing. The paper honestly discloses several limitations in Sec. 5 and includes ablations for the SAM2 mask, BLIP-2 prompts, backward propagation, and initial latent index. However, the quantitative evidence in the supplementary material is not currently interpretable as a comparison of colour-editing quality, the intermediate-frame scenario lacks any junction-consistency evaluation despite acknowledged artefacts in backward propagation, and several control parameters are left unspecified. The contribution is therefore plausible but not yet convincingly established at the level claimed in the abstract.

major comments (4)
  1. [Supplementary Sec. B, Table S1] The quantitative evaluation is not specified well enough to support the 'matches or exceeds SOTA' claim. FID, LPIPS, PSNR, and SSIM are reported on DAVIS, but the reference ('ground truth') used for these metrics is never defined; if the original unedited DAVIS frames are used, any method that changes the requested object colours is penalised for exactly the edit the user asked for, making the numbers uninterpretable as a colour-editing quality measure. Please specify the protocol completely, report the number of videos and edits, give per-video statistics, and either use mask-aware metrics (colour accuracy inside the edited mask, background preservation outside it) or a user study.
  2. [Sec. 3.3.2 and Fig. 10] Intermediate-frame editing concatenates two independently generated segments at the edited frame m, but no mechanism enforces consistency of motion, appearance, or colour across that junction, and no evaluation of the junction is provided. The paper's own ablation in Fig. 10 reports 'minor shifts in colour accuracy and occasional artefacts' for backward propagation, so the any-frame editing claim needs direct evidence that the transition from backward to forward generation is temporally coherent. Please add junction-specific temporal-consistency metrics or side-by-side visual comparisons around the edited frame.
  3. [Sec. 3.3.1 and Supplementary Sec. C] The spatio-temporal feature injection is controlled by thresholds tau_conv, tau_sa, and tau_ta, but their numerical values are never given, and no sensitivity analysis is reported for them. Only the initial latent index tau_idx is ablated (Supplementary Sec. C). Because these thresholds determine how strongly the source video's structure and motion are imposed on the edited generation, omitting them prevents reproduction and leaves unclear whether the reported results depend on fine-tuned hyperparameters despite the 'training-free' claim.
  4. [Abstract, Sec. 1, and Sec. 3.3.3] The headline contribution of 'motion-aware blending' is not actually described as motion-aware in the method. Sec. 3.3.3 uses a proximity-based weighted sum (a 'linear blend operator') followed by DDIM inversion and resampling with a text prompt, with no explicit motion estimation or motion-dependent weighting. Please either clarify how motion is incorporated into the blending or rename the component so that the described procedure matches the claimed mechanism.
minor comments (5)
  1. [Sec. 3.1] There is a typo in 'f orwardand backward diffusion', and the notation for the noisy latent is inconsistent: x_t is defined with sqrt(alpha_t) while z_t is later defined with alpha_t; please unify the notation.
  2. [Sec. 3.2.1] The '20-pixel Euclidean distance' exclusion threshold and the relation between the 16x16 colour grid and SLIC superpixels need clarification: are the exclusion distances computed in the original image resolution or in the grid resolution?
  3. [Fig. 10 caption] The caption repeats 'propagate forward (frame1 to 16)' and 'backward propagate (frame16 to 1)' twice; the duplicated text should be removed.
  4. [Sec. 3.3.1] The list of negative prompts ends with '...'; please provide the full list used in the experiments, as prompt wording can affect the results.
  5. [Supplementary Sec. B] The DAVIS dataset name is typeset with an unusual space ('DA VIS') in several places; please correct this and also report implementation details such as resolution, number of sampled frames, DDIM steps, and inference time.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the pipeline composes external pre-trained models, standard DDIM inversion, and explicit blending operations; no fitted parameter or self-citation back-feeds into the result.

full rationale

DreamColour's derivation chain is a composition of well-specified stages: user colour hints and SAM2 masks are passed to UniColor to produce an edited first frame; then DDIM inversion of the source video plus spatio-temporal feature injection is used with the external I2VGen-XL model to regenerate the video from that edited frame; intermediate-frame editing applies the same first-frame procedure separately on forward and backward subsequences and concatenates the results; multi-frame editing uses an explicit weighted sum and a resampling prompt. Nothing in this chain estimates a parameter from the target output or from the evaluation set. The few tunable quantities, such as the feature-injection thresholds tau_conv, tau_sa, tau_ta and the initial latent index tau_idx, are operation choices whose effects are ablated in supplementary Sec. C; they are not fitted to the videos being edited, so the generated colours are not forced by construction. The paper's claim that the approach 'matches or exceeds state-of-the-art methods' is an empirical claim evaluated against DAVIS with standard metrics (FID, LPIPS, colourfulness, CDC, PSNR, SSIM) and external baselines, not a derivation from a fitted quantity. All load-bearing references point to external works: UniColor [28], SAM2 [50], I2VGen-XL [72], BLIP-2 [36], DDIM [57], and AnyV2V [34]; there is no self-citation invoked as a uniqueness theorem or as justification of the central premise. The limitations the paper itself reports, such as colour bleeding on thin or motion-blurred regions and minor shifts in backward propagation, are correctness and robustness caveats rather than signs that an output was reinserted as an input. Consequently, no circular step can be exhibited from the paper's equations or references, and the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework adds no new physical or learned entity; it assembles existing pre-trained models. The central claim draws its power from assumptions about those models' behaviour rather than from new first-principles content. The hand-set thresholds, hint exclusion distance, prompt wording, and initial latent index are the main free knobs.

free parameters (4)
  • Feature injection thresholds (tau_conv, tau_sa, tau_ta) = not reported
    Sec. 3.3.1 says features are injected only within 'early steps' controlled by these thresholds, but no values are given; the output's structural fidelity and temporal consistency depend on them.
  • Initial latent index (tau_idx) = 3 or 9 (best in ablation)
    Supplementary Sec. C ablates tau_idx in {0,3,9,20} and shows colour propagation quality depends strongly on the choice; the paper does not fix a single default.
  • Hint boundary exclusion distance = 20 pixels
    Sec. 3.2.1 excludes hint points within 20 pixels of the SAM2 mask boundary to prevent leakage; this hand-set distance affects edge sharpness and was not swept.
  • Negative prompt wording for BLIP-2 guidance = 'desaturated colour, greyish, unrealistic'
    Sec. 3.3.1 uses hand-crafted negative prompts to suppress artefacts; prompt choice is a design parameter that influences output quality.
assumptions (4)
  • standard math DDIM inversion and sampling equations correctly approximate the reverse diffusion process of I2VGen-XL.
    Invoked in Sec. 3.1 and Sec. 3.3.1 as the backbone of both the inversion and the generation passes.
  • domain assumption The external pre-trained models (SAM2, UniColor, I2VGen-XL, BLIP-2) have the capabilities they are credited with: point-based segmentation, region colourisation, image-to-video generation, and visual question answering.
    The entire pipeline in Sec. 3.2 and 3.3 composes these off-the-shelf models; if any underperforms on a given video, the pipeline inherits the failure.
  • domain assumption The I2V model's latent space encodes 'inherent video motion patterns' that remain usable when the conditioning frame is recoloured and when the video is reversed.
    Sec. 3.3.2 and the backward-propagation ablation (Fig. 10) rest on this; the paper's own results show artefacts in early frames.
  • domain assumption SAM2 point prompts produce masks accurate enough for colour hints, including multiple regions using positive and negative prompts.
    Sec. 3.2.2 relies on SAM2's dual-prompt segmentation to separate nearby regions; the paper notes it does not achieve perfect segmentation for extremely close colour hints.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DreamColour: Controllable Video Colour Editing without Training." pith.science (2026). https://pith.science/paper/ESCEVJZE

@misc{pith2026241205180,
  author       = {Pith},
  title        = {Pith review of: DreamColour: Controllable Video Colour Editing without Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ESCEVJZE}},
  note         = {Machine review of arXiv:2412.05180}
}
read the original abstract

Video colour editing is a crucial task for content creation, yet existing solutions either require painstaking frame-by-frame manipulation or produce unrealistic results with temporal artefacts. We present a practical, training-free framework that makes precise video colour editing accessible through an intuitive interface while maintaining professional-quality output. Our key insight is that by decoupling spatial and temporal aspects of colour editing, we can better align with users' natural workflow -- allowing them to focus on precise colour selection in key frames before automatically propagating changes across time. We achieve this through a novel technical framework that combines: (i) a simple point-and-click interface merging grid-based colour selection with automatic instance segmentation for precise spatial control, (ii) bidirectional colour propagation that leverages inherent video motion patterns, and (iii) motion-aware blending that ensures smooth transitions even with complex object movements. Through extensive evaluation on diverse scenarios, we demonstrate that our approach matches or exceeds state-of-the-art methods while eliminating the need for training or specialized hardware, making professional-quality video colour editing accessible to everyone.

Figures

Figures reproduced from arXiv: 2412.05180 by the authors.

Figure 1
Figure 1. Our training-free framework enables intuitive video colour editing in two stages. First, users simply select colours from a [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our single-region colour editing begins with greyscale [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Multi-region colour editing pipeline using SAM2 and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The primary pathway (top) performs DDIM inversion on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: To edit the mth intermediate frame, the video is divided into forward (Im → In) and backward (Im → I1) subsequences. First-frame colour editing is then applied separately in each direc￾tion, with colour changes propagated through denoising steps. The edited segments ar…
Figure 6
Figure 6. Figure 6: Smooth colour transition using intermediate frame edit [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Comparison of our method with FateZero [ [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 11
Figure 11. Figure 11: For smooth colour transitions across multiple edited [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Limitations of our proposed method, especially of thin [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]
Figure 10
Figure 10. Figure 10: Ablation study on the flexibility of pre-trained video [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 60 canonical work pages

  1. [1]

    SLIC Superpixels Compared to State-of-the-Art Superpixel Methods

    Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine S¨usstrunk. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods. TPAMI,

    work page

  2. [2]

    Blended Diffusion for Text-driven Editing of Natural Images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended Diffusion for Text-driven Editing of Natural Images. In CVPR, 2022. 3

    work page 2022

  3. [3]

    Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models. In CVPR, 2023. 2

    2023

  4. [4]

    Versatile Vi- sion Foundation Model for Image and Video Colorization

    Vukasin Bozic, Abdelaziz Djelouah, Yang Zhang, Radu Tim- ofte, Markus Gross, and Christopher Schroers. Versatile Vi- sion Foundation Model for Image and Video Colorization. In SIGGRAPH, 2024. 2

    work page 2024

  5. [5]

    LEDITS++: Limitless Image Editing us- ing Text-to-Image Models

    Manuel Brack, Felix Friedrich, Katharina Kornmeier, Linoy Tsaban, Patrick Schramowski, Kristian Kersting, and Apolin´ario Passos. LEDITS++: Limitless Image Editing us- ing Text-to-Image Models. In CVPR, 2024. 12, 13

    work page 2024

  6. [6]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structPix2Pix: Learning to Follow Image Editing Instruc- tions. In CVPR, 2023. 12, 13

    work page 2023

  7. [7]

    DYffusion: A Dynamics-informed Diffusion Model for Spa- tiotemporal Forecasting

    Salva R ¨uhling Cachay, Bo Zhao, Hailey Joren, and Rose Yu. DYffusion: A Dynamics-informed Diffusion Model for Spa- tiotemporal Forecasting. In NeurIPS, 2023. 6

    work page 2023

  8. [8]

    The Animation Transformer: Visual Correspondence via Segment Matching

    Evan Casey, P ´erez V´ıctor, Zhuoru Li, Harry Teitelman, Nick Boyajian, Tim Pulver, Mike Manh, and William Grisaitis. The Animation Transformer: Visual Correspondence via Segment Matching. In ICCV, 2021. 2

    work page 2021

Show all 74 references
  1. [9]

    Duygu Ceylan, Chun-Hao Paul Huang, and Niloy J. Mitra. Pix2Video: Video Editing using Image Diffusion. In ICCV,

  2. [10]

    L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors

    Zheng Chang, Shuchen Weng, Peixuan Zhang, Yu Li, Si Li, and Boxin Shi. L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors. In NeurIPS,

  3. [11]

    L-CoIns: Language-based Colorization with Instance Awareness

    Zheng Chang, Shuchen Weng, Peixuan Zhang, Yu Li, Si Li, and Boxin Shi. L-CoIns: Language-based Colorization with Instance Awareness. In CVPR, 2023. 4

    2023

  4. [12]

    Language-Based Image Editing with Recur- rent Attentive Models

    Jianbo Chen, Yelong Shen, Jianfeng Gao, Jingjing Liu, and Xiaodong Liu. Language-Based Image Editing with Recur- rent Attentive Models. In CVPR, 2018. 2

    2018

  5. [13]

    Video-Story Composition via Plot Analysis

    Jinsoo Choi, Tae-Hyun Oh, and In So Kweon. Video-Story Composition via Plot Analysis. In CVPR, 2016. 2

    2016

  6. [14]

    ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models

    Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models. In ICCV, 2021. 3

    2021

  7. [15]

    Automatic Controllable Colorization via Imagination

    Xiaoyan Cong, Yue Wu, Qifeng Chen, and Chenyang Lei. Automatic Controllable Colorization via Imagination. In CVPR, 2024. 1

    2024

  8. [16]

    Paint Bucket Coloriza- tion Using Anime Character Color Design Sheets

    Yuekun Dai, Qinyue Li, Shangchen Zhou, Yihang Luo, Chongyi Li, and Chen Change Loy. Paint Bucket Coloriza- tion Using Anime Character Color Design Sheets. arXiv preprint arXiv:19424v1, 2024. 2

    2024

  9. [17]

    Diffusion Models Beat GANs on Image Synthesis

    Prafulla Dhariwal and Alex Nichol. Diffusion Models Beat GANs on Image Synthesis. In NeurIPS, 2021. 2, 3

    2021

  10. [18]

    Efros, and Aleksander Holynski

    Dave Epstein, Allan Jabri, Ben Poole, Alexei A. Efros, and Aleksander Holynski. Diffusion Self-Guidance for Control- lable Image Generation. In NeurIPS, 2023. 3

    2023

  11. [19]

    TokenFlow: Consistent Diffusion Features for Consistent Video Editing

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. TokenFlow: Consistent Diffusion Features for Consistent Video Editing. In ICLR, 2024. 3

    2024

  12. [20]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Networks. In NeurIPS, 2014. 2

    2014

  13. [21]

    Measuring colorfulness in natural images

    David Hasler and Sabine S ¨usstrunk. Measuring colorfulness in natural images. In IS&T/SPIE Electronic Imaging, 2003. 12

    2003

  14. [22]

    Prompt-to-Prompt Image Editing with Cross Attention Control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-Prompt Image Editing with Cross Attention Control. In ICLR, 2023. 3

    2023

  15. [23]

    GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In NeurIPS, 2017. 12

    2017

  16. [24]

    Denoising Dif- fusion Probabilistic Models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Dif- fusion Probabilistic Models. In NeurIPS, 2020. 2, 3

    2020

  17. [25]

    Kingma, Ben Poole, Mohammad Norouzi, David J

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Sal- imans. Imagen Video: High Definition Video Generation with Diffusion Models. arXiv preprint arXiv:2210.02303 ,

    arXiv

  18. [26]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video Dif- fusion Models. arXiv preprint arXiv:2204.03458, 2022. 1, 2

    2022 arXiv

  19. [27]

    Learning Where to Cut from Edited Videos

    Yuzhong Huang, Xue Bai, Oliver Wang, Fabian Caba, and Aseem Agarwala. Learning Where to Cut from Edited Videos. In ICCVW, 2021. 2

    2021

  20. [28]

    UniColor: A Unified Framework for Multi-Modal Colorization with Transformer

    Zhitong Huang, Nanxuan Zhao, and Jing Liao. UniColor: A Unified Framework for Multi-Modal Colorization with Transformer. ACM-TOG, 2022. 3, 4, 7

    2022

  21. [29]

    LVCD: Reference-based Lineart Video Colorization with Diffusion Models

    Zhitong Huang, Mohan Zhang, and Jing Liao. LVCD: Reference-based Lineart Video Colorization with Diffusion Models. arXiv preprint arXiv:2409.12960, 2024. 2

    2024 arXiv

  22. [30]

    DeepRemaster: Tem- poral Source-Reference Attention Networks for Comprehen- sive Video Enhancement

    Satoshi Iizuka and Edgar Simo-Serra. DeepRemaster: Tem- poral Source-Reference Attention Networks for Comprehen- sive Video Enhancement. In SIGGRAPH, 2019. 2

    2019

  23. [31]

    Imagic: Text-Based Real Image Editing with Diffusion Models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-Based Real Image Editing with Diffusion Models. In CVPR, 2023. 3

    2023

  24. [32]

    Text2Video-Zero: Text- to-Image Diffusion Models are Zero-Shot Video Generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2Video-Zero: Text- to-Image Diffusion Models are Zero-Shot Video Generators. In ICCV, 2023. 3

    2023

  25. [33]

    Auto-Encoding Vari- ational Bayes

    Diederik P Kingma and Max Welling. Auto-Encoding Vari- ational Bayes. arXiv preprint arXiv:1312.6114, 2013. 2

    2013 arXiv

  26. [34]

    AnyV2V: A Tuning-Free Framework For Any Video- to-Video Editing Tasks

    Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. AnyV2V: A Tuning-Free Framework For Any Video- to-Video Editing Tasks. TMLR, 2024. 2, 6, 12, 13 9

    2024

  27. [35]

    PopStage: The Genera- tion of Stage Cross-Editing Video based on Spatio-Temporal Matching

    Dawon Lee, Jung Eun Yoo, Kyungmin Cho, Bumki Kim, Gyeonghun Im, and Junyong Noh. PopStage: The Genera- tion of Stage Cross-Editing Video based on Spatio-Temporal Matching. ACM-TOG, 2022. 2

    2022

  28. [36]

    BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. arXiv preprint arXiv:2301.12597, 2023. 3, 5, 6, 14

    2023 arXiv

  29. [37]

    VidToMe: Video Token Merging for Zero-Shot Video Edit- ing

    Xirui Li, Chao Ma, Xiaokang Yang, and Ming-Hsuan Yang. VidToMe: Video Token Merging for Zero-Shot Video Edit- ing. In CVPR, 2024. 12, 13

    2024

  30. [38]

    Video Colorization with Pre- trained Text-to-Image Diffusion Models

    Hanyuan Liu, Minshan Xie, Jinbo Xing, Chengze Li, and Tien-Tsin Wong. Video Colorization with Pre- trained Text-to-Image Diffusion Models. arXiv preprint arXiv:2306.01732, 2023. 12

    2023 arXiv

  31. [39]

    Video-P2P: Video Editing with Cross-attention Control

    Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-P2P: Video Editing with Cross-attention Control. In CVPR, 2023. 3

    2023

  32. [40]

    Yihao Liu, Hengyuan Zhao, Kelvin C. K. Chan, Xintao Wang, Chen Change Loy, Yu Qiao, and Chao Dong. Tem- porally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning.arXiv preprint arXiv:2110.04562, 2021. 12

    arXiv

  33. [41]

    T2I- Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2I- Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models. arXiv preprint arXiv:2302.08453, 2023. 2

    2023 arXiv

  34. [42]

    Improved Denoising Diffusion Probabilistic Models

    Alex Nichol and Prafulla Dhariwal. Improved Denoising Diffusion Probabilistic Models. In ICML, 2021. 2

    2021

  35. [43]

    GLIDE: Towards Photorealistic Image Gener- ation and Editing with Text-Guided Diffusion Models

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards Photorealistic Image Gener- ation and Editing with Text-Guided Diffusion Models. In ICML, 2022. 2

    2022

  36. [44]

    Understand the Latent Space of Diffu- sion Models through the Lens of Riemannian Geometry

    Yong-Hyun Park, Mingi Kwon, Jaewoong Choi, Junghyo Jo, and Youngjung Uh. Understand the Latent Space of Diffu- sion Models through the Lens of Riemannian Geometry. In NeurIPS, 2023. 5

    2023

  37. [45]

    Zero-shot Image-to- Image Translation

    Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yi- jun Li, Jingwan Lu, and Jun-Yan Zhu. Zero-shot Image-to- Image Translation. In SIGGRAPH, 2023. 3

    2023

  38. [46]

    The 2017 DA VIS Challenge on Video Object Segmentation

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 DA VIS Challenge on Video Object Segmentation. arXiv preprint arXiv:1704.00675, 2018. 12, 13

    2017 arXiv

  39. [47]

    FateZero: Fus- ing Attentions for Zero-shot Text-based Video Editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. FateZero: Fus- ing Attentions for Zero-shot Text-based Video Editing. In ICCV, 2023. 3, 6, 7, 12, 13

    2023

  40. [48]

    Zero-Shot Text-to-Image Generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-Shot Text-to-Image Generation. In ICML, 2021. 2

    2021

  41. [49]

    Improving Meeting Capture by Applying Television Production Principles with Audio and Motion Detection

    Abhishek Ranjan, Jeremy Birnholtz, and Ravin Balakrish- nan. Improving Meeting Capture by Applying Television Production Principles with Audio and Motion Detection. In CHI, 2008. 2

    2008

  42. [50]

    SAM 2: Segment Anything in Images and Videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junt- ing Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Doll´ar, and Christoph Feic...

    2024 arXiv

  43. [51]

    High-Resolution Image Synthesis with Latent Diffusion Models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models. In CVPR, 2022. 2, 3, 4

    2022

  44. [52]

    U- Net: Convolutional Networks for Biomedical Image Seg- mentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- Net: Convolutional Networks for Biomedical Image Seg- mentation. In MICCAI, 2015. 4

    2015

  45. [53]

    Lee, Jonathan Ho, Tim Salimans, David J

    Chitwan Saharia, William Chan, Huiwen Chang, Chris A. Lee, Jonathan Ho, Tim Salimans, David J. Fleet, and Mo- hammad Norouzi. Palette: Image-to-Image Diffusion Mod- els. In SIGGRAPH, 2022. 2

    2022

  46. [54]

    Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic Text-to- Image Diffusion Mo...

    2022

  47. [55]

    Edit-A-Video: Single Video Editing with Object-Aware Consistency

    Chaehun Shin, Heeseung Kim, Che Hyun Lee, Sang gil Lee, and Sungroh Yoon. Edit-A-Video: Single Video Editing with Object-Aware Consistency. In ACML, 2023. 3

    2023

  48. [56]

    Make-A-Video: Text-to-Video Generation without Text-Video Data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taig- man. Make-A-Video: Text-to-Video Generation without Text-Video Data. In ICLR, 2022. 2

    2022

  49. [57]

    Denois- ing Diffusion Implicit Models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing Diffusion Implicit Models. In ICLR, 2021. 1, 2, 3, 5

    2021

  50. [58]

    Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equa- tions. In ICLR, 2021. 2

    2021

  51. [59]

    Computer Vision: Algorithms and Appli- cations

    Richard Szeliski. Computer Vision: Algorithms and Appli- cations. Springer, 2nd edition, 2021. 1, 3, 6

    2021

  52. [60]

    QuickCut: An Interactive Tool for Editing Nar- rated Video

    Anh Truong, Floraine Berthouzoz, Wilmot Li, and Maneesh Agrawala. QuickCut: An Interactive Tool for Editing Nar- rated Video. In UIST, 2016. 2

    2016

  53. [61]

    Plug-and-Play Diffusion Features for Text- Driven Image-to-Image Translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-Play Diffusion Features for Text- Driven Image-to-Image Translation. arXiv preprint arXiv:2211.12572, 2022. 3, 6, 12, 13

    2022 arXiv

  54. [62]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention Is All You Need. In NeurIPS, 2017. 4

    2017

  55. [63]

    Tracking Emerges by Col- orizing Videos

    Carl V ondrick, Abhinav Shrivastava, Alireza Fathi, Sergio Guadarrama, and Kevin Murphy. Tracking Emerges by Col- orizing Videos. In ECCV, 2018. 2

    2018

  56. [64]

    Sketch-Guided Text-to-Image Diffusion Models

    Andrey V oynov, Kfir Aberman, and Daniel Cohen-Or. Sketch-Guided Text-to-Image Diffusion Models. In SIG- GRAPH, 2023. 2 10

    2023

  57. [65]

    Pretraining is All You Need for Image-to-Image Translation

    Tengfei Wang, Ting Zhang, Bo Zhang, Hao Ouyang, Dong Chen, Qifeng Chen, and Fang Wen. Pretraining is All You Need for Image-to-Image Translation. arXiv preprint arXiv:2205.12952, 2022. 2

    2022 arXiv

  58. [66]

    Story-driven Video Editing

    Zheng Wang, Jianguo Li, and Yu-Gang Jiang. Story-driven Video Editing. TMM, 2021. 2

    2021

  59. [67]

    Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation. In ICCV, 2023. 3

    2023

  60. [68]

    Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation. In SIGGRAPH, 2023. 3

    2023

  61. [69]

    BiSTNet: Semantic Im- age Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-Based Video Colorization

    Yixin Yang, Jinshan Pan, Zhongzheng Peng, Xiaoyu Du, Zhulin Tao, and Jinhui Tang. BiSTNet: Semantic Im- age Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-Based Video Colorization. TPAMI, 2024. 2

    2024

  62. [70]

    Adding Conditional Control to Text-to-Image Diffusion Models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding Conditional Control to Text-to-Image Diffusion Models. In ICCV, 2023. 2

    2023

  63. [71]

    Efros, Eli Shecht- man, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR, 2018. 12

    2018

  64. [72]

    I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models. arXiv preprint arXiv:2311.04145, 2023. 3, 4, 5

    2023 arXiv

  65. [73]

    ControlVideo: Training-free Controllable Text-to-Video Generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. ControlVideo: Training-free Controllable Text-to-Video Generation. In ICLR, 2024. 3

    2024

  66. [74]

    SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation

    Yuzhi Zhao, Lai-Man Po, Kangcheng Liu, Xuehui Wang, Wing-Yin Yu, Pengfei Xian, Yujia Zhang, and Mengyang Liu. SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation. TIP, 2023. 2 11 Supplementary material for DreamColour: Controllable Video Colour Editing ...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.