Pith. sign in

REVIEW 4 major objections 6 minor 47 references

VipDiff: Towards Coherent and Diverse Video Inpainting via Training-free Denoising Diffusion Models

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VipDiff claims that a pre-trained image-level diffusion model can be steered by optical-flow-propagated pixels to produce temporally coherent, diverse video inpainting with no training.

desk verdict VipDiff's combination of flow propagation and noise optimization is genuinely new, but the SOTA claim rests on unvalidated flow accuracy inside masks and test-set-tuned hyperparameters. read the letter →

arxiv 2501.12267 v1 pith:MNIPAEXV submitted 2025-01-21 cs.CV

Chaohao Xie , Kai Han , Kwan-Yee K. Wong This is my paper · ORCID
classification cs.CV
keywords videoinpaintingdiffusionmodelstraining-freeopticalflownoiseoptimizationtemporalcoherencelatentdiversegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VipDiff tries to establish that a pre-trained image-level diffusion model can be turned into a video inpainting method without any training data or fine-tuning. The recipe is to use completed optical flows to copy valid pixels from other frames into the masked region, then optimize the diffusion model's input noise so the generated frame matches both those propagated pixels and its own unmasked pixels. If this works, video inpainting no longer needs a large trained video-diffusion model, and users with one GPU can generate temporally coherent, diverse completions in minutes. The reported experiments on YouTube-VOS and DAVIS back this up on temporal-consistency metrics, and the method also attacks the mask-center artifacts that plague flow-propagation baselines.

What carries the argument

The central machinery is noise-optimized reverse diffusion. Instead of sampling noise once and denoising, VipDiff optimizes the initial Gaussian noise $z$ so that the frozen latent-diffusion U-Net's output $\hat{y}_k$ matches the partially filled frame $\tilde{x}^k_0$ wherever valid pixels exist, using the loss $L_{\mathrm{cond}} = \|\hat{y}_k \odot (1 - \tilde{m}_k) - \tilde{x}^k_0 \odot (1 - \tilde{m}_k)\|^2$ plus a regularization term $\gamma\|z - z_0\|^2$. The second load-bearing piece is optical-flow-guided pixel propagation: completed flows $\tilde{f}_{k \to j}$ backward-warp valid pixels from reference frame $j$ into target frame $k$, updating the valid mask. Together, these convert an image-level diffusion prior into a video-consistent prior without changing any network weights.

What would settle it

Take a masked video where the true motion inside the mask is known, for example a synthetic scene with known depth and camera path. Run VipDiff once with the completed flows it normally uses and once with the known true flows; if the two outputs differ significantly in VFID or flow-warping error, flow accuracy inside the mask is load-bearing. If they do not differ, the diffusion prior is doing the work and the flow-completion assumption is not what carries the method.

Watch

Extended reading notes

Core claim

VipDiff's central claim is that temporal coherence in video inpainting does not require a video-specific diffusion model or any training. Working on a frozen image-level latent diffusion model, it first completes optical flows between frames and warps valid pixels from reference frames into the masked target frame, producing a partially filled frame. The method then treats the randomly sampled Gaussian noise as a parameter and backpropagates a reconstruction loss that compares the denoised output with this partially filled frame, regularized to keep the noise distribution Gaussian. Once a frame is generated, its pixels are propagated to neighboring frames, shrinking the remaining mask; the authors report this loop yields the best SSIM, VFID, and flow-warping error on YouTube-VOS and DAVIS, while still allowing multiple diverse completions from different noise samples. They state this is the first work to successfully use pre-trained image-level diffusion models for video inpainting.

Load-bearing premise

The load-bearing premise is that the pre-trained model that guesses how pixels move inside the missing region gets those guesses right, so the pixels it copies from other frames are genuinely valid guides for the generator.

Editorial extensions

If this is right

  • On YouTube-VOS and DAVIS, VipDiff reports the highest SSIM, VFID, and flow-warping error among the compared methods, indicating both sharper frames and better temporal consistency than flow-based and transformer baselines.
  • Because the pipeline freezes all network weights, the same framework can be carried over to any pre-trained image-level diffusion model without retraining, which the authors demonstrate in the supplementary material.
  • Large masks whose centers have no pixel correspondences in reference frames are filled by the diffusion prior, removing the mask-center artifacts typical of flow-guided video inpainting.
  • Sampling different Gaussian noises yields different yet temporally coherent completions, giving users a choice among plausible fills for the same masked video.
  • A video of about 100 to 200 frames is completed in roughly 4 to 6 minutes on a single RTX 3090 GPU, making diffusion-based video inpainting accessible without large-scale training infrastructure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the completed optical flows are wrong inside the mask, the propagated pixels in Eq. 5 become wrong constraints, so the noise-optimized diffusion will be steered toward an incorrect target; the paper does not measure flow error inside masked regions, leaving this as a testable vulnerability.
  • Editorial inference: diversity is only sampled in regions where no valid pixels arrive; for masks with strong flow guidance, the propagated constraints tightly pin down the output, so diversity should be more visible for large stationary masks than for object-removal tasks.
  • Editorial inference: the same noise-optimization loop should extend to other video inverse problems by swapping the constraint in Eq. 6 for a different forward model, such as a blur kernel for video deblurring or a downsampling operator for video super-resolution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes VipDiff, a training-free video inpainting framework that combines optical-flow-guided pixel propagation from reference frames with noise-optimized reverse diffusion in a pre-trained image-level latent diffusion model. Valid pixels from completed flows are used as hard constraints during noise optimization, and after a frame is generated its pixels are propagated to neighboring frames. The authors report state-of-the-art or competitive SSIM, VFID, and Ewarp on YouTube-VOS and DAVIS, show qualitative diversity across random noise samples, and ablate the contributions of pixel propagation and noise optimization.

Significance. If the claims hold, the result is significant: it shows that a training-free combination of a pre-trained flow completion model and a pre-trained image diffusion model can produce temporally coherent video inpainting, avoiding the need to train a video diffusion model. The pipeline is clearly described, the components are individually motivated, and the central method is not circular because it is evaluated against held-out ground truth. However, the evaluation currently has unresolved reliability issues: the method is stochastic but Table 1 reports single-run numbers, the regularization weight is selected on the same DAVIS test set used for the headline comparison, and the accuracy of the completed optical flows inside masks is never validated even though Eqs. (5) and (6) convert those flows into hard constraints. These issues weaken the strength of the state-of-the-art claim and require additional experiments rather than simple text changes.

major comments (4)
  1. [Section 4.2.1, Table 1] The method is stochastic because the noise z is randomly sampled and then optimized, yet Table 1 reports a single run with no error bars or significance tests. Several margins are very small: PSNR is slightly below ProPainter on both datasets (34.21 vs. 34.23 on YouTube-VOS and 34.23 vs. 34.27 on DAVIS), and SSIM margins are around 0.001. The claim of 'largely outperforming' state-of-the-art methods is not supported without multi-seed statistics. Please report mean and standard deviation over at least 3-5 runs and state whether the differences are statistically significant. The text also says VipDiff achieved 'the most highest metric scores,' but PSNR is not the best on either dataset.
  2. [Sections 4.1.3 and 4.3] The regularization weight gamma is set to 0.001 for the main experiments, and the ablation that recommends this value is performed on the DAVIS test set, which is the same dataset used for the headline numbers in Table 1. This is test-set tuning. Please select gamma on a validation split (for example, the DAVIS validation set or the YouTube-VOS validation set) and report the chosen value, or provide a sensitivity analysis showing stable results across gamma in the recommended range [0.01, 0.001].
  3. [Sections 3.2 and 3.3, Eqs. (5)-(6)] The temporal constraints come from completed optical flows inside the masked region: Eq. (5) uses the completed flow to warp valid pixels into the mask, and Eq. (6) enforces the diffusion output to match those propagated pixels on the valid region. The paper never evaluates flow accuracy in masked areas, even though a wrong flow does not merely leave a hole but creates a hard constraint that the noise-optimized reverse diffusion must reproduce. Please report flow error (e.g., EPE or outlier rate) inside masks on a subset of the evaluation videos, and include an ablation that substitutes ground-truth flows for the completed flows or adds controlled perturbations, to quantify the sensitivity of VipDiff to flow errors.
  4. [Section 4.3] The ablation omits the variant that runs noise-optimized reverse diffusion for every single frame, with only a qualitative statement that it was slow and produced flickering. This variant is directly relevant to the claim that the propagation branch is necessary. Please report its quantitative metrics in a supplementary table alongside LDM, LDM+PP, w/o Opt, and Ours, or explicitly provide the numbers in the main text.
minor comments (6)
  1. [Throughout] There are several typos: 'DA VIS' should be 'DAVIS', 'noize' should be 'noise', 'sinlge' should be 'single', 'stationery masks' should be 'stationary masks', and 'most highest metric scores' should be 'highest metric scores'.
  2. [Sections 3.2 and 4.1.3] The flow completion model is called both a 'pre-trained RAFT model [30]' in Eq. (4) and a 'pre-trained RAFT provided by [14]' in the implementation details. Please clarify the exact checkpoint and whether it is the ECFVI-trained flow completion model.
  3. [Section 4.1.3] The phrase 'the number of reverse optimization steps is set to 50' is ambiguous: it is not clear whether 50 is the total number of denoising steps, the number of gradient steps per denoising step, or the number of optimization iterations. Please clarify the schedule of eta_t as well.
  4. [Section 4.3, Fig. 6] The gamma ablation is qualitative only. Adding a quantitative table with PSNR, VFID, and Ewarp for gamma = 0.1, 0.01, 0.001, and 0 would strengthen the recommendation on choosing gamma.
  5. [Section 5] The claim of being 'the first work that successfully tames pre-trained image-level diffusion models for video inpainting' should be reconciled with the cited video diffusion methods [10, 46], which are not included in the quantitative comparisons.
  6. [Section 3.3 and Fig. 1] Diversity is demonstrated only through three samples in Fig. 1. A quantitative diversity metric, such as the average LPIPS distance between inpainted samples over the same video, would support the diversity claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: VipDiff's derivation chain is self-contained and evaluated on external benchmarks; the flow-completion dependency is an unvalidated external assumption, not a circular step.

full rationale

VipDiff's claimed contribution is a training-free video inpainting framework that optimizes a sampled noise so that a frozen image-level diffusion model matches optical-flow-propagated pixels. The derivation chain does not reduce to its inputs at any point: Eq. (5) defines the propagation constraints from completed flows, Eq. (6) defines the conditioning loss against the partially filled frame, and Eq. (7) optimizes the noise. These are construction steps for a generative objective, not fitted parameters renamed as predictions. The final frames are compared against held-out YouTube-VOS and DAVIS test frames using standard metrics, so the headline results are not forced by the method's equations. There is no self-citation chain: the pretrained LDM, RAFT-based flow completion, and error-compensation network are all external prior works cited as components, not as an unverified uniqueness theorem by the present authors. The main legitimate concern is that the regularization weight gamma is ablated on the same DAVIS test split later used for the main comparison, which is a benchmarking-protocol weakness that compromises benchmark independence; however, gamma is a hyperparameter, not a fitted quantity whose reported performance is determined by construction. Likewise, the reliance on ECFVI's flow completion inside large masks is an unvalidated external dependency and a correctness risk, but it is not circularity because the paper does not claim to derive flow accuracy from its own outputs. Overall, the central derivation is self-contained against external benchmarks, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method rests on off-the-shelf pre-trained components (LDM, RAFT/flow completion, ECFVI error compensation) and hand-set hyperparameters (gamma, learning rate, steps). No new physical entities are introduced. The most consequential assumption is the accuracy of flow completion inside the mask, which is the foundation of the temporal constraints.

free parameters (3)
  • gamma (regularization weight) = 0.001
    Chosen by ablation on DAVIS test set (Fig. 6, Section 4.3), then used for final DAVIS results, tuning on the evaluation set.
  • learning rate eta0 = 0.01
    Set by hand in Section 4.1.3; no sensitivity analysis.
  • number of noise optimization steps = 50
    Set by hand in Section 4.1.3; no sensitivity analysis.
assumptions (4)
  • domain assumption The pre-trained LDM can serve as a valid generative prior for video frames, even though it was trained on images.
    Used in Section 3.3: the denoising U-Net is applied to each frame independently with only noise-level constraints.
  • domain assumption The pre-trained flow completion model produces accurate complete flows inside masked regions.
    Section 3.2 states the flow estimator predicts complete optical flows for images with missing areas; no direct validation on masked regions is provided.
  • domain assumption The error compensation network from ECFVI corrects color discrepancy in propagated pixels.
    Section 3.2.1 adopts the ECFVI error compensation network; the paper provides no analysis of its failure cases.
  • standard math The noise optimization procedure (Eq. 7) yields a valid sample from the diffusion model.
    Follows from CoPaint; the paper assumes the regularized objective and gradient descent on z produce a meaningful result without proof.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VipDiff: Towards Coherent and Diverse Video Inpainting via Training-free Denoising Diffusion Models." pith.science (2026). https://pith.science/paper/MNIPAEXV

@misc{pith2026250112267,
  author       = {Pith},
  title        = {Pith review of: VipDiff: Towards Coherent and Diverse Video Inpainting via Training-free Denoising Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MNIPAEXV}},
  note         = {Machine review of arXiv:2501.12267}
}
read the original abstract

Recent video inpainting methods have achieved encouraging improvements by leveraging optical flow to guide pixel propagation from reference frames either in the image space or feature space. However, they would produce severe artifacts in the mask center when the masked area is too large and no pixel correspondences can be found for the center. Recently, diffusion models have demonstrated impressive performance in generating diverse and high-quality images, and have been exploited in a number of works for image inpainting. These methods, however, cannot be applied directly to videos to produce temporal-coherent inpainting results. In this paper, we propose a training-free framework, named VipDiff, for conditioning diffusion model on the reverse diffusion process to produce temporal-coherent inpainting results without requiring any training data or fine-tuning the pre-trained diffusion models. VipDiff takes optical flow as guidance to extract valid pixels from reference frames to serve as constraints in optimizing the randomly sampled Gaussian noise, and uses the generated results for further pixel propagation and conditional generation. VipDiff also allows for generating diverse video inpainting results over different sampled noise. Experiments demonstrate that VipDiff can largely outperform state-of-the-art video inpainting methods in terms of both spatial-temporal coherence and fidelity.

Figures

Figures reproduced from arXiv: 2501.12267 by the authors.

Figure 1
Figure 1. Video inpainting results on Davis dataset, we randomly select 3 frames from ’bear’ and ’cows’ videos. LDM generates different [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overall framework of our VipDiff. Given a target frame x k 0 , we first adopt a flow completion model to predict the optical flows. Then the flows are utilized for pixel propagation to extract temporal prior from reference frames to get partially inpainted image xe k 0 . Next, xe k 0 will act as constrains for optimizing the random sampled Gaussian nosie z which is feed for the reverse denoising U-Net. Through backp… view at source ↗
Figure 3
Figure 3. Process for propagating constrained generated pixels to [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons with SOTA video inpainting methods. Best viewed in PDF with zoom. Please refer to the supplementary [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Ablation study on regularization weight γ. ent weights of γ, ranging from 0.1 to 0, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 5
Figure 5. Figure 5: Ablation study on different variants. From top row to [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 41 canonical work pages

  1. [1]

    Patchmatch: A randomized correspon- dence algorithm for structural image editing

    Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. Patchmatch: A randomized correspon- dence algorithm for structural image editing. ACM Trans- actions on Graphics (TOG), 2009. 3

    work page 2009

  2. [2]

    Navier-stokes, fluid dynamics, and image and video inpainting

    Marcelo Bertalmio, Andrea L Bertozzi, and Guillermo Sapiro. Navier-stokes, fluid dynamics, and image and video inpainting. In IEEE Conference on Computer Vision and Pat- tern Recognition, 2001. 3

    work page 2001

  3. [3]

    Free-form video inpainting with 3d gated convolution and temporal patchgan

    Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, and Winston Hsu. Free-form video inpainting with 3d gated convolution and temporal patchgan. In IEEE/CVF International Confer- ence on Computer Vision, 2019. 1, 3

    work page 2019

  4. [4]

    Learnable gated temporal shift module for deep video inpainting

    Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, and Winston Hsu. Learnable gated temporal shift module for deep video inpainting. In British Machine Vision Conference, 2019. 1, 3, 5, 6

    work page 2019

  5. [5]

    Diffusion pos- terior sampling for general noisy inverse problems

    Hyungjin Chung, Jeongsol Kim, Michael Thompson Mc- cann, Marc Louis Klasky, and Jong Chul Ye. Diffusion pos- terior sampling for general noisy inverse problems. In Inter- national Conference on Learning Representations, 2023. 1, 3

    work page 2023

  6. [6]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In M. Ranzato, A. Beygelz- imer, Y . Dauphin, P.S. Liang, and J. Wortman Vaughan, ed- itors, Advances in Neural Information Processing Systems ,

    work page

  7. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representa- tions, 2021. 3

    2021

  8. [8]

    Flow-edge guided video completion

    Chen Gao, Ayush Saraf, Jia-Bin Huang, and Johannes Kopf. Flow-edge guided video completion. In European Confer- ence on Computer Vision, 2020. 1, 3, 5, 6

    work page 2020

Show all 47 references
  1. [9]

    How not to be seen—object removal from videos of crowded scenes

    Miguel Granados, James Tompkin, K Kim, Oliver Grau, Jan Kautz, and Christian Theobalt. How not to be seen—object removal from videos of crowded scenes. In Computer Graphics Forum, 2012. 3

    2012

  2. [10]

    Flow-guided diffusion for video inpainting

    Bohai Gu, Yongsheng Yu, Heng Fan, and Libo Zhang. Flow-guided diffusion for video inpainting. arXiv preprint arXiv:2311.15368, 2023. 3

    2023 arXiv

  3. [11]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 2020. 1, 3

    2020

  4. [12]

    Proposal-based video com- pletion

    Yuan-Ting Hu, Heng Wang, Nicolas Ballas, Kristen Grau- man, and Alexander G Schwing. Proposal-based video com- pletion. In European Conference on Computer Vision, 2020. 1, 3

    2020

  5. [13]

    Temporally coherent completion of dynamic video

    Jia-Bin Huang, Sing Bing Kang, Narendra Ahuja, and Jo- hannes Kopf. Temporally coherent completion of dynamic video. ACM Transactions on Graphics (ToG), 2016. 3

    2016

  6. [14]

    Error compensation framework for flow-guided video inpainting

    Jaeyeon Kang, Seoung Wug Oh, and Seon Joo Kim. Error compensation framework for flow-guided video inpainting. In European Conference on Computer Vision, 2022. 1, 2, 3, 5, 6, 7

    2022

  7. [15]

    Deep video inpainting

    Dahun Kim, Sanghyun Woo, Joon-Young Lee, and In So Kweon. Deep video inpainting. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 5, 6

    2019

  8. [16]

    Learning blind video temporal consistency

    Wei-Sheng Lai, Jia-Bin Huang, Oliver Wang, Eli Shechtman, Ersin Yumer, and Ming-Hsuan Yang. Learning blind video temporal consistency. In European conference on computer vision, 2018. 6

    2018

  9. [17]

    Copy-and-paste networks for deep video inpainting

    Sungho Lee, Seoung Wug Oh, DaeYeun Won, and Seon Joo Kim. Copy-and-paste networks for deep video inpainting. In IEEE/CVF international conference on computer vision ,

  10. [18]

    Short-term and long-term context aggrega- tion network for video inpainting

    Ang Li, Shanshan Zhao, Xingjun Ma, Mingming Gong, Jianzhong Qi, Rui Zhang, Dacheng Tao, and Ramamoha- narao Kotagiri. Short-term and long-term context aggrega- tion network for video inpainting. In European Conference on Computer Vision, 2020. 1, 3

    2020

  11. [19]

    Towards an end-to-end framework for flow-guided video inpainting

    Zhen Li, Cheng-Ze Lu, Jianhua Qin, Chun-Le Guo, and Ming-Ming Cheng. Towards an end-to-end framework for flow-guided video inpainting. In IEEE/CVF conference on computer vision and pattern recognition, 2022. 1, 3, 5, 6

    2022

  12. [20]

    Fuseformer: Fusing fine-grained information in transformers for video inpainting

    Rui Liu, Hanming Deng, Yangyi Huang, Xiaoyu Shi, Lewei Lu, Wenxiu Sun, Xiaogang Wang, Jifeng Dai, and Hong- sheng Li. Fuseformer: Fusing fine-grained information in transformers for video inpainting. In IEEE/CVF interna- tional conference on computer vision, 2021. 1, 3, 5, 6

    2021

  13. [21]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition ,

  14. [22]

    Full-frame video stabilization with motion inpainting

    Yasuyuki Matsushita, Eyal Ofek, Weina Ge, Xiaoou Tang, and Heung-Yeung Shum. Full-frame video stabilization with motion inpainting. IEEE Transactions on pattern analysis and Machine Intelligence, 2006. 3

    2006

  15. [23]

    Video inpainting of complex scenes

    Alasdair Newson, Andr ´es Almansa, Matthieu Fradet, Yann Gousseau, and Patrick P ´erez. Video inpainting of complex scenes. Siam journal on imaging sciences, 2014. 3

    2014

  16. [24]

    Video inpainting of occluding and occluded ob- jects

    Kedar A Patwardhan, Guillermo Sapiro, and Marcelo Bertalmio. Video inpainting of occluding and occluded ob- jects. In IEEE International Conference on Image Process- ing 2005, 2005. 3

    2005

  17. [25]

    Video inpainting under constrained camera mo- tion

    Kedar A Patwardhan, Guillermo Sapiro, and Marcelo Bertalm´ıo. Video inpainting under constrained camera mo- tion. IEEE Transactions on Image Processing, 2007. 3

    2007

  18. [26]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In IEEE conference on computer vision and pattern recognition, 2016. 6

    2016

  19. [27]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InIEEE/CVF conference on computer vision and pattern recognition, 2022. 1, 2, 3, 4, 6, 7

    2022

  20. [28]

    Deep unsupervised learning using 9 nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using 9 nonequilibrium thermodynamics. In International confer- ence on machine learning, 2015. 1, 3

    2015

  21. [29]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021. 3

    2021

  22. [30]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In European Conference on Computer Vision, 2020. 5

    2020

  23. [31]

    Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S

    Brian L. Trippe, Jason Yim, Doug Tischer, David Baker, Tamara Broderick, Regina Barzilay, and Tommi S. Jaakkola. Diffusion probabilistic modeling of protein backbones in 3d for the motif-scaffolding problem. In International Confer- ence on Learning Representations, 2023. 1, 3

    2023

  24. [32]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 2017. 3

    2017

  25. [33]

    Video inpainting by jointly learning temporal structure and spatial details

    Chuan Wang, Haibin Huang, Xiaoguang Han, and Jue Wang. Video inpainting by jointly learning temporal structure and spatial details. In AAAI Conference on Artificial Intelligence,

  26. [34]

    Video-to- video synthesis

    Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. Video-to- video synthesis. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Ad- vances in Neural Information Processing Systems, 2018. 6

    2018

  27. [35]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing,

  28. [36]

    Space- time completion of video

    Yonatan Wexler, Eli Shechtman, and Michal Irani. Space- time completion of video. IEEE Transactions on pattern analysis and machine intelligence, 2007. 3

    2007

  29. [37]

    Towards language-driven video inpainting via multimodal large language models

    Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yunhai Tong, Ziwei Liu, et al. Towards language-driven video inpainting via multimodal large language models. In IEEE conference on computer vision and pattern recogni- ...

    2024

  30. [38]

    Youtube-vos: Sequence-to-sequence video object segmentation

    Ning Xu, Linjie Yang, Yuchen Fan, Jianchao Yang, Dingcheng Yue, Yuchen Liang, Brian Price, Scott Cohen, and Thomas Huang. Youtube-vos: Sequence-to-sequence video object segmentation. In European conference on com- puter vision, 2018. 6

    2018

  31. [39]

    Deep flow-guided video inpainting

    Rui Xu, Xiaoxiao Li, Bolei Zhou, and Chen Change Loy. Deep flow-guided video inpainting. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2019. 1, 3, 5, 6

    2019

  32. [40]

    Frequency-aware spatiotemporal transformers for video in- painting detection

    Bingyao Yu, Wanhua Li, Xiu Li, Jiwen Lu, and Jie Zhou. Frequency-aware spatiotemporal transformers for video in- painting detection. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 3

    2021

  33. [41]

    Learning joint spatial-temporal transformations for video inpainting

    Yanhong Zeng, Jianlong Fu, and Hongyang Chao. Learning joint spatial-temporal transformations for video inpainting. In European Conference on Computer Vision, 2020. 1, 3, 5, 6

    2020

  34. [42]

    Towards coherent image in- painting using denoising diffusion implicit models, 2023

    Guanhua Zhang, Jiabao Ji, Yang Zhang, Mo Yu, Tommi Jaakkola, and Shiyu Chang. Towards coherent image in- painting using denoising diffusion implicit models, 2023. 1, 3, 6

    2023

  35. [43]

    An internal learning approach to video inpainting

    Haotian Zhang, Long Mai, Ning Xu, Zhaowen Wang, John Collomosse, and Hailin Jin. An internal learning approach to video inpainting. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, 2019. 1, 3

    2019

  36. [44]

    Flow-guided transformer for video inpainting

    Kaidong Zhang, Jingjing Fu, and Dong Liu. Flow-guided transformer for video inpainting. In European Conference on Computer Vision, 2022. 3, 5, 6, 7

    2022

  37. [45]

    Ex- ploiting optical flow guidance for transformer-based video inpainting

    Kaidong Zhang, Jialun Peng, Jingjing Fu, and Dong Liu. Ex- ploiting optical flow guidance for transformer-based video inpainting. arXiv preprint arXiv:2301.10048, 2023. 3

    2023 arXiv

  38. [46]

    Avid: Any-length video inpainting with diffusion model

    Zhixing Zhang, Bichen Wu, Xiaoyan Wang, Yaqiao Luo, Luxin Zhang, Yinan Zhao, Peter Vajda, Dimitris Metaxas, and Licheng Yu. Avid: Any-length video inpainting with diffusion model. arXiv preprint arXiv:2312.03816, 2023. 3

    2023 arXiv

  39. [47]

    ProPainter: Improving propagation and transformer for video inpainting

    Shangchen Zhou, Chongyi Li, Kelvin C.K Chan, and Chen Change Loy. ProPainter: Improving propagation and transformer for video inpainting. InIEEE/CVF International Conference on Computer Vision (ICCV), 2023. 1, 2, 3, 5, 6, 7 10

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.