Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

OutDreamer: Video Outpainting with a Diffusion Transformer

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read OutDreamer, a zero-shot diffusion-transformer video outpainting method, reports better SSIM, LPIPS, and FVD than existing zero-shot baselines on DAVIS and YouTube-VOS, with PSNR equal or better, and parity with a one-shot method that…

desk verdict Solid DiT-based zero-shot video outpainting architecture with real ablations; the SOTA claim rests on unverified baseline numbers, so the evaluation needs a serious pass before this is citable as a benchmark win. read the letter →

arxiv 2506.22298 v1 pith:6HG7UWUN submitted 2025-06-27 cs.CV

Linhao Zhong , Fan Li , Yi Huang , Jianzhuang Liu , Renjing Pei , Fenglong Song This is my paper
classification cs.CV
keywords videooutpaintingdiffusiontransformerzero-shotgenerationmask-drivenself-attentionlatentalignmentlosslongtemporalconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces OutDreamer, a video outpainting method that extends video frames beyond their borders without any per-video fine-tuning. It claims that a diffusion transformer backbone, fed with masked-video conditions through a lightweight control branch and a mask-driven self-attention layer, generates visually realistic and temporally consistent outpainted content. The authors report that this zero-shot approach beats current zero-shot baselines on DAVIS and YouTube-VOS on SSIM, LPIPS, and FVD, and reaches parity with a one-shot method that is allowed to adapt to each video. If true, this would make high-quality video outpainting practical for both short and long clips without costly per-video adaptation.

What carries the argument

The load-bearing mechanism is the mask-driven self-attention of Eq. (3): $$ \mathrm{Attn}(Q,K,V) = \mathrm{softmax}\left(\frac{Q(K \cdot (1+\gamma F_s(m)))^T}{\sqrt{d_k}}\right) V, $$ where $F_s(m)$ maps the mask to per-key scaling factors in $[-1,1]$, so patches in the known region attract more attention during denoising. Around this sit two supporting pieces: an efficient control branch that extracts the masked-video latent and adds its features into the first DiT block (Eq. (2)), and a latent alignment loss that penalizes per-frame mean and variance differences between predicted and ground-truth latents (Eq. (5)). A cross-video-clip refiner, doing mean-variance alignment and histogram matching stitch, keeps iteratively generated long clips consistent.

What would settle it

Run the released masked 3D diffusion code (and the non-masked variant if a checkpoint can be obtained) through OutDreamer's exact evaluation pipeline—same mask ratios 0.25 and 0.66, 256x256 resizing, 29-frame clips, 16-frame FVD sampling, and same captions—and check whether the resulting scores reproduce Table 1; if the baseline numbers change materially, the claimed lead may not hold.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that replacing a U-Net denoiser with a diffusion transformer, injecting masked-video conditions right after the first transformer block, and steering self-attention with mask information yields zero-shot video outpainting that is more realistic and temporally consistent than existing zero-shot approaches. In the reported tables, OutDreamer leads the zero-shot baselines on DAVIS and YouTube-VOS for SSIM, LPIPS, and FVD, and its PSNR is equal to or better than theirs; against a one-shot method that fine-tunes per video, it achieves comparable scores without any adaptation. The ablation study attributes these gains to the latent alignment loss and the mask-driven self-attention layer, and to the cross-video-clip refiner for long sequences.

Load-bearing premise

The state-of-the-art claim assumes the SDM scores in Table 1 came from the same evaluation protocol as OutDreamer's, yet the paper says SDM's code is not released and excludes it from qualitative comparisons.

Editorial extensions

If this is right

  • If the reported numbers hold, zero-shot video outpainting does not need a U-Net backbone: the diffusion transformer version beats the U-Net-based zero-shot baselines on SSIM, LPIPS, and FVD in the paper's tables.
  • The mask-driven self-attention and latent alignment loss each add measurable quality; Table 3 shows the full model scores best, so future outpainting systems can adopt these two modules independently.
  • Long-video outpainting can be done iteratively, clip by clip, with overlapping condition frames and a refiner; Table 4 shows the refiner improves all metrics on 315-frame outputs.
  • Zero-shot outpainting can reach roughly the same quality as one-shot adaptation that fine-tunes per video, without the per-video cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the protocol caveat about the non-masked 3D diffusion baseline is resolved, the architecture is likely transferable to other masked video generation tasks such as object removal or region-based video editing, because the mask-driven attention and early condition injection are not outpainting-specific.
  • The cross-video-clip refiner relies on hand-crafted color statistics; a learned refiner trained to minimize inter-clip discontinuities could plausibly push long-video consistency further than the reported histogram matching.
  • The paper evaluates temporal consistency only through FVD and qualitative inspection; a direct temporal-coherence metric, such as inter-frame warping error or flicker measurement, could test whether the claimed frame-to-frame consistency is as strong as stated.
  • Because training uses mask ratios between 0.1 and 0.8, performance at more extreme masks (for example 0.9) is an open question that would tell whether the method scales to near-total outpainting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes OutDreamer, a zero-shot video outpainting framework built on a diffusion transformer backbone (Open-Sora-Plan). The method has two main branches: an efficient video control branch that injects masked video latents into early DiT blocks, and a conditional outpainting branch with mask-driven self-attention that scales attention keys according to the mask. A latent alignment loss aligns per-frame latent means and variances, and long videos are generated iteratively with a cross-video-clip refiner that performs mean-variance alignment and histogram matching. The authors report quantitative comparisons on DAVIS and YouTube-VOS against Dehan, SDM, M3DDM, and MOTIA, and present ablations for the latent alignment loss, mask-driven self-attention, the refiner, and the mask downsampling function. The central claim is that OutDreamer outperforms existing zero-shot video outpainting methods on standard benchmarks.

Significance. If the reported numbers are reliable, OutDreamer is a useful step toward DiT-based video outpainting: the control-branch injection at early DiT blocks, the mask-driven key scaling, and the cross-video-clip refiner are reasonable and the ablations are internally consistent. The paper also provides explicit loss equations and pseudocode for the refiner, which aids reproducibility. However, the headline claim rests on a benchmark comparison whose baseline protocol is not established, and the PSNR margins over the strongest baseline are tiny (0.04 dB and 0.01 dB). The architecture is promising, but the empirical support for state-of-the-art status needs to be verified with a fairer protocol and statistical grounding.

major comments (4)
  1. [§5.1.3, Table 1, Figure 5] The evaluation protocol for OutDreamer is described in detail (average of 0.25 and 0.66 mask ratios, resizing to 256×256, input blending, ShareGPT4V captions, and 16-frame FVD sampling), but the paper never states that Dehan, SDM, or M3DDM were rerun under the same protocol. The contradiction with Figure 5 is particularly important: its caption says "SDM is not included since its code is not released," yet Table 1 lists SDM scores for both datasets. If the SDM and M3DDM numbers are copied from the M3DDM paper under different mask ratios, resolutions, blending, or FVD sampling, the comparison is not fair and the claimed state-of-the-art status is not verifiable. The authors should rerun all baselines under the exact protocol used for OutDreamer, or if that is impossible, state the exact source of each baseline number and justify why the protocols are equivalent.
  2. [Table 1, §5.1.2] No error bars, confidence intervals, or significance tests accompany the quantitative results. The PSNR differences over M3DDM are 0.04 dB on DAVIS and 0.01 dB on YouTube-VOS, which are within the run-to-run variability expected from stochastic diffusion sampling. Additionally, the reported scores are averaged over mask ratios 0.25 and 0.66; without per-ratio breakdowns or multiple-seed evaluation, the statement that OutDreamer "outperforms them in all the scores" is not supported, particularly for PSNR.
  3. [§4.4, §5.1.3, Supplementary III] The training setup is not described with sufficient detail to assess the method or reproduce it. The paper specifies loss functions, hyperparameters β and T_latent, and a mask-ratio sampling rule, but it never states which video dataset or datasets were used for training, how many videos, at what resolution and frame length, or for how many training iterations. Since OutDreamer is a learned zero-shot method, the training data is a core part of the method and must be reported.
  4. [§5.4, Table 4] The long-video evaluation is underspecified. The paper states that 20 long videos from Pexels were used, videos were generated with varying frame lengths at mask ratio 0.5, and scores were computed for the last clip, but it does not describe how ground truth is obtained, whether the outpainting target regions are synthetically masked versions of the original videos, how the 20 videos are split, or how the scores are aggregated. The refiner ablation in Table 4 therefore lacks enough protocol detail to be independently reproduced or interpreted.
minor comments (5)
  1. [Eq. (3), Supplementary III] The hyperparameter γ in the mask-driven self-attention is never given a value; please report it together with β and T_latent.
  2. [Table 3] The first row of the ablation table has no method label; it should be labeled "Baseline" for clarity.
  3. [§4.5.1, Figure 2] The advanced version of the efficient video control branch is described only through Figure 2 and a short sentence; an explicit formulation analogous to Eq. (2) would clarify how the last K frames from the previous clip are encoded and injected.
  4. [Figure 7] The figure reports "maximum normalization with proportional scaling," which makes the y-axis values nonstandard and hard to interpret; please show raw metric values or describe the normalization exactly, and consider adding error bars.
  5. [Eq. (6)] The notation g_t is written as "gt" in the equation; please use a subscript for readability and define the range of t relative to the total number of diffusion timesteps.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper is an empirical benchmark and architecture contribution, and no claimed result reduces to its own inputs by construction.

full rationale

OutDreamer's central claims are architectural (efficient video control branch, mask-driven self-attention in Eq. (3), latent alignment loss in Eq. (5), cross-video-clip refiner) and empirical (Table 1). None of the proposed objectives or conditions is defined in terms of the evaluation metric being predicted. The mask-driven self-attention is a learned mechanism with hyperparameters beta and T_latent fixed before evaluation, so there is no fitted parameter being renamed as a prediction. The latent alignment loss supervises the predicted latent Z_0 against ground-truth latent Z_0 during training; this is standard supervision, not a circular derivation. The method builds on Open-Sora-Plan and ControlNeXt through citations, but those citations supply a pretrained backbone and design inspiration only; they are not invoked as an external uniqueness theorem, and no load-bearing argument reduces to a self-citation. The self-citation in the reference list (MagicEraser, reference [17]) appears only in a related-work survey context and is not needed to establish any contribution. The remaining weakness is benchmark-fairness rather than circularity: Figure 5's caption says 'SDM is not included since its code is not released,' while Table 1 lists SDM scores, so the protocol under which those baseline numbers were produced is not fully verified. That concern affects the credibility of the SOTA comparison but is a measurement issue external to the derivation chain, not a case where Eq. X is Eq. Y by construction or where a fitted quantity is presented as a prediction. Under the stipulated circularity definitions, the paper's derivation chain is self-contained: its improvements are validated by controlled ablations (Tables 3-5), and its headline benchmark claim is an empirical comparison rather than a mathematically forced consequence of its definitions.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the pretrained Open-Sora-Plan backbone and on several design choices (control injection position, mask-driven key scaling, latent alignment loss, refiner) that are only validated on the reported benchmarks. No new physical entities are introduced.

free parameters (8)
  • gamma (gamma) in mask-driven self-attention = not specified
    Scaling factor for mask-driven key modulation in Eq. (3); its value is not reported.
  • beta (weight of latent alignment loss) = 0.02
    Set in the supplementary material (Section III).
  • T_latent (max step for latent alignment loss) = 200
    Set in the supplementary material (Section III).
  • CFG scale = 3
    Classifier-free guidance scale during sampling (Section 5.1.3).
  • Diffusion steps = 100
    Number of sampling steps (Section 5.1.3).
  • K (condition frames between clips) = 3
    Number of overlapping frames in long video generation (Section 5.1.3).
  • S (frames per clip) = 29
    Clip length for generation (Section 5.1.3).
  • Training mask ratio range = 0.1 to 0.8
    Randomly chosen per video during training (supplementary Section III); affects learned behavior.
assumptions (5)
  • domain assumption The Open-Sora-Plan DiT backbone provides a strong pretrained video prior that can be adapted to outpainting.
    The method fine-tunes the pretrained T2V model; the central results depend on this prior.
  • domain assumption Injecting control features after the first DiT block, following ControlNeXt, is an effective conditioning strategy.
    Section 4.2; the paper adopts this design without a dedicated ablation comparing injection positions.
  • ad hoc to paper The mask-driven self-attention (Eq. 3) with key scaling in [-1,1] is beneficial for outpainting.
    The paper's main new mechanism; supported only by the ablation in Table 3.
  • domain assumption ShareGPT4V-generated captions are sufficient text conditions for the benchmark videos.
    Section 5.1.3; caption quality could affect text-conditioned generation, though all videos receive the same rough-caption pipeline.
  • domain assumption The evaluation protocol (mask ratios 0.25 and 0.66, blending input into output, resizing to 256x256, FVD on 16 sampled frames) is consistent with the baselines.
    The paper says it follows previous works but does not verify that the SDM numbers in Table 1 were produced under this exact protocol.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OutDreamer: Video Outpainting with a Diffusion Transformer." pith.science (2026). https://pith.science/paper/6HG7UWUN

@misc{pith2026250622298,
  author       = {Pith},
  title        = {Pith review of: OutDreamer: Video Outpainting with a Diffusion Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6HG7UWUN}},
  note         = {Machine review of arXiv:2506.22298}
}
read the original abstract

Video outpainting is a challenging task that generates new video content by extending beyond the boundaries of an original input video, requiring both temporal and spatial consistency. Many state-of-the-art methods utilize latent diffusion models with U-Net backbones but still struggle to achieve high quality and adaptability in generated content. Diffusion transformers (DiTs) have emerged as a promising alternative because of their superior performance. We introduce OutDreamer, a DiT-based video outpainting framework comprising two main components: an efficient video control branch and a conditional outpainting branch. The efficient video control branch effectively extracts masked video information, while the conditional outpainting branch generates missing content based on these extracted conditions. Additionally, we propose a mask-driven self-attention layer that dynamically integrates the given mask information, further enhancing the model's adaptability to outpainting tasks. Furthermore, we introduce a latent alignment loss to maintain overall consistency both within and between frames. For long video outpainting, we employ a cross-video-clip refiner to iteratively generate missing content, ensuring temporal consistency across video clips. Extensive evaluations demonstrate that our zero-shot OutDreamer outperforms state-of-the-art zero-shot methods on widely recognized benchmarks.

Figures

Figures reproduced from arXiv: 2506.22298 by the authors.

Figure 1
Figure 1. OutDreamer is a zero-shot video outpainting method based on the DiT backbone, producing high-quality missing content [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of OutDreamer. OutDreamer consists of two main components: an efficient video control branch for extracting essential video conditions and a conditional outpainting branch for generating missing content based on these conditions and iterative generation for long videos. The efficient video control branch has two versions: a basic version shown in the upper-left part and an advanced version in the upper-righ… view at source ↗
Figure 3
Figure 3. Mask-Driven Self-Attention. The green embeddings represent patches corresponding to the given region, while the gray embeddings represent patches corresponding to the outpainted re￾gion. W is the scaling factor vector for K, and the different gray levels of W represent the magnitudes of the scaling factor for K. where Θd′ represents the parameters of the first block of the backbone, Θd′ ⊆ Θd, Θc << Θd, Zt ∈ R h×w×s×… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Iterative long video generation. Starting from the sec￾ond video clip, each clip with total S frames utilizes the last K frames of the previously generated clip as condition information. This example shows the case for S = 6 and K = 2. mal outpainting masks. Similarly …
Figure 5
Figure 5. Figure 5: Qualitative comparison of video outpainting methods. The mask ratio is 0.66. We compare our method with Dehan, M3DDM, and MOTIA. SDM is not included since its code is not released. Zero-shot method DAVIS dataset YouTube-VOS dataset SSIM↑ PSNR↑ LPIPS↓ FVD↓ SSIM↑ PSNR↑ L…
Figure 6
Figure 6. Figure 6: A long video generation example. The mask ratio is 0.5. Method SSIM↑ PSNR↑ LPIPS↓ FVD↓ One-shot 0.7636 20.25 0.1727 58.99 MOTIA Zero-shot 0.7644 20.21 0.1827 56.02 OutDreamer [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Quantitative evaluation of long video generation. For each metric score (PSNR, SSIM, LPIPS, and FVD), we perform maximum normalization with proportional scaling, making the re￾sults more intuitive. 5.4. Long Video Generation We assess the performance of our iterative g…
Figure 8
Figure 8. Figure 8: Additional outpainting results. Left: input videos. Right: outpainting results generated by OutDreamer. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CameraAnything: Refilming Videos with Arbitrary Camera Control

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A video diffusion editor jointly controls extrinsic pose, multi-shot cuts, focal length, and native resolution via Plücker rays in resolution-aware 3D RoPE, trained on synthetic multi-camera pairs.

Reference graph

Works this paper leans on

50 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315–11325, 2022. 1

    work page 2022

  2. [2]

    Free-form video inpainting with 3d gated convolution and temporal patchgan

    Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Lee, and Winston Hsu. Free-form video inpainting with 3d gated convolution and temporal patchgan. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9066– 9075, 2019. 3

    work page 2019

  3. [3]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12793, 2023. 7

    arXiv 2023

  4. [4]

    Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    arXiv 2024

  5. [5]

    Sharegpt4video: Improving video understand- ing and generation with better captions

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Bin Lin, Zhenyu Tang, et al. Sharegpt4video: Improving video understand- ing and generation with better captions. arXiv preprint arXiv:2406.04325, 2024. 7

    arXiv 2024

  6. [6]

    Inout: Diverse image outpainting via gan inversion

    Yen-Chi Cheng, Chieh Hubert Lin, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, and Ming-Hsuan Yang. Inout: Diverse image outpainting via gan inversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11431–11440, 2022. 1, 3

    work page 2022

  7. [7]

    Complete and temporally consistent video out- painting

    Lo ¨ıc Dehan, Wiebe Van Ranst, Patrick Vandewalle, and Toon Goedem´e. Complete and temporally consistent video out- painting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 687–695,

    work page

  8. [8]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 3

    arXiv 2010

Show all 50 references
  1. [9]

    Hierar- chical masked 3d diffusion model for video outpainting

    Fanda Fan, Chaoxu Guo, Litong Gong, Biao Wang, Tiezheng Ge, Yuning Jiang, Chunjie Luo, and Jianfeng Zhan. Hierar- chical masked 3d diffusion model for video outpainting. In Proceedings of the 31st ACM International Conference on Multimedia, pages 7890–7900, 2023. 1, 3, 6

    2023

  2. [10]

    Flow-edge guided video completion

    Chen Gao, Ayush Saraf, Jia-Bin Huang, and Johannes Kopf. Flow-edge guided video completion. In European Confer- ence on Computer Vision , pages 713–729. Springer, 2020. 3

    2020

  3. [11]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 3

    2023 arXiv

  4. [12]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. 3

    2020

  5. [13]

    Proposal-based video com- pletion

    Yuan-Ting Hu, Heng Wang, Nicolas Ballas, Kristen Grau- man, and Alexander G Schwing. Proposal-based video com- pletion. In European Conference on Computer Vision, pages 38–54. Springer, 2020. 3

    2020

  6. [14]

    Diffusion model-based image editing: A survey

    Yi Huang, Jiancheng Huang, Yifan Liu, Mingfu Yan, Jiaxi Lv, Jianzhuang Liu, Wei Xiong, He Zhang, Shifeng Chen, and Liangliang Cao. Diffusion model-based image editing: A survey. arXiv preprint arXiv:2402.17525, 2024. 3

    2024 arXiv

  7. [15]

    Open-sora-plan, 2024

    PKU-Yuan Lab and Tuzhan AI etc. Open-sora-plan, 2024. 6

    2024

  8. [16]

    Srinpaintor: When super-resolution meets transformer for image inpainting

    Feng Li, Anqi Li, Jia Qin, Huihui Bai, Weisi Lin, Runmin Cong, and Yao Zhao. Srinpaintor: When super-resolution meets transformer for image inpainting. IEEE Transactions on Computational Imaging, 8:743–758, 2022. 3

    2022

  9. [17]

    Magiceraser: Eras- ing any objects via semantics-aware control

    Fan Li, Zixiao Zhang, Yi Huang, Jianzhuang Liu, Ren- jing Pei, Bin Shao, and Songcen Xu. Magiceraser: Eras- ing any objects via semantics-aware control. arXiv preprint arXiv:2410.10207, 2024. 3

    2024 arXiv

  10. [18]

    Context-aware semantic inpainting.IEEE Trans- actions on Cybernetics, 49(12):4398–4411, 2018

    Haofeng Li, Guanbin Li, Liang Lin, Hongchuan Yu, and Yizhou Yu. Context-aware semantic inpainting.IEEE Trans- actions on Cybernetics, 49(12):4398–4411, 2018. 3

    2018

  11. [19]

    Mat: Mask-aware transformer for large hole im- age inpainting

    Wenbo Li, Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, and Ji- aya Jia. Mat: Mask-aware transformer for large hole im- age inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10758– 10768, 2022. 3

    2022

  12. [20]

    Image inpainting via iteratively decoupled probabilistic modeling

    Wenbo Li, Xin Yu, Kun Zhou, Yibing Song, and Zhe Lin. Image inpainting via iteratively decoupled probabilistic modeling. In International Conference on Learning Repre- sentations, 2024. 3

    2024

  13. [21]

    Edge guided progressively generative image outpainting

    Han Lin, Maurice Pagnucco, and Yang Song. Edge guided progressively generative image outpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 806–815, 2021. 1

    2021

  14. [22]

    Fuseformer: Fusing fine-grained information in transformers for video inpainting

    Rui Liu, Hanming Deng, Yangyi Huang, Xiaoyu Shi, Lewei Lu, Wenxiu Sun, Xiaogang Wang, Jifeng Dai, and Hong- sheng Li. Fuseformer: Fusing fine-grained information in transformers for video inpainting. In Proceedings of the IEEE/CVF International Conference on Computer Vision , ...

    2021

  15. [23]

    Coordfill: Efficient high- resolution image inpainting via parameterized coordinate querying

    Weihuang Liu, Xiaodong Cun, Chi-Man Pun, Menghan Xia, Yong Zhang, and Jue Wang. Coordfill: Efficient high- resolution image inpainting via parameterized coordinate querying. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, pages 1746–1754, 2023. 3

    2023

  16. [24]

    9 Hd-painter: high-resolution and prompt-faithful text-guided image inpainting with diffusion models

    Hayk Manukyan, Andranik Sargsyan, Barsegh Atanyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. 9 Hd-painter: high-resolution and prompt-faithful text-guided image inpainting with diffusion models. arXiv preprint arXiv:2312.14091, 2023. 3

    2023 arXiv

  17. [25]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162–8171. PMLR,

  18. [26]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 4195–4205,

  19. [27]

    Controlnext: Powerful and effi- cient control for image and video generation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 1, 4

    2024 arXiv

  20. [28]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15932–15942, 2023. 3

    2023

  21. [29]

    Deep learning-based image and video inpainting: A survey

    Weize Quan, Jiaxi Chen, Yanli Liu, Dong-Ming Yan, and Pe- ter Wonka. Deep learning-based image and video inpainting: A survey. International Journal of Computer Vision, 132(7): 2367–2400, 2024. 3

    2024

  22. [30]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 3

    2022 arXiv

  23. [31]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022. 3

    2022

  24. [32]

    Palette: Image-to-image diffusion models

    Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings , pages 1– 10, 2022. 1

    2022

  25. [33]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3

    2010 arXiv

  26. [34]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In International Conference on Learning Represen- tations, 2021. 3

    2021

  27. [35]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. In Proceedings of the IEEE/CVF Win- ...

    2022

  28. [36]

    Be-your-outpainter: Mastering video outpainting through input-specific adaptation

    Fu-Yun Wang, Xiaoshi Wu, Zhaoyang Huang, Xiaoyu Shi, Dazhong Shen, Guanglu Song, Yu Liu, and Hongsheng Li. Be-your-outpainter: Mastering video outpainting through input-specific adaptation. In European Conference on Com- puter Vision, pages 153–168. Springer, 2025. 1, 3, 6

    2025

  29. [37]

    Zero-shot video editing using off-the-shelf image diffusion models

    Wen Wang, Yan Jiang, Kangyang Xie, Zide Liu, Hao Chen, Yue Cao, Xinlong Wang, and Chunhua Shen. Zero-shot video editing using off-the-shelf image diffusion models. arXiv preprint arXiv:2303.17599, 2023. 3

    2023 arXiv

  30. [38]

    Sketch-guided scenery image outpainting

    Yaxiong Wang, Yunchao Wei, Xueming Qian, Li Zhu, and Yi Yang. Sketch-guided scenery image outpainting. IEEE Transactions on Image Processing, 30:2643–2655, 2021. 1

    2021

  31. [39]

    Deep learning for im- age inpainting: A survey

    Hanyu Xiang, Qin Zou, Muhammad Ali Nawaz, Xianfeng Huang, Fan Zhang, and Hongkai Yu. Deep learning for im- age inpainting: A survey. Pattern Recognition, 134:109046,

  32. [40]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22428–22437, 2023. 3

    2023

  33. [41]

    Deep flow-guided video inpainting

    Rui Xu, Xiaoxiao Li, Bolei Zhou, and Chen Change Loy. Deep flow-guided video inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3723–3732, 2019. 3

    2019

  34. [42]

    High-resolution image inpainting us- ing multi-scale neural patch synthesis

    Chao Yang, Xin Lu, Zhe Lin, Eli Shechtman, Oliver Wang, and Hao Li. High-resolution image inpainting us- ing multi-scale neural patch synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6721–6729, 2017. 3

    2017

  35. [43]

    Scene graph expansion for semantics-guided image outpainting

    Chiao-An Yang, Cheng-Yo Tan, Wan-Cyuan Fan, Cheng-Fu Yang, Meng-Lin Wu, and Yu-Chiang Frank Wang. Scene graph expansion for semantics-guided image outpainting. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 15617–15626, 2022. 1

    2022

  36. [44]

    Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model

    Shiyuan Yang, Xiaodong Chen, and Jing Liao. Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model. In Proceedings of the 31st ACM International Conference on Multimedia, pages 3190–3199,

  37. [45]

    Flow-guided transformer for video inpainting

    Kaidong Zhang, Jingjing Fu, and Dong Liu. Flow-guided transformer for video inpainting. In European Conference on Computer Vision, pages 74–90. Springer, 2022. 3

    2022

  38. [46]

    Inertia-guided flow completion and style fusion for video inpainting

    Kaidong Zhang, Jingjing Fu, and Dong Liu. Inertia-guided flow completion and style fusion for video inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 5982–5991, 2022. 3

    2022

  39. [47]

    Avid: Any-length video inpainting with dif- fusion model

    Zhixing Zhang, Bichen Wu, Xiaoyan Wang, Yaqiao Luo, Luxin Zhang, Yinan Zhao, Peter Vajda, Dimitris Metaxas, and Licheng Yu. Avid: Any-length video inpainting with dif- fusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 71...

    2024

  40. [48]

    Large scale image comple- tion via co-modulated generative adversarial networks.arXiv preprint arXiv:2103.10428, 2021

    Shengyu Zhao, Jonathan Cui, Yilun Sheng, Yue Dong, Xiao Liang, Eric I Chang, and Yan Xu. Large scale image comple- tion via co-modulated generative adversarial networks.arXiv preprint arXiv:2103.10428, 2021. 3

    2021 arXiv

  41. [49]

    Propainter: Improving propagation and transformer for video inpainting

    Shangchen Zhou, Chongyi Li, Kelvin CK Chan, and Chen Change Loy. Propainter: Improving propagation and transformer for video inpainting. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10477–10486, 2023. 3 10

    2023

  42. [50]

    Cococo: Improving text-guided video inpainting for better consistency, controllability and compatibility.arXiv preprint arXiv:2403.12035, 2024

    Bojia Zi, Shihao Zhao, Xianbiao Qi, Jianan Wang, Yukai Shi, Qianyu Chen, Bin Liang, Kam-Fai Wong, and Lei Zhang. Cococo: Improving text-guided video inpainting for better consistency, controllability and compatibility.arXiv preprint arXiv:2403.12035, 2024. 3 11 I. Details of t...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.