Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that one blended image can be decomposed into its constituent foreground and background layers by a diffusion transformer that learns a posterior over possible decompositions, bypassing masks and direct…

desk verdict A genuinely new task and a substantial synthetic dataset, but the headline numbers rest on a single-sample evaluation of an ill-posed inverse problem, so read the margins with caution. read the letter →

arxiv 2505.21541 v3 pith:T4CVFB52 submitted 2025-05-24 cs.CV cs.AI

Zitong Wang , Hang Zhao , Qianyu Zhou , Xuequan Lu , Xiangtai Li , Yiren Song This is my paper
classification cs.CVcs.AI
keywords alphacompositinglayer-wisedecompositiontransparentobjectssemi-transparentocclusiondiffusiontransformerin-contextwatermarkremovalX-raycontraband
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a new task: given a single image formed by semi-transparent or transparent layers, recover the constituent foreground and background layers. It argues that mask-based inpainting cannot do this because transparency entangles color with alpha, and that the right framing is to learn a distribution over plausible decompositions rather than to inpaint masked regions. To support the task, it introduces AlphaBlend, a large synthetic dataset with six subtasks covering lens flares, translucent occlusion, watermarks, glassware, cells, and X-ray contraband, and DiffDecompose, a diffusion transformer that jointly generates both layers from the observed composite. If correct, this gives a mask-free way to edit transparent overlays in real photographs, with the paper reporting large average margins over inpainting baselines. The paper itself notes a residual limitation: without explicit per-layer supervision, recovered layers can show pixel-level drift that is visually imperceptible but not positionally exact.

What carries the argument

The carrying mechanism is In-Context Decomposition (ICD): the composited image is encoded into a clean condition latent, concatenated along the sequence dimension with separately noised foreground and background latents together with text tokens, and processed by bidirectional attention inside a diffusion transformer, so that both layers are generated conditioned on the observed image and a task prompt. Layer Position Encoding Cloning (LPEC) clones the positional encoding of the composite into the background and composite tokens while leaving the foreground in its own positional space, which keeps the background locked to the observed image coordinates and prevents spatial entanglement between layers. Training uses a conditional flow-matching objective over the joint velocity field of both layers.

What would settle it

Run the released model on a held-out set of real photographs with measured ground truth, for example a camera photo of a known background through a real glass pane or a real X-ray scan with a known concealed object, and compute RMSE and LPIPS against the known layers; if the error margins over mask-based inpainting collapse or the recovered layers show visible pixel drift at object boundaries, the claimed generalization would be falsified.

Watch

Extended reading notes

Core claim

DiffDecompose claims to recover constituent layers from single overlapped images under semi-transparent and transparent nonlinear $\alpha$ occlusion. Formally, given an observed composite $z = G(x, y)$, where $x$ is a foreground RGBA layer, $y$ is a background RGB layer, and $G$ is an unknown or variable blend operator such as $\alpha$ blending, additive, multiply, screen, or overlay, the model jointly predicts $x$ and $y$ without any mask prior or per-layer supervision. On the proposed AlphaBlend benchmark and the public LOGO watermark test sets, the paper reports that DiffDecompose outperforms the second-best methods by average margins of $36.3\%$ in RMSE, $+1.2\%$ in SSIM, and $52.8\%$ in LPIPS, and that its layer-position encoding component is responsible for large ablated gains in tasks such as X-ray contraband separation and transparent glassware decomposition.

Load-bearing premise

The load-bearing premise is that the synthetic alpha-compositing formulas used to create AlphaBlend (alpha blending, additive, multiply, and screen modes) capture real transparent and semi-transparent occlusion well enough that a model trained on them transfers to real photographs.

Editorial extensions

If this is right

  • Mask-free decomposition becomes usable for real editing: watermarks, lens flares, window condensation, glassware, cells, and X-ray overlays can be separated without a user-provided mask.
  • One trained model spans both global layer-level effects and localized object-level transparency, because all six subtasks share the same composition-conditioned posterior formulation.
  • Nonlinear blend modes beyond straight alpha can be inverted by learning the posterior rather than by fitting an explicit composition formula.
  • The model also produces usable foreground RGBA layers, not just a cleaned background, which supports downstream matting, compositing, and content reuse.
  • On public LOGO test sets, the mask-free method reports lower RMSE and LPIPS than mask-based inpainting baselines, with the single exception of a slightly higher FID on LOGO-L.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the synthetic-to-real transfer holds, the same ICD/LPEC design could be applied to other ill-posed layer inversions such as reflection separation, dehazing, or video layer decomposition, where dense per-layer ground truth is scarce.
  • Beyond the paper: because the reported margins are measured on AlphaBlend test images generated by the same composition formulas used for training, the benchmark may overstate real-world gains; a paired real-capture benchmark would settle the gap.
  • Beyond the paper: since the model samples from a posterior, repeated sampling could quantify decomposition ambiguity, which is itself useful signal for how much an observed composite constrains its layers.
  • Beyond the paper: the stated pixel-drift limitation suggests a cheap extension, namely a reconstruction consistency term that re-composes the predicted layers through $G$ and penalizes deviation from the observed $z$, reducing drift without adding masks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript introduces a new task, Layer-Wise Decomposition of Alpha-Composited Images, in which a single RGB image formed by compositing a semi-transparent or transparent foreground over a background must be separated into its constituent layers. The authors contribute the AlphaBlend dataset, a synthetic resource spanning six subtasks with distinct composition operators (additive, screen-like, alpha blending, and piecewise blends), and propose DiffDecompose, a diffusion-Transformer framework fine-tuned from FLUX that learns a posterior over plausible decompositions via In-Context Decomposition and Layer Position Encoding Cloning. The paper reports large quantitative gains over mask-based inpainting baselines on AlphaBlend and on the public LOGO watermark-removal benchmarks, along with qualitative results, ablations, and a user study.

Significance. If the quantitative claims held, the paper would make a useful contribution: the task definition is timely, the probabilistic framing is appropriate for an ill-posed inverse problem, and the promise of a large-scale dataset plus code could support further work. The paper also includes explicit ablations of LPEC and ICD and a user study, which are helpful complements to the automatic metrics. However, the current evaluation protocol does not substantiate the central claim of large margins over the state of the art: the main benchmarks are generated with the same formulas used for training, the baselines do not include prior layer-decomposition methods, and the metrics are mismatched to the ill-posed nature of the task. The significance is therefore conditional on a substantially strengthened evaluation.

major comments (4)
  1. [Sec. 4.2 and Eq. (2)] The evaluation protocol is mismatched to the problem definition. The paper defines the task as learning a posterior pθ(x, y | z, τ) over a 'highly ill-posed' inverse problem, but Table 1 and Table 3 report point metrics (RMSE, SSIM, LPIPS, FID) between a single sampled output and the single ground-truth pair used to synthesize each test image. No sampling repetitions, no multiple-sample statistics, and no recomposition-consistency check (recomposing the predicted layers with the known blending operator and comparing to the input) are reported. For non-injective composition functions such as the additive model I = A + B used in the cell subtask, shifting a constant between layers produces a valid decomposition that can incur arbitrarily large RMSE against the original pair. The headline '36.3% RMSE' margin may therefore penalize valid alternative decompositions rather than measure decomposition quality. This issue is load-bearing and should be addressed with solution-set-aware or recomposition-based metrics, variance reporting across samples, or a reframing of the claimed margins.
  2. [Sec. 4.1 Baselines] The comparison set includes only mask-based inpainting methods. The paper's own Related Work cites layer-decomposition methods such as Layer Diffusion [71] and Generative Image Layer Decomposition [64], which are the most direct competitors for the proposed task, yet none is included in any table or figure. Without these baselines, the claim that DiffDecompose 'outperforms the second-best methods' is only established against inpainting models, not against the existing layer-decomposition literature. The authors should add such methods to Table 1 and to the AlphaBlend evaluation where feasible, or explicitly justify their exclusion.
  3. [Secs. 3.1 and 4.2] The core quantitative evaluation on AlphaBlend is conducted on test images synthesized with the same task-specific composition formulas used to create the training data. This does not test generalization to real semi-transparent or transparent scenes. The only real-image evaluation is the LOGO watermark-removal benchmark, which is a single linear alpha-blending task and does not cover the nonlinear operators (X-ray, glass, cell, flare, occlusion) that are central to the paper's claims. The conclusion that the method 'generalizes to real-world transparent and semi-transparent layer decomposition scenarios' is therefore unsupported. At minimum, the authors should add real-image qualitative results for several subtasks and, where possible, real-image metrics, or restrict the generalization claims accordingly.
  4. [Section E, Limitations, and Table 3] The limitations section states that the model suffers from 'pixel drift' and 'lack pixel-level position accuracy' in the separated layers. This is in direct tension with Table 1 and Table 3, which report small RMSE and near-perfect SSIM on synthetic pixel-aligned ground truth, and with the abstract's claim of accurate decomposition. Additionally, Table 3 reports foreground-separation metrics for DiffDecompose only, with no competing method for comparison, so it cannot support a superiority claim. The authors should reconcile the stated limitation with the pixel-level metrics, or explicitly state that the synthetic metrics are not indicative of real-world pixel accuracy, and should either add comparison methods to Table 3 or clarify its purpose.
minor comments (6)
  1. [Sec. 3.2, Eq. (1)] Equation (1) contains the incomplete sentence fragment 'The goal is to' and uses an undefined z0 in the integral; the notation should be cleaned up so that the formulation is self-contained.
  2. [Abstract and Sec. 1] The abstract and Section 1 both promise code and dataset release, and the abstract contains two nearly identical 'available' sentences; the dataset URL is missing, so please unify the statements and provide a single link.
  3. [Sec. 4.3, Table 2] The ablation text says 'after studying the LPEC' rather than 'after ablating LPEC' and the reported improvements for translucent occlusion (2.3/0.0242/0.0227) are presented as if they were independent of the full model; please align the wording and clarify that these are gains from adding LPEC to the baseline.
  4. [Sec. 3.3 and Fig. 3] The abbreviation MMA is used in the caption and in Eq. (4) before 'Multi-Modality Attention' is explicitly defined in the text; define the term at first use and ensure the caption notation matches the text.
  5. [Appendix D.1] The metric descriptions refer to 'correct object removal' and 'inpainted image distributions', which are inconsistent with the layer-decomposition framing of the paper; adapt the descriptions to the actual task.
  6. [References] Reference [28] lacks an author list, and reference [18] is listed twice with duplicated entries; please clean up the bibliography.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DiffDecompose's central claim is an empirical evaluation on a held-out synthetic benchmark plus a public LOGO benchmark; no fitted parameter is renamed as a prediction and no load-bearing self-citation is present.

full rationale

The paper's derivation chain is a standard supervised learning pipeline. AlphaBlend is constructed by explicit composition formulas (Sec. 3.1), and DiffDecompose is trained with a conditional flow matching objective (Eq. 5) without fitting any parameter to the test labels or to the reported metrics. The headline margins are measured on a held-out test split of AlphaBlend and on the public LOGO dataset, against external baselines; hence the numbers are not forced by construction. The formulation p_theta(x, y | z, tau) (Eq. 2) is a posterior over decompositions, and the ill-posedness is acknowledged, but acknowledging ill-posedness is not circular. Self-citations in Related Work (e.g., [20, 48, 54, 59, 60, 65]) are contextual literature references, not load-bearing uniqueness theorems or ansatz justifications. The main weakness is external validity: the AlphaBlend test set is generated with the same formulas as training, so strong in-domain numbers do not by themselves establish real-world generalization. That is a benchmarking and generalization concern, not a circularity in the paper's reasoning, and it is partially mitigated by the public LOGO evaluation and the admitted limitation about pixel drift. Under the rule that circularity requires exhibiting a specific reduction of a claimed result to its own inputs, no such reduction is present.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest on the synthetic composition model, the frozen VAE representation, and the implicit learning of composition consistency. No new physical entities are introduced; the free parameters are dataset design choices rather than scientific constants.

free parameters (3)
  • Task-specific blending operator for each subtask = X-ray: I=(1-alpha)*B + alpha*(A_alpha/255*B/255*255); glass: multiplicative/screen; cells: additive; etc.
    The dataset's composition functions are hand-designed to simulate different physical phenomena. The model is trained and evaluated on data produced with these exact formulas, so its reported performance is coupled to these modeling choices.
  • Watermark size and opacity ranges = size 96-128, opacity below 0.25
    Appendix A states these ranges were chosen to make the watermark removal task harder than prior datasets.
  • Training set size per subtask = 5000-10000 training images, 300-500 test images per task
    Dataset scale is a design choice; the paper claims it is large-scale.
assumptions (3)
  • domain assumption The synthetic composition formulas in AlphaBlend faithfully model real-world semi-transparent and transparent occlusion phenomena.
    The entire evaluation and generalization claim rests on the assumption that additive, screen, multiplicative, and alpha blending formulas reproduce real transparency well enough that a model trained on them transfers to real images. This is stated in Sec. 3.1 but not validated against real data beyond the LOGO watermark set.
  • domain assumption The Flux VAE latent space preserves the information needed to separate layers.
    DiffDecompose operates entirely in the frozen VAE latent space (Sec. 3.3). If the VAE discards alpha-related or fine-grained spatial information, layer separation would be impossible.
  • domain assumption Conditional flow matching with independent noise on foreground and background latents can learn the joint posterior p(x,y|z).
    The training loss (Eq. 5) treats xt and yt as independent noisy latent variables; the composition consistency G(x,y)=z is not enforced in the loss, only learned implicitly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers." pith.science (2026). https://pith.science/paper/T4CVFB52

@misc{pith2026250521541,
  author       = {Pith},
  title        = {Pith review of: DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T4CVFB52}},
  note         = {Machine review of arXiv:2505.21541}
}
read the original abstract

Diffusion models have recently motivated great success in many generation tasks like object removal. Nevertheless, existing image decomposition methods struggle to disentangle semi-transparent or transparent layer occlusions due to mask prior dependencies, static object assumptions, and the lack of datasets. In this paper, we delve into a novel task: Layer-Wise Decomposition of Alpha-Composited Images, aiming to recover constituent layers from single overlapped images under the condition of semi-transparent/transparent alpha layer non-linear occlusion. To address challenges in layer ambiguity, generalization, and data scarcity, we first introduce AlphaBlend, the first large-scale and high-quality dataset for transparent and semi-transparent layer decomposition, supporting six real-world subtasks (e.g., translucent flare removal, semi-transparent cell decomposition, glassware decomposition). Building on this dataset, we present DiffDecompose, a diffusion Transformer-based framework that learns the posterior over possible layer decompositions conditioned on the input image, semantic prompts, and blending type. Rather than regressing alpha mattes directly, DiffDecompose performs In-Context Decomposition, enabling the model to predict one or multiple layers without per-layer supervision, and introduces Layer Position Encoding Cloning to maintain pixel-level correspondence across layers. Extensive experiments on the proposed AlphaBlend dataset and public LOGO dataset verify the effectiveness of DiffDecompose. The code and dataset will be available upon paper acceptance. Our code will be available at: https://github.com/Wangzt1121/DiffDecompose.

Figures

Figures reproduced from arXiv: 2505.21541 by the authors.

Figure 1
Figure 1. We propose a novel generative task, Layer-Wise Decomposition of Alpha-Composited [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The comparison of conventional inpainting methods with our proposed DiffDecompose. The [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The DiffDecompose framework comprises two steps: (1) VAE encodes the source image [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Our DiffDecompose shows impressive layer-level decomposition results of the image. It can [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons between our method and other methods. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The visualization of LPEC ablation. Full [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: The presentation of the six subtasks’ dataset. Each foreground has its respective properties. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: The SDXL-inpainting results under different strengths. We input the prompt like "Remove [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Qualitative results of our and compared methods on the proposed AlphaBlend dataset [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Additional Qualitative Results for Subtasks I–III. Extended visualizations demonstrating [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Additional Qualitative Results for Subtasks IV–VI. Extended results showcasing DiffDe [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: User study results. The voting results of DiffDecompose and the baseline method are [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: A user study voting interface was provided to participants. We present the performance of [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Results of failed decomposition by DiffDecompose. [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generation

    cs.CV 2025-11 conditional novelty 5.0 of 10

    A bidirectional egocentric-to-exocentric video translation framework trained with in-context attention on a new synthetic+real dataset, with evaluation flaws around reference leakage and missing direct baselines.

Reference graph

Works this paper leans on

77 extracted references · 49 canonical work pages · cited by 1 Pith paper

  1. [71]

    Transparent image layer diffusion using latent transparency

    Lvmin Zhang and Maneesh Agrawala. Transparent image layer diffusion using latent transparency. arXiv preprint arXiv:2402.17113, 2024

    arXiv 2024

  2. [64]

    Generative image layer decomposition with visual effects

    Jinrui Yang, Qing Liu, Yijun Li, Soo Ye Kim, Daniil Pakhomov, Mengwei Ren, Jianming Zhang, Zhe Lin, Cihang Xie, and Yuyin Zhou. Generative image layer decomposition with visual effects. arXiv preprint arXiv:2411.17864, 2024

    arXiv 2024

  3. [1]

    Machine learning approach of automatic identification and counting of blood cells

    Mohammad Mahmudul Alam and Mohammad Tariqul Islam. Machine learning approach of automatic identification and counting of blood cells. Healthcare Technology Letters, 6(4):103–108, 2019

    work page 2019

  4. [2]

    Segdiff: Image segmentation with diffusion probabilistic models

    Tomer Amit, Tal Shaharbany, Eliya Nachmani, and Lior Wolf. Segdiff: Image segmentation with diffusion probabilistic models. arXiv preprint arXiv:2112.00390, 2021

    arXiv 2021

  5. [3]

    Layer-animate for transparent video generation

    Jingqi Bai, Jingkai Zhou, Benzhi Wang, Weihua Chen, Yang Yang, Zhen Lei, and Fan Wang. Layer-animate for transparent video generation. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1–5, 2025

    work page 2025

  6. [4]

    Sequential modeling enables scalable learning for large vision models

    Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan L Yuille, Trevor Darrell, Jitendra Malik, and Alexei A Efros. Sequential modeling enables scalable learning for large vision models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22861–22872, 2024

    work page 2024

  7. [5]

    Lumiere: A space-time diffusion model for video generation

    Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, et al. Lumiere: A space-time diffusion model for video generation. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024

    2024

  8. [6]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science, 2(3):8, 2023

    work page 2023

Show all 77 references
  1. [7]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023

    2023

  2. [8]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901, 2020

    1901

  3. [9]

    Inverse painting: Reconstructing the painting process

    Bowei Chen, Yifan Wang, Brian Curless, Ira Kemelmacher-Shlizerman, and Steven M Seitz. Inverse painting: Reconstructing the painting process. In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024

    2024

  4. [10]

    Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Zhongdao Wang, James T. Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In The Twelfth International Conference on Learning R...

    2024

  5. [11]

    Edit transfer: Learning image editing via vision in-context relations

    Lan Chen, Qi Mao, Yuchao Gu, and Mike Zheng Shou. Edit transfer: Learning image editing via vision in-context relations. arXiv preprint arXiv:2503.13327, 2025

    2025 arXiv

  6. [12]

    Zero-shot image editing with reference imitation.Advances in Neural Information Processing Systems, 37:84010–84032, 2024

    Xi Chen, Yutong Feng, Mengting Chen, Yiyang Wang, Shilong Zhang, Yu Liu, Yujun Shen, and Heng- shuang Zhao. Zero-shot image editing with reference imitation.Advances in Neural Information Processing Systems, 37:84010–84032, 2024

    2024

  7. [13]

    Transanimate: Taming layer diffusion to generate rgba video

    Xuewei Chen, Zhimin Chen, and Yiren Song. Transanimate: Taming layer diffusion to generate rgba video. arXiv preprint arXiv:2503.17934, 2025

    2025 arXiv

  8. [14]

    An improved method for semantic image inpainting with gans: progressive inpainting

    Yizhen Chen and Haifeng Hu. An improved method for semantic image inpainting with gans: progressive inpainting. Neural Processing Letters, 49:1355–1367, 2019

    2019

  9. [15]

    Split then refine: stacked attention-guided resunets for blind single image visible watermark removal

    Xiaodong Cun and Chi-Man Pun. Split then refine: stacked attention-guided resunets for blind single image visible watermark removal. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35-2, pages 1184–1192, 2021

    2021

  10. [16]

    Enhance image-to-image generation with llava- generated prompts

    Zhicheng Ding, Panfeng Li, Qikai Yang, and Siyang Li. Enhance image-to-image generation with llava- generated prompts. In 5th International Conference on Information Science, Parallel and Distributed Systems, pages 77–81, 2024

    2024

  11. [17]

    Clipaway: Harmonizing focused embeddings for removing objects via diffusion models

    Yi˘git Ekin, Ahmet Burak Yildirim, Erdem Eren Ça˘glar, Aykut Erdem, Erkut Erdem, and Aysegul Dundar. Clipaway: Harmonizing focused embeddings for removing objects via diffusion models. Advances in Neural Information Processing Systems, 37:17572–17601, 2024

    2024

  12. [18]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthes...

    2024

  13. [19]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthes...

    2024

  14. [20]

    Explore in-context learning for 3d point cloud understanding

    Zhongbin Fang, Xiangtai Li, Xia Li, Joachim M Buhmann, Chen Change Loy, and Mengyuan Liu. Explore in-context learning for 3d point cloud understanding. Advances in Neural Information Processing Systems, 36:42382–42395, 2023

    2023

  15. [21]

    Factormatte: Redefining video matting for re- composition tasks

    Zeqi Gu, Wenqi Xian, Noah Snavely, and Abe Davis. Factormatte: Redefining video matting for re- composition tasks. ACM Transactions on Graphics, 42(4):1–14, 2023

    2023

  16. [22]

    Diffusioninst: Diffusion model for instance segmentation

    Zhangxuan Gu, Haoxing Chen, and Zhuoer Xu. Diffusioninst: Diffusion model for instance segmentation. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 2730–2734, 2024

    2024

  17. [23]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems, 30, 2017

    2017

  18. [24]

    Root mean square error (rmse) or mean absolute error (mae): When to use them or not

    Timothy O Hodson. Root mean square error (rmse) or mean absolute error (mae): When to use them or not. Geoscientific Model Development Discussions, 2022:1–10, 2022

    2022

  19. [25]

    Hq-edit: A high-quality dataset for instruction-based image editing

    Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang, Yuyin Zhou, and Cihang Xie. Hq-edit: A high-quality dataset for instruction-based image editing. arXiv preprint arXiv:2404.09990, 2024

    2024 arXiv

  20. [26]

    Smarteraser: Remove anything from images using masked-region guidance

    Longtao Jiang, Zhendong Wang, Jianmin Bao, Wengang Zhou, Dongdong Chen, Lei Shi, Dong Chen, and Houqiang Li. Smarteraser: Remove anything from images using masked-region guidance. arXiv preprint arXiv:2501.08279, 2025

    2025 arXiv

  21. [27]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, 2014

    2014

  22. [28]

    Black Forest Labs. Flux. https://github.com/black-forest-labs/flux , 2024

    2024

  23. [29]

    Exploiting diffusion prior for real-world image dehazing with unpaired training

    Yunwei Lan, Zhigao Cui, Chang Liu, Jialun Peng, Nian Wang, Xin Luo, and Dong Liu. Exploiting diffusion prior for real-world image dehazing with unpaired training. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39-4, pages 4455–4463, 2025

    2025

  24. [30]

    Generative omnimatte: Learning to decompose video into layers

    Yao-Chih Lee, Erika Lu, Sarah Rumbley, Michal Geyer, Jia-Bin Huang, Tali Dekel, and Forrester Cole. Generative omnimatte: Learning to decompose video into layers. arXiv preprint arXiv:2411.16683, 2024

    2024 arXiv

  25. [31]

    Garmentdiffusion: 3d garment sewing pattern generation with multimodal diffusion transformers

    Xinyu Li, Qi Yao, and Yuanda Wang. Garmentdiffusion: 3d garment sewing pattern generation with multimodal diffusion transformers. arXiv preprint arXiv:2504.21476, 2025

    2025

  26. [32]

    Nighthaze: Nighttime image dehazing via self-prior learning

    Beibei Lin, Yeying Jin, Yan Wending, Wei Ye, Yuan Yuan, and Robby T Tan. Nighthaze: Nighttime image dehazing via self-prior learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39-5, pages 5209–5217, 2025

    2025

  27. [33]

    Partial convolution for padding, inpainting, and image synthesis

    Guilin Liu, Aysegul Dundar, Kevin J Shih, Ting-Chun Wang, Fitsum A Reda, Karan Sapra, Zhiding Yu, Xiaodong Yang, Andrew Tao, and Bryan Catanzaro. Partial convolution for padding, inpainting, and image synthesis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4...

    2022

  28. [34]

    Image inpainting for irregular holes using partial convolutions

    Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European Conference on Computer Vision, pages 85–100, 2018

    2018

  29. [35]

    Efficient portrait matte creation with layer diffusion and connectivity priors

    Zhiyuan Lu, Hao Lu, and Hua Huang. Efficient portrait matte creation with layer diffusion and connectivity priors. arXiv preprint arXiv:2501.16147, 2025

    2025 arXiv

  30. [36]

    Image segmentation using deep learning: A survey

    Shervin Minaee, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos. Image segmentation using deep learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3523–3542, 2021

    2021

  31. [37]

    Model see model do: Speech-driven facial animation with style control

    Yifang Pan, Karan Singh, and Luiz Gustavo Hafemann. Model see model do: Speech-driven facial animation with style control. arXiv preprint arXiv:2505.01319, 2025

    2025 arXiv

  32. [38]

    Context encoders: Feature learning by inpainting

    Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2536–2544, 2016. 12

    2016

  33. [39]

    Sdxl inpainting

    Suraj Patil. Sdxl inpainting. https://huggingface.co/spaces/diffusers/ stable-diffusion-xl-inpainting/tree/main , 2024

    2024

  34. [40]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Cision, pages 4195–4205, 2023

    2023

  35. [41]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020

    2020

  36. [42]

    Encoding in style: a stylegan encoder for image-to-image translation

    Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen- Or. Encoding in style: a stylegan encoder for image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2287–2296, 2021

    2021

  37. [43]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022

    2022

  38. [44]

    Foodfusion: A novel approach for food image composition via diffusion models

    Chaohua Shi, Xuan Wang, Si Shi, Xule Wang, Mingrui Zhu, Nannan Wang, and Xinbo Gao. Foodfusion: A novel approach for food image composition via diffusion models. arXiv preprint arXiv:2408.14135, 2024

    2024 arXiv

  39. [45]

    Learning to generate images with perceptual similarity metrics

    Jake Snell, Karl Ridgeway, Renjie Liao, Brett D Roads, Michael C Mozer, and Richard S Zemel. Learning to generate images with perceptual similarity metrics. In 2017 IEEE International Conference on Image Processing, pages 4277–4281. IEEE, 2017

    2017

  40. [46]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In9th International Conference on Learning Representations, 2021

    2021

  41. [47]

    Insert anything: Image insertion via in-context editing in dit

    Wensong Song, Hong Jiang, Zongxing Yang, Ruijie Quan, and Yi Yang. Insert anything: Image insertion via in-context editing in dit. arXiv preprint arXiv:2504.15009, 2025

    2025 arXiv

  42. [48]

    Makeanything: Harnessing diffusion transformers for multi-domain procedural sequence generation

    Yiren Song, Cheng Liu, and Mike Zheng Shou. Makeanything: Harnessing diffusion transformers for multi-domain procedural sequence generation. arXiv preprint arXiv:2502.01572, 2025

    2025 arXiv

  43. [49]

    Alpha-clip: A clip model focusing on wherever you want

    Zeyi Sun, Ye Fang, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. Alpha-clip: A clip model focusing on wherever you want. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13019–13029, 2024

    2024

  44. [50]

    Ominicontrol: Minimal and universal control for diffusion transformer

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint arXiv:2411.15098, 2024

    2024 arXiv

  45. [51]

    Flux.1-dev-controlnet-inpainting-alpha

    AlimamaCreative Team. Flux.1-dev-controlnet-inpainting-alpha. https://github.com/ alimama-creative/FLUX-Controlnet-Inpainting.git , 2024

    2024

  46. [52]

    Anywhere: A multi-agent framework for user-guided, reliable, and diverse foreground-conditioned image generation

    Xie Tianyidan, Rui Ma, Qian Wang, Xiaoqian Ye, Feixuan Liu, Ying Tai, Zhenyu Zhang, Lanjun Wang, and Zili Yi. Anywhere: A multi-agent framework for user-guided, reliable, and diverse foreground-conditioned image generation. In Proceedings of the AAAI Conference on Artificial I...

    2025

  47. [53]

    Mulan: A multi layer annotated dataset for controllable text-to-image generation

    Petru-Daniel Tudosiu, Yongxin Yang, Shifeng Zhang, Fei Chen, Steven McDonagh, Gerasimos Lampouras, Ignacio Iacobacci, and Sarah Parisot. Mulan: A multi layer annotated dataset for controllable text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Visi...

    2024

  48. [54]

    Explore in-context segmentation via latent diffusion models

    Chaoyang Wang, Xiangtai Li, Henghui Ding, Lu Qi, Jiangning Zhang, Yunhai Tong, Chen Change Loy, and Shuicheng Yan. Explore in-context segmentation via latent diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 7545–7553, 2025

    2025

  49. [55]

    Semflow: Binding semantic segmentation and image synthesis via rectified flow

    Chaoyang Wang, Xiangtai Li, Lu Qi, Henghui Ding, Yunhai Tong, and Ming-Hsuan Yang. Semflow: Binding semantic segmentation and image synthesis via rectified flow. Advances in Neural Information Processing Systems, 37:138981–139001, 2024

    2024

  50. [56]

    A universal image quality index

    Zhou Wang and Alan C Bovik. A universal image quality index. IEEE Signal Srocessing Letters , 9(3):81–84, 2002

    2002

  51. [57]

    Occluded prohibited items detection: An x-ray security inspection benchmark and de-occlusion attention module

    Yanlu Wei, Renshuai Tao, Zhangjie Wu, Yuqing Ma, Libo Zhang, and Xianglong Liu. Occluded prohibited items detection: An x-ray security inspection benchmark and de-occlusion attention module. InProceedings of the 28th ACM International Conference on Multimedia, pages 138–146, 2020. 13

    2020

  52. [58]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  53. [59]

    Towards language-driven video inpainting via multimodal large language models

    Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li, Kai Chen, Yunhai Tong, Ziwei Liu, et al. Towards language-driven video inpainting via multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Visio...

    2024

  54. [60]

    Motionbooth: Motion-aware customized text-to-video generation

    Jianzong Wu, Xiangtai Li, Yanhong Zeng, Jiangning Zhang, Qianyu Zhou, Yining Li, Yunhai Tong, and Kai Chen. Motionbooth: Motion-aware customized text-to-video generation. arXiv preprint arXiv:2406.17758, 2024

    2024 arXiv

  55. [61]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22428–22437, 2023

    2023

  56. [62]

    Ufogen: You forward once large scale text-to- image generation via diffusion gans

    Yanwu Xu, Yang Zhao, Zhisheng Xiao, and Tingbo Hou. Ufogen: You forward once large scale text-to- image generation via diffusion gans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8196–8206, 2024

    2024

  57. [63]

    Pixelhacker: Image inpainting with structural and semantic consistency

    Ziyang Xu, Kangsheng Duan, Xiaolei Shen, Zhifeng Ding, Wenyu Liu, Xiaohu Ruan, Xiaoxin Chen, and Xinggang Wang. Pixelhacker: Image inpainting with structural and semantic consistency. arXiv preprint arXiv:2504.20438, 2025

    2025 arXiv

  58. [65]

    Unified dense prediction of video diffusion

    Lehan Yang, Lu Qi, Xiangtai Li, Sheng Li, Varun Jampani, and Ming-Hsuan Yang. Unified dense prediction of video diffusion. arXiv preprint arXiv:2503.09344, 2025

    2025 arXiv

  59. [66]

    Layerpano3d: Layered 3d panorama for hyper-immersive scene generation

    Shuai Yang, Jing Tan, Mengchen Zhang, Tong Wu, Yixuan Li, Gordon Wetzstein, Ziwei Liu, and Dahua Lin. Layerpano3d: Layered 3d panorama for hyper-immersive scene generation. arXiv preprint arXiv:2408.13252, 2024

    2024 arXiv

  60. [67]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  61. [68]

    Anyedit: Mastering unified high-quality image editing for any idea

    Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. Anyedit: Mastering unified high-quality image editing for any idea. arXiv preprint arXiv:2411.15738, 2024

    2024 arXiv

  62. [69]

    Inpaint anything: Segment anything meets image inpainting

    Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, and Zhibo Chen. Inpaint anything: Segment anything meets image inpainting. arXiv preprint arXiv:2304.06790, 2023

    2023 arXiv

  63. [70]

    Promptfix: You prompt and we fix the photo

    Yongsheng Yu, Ziyun Zeng, Hang Hua, Jianlong Fu, and Jiebo Luo. Promptfix: You prompt and we fix the photo. arXiv preprint arXiv:2405.16785, 2024

    2024 arXiv

  64. [72]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023

    2023

  65. [73]

    What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems, 36:17773–17794, 2023

    Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems, 36:17773–17794, 2023

    2023

  66. [74]

    In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer

    Zechuan Zhang, Ji Xie, Yu Lu, Zongxin Yang, and Yi Yang. In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer. arXiv preprint arXiv:2504.20690, 2025

    2025 arXiv

  67. [75]

    Fast and robust segmentation of white blood cell images by self-supervised learning

    Xin Zheng, Yong Wang, Guoyou Wang, and Jianguo Liu. Fast and robust segmentation of white blood cell images by self-supervised learning. Micron, 107:55–71, 2018

    2018

  68. [76]

    Magicvideo: Efficient video generation with latent diffusion models

    Daquan Zhou, Weimin Wang, Hanshu Yan, Weiwei Lv, Yizhe Zhu, and Jiashi Feng. Magicvideo: Efficient video generation with latent diffusion models. arXiv preprint arXiv:2211.11018, 2022. 14

    2022 arXiv

  69. [77]

    Remove the light and make the scenario darker

    Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. In European Conference on Computer Vision, pages 195–211, 2024. 15 Overview In the Appendix, we first present mor...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.