Pith. sign in

REVIEW 2 major objections 6 minor 57 references

SDMatte: Grafting Diffusion Models for Interactive Matting

T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SDMatte claims that grafting Stable Diffusion's generative priors onto interactive matting yields finer edge detail and state-of-the-art alpha mattes from point, box, or mask prompts.

desk verdict Solid idea, code, and broad experiments, but the SOTA claim is undercut by an unreported mismatch between ablation tables and Table 1, plus a privileged opacity cue. read the letter →

arxiv 2508.00443 v2 pith:DHR6XIOX submitted 2025-08-01 cs.CV

Longfei Huang , Yu Liang , Hao Zhang , Jinwei Chen , Wei Dong , Lunde Chen , Wanyu Liu , Bo Li
show 1 more author
Peng-Tao Jiang
This is my paper
classification cs.CV
keywords interactivemattingalphamattediffusionmodelsvisualpromptconditioningcross-attentionmaskedself-attentionopacityembeddingStable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SDMatte is a method for interactive image matting—cutting a foreground object out of a photo while preserving delicate semi-transparent edges such as hair, fur, and glass—using simple user prompts: a point, a bounding box, or a mask. The paper sets out to show that the generative priors of a large text-to-image diffusion model can be repurposed for this task and that they outperform existing interactive matting methods, whose fine-edge predictions degrade on transparent and detailed objects. To make the transfer work, the authors replace text conditioning with a visual-prompt embedding in cross-attention, feed coordinate and opacity embeddings into the U-Net in place of the time embedding, and add a masked self-attention mechanism that concentrates the model on prompt-specified regions. They report consistently lower error on AIM-500, AM-2K, P3M-500-NP, and RefMatte-RW-100 across all prompt types, and describe a lightweight variant that keeps most of the gain at a fraction of the compute. The reason to care is that interactive matting fails precisely where diffusion models are strong, namely in fine, translucent boundary structure.

What carries the argument

The load-bearing machinery is a conditioning swap plus two attention modifications inside a one-step deterministic diffusion pipeline. First, visual prompt-driven cross-attention: a zero-convolution layer maps the latent encoding of the point, box, or mask prompt to the dimension of the text embedding and feeds it into the middle-block cross-attention, converting Stable Diffusion's text-driven interaction into prompt-driven interaction while reusing the pre-trained weights. Second, opacity and coordinate embeddings: sinusoidal position encodings of the prompt coordinates and of the object's annotated opacity (0 for transparent, 1 for opaque) are linearly combined and substituted for the time embedding, which is meaningless once denoising is removed. Third, masked self-attention: hard binary masks for box and mask prompts and a Gaussian soft mask for point prompts are added to the self-attention logits so the model focuses on prompt-specified areas. The one-step deterministic paradigm removes noise addition and denoising, so the pre-trained U-Net is supervised directly in pixel space.

What would settle it

Run SDMatte on AIM-500 or RefMatte-RW-100 with the opacity embedding disabled or set to a guessed value and compare MSE and SAD with the reported numbers; if the gap over SmartMatting and SEMat shrinks or reverses, the claimed advantage depends on the oracle opacity cue rather than on the prompt interaction mechanisms.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a latent diffusion model trained for text-to-image generation can be grafted into an interactive matting system without sacrificing either its generative priors or its interaction mechanism. SDMatte keeps Stable Diffusion v2's U-Net and VAE, drops the noise schedule for a one-step deterministic pass, and swaps the text embedding for an embedding of the user's visual prompt, which a zero-convolution layer projects into the cross-attention of the U-Net's middle block. Coordinate embeddings built from the prompt and opacity embeddings of the target object replace the time embedding, and a masked self-attention mechanism biases attention toward prompt-indicated regions. With these changes the model predicts a full-resolution alpha matte in one forward pass, and the paper reports that it outperforms existing interactive matting methods on every benchmark and prompt type tested.

Load-bearing premise

The load-bearing premise is that every test image is accompanied by a binary opacity label—transparent or opaque—that is fed to the model as a conditioning cue, which is annotation a real interactive user would not normally provide.

Editorial extensions

If this is right

  • If the reported results hold, SDMatte becomes the best interactive matting method among those compared on AIM-500, AM-2K, P3M-500-NP, and RefMatte-RW-100 for point, box, and mask prompts, with the largest gains on fine-edge metrics.
  • Because the noise schedule is removed, the method gets diffusion priors without iterative denoising, suggesting that one-step deterministic repurposing is sufficient for pixel-level prediction tasks.
  • LiteSDMatte indicates that the same design can be distilled into a much lighter model, cutting FLOPs by roughly a factor of five while retaining most of the accuracy.
  • The visual-prompt cross-attention and masked self-attention are general conditioning mechanisms, so a similar grafting recipe could be applied to other dense prediction tasks that need point-or-box guidance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The opacity embedding is an oracle cue: because the authors annotate opacity on all training and test data, the reported numbers measure performance under a condition real users cannot satisfy, and a fair comparison would require predicting opacity from the image or omitting it.
  • The same grafting recipe should transfer to other interactive dense prediction tasks such as depth estimation or segmentation, where a diffusion prior plus point or box prompts could replace task-specific heads.
  • One direct test of the claimed interaction transfer would be to freeze the visual-prompt cross-attention weights and fine-tune only the matting head on a new domain; preserved generalization would indicate the mechanism truly inherits Stable Diffusion's text-conditioning behavior.
  • The coordinate embedding's largest gains occur on the multi-instance benchmark RefMatte-RW-100, suggesting that prompt ambiguity is a remaining bottleneck and that methods combining multiple prompt types per object may push further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper introduces SDMatte, an interactive image matting method built on Stable Diffusion v2 in a one-step deterministic regime. The main novelty is three conditioning mechanisms: a visual-prompt-driven cross-attention layer that replaces the text embedding with prompt latents, coordinate and opacity embeddings that replace the time embedding, and a masked self-attention mechanism that focuses attention on prompt-specified regions. The method supports point, box, and mask prompts. It is evaluated on AIM-500, AM-2K, P3M-500-NP, and RefMatte-RW-100, and compared with MAM, MatAny, SmartMatting, SEMat, and MGMatting. The authors report consistent improvements across all benchmarks and prompts, a lightweight variant (LiteSDMatte) with distillation, and release code and model weights.

Significance. The idea of repurposing Stable Diffusion's text-conditioning machinery for visual-prompt-driven matting is timely, and the masked-attention and embedding designs are plausible. The code release, the distillation scheme for LiteSDMatte, and the breadth of benchmarks and metrics are positive features. If the reported results are reproducible, SDMatte would be a meaningful advance in interactive matting, especially for fine edge details and transparent objects. However, the quantitative support is currently weakened by an internal inconsistency between the main table and the ablation tables, and by a test-time opacity condition that is not shared with the baseline methods. These issues are load-bearing for the claimed SOTA and must be resolved.

major comments (2)
  1. [Sec. 4.3, Tabs. 3-5 vs Tab. 1] Section 4.3 states that all ablations use the same training settings as the best result, so the optimal rows of Tables 3, 4, and 5 should reproduce SDMatte in Table 1. They do not. For example, Table 4's opacity+coordinate row reports AIM-500 point-prompt MSE 0.0139 and RefMatte-RW-100 point-prompt MSE 0.0107, whereas Table 1 lists 0.0109 and 0.0091; the AIM-500 box SAD is 24.26 in Table 4 versus 19.45 in Table 1, and the AIM-500 box MSE differs by about 57% (0.0077 vs 0.0049). Table 3's optimal middle-block row reports AIM-500 point SAD 38.02 versus 31.80 in Table 1, and Table 5's all-blocks row reports 0.0114 vs 0.0109 and 0.0099 vs 0.0091 for the same point-prompt MSE entries. These differences are far beyond run-to-run noise and prevent the reader from attributing the gains to the proposed components. The authors should reconcile the tables or explicitly state which training/evaluation protocol produced each table.
  2. [Sec. 3.3, Eq. (2); Tab. 4] The opacity embedding is an oracle-like test-time condition. The manuscript says 'we annotate all training and testing data with opacity information' and Eq. (2) feeds that information into the U-Net. This cue is not part of the visual prompt for the baseline methods and would not be supplied by the benchmark protocols. Tab. 4 shows that it materially helps: adding opacity embedding alone reduces AIM-500 point-prompt MSE from 0.0169 to 0.0149 and box MSE from 0.0098 to 0.0079, so part of the reported superiority over SmartMatting, MAM, and SEMat may come from an annotation that the baselines cannot access. Please evaluate SDMatte without the test-time opacity cue in the main comparison, or give baselines the same information and explain how the opacity is obtained by the user in the interactive scenario.
minor comments (6)
  1. [Fig. 2] Figure 2 contains the typo 'Maked Self Attention'; it should be 'Masked Self-Attention'.
  2. [Tab. 2] Table 2 lists one method as 'Smat'; use the full name 'SmartMatting' to avoid ambiguity.
  3. [Sec. 4.1] Section 4.1 does not specify the number of point prompts, the point sampling strategy, or the protocol for generating box/mask prompts; please add these details or refer to the released code exactly.
  4. [Sec. 4.2, Tab. 1] In Table 1, the P3M-500-NP point-prompt row shows LiteSDMatte with lower MSE (0.0121) and SAD (29.94) than SDMatte (0.0134 and 32.02), which appears to contradict the statement in Sec. 4.2 that LiteSDMatte shows only slight performance degradation; please qualify this claim.
  5. [References] References [11] and [12] cite the same paper ('Diffusion for natural image matting') with different venues and years; please merge or differentiate them.
  6. [Tab. 1 caption] The caption of Table 1 refers to colored text for best/second-best methods, but no such formatting is visible in the provided manuscript text; please make the highlighting explicit.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; external benchmarks support the central claim, with minor self-citation overlap and a test-time opacity cue as caveats.

full rationale

SDMatte is an empirical engineering paper: there is no derivation chain that reduces to fitted constants or to the outputs it predicts. The main SOTA claims are tested against external baselines (MAM, MatAny, SmartMatting, SEMat) on held-out benchmarks (AIM-500, AM-2K, P3M-500-NP, RefMatte-RW-100) using official baseline models, so the central comparison is not equivalent to the paper's own inputs. Self-citations exist ([43] SEMat and [53] DiffDIS share authors with this paper), but they are used only as prior work and as a training-set source; they do not supply the evidence for SDMatte's effectiveness, so they are not load-bearing. The opacity embedding in Sec. 3.3 is a real validity caveat: 'we annotate all training and testing data with opacity information,' and Tab. 4 shows this cue improves AIM-500, meaning part of the transparent-object advantage comes from a test-time transparency label rather than from the user prompt alone. This is test-label leakage or evaluation unfairness, not a circular reduction, because the binary opacity label is a coarse, user-knowable property and not the predicted alpha matte itself; even the no-opacity baseline row in Tab. 4 is highly competitive. Separately, the full-configuration rows of Tabs. 3-5 do not match Tab. 1 (e.g., AIM-500 box MSE 0.0077 vs 0.0049), which weakens the attribution of the SOTA numbers to the ablated components; this is an internal-consistency problem, not circularity. Overall, the core claim has independent empirical content, so the circularity score is low.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central method rests on external pretrained components (SD v2, GenPercept paradigm) plus a paper-specific binary opacity annotation at test time. The hand-chosen encoding dimensions are minor. The opacity annotation is the most consequential ledger item because it supplies information beyond the user prompt.

free parameters (2)
  • Coordinate encoding dimension C for point prompts = 1680
    Chosen in Eq. (1) to be divisible by most prime numbers, minimizing zero padding for N point prompts; a hand-picked architecture constant.
  • Coordinate encoding dimension C for box/mask prompts = 1280
    Set according to Stable Diffusion's time embedding configuration (4 times the 320-dim scalar embedding) in Eq. (1).
assumptions (4)
  • domain assumption Stable Diffusion v2 pre-trained on large image-text data provides useful priors for image matting.
    The method initializes U-Net and VAE from SD v2 (Sec. 3.1) and inherits its representations; this assumption is external to the paper.
  • domain assumption One-step deterministic denoising, without noise addition, is sufficient for dense prediction.
    Adopted from GenPercept [45] and used in Sec. 3.1; the paper does not re-derive this.
  • ad hoc to paper Opacity of the target object is binary and known in advance at test time.
    Sec. 3.3 annotates all testing data with opacity labels and feeds them via E_opacity. This is a paper-specific assumption not shared by baselines and not justified as a realistic user input.
  • domain assumption Masked attention with additive masks is a valid way to bias attention toward prompt regions.
    Inspired by Mask2Former [4], used in Sec. 3.4; the soft-mask variant is a new design but its benefit is only shown empirically.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SDMatte: Grafting Diffusion Models for Interactive Matting." pith.science (2026). https://pith.science/paper/DHR6XIOX

@misc{pith2026250800443,
  author       = {Pith},
  title        = {Pith review of: SDMatte: Grafting Diffusion Models for Interactive Matting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHR6XIOX}},
  note         = {Machine review of arXiv:2508.00443}
}
read the original abstract

Recent interactive matting methods have shown satisfactory performance in capturing the primary regions of objects, but they fall short in extracting fine-grained details in edge regions. Diffusion models trained on billions of image-text pairs, demonstrate exceptional capability in modeling highly complex data distributions and synthesizing realistic texture details, while exhibiting robust text-driven interaction capabilities, making them an attractive solution for interactive matting. To this end, we propose SDMatte, a diffusion-driven interactive matting model, with three key contributions. First, we exploit the powerful priors of diffusion models and transform the text-driven interaction capability into visual prompt-driven interaction capability to enable interactive matting. Second, we integrate coordinate embeddings of visual prompts and opacity embeddings of target objects into U-Net, enhancing SDMatte's sensitivity to spatial position information and opacity information. Third, we propose a masked self-attention mechanism that enables the model to focus on areas specified by visual prompts, leading to better performance. Extensive experiments on multiple datasets demonstrate the superior performance of our method, validating its effectiveness in interactive matting. Our code and model are available at https://github.com/vivoCameraResearch/SDMatte.

Figures

Figures reproduced from arXiv: 2508.00443 by the authors.

Figure 1
Figure 1. Interactive image matting results of our SDMatte with box prompts. SDMatte leverages strong diffusion priors, ensuring robust generalization. Meanwhile, it transforms the text-driven image generation capability of Stable Diffusion into a visual prompt-driven interactive capability, enabling precise alpha matte prediction based on simple user-provided visual prompts (points, boxes, masks). Abstract Recent interactive… view at source ↗
Figure 2
Figure 2. The overall framework of SDMatte. We map the input image and visual prompt into the latent space and concatenate them as the input to the U-Net. Subsequently, we substitute the time embedding in Stable Diffusion with coordinate embeddings of visual prompts and opacity embeddings of target objects to enhance SDMatte’s sensitivity to spatial position and opacity information. Finally, we leverage the masked self-attent… view at source ↗
Figure 3
Figure 3. Visualization of the attention maps in U-Net’s final cross-attention layer. It visually demonstrates the model’s focus on the regions indicated by the visual prompts, proving the effec￾tiveness of the visual prompt-driven cross-attention mechanism. To enhance SDMatte’s ability to recognize transparent ob￾jects, we annotate all training and testing data with opacity information. If an object is transparent, its opaci… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visual comparison with existing interactive image matting methods. Compared to other methods, our approach demonstrates significantly better generalization and superior extraction capabilities for transparent and detail-rich objects. Down Blocks Mid Block Up Blocks AIM…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 35 canonical work pages

  1. [1]

    Segdiff: Image segmentation with diffusion proba- bilistic models.arXiv preprint arXiv:2112.00390, 2021

    Tomer Amit, Tal Shaharbany, Eliya Nachmani, and Lior Wolf. Segdiff: Image segmentation with diffusion proba- bilistic models.arXiv preprint arXiv:2112.00390, 2021. 2

    arXiv 2021

  2. [2]

    taesd: A tiny autoen- coder for fast sampling of stable diffusion

    Ollin Boer Bohan. taesd: A tiny autoen- coder for fast sampling of stable diffusion. https://github.com/madebyollin/taesd, 2023. Accessed: 2025-07-31. 6

    work page 2023

  3. [3]

    Pp-matting: high-accuracy natural image matting

    Guowei Chen, Yi Liu, Jian Wang, Juncai Peng, Yuying Hao, Lutao Chu, Shiyu Tang, Zewu Wu, Zeyu Chen, Zhiliang Yu, et al. Pp-matting: high-accuracy natural image matting. arXiv preprint arXiv:2204.09433, 2022. 2

    arXiv 2022

  4. [4]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 5

    2022

  5. [5]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. InForty-first International Conference on Machine Learn- ing, 2024. 2, 3

    work page 2024

  6. [6]

    Marco Forte and Franc ¸ois Piti´e.f,b, alpha matting.arXiv preprint arXiv:2003.07711, 2020. 2

    work page Pith review arXiv 2003

  7. [7]

    In-context matting

    He Guo, Zixuan Ye, Zhiguo Cao, and Hao Lu. In-context matting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3711– 3720, 2024. 2

    work page 2024

  8. [8]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. 3

    arXiv 2022

Show all 57 references
  1. [9]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 2, 3

    2020

  2. [10]

    Diffumatting: Synthe- sizing arbitrary objects with matting-level annotation

    Xiaobin Hu, Xu Peng, Donghao Luo, Xiaozhong Ji, Jin- long Peng, Zhengkai Jiang, Jiangning Zhang, Taisong Jin, Chengjie Wang, and Rongrong Ji. Diffumatting: Synthe- sizing arbitrary objects with matting-level annotation. In European Conference on Computer Vision, pages 396–413....

    2024

  3. [11]

    Diffusion for natural image matting

    Yihan Hu, Yiheng Lin, Wei Wang, Yao Zhao, Yunchao Wei, and Humphrey Shi. Diffusion for natural image matting. In European Conference on Computer Vision, pages 181–199. Springer, 2024. 2

    2024

  4. [12]

    Diffusion for natural image matting

    Yihan Hu, Yiheng Lin, Wei Wang, Yao Zhao, Yunchao Wei, and Humphrey Shi. Diffusion for natural image matting. In European Conference on Computer Vision, pages 181–199. Springer, 2025. 2

    2025

  5. [13]

    Maggie: Masked guided gradual human in- stance matting

    Chuong Huynh, Seoung Wug Oh, Abhinav Shrivastava, and Joon-Young Lee. Maggie: Masked guided gradual human in- stance matting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3870– 3879, 2024. 2

    2024

  6. [14]

    Ddp: Diffusion model for dense visual prediction

    Yuanfeng Ji, Zhe Chen, Enze Xie, Lanqing Hong, Xihui Liu, Zhaoqiang Liu, Tong Lu, Zhenguo Li, and Ping Luo. Ddp: Diffusion model for dense visual prediction. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 21741–21752, 2023. 2

    2023

  7. [15]

    Trimap-guided feature mining and fusion network for natural image matting.Computer Vision and Image Understanding, 230:103645, 2023

    Weihao Jiang, Dongdong Yu, Zhaozhi Xie, Yaoyi Li, Zehuan Yuan, and Hongtao Lu. Trimap-guided feature mining and fusion network for natural image matting.Computer Vision and Image Understanding, 230:103645, 2023. 2

    2023

  8. [16]

    Repurposing stable diffusion attention for training-free unsupervised interactive segmentation.arXiv preprint arXiv:2411.10411, 2024

    Markus Karmann and Onay Urfalioglu. Repurposing stable diffusion attention for training-free unsupervised interactive segmentation.arXiv preprint arXiv:2411.10411, 2024. 2, 3

    2024 arXiv

  9. [17]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9492– ...

    2024

  10. [18]

    Segment anything in high qual- ity.Advances in Neural Information Processing Systems, 36: 29914–29934, 2023

    Lei Ke, Mingqiao Ye, Martin Danelljan, Yu-Wing Tai, Chi- Keung Tang, Fisher Yu, et al. Segment anything in high qual- ity.Advances in Neural Information Processing Systems, 36: 29914–29934, 2023. 2, 3

    2023

  11. [19]

    Modnet: Real-time trimap-free portrait mat- ting via objective decomposition

    Zhanghan Ke, Jiayu Sun, Kaican Li, Qiong Yan, and Ryn- son WH Lau. Modnet: Real-time trimap-free portrait mat- ting via objective decomposition. InProceedings of the AAAI Conference on Artificial Intelligence, pages 1140– 1147, 2022. 2

    2022

  12. [20]

    Bk-sdm: A lightweight, fast, and cheap ver- sion of stable diffusion

    Bo-Kyeong Kim, Hyoung-Kyu Song, Thibault Castells, and Shinkook Choi. Bk-sdm: A lightweight, fast, and cheap ver- sion of stable diffusion. InEuropean Conference on Com- puter Vision, pages 381–399. Springer, 2024. 7

    2024

  13. [21]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023...

    2023

  14. [22]

    Privacy- preserving portrait matting

    Jizhizi Li, Sihan Ma, Jing Zhang, and Dacheng Tao. Privacy- preserving portrait matting. InProceedings of the 29th ACM international conference on multimedia, pages 3501–3509,

  15. [23]

    Deep automatic natural image matting.arXiv preprint arXiv:2107.07235,

    Jizhizi Li, Jing Zhang, and Dacheng Tao. Deep automatic natural image matting.arXiv preprint arXiv:2107.07235,

    arXiv

  16. [24]

    Bridging composite and real: towards end-to-end deep image matting.International Journal of Computer Vision, 130(2):246–266, 2022

    Jizhizi Li, Jing Zhang, Stephen J Maybank, and Dacheng Tao. Bridging composite and real: towards end-to-end deep image matting.International Journal of Computer Vision, 130(2):246–266, 2022. 2, 5

    2022

  17. [25]

    Referring im- age matting

    Jizhizi Li, Jing Zhang, and Dacheng Tao. Referring im- age matting. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22448– 22457, 2023. 2, 5

    2023

  18. [26]

    Matting anything

    Jiachen Li, Jitesh Jain, and Humphrey Shi. Matting anything. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1775–1785, 2024. 2, 3, 5, 6

    2024

  19. [27]

    Rethinking portrait matting with privacy preserving

    Sihan Ma, Jizhizi Li, Jing Zhang, He Zhang, and Dacheng Tao. Rethinking portrait matting with privacy preserving. International journal of computer vision, 131(8):2172–2197,

  20. [28]

    Diffusion models beat gans on image classification.arXiv preprint arXiv:2307.08702, 2023

    Soumik Mukhopadhyay, Matthew Gwilliam, Vatsal Agar- wal, Namitha Padmanabhan, Archana Swaminathan, Srinidhi Hegde, Tianyi Zhou, and Abhinav Shrivastava. Diffusion models beat gans on image classification.arXiv preprint arXiv:2307.08702, 2023. 3

    2023 arXiv

  21. [29]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 6

    2023 arXiv

  22. [30]

    Mask-guided matting in the wild

    Kwanyong Park, Sanghyun Woo, Seoung Wug Oh, In So Kweon, and Joon-Young Lee. Mask-guided matting in the wild. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 1992–2001,

    1992

  23. [31]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 2, 3, 4, 8

    2023 arXiv

  24. [32]

    Attention-guided hi- erarchical structure aggregation for image matting

    Yu Qiao, Yuhao Liu, Xin Yang, Dongsheng Zhou, Mingliang Xu, Qiang Zhang, and Xiaopeng Wei. Attention-guided hi- erarchical structure aggregation for image matting. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13676–13685, 2020. 5

    2020

  25. [33]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024. 2, 3

    2024 arXiv

  26. [34]

    A perceptu- ally motivated online benchmark for image matting

    Christoph Rhemann, Carsten Rother, Jue Wang, Margrit Gelautz, Pushmeet Kohli, and Pamela Rott. A perceptu- ally motivated online benchmark for image matting. In2009 IEEE conference on computer vision and pattern recogni- tion, pages 1826–1833. IEEE, 2009. 5

    2009

  27. [35]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2, 3

    2022

  28. [36]

    Palette: Image-to-image diffusion models

    Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10, 2022

    2022

  29. [37]

    Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020. 2, 3

    2010 arXiv

  30. [38]

    Semantic image matting: General and specific semantics.Interna- tional Journal of Computer Vision, 132(3):710–730, 2024

    Yanan Sun, Chi-Keung Tang, and Yu-Wing Tai. Semantic image matting: General and specific semantics.Interna- tional Journal of Computer Vision, 132(3):710–730, 2024. 2

    2024

  31. [39]

    Learning-based sampling for natural image matting

    Jingwei Tang, Yagiz Aksoy, Cengiz Oztireli, Markus Gross, and Tunc Ozan Aydin. Learning-based sampling for natural image matting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3055– 3063, 2019. 2

    2019

  32. [40]

    Diffuse attend and segment: Un- supervised zero-shot segmentation using stable diffusion

    Junjiao Tian, Lavisha Aggarwal, Andrea Colaco, Zsolt Kira, and Mar Gonzalez-Franco. Diffuse attend and segment: Un- supervised zero-shot segmentation using stable diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 3554–3563...

    2024

  33. [41]

    Matting by gen- eration

    Zhixiang Wang, Baiang Li, Jian Wang, Yu-Lun Liu, Jinwei Gu, Yung-Yu Chuang, and Shin’Ichi Satoh. Matting by gen- eration. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 2, 3

    2024

  34. [42]

    Improved im- age matting via real-time user clicks and uncertainty estima- tion

    Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Han- qing Zhao, Weiming Zhang, and Nenghai Yu. Improved im- age matting via real-time user clicks and uncertainty estima- tion. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 15374–15383,

  35. [43]

    Towards natural im- age matting in the wild via real-scenario prior.arXiv preprint arXiv:2410.06593, 2024

    Ruihao Xia, Yu Liang, Peng-Tao Jiang, Hao Zhang, Qianru Sun, Yang Tang, Bo Li, and Pan Zhou. Towards natural im- age matting in the wild via real-scenario prior.arXiv preprint arXiv:2410.06593, 2024. 2, 3, 5, 6

    2024 arXiv

  36. [44]

    Pyramid grafting network for one- stage high resolution saliency detection

    Chenxi Xie, Changqun Xia, Mingcan Ma, Zhirui Zhao, Xi- aowu Chen, and Jia Li. Pyramid grafting network for one- stage high resolution saliency detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11717–11726, 2022. 5

    2022

  37. [45]

    What matters when repurposing diffusion models for general dense perception tasks?arXiv preprint arXiv:2403.06090,

    Guangkai Xu, Yongtao Ge, Mingyu Liu, Chengxiang Fan, Kangyang Xie, Zhiyue Zhao, Hao Chen, and Chunhua Shen. What matters when repurposing diffusion models for general dense perception tasks?arXiv preprint arXiv:2403.06090,

    arXiv

  38. [46]

    Deep image matting

    Ning Xu, Brian Price, Scott Cohen, and Thomas Huang. Deep image matting. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2970– 2979, 2017. 2, 5

    2017

  39. [47]

    Exploring the interactive guidance for unified and effec- tive image matting.arXiv preprint arXiv:2205.08324, 2022

    Dinghao Yang, Bin Wang, Weijia Li, YiQi Lin, and Conghui He. Exploring the interactive guidance for unified and effec- tive image matting.arXiv preprint arXiv:2205.08324, 2022. 2

    2022 arXiv

  40. [48]

    Vitmatte: Boosting image matting with pre- trained plain vision transformers.Information Fusion, 103: 102091, 2024

    Jingfeng Yao, Xinggang Wang, Shusheng Yang, and Baoyuan Wang. Vitmatte: Boosting image matting with pre- trained plain vision transformers.Information Fusion, 103: 102091, 2024. 2

    2024

  41. [49]

    Matte anything: Interactive natural image matting with seg- ment anything model.Image and Vision Computing, 147: 105067, 2024

    Jingfeng Yao, Xinggang Wang, Lang Ye, and Wenyu Liu. Matte anything: Interactive natural image matting with seg- ment anything model.Image and Vision Computing, 147: 105067, 2024. 2, 3, 5, 6

    2024

  42. [50]

    Diffusionedge: Diffusion probabilistic model for crisp edge detection

    Yunfan Ye, Kai Xu, Yuhang Huang, Renjiao Yi, and Zhiping Cai. Diffusionedge: Diffusion probabilistic model for crisp edge detection. InProceedings of the AAAI conference on artificial intelligence, pages 6675–6683, 2024. 2

    2024

  43. [51]

    Unifying automatic and in- teractive matting with pretrained vits

    Zixuan Ye, Wenze Liu, He Guo, Yujia Liang, Chaoyi Hong, Hao Lu, and Zhiguo Cao. Unifying automatic and in- teractive matting with pretrained vits. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 25585–25594, 2024. 2, 3, 5, 6

    2024

  44. [52]

    Mask guided matting via progressive refinement network

    Qihang Yu, Jianming Zhang, He Zhang, Yilin Wang, Zhe Lin, Ning Xu, Yutong Bai, and Alan Yuille. Mask guided matting via progressive refinement network. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1154–1163, 2021. 2, 6

    2021

  45. [53]

    High-precision dichotomous image segmentation via probing diffusion capacity.arXiv preprint arXiv:2410.10105, 2024

    Qian Yu, Peng-Tao Jiang, Hao Zhang, Jinwei Chen, Bo Li, Lihe Zhang, and Huchuan Lu. High-precision dichotomous image segmentation via probing diffusion capacity.arXiv preprint arXiv:2410.10105, 2024. 3

    2024 arXiv

  46. [54]

    Primedepth: Efficient monocular depth estimation with a sta- ble diffusion preimage

    Denis Zavadski, Damjan Kal ˇsan, and Carsten Rother. Primedepth: Efficient monocular depth estimation with a sta- ble diffusion preimage. InProceedings of the Asian Confer- ence on Computer Vision, pages 922–940, 2024. 2

    2024

  47. [55]

    Betterdepth: Plug-and-play diffu- sion refiner for zero-shot monocular depth estimation.arXiv preprint arXiv:2407.17952, 2024

    Xiang Zhang, Bingxin Ke, Hayko Riemenschneider, Nando Metzger, Anton Obukhov, Markus Gross, Konrad Schindler, and Christopher Schroers. Betterdepth: Plug-and-play diffu- sion refiner for zero-shot monocular depth estimation.arXiv preprint arXiv:2407.17952, 2024

    2024 arXiv

  48. [56]

    Ar-1-to-3: Single image to consistent 3d object generation via next-view prediction.arXiv preprint arXiv:2503.12929, 2025

    Xuying Zhang, Yupeng Zhou, Kai Wang, Yikai Wang, Zhen Li, Shaohui Jiao, Daquan Zhou, Qibin Hou, and Ming- Ming Cheng. Ar-1-to-3: Single image to consistent 3d object generation via next-view prediction.arXiv preprint arXiv:2503.12929, 2025. 2

    2025 arXiv

  49. [57]

    Sampling propagation attention with trimap gen- eration network for natural image matting.IEEE Transac- tions on Circuits and Systems for Video Technology, 33(10): 5828–5843, 2023

    Yuhongze Zhou, Liguang Zhou, Tin Lun Lam, and Yang- sheng Xu. Sampling propagation attention with trimap gen- eration network for natural image matting.IEEE Transac- tions on Circuits and Systems for Video Technology, 33(10): 5828–5843, 2023. 2

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.