Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Instruct2See: Learning to Remove Any Obstructions Across Distributions

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Instruct2See claims that a single model can remove any obstruction, seen or unseen, by reformulating removal as a soft–hard mask restoration problem guided by text instructions and visual semantics.

desk verdict Competent all-in-one obstruction removal with a genuinely useful soft/hard mask switch, but the 'any obstruction' zero-shot claim outruns the evidence, and the undisclosed mask detector for two of the three unseen test categories is a real leak risk. read the letter →

arxiv 2505.17649 v1 pith:KQ45RD6X submitted 2025-05-23 cs.CV

Junhang Li , Yu Guo , Chuhua Xian , Shengfeng He This is my paper · ORCID
classification cs.CV
keywords obstructionremovalzero-shotgeneralizationsoft-hardmaskrestorationmulti-modalpromptscross-attentionimageCLIPtunableadapter
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that one image-restoration model can remove almost any visual obstruction—not just the fences, raindrops, and lens flares it was trained on, but also snow, rain streaks, strokes, power cables, shadows, and other blockers it has never seen. The proposed framework, Instruct2See, treats removal as a soft–hard mask restoration problem: it takes an image, a mask of the obstruction, and a short text description, then reconstructs the hidden background. The key move is to let the text decide whether the blocker is opaque (hard mask) or semi-transparent (soft mask), and to refine imperfect masks with a tunable adapter. If the claim is correct, users would no longer need a specialized model per obstruction type; a single instruction-following model could be deployed on new and unpredictable occluders.

What carries the argument

The central object is the soft–hard mask restoration formulation, carried by three components. First, a mask detector (SAM2 for most obstructions, a U-Net for rain streaks and snow) produces an initial mask. Second, a CLIP text encoder compares the user's instruction against fixed embeddings for "opaque" and "semi-transparent"; if the semi-transparent similarity exceeds a threshold, the mask is refined by a tunable adapter network, otherwise it is used directly. Third, the image is cut out with the refined mask, and a Restormer-based transformer with a cross-attention unit fuses the masked image with concatenated CLIP text and visual embeddings to reconstruct the clean background. The tunable adapter—a small transformer that adjusts masks with ambiguous boundaries—is what lets imperfect masks be corrected at test time, and the text-driven mode switch is what lets one model handle both opaque and semi-transparent blockers.

What would settle it

Rerun the unseen-obstruction experiments with ground-truth masks instead of the U-Net/SAM2 outputs and compare PSNR; a large drop would show the claimed zero-shot behavior lives in the mask detector, while a small drop would confirm it lives in the restoration model.

Watch

Extended reading notes

Core claim

Instruct2See claims that obstruction removal can be unified as a soft–hard mask restoration problem and that this formulation transfers to obstructions never seen in training. Given an input image, an initial mask, and a text instruction such as "remove the semi-transparent obstruction," the model uses a CLIP text encoder to choose between hard masking (opaque blockers, mask used as-is) and soft masking (semi-transparent blockers, mask refined by a tunable adapter), cuts out the masked region, and reconstructs the background with a Restormer-based transformer that fuses the masked image with concatenated text and visual embeddings via cross-attention. The paper reports that on seen obstructions the method is competitive with or better than all-in-one restoration baselines (average PSNR 29.31 dB), and on unseen obstructions—rain streaks, snow, and strokes—it leads all compared methods (average PSNR 31.37 dB) and beats inpainting and editing baselines such as LaMa, RePaint, and DiffEdit. The strong claim is that this generalization is distribution-agnostic: the model performs well "regardless of whether the obstacles were present during the training phase."

Load-bearing premise

The zero-shot claim collapses if the mask detector or the text-driven mode switch has effectively seen the test obstruction types—for instance, if the U-Net used for rain and snow masks was trained on those same test images, the reported gains would be a mask-detection result rather than a restoration generalization result.

Editorial extensions

If this is right

  • A single model trained on three obstruction types would replace the current practice of training a separate model for each obstruction category, cutting data-collection and retraining costs for new blockers.
  • Because mask estimation is separated from restoration, replacing the mask detector with a better one should directly improve removal quality without retraining the restorer.
  • The text-driven mode switch makes the user's description part of the interface: a correct description is required to choose the right masking strategy, and an incorrect one degrades results.
  • On unseen obstructions the reported margin over inpainting baselines suggests that obstruction removal is better framed as mask-guided restoration than as free-form inpainting, even in the zero-shot setting.
  • On seen obstructions the method remains competitive (average PSNR 29.31 dB versus 28.45 dB for the second-best method), so the generalization gain is not bought at the cost of in-distribution quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The zero-shot claim is only as strong as the mask detector: the paper does not disclose whether the U-Net used for rain streaks and snow was trained on those test categories, so a fair test of the restorer's generalization would swap in ground-truth masks and compare results.
  • The mode switch is a binary choice between hard and soft masking based on text similarity; a confidence-weighted blend of the two modes might handle obstructions with mixed or ambiguous transparency more gracefully.
  • The paper's own failure analysis (Appendix C) shows that an incorrect text description degrades removal; in real deployments the method would need an automatic obstruction captioner, and the reliability of that captioner on arbitrary unseen obstructions is unmeasured.
  • The unified imaging model assumes a single mask per obstruction, so the 'any obstruction' claim is best read as 'any single texture-level obstruction' rather than a guarantee for large, multi-layered, or overlapping occluders.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Instruct2See, a unified obstruction-removal framework that treats any obstruction as a soft-hard mask restoration problem. The method takes an input image, an estimated mask, and a text instruction; it uses CLIP encoders to build multi-modal prompts and a Restormer-based encoder-decoder with cross-attention to reconstruct the clean background. A tunable adapter adjusts masks for semi-transparent obstructions, and a text-based switch decides between hard and soft masking. The model is trained on fences, flares, and raindrops, and evaluated on in-distribution data as well as out-of-distribution rain streaks, snow, and strokes, plus qualitative results on several additional categories. The authors report state-of-the-art PSNR, SSIM, CLIP score, and LPIPS compared with seven restoration baselines and three inpainting/editing methods, and provide ablations for the proposed modules and prompts.

Significance. If the zero-shot claim holds, the paper is a useful step toward a single restoration model that handles diverse obstructions without per-category training. The work has several concrete strengths: it reports broad experiments with multiple baselines, includes seen/unseen splits, ablates the proposed cross-attention and adapter, and plans to release code and data. The soft-hard mask formulation with a tunable adapter is a reasonable and potentially reusable idea. However, the validity of the central zero-shot claim hinges on whether the full inference pipeline is genuinely out-of-distribution for the unseen categories, and this is not currently established because the mask generator used for two of the three quantitative unseen categories is not characterized.

major comments (3)
  1. [§5.1, Table 2, Appendix A.1, Algorithm 1] The central claim that the model generalizes to unseen obstructions is not fully supported because the mask generator used for two of the three quantitative unseen categories is never characterized. In Appendix A.1, the initial mask for rain streaks and snow is produced by a U-Net-based model, but the manuscript does not state what data or objective that U-Net was trained on. Since Table 2 evaluates on the rain streak dataset of Yang et al. (2017) and the snow dataset of Liu et al. (2018), if the U-Net was trained on those datasets or on the same distribution, the reported gains on those rows are at least partly attributable to a mask detector that is not zero-shot, rather than to the proposed restoration model. Please disclose the U-Net's training set, and provide an unseen evaluation using either a mask detector that is explicitly not trained on these categories or ground-truth masks, following the design of Table 5.
  2. [§4.2, Eq. (7), Algorithm 1] The soft masking switch relies on a threshold θ in Eq. (7) and Algorithm 1 line 5, but the value of θ is never specified, nor is the procedure by which it was chosen. If θ was selected using the unseen test sets, the zero-shot evaluation leaks tuning information. The paper should report the exact value of θ, state whether it was fixed a priori or tuned on a validation split, and include a sensitivity analysis over θ on seen data.
  3. [Abstract, §5.2, Appendix C] The 'zero-shot' framing overstates what the full pipeline requires. The inference procedure in Algorithm 1 needs an external mask and a user-supplied text instruction; Appendix C and Fig. 14 show that incorrect descriptions lead to failure, and the text instruction is effectively reduced to a binary opaque/semi-transparent decision. Thus the restoration network alone is zero-shot with respect to the training classes, but the complete system is not fully automatic and depends on correct user input and a suitable mask detector. The claims should be revised to state these dependencies explicitly and to avoid the impression that the model recognizes unseen obstructions on its own.
minor comments (4)
  1. [§5.1] The progressive training schedule lists patch sizes and iterations in a confusing order: '128×128, 160×160, 192×192, and 256×256 at iterations 115,000, 80,000, 60,000, and 45,000' appears to be sorted by iteration descending rather than by training time, and the text says the patch size is 'progressively updated to 128×128' after already stating the initial patch size is 128×128. Please clarify the schedule.
  2. [Table 2 and Table 7] There are typos in the method names: 'XResrormer' in Table 2 and 'XRestromer' in Table 7 should both be 'XRestormer'.
  3. [§5.2] The statement that seen-obstruction comparisons are made 'by using detected masks' is vague because the mask detector is not named for the fence, flare, and raindrop cases in the main text. Please specify the mask source for each seen category or refer to Appendix A.
  4. [§1, Contributions] The claim of introducing the 'first unified obstruction formulation' is not substantiated given the existence of all-in-one restoration models such as AirNet and PromptIR, which also aim to handle multiple degradation types; consider softening this wording.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation in the core method; the main caveats are external-validity and potential-leakage concerns, not circularity.

full rationale

The paper's derivation chain is transparent: Eq. (1) is the standard image-formation model for obstructions, Eqs. (2)-(5) define a mask-based preprocessing and a learned reconstruction objective, and the restoration network is trained on fence, flare, and raindrop pairs while tested on external rain-streak, snow, and stroke datasets. The final metrics compare the output against the same ground-truth background B used in the training loss, but this is ordinary supervised evaluation, not a tautology, because the test images and obstacle types are outside the training distribution of the restoration model. The mask detector does feed the pipeline (Algorithm 1), so the quantitative 'unseen' claim depends in part on a module whose training data is not disclosed in Appendix A.1; the paper even states in Appendix B.1 that 'the mask detector is plug-and-play' and that future stronger detectors could improve results. That is an external-validity or potential-leakage concern, not a circularity: the detector's output is not a fitted parameter of the proposed restoration model, and the restoration equations do not reduce to the detector's output by construction. Similarly, the CLIP text encoder is contrastively fine-tuned on the authors' own GPT-4 instruction database to separate opaque from semi-transparent instructions, and the same embeddings are then used to activate the soft/hard switch (Sec. 4.2, Appendix A.2). This is a trained classifier operating in-distribution, and Appendix C explicitly shows that wrong user descriptions break the switch; it does not make the restoration prediction equivalent to its training labels. No load-bearing self-citation, imported uniqueness theorem, or ansatz smuggled in by citation was found. The core claim therefore has independent empirical content, with the main risk being undisclosed mask-detector training data rather than circular reasoning.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical entities are proposed. The central claim depends on hand-set hyperparameters (threshold θ, temperature λ), a simplified image formation model, and the transferability of masks and CLIP text embeddings to unseen obstructions.

free parameters (2)
  • Soft masking switch threshold θ
    Algorithm 1 line 5 and Eq. (7): the adapter is enabled when cosine similarity to Γt(Ts) exceeds θ. The value is not reported, and the text encoder is fine-tuned on the authors' instruction database, so the switch is tuned to that distribution.
  • Cross-attention temperature λ
    Eq. (8) divides attention logits by λ. The paper does not report its value or whether it is learned. It scales prompt-image interaction in every transformer block.
assumptions (4)
  • domain assumption Images obey the layer composition I = B∘(1-M) + R∘M with a binary mask M.
    Eq. (1) is the basis of the whole method. It ignores refraction, reflection, scattering, and depth-dependent transparency, which are relevant for flares, snow, and shadows.
  • domain assumption Removing the estimated mask region leaves B∘(1-M), a clean background.
    Eqs. (2)-(4) assume the mask is accurate and the obstruction is layer-separable. The tunable adapter is introduced precisely because this fails for soft-boundary obstacles; no guarantee is given for unseen ones.
  • domain assumption Fine-tuned CLIP text embeddings can identify the correct masking mode (soft vs hard) for unseen obstruction descriptions.
    Sec. 4.2 and Appendix A.2 fine-tune CLIP on a two-category GPT-4 instruction database. Appendix C shows incorrect descriptions cause visible failures, so this assumption is load-bearing.
  • domain assumption SAM2 and the U-Net detector can segment seen and unseen obstructions.
    Appendix A.1 states U-Net is used for rain streaks and snow, but does not report whether the U-Net was trained on masks from those 'unseen' datasets. The zero-shot claim of the full pipeline rests on this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Instruct2See: Learning to Remove Any Obstructions Across Distributions." pith.science (2026). https://pith.science/paper/KQ45RD6X

@misc{pith2026250517649,
  author       = {Pith},
  title        = {Pith review of: Instruct2See: Learning to Remove Any Obstructions Across Distributions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KQ45RD6X}},
  note         = {Machine review of arXiv:2505.17649}
}
read the original abstract

Images are often obstructed by various obstacles due to capture limitations, hindering the observation of objects of interest. Most existing methods address occlusions from specific elements like fences or raindrops, but are constrained by the wide range of real-world obstructions, making comprehensive data collection impractical. To overcome these challenges, we propose Instruct2See, a novel zero-shot framework capable of handling both seen and unseen obstacles. The core idea of our approach is to unify obstruction removal by treating it as a soft-hard mask restoration problem, where any obstruction can be represented using multi-modal prompts, such as visual semantics and textual instructions, processed through a cross-attention unit to enhance contextual understanding and improve mode control. Additionally, a tunable mask adapter allows for dynamic soft masking, enabling real-time adjustment of inaccurate masks. Extensive experiments on both in-distribution and out-of-distribution obstacles show that Instruct2See consistently achieves strong performance and generalization in obstruction removal, regardless of whether the obstacles were present during the training phase. Code and dataset are available at https://jhscut.github.io/Instruct2See.

Figures

Figures reproduced from arXiv: 2505.17649 by the authors.

Figure 1
Figure 1. We present Instruct2See, a zero-shot framework for obstruction removal that handles arbitrary obstructions. It [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Examples of in- and out-of-distribution obstacles. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Flowchart of our Instruct2See. Instruct2See accepts instructions (randomly sampled from a database of instructions [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Visual comparisons of our method with other approaches on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Visual comparisons of our method with other approaches on [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 8
Figure 8. Figure 8: Visual comparisons of using our tunable adapter [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 6
Figure 6. Figure 6: Visual comparisons of our method with inpainting [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Visual comparisons of using different prompt [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 9
Figure 9. Figure 9: Contrastive fine-tuning of CLIP text encoder. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: PSNR and SSIM comparisons of different methods on seen and unseen obstructions. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Visual comparisons on three classic unseen obstructions (rain streak, snow, and stroke). [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Visual comparisons on more uncommon obstructions. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Visual results on multiple obstruction removal. [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Visual comparisons of using different text descriptions. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. V2VCrafter: Consistent Street-View Image Generation Across Vehicles

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    V2XCrafter introduces a progressive multi-agent diffusion model with cross-agent attention to generate controllable, consistent collaborative driving scenes for V2X data augmentation.

Reference graph

Works this paper leans on

54 extracted references · 52 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Uav-rain1k: A benchmark for raindrop removal from uav aerial imagery

    Chang, W., Chen, H., He, X., Chen, X., and Shen, L. Uav-rain1k: A benchmark for raindrop removal from uav aerial imagery. In CVPR, pp.\ 15--22, 2024

    work page 2024

  3. [3]

    Pre-trained image processing transformer

    Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., and Gao, W. Pre-trained image processing transformer. In CVPR, pp.\ 12299--12310, 2021

    work page 2021

  4. [4]

    Sparse sampling transformer with uncertainty-driven ranking for unified removal of raindrops and rain streaks

    Chen, S., Ye, T., Bai, J., Chen, E., Shi, J., and Zhu, L. Sparse sampling transformer with uncertainty-driven ranking for unified removal of raindrops and rain streaks. In ICCV, pp.\ 13106--13117, 2023 a

    work page 2023

  5. [5]

    Lightweight image de-snowing: A better trade-off between network capacity and performance

    Chen, Z., Sun, Y., Bi, X., and Yue, J. Lightweight image de-snowing: A better trade-off between network capacity and performance. Neural Networks, 165: 0 896--908, 2023 b

    work page 2023

  6. [6]

    Neural spline fields for burst image fusion and layer separation

    Chugunov, I., Shustin, D., Yan, R., Lei, C., and Heide, F. Neural spline fields for burst image fusion and layer separation. In CVPR, pp.\ 25763--25773, 2024

    work page 2024

  7. [7]

    Diffusion-based semantic image editing with mask guidance

    Couairon, G., Verbeek, J., Schwenk, H., and Diffedit, M. Diffusion-based semantic image editing with mask guidance. In ICLR, 2023

    work page 2023

  8. [8]

    Dai, Y., Li, C., Zhou, S., Feng, R., and Loy, C. C. Flare7k: A phenomenological nighttime flare removal dataset. NeurIPS, 35: 0 3926--3937, 2022

    work page 2022

Show all 54 references
  1. [9]

    Dai, Y., Luo, Y., Zhou, S., Li, C., and Loy, C. C. Nighttime smartphone reflective flare removal using optical center symmetry prior. In CVPR, pp.\ 20783--20791, 2023

    2023

  2. [10]

    Learning contextual transformer network for image inpainting

    Deng, Y., Hui, S., Zhou, S., Meng, D., and Wang, J. Learning contextual transformer network for image inpainting. In MM, pp.\ 2529--2538, 2021

    2021

  3. [11]

    Accurate and efficient video de-fencing using convolutional neural networks and temporal information

    Du, C., Kang, B., Xu, Z., Dai, J., and Nguyen, T. Accurate and efficient video de-fencing using convolutional neural networks and temporal information. In ICME, pp.\ 1--6, 2018

    2018

  4. [12]

    Blind image denoising via dynamic dual learning

    Du, Y., Han, G., Tan, Y., Xiao, C., and He, S. Blind image denoising via dynamic dual learning. IEEE TMM, 23: 0 2139--2152, 2020

    2020

  5. [13]

    Gradpaint: Gradient-guided inpainting with diffusion models

    Grechka, A., Couairon, G., and Cord, M. Gradpaint: Gradient-guided inpainting with diffusion models. CVIU, 240: 0 103928, 2024

    2024

  6. [14]

    W., and He, S

    Guo, Y., Gao, Y., Lu, Y., Zhu, H., Liu, R. W., and He, S. Onerestore: A universal restoration framework for composite degradation. ECCV, 2024

    2024

  7. [15]

    A., Petersson, L., and Li, H

    Han, J., Li, W., Fang, P., Sun, C., Hong, J., Armin, M. A., Petersson, L., and Li, H. Blind image decomposition. In ECCV, pp.\ 218--237, 2022

    2022

  8. [16]

    M-rrfs: A memory-based robust region feature synthesizer for zero-shot object detection

    Huang, P., Zhang, D., Cheng, D., Han, L., Zhu, P., and Han, J. M-rrfs: A memory-based robust region feature synthesizer for zero-shot object detection. IJCV, pp.\ 1--22, 2024 a

    2024

  9. [17]

    Portrait shadow removal via self-exemplar illumination equalization

    Huang, Q., Xu, C., Li, G., Wu, Z., Liu, S., and He, S. Portrait shadow removal via self-exemplar illumination equalization. In ACM MM, pp.\ 7474--7482, 2024 b

    2024

  10. [18]

    All-in-one image restoration for unknown corruption

    Li, B., Liu, X., Hu, P., Wu, Z., Lv, J., and Peng, X. All-in-one image restoration for unknown corruption. In CVPR, pp.\ 17452--17462, 2022 a

    2022

  11. [19]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Li, J., Li, D., Xiong, C., and Hoi, S. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, 2022 b

    2022

  12. [20]

    T., and Cheong, L.-F

    Li, R., Tan, R. T., and Cheong, L.-F. All in one bad weather removal using architectural search. In CVPR, pp.\ 3175--3185, 2020

    2020

  13. [21]

    Mat: Mask-aware transformer for large hole image inpainting

    Li, W., Lin, Z., Zhou, K., Qi, L., Wang, Y., and Jia, J. Mat: Mask-aware transformer for large hole image inpainting. In CVPR, pp.\ 10758--10768, 2022 c

    2022

  14. [22]

    Dual-pixel raindrop removal

    Li, Y., Monno, Y., and Okutomi, M. Dual-pixel raindrop removal. IEEE TPAMI, 2024

    2024

  15. [23]

    Desnownet: Context-aware deep network for snow removal

    Liu, Y.-F., Jaw, D.-W., Huang, S.-C., and Hwang, J.-N. Desnownet: Context-aware deep network for snow removal. IEEE TIP, 27 0 (6): 0 3064--3073, 2018

    2018

  16. [24]

    Learning to see through obstructions with layered decomposition

    Liu, Y.-L., Lai, W.-S., Yang, M.-H., Chuang, Y.-Y., and Huang, J.-B. Learning to see through obstructions with layered decomposition. IEEE TPAMI, 44 0 (11): 0 8387--8402, 2021

    2021

  17. [25]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., and Van Gool, L. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR, pp.\ 11461--11471, 2022

    2022

  18. [26]

    A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

    Martin, D., Fowlkes, C., Tal, D., and Malik, J. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, volume 2, pp.\ 416--423, July 2001

    2001

  19. [27]

    F., Xian, Y., Gool, L

    Naeem, M. F., Xian, Y., Gool, L. V., and Tombari, F. I2dformer+: Learning image to document summary attention for zero-shot image classification. IJCV, pp.\ 1--17, 2024

    2024

  20. [28]

    and Legenstein, R

    \"O zdenizci, O. and Legenstein, R. Restoring vision in adverse weather conditions with patch-based denoising diffusion models. IEEE TPAMI, 45 0 (8): 0 10346--10357, 2023

    2023

  21. [29]

    W., Khan, S., and Khan, F

    Potlapalli, V., Zamir, S. W., Khan, S., and Khan, F. Promptir: Prompting for all-in-one image restoration. In NeurIPS, 2023

    2023

  22. [30]

    T., Yang, W., Su, J., and Liu, J

    Qian, R., Tan, R. T., Yang, W., Su, J., and Liu, J. Attentive generative adversarial network for raindrop removal from a single image. In CVPR, pp.\ 2482--2491, 2018

    2018

  23. [31]

    Deep video demoir \'e ing via compact invertible dyadic decomposition

    Quan, Y., Huang, H., He, S., and Xu, R. Deep video demoir \'e ing via compact invertible dyadic decomposition. In ICCV, pp.\ 12677--12686, 2023 a

    2023

  24. [32]

    Image desnowing via deep invertible separation

    Quan, Y., Tan, X., Huang, Y., Xu, Y., and Ji, H. Image desnowing via deep invertible separation. IEEE TCSVT, 33 0 (7): 0 3133--3144, 2023 b

    2023

  25. [33]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In ICML, pp.\ 8748--8763, 2021

    2021

  26. [34]

    Sam 2: Segment anything in images and videos

    Ravi, N., Gabeur, V., Hu, Y.-T., Hu, R., Ryali, C., Ma, T., Khedr, H., R \"a dle, R., Rolland, C., Gustafson, L., et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024

    2024 arXiv

  27. [35]

    U-net: Convolutional networks for biomedical image segmentation

    Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, pp.\ 234--241, 2015

    2015

  28. [36]

    and Stich, M

    Schaefer, G. and Stich, M. Ucid: An uncompressed color image database. In Storage and retrieval methods and applications for multimedia 2004, volume 5307, pp.\ 472--480. SPIE, 2003

    2004

  29. [37]

    Restoring images in adverse weather conditions via histogram transformer

    Sun, S., Ren, W., Gao, X., Wang, R., and Cao, X. Restoring images in adverse weather conditions via histogram transformer. ECCV, 2024

    2024

  30. [38]

    Resolution-robust large mask inpainting with fourier convolutions

    Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., and Lempitsky, V. Resolution-robust large mask inpainting with fourier convolutions. In WACV, pp.\ 2149--2159, 2022

    2022

  31. [39]

    Efficient flow-guided multi-frame de-fencing

    Tsogkas, S., Zhang, F., Jepson, A., and Levinshtein, A. Efficient flow-guided multi-frame de-fencing. In WACV, pp.\ 1838--1847, 2023

    2023

  32. [40]

    Valanarasu, J. M. J., Yasarla, R., and Patel, V. M. Transweather: Transformer-based restoration of images degraded by adverse weather conditions. In CVPR, pp.\ 2353--2363, 2022

    2022

  33. [41]

    Context-aware pretraining for efficient blind image decomposition

    Wang, C., Zheng, Z., Quan, R., Sun, Y., and Yang, Y. Context-aware pretraining for efficient blind image decomposition. In CVPR, pp.\ 18186--18195, 2023 a

    2023

  34. [42]

    Smartassign: Learning a smart knowledge assignment strategy for deraining and desnowing

    Wang, Y., Ma, C., and Liu, J. Smartassign: Learning a smart knowledge assignment strategy for deraining and desnowing. In CVPR, pp.\ 3677--3686, 2023 b

    2023

  35. [43]

    Single image reflection removal beyond linearity

    Wen, Q., Tan, Y., Qin, J., Liu, W., Han, G., and He, S. Single image reflection removal beyond linearity. In CVPR, pp.\ 3771--3779, 2019

    2019

  36. [44]

    Mask-guided progressive network for joint raindrop and rain streak removal in videos

    Wu, H., Yang, Y., Chen, H., Ren, J., and Zhu, L. Mask-guided progressive network for joint raindrop and rain streak removal in videos. In MM, pp.\ 7216--7225, 2023

    2023

  37. [45]

    T., Feng, J., Liu, J., Guo, Z., and Yan, S

    Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., and Yan, S. Deep joint rain detection and removal from a single image. In CVPR, pp.\ 1357--1366, 2017

    2017

  38. [46]

    W., Arora, A., Khan, S., Hayat, M., Khan, F

    Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., and Yang, M.-H. Restormer: Efficient transformer for high-resolution image restoration. In CVPR, pp.\ 5728--5739, 2022

    2022

  39. [47]

    Learning pyramid-context encoder network for high-quality image inpainting

    Zeng, Y., Fu, J., Chao, H., and Guo, B. Learning pyramid-context encoder network for high-quality image inpainting. In CVPR, pp.\ 1486--1494, 2019

    2019

  40. [48]

    Ff-former: Swin fourier transformer for nighttime flare removal

    Zhang, D., Ouyang, J., Liu, G., Wang, X., Kong, X., and Jin, Z. Ff-former: Swin fourier transformer for nighttime flare removal. In CVPR, pp.\ 2824--2832, 2023 a

    2023

  41. [49]

    Fast user-guided single image reflection removal via edge-aware cascaded networks

    Zhang, H., Xu, X., He, H., He, S., Han, G., Qin, J., and Wu, D. Fast user-guided single image reflection removal via edge-aware cascaded networks. IEEE TMM, 22 0 (8): 0 2012--2023, 2019

    2012

  42. [50]

    Vision-language models for vision tasks: A survey

    Zhang, J., Huang, J., Jin, S., and Lu, S. Vision-language models for vision tasks: A survey. IEEE TPAMI, 2024

    2024

  43. [51]

    Single image reflection separation with perceptual losses

    Zhang, X., Ng, R., and Chen, Q. Single image reflection separation with perceptual losses. In CVPR, pp.\ 4786--4794, 2018

    2018

  44. [52]

    Data-driven single image deraining: A comprehensive review and new perspectives

    Zhang, Z., Wei, Y., Zhang, H., Yang, Y., Yan, S., and Wang, M. Data-driven single image deraining: A comprehensive review and new perspectives. PR, 143: 0 109740, 2023 b

    2023

  45. [53]

    Improving lens flare removal with general-purpose pipeline and multiple light sources recovery

    Zhou, Y., Liang, D., Chen, S., Huang, S.-J., Yang, S., and Li, C. Improving lens flare removal with general-purpose pipeline and multiple light sources recovery. In ICCV, pp.\ 12969--12979, 2023

    2023

  46. [54]

    Zero-shot object counting with good exemplars

    Zhu, H., Yuan, J., Yang, Z., Guo, Y., Wang, Z., Zhong, X., and He, S. Zero-shot object counting with good exemplars. ECCV, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.