Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

RORem: Training a Robust Object Remover with Human-in-the-Loop

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Object remover trained with human feedback beats rivals by 18 points.

desk verdict Solid dataset-plus-pipeline paper whose headline 18% margin is plausible but rests on a small, under-reported user study; worth reviewing for the 200K-pair dataset alone. read the letter →

arxiv 2501.00740 v3 pith:SJNKBMIE submitted 2025-01-01 cs.CV

Ruibin Li , Tao Yang , Song Guo , Lei Zhang This is my paper · ORCID
classification cs.CV
keywords objectremovalimageinpaintinghuman-in-the-loopsemi-supervisedlearningdiffusionmodelstrainingdatagenerationdiscriminatormodeldistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the low success rate of object removal models comes from a training-data problem: self-supervised inpainting teaches the model to reconstruct whatever was masked, so at test time it hesitates between redrawing the object and restoring the background. The fix is a semi-supervised, human-in-the-loop pipeline that builds a large dataset of true removal triplets (source image, mask, clean removal result). Starting from 60K triplets from existing datasets, an initial model proposes removals, humans filter the good ones, and a discriminator trained on those judgments automates further filtering; several rounds produce over 200K pairs. Fine-tuning an SDXL inpainting model on this data yields RORem, which human judges mark as successful 76.2% of the time at 512x512 and 70.2% at 1024x1024, about 18 points above the next-best method. A distilled four-step version runs in about half a second per image, making the reliability gain usable in practice.

What carries the argument

The load-bearing object is the training triplet (source image, mask, true removal result), which removes the self-supervised ambiguity. Around it the paper builds a data flywheel: an initial SDXL inpainting model proposes removals, human annotators label each output as success or failure, and a discriminator built from the down/middle blocks of SDXL with trainable LoRA layers learns to predict those labels, scoring candidate pairs above a 0.9 threshold for automatic inclusion. The same triplet formulation is used in every training round, with the source image's masked region blanked before concatenation, and the final fast model is obtained by distilling the trained RORem into four diffusion steps using a latent-consistency-style objective with LoRA layers.

What would settle it

Recruit a fresh panel of at least 20 annotators who are blind to method identity and method order, have them judge the same 500 pairs at each resolution, and compute the mean success-rate gap between RORem and the runner-up with a 95% confidence interval; if the interval includes zero or the gap is materially below 18 points, the paper's central performance claim fails.

Watch

Extended reading notes

Core claim

The central claim is that object removal reliability is bottlenecked by the ambiguity of self-supervised random-mask training, and that the bottleneck can be broken by supplying the model with ground-truth removal pairs at scale. The paper shows that a model trained on 60K pairs from existing video and synthetic datasets succeeds less than half the time, but that each round of human filtering plus discriminator-automated filtering adds tens of thousands of high-quality triplets and steadily raises success rate from 7.6% to 76.2% as the training set grows past 200K. The resulting RORem outperforms seven prior methods on human-judged success rate at both 512x512 and 1024x1024, and its distilled variant keeps most of the gain while cutting inference time by roughly 88% at 512x512.

Load-bearing premise

The headline 18-point advantage rests on five volunteers' yes/no judgments of 500 image pairs per resolution, with no reported inter-rater agreement, variance, or confidence intervals; if those judgments are noisy or biased toward the authors' model, the margin is not established.

Editorial extensions

If this is right

  • At 512x512, human judges rate RORem as successful on 76.2% of cases versus 55.8% for the second-best method, and at 1024x1024 the margin is 70.2% versus 52.4%.
  • The 200K-pair dataset and trained model are released publicly, letting downstream applications fine-tune on top of a reliable removal prior rather than starting from ambiguous self-supervised weights.
  • The four-step distilled variant runs in 0.50 seconds per image at 512x512 and 0.83 seconds at 1024x1024, with success rate drops of only 1.4 and 2.8 points.
  • Success rate rises monotonically with dataset size across the annotation rounds, from 38.6% after initialization to 76.2% after the final high-resolution fine-tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same triplet-plus-discriminator flywheel should transfer to other image-editing tasks such as object insertion, replacement, or style harmonization, because the paper's mechanism only assumes that a preference discriminator can imitate human quality judgments.
  • A stronger test of the method would be an independent user study with more annotators, inter-annotator agreement, and confidence intervals; the paper reports averages only, so the 18-point claim is as strong as those five volunteers' consistency.
  • The released discriminator could serve as a cheap automated evaluation metric for future object removers, provided it is periodically re-calibrated on new failure modes, since the paper shows it aligns with human labels within about 3% on its own method.
  • A direct extension would test RORem on images outside the OpenImages sampling distribution (e.g., user-captured photos with unusual backgrounds), since the test set is filtered by the same keyword and mask-size rules as the training data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces RORem, a diffusion-based object remover trained with a human-in-the-loop data generation pipeline. Starting from two open-source datasets (RORD and Mulan) with about 60K triplets, the authors fine-tune an SDXL inpainting model, use it to generate candidate removal pairs from OpenImages, filter them through human annotation and a learned discriminator Dphi, iteratively retrain, and finally obtain 201,134 training pairs. A four-step distilled variant (RORem-4S) runs in about 0.5 seconds per image. The central claim, stated in the abstract, is that RORem improves the object removal success rate over previous methods by more than 18%, based on Table 2's human-judged success rates (76.2% vs 55.8% at 512x512; 70.2% vs 52.4% at 1024x1024).

Significance. If the reported success-rate gains are confirmed, RORem would be a substantial practical advance in object removal reliability, and the released dataset, code, and model would be a useful community resource. The paper also demonstrates a credible pipeline for semi-supervised data collection with human feedback and a strong distilled efficient variant. However, the headline claim is currently supported mainly by a small, possibly unblinded user study, and the paper's secondary Dphi corroboration is not independent; these issues must be addressed before the central claim can be considered established.

major comments (3)
  1. [Sec. 4.1 and Appendix A] The headline claim of 'more than 18%' improvement is established solely by the user study in Sec. 4.1 and Appendix A, which used five volunteers, each annotating 500 pairs per resolution, and reports only the averaged success rate without inter-annotator agreement, per-method variance, or confidence intervals. Because the volunteer recruitment and their relationship to the authors are not described, and because the interface in Fig. 9 shows the editing results of all methods side by side without blinding, the measured 20.4-point (512) and 17.8-point (1024) margins may be systematically biased toward RORem. Please provide a blinded, preregistered user study with a larger and independent rater pool, report agreement statistics (e.g., Fleiss' kappa) and per-method confidence intervals, and describe how volunteers were recruited and instructed.
  2. [Abstract and Sec. 4.2] The abstract's 'more than 18%' is inaccurate for the 1024x1024 resolution, where Table 2 shows a margin of 70.2% - 52.4% = 17.8 percentage points. In addition, Sec. 4.2 states that Dphi's success rates deviate from human annotation by less than 3% in most cases, but Table 2 shows deviations of 6.8 points for Lama at 512 (55.4 vs 48.6), 7.8 points for PPT at 1024 (46.8 vs 54.6), and 5.4 points for CLIPAway at 1024 (23.8 vs 29.2). Please rephrase the quantitative claims and re-analyze the Dphi alignment, or restrict the claim to the 512-resolution result.
  3. [Sec. 4.3 and Table 3] The Dphi metric is not an independent corroboration of the human study. Dphi is trained on labels from the same human-feedback protocol, partly on RORem outputs (Table 4), and its validation in Table 3 and Appendix B uses a test set of RORem-generated outputs only. Therefore, the Dphi success-rate columns in Table 2 for the baseline methods are not validated, and the statement that Dphi 'effectively aligns with human preferences' overstates the evidence. Please either validate Dphi on held-out human labels for each method separately or remove the Dphi-based success-rate comparisons from the main table.
minor comments (5)
  1. [Sec. 4.1] The test protocol applies a substantial mask dilation (kernel sizes 50 and 100 for 512 and 1024 resolutions) to accommodate methods like PPT, but the manuscript does not state whether RORem was trained with masks of comparable dilation. If RORem was trained on tighter masks, the shared dilated mask could disadvantage RORem or, conversely, the dilation could mask RORem's sensitivity to fine masks. Please clarify the mask preprocessing for training versus testing and, if necessary, evaluate with both original and dilated masks.
  2. [Sec. 3.2 and Appendix A] The relationship between the training-time human annotation (10 annotators) and the final evaluation (5 volunteers) should be clarified; currently the text in Sec. 4.1 and Appendix A could be misread as the same group, leading to confusion about whether the final evaluation is independent.
  3. [Table 4] Table 4 contains a typo: 'dowmsample' should be 'downsample', and the column header 'Baseline' should be 'Baselines'. Please correct these in the final version.
  4. [Sec. 3.4] The phrase 'four diffussion steps' should be corrected to 'four diffusion steps'. Additionally, the rank of LoRA layers (64 for distillation, 4 for Dphi) should be mentioned explicitly in the main text to avoid ambiguity.
  5. [Sec. 4.1] Because the test set is sampled from OpenImages and filtered with the same keyword and mask-size rules as the training data, the evaluation is in-distribution for RORem. The paper should acknowledge this as a limitation and ideally include an additional out-of-distribution test set to ensure the reported improvement is not an artifact of distribution matching.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the headline success-rate claim rests on an independent human user study, and the Dphi proxy is a trained evaluator, not a fitted input to the target claim.

full rationale

The paper's central claim is that RORem improves object-removal success rate by more than 18%, and this is operationalized in Table 2 through a user study in which five volunteers judge 500 test pairs per resolution against the original image and mask. That evaluation is external to the training loop and not derived from the model's own training objective. The human-in-the-loop data-generation pipeline is a self-bootstrap, but it is not a definitional reduction: the model generates candidate pairs, humans label them, a discriminator approximates those labels, accepted pairs are added to the training set, and the model is retrained. The final success rate is measured on held-out human judgments, not on the discriminator's training labels, so the improvement claim is not forced by construction. The Dphi discriminator is trained to mimic human labels and is then used as a secondary evaluation metric; even if this proxy is imperfect or biased, it is not used to fit RORem's parameters, and the paper explicitly augments Dphi's training data with baseline-method outputs, synthetic degradations, and RORD positives. No load-bearing self-citation or imported uniqueness theorem appears: the authors' prior works cited as references [24] and [25] are mentioned only in the related-work discussion of inversion-based editing methods and are not used to justify the RORem architecture or training strategy. The discrepancy between the paper's claim that Dphi deviates from human annotation by less than 3% in most cases and the larger deviations visible in Table 2 is a correctness/statistics concern about the proxy, not a circularity in the main derivation. Overall, the central result is supported by an independent evaluation protocol, and no step in the claimed derivation reduces to its own inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The ledger lists the protocol choices and assumptions the central claims rest on. Routine training hyperparameters (learning rate 5e-5, batch 192, 50K steps per round, LoRA ranks 4 and 64, k=20 for distillation) are treated as standard engineering knobs rather than free parameters; the entries above are the hand-chosen decisions that shape the dataset, the evaluation, and the headline numbers.

free parameters (4)
  • Dphi selection threshold = 0.9
    Samples with discriminator confidence above 0.9 are admitted to the training set; hand-chosen to trade off dataset size against precision (Sec. 3.3).
  • Mask dilation kernel sizes = 50 (512x512), 100 (1024x1024)
    Test masks are dilated with OpenCV to accommodate methods that struggle with fine-grained masks; protocol choice that affects all reported success rates (Sec. 4.1).
  • Mask size filter for sampling = 3%-70% of image
    OpenImages samples with masks smaller than 3% or larger than 70% are excluded from both training and test sets; hand-chosen and applies to both distributions (Sec. 4.1).
  • Per-class sampling cap = 500 instances per class
    Sampling of a class stops once 500 instances are collected, to control category balance (Sec. 3.2).
assumptions (4)
  • domain assumption Binary human yes/no labels are a reliable ground truth for removal quality
    Used for annotator feedback (Sec. 3.2), user study (Sec. 4.1), and as supervision for Dphi (Eq. 2); no agreement statistics are reported.
  • domain assumption An SDXL-inpainting backbone fine-tuned on this data can solve the object-removal task
    The model is initialized from the pretrained SDXL-inpainting checkpoint (Sec. 3.1); the paper assumes this prior is a sufficient inductive base.
  • domain assumption The OpenImages test set is representative of object-removal deployment
    Both training and test pairs are sampled from OpenImages with identical preprocessing and filters (Sec. 4.1); no cross-dataset evaluation is reported.
  • ad hoc to paper High-precision filtering by Dphi monotonically improves the training set
    The iterative bootstrap (Sec. 3.3) assumes that adding only samples with Dphi score at least 0.9 improves the model; the causal contribution of filter quality is not isolated from training-round compute.
invented entities (1)
  • Dphi (preference-aligned discriminator) independent evidence
    purpose: Automated annotation of removal pairs and as an auxiliary evaluation metric
    Dphi is trained on human-labeled quadruples and evaluated against a held-out human-labeled test set (Tab. 3, Sec. 4.3), so it has a falsifiable handle; however, its positive training class is dominated by RORem outputs (17,322 of 18,859 positives), biasing it as an evaluator of other methods.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RORem: Training a Robust Object Remover with Human-in-the-Loop." pith.science (2026). https://pith.science/paper/SJNKBMIE

@misc{pith2026250100740,
  author       = {Pith},
  title        = {Pith review of: RORem: Training a Robust Object Remover with Human-in-the-Loop},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SJNKBMIE}},
  note         = {Machine review of arXiv:2501.00740}
}
read the original abstract

Despite the significant advancements, existing object removal methods struggle with incomplete removal, incorrect content synthesis and blurry synthesized regions, resulting in low success rates. Such issues are mainly caused by the lack of high-quality paired training data, as well as the self-supervised training paradigm adopted in these methods, which forces the model to in-paint the masked regions, leading to ambiguity between synthesizing the masked objects and restoring the background. To address these issues, we propose a semi-supervised learning strategy with human-in-the-loop to create high-quality paired training data, aiming to train a Robust Object Remover (RORem). We first collect 60K training pairs from open-source datasets to train an initial object removal model for generating removal samples, and then utilize human feedback to select a set of high-quality object removal pairs, with which we train a discriminator to automate the following training data generation process. By iterating this process for several rounds, we finally obtain a substantial object removal dataset with over 200K pairs. Fine-tuning the pre-trained stable diffusion model with this dataset, we obtain our RORem, which demonstrates state-of-the-art object removal performance in terms of both reliability and image quality. Particularly, RORem improves the object removal success rate over previous methods by more than 18\%. The dataset, source code and trained model are available at https://github.com/leeruibin/RORem.

Figures

Figures reproduced from arXiv: 2501.00740 by the authors.

Figure 1
Figure 1. Given an input image and a mask (see (a)), existing [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our training data generation and model training process. In stage 1, we gather 60K training triplets from open [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. We finetune the pre-trained SDXL-inpaiting model with [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Training of the discriminator for automated data anno [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Efficient model distillation. We integrate trainable LoRA [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: The category distribution of our constructed dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Visual comparison of the object removal results by RORem and other methods on [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: The interface for human annotation. The annotators are asked to give feedback by clicking ”Yes” or ”No” button. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: The interface for human evaluation. The volunteers make selections by checking the multiple-choice check-boxes at the bottom [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Confusion matrix and the definition of metrics for evaluating our discriminator. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Visual results of RORem at each training round, one can see that the removal capacity of RORem improves with the increase of [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Visual results of IP2P and CLIPAway on 1024 × 1024 resolution images. E. Visual results on images of 512 × 512 resolution The qualitative comparisons on images of 512 × 512 resolution are illustrated in [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]
Figure 13
Figure 13. Figure 13: Visual comparison of the object removal results by RORem and other methods on [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Failure cases of RORem. 6 [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OmniEraser: Remove Objects and Their Effects in Images with Paired Video-Frame Data

    cs.CV 2025-01 conditional novelty 6.0 of 10

    OmniEraser removes objects along with their shadows and reflections by conditioning a FLUX diffusion model on separate object and background latents, trained on a 134,281-sample video-derived dataset.

Reference graph

Works this paper leans on

72 extracted references · 45 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gradio: Hassle-free sharing and testing of ml models in the wild

    Abubakar Abid, Ali Abdalla, Ali Abid, Dawood Khan, Ab- dulrahman Alfozan, and James Zou. Gradio: Hassle-free sharing and testing of ml models in the wild. arXiv preprint arXiv:1906.02569, 2019. 1

    arXiv 1906

  2. [2]

    Ntire 2017 challenge on single image super-resolution: Dataset and study

    Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In The IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR) Workshops, 2017. 5, 3

    work page 2017

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 3

    arXiv 2023

  4. [4]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18392–18402, 2023. 3, 6, 8

    work page 2023

  5. [5]

    Zits++: Im- age inpainting by improving the incremental transformer on structural priors

    Chenjie Cao, Qiaole Dong, and Yanwei Fu. Zits++: Im- age inpainting by improving the incremental transformer on structural priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10):12667–12684, 2023. 3

    work page 2023

  6. [6]

    Re- gion filling and object removal by exemplar-based image in- painting

    Antonio Criminisi, Patrick P ´erez, and Kentaro Toyama. Re- gion filling and object removal by exemplar-based image in- painting. IEEE Transactions on image processing , 13(9): 1200–1212, 2004. 1

    work page 2004

  7. [7]

    Incremental transformer structure enhanced image inpainting with mask- ing positional encoding

    Qiaole Dong, Chenjie Cao, and Yanwei Fu. Incremental transformer structure enhanced image inpainting with mask- ing positional encoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11358–11368, 2022. 1, 3

    work page 2022

  8. [8]

    Clipaway: Harmonizing focused embeddings for removing objects via diffusion models

    Yigit Ekin, Ahmet Burak Yildirim, Erdem Eren Caglar, Aykut Erdem, Erkut Erdem, and Aysegul Dundar. Clipaway: Harmonizing focused embeddings for removing objects via diffusion models. arXiv preprint arXiv:2406.09368, 2024. 6, 8

    arXiv 2024

Show all 72 references
  1. [9]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik 9 Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Ma...

    2024

  2. [10]

    Instructdiffusion: A generalist modeling inter- face for vision tasks

    Zigang Geng, Binxin Yang, Tiankai Hang, Chen Li, Shuyang Gu, Ting Zhang, Jianmin Bao, Zheng Zhang, Houqiang Li, Han Hu, et al. Instructdiffusion: A generalist modeling inter- face for vision tasks. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Reco...

    2024

  3. [11]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commu- nications of the ACM, 63(11):139–144, 2020. 3

    2020

  4. [12]

    Shadowd- iffusion: When degradation prior meets diffusion model for shadow removal

    Lanqing Guo, Chong Wang, Wenhan Yang, Siyu Huang, Yufei Wang, Hanspeter Pfister, and Bihan Wen. Shadowd- iffusion: When degradation prior meets diffusion model for shadow removal. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 140...

    2023

  5. [13]

    Proxedit: Improving tuning-free real image editing with proximal guidance

    Ligong Han, Song Wen, Qi Chen, Zhixing Zhang, Kunpeng Song, Mengwei Ren, Ruijiang Gao, Anastasis Stathopou- los, Xiaoxiao He, Yuxiao Chen, et al. Proxedit: Improving tuning-free real image editing with proximal guidance. In Proceedings of the IEEE/CVF Winter Conference on Appl...

    2024

  6. [14]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 3

    2022 arXiv

  7. [15]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3, 5

    2020

  8. [16]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 4

    2021 arXiv

  9. [17]

    Designedit: Multi- layered latent decomposition and fusion for unified & accu- rate image editing

    Yueru Jia, Yuhui Yuan, Aosong Cheng, Chuke Wang, Ji Li, Huizhu Jia, and Shanghang Zhang. Designedit: Multi- layered latent decomposition and fusion for unified & accu- rate image editing. arXiv preprint arXiv:2403.14487, 2024. 3, 6, 8

    2024 arXiv

  10. [18]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 5

    2023

  11. [19]

    Continuously masked transformer for image inpainting

    Keunsoo Ko and Chang-Su Kim. Continuously masked transformer for image inpainting. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 13169–13178, 2023. 1, 3

    2023

  12. [20]

    The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale

    Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Ui- jlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection...

    1956

  13. [21]

    Prior guided gan based semantic inpainting

    Avisek Lahiri, Arnav Kumar Jain, Sanskar Agrawal, Pabitra Mitra, and Prabir Kumar Biswas. Prior guided gan based semantic inpainting. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 13696–13705, 2020. 1

    2020

  14. [22]

    Rgbd2: Generative scene synthesis via incremental view inpainting using rgbd diffusion models

    Jiabao Lei, Jiapeng Tang, and Kui Jia. Rgbd2: Generative scene synthesis via incremental view inpainting using rgbd diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 8422–8434, 2023. 1

    2023

  15. [23]

    Recurrent feature reasoning for image inpainting

    Jingyuan Li, Ning Wang, Lefei Zhang, Bo Du, and Dacheng Tao. Recurrent feature reasoning for image inpainting. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 7760–7768, 2020. 3

    2020

  16. [24]

    Freepih: Training-free painterly image harmoniza- tion with diffusion model

    Ruibin Li, Jingcai Guo, Song Guo, Qihua Zhou, and Jie Zhang. Freepih: Training-free painterly image harmoniza- tion with diffusion model. CoRR, abs/2311.14926, 2023. 1

    2023 arXiv

  17. [25]

    Source prompt disentangled inversion for boosting image editability with diffusion models

    Ruibin Li, Ruihuang Li, Song Guo, and Lei Zhang. Source prompt disentangled inversion for boosting image editability with diffusion models. In European Conference on Com- puter Vision, 2024. 3

    2024

  18. [26]

    Mat: Mask-aware transformer for large hole im- age inpainting

    Wenbo Li, Zhe Lin, Kun Zhou, Lu Qi, Yi Wang, and Ji- aya Jia. Mat: Mask-aware transformer for large hole im- age inpainting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10758– 10768, 2022. 1, 3

    2022

  19. [27]

    Cv2 dilate function, 2024

    Open Computer Vision Library. Cv2 dilate function, 2024. 6

    2024

  20. [28]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  21. [29]

    Image inpainting for ir- regular holes using partial convolutions

    Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. Image inpainting for ir- regular holes using partial convolutions. In Proceedings of the European conference on computer vision (ECCV), pages 85–100, 2018. 3

    2018

  22. [30]

    Coherent semantic attention for image inpainting

    Hongyu Liu, Bin Jiang, Yi Xiao, and Chao Yang. Coherent semantic attention for image inpainting. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4170–4179, 2019. 3

    2019

  23. [31]

    Rethinking image inpainting via a mutual encoder- decoder with feature equalizations

    Hongyu Liu, Bin Jiang, Yibing Song, Wei Huang, and Chao Yang. Rethinking image inpainting via a mutual encoder- decoder with feature equalizations. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, Au- gust 23–28, 2020, Proceedings, Part II 16 , pages 725–7...

    2020

  24. [32]

    Pd-gan: Probabilistic diverse gan for image inpainting

    Hongyu Liu, Ziyu Wan, Wei Huang, Yibing Song, Xintong Han, and Jing Liao. Pd-gan: Probabilistic diverse gan for image inpainting. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9371–9381, 2021. 1

    2021

  25. [33]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 6

    2024

  26. [34]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 6 10

    2017 arXiv

  27. [35]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11461–11471, 2022. 1, 3

    2022

  28. [36]

    Latent consistency models: Synthesizing high- resolution images with few-step inference

    Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high- resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023. 2, 5

    2023 arXiv

  29. [37]

    Regionwise gen- erative adversarial image inpainting for large missing areas

    Yuqing Ma, Xianglong Liu, Shihao Bai, Lei Wang, Aishan Liu, Dacheng Tao, and Edwin R Hancock. Regionwise gen- erative adversarial image inpainting for large missing areas. IEEE transactions on cybernetics , 53(8):5226–5239, 2022. 1, 3

    2022

  30. [38]

    Defacto: Image and face manipulation dataset

    Ga ¨el Mahfoudi, Badr Tajini, Florent Retraint, Frederic Morain-Nicolier, Jean Luc Dugelay, and PIC Marc. Defacto: Image and face manipulation dataset. In 2019 27Th euro- pean signal processing conference (EUSIPCO) , pages 1–5. IEEE, 2019. 3

    2019

  31. [39]

    Null-text inversion for editing real images using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. arXiv preprint arXiv:2211.09794, 2022. 3

    2022 arXiv

  32. [40]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 6

    2023 arXiv

  33. [41]

    Context encoders: Feature learning by inpainting

    Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 2536–2544, 2016. 1, 3

    2016

  34. [42]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1, 2, 3, 4, 6, 8

    2023 arXiv

  35. [43]

    Deep learning-based image and video inpainting: A survey

    Weize Quan, Jiaxi Chen, Yanli Liu, Dong-Ming Yan, and Pe- ter Wonka. Deep learning-based image and video inpainting: A survey. International Journal of Computer Vision, 132(7): 2367–2400, 2024. 1

    2024

  36. [44]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  37. [45]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  38. [46]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  39. [47]

    Rord: A real-world object removal dataset

    Min-Cheol Sagong, Yoon-Jae Yeo, Seung-Won Jung, and Sung-Jea Ko. Rord: A real-world object removal dataset. In BMVC, page 542, 2022. 1, 2, 3, 6

    2022

  40. [48]

    Palette: Image-to-image diffusion models

    Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10,

    2022

  41. [49]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural In- f...

    2022

  42. [50]

    Transinpaint: Transformer-based image inpainting with context adaptation

    Pourya Shamsolmoali, Masoumeh Zareapoor, and Eric Granger. Transinpaint: Transformer-based image inpainting with context adaptation. InProceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 849–858,

  43. [51]

    Emu edit: Precise image editing via recognition and gen- eration tasks

    Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and gen- eration tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pa...

    2024

  44. [52]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3, 5

    2010 arXiv

  45. [53]

    Contextual-based image inpainting: Infer, match, and translate

    Yuhang Song, Chao Yang, Zhe Lin, Xiaofeng Liu, Qin Huang, Hao Li, and C-C Jay Kuo. Contextual-based image inpainting: Infer, match, and translate. In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018. 3

    2018

  46. [54]

    Resolution-robust large mask inpainting with fourier convolutions

    Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robust large mask inpainting with fourier convolutions. InProceedings of the IEEE/CVF winter...

    2022

  47. [55]

    Ntire 2017 challenge on single image super-resolution: Methods and results

    Radu Timofte, Eirikur Agustsson, Luc Van Gool, Ming- Hsuan Yang, and Lei Zhang. Ntire 2017 challenge on single image super-resolution: Methods and results. In Proceed- ings of the IEEE conference on computer vision and pattern recognition workshops, pages 114–125, 2017. 5, 3

    2017

  48. [56]

    Mulan: A multi layer anno- tated dataset for controllable text-to-image generation

    Petru-Daniel Tudosiu, Yongxin Yang, Shifeng Zhang, Fei Chen, Steven McDonagh, Gerasimos Lampouras, Ignacio Iacobacci, and Sarah Parisot. Mulan: A multi layer anno- tated dataset for controllable text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vi...

    2024

  49. [57]

    Perceptual adversarial networks for image-to-image 11 transformation

    Chaoyue Wang, Chang Xu, Chaohui Wang, and Dacheng Tao. Perceptual adversarial networks for image-to-image 11 transformation. IEEE Transactions on Image Processing, 27 (8):4066–4079, 2018. 1

    2018

  50. [58]

    Imagen editor and editbench: Advancing and evaluating text-guided im- age inpainting

    Su Wang, Chitwan Saharia, Ceslee Montgomery, Jordi Pont- Tuset, Shai Noy, Stefano Pellegrini, Yasumasa Onoe, Sarah Laszlo, David J Fleet, Radu Soricut, et al. Imagen editor and editbench: Advancing and evaluating text-guided im- age inpainting. In Proceedings of the IEEE/CVF c...

    2023

  51. [59]

    Objectdrop: Bootstrap- ping counterfactuals for photorealistic object removal and in- sertion

    Daniel Winter, Matan Cohen, Shlomi Fruchter, Yael Pritch, Alex Rav-Acha, and Yedid Hoshen. Objectdrop: Bootstrap- ping counterfactuals for photorealistic object removal and in- sertion. arXiv preprint arXiv:2403.18818, 2024. 1, 3, 4, 6

    2024 arXiv

  52. [60]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 22428–22437, 2023. 1

    2023

  53. [61]

    Shift-net: Image inpainting via deep feature rearrangement

    Zhaoyi Yan, Xiaoming Li, Mu Li, Wangmeng Zuo, and Shiguang Shan. Shift-net: Image inpainting via deep feature rearrangement. In Proceedings of the European conference on computer vision (ECCV), pages 1–17, 2018. 3

    2018

  54. [62]

    High-resolution image inpainting using multi- scale neural patch synthesis

    Chao Yang, Xin Lu, Zhe Lin, Eli Shechtman, Oliver Wang, and Hao Li. High-resolution image inpainting using multi- scale neural patch synthesis. InProceedings of the IEEE con- ference on computer vision and pattern recognition , pages 6721–6729, 2017. 3

    2017

  55. [63]

    Se- mantic image inpainting with deep generative models

    Raymond A Yeh, Chen Chen, Teck Yian Lim, Alexander G Schwing, Mark Hasegawa-Johnson, and Minh N Do. Se- mantic image inpainting with deep generative models. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5485–5493, 2017. 3

    2017

  56. [64]

    Inst-inpaint: Instructing to remove objects with diffusion models

    Ahmet Burak Yildirim, Vedat Baday, Erkut Erdem, Aykut Erdem, and Aysegul Dundar. Inst-inpaint: Instructing to remove objects with diffusion models. arXiv preprint arXiv:2304.03246, 2023. 1, 3, 6, 8

    2023 arXiv

  57. [65]

    Generative image inpainting with con- textual attention

    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. Generative image inpainting with con- textual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5505–5514,

  58. [66]

    Diverse image inpainting with bidirec- tional and autoregressive transformers

    Yingchen Yu, Fangneng Zhan, Rongliang Wu, Jianxiong Pan, Kaiwen Cui, Shijian Lu, Feiying Ma, Xuansong Xie, and Chunyan Miao. Diverse image inpainting with bidirec- tional and autoregressive transformers. InProceedings of the 29th ACM International Conference on Multimedia , pa...

    2021

  59. [67]

    Learning pyramid-context encoder network for high- quality image inpainting

    Yanhong Zeng, Jianlong Fu, Hongyang Chao, and Baining Guo. Learning pyramid-context encoder network for high- quality image inpainting. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1486–1494, 2019. 3

    2019

  60. [68]

    Learning joint spatial-temporal transformations for video inpainting

    Yanhong Zeng, Jianlong Fu, and Hongyang Chao. Learning joint spatial-temporal transformations for video inpainting. In Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16, pages 528–543. Springer, 2020. 1

    2020

  61. [69]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6

    2018

  62. [70]

    Hive: Harnessing human feedback for instructional visual editing

    Shu Zhang, Xinyi Yang, Yihao Feng, Can Qin, Chia-Chih Chen, Ning Yu, Zeyuan Chen, Huan Wang, Silvio Savarese, Stefano Ermon, et al. Hive: Harnessing human feedback for instructional visual editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2024

  63. [71]

    Propainter: Improving propagation and transformer for video inpainting

    Shangchen Zhou, Chongyi Li, Kelvin CK Chan, and Chen Change Loy. Propainter: Improving propagation and transformer for video inpainting. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10477–10486, 2023. 3

    2023

  64. [72]

    A task is worth one word: Learning with task prompts for high-quality versatile image inpainting

    Junhao Zhuang, Yanhong Zeng, Wenran Liu, Chun Yuan, and Kai Chen. A task is worth one word: Learning with task prompts for high-quality versatile image inpainting. arXiv preprint arXiv:2312.03594, 2023. 1, 3, 6, 8 12 RORem: Training a Robust Object Remover with Human-in-the-Lo...

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.