Pith. sign in

REVIEW 3 major objections 5 minor 59 references

Can We Challenge Open-Vocabulary Object Detectors with Generated Content in Street Scenes?

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Open-vocabulary detectors fail by object location, not object type.

desk verdict A useful synthetic stress-test pipeline with a promising location-over-semantics claim that is under-supported because the random-location experiments don't control for inpainting fidelity or bbox alignment. read the letter →

arxiv 2506.23751 v1 pith:4YJYEZUL submitted 2025-06-30 cs.CV

Annika Mütze , Sadia Ilyas , Christian Dörpelkus , Matthias Rottmann This is my paper
classification cs.CV
keywords open-vocabularyobjectdetectionsyntheticdatadiffusioninpaintingstreetscenesfailuremodesfalsenegativeslocationGroundingDINO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether objects generated by diffusion inpainting and placed into real street scenes can expose limitations of open-vocabulary object detectors, and whether those limitations are systematic. The authors build two automated pipelines that insert semantically diverse objects into LostAndFound and NuImages frames, then test Grounding DINO, YOLO-World, MDETR, OmDet, and Faster R-CNN with five different text prompts. They find that the synthetic content produces false negatives at rates comparable to real out-of-distribution data, and that the location of the inpainted object and the prompt have more influence on detection than the object's semantic class. If true, this means open-vocabulary detectors have spatial blind spots that can be mapped ahead of deployment and targeted with better data, rather than limits that only appear when a new category is encountered.

What carries the argument

The load-bearing mechanism is a pair of automated stable-diffusion inpainting pipelines that turn real street frames into controlled test cases. The hybrid-concept pipeline samples up to four substantives from WordNet, composes prompts such as 'word1_word2_wordn_hybrid', and inpaints into an oval mask derived from the original bounding box, repeated ten times per image to yield 3,580 images. The single-concept pipeline takes a ChatGPT-supplied list of objects unusual for street scenes and inpaints them on drivable regions with one of three crop sizes, yielding 4,102 images after manual filtering. The analysis device that carries the location claim is the pixel-wise recall heatmap: 2,000 random road locations per scene are inpainted with a fixed 100x130 box, and the resulting false-negative positions are mapped onto the scene so spatial clusters become visible.

What would settle it

Place a fixed set of real objects at the same image locations used in the random-location inpainting study, run the same detectors with the same prompts, and compare pixel-wise false-negative maps; if the spatial clusters disappear with real objects, the reported blind spots are artifacts of inpainting quality rather than genuine detector limitations.

Watch

Extended reading notes

Core claim

The paper's central claim is that open-vocabulary object detectors can be systematically challenged with diffusion-inpainted objects, and that the failures these challenges expose are governed primarily by object location and prompt wording rather than by the semantic class of the object. On real LostAndFound images and 3,580 hybrid-inpainted variants, Grounding DINO's AUPRC is similar on real and synthetic data, yet AP and AR drop sharply on synthetic objects, showing that clearly visible synthetic objects are often missed. When scenes are ranked by false-negative count, rank correlations between synthetic variants are between 0.90 and 0.96, meaning the same scenes keep producing misses under different object semantics. In the random-location experiments with a fixed 100x130 box and a fixed 'robot' object on NuImages, pixel-wise recall heatmaps show persistent false-negative clusters whose position changes from scene to scene and from model to model. The authors conclude that these clusters are spatial blind spots of the detectors, so generated content can locate them.

Load-bearing premise

The finding that location dominates semantics assumes the inpainting model produces objects of comparable visibility, size, and realism at every road location; if some regions yield systematically worse or smaller inpainted objects, the false-negative clusters could be generation artifacts rather than detector blind spots.

Editorial extensions

If this is right

  • Synthetic inpainted content can act as a controllable test bench: open-vocabulary detectors miss generated objects about as often as they miss real out-of-distribution objects, so models can be probed for weak spots before deployment.
  • False-negative clusters are scene-specific and model-specific, so aggregate metrics like AUPRC understate the risk that a particular road region will be overlooked.
  • Prompt wording shifts both overall performance and the location of failure clusters, making prompt selection a substantive part of the safety case.
  • Because object identity matters less than placement, collecting more object categories is not the obvious fix; collecting data from known failure locations is a more direct route to improvement.
  • Even with the object fixed to 'robot', clustered misses persist on NuImages, so the failure mode is not simply a vocabulary gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the claim would be to place real objects at the same mapped locations and re-measure false-negative rates; matching clusters would confirm the blind spots are properties of the detector, not of the inpainting model.
  • If location dominates, the same heatmap audit could be applied to other perception tasks such as semantic segmentation and multi-object tracking, where spatial priors may create analogous blind spots.
  • The location dependence is plausibly a learned spatial prior: training data concentrated on certain regions and scales could make detectors systematically weaker elsewhere, suggesting position-aware augmentation or scale rebalancing as a remedy.
  • Per-location generation fidelity is not verified in the paper, so a future study could score each inpainted patch for object visibility and realistic scale, then check whether false-negative clusters survive after controlling for that score.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates whether synthetically inpainted objects in street scenes can challenge open-vocabulary object detectors, and whether systematic failure modes can be identified. Two inpainting pipelines are proposed: hybrid-concept inpainting using WordNet-sampled substantives and single-concept inpainting using ChatGPT-sampled objects. The authors evaluate five detectors (MDETR, OmDet, Grounding DINO, YOLO-World, Faster R-CNN) on synthetic versions of LostAndFound and NuImages, reporting AUPRC/AP/AR, TP/FP/FN counts, per-scene rank correlations, and per-pixel recall heatmaps. The central claim is that object location and prompt choice have a greater influence on detection performance than the semantic content of the generated objects, with location identified as a highly influential factor.

Significance. If the central claim is correct, this work offers a practical, automated test bench for probing open-vocabulary detectors in safety-critical settings and suggests that failure modes are spatially predictable. The paper is transparent about its protocol, includes multiple models and prompts, reports manual filtering for the fixed-location datasets, and provides additional control experiments (noise, pattern, removed-object, brightness) in Appendix D/E. These are genuine strengths. However, the location-over-semantics conclusion is currently supported mainly by qualitative heatmaps and correlational evidence that does not yet rule out confounds from inpainting fidelity and bounding-box alignment.

major comments (3)
  1. [§5 and App. C] The random-position experiment, which underpins the location-over-semantics claim, does not report any quality control for the inpainted images. For the fixed-location datasets the authors state that failed inpaintings were manually filtered (§3), and Appendix B reports a reduced IoU threshold for small single-concept objects. In contrast, the 2,000 random-position images per scene in §5 appear to receive no analogous filtering or per-location fidelity verification. If the diffusion model produces systematically worse, smaller, or less recognizable objects in certain image regions (e.g., near borders or on textured road), the resulting per-pixel recall heatmaps will confound generation artifacts with genuine detector blind spots. The manuscript should either provide per-location generation quality checks (e.g., human or automatic evaluation of whether each inpainted region contains a coherent object) or explicitly argue why such artifacts cannot explain the observed FN clusters.
  2. [§4 and §5] The TP definition uses IoU > 0.5 against the ground-truth bounding box (§4), but in the random-position experiment the ground truth is the fixed 100×130 px bbox, not the actual extent of the generated object. Appendix B states that single-concept inpaintings often produced smaller objects and required lowering the IoU threshold to 0.1; the random-position protocol does not mention applying this correction. A coherent object that occupies only part of the fixed bbox will therefore be counted as a false negative, and because object scale and placement vary with road location, this misalignment is location-dependent. The authors should report whether the 0.1 threshold was used in §5, or otherwise validate that the fixed bbox accurately covers the inpainted object across all sampled locations.
  3. [Table 2 and §5] The quantitative support for 'location is more influential than semantics' is currently limited. Table 2 reports Pearson correlations of per-scene FN counts across the real and synthetic datasets, but without confidence intervals or hypothesis tests, and the heatmaps in §5 are qualitative. The high correlation across datasets that share the same scene and object location but differ in semantic content is suggestive, but it does not separate the effect of location from other scene-level factors or from generation fidelity. A formal analysis—for example, a mixed-effects model with scene, location, and object semantics as factors, or a permutation test comparing FN maps across semantic conditions—would be needed to support the claim that semantics is less relevant. Additionally, the paper does not report any quantitative measure of semantic diversity or per-semantic-class detection rates, so the 'semantics is less relevant' part of the claim is not directly tested.
minor comments (5)
  1. [Abstract / §4] The dataset size is reported inconsistently: the text in §4 states 3,580 images for LostAndFound w/ hybrid-concept inpainting, while Table 1 reports 3,539 images.
  2. [§5] The text says 'In section 5 we show two examples...' from within Section 5; this likely refers to a figure in the same section but the reference is unclear.
  3. [Fig. 4 caption] The caption states the prompt used is 'object on the street' while the body text says the fixed object prompt 'robot' was used; please clarify which prompt was used for the detections shown.
  4. [Appendix B] The sentence about reducing the IoU threshold to 0.1 is important for reproducibility and should also be stated in the main text where TP/FN counts are first defined, not only in the appendix.
  5. [Appendix D] The additional experiments on noise ovals, patterns, and removed objects are on very small image sets (16–179 images) and are reported without confidence intervals; a brief caveat would help readers gauge their evidential weight.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the evaluation is empirically self-contained, and the location-vs-semantics claim rests on randomized inpainting and external detector outputs rather than on fitted parameters, definitional identities, or load-bearing self-citations.

full rationale

This is an empirical evaluation without a mathematical derivation chain, so the circularity patterns of definitional equivalence or fitted-input-called-prediction do not apply. The central claims are that synthetic inpainted content can induce false negatives and that object location matters more than semantic class. Both claims are supported by experiments that vary prompts, objects, and locations while measuring standard detection metrics (AUPRC, AP, AR, TP/FP/FN) against real benchmark datasets. In the random-location experiment, the paper independently repeats the same semantic object ('robot') across random road positions and still observes clustered FN regions, which directly supports the location-dominance claim without making the outcome true by construction. The one author self-citation, reference [8], is used only as contextual motivation for the general competence of open-vocabulary models on OOD data and is not load-bearing for any derived result. The IoU-threshold change in Appendix B and the fixed 100x130 bbox in the location experiment are measurement or generation-fidelity concerns, not circular reductions: they could confound the interpretation of where false negatives occur, but they do not make the paper's conclusions equivalent to its inputs. No parameter is fitted and then relabeled as a prediction, and no known result is merely renamed. The paper is self-contained against external benchmarks, and its claims are externally falsifiable, so the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central finding depends on several design choices and assumptions. The main free parameters are experimental settings (bbox size, score thresholds, crop sizes) chosen by hand but not fitted to the result. The axioms are domain assumptions about the validity of synthetic proxies and ground truth alignment. The manual filtering of failed inpaintings is a post-hoc step that could introduce selection bias.

free parameters (4)
  • object bbox size for random placement = 100 x 130 pixels
    Chosen by hand for inpainting; may not match actual generated object size, affecting ground truth alignment.
  • detection score threshold for TP/FP counting = 0.1
    Standard but arbitrary; affects reported TP/FP/FN numbers.
  • pixel-wise recall score threshold = 0.2
    Chosen for heatmaps; not justified against alternative thresholds.
  • inpainting crop sizes = 512x512, 256x256, 128x128
    Selected after experiments; could influence object quality by location.
assumptions (3)
  • domain assumption Synthetic inpainted objects are a valid proxy for real-world OOD objects for evaluating detector generalization.
    The paper's entire method rests on this; they acknowledge photorealism limits but argue diversity matters.
  • domain assumption The ground truth bounding box for each inpainted object accurately represents the generated object's location and extent.
    Used for IoU and pixel-wise recall; if inpainting doesn't align, metrics misattribute failures.
  • ad hoc to paper Manual filtering of failed inpaintings does not introduce systematic bias across image locations.
    They discarded images where inpainting failed; if failures correlate with location, this biases the blind-spot analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can We Challenge Open-Vocabulary Object Detectors with Generated Content in Street Scenes?." pith.science (2026). https://pith.science/paper/4YJYEZUL

@misc{pith2026250623751,
  author       = {Pith},
  title        = {Pith review of: Can We Challenge Open-Vocabulary Object Detectors with Generated Content in Street Scenes?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4YJYEZUL}},
  note         = {Machine review of arXiv:2506.23751}
}
read the original abstract

Open-vocabulary object detectors such as Grounding DINO are trained on vast and diverse data, achieving remarkable performance on challenging datasets. Due to that, it is unclear where to find their limitations, which is of major concern when using in safety-critical applications. Real-world data does not provide sufficient control, required for a rigorous evaluation of model generalization. In contrast, synthetically generated data allows to systematically explore the boundaries of model competence/generalization. In this work, we address two research questions: 1) Can we challenge open-vocabulary object detectors with generated image content? 2) Can we find systematic failure modes of those models? To address these questions, we design two automated pipelines using stable diffusion to inpaint unusual objects with high diversity in semantics, by sampling multiple substantives from WordNet and ChatGPT. On the synthetically generated data, we evaluate and compare multiple open-vocabulary object detectors as well as a classical object detector. The synthetic data is derived from two real-world datasets, namely LostAndFound, a challenging out-of-distribution (OOD) detection benchmark, and the NuImages dataset. Our results indicate that inpainting can challenge open-vocabulary object detectors in terms of overlooking objects. Additionally, we find a strong dependence of open-vocabulary models on object location, rather than on object semantics. This provides a systematic approach to challenge open-vocabulary models and gives valuable insights on how data could be acquired to effectively improve these models.

Figures

Figures reproduced from arXiv: 2506.23751 by the authors.

Figure 1
Figure 1. Range of detection capability of Grounding DINO on LostAndFound. All panels are prompted with “object on the street” and detections are visualized in yellow. From left to right: 1) detected real object; 2) overlooked real object; 3) detected synthetic object; 4) overlooked synthetic object where, however, small leaves by side are detected. Therein, for the first procedure we sample multiple substantives from WordNet… view at source ↗
Figure 2
Figure 2. The left image shows the Single-concept inpainting on NuImages and the right one shows the Hybrid-concept inpainting. The blue bounding box indicates the inpainted object. 4 Experimental Setup In the following, we briefly introduce our experimental setup to challenge open-vocabulary object detectors with generated content in street scenes. This includes a description of the prompts provided to the detection models, … view at source ↗
Figure 3
Figure 3. Heatmaps of pixel-wise recall (first and second row) for Grounding DINO tested on inpainted objects at random locations in the street for scenes with (a) a low number of FN and (b) a high number of FN in our fixed location experiments in table 1. Heatmaps in the second and fourth row depict the number of FN corresponding to the above recall heatmap. overlooked at the original location, see section 5, show high pixel… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Heatmaps of the prediction performance of Grounding DINO given the prompt “object on the street” when detecting a “robot”, synthetically generated and inpainted, at random locations on the road of an image of NuImages. 6 Conclusion In this work, we investigated whether…
Figure 5
Figure 5. Figure 5: Based on the purple region of interest from the dataset ground truth, we created two sets of pixels (a) road pixels and (b) road border pixels, visualized in green. These are used to draw random centers for bboxes. To maintain our inpainting pipeline, we ensured that p…
Figure 6
Figure 6. Figure 6: Pixel-wise recall heatmaps for the same scene with varying models used for detection. The open vocabulary models Grounding DINO and YOLO-World were tested based on the prompt “object on the street”. F Additional visualizations We present random examples of inpainted ob…
Figure 7
Figure 7. Figure 7: Grounding DINO [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: YOLO-World [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 10
Figure 10. Figure 10: Prediction quality of Grounding DINO with varying prompt. Heatmaps show pixel-wise recall as well as the number of FN for identic scenes from LostAndFound with different detection prompts. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Examples of different types of synthetic content added to street scenes. Ground truth bounding boxes are shown in red and predictions in yellow, annotated with the confidence score. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Examples for TP predictions using Grounding DINO with prompt “object on the street” and a confidence score of 0.1. Red bboxes annotated with gt denoting the ground truth of the original object used for inpainting. Yellow bboxes show model predictions with the respecti…
Figure 13
Figure 13. Figure 13: Examples for FN predictions using Grounding DINO with prompt “object on the street” and a confidence score of 0.1. Red bboxes annotated with gt denoting the ground truth of the original object used for inpainting. Yellow bboxes show model predictions with the respecti…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 43 canonical work pages

  1. [1]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Liu, S.et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499(2023)

    arXiv 2023

  2. [2]

    Cheng, T. et al. Yolo-world: Real-time open-vocabulary object detection. arXiv preprint arXiv:2401.17270 (2024)

    arXiv 2024

  3. [3]

    InProceedings of the IEEE/CVF International Conference on Computer Vision, 1780–1790 (2021)

    Kamath, A.et al.Mdetr-modulated detection for end-to-end multi-modal understanding. InProceedings of the IEEE/CVF International Conference on Computer Vision, 1780–1790 (2021). 8

    work page 2021

  4. [4]

    Glipv2: Unifying localization and vision-language understanding.Advances in Neural Information Processing Systems35, 36067–36080 (2022)

    Zhang, H.et al. Glipv2: Unifying localization and vision-language understanding.Advances in Neural Information Processing Systems35, 36067–36080 (2022)

    work page 2022

  5. [5]

    InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16793–16803 (2022)

    Zhong, Y.et al.Regionclip: Region-based language-image pretraining. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16793–16803 (2022)

    work page 2022

  6. [6]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6339–6350 (2023)

    Mao, X.et al.Coco-o: Abenchmarkforobjectdetectorsundernaturaldistributionshifts. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 6339–6350 (2023)

    work page 2023

  7. [7]

    A simple framework for open-vocabulary segmentation and detection

    Zhang, H.et al. A simple framework for open-vocabulary segmentation and detection. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 1020–1031 (2023)

    work page 2023

  8. [8]

    & Rottmann, M

    Ilyas, S., Freeman, I. & Rottmann, M. On the potential of open-vocabulary models for object detection in unusual street scenes.arXiv preprint arXiv:2408.11221(2024)

    arXiv 2024

Show all 59 references
  1. [9]

    & Sünderhauf, N

    Miller, D., Nicholson, L., Dayoub, F. & Sünderhauf, N. Dropout sampling for robust object detection in open-set conditions. In2018 IEEE International Conference on Robotics and Automation (ICRA), 3243–3249 (IEEE, 2018)

    2018

  2. [10]

    & Sünderhauf, N

    Miller, D., Dayoub, F., Milford, M. & Sünderhauf, N. Evaluating merging strategies for sampling- based uncertainty techniques in object detection. In2019 International Conference on Robotics and Automation (ICRA), 2348–2354 (2019)

    2019

  3. [11]

    & Rottmann, M

    Schubert, M., Kahl, K. & Rottmann, M. Metadetect: Uncertainty quantification and prediction quality estimates for object detection. In2021 International Joint Conference on Neural Networks (IJCNN), 1–10 (IEEE, 2021)

    2021

  4. [12]

    & Gottschalk, H

    Riedlinger, T., Rottmann, M., Schubert, M. & Gottschalk, H. Gradient-based quantification of epis- temic uncertainty for deep object detectors. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 3921–3931 (2023)

    2023

  5. [13]

    Du, X., Wang, Z., Cai, M. & Li, Y. Vos: Learning what you don’t know by virtual outlier synthesis. arXiv preprint arXiv:2202.01197(2022)

    2022 arXiv

  6. [14]

    & Sünderhauf, N

    Wilson, S., Fischer, T., Dayoub, F., Miller, D. & Sünderhauf, N. Safe: Sensitivity-aware features for out-of-distribution object detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, 23565–23576 (2023)

    2023

  7. [15]

    Saharia, C. et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems35, 36479–36494 (2022)

    2022

  8. [16]

    & Chen, M

    Ramesh, A., Dhariwal, P., Nichol, A., Chu, C. & Chen, M. Hierarchical text-conditional image genera- tion with clip latents.arXiv preprint arXiv:2204.061251, 3 (2022)

    2022 arXiv

  9. [17]

    & Nguyen, K

    Nguyen, Q., Vu, T., Tran, A. & Nguyen, K. Dataset diffusion: Diffusion-based synthetic data generation for pixel-level semantic segmentation.Advances in Neural Information Processing Systems36 (2024)

    2024

  10. [18]

    Miller, G. A. Wordnet: a lexical database for english.Communications of the ACM38, 39–41 (1995)

    1995

  11. [19]

    & Lee, K

    Zhao, T., Liu, P., He, X., Zhang, L. & Lee, K. Real-time transformer-based open-vocabulary detection with efficient fusion head.arXiv preprint arXiv:2403.06892(2024)

    2024 arXiv

  12. [20]

    & Sun, J

    Ren, S., He, K., Girshick, R. & Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks (2016). URLhttps://arxiv.org/abs/1506.01497. 1506.01497

    2016 arXiv

  13. [21]

    Lost and found: detecting small road hazards for self-driving vehicles

    Pinggera, P.et al. Lost and found: detecting small road hazards for self-driving vehicles. in 2016 ieee. In RSJ International Conference on Intelligent Robots and Systems (IROS), 1099–1106

    2016

  14. [22]

    Caesar, H. et al. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11621–11631 (2020). 9

    2020

  15. [23]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models

    Nichol, A.et al. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741(2021)

    2021 arXiv

  16. [24]

    & Ommer, B

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684–10695 (2022)

    2022

  17. [25]

    Betker, J. et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf2, 8 (2023)

    2023

  18. [26]

    & Zhang, K

    Xie, S., Zhang, Z., Lin, Z., Hinz, T. & Zhang, K. Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 22428–22437 (2023)

    2023

  19. [27]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Lugmayr, A.et al. Repaint: Inpainting using denoising diffusion probabilistic models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11461–11471 (2022)

    2022

  20. [28]

    & Martinez, A

    Corneanu, C., Gadde, R. & Martinez, A. M. Latentpaint: Image inpainting in latent space with diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 4334–4343 (2024)

    2024

  21. [29]

    & Yang, W

    Ye, H., Zhang, J., Liu, S., Han, X. & Yang, W. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models (2023)

    2023

  22. [30]

    Stable Diffusion Web UI (2022)

    AUTOMATIC1111. Stable Diffusion Web UI (2022). URL https://github.com/AUTOMATIC1111/ stable-diffusion-webui

    2022

  23. [31]

    Imagenhub: Standardizing the evaluation of conditional image generation models.arXiv preprint arXiv:2310.01596 (2023)

    Ku, M.et al. Imagenhub: Standardizing the evaluation of conditional image generation models.arXiv preprint arXiv:2310.01596 (2023)

    2023 arXiv

  24. [32]

    & Arcucci, R

    Liu, C., Shah, A., Bai, W. & Arcucci, R. Utilizing synthetic data for medical vision-language pre- training: Bypassing the need for real images.arXiv preprint arXiv:2310.07027(2023)

    2023 arXiv

  25. [33]

    & Tang, Y

    Gui, S., Song, S., Qin, R. & Tang, Y. Remote sensing object detection in the deep learning era—a review. Remote Sensing16, 327 (2024)

    2024

  26. [34]

    E.et al.How much real data do we actually need: Analyzing object detection performance using synthetic and real data (2019)

    Nowruzi, F. E.et al.How much real data do we actually need: Analyzing object detection performance using synthetic and real data (2019). URLhttps://arxiv.org/abs/1907.07061. 1907.07061

    2019 arXiv

  27. [35]

    & Zhao, R

    Lin, S., Wang, K., Zeng, X. & Zhao, R. Explore the power of synthetic data on few-shot object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 638–647 (2023)

    2023

  28. [36]

    Arents, J. et al. Synthetic data of randomly piled, similar objects for deep learning-based object detection. vol. 13232, 706–717 (Springer, 2022)

    2022

  29. [37]

    Liu, J., Wen, X., Zhao, S., Chen, Y. & Qi, X. Can ood object detectors learn from foundation models? arXiv preprint arXiv:2409.05162(2024)

    2024 arXiv

  30. [38]

    & Cadena, C

    Blum, H., Sarlin, P.-E., Nieto, J., Siegwart, R. & Cadena, C. The fishyscapes benchmark: Measuring blind spots in semantic segmentation.International Journal of Computer Vision129, 3119–3135 (2021)

    2021

  31. [39]

    & Cord, M

    Loiseau, T., Vu, T.-H., Chen, M., Pérez, P. & Cord, M. Reliability in semantic segmentation: Can we use synthetic data? InEuropean Conference on Computer Vision, 442–459 (Springer, 2024)

    2024

  32. [40]

    In 2021 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops), 182–189 (IEEE, 2021)

    Rosenzweig, J.et al.Validation of simulation-based testing: Bypassing domain shift with label-to-image synthesis. In 2021 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops), 182–189 (IEEE, 2021)

    2021

  33. [41]

    Li, L. H.et al. Grounded language-image pre-training. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10965–10975 (2022). 10

    2022

  34. [42]

    Minderer, M. et al. Simple open-vocabulary object detection. In European conference on computer vision, 728–755 (Springer, 2022)

    2022

  35. [43]

    InComputer vision–ECCV 2014: 13th Eu- ropean conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, 740–755 (Springer, 2014)

    Lin, T.-Y.et al.Microsoft coco: Common objects in context. InComputer vision–ECCV 2014: 13th Eu- ropean conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, 740–755 (Springer, 2014)

    2014

  36. [44]

    & Girshick, R

    Gupta, A., Dollar, P. & Girshick, R. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5356–5364 (2019)

    2019

  37. [45]

    Shao, S. et al. Objects365: A large-scale, high-quality dataset for object detection. InProceedings of the IEEE/CVF international conference on computer vision, 8430–8439 (2019)

    2019

  38. [46]

    & Falchi, F

    Bianchi, L., Carrara, F., Messina, N., Gennaro, C. & Falchi, F. The devil is in the fine-grained de- tails: Evaluating open-vocabulary object detectors for fine-grained understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 22520–22...

    2024

  39. [47]

    Dreamshaper (revision 8c1bfc6) (2023)

    -. Dreamshaper (revision 8c1bfc6) (2023). URLhttps://huggingface.co/Lykon/DreamShaper

    2023

  40. [48]

    Stable Diffusion Inpainting

    RunwayML. Stable Diffusion Inpainting. https://huggingface.co/runwayml/ stable-diffusion-inpainting (2023). Accessed: 2025-05-16

    2023

  41. [49]

    Zhang, H. et al. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605(2022)

    2022 arXiv

  42. [50]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Liu, Z.et al. Swin transformer: Hierarchical vision transformer using shifted windows. InProceedings of the IEEE/CVF international conference on computer vision, 10012–10022 (2021)

    2021

  43. [51]

    & Toutanova, K

    Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805(2018)

    2018 arXiv

  44. [52]

    You only look once: Unified, real-time object detection

    Redmon, J. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition(2016)

    2016

  45. [53]

    & Qiu, J

    Jocher, G., Chaurasia, A. & Qiu, J. Ultralytics YOLO (2023). URL https://github.com/ ultralytics/ultralytics

    2023

  46. [54]

    Learning transferable visual models from natural language supervision

    Radford, A.et al. Learning transferable visual models from natural language supervision. InInterna- tional conference on machine learning, 8748–8763 (PMLR, 2021)

    2021

  47. [55]

    End-to-end object detection with transformers

    Carion, N.et al. End-to-end object detection with transformers. InEuropean conference on computer vision, 213–229 (Springer, 2020)

    2020

  48. [56]

    arXiv preprint arXiv:1907.11692 (2019)

    Liu, Y.et al.Roberta: Arobustlyoptimizedbertpretrainingapproach. arXiv preprint arXiv:1907.11692 (2019)

    2019 arXiv

  49. [57]

    Lu, C. et al. Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models.arXiv preprint arXiv:2211.01095 (2022)

    2022 arXiv

  50. [58]

    & Ermon, S

    Song, J., Meng, C. & Ermon, S. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502 (2020). 11 In the appendix, we provide technical details as well as additional visualizations for the discussions in the main paper. A Technical and implementation details A.1 Mo...

    2020 arXiv

  51. [59]

    object in the street

    by incorporating open-vocabulary detection through vision-language modeling. The open-set capabili- ties are based on a new re-parameterizable Vision-Language Path Aggregation Network (RepVL-PAN) and region-text contrastive loss to learn interaction between region-text pairs a...

    1980

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.