Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

AnySynth: Harnessing the Power of Image Synthetic Data Generation for Generalized Vision-Language Tasks

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AnySynth claims that a single layout-image-annotation pipeline can synthesize training data for many vision-language tasks at once.

desk verdict A clean integrated synthetic-data pipeline with honest ablations, but the headline few-shot gains are partly explained by the verifier already knowing the target classes. read the letter →

arxiv 2411.16749 v2 pith:GDUAFJWV submitted 2024-11-24 cs.CV

You Li , Fan Ma , Yi Yang This is my paper · ORCID
classification cs.CV
keywords syntheticdatagenerationdiffusionmodelslayout-to-imagefew-shotobjectdetectionzero-shotcomposedimageretrievalcross-domainmultimodalgroundingaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that one general-purpose pipeline can replace the task-specific data generators that vision models currently rely on. AnySynth produces synthetic training data in three stages: a large language model reasons about the scene and places object boxes, a controllable diffusion model renders the image, and task-oriented tools add the required labels. The authors report consistent gains over baselines in few-shot object detection on VOC and COCO, zero-shot composed image retrieval on CIRCO and CIRR, cross-domain detection on several Roboflow domains, and region captioning and referring expression segmentation with multimodal models. If the claim is right, a single system can serve many tasks by swapping inputs and annotation formats rather than redesigning the synthesis loop.

What carries the argument

The mechanism is the three-stage pipeline itself. Layout generation combines large-language-model reasoning with empirical width and aspect-ratio priors from real datasets so that boxes are plausible and minimally overlapping. Image generation uses an instance-wise controllable diffusion generator, extended with reference-image embedding and style injection, and then filters candidates by detector confidence and visual quality scores. Annotation generation relies on open-set detectors to post-refine boxes and on segmentation, captioning, relation, and VQA tools to produce labels. The filtering and post-refinement steps are what keep synthetic annotations usable: they discard images where the generator failed and correct boxes where it roughly succeeded.

What would settle it

Measure, per category, the fraction of AnySynth candidate images that survive the verification step, and train a few-shot detector on the surviving images for a category with a high discard rate. If a large share of common categories is discarded, or if detection accuracy on the surviving data is no better than training without synthetic images, the layout-image-annotation loop is not producing learnable signal for that category.

Watch

Extended reading notes

Core claim

The authors propose that the differences between synthetic-data tasks can be captured by three knobs — image layout, content, and annotation format — and that a fixed Layout-Image-Annotation structure can handle them all. Given object categories, layout rules, initial boxes, reference images, and style images, AnySynth first asks a large language model to infer a scene and instance layout, then adjusts boxes using size and aspect-ratio statistics from a reference dataset. A multi-conditioned generator renders the layout, with reference-image content and style-image domain injected during diffusion, and a verification step keeps only images whose instances are detected in the right places. Finally, task-oriented annotation tools refine boxes, add masks, captions, relationships, and question-answer pairs. The paper's evidence is a set of benchmark experiments in which models trained on AnySynth-generated data outperform their baselines across the four task families.

Load-bearing premise

The pipeline assumes the image generator actually draws the objects at the positions and with the attributes the language model specified, well enough that category-recognition detectors can verify them and the corrected boxes match the pixels.

Editorial extensions

If this is right

  • Few-shot detectors trained with AnySynth images improve novel-class accuracy on VOC and COCO across multiple shot settings.
  • Zero-shot composed image retrieval improves when synthetic images are added as retrieval proxies, raising mAP on CIRCO and recall on CIRR.
  • Cross-domain detection improves when style injection aligns synthetic images with the target domain, including art and underwater domains.
  • Multimodal large models gain in region captioning and referring expression segmentation when trained with AnySynth region captions and masks.
  • Ablations show that LLM-adjusted layouts, detector-based filtering, and post-refined boxes each contribute to the gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The verification step implies that AnySynth can only produce data for objects that open-set detectors already recognize; categories those detectors miss will be filtered out, so the framework may silently narrow the label space it can augment.
  • Because the pipeline is modular, swapping the base generator or adding new annotation tools could extend AnySynth to tasks like keypoint detection or depth estimation, but the paper does not test those extensions.
  • The authors' stated limitation that difficult domains still require fine-tuning suggests that the ceiling on gains is set by generator fidelity, not by layout or annotation quality; one testable extension is to measure how per-category gains scale with generator and detector accuracy.
  • In zero-shot retrieval, synthetic images act as a learned proxy between query and target; this suggests a broader use: AnySynth could generate negative or hard examples to sharpen retrieval models, not just positive proxies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces AnySynth, a three-stage pipeline for synthetic training-data generation: a Task-Specific Layout Generation Module that combines an LLM with dataset statistics to produce layouts, a Uni-Controlled Image Generation Module built on MIGC/MIGC++ with style and reference-image injection, and a Task-Oriented Annotation Module that uses detectors, SAM, and captioning tools to label the generated images. The framework is evaluated on few-shot object detection (VOC and COCO), few-shot image classification, zero-shot composed image retrieval (CIRCO and CIRR), cross-domain detection, and multimodal perception/grounding (MME, region captioning, referring expression segmentation). The authors report consistent improvements over baselines and provide ablations of layout source, filtering, post-refinement, generator choice, and domain alignment. The central claim is that one unified framework can replace task-specific synthetic-data pipelines across diverse tasks.

Significance. If the findings hold, AnySynth would be a useful step toward a single synthetic-data engine for multiple vision-language tasks, reducing the manual redesign currently needed for each downstream task. The paper's breadth of evaluation is a strength, and the ablations (Table 5, Table 8) isolate several design choices, including layout quality, filtering, and style alignment. However, the significance is conditional: the headline few-shot detection results are confounded by the verifier's target-class competence, and the ZSCIR gains depend on an undisclosed combination weight. The paper does not release code or data, which further limits immediate reproducibility. The authors do explicitly acknowledge generator limitations in Appendix D, but they do not address the verifier-dependency issue, which is load-bearing for the claimed generality.

major comments (3)
  1. [§3.3, Eq. (8); §3.4; Appendix A.2] The filtering and post-refinement stages rely on GLIP and Grounding-DINO, both pretrained on COCO and other detection corpora, to decide whether a generated layout instance is acceptable (IoU > 0.5) and to output the training boxes. In the VOC and COCO few-shot detection experiments, the evaluated 'novel' classes are exactly within the verifier's vocabulary. The observed gains (e.g., COCO 10-shot AP from 15.8 to 18.8 in Table 2) are therefore consistent with distillation of the verifier's target-class knowledge into the student detector, independent of the proposed layout-generation and style-injection modules. This does not invalidate the framework, but it means the experiments do not support the generality claim made in Section 1. I request a control experiment in which the verifier has no access to the target classes (e.g., novel classes outside the GLIP/G-DINO vocabulary, or a verifier trained without those classes) and, if possible, a baseline that filters using the base-trained downstream detector itself.
  2. [§4.2, Table 3; Appendix B (ZSCIR)] For zero-shot composed image retrieval, the final score is a weighted average of the baseline similarity and the auxiliary similarity from the generated images, but the manuscript never states the weight or whether it was tuned per dataset. Since Table 3 reports improvements of only 1.2 mAP on CIRCO and 0.8 Recall@1 on CIRR, the result is sensitive to this undisclosed hyperparameter. Please report the exact combination formula, the chosen weight, and an ablation over weights; otherwise the retrieval gains cannot be attributed to the generated data.
  3. [All experiment tables (§4.2, Appendix B)] Most reported improvements are 1–3 points, and some, such as the CIRR Recall@5 change from 55.57 to 55.74, are below 0.2 points, but no error bars, multiple seeds, or significance tests are provided. In few-shot detection with K=1, the VOC Split 1 improvement from 45.8 to 63.6 is large, yet the 1- and 2-shot settings are exactly where variance is highest. The paper's central claim that synthetic data 'significantly improves' performance needs at least three seeds with mean and standard deviation for the headline few-shot experiments, or a clear statement of which improvements are outside noise.
minor comments (6)
  1. [Table 5, §4.3] The table header appears garbled: the row labels 'Layout' and 'Filter' are followed by duplicate AP-column groups, and the text refers to 'llm†' as 'without post-refine' while the row label says 'w/o MIGC'. Please clarify the columns and align the notation with the narrative.
  2. [§3.2, Eq. (5)] The adjustment function f is used before being defined; Eq. (5) writes f(b,E) but the text only loosely describes 'the adjustment algorithm'. Please provide the explicit formula or pseudocode for the size and position updates.
  3. [§4.2, text and captions] There are several typos: 'METOR' should be 'METEOR', 'position-bassed' should be 'position-based', and 'LLA V A' appears with a stray space. Please proofread the final text.
  4. [§3.3, Eq. (7)] The style-fusion weight \lambda is introduced as a constant in Eq. (7), but Appendix A.2 later uses a time-dependent schedule (0.7 for early steps, 0.3 for later). Define \lambda(t) in the main text or state explicitly that Eq. (7) holds per time step with a schedule.
  5. [§4.1; Appendix B] The number of synthetic images per query for ZSCIR is not stated in the main text (5 in the appendix), and the final similarity combination is described only as a 'weighted average'. Move these implementation details to the main text for reproducibility.
  6. [Throughout] No code, model weights, or hyperparameter configuration files are released; given the many interacting components (LLM prompts, layout statistics, filtering thresholds, style schedules), releasing the pipeline would substantially improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AnySynth is an empirical system paper whose gains are measured on external benchmarks; its self-citations (MIGC/MIGC++) are components, not derived evidence.

full rationale

The paper makes no prediction that reduces to a fitted value. The central derivation chain is the Layout-Image-Annotation pipeline (Eqs. 1-3), and each claimed improvement is measured on external benchmarks (VOC, COCO, CIRCO, CIRR, MME, Roboflow, etc.), so the framework's value is not established by a self-referential equation. The quality/position filter in Eq. 8 selects among candidate images using detector confidence and image-quality scores; this is a data-selection step, not a quantity that is later reported as a prediction. The use of GLIP and Grounding-DINO for verification and post-refinement (Sec. 3.3-3.4) does mean the synthetic boxes are teacher-derived, and Appendix D concedes that challenging categories still require target-domain fine-tuning; this is a legitimate generalization and distillation concern, but it is not a circular reduction because the downstream gains are still evaluated against external test sets and the verifier is a fixed external model rather than a parameter fitted to those test results. The authors' own MIGC/MIGC++ and related works are cited as building blocks, but the paper does not invoke a uniqueness theorem or import its central claim from those citations; the empirical ablations (Tab. 5) independently compare generators and filtering choices. Therefore, no step in the derivation reduces to its own input, and the paper is best characterized as an empirical system contribution with external validation rather than a circular argument.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical and relies on several hand-chosen hyperparameters and on the reliability of external models. The free parameters listed above can materially change the reported gains, and the retrieval weight is not disclosed.

free parameters (5)
  • Style fusion lambda = 0.7 (early steps), 0.3 (late steps)
    Hand-chosen weighting for style information injection (Appendix A.2); affects domain alignment and could be tuned per task.
  • Layout empirical weight = 0.1 to 0.2
    Weighted average between empirical size and LLM initial size in layout adjustment (Appendix A.1).
  • Candidate image count K = 4
    Number of candidate images generated in filtering (Section 3.3).
  • IoU match threshold = 0.5
    Threshold for matching detected instances to layout boxes in filtering (Appendix A.2).
  • Retrieval similarity combination weight = unspecified
    The final retrieval score is a weighted average of baseline similarity and synthetic-image similarity (Appendix B). The weight is not reported, leaving open the possibility of per-dataset tuning.
assumptions (4)
  • domain assumption COCO layout statistics can serve as a reference distribution for adjusting layouts in other domains.
    Used in Layout Adjustment (Section 3.2, Eq. 5) to fix sizes and positions for tasks like cross-domain detection and retrieval; if the target domain's object geometry differs from COCO, the adjustment may degrade image quality.
  • domain assumption Object detection confidence scores (from GLIP and Grounding-DINO) correlate with how useful an image will be for training a downstream model.
    Used in Quality and Position based Filtering (Section 3.3) to select candidate images; there is no validation in the paper that detector confidence predicts training utility.
  • domain assumption The large language model (Qwen1.5-32B) can reliably reason about scenes, relative positions, and object sizes from a short list of categories and rules.
    The entire layout generation module (Section 3.2) depends on LLM reasoning; no quantitative evaluation of layout quality from the LLM is provided.
  • domain assumption External models (MIGC, MIGC++, ELITE, StyleID, SAM, etc.) behave as described in their respective papers.
    The framework builds on these as fixed components; if any of these models underperforms, the synthetic data quality suffers, as the authors note for challenging domains (Appendix D).

how reviews work

0 comments
Cite this review

Pith. "Pith review of AnySynth: Harnessing the Power of Image Synthetic Data Generation for Generalized Vision-Language Tasks." pith.science (2026). https://pith.science/paper/GDUAFJWV

@misc{pith2026241116749,
  author       = {Pith},
  title        = {Pith review of: AnySynth: Harnessing the Power of Image Synthetic Data Generation for Generalized Vision-Language Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GDUAFJWV}},
  note         = {Machine review of arXiv:2411.16749}
}
read the original abstract

Diffusion models have recently been employed to generate high-quality images, reducing the need for manual data collection and improving model generalization in tasks such as object detection, instance segmentation, and image perception. However, the synthetic framework is usually designed with meticulous human effort for each task due to various requirements on image layout, content, and annotation formats, restricting the application of synthetic data on more general scenarios. In this paper, we propose AnySynth, a unified framework integrating adaptable, comprehensive, and highly controllable components capable of generating an arbitrary type of synthetic data given diverse requirements. Specifically, the Task-Specific Layout Generation Module is first introduced to produce reasonable layouts for different tasks by leveraging the generation ability of large language models and layout priors of real-world images. A Uni-Controlled Image Generation Module is then developed to create high-quality synthetic images that are controllable and based on the generated layouts. In addition, user specific reference images, and style images can be incorporated into the generation to task requirements. Finally, the Task-Oriented Annotation Module offers precise and detailed annotations for the generated images across different tasks. We have validated our framework's performance across various tasks, including Few-shot Object Detection, Cross-domain Object Detection, Zero-shot Composed Image Retrieval, and Multi-modal Image Perception and Grounding. The specific data synthesized by our framework significantly improves model performance in these tasks, demonstrating the generality and effectiveness of our framework.

Figures

Figures reproduced from arXiv: 2411.16749 by the authors.

Figure 1
Figure 1. The Differences of AnySynth from Other Frame￾works. In (a), we showcase several typical tasks with diverse re￾quirements, in (b), we show the common Synthetic data collection frameworks, which need specific desgin on different tasks. In (c), we show our AnySynth Framework, handle diverse tasks in one unified framework, enhance the generality of synthetic data. multi-modal understanding [33–35, 42], largely driven by… view at source ↗
Figure 2
Figure 2. Overview of our AnySynth. Our AnySynth consists of three modules. (a) Represents the Task-Specific Layout Generation Module, which parses various layout parameters using LLMs and combines them with dataset statistics to derive object layouts and basic scenes. (b) Represents the Uni-Controlled Image Generation Module, which achieves comprehensive high-quality image generation by integrating layout, instance reference… view at source ↗
Figure 3
Figure 3. Quantitative results in Few-Shot Image Classification. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 6
Figure 6. Figure 6: Improvement in clipart in Few-shot Cross￾Domain detection task. Confi:0.5336 Ps:0.5848 Confi:0.5308 Ps:0.4152 Confi:0.6105 Ps:0.5848 Confi:0.6297 Ps:0.6229 Confi:0.3986 Ps:0.4097 Confi:0.5330 Ps:0.5902 Case 1 Case 2 Case 3 [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results. Based on Position and quality scores, we can filter the low-quality and low-discriminability im￾ages. The Confi indicating the confidence of the detector, and Ps indicating the score of Pick-a-Score model. more detailed descriptions through the Tas…
Figure 8
Figure 8. Figure 8: We present a comparison between (a) the image gener [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 1
Figure 1. Figure 1: Overview of our system prompt, Input and output. signed for comprehensive evaluation. The training pro￾cess uses the train/val sets from VOC07 and VOC12, while the VOC07 test set is used for evaluation. Performance is measured using mAP50 under K-shot settings, where K…
Figure 2
Figure 2. Figure 2: The Image generated by our Framework in ZSCIR. of 128. The synthetic and real data are sampled at a 3:1 ratio. Relation Extraction with Synthesis. For the RES task, we use the 1,000 images constructed for RC as training data. The synthetic data is directly used for tra…
Figure 3
Figure 3. Figure 3: The Image generated by our Framework in Cross-Domain Detection. still some deficiencies in the details of the final regions in the generated images. D. Limitation and Future Work Limitation. Due to the generator’s performance limita￾tions, certain challenging categorie…
Figure 4
Figure 4. Figure 4: The Image generated by our Framework in Multi-modal Understanding. 5 [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxy

    cs.CV 2024-11 conditional novelty 6.0 of 10

    IP-CIR creates imagined proxy images from a query image and caption via LLM-based layout and conditional generation, then blends proxy, query, and text features to improve zero-shot composed image retrieval.

Reference graph

Works this paper leans on

72 extracted references · 39 canonical work pages · cited by 1 Pith paper

  1. [1]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xi- aodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Day- iheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfe...

    arXiv 2023

  2. [2]

    ediff-i: Text-to-image diffusion models with ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Qinsheng Zhang, Karsten Kreis, Miika Ait- tala, Timo Aila, Samuli Laine, Bryan Catanzaro, Tero Kar- ras, and Ming-Yu Liu. ediff-i: Text-to-image diffusion models with ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 2

    arXiv 2022

  3. [3]

    Zero-shot composed image retrieval with textual inversion, 2023

    Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, and Al- berto Del Bimbo. Zero-shot composed image retrieval with textual inversion, 2023. 5, 2

    work page 2023

  4. [4]

    Murphy, William T

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jos´e Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin P. Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative transformers. 2023. 2

    2023

  5. [5]

    Training-free layout control with cross-attention guidance

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. arXiv preprint arXiv:2304.03373, 2023. 2

    arXiv 2023

  6. [6]

    Wenhu Chen, Hexiang Hu, Chitwan Saharia, and William W. Cohen. Re-imagen: Retrieval-augmented text-to-image gen- erator, 2022. 2

    work page 2022

  7. [7]

    Auto cherry-picker: Learning from high-quality generative data driven by lan- guage, 2024

    Yicheng Chen, Xiangtai Li, Yining Li, Yanhong Zeng, Jian- zong Wu, Xiangyu Zhao, and Kai Chen. Auto cherry-picker: Learning from high-quality generative data driven by lan- guage, 2024. 2, 3

    work page 2024

  8. [8]

    Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer

    Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injec- tion in diffusion: A training-free approach for adapting large- scale diffusion models for style transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8795–8805, 2024. 4, 1

    work page 2024

Show all 72 references
  1. [9]

    Roboflow 100: A rich, multi-domain object detection benchmark, 2022

    Floriana Ciaglia, Francesco Saverio Zuppichini, Paul Guer- rie, Mark McQuade, and Jacob Solawetz. Roboflow 100: A rich, multi-domain object detection benchmark, 2022. 1, 6

    2022

  2. [10]

    Diffusion models beat gans on image synthesis, 2021

    Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis, 2021. 1, 2

    2021

  3. [11]

    Hierarchical text-conditional image generation with clip latents, 2022

    Aditya Ramesh et al. Hierarchical text-conditional image generation with clip latents, 2022. 1, 2

    2022

  4. [12]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2): 303–338, 2010. 1, 5

    2010

  5. [13]

    Mme: A compre- hensive evaluation benchmark for multimodal large language models, 2024

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A compre- hensive evaluation benchmark for multimodal large language models, 2024. 6

    2024

  6. [15]

    Cross-domain few-shot object detec- tion via enhanced open-set object detector

    Yuqian Fu, Yu Wang, Yixuan Pan, Lian Huai, Xingyu Qiu, Zeyu Shangguan, Tong Liu, Lingjie Kong, Yanwei Fu, Luc Van Gool, et al. Cross-domain few-shot object detec- tion via enhanced open-set object detector. arXiv preprint arXiv:2402.03094, 2024. 6

    2024 arXiv

  7. [16]

    Apollo home page

    gi. Apollo home page. https : / / github . com / apolloauto, 2019. 5

    2019

  8. [17]

    Classifier-free diffusion guidance

    Jonathan Ho. Classifier-free diffusion guidance. ArXiv, abs/2207.12598, 2022. 2

    2022 arXiv

  9. [18]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  10. [19]

    Egtr: Extracting graph from trans- former for scene graph generation

    Jinbae Im, JeongYeon Nam, Nokyung Park, Hyungmin Lee, and Seunghyun Park. Egtr: Extracting graph from trans- former for scene graph generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24229–24238, 2024. 5

    2024

  11. [20]

    Cross-domain weakly-supervised object de- tection through progressive domain adaptation

    Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiy- oharu Aizawa. Cross-domain weakly-supervised object de- tection through progressive domain adaptation. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5001–5009, 2018. 6

    2018

  12. [21]

    YOLOv5 by Ultralytics

    Glenn Jocher, Ayush Chaurasia, Adam Polishchuk, and Jing Qiu. YOLOv5 by Ultralytics. https://github.com/ ultralytics/yolov5, 2020. Accessed: 2024-10-12. 6

    2020

  13. [22]

    FOCUS: Familiar objects in common and uncommon settings

    Priyatham Kattakinda and Soheil Feizi. FOCUS: Familiar objects in common and uncommon settings. In Proceedings of the 39th International Conference on Machine Learning , pages 10825–10847. PMLR, 2022. 5

    2022

  14. [23]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 5

    2023 arXiv

  15. [24]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation

  16. [25]

    Elevater: A benchmark 9 and toolkit for evaluating language-augmented visual mod- els

    Chunyuan Li*, Haotian Liu*, Liunian Harold Li, Pengchuan Zhang, Jyoti Aneja, Jianwei Yang, Ping Jin, Yong Jae Lee, Houdong Hu, Zicheng Liu, et al. Elevater: A benchmark 9 and toolkit for evaluating language-augmented visual mod- els. arXiv preprint arXiv:2204.08790, 2022. 5

    2022 arXiv

  17. [26]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML,

  18. [27]

    Grounded language-image pre-training

    Liunian Harold Li*, Pengchuan Zhang*, Haotian Zhang*, Jianwei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language-image pre-training. In CVPR, 2022. 5, 1

    2022

  19. [28]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. CVPR,

  20. [29]

    Imagine and seek: Improving composed image retrieval with an imagined proxy

    You Li, Fan Ma, and Yi Yang. Imagine and seek: Improving composed image retrieval with an imagined proxy. arXiv preprint arXiv:2411.16752, 2024. 1

    2024 arXiv

  21. [30]

    Caphuman: Capture your moments in parallel uni- verses

    Chao Liang, Fan Ma, Linchao Zhu, Yingying Deng, and Yi Yang. Caphuman: Capture your moments in parallel uni- verses. In CVPR, pages 6400–6409, 2024. 1

    2024

  22. [31]

    Lawrence Zitnick, and Piotr Doll ´ar

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Doll ´ar. Microsoft coco: Common objects in context, 2015. 1, 5

    2015

  23. [32]

    Di- agen: Diverse image augmentation with generative models

    Tobias Lingenberg, Markus Reuter, Gopika Sudhakaran, Do- minik Gojny, Stefan Roth, and Simone Schaub-Meyer. Di- agen: Diverse image augmentation with generative models. arXiv preprint arXiv:2408.14584, 2024. 5

    2024 arXiv

  24. [33]

    Improved baselines with visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023. 1, 5, 6

    2023

  25. [34]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 2

    2023

  26. [35]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 1

    2024

  27. [36]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 5, 1

    2023 arXiv

  28. [37]

    Image retrieval on real-life images with pre-trained vision-and-language models

    Zheyuan Liu, Cristian Rodriguez-Opazo, Damien Teney, and Stephen Gould. Image retrieval on real-life images with pre-trained vision-and-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2125–2134, 2021. 5, 2

    2021

  29. [38]

    Ovis: Structural em- bedding alignment for multimodal large language model

    Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. Ovis: Structural em- bedding alignment for multimodal large language model. arXiv:2405.20797, 2024. 5

    2024 arXiv

  30. [39]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models, 2022

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models, 2022. 2

    2022

  31. [40]

    Gpt-4 technical report, 2023

    OpenAI. Gpt-4 technical report, 2023. 3, 1

    2023

  32. [41]

    Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever

    Alec Radford, Jong Wook Kim, Chris Hallacy, A. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from nat- ural language supervision. In ICML, 2021. 1, 5

    2021

  33. [42]

    Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S

    Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdel- rahman Shaker, Salman Khan, Hisham Cholakkal, Rao M. Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S. Khan. Glamm: Pixel grounding large multimodal model. The IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 1, 6

    2024

  34. [43]

    Generative ad- versarial text to image synthesis, 2016

    Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Lo- geswaran, Bernt Schiele, and Honglak Lee. Generative ad- versarial text to image synthesis, 2016. 2

    2016

  35. [44]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 1, 2

    2021

  36. [45]

    Ronneberger, P.Fischer, and T

    O. Ronneberger, P.Fischer, and T. Brox. U-net: Convolu- tional networks for biomedical image segmentation. InMed- ical Image Computing and Computer-Assisted Intervention (MICCAI), pages 234–241. Springer, 2015. (available on arXiv:1505.04597 [cs.CV]). 2

    2015 arXiv

  37. [46]

    Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffu...

    2022

  38. [47]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. arXiv:2010.02502, 2020. 4, 1

    2010 arXiv

  39. [48]

    Gen2det: Generate to detect, 2023

    Saksham Suri, Fanyi Xiao, Animesh Sinha, Sean Chang Cu- latana, Raghuraman Krishnamoorthi, Chenchen Zhu, and Abhinav Shrivastava. Gen2det: Generate to detect, 2023. 2, 3

    2023

  40. [49]

    Effective data augmentation with diffusion models, 2023

    Brandon Trabucco, Kyle Doherty, Max Gurinas, and Ruslan Salakhutdinov. Effective data augmentation with diffusion models, 2023. 5

    2023

  41. [50]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 2

    2017

  42. [51]

    YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors

    Chien-Yao Wang, Alexey Bochkovskiy, and Hong- Yuan Mark Liao. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 6

    2023

  43. [52]

    Learning from synthetic data for crowd counting in the wild, 2019

    Qi Wang, Junyu Gao, Wei Lin, and Yuan Yuan. Learning from synthetic data for crowd counting in the wild, 2019. 2

    2019

  44. [53]

    Instancediffusion: Instance-level control for image generation, 2024

    Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, and Ishan Misra. Instancediffusion: Instance-level control for image generation, 2024. 2

    2024

  45. [54]

    Fine-grained prototypes distillation for few-shot object de- tection

    Zichen Wang, Bo Yang, Haonan Yue, and Zhenghao Ma. Fine-grained prototypes distillation for few-shot object de- tection. In Proceedings of the 38th AAAI Conference on Ar- tificial Intelligence (AAAI-24), 2024. 5, 2 10

    2024

  46. [55]

    Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation

    Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual con- cepts into textual embeddings for customized text-to-image generation. arXiv preprint arXiv:2302.13848, 2023. 4, 1

    2023 arXiv

  47. [56]

    Cashman, and Jamie Shotton

    Erroll Wood, Tadas Baltru ˇsaitis, Charlie Hewitt, Sebastian Dziadzio, Matthew Johnson, Virginia Estellers, Thomas J. Cashman, and Jamie Shotton. Fake it till you make it: Face analysis in the wild using synthetic data alone, 2021. 2

    2021

  48. [57]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  49. [58]

    Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion

    Jinheng Xie, Yuexiang Li, Yawen Huang, Haozhe Liu, Wen- tian Zhang, Yefeng Zheng, and Mike Zheng Shou. Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion. arXiv preprint arXiv:2307.10816, 2023. 2

    2023 arXiv

  50. [59]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation, 2023

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation, 2023. 5

    2023

  51. [60]

    Attngan: Fine- grained text to image generation with attentional generative adversarial networks, 2017

    Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He. Attngan: Fine- grained text to image generation with attentional generative adversarial networks, 2017. 2

    2017

  52. [61]

    Ldre: Llm-based divergent reasoning and ensemble for zero-shot composed image re- trieval

    Zhenyu Yang, Dizhan Xue, Shengsheng Qian, Weiming Dong, and Changsheng Xu. Ldre: Llm-based divergent reasoning and ensemble for zero-shot composed image re- trieval. In Proceedings of the 47th International ACM SI- GIR Conference on Research and Development in Informa- tion Re...

    2024

  53. [62]

    Glipv2: Unifying localization and vision-language understanding

    Haotian* Zhang, Pengchuan* Zhang, Xiaowei Hu, Yen- Chun Chen, Liunian Harold Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq-Neng Hwang, and Jianfeng Gao. Glipv2: Unifying localization and vision-language understanding. arXiv preprint arXiv:2206.05836, 2022. 5

    2022 arXiv

  54. [63]

    Dynrefer: Delving into region-level multi-modality tasks via dynamic resolution,

    Yuzhong Zhao, Feng Liu, Yue Liu, Mingxiang Liao, Chen Gong, Qixiang Ye, and Fang Wan. Dynrefer: Delving into region-level multi-modality tasks via dynamic resolution,

  55. [64]

    Layoutdiffusion: Controllable diffu- sion model for layout-to-image generation

    Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. Layoutdiffusion: Controllable diffu- sion model for layout-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 22490–22499, 2023. 2

    2023

  56. [65]

    Pyramid diffusion models for low-light image enhancement

    Dewei Zhou, Zongxin Yang, and Yi Yang. Pyramid diffusion models for low-light image enhancement. arXiv preprint arXiv:2305.10028, 2023. 1

    2023 arXiv

  57. [66]

    Migc: Multi-instance generation controller for text-to-image synthesis

    Dewei Zhou, You Li, Fan Ma, Zongxin Yang, and Yi Yang. Migc: Multi-instance generation controller for text-to-image synthesis. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 6818–6828, 2024. 2, 4, 1

    2024

  58. [67]

    Migc++: Advanced multi-instance generation controller for image synthesis

    Dewei Zhou, You Li, Fan Ma, Zongxin Yang, and Yi Yang. Migc++: Advanced multi-instance generation controller for image synthesis. ArXiv, abs/2407.02329, 2024. 2, 4, 1

    2024 arXiv

  59. [68]

    3dis: Depth-driven decoupled instance synthesis for text-to-image generation

    Dewei Zhou, Ji Xie, Zongxin Yang, and Yi Yang. 3dis: Depth-driven decoupled instance synthesis for text-to-image generation. ArXiv, abs/2410.12669, 2024. 1

    2024

  60. [69]

    Odgen: Domain-specific object detection data generation with diffusion models, 2024

    Jingyuan Zhu, Shiyu Li, Yuxuan Liu, Ping Huang, Jiulong Shan, Huimin Ma, and Jian Yuan. Odgen: Domain-specific object detection data generation with diffusion models, 2024. 2, 3 11 Appendix A. Framework Details A.1. Task-Specific Layout Generation Layout Initialization. To ach...

    2024

  61. [70]

    Imagine a basic scene, including whether the scene is near or far, and give a scene label, such as on the grass or in the hospital

    Conceptualize a suitable scene. Imagine a basic scene, including whether the scene is near or far, and give a scene label, such as on the grass or in the hospital. In this step, Give a description of the scene in format of [## Scene: xxx ##]

  62. [71]

    users may provide reference images and modify the content of the reference images

    Identifying the instances in the image. users may provide reference images and modify the content of the reference images. You need to analyze inputs to determine what objects will actually be present in the final image

  63. [72]

    Reasoning between instances. You need to perform appropriate reasoning within the parsed objects and the overall scene, anticipating each object's relative size, dimensions, relative position, possible attributes, and relationships

  64. [73]

    Summarize output in format: … Make sure that each layout contains only one instance

    Summarize and output. Summarize output in format: … Make sure that each layout contains only one instance. If multiple objects of the same type are mentioned, you must assign a separate layout for each instance. (b) Input: Object: Inst1:Label: a dog, Reference: text, Initial B...

    2017

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.