Pith. sign in

REVIEW 4 major objections 5 minor 23 references

DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read As text-to-image models scale up, they get better at following explicit attribute prompts but worse at producing varied attributes on their own; the same measurement finds default diversity tracks training-image attribute diversity at…

desk verdict DIMCIM's Does/Can split is a genuinely useful evaluation framework, but the headline trade-off and training correlation need error bars and a scorer-bias check before being taken at face value. read the letter →

arxiv 2506.05108 v1 pith:XREO2B7L submitted 2025-06-05 cs.CV

Revant Teotia , Candace Ross , Karen Ullrich , Sumit Chopra , Adriana Romero-Soriano , Melissa Hall , Matthew J. Muckley This is my paper
classification cs.CV
keywords text-to-imagegenerationdiversityevaluationdefault-modebiasgeneralizationcapacityVQAScorediffusionmodelstrainingdataskewCOCO-DIMCIMbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that default-mode diversity and generalization capacity are distinct properties of text-to-image models and that the two trade off as models scale. To show this, it introduces DIMCIM, a reference-free framework that measures whether a model produces varied attributes on its own (Does-It Metric) and whether it can produce a requested attribute when explicitly prompted (Can-It Metric). On the COCO-DIMCIM benchmark, smaller models LDM2.1 and Flow-Int score higher on DIM while larger models LDM3.5L and FLUX.1-dev score higher on CIM, and the paper reports the same trade-off when classifier-free guidance is increased. The framework also finds that default-mode diversity in generated images correlates at $r = 0.85$ with attribute diversity in the training images of a Flow-Int model trained on CC12M, and it pinpoints failure modes such as negation and state attributes that explicit prompting cannot overcome.

What carries the argument

The load-bearing object is the attribute-concept score $S(p_c, a)$, defined in Eq. (1) as the mean VQAScore image-attribute score for attribute $a$ across $n$ generated images minus the mean score for all other attributes in the same attribute type $A_{c,t}$. VQAScore is run on truncated text containing only the concept and attribute (e.g. 'a metal dining table') to keep background context out of the score. The Does-It Metric is then $1 - \text{mean}|S(p_{c,-}, a)|$ over coarse prompts, so a high value means balanced default attributes, while the Can-It Metric is the mean of $S(p_{c,a}, a)$ over dense prompts, measuring whether the requested attribute dominates when explicitly prompted. This same scoring recipe is applied both to model-generated images and to training images, which is what makes the $r = 0.85$ training-data comparison possible.

What would settle it

Have human annotators label attribute presence on a stratified subset of COCO-DIMCIM images, oversampling the negation and state attributes the paper flags as failures, and recompute the Does-It and Can-It metrics and the training-data correlation on that subset. If the size trade-off in Table 1 disappears or the $r = 0.85$ correlation drops toward zero, the central claim fails; if both survive human labels, it holds.

Watch

Extended reading notes

Core claim

The central claim is that default-mode diversity and generalization capacity are separable and move in opposite directions. Measured on COCO-DIMCIM, LDM2.1 (1.5B) and Flow-Int (1.9B) have higher Does-It Metric scores (0.815 and 0.802), while LDM3.5L (8.1B) and FLUX.1-dev (12B) have higher Can-It Metric scores (0.374 and 0.326), which the paper summarizes as 'as model size increases, the Does-It Metric decreases while Can-It Metric increases.' The paper also reports that increasing classifier-free guidance improves the Can-It Metric at the cost of the Does-It Metric, and that for a Flow-Int model trained on CC12M, default-mode diversity correlates at $r = 0.85$ with the attribute diversity measured in the training images. It further identifies fine-grained failures—negation, and state attributes like 'closed,' 'full,' and 'inverted'—where explicit prompting does not reliably produce the requested attribute.

Load-bearing premise

The load-bearing premise is that VQAScore with truncated 'concept attribute' text measures attribute presence in generated and training images without systematic bias; if it undercounts attributes like negation, 'closed', 'full', or 'inverted', the size trade-off and the $r = 0.85$ correlation could be artifacts of the scorer rather than properties of the models.

Editorial extensions

If this is right

  • If the trade-off holds, evaluation reports that only track prompt-following will systematically miss a decline in default diversity as models scale.
  • The $r = 0.85$ training correlation implies that attribute skew in training images is a leading driver of default-mode skew in generations, so adjusting training-data composition should be a direct lever on the Does-It Metric.
  • Attributes like 'military airplane' or 'glass table' that are under-represented by default but score high on the Can-It Metric can be recovered by explicit prompting or guidance, meaning the default skew is not a hard capability limit.
  • Attributes like negation, 'closed', 'full', and 'inverted' fail even when explicitly requested, indicating failures that cannot be fixed by prompt changes alone.
  • Because DIMCIM needs no reference image set, the same pipeline can be adapted to new concepts and attributes without collecting comparison images for each axis of diversity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A causal reading of the $r = 0.85$ correlation is a testable claim the paper does not run: rebalancing the attribute distribution of a training set should raise the Does-It Metric for previously under-represented attributes.
  • The failure attributes the paper highlights are precisely the ones a VQA scorer is likely to mis-count, so human annotation on the negation and state attribute subset would separate genuine model failure from measurement failure.
  • The current benchmark filters out people, so extending DIMCIM to human concepts would require treating attributes as socially loaded rather than neutral visual axes.
  • The same pipeline could be used as a pre-training data audit: compute training-image Does-It scores per attribute and use them to predict which attributes a new model will skew by default.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces DIMCIM, a reference-free framework for measuring two axes of diversity in text-to-image (T2I) models: default-mode diversity (Does-it Metric, DIM) and prompt-conditioned generalization (Can-it Metric, CIM). The framework uses VQAScore with truncated concept-attribute text to compute attribute-concept scores S(pc,a) (Eq. 1), then aggregates them over a new benchmark, COCO-DIMCIM, built from 30 COCO concepts, 494 attributes, 930 coarse prompts, and 14,641 dense prompts. The authors report that DIM decreases and CIM increases with model size across four models (LDM2.1, Flow-Int, LDM3.5L, FLUX.1-dev), identify interpretable failure modes (e.g., negation, closed/full states), and report a Pearson correlation of 0.85 between training-image DIM and generated-image DIM for a model trained on CC12M.

Significance. If the empirical claims hold, DIMCIM is a valuable addition to T2I evaluation: it is reference-free, provides fine-grained, interpretable failure analyses, and offers a diagnostic connecting training-data attribute skew to default-generation diversity. The public benchmark and code are contributions. However, the central empirical claims rest on a small number of models, without statistical uncertainty quantification, and on a scoring pipeline whose attribute-level reliability is not validated. The headline trade-off is also not strictly monotonic in the reported table. These issues currently limit the strength of the paper's conclusions.

major comments (4)
  1. [Section 5.1, Table 1] The stated headline trade-off is not monotonic: FLUX.1-dev (12B) has CIM=0.326, which is lower than LDM3.5L (8.1B) CIM=0.374, while DIM continues to decrease from 0.799 to 0.785. The caption's 'as model size increases, the Does-It Metric decreases while Can-It Metric increases' therefore overstates the pattern. Moreover, with only four models and no error bars, confidence intervals, or significance tests, the small DIM differences (0.815 vs 0.802 vs 0.799 vs 0.785) are not established as robust. Please report per-model variability (e.g., bootstrap across prompts and attributes) and reframe the claim to match the actual ordering.
  2. [Section 3.2, Eq. (1); Section 5.1] The framework crucially depends on VQAScore with truncated text, but the paper provides no validation that this scoring gives unbiased, comparable attribute-presence estimates across models and attribute types. The paper itself acknowledges in Section 2 that VQAScore-style metrics are susceptible to linguistic biases (Ross et al. 2024), and its own failure cases in Section 5.2.2 and Appendix B involve negation, 'closed', 'full', and 'inverted' - exactly state/negation attributes where VLMs are known to be unreliable. Because the summary DIM is 1 minus the mean absolute value of S, systematic under-detection of such attributes pushes |S| toward 0 and inflates DIM; if larger models generate higher-fidelity images that the scorer can discriminate more easily, the DIM decrease and CIM increase across model sizes could be at least partly manufactured by the scorer. The footnote reporting consistency across other VQAScore models does not address this, as those models can share the same bias. A human-labeled validation of the truncated attribute scoring on a sample of generated and training images is needed before the trade-off and correlation claims can be accepted.
  3. [Section 5.3] The r=0.85 correlation between training-data DIM and model DIM uses the same VQAScore in both paths: it is used to filter CC12M images with a concept-text threshold of 0.8, and then to compute attribute scores for both training and generated images. This shared measurement pipeline can inflate the correlation if the concept filter or the attribute detection is systematically biased. The paper should report the number of concept-attribute pairs and a confidence interval for r, show correlations after excluding the unreliable state/negation attributes, and examine sensitivity to the 0.8 threshold.
  4. [Sections 5 and 5.3; Table 2] The framework contains free parameters - n=30 images per prompt, the VQAScore concept-filter threshold of 0.8, and the classifier-free guidance scale of 7.5 - but only the CFG scale is varied (Table 2). A sensitivity analysis for n and for the concept-filter threshold is necessary to show that the reported DIM/CIM scores and the trade-off are not artifacts of these choices.
minor comments (5)
  1. [Section 3.3] The aggregation formula for the summary DIM should state whether each attribute type is given equal weight or whether all attributes are weighted equally; because attribute types have different sizes, this choice changes the final score.
  2. [Section 4] Please state the exact version of Llama3.1 used and provide the random seed or the list of the 31 seed prompts per concept to make benchmark construction reproducible.
  3. [Section 5.3] Please report the number of concept-attribute pairs entering the r=0.85 correlation and provide confidence intervals for the coefficient, since the scatter plot in Figure 6 shows one concept only.
  4. [Appendix A] The Flow-Int training details mention an 'internally licensed dataset' and describe the model as proprietary; please clarify which public components are used and whether the exact weights are available, since this limits reproducibility.
  5. [Figure 6 and Table 1] Adding error bars or attribute-level scatter to the summary results would help readers assess the variability behind the aggregate DIM and CIM values.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: DIM/CIM and the r=0.85 training correlation are measured, not fitted; self-citations are non-load-bearing.

full rationale

The paper's headline findings are empirical observations rather than quantities forced by construction. The Does-It Metric and Can-It Metric are computed by applying the attribute-concept score S(pc,a) from Eq. (1) to VQAScore outputs for coarse and dense prompts, and the model-size trade-off in Table 1 is an observed ranking across models with no parameters fitted to that trade-off. The r=0.85 correlation between default-mode diversity in generated images and attribute diversity in CC12M training images is also measured: both sides use VQAScore, and the training-image set is filtered by a VQAScore concept threshold, but no term in the correlation is constructed to force agreement, and the two sets of images are independent. The self-citations to the authors' prior diversity work (Hall et al., 2024; Hemmat et al., 2024) and to Ross et al. (2024) for VQAScore's linguistic biases motivate the problem and the truncated-scoring design, but they do not supply the equations or the conclusions; the central claims rest on the reported computations rather than on a self-citation chain. The VQAScore-bias caveat is a validity and correctness risk, not a circularity: if attribute detection is systematically inaccurate for state or negation attributes, both the trade-off and the correlation could be scorer artifacts, but that would not make the derivation equivalent to its inputs. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no known result is merely relabeled. The low score reflects only the presence of minor, non-load-bearing self-citations; no formal circular step is identified.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The framework rests on VQAScore as a trusted attribute detector, on LLM/manual construction of attributes, and on several arbitrary evaluation settings (n=30, CFG 7.5, threshold 0.8). No physical or mathematical entities are invented; the metrics are definitions. The r=0.85 training correlation shares the same scoring procedure on both sides, which is a design choice that can inflate agreement.

free parameters (3)
  • n, images per prompt = 30
    30 images are generated for each dense prompt and an equivalent number for coarse prompts (Section 5). No sensitivity analysis is reported, so the stability of S(p_c,a) at this sample size is unmeasured.
  • VQAScore concept-filter threshold for training images = 0.8
    In Section 5.3, CC12M images with VQAScore below 0.8 are removed before computing training-data DIM; this choice directly shapes the training-side diversity scores and the reported 0.85 correlation.
  • Classifier-free guidance scale = 7.5
    The main comparisons use CFG=7.5; Table 2 shows DIM and CIM change with CFG, so model rankings may depend on this evaluation setting.
assumptions (5)
  • domain assumption VQAScore with truncated text correctly measures attribute presence in generated and training images.
    Section 3.2 uses VQAScore to compute image-attribute scores, and the authors truncate the prompt to the concept-attribute pair to reduce background influence. If VQAScore is biased for certain attributes, both metrics inherit the bias.
  • domain assumption Attributes within each attribute type are mutually exclusive and imageable.
    Section 4 describes manual filtering of attributes that are visually ambiguous or non-exclusive; the attribute-concept score in Eq. (1) treats the attribute set as a meaningful comparison class.
  • domain assumption LLM-generated coarse prompts do not leak visual attributes of the target concept.
    The Does-It Metric relies on coarse prompts with attribute information removed. If the LLM leaves or re-introduces attributes, the default-mode diversity measurement is contaminated.
  • domain assumption LLM-generated and manually filtered attributes are representative of real-world visual diversity.
    The COCO-DIMCIM benchmark is built from Llama3.1 suggestions plus manual filtering; the generality of conclusions about diversity depends on this attribute coverage.
  • domain assumption Comparing models at fixed CFG=7.5 isolates model size/architecture effects.
    Table 2 shows CFG changes DIM/CIM, so the cross-model comparison may confound model properties with the chosen guidance setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models." pith.science (2026). https://pith.science/paper/XREO2B7L

@misc{pith2026250605108,
  author       = {Pith},
  title        = {Pith review of: DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XREO2B7L}},
  note         = {Machine review of arXiv:2506.05108}
}
read the original abstract

Recent advances in text-to-image (T2I) models have achieved impressive quality and consistency. However, this has come at the cost of representation diversity. While automatic evaluation methods exist for benchmarking model diversity, they either require reference image datasets or lack specificity about the kind of diversity measured, limiting their adaptability and interpretability. To address this gap, we introduce the Does-it/Can-it framework, DIM-CIM, a reference-free measurement of default-mode diversity ("Does" the model generate images with expected attributes?) and generalization capacity ("Can" the model generate diverse attributes for a particular concept?). We construct the COCO-DIMCIM benchmark, which is seeded with COCO concepts and captions and augmented by a large language model. With COCO-DIMCIM, we find that widely-used models improve in generalization at the cost of default-mode diversity when scaling from 1.5B to 8.1B parameters. DIMCIM also identifies fine-grained failure cases, such as attributes that are generated with generic prompts but are rarely generated when explicitly requested. Finally, we use DIMCIM to evaluate the training data of a T2I model and observe a correlation of 0.85 between diversity in training images and default-mode diversity. Our work provides a flexible and interpretable framework for assessing T2I model diversity and generalization, enabling a more comprehensive understanding of model performance.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 8 canonical work pages

  1. [4]

    A red table sitting under a ceiling fan

    A wooden table in a living room. A red table sitting under a ceiling fan. … A yellow table with boxes on it. COCO Captions with selected concept (table) as main subject LLM A table in a living room. A table sitting under a ceiling fan. … A table with boxes on it. material: [wooden, metal, plastic] color: [brown, white, black, gray] shape: [rectangular, ro...

    work page 2015

  2. [5]

    https://arxiv.org/abs/2404.07990. Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Do- minik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, ...

    arXiv 2024

  3. [6]

    Melissa Hall, Candace Ross, Adina Williams, Nicolas Carion, Michal Drozdzal, and Adriana Romero Soriano

    https://arxiv.org/abs/2210.02410. Melissa Hall, Candace Ross, Adina Williams, Nicolas Carion, Michal Drozdzal, and Adriana Romero Soriano. Dig in: Evaluating disparities in image generations with indicators for geographic diversity, 2024.https: //arxiv.org/abs/2308.06198. Reyhane Askari Hemmat, Melissa Hall, Alicia Sun, Can- dace Ross, Michal Drozdzal, an...

    arXiv

  4. [7]

    Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidance

    https://arxiv.org/abs/2406.04551. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference- free evaluation metric for image captioning. In Marie- Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors,Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pag...

    work page Pith review arXiv 2021

  5. [8]

    doi: 10.18653/v1/ 2021.emnlp-main.595

    Associa- tion for Computational Linguistics. doi: 10.18653/v1/ 2021.emnlp-main.595. https://aclanthology.org/2021. emnlp-main.595/. Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained 10 by a two time-scale update rule converge to a local nash equilibrium, 2018.https://arxiv.org/abs/1706.08500. Jonathan ...

    arXiv 2021

  6. [12]

    https://arxiv.org/abs/2210. 02747. Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models, 2020.https: //arxiv.org/abs/2002.09797. Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving late...

    work page Pith review arXiv 2020

  7. [13]

    Royi Rassin, Aviv Slobodkin, Shauli Ravfogel, Yanai Elazar, and Yoav Goldberg

    https://arxiv.org/abs/2307.01952. Royi Rassin, Aviv Slobodkin, Shauli Ravfogel, Yanai Elazar, and Yoav Goldberg. GRADE: Quantifying sample diversity in text-to-image models, 2025.https: //openreview.net/forum?id=JddNOaw66n. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution im- age synthesis with latent diffu...

    arXiv 2025

  8. [15]

    Cads: Unleashing the diversity of diffusion models through condition- annealed sampling.arXiv preprint arXiv:2310.17347,

    Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Ot- mar Hilliges, and Romann M Weber. Cads: Unleashing the diversity of diffusion models through condition- annealed sampling.arXiv preprint arXiv:2310.17347,

    arXiv

Show all 23 references
  1. [16]

    Evalalign: Supervised fine-tuning multimodal llms with human-aligned data for evaluating text-to-image models.arXiv preprint arXiv:2406.16562,

    Zhiyu Tan, Xiaomeng Yang, Luozheng Qin, Mengping Yang, Cheng Zhang, and Hao Li. Evalalign: Supervised fine-tuning multimodal llms with human-aligned data for evaluating text-to-image models.arXiv preprint arXiv:2406.16562,

    arXiv

  2. [17]

    Bart Thomee, David A

    https: //arxiv.org/abs/2407.21783. Bart Thomee, David A. Shamma, Gerald Friedland, Ben- jamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. The new data and new chal- lenges in multimedia research.CoRR, abs/1503.01817, 2015.http://arxiv.org/abs/1503.01817. Xi...

    2015 arXiv

  3. [18]

    Diverse diffusion: Enhancing image diversity in text- to-image generation.arXiv preprint arXiv:2310.12583,

    Mariia Zameshina, Olivier Teytaud, and Laurent Najman. Diverse diffusion: Enhancing image diversity in text- to-image generation.arXiv preprint arXiv:2310.12583,

    arXiv

  4. [19]

    A contrastive compositional benchmark for text-to-image synthesis: A study with unified text-to-image fidelity metrics

    Xiangru Zhu, Penglei Sun, Chengyu Wang, Jingping Liu, Zhixu Li, Yanghua Xiao, and Jun Huang. A contrastive compositional benchmark for text-to-image synthesis: A study with unified text-to-image fidelity metrics. arXiv preprint arXiv:2312.02338,

    arXiv

  5. [20]

    11 Appendix A Flow-Int Training Details TheFlow-IntversionevaluatedinSection5.1wastrainedleveragingflowmatching(Lipmanetal.,2023), control conditions (Berrada et al., 2025; Podell et al.,

    2023

  6. [21]

    broken”umbrellaand “foal

    and latent perceptual loss (Berrada et al., 2025). The training dataset consisted of image-caption pairs including ImageNet (Deng et al., 2009), CC12M (Changpinyo et al., 2021), YFCC (Thomee et al., 2015), and an internally licensed dataset. The version of Flow-Int evaluated i...

    2022

  7. [22]

    an empty refrigerator in

    • Figure 9 has examples that show that negations are difficult for all models. When these models are prompted to not to generate an attribute, more often than not they generate those attributes in the images. • Figure 10 has examples that show that some attribute types likemat...

    2022

  8. [2009]

    Moreno D’Incà, Elia Peruzzo, Massimiliano Mancini, De- jia Xu, Vidit Goel, Xingqian Xu, Zhangyang Wang, Humphrey Shi, and Nicu Sebe

    doi: 10.1109/CVPR.2009.5206848. Moreno D’Incà, Elia Peruzzo, Massimiliano Mancini, De- jia Xu, Vidit Goel, Xingqian Xu, Zhangyang Wang, Humphrey Shi, and Nicu Sebe. Openbias: Open-set bias detection in text-to-image generative models,

    2009

  9. [2015]

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan

    https://arxiv.org/abs/1405.0312. Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual genera- tion with image-to-text generation.arXiv preprint arXiv:2404.01291,

    arXiv

  10. [2019]

    Black Forest Labs

    https://arxiv.org/abs/1904.06991. Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux,

    1904 arXiv

  11. [2020]

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith

    doi: 10.5281/ zenodo.1212303. Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. InProceedings of the IEEE/CVF International Conferen...

    arXiv

  12. [2021]

    Davidsonian scene graph: Improving reliability in fine-grained evaluation for text- to-image generation.arXiv preprint arXiv:2310.18235, 2023a

    Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, and Su Wang. Davidsonian scene graph: Improving reliability in fine-grained evaluation for text- to-image generation.arXiv preprint arXiv:2310.18235, 2023a. Jaem...

    arXiv

  13. [2022]

    What makes a good metric? eval- uating automatic metrics for text-to-image consistency

    Candace Ross, Melissa Hall, Adriana Romero Soriano, and Adina Williams. What makes a good metric? eval- uating automatic metrics for text-to-image consistency. arXiv preprint arXiv:2412.13989,

    arXiv

  14. [2023]

    Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023.https://arxiv.org/abs/2305

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023.https://arxiv.org/abs/2305. 06500. Jia Deng, Wei Dong, Ri...

    2023

  15. [2024]

    Gabriele Corso, Yilun Xu, Valentin De Bortoli, Regina Barzilay, and Tommi Jaakkola

    https://arxiv.org/abs/2411.17066. Gabriele Corso, Yilun Xu, Valentin De Bortoli, Regina Barzilay, and Tommi Jaakkola. Particle guidance: non-iid diverse sampling with diffusion models.arXiv preprint arXiv:2310.13102,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.