Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Evaluating Hallucination in Text-to-Image Diffusion Models with Scene-Graph based Question-Answering Agent

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that prompt-image consistency in text-to-image models can be scored automatically by asking an LLM to answer prompt-derived questions against a scene graph extracted from the generated image, and that this score tracks…

desk verdict A plausible interpretable metric with a valuable human-scored dataset, but the headline claim of closer human alignment rests on statistically untested correlation differences of ~0.02. read the letter →

arxiv 2412.05722 v1 pith:XFQFQYEF submitted 2024-12-07 cs.CV

Ziyuan Qin , Dongjie Cheng , Haoyu Wang , Huahui Yi , Yuting Shao , Zhiyuan Fan , Kang Li , Qicheng Lao This is my paper · ORCID
classification cs.CV
keywords text-to-imageevaluationhallucinationdetectionscenegraphquestionansweringdiffusionmodelshuman-alignedscoringattributebindingT2Iconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the consistency between a text prompt and a generated image can be measured automatically as a question-answering task: build a scene graph of the image, turn the prompt into templated questions, and let an LLM answer them by retrieving triples from the graph. Wrong answers indicate what the authors call 'hallucinations' in the image, and the pattern of errors reveals whether the failure is an attribute, a relation, a missing object, or an extraneous object. On 12,000 images generated from 1,000 composite prompts by three Stable Diffusion models, the proposed score correlates with human ratings more closely than CLIP-score, BLIP-score, LLM-score, and T2ICompBench. The authors will release the images and human scores, giving the field a reusable benchmark for evaluating evaluators.

What carries the argument

The load-bearing mechanism is the image-specific scene graph combined with a retrieval-based question-answering loop. Each detected object becomes a node, attributes are bound to object nodes, and edges encode spatial relations (from bounding boxes) and non-spatial relations (from BLIP-2 question answering). On the prompt side, a dependency parser splits the text into triples of subject, relation, and object, and an LLM generates template questions and ground-truth answers from those triples. A graph-retrieval module extracts entities from each question, pulls matching triples from the scene graph as memory, and the LLM decides whether that memory answers the question; a rule-based scorer then converts wrong answers into a 1-to-7 human-style rating.

What would settle it

Take images for which the open-vocabulary detector is known to miss prompt-relevant objects (the authors report this for landscape background elements), have humans score them, and check whether the metric's scores diverge from human scores on exactly those images while agreeing elsewhere. A sharper test is to feed a hand-verified complete scene graph into the question-answering stage instead of the extracted one; if the score still fails to track human ratings, the bottleneck is not the graph but the question-answering or scoring design.

Watch

Extended reading notes

Core claim

The central claim is that a graph question-answering agent can serve as a faithful proxy for human judgment of text-to-image consistency. The paper states the operating hypothesis explicitly: if a generated image faithfully reflects the prompt, the agent should be able to answer all prompt-derived questions by retrieving the relevant triples from the image's scene graph, so an erroneous answer marks a hallucinated part of the image. The scene graph is built with an open-vocabulary detector (GroundedSAM) for objects and a VQA model (BLIP-2) for attributes and non-spatial relations, with spatial relations computed from bounding-box coordinates; questions are generated from the prompt's dependency tree. The main quantitative evidence is that the method's Pearson, Kendall, and Spearman correlations with human scores on the 12,000-image dataset exceed those of four baseline metrics, and that its F1 scores for identifying attribute, omission, relation, and extraneous-object hallucinations exceed T2ICompBench's.

Load-bearing premise

The entire metric rests on the assumption that the automatically extracted scene graph is a complete and correct representation of everything in the image that the prompt cares about; if the detector misses a prompt-relevant object or the VQA model misreads an attribute, the score is wrong regardless of how faithful the image actually is.

Editorial extensions

If this is right

  • The metric can rank and compare text-to-image models automatically, producing both an overall score and a breakdown of hallucination types for each model.
  • Model developers can use the per-type error counts (attribute, relation, omission, extraneous) to localize weaknesses, such as attribute-binding failures in the text encoder.
  • The released 12,000 human-scored image-prompt pairs become a benchmark for testing any future automatic evaluation metric against human judgment.
  • Because the pipeline uses detection, VQA, and LLM components with no training, it can be rerun on new prompts and models as they appear.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • As open-vocabulary detection improves to handle background and overlapping objects, the same graph-QA design should extend to the landscape images where the current version demonstrably fails.
  • The framework could be inverted into a prompt-repair tool: the specific triples the agent fails to verify identify which phrase of the prompt the image did not realize, and could be fed back to the generator as a corrective signal.
  • A controlled perturbation test (swapping attributes or relations in the prompt and checking that the score drops accordingly) would strengthen the causal reading of the method, since the current evidence is correlational.
  • The hallucination-type frequencies, if published per model, could serve as a diagnostic of text-encoder capability independent of image quality, since the authors observe that attribute-binding errors are common across models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes an automatic evaluation metric for text-to-image (T2I) faithfulness, named scene-graph-based question-answering (SGQA). The pipeline first generates images from composite prompts, then builds an image-specific scene graph using GroundedSAM for object detection and BLIP-2 for attribute and relation extraction. Template questions are derived from the dependency tree of the prompt, and an LLM answers these questions by retrieving triples from the scene graph. A rule-based module converts the QA results into a hallucination severity score and categorizes errors into attribute, relation, object omission, and extraneous-object types. The authors also construct a dataset of 12,000 human-scored images from 1,000 prompts using three Stable Diffusion models. The central claim is that the proposed metric correlates more closely with human scoring than existing metrics such as CLIP-Score, BLIP-Score, LLM-Score, and T2ICompBench.

Significance. If the central claim holds, the method would provide a more interpretable and granular automatic evaluation for T2I models, with the additional contribution of a publicly released human-scored dataset. The paper explicitly ships the threat of a load-bearing assumption: the scene-graph completeness issue is acknowledged (Section 5), and the human gold standard is central to all correlation analyses. The paper also offers a useful categorization of hallucination types and a machine-checkable recipe for evaluation. However, the quantitative evidence for the headline claim is currently weak: the correlation advantages over T2ICompBench are small (Pearson 0.26 vs. 0.24; Spearman tied), no significance tests or confidence intervals are reported, and the best variant (GPT-4) was evaluated on only a 10% subset. These issues undermine the comparative claim as stated.

major comments (4)
  1. [§4.2.1, Table 1] The central claim that the proposed metric 'aligns more closely with human scoring patterns' is not established by the reported numbers. The best full-data variant shows Pearson 0.26 vs. T2ICompBench 0.24, Kendall 0.23 vs. 0.20, and Spearman tied at 0.27. No confidence intervals or significance tests are provided, and with N≈12,000 the Pearson difference is not significant (approximate Fisher z-test p≈0.11). Moreover, Section 4.2.2 states that GPT-4 was run on only ~10% of the data, but Table 1 does not indicate whether its correlations are subset-restricted; if they are, the comparison against full-data baselines is not apples-to-apples. The text in Section 4.2.1 also refers to 'our method(w/GPT-3.5)', a configuration absent from Table 1, leaving the headline result ambiguous. Please report the exact configuration, run the best variant on the full data or state subset-restricted results explicitly, and provide confidence intervals or significance tests for all correlation differences.
  2. [§3.2 and §5] The metric's validity rests on the completeness and correctness of the extracted scene graph. The authors acknowledge that GroundedSAM 'struggles to effectively detect key objects in synthesized landscape images, possibly due to these being treated as background'; in such cases every downstream QA answer and the final score are wrong regardless of actual image faithfulness. This is a direct violation of the core operating hypothesis stated in Section 3.5, and it limits the generality of the proposed metric. The manuscript should quantify the frequency of such detection failures (e.g., per prompt category) and either restrict the method's claimed scope or provide a recovery mechanism (e.g., additional VQA-based object verification).
  3. [§4.1] The human gold-standard scores are used as the target for all correlations, but the manuscript does not report inter-rater agreement (e.g., Cohen's kappa or Krippendorff's alpha), the number of annotators per image, or the annotator instructions beyond the 7-point rubric. Without reliability evidence, it is unclear how much of the correlation differences are attributable to the metric versus noise in the human labels. This is particularly important because the rubric requires subjective judgments such as 'more than two missing objects' and whether extraneous objects are 'unreasonable' (Section 1). Please report agreement statistics and, if feasible, the per-image majority-vote construction.
  4. [§4.2.2, Table 2] The claim that 'our method achieves higher F1 scores across different hallucination types' is not supported by the table. For relation errors, T2ICompBench achieves an F1 of 0.62, equal to Ours(w/GPT-3.5-turbo) and higher than Ours(w/Gemini) at 0.61. Also, T2ICompBench is simply not evaluated on omission or extraneous-object types (shown as '–'), so the comparison is incomplete. The text should either qualify the claim to the types where the method is actually better or provide a fairer head-to-head evaluation with the missing cells filled in.
minor comments (6)
  1. [§3.3] The section heading 'Rasing Templated Questions' contains a typo; it should be 'Raising Templated Questions'.
  2. [§3.5] In the sentence 'the graph question-answering agent should be able to answer all the questions by retiring the relevant triples', 'retiring' should be 'retrieving'.
  3. [§5] The heading 'Future works and Limiations' contains a typo; it should be 'Future Works and Limitations'.
  4. [§4.2.1] The text says 'Tabel 1' and later 'From Tabel 1'; these should be 'Table 1'.
  5. [§1] In the sentence 'From out experiments, except for explicitly unmatched relationships...', 'out' should be 'our'.
  6. [Abstract and §4.1] The dataset is described as 'will be made publicly available in the future' but no repository or release plan is given; please provide a concrete availability statement (e.g., URL or expected hosting platform) to enable reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: the scene-graph QA metric is validated against independent human labels; no parameter is fitted to the target scores and no self-citation chain forces the reported correlations.

full rationale

The paper's derivation chain is an empirical evaluation pipeline, not a derivation whose conclusion is encoded in its inputs. In Sections 3.2-3.5, the metric is built from external components (GroundedSAM, BLIP-2, spaCy, LangChain) and a hand-written rule-based scoring module; the final score is a function of GraphQA errors on the extracted scene graph. Human scores enter only as a gold standard in Section 4.2.1 and Table 1; there is no step in which a parameter, threshold, or equation is fitted to those human ratings and then reported as a prediction. The central hypothesis in Section 3.5 ('if the generated images faithfully reflect the content in the text prompts, the graph question-answering agent should be able to answer all the questions') is an empirical claim tested by correlation, not a definitional identity. The human scoring rubric shares a severity taxonomy with the metric (omission and relation errors more severe than attribute errors), but that taxonomy is a design choice, not numerically optimized against the validation labels; the paper also states that extraneous-object criteria could not be reliably automated. No load-bearing self-citation appears in the reference list, and the acknowledged GroundedSAM limitation in Section 5 is a measurement-validity concern rather than circularity. Statistical weaknesses such as small correlation differences, absent confidence intervals, and the GPT-4 variant running on only about 10% of data are separate from the question of whether the claimed result reduces to its own inputs. On the exhibited evidence, no circular step can be quoted with a specific reduction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim is an empirical correlation claim, not a derivation, so there are no fitted numerical parameters in a mathematical sense; however, the hand-chosen 7-point rubric thresholds, the unspecified rule-based scoring step, and the assumed accuracy of the scene-graph extraction pipeline are the main unvalidated inputs. No new physical or theoretical entities are introduced.

free parameters (2)
  • hallucination severity score thresholds = 1-7 rubric with thresholds such as more than two missing objects = 2pt and attribute errors = 4pt
    Section 4.1 defines the human scoring scale and severity categories; these thresholds are chosen by hand and define the ground truth that the automated metric is validated against.
  • rule-based scoring aggregation = not specified in the paper
    Section 3.5 says a rule-based scoring system converts QA errors into the final rating, but the actual rules and any weights are not given, so the mapping from error counts to the reported score is a hand-designed black box.
assumptions (4)
  • domain assumption The 7-point human scoring rubric is a valid gold standard for prompt-image consistency.
    Section 4.1 states the annotator scoring criteria; the entire correlation analysis treats these scores as ground truth, but no inter-rater agreement statistics are reported.
  • domain assumption GroundedSAM and BLIP-2 produce a sufficiently complete and correct scene graph.
    Section 3.2 builds all graph nodes and attribute edges from these models, and Section 5 admits landscape key objects are missed, so completeness is not guaranteed.
  • domain assumption The T2I-CompBench prompts are representative of composite text-to-image tasks.
    Section 4.1 uses 1,000 prompts from T2I-CompBench; performance on other prompt distributions is untested.
  • domain assumption The LLM GraphQA agent answers correctly when the graph contains the relevant triples.
    Section 3.4 uses LangChain GraphQA with LLMs (GPT-3.5, Gemini, GPT-4) to retrieve and answer; there is no independent verification of answer correctness apart from final correlation with human scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Hallucination in Text-to-Image Diffusion Models with Scene-Graph based Question-Answering Agent." pith.science (2026). https://pith.science/paper/XFQFQYEF

@misc{pith2026241205722,
  author       = {Pith},
  title        = {Pith review of: Evaluating Hallucination in Text-to-Image Diffusion Models with Scene-Graph based Question-Answering Agent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XFQFQYEF}},
  note         = {Machine review of arXiv:2412.05722}
}
read the original abstract

Contemporary Text-to-Image (T2I) models frequently depend on qualitative human evaluations to assess the consistency between synthesized images and the text prompts. There is a demand for quantitative and automatic evaluation tools, given that human evaluation lacks reproducibility. We believe that an effective T2I evaluation metric should accomplish the following: detect instances where the generated images do not align with the textual prompts, a discrepancy we define as the `hallucination problem' in T2I tasks; record the types and frequency of hallucination issues, aiding users in understanding the causes of errors; and provide a comprehensive and intuitive scoring that close to human standard. To achieve these objectives, we propose a method based on large language models (LLMs) for conducting question-answering with an extracted scene-graph and created a dataset with human-rated scores for generated images. From the methodology perspective, we combine knowledge-enhanced question-answering tasks with image evaluation tasks, making the evaluation metrics more controllable and easier to interpret. For the contribution on the dataset side, we generated 12,000 synthesized images based on 1,000 composited prompts using three advanced T2I models. Subsequently, we conduct human scoring on all synthesized images and prompt pairs to validate the accuracy and effectiveness of our method as an evaluation metric. All generated images and the human-labeled scores will be made publicly available in the future to facilitate ongoing research on this crucial issue. Extensive experiments show that our method aligns more closely with human scoring patterns than other evaluation metrics.

Figures

Figures reproduced from arXiv: 2412.05722 by the authors.

Figure 1
Figure 1. The complete framework for the text-to-image (T2I) generation to evaluation process is as [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Qualitative Evaluation: We select several hallucinated images not aligned with the prompts [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Models

    cs.AI 2026-08 conditional novelty 6.0 of 10

    An agentic framework and a distilled 3B model evaluate text-to-image and text-to-video generators with roughly 10% of the samples of standard benchmarks, at the cost of only partial accuracy.

  2. Mitigating Diffusion Model Hallucinations with Dynamic Guidance

    cs.CV 2025-10 conditional novelty 6.0 of 10

    Dynamic per-step selection of the classifier-guidance target reduces diffusion-model hallucinations on 2D, shape, and hand-image benchmarks, with proxy-metric gains on ImageNet.

Reference graph

Works this paper leans on

30 extracted references · 9 canonical work pages · cited by 2 Pith papers

  1. [1]

    Image quality metrics: Psnr vs

    Alain Horé and Djemel Ziou. Image quality metrics: Psnr vs. ssim. 2010 20th International Conference on Pattern Recognition, pages 2366–2369, 2010

    work page 2010

  2. [2]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Interna- tional Conference on Machine Learning , 2021

    work page 2021

  3. [3]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, 2022

    work page 2022

  4. [4]

    T2i-compbench: A comprehen- sive benchmark for open-world compositional text-to-image generation

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A comprehen- sive benchmark for open-world compositional text-to-image generation. ArXiv, abs/2307.06350, 2023

    arXiv 2023

  5. [5]

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A. Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. ArXiv, abs/2303.11897, 2023

    arXiv 2023

  6. [6]

    Holistic evaluation of text-to-image models

    Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, Minguk Kang, Taesung Park, Jure Leskovec, Jun-Yan Zhu, Fei-Fei Li, Jiajun Wu, Stefano Ermon, and Percy Liang. Holistic evaluation of text-to-image models. ArXiv, abs/2311.04287, 2023

    arXiv 2023

  7. [7]

    Attribute2image: Conditional image generation from visual attributes, 2016

    Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. Attribute2image: Conditional image generation from visual attributes, 2016

    2016

  8. [8]

    Neural discrete representation learning, 2018

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning, 2018

    2018

Show all 30 references
  1. [9]

    Generating diverse high-fidelity images with vq-vae-2

    Ali Razavi, Aäron van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. In Neural Information Processing Systems, 2019

    2019

  2. [10]

    Zero-shot text-to-image generation, 2021

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation, 2021

    2021

  3. [11]

    Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer

    Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, 2021

    2022

  4. [12]

    Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David J

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image di...

    2022 arXiv

  5. [13]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. ArXiv, abs/2204.06125, 2022

    2022 arXiv

  6. [14]

    Jonathan Ho, Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models. ArXiv, abs/2006.11239, 2020

    2006 arXiv

  7. [15]

    Openclip, July 2021

    Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. If you use this software, please cite it as below

    2021

  8. [16]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023. 11

    2023

  9. [17]

    Siren’s song in the ai ocean: A survey on hallucination in large language models, 2023

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren’s song in the ai ocean: A survey on hallucination in large language models, 2023

    2023

  10. [18]

    Analyzing and mitigating object hallucination in large vision-language models

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models. ArXiv, abs/2310.00754, 2023

    2023 arXiv

  11. [19]

    Woodpecker: Hallucination correction for multimodal large language models

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xingguo Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models. ArXiv, abs/2310.16045, 2023

    2023 arXiv

  12. [20]

    Effectively unbiased fid and inception score and where to find them

    Min Jin Chong and David Alexander Forsyth. Effectively unbiased fid and inception score and where to find them. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6069–6078, 2019

    2020

  13. [21]

    Barratt and Rishi Sharma

    Shane T. Barratt and Rishi Sharma. A note on the inception score. ArXiv, abs/1801.01973, 2018

    2018 arXiv

  14. [22]

    Improved precision and recall metric for assessing generative models

    Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. ArXiv, abs/1904.06991, 2019

    1904 arXiv

  15. [23]

    Benchmark for compositional text-to-image synthesis

    Dong Huk Park, Samaneh Azadi, Xihui Liu, Trevor Darrell, and Anna Rohrbach. Benchmark for compositional text-to-image synthesis. In NeurIPS Datasets and Benchmarks , 2021

    2021

  16. [24]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. ArXiv, abs/2104.08718, 2021

    2021 arXiv

  17. [25]

    Llmscore: Unveiling the power of large language models in text-to-image synthesis evaluation

    Yujie Lu, Xianjun Yang, Xiujun Li, Xin Eric Wang, and William Yang Wang. Llmscore: Unveiling the power of large language models in text-to-image synthesis evaluation. ArXiv, abs/2305.11116, 2023

    2023 arXiv

  18. [26]

    Dall-eval: Probing the reasoning skills and social biases of text-to-image generative transformers

    Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to-image generative transformers. CoRR, abs/2202.04053, 2022

    2022 arXiv

  19. [27]

    Grounded-Segment-Anything, April 2023

    Grounded-SAM Contributors. Grounded-Segment-Anything, April 2023

    2023

  20. [28]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning, 2023

    2023

  21. [29]

    spaCy: Industrial- strength Natural Language Processing in Python

    Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial- strength Natural Language Processing in Python. 2020

    2020

  22. [30]

    LangChain, October 2022

    Harrison Chase. LangChain, October 2022. 12

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.