Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation

T0 review · 5 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training-free pipeline that explicitly decomposes metaphors into source, target, and meaning, then iteratively refines generated images using vision-language feedback, outperforms GPT-4o and Imagen-3 on automatic metaphor-alignment…

desk verdict Useful framework, but the headline 'strongest alignment' claim is weaker than it looks because the same VLM is both the judge and the optimizer; the paper's own user study reverses the ranking. read the letter →

arxiv 2508.18569 v1 pith:P37EIKSR submitted 2025-08-26 cs.CL cs.CV

Girish A. Koushik , Fatemeh Nazarieh , Katherine Birch , Shenbin Qian , Diptesh Kanojia This is my paper · ORCID
classification cs.CLcs.CV
keywords visualmetaphorgenerationsource-target-meaningdecompositionVLM-as-judgeself-evaluationrewardGRPOreinforcementlearningiterativepromptrefinementtext-to-imagealignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether visual metaphor generation can be guided by explicit semantic structure instead of left to literal text-to-image rendering. It proposes decomposing each metaphor into its concrete source, abstract target, and intended meaning, generating an image, then using a vision-language model to score the result and feed those scores back into prompt rewriting or reinforcement learning. On the 462-metaphor HAIVMet test set, the training-free pipeline beats GPT-4o and Imagen-3 on CLIP score, meaning alignment, and decomposition score, with the GRPO-tuned variant close behind. A user study tempers the claim: human raters preferred GPT-4o overall, while the training-free pipeline led open-source methods and edged Imagen-3 on abstract metaphors. The significance is that explicit metaphor decomposition plus iterative feedback improves semantic alignment under modest compute, and the remaining gap to human preference appears to be aesthetics and sampling rather than metaphor understanding.

What carries the argument

The load-bearing mechanism is the source-target-meaning (S-T-M) decomposition: for each metaphor, the LLM names the concrete source, the abstract target, and the intended meaning, then writes a visual prompt from that structure. Around this sits a composite reward $R(I,P,D)=\sum_k w_k r_k$ that combines a decomposition-quality score, CLIPScore, VLM judgments of source presence, target presence, and meaning alignment, and BERTScore similarities between the original and perceived S, T, and M. In the training-free pipeline this reward drives ten rounds of generate-evaluate-refine, with the highest-scoring image selected; in the GRPO pipeline the same reward updates LoRA weights of a small LLM. The mechanism matters because it converts 'does the image capture the metaphor' from an unstated goal into a numerically guided feedback loop.

What would settle it

Conduct a larger human study on a random sample of the full 462-metaphor test set, asking raters to judge only metaphor faithfulness while hiding style and realism; if raters systematically prefer GPT-4o's images while the Qwen-based scores keep favoring the training-free pipeline, the automatic alignment signal is not measuring human metaphor alignment. A cheaper check is to swap the VLM judge for an independently trained model and observe whether the pipeline rankings change.

Watch

Extended reading notes

Core claim

The central claim is that a training-free pipeline — LLM decomposition into source-target-meaning, image synthesis, VLM critique, iterative prompt rewriting over ten rounds, then selection of the highest-reward image — achieves the strongest automatic metaphor alignment, surpassing both GPT-4o and Imagen-3 on CLIP, meaning-alignment, and decomposition scores. The best configuration, GEMMA-3-27B with Janus-Pro-7B and Qwen2.5-VL-32B, reaches 0.2960 CLIP, 0.8760 meaning alignment, and 0.8668 decomposition, compared with GPT-4o's 0.2296, 0.8180, and 0.8072. The paper also claims that GRPO fine-tuning with low-rank adapters on a 4B Gemma model performs competitively on CLIP and source presence with far less compute, and that S-T-M prompting specifically helps longer or more abstract metaphors while closed models excel on short, concrete ones. It openly reports that a 15-annotator user study favored GPT-4o by 127 to 79 votes over the training-free pipeline, attributing the divergence to aesthetics and sampling rather than semantic alignment.

Load-bearing premise

The automatic judgment scores — especially the vision-language model's meaning-alignment and decomposition scores — are a valid substitute for human metaphor alignment, even though the same model provides the training reward and the evaluation, and even though human raters preferred GPT-4o in the user study.

Editorial extensions

If this is right

  • Structured S-T-M prompting and iterative VLM feedback can improve metaphor alignment for longer or more abstract prompts, where closed models like GPT-4o perform worse.
  • The training-free pipeline is immediately deployable with existing text-to-image and vision-language models, since it requires no retraining or labeled data.
  • Lightweight GRPO fine-tuning gives a smaller LLM competitive CLIP and source-presence scores under limited compute, making metaphor-aware prompting feasible without large-scale training.
  • The gap between automatic alignment scores and human preference is attributed by the paper to aesthetics and sampling, so adding style-aware rewards is a direct next step.
  • The framework is model-agnostic and can be lifted across text-to-image backbones, but performance is sensitive to sampler settings such as guidance scale and inference steps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: the S-T-M decomposition is a textual intervention, so the same reward loop may transfer to other figurative-generation tasks, such as diagrammatic explanations or visual puns, where source-target structure is explicit.
  • Our inference: because the VLM judge also supplies the training reward, a meaningful next experiment is to re-evaluate the best pipeline with an independent, human-calibrated judge; the paper's own user study suggests the ranking could flip.
  • Our inference: the composite reward weights are fixed by hand (0.20 for decomposition and CLIP, 0.10 for the rest); tuning these weights, or learning them from human preference, would likely change which configurations win.
  • Our inference: testing the pipeline on non-HAIVMet metaphors, including cross-lingual or culturally specific metaphors, would test whether S-T-M decomposition generalizes beyond the benchmark's distribution.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a framework for visual metaphor generation built around an explicit Source–Target–Meaning (S–T–M) decomposition. An LLM decomposes the input metaphor and writes a visual prompt; a text-to-image model (Stable Diffusion 3.5 Medium or Janus-Pro-7B) renders the image; and a multi-faceted reward, combining CLIPScore, BERTScore, and Qwen2.5-VL judgments, drives either iterative prompt refinement (training-free pipeline) or GRPO-based LoRA fine-tuning of Gemma-3-4B. The authors evaluate on the HAIVMet test set against GPT-4o and Imagen-3, report automatic scores (CLIP, MA, decomposition, presence, BERT-based similarities), and conduct a 15-annotator user study on 50 metaphors. The paper claims that the training-free pipeline achieves the strongest alignment, surpassing GPT-4o and Imagen-3 on CLIP, MA, and decomposition scores, while the user study preferred GPT-4o overall.

Significance. If the central claim were established, the paper would make a useful practical contribution: structured S–T–M prompting plus iterative VLM feedback is a low-compute way to improve metaphorical alignment, and the dual training-free / GRPO design is sensible. The paper is strong on transparency: it provides full prompt templates in Appendix C, hyperparameters in Appendix A, a user study, and an explicit limitations paragraph. However, the headline automatic-score advantage is not currently convincing because the same Qwen2.5-VL model provides the reward signal, the refinement feedback, and the final evaluation; the evaluation prompts are not matched across systems; and the main human-preference result reverses the automatic ranking. These issues are load-bearing for the paper's core claim, so the contribution cannot be assessed until they are addressed.

major comments (5)
  1. [Sections 2.2, 2.3, Table 2] The central alignment claim is vulnerable to reward hacking. In Section 2.2, the MA score, decomposition reward, source-presence, and target-presence scores are all outputs of Qwen2.5-VL. In Section 2.3, the training-free pipeline selects the prompt that maximizes the composite reward R, which includes those VLM scores. Table 2 then reports the same VLM's scores as evidence of improvement. This is a self-referential loop: the system is optimized to satisfy Qwen2.5-VL's rubric, so high MA and decomposition scores may reflect overfitting to one judge's biases rather than genuinely better metaphorical images. The paper needs an independent evaluation, for example a different VLM judge plus human annotation on a sample, and a report of agreement between automatic scores and human judgments.
  2. [Section 3.2, Figure 4, Section 3.3] The user study contradicts the automatic-score ranking. Human raters chose GPT-4o as best in 127 cases versus 79 for the training-free pipeline, even though the study instructions asked participants to judge metaphor alignment and conceptual clarity rather than style. The paper attributes this to 'aesthetics bias' and subset composition, but provides no supporting analysis, such as separate style/aesthetics ratings or a breakdown showing that GPT-4o wins occur mainly on short, concrete metaphors. Without such evidence, the claim in Section 3.3 that the remaining gap is 'driven more by aesthetics and sampling than by semantic understanding' is unsupported.
  3. [Appendix C, Tables 1 and 2] The automatic evaluation is not matched across systems. Imagen-3 images are evaluated with prompt C.2, which gives the judge only the original metaphor and asks for a single alignment score, while GPT-4o images are evaluated with C.1 and the proposed pipelines are evaluated with C.4, both of which supply the intended S–T–M decomposition to the judge. The higher MA scores for the proposed pipelines could therefore reflect that the judge was provided with the intended decomposition, not that the images are better metaphors. All systems should be evaluated with identical prompts, or the paper should report results under both with-STM and without-STM protocols.
  4. [Section 3.1, Table 2, Abstract] The best configuration appears to be selected on the held-out test set, which inflates the reported numbers. Table 2 reports several configurations with different image-generation hyperparameters and different VLM sizes; the abstract and Section 3.1 report the single best training-free configuration. No validation split or model-selection procedure is described. The authors should either define a validation set for configuration selection or report results for all configurations with variability estimates, and avoid claiming that the best test-set configuration represents expected performance.
  5. [Section 3.3, Tables 1 and 2] The claim that the training-free pipeline 'surpasses both GPT-4o and Imagen-3 on CLIP, MA, and decomposition scores' is not fully supported by the tables. For Imagen-3, the decomposition score is not reported ('-' in Table 1), so the decomposition-score comparison against Imagen-3 is vacuous. Moreover, the CLIP score is computed between the generated image and the visual prompt, not the metaphor itself; for different systems the visual prompts are produced by different processes, making cross-system CLIP comparisons difficult to interpret as metaphor alignment.
minor comments (5)
  1. [Section 2.2, Appendix A.3] The reward weights in Appendix A.3 do not sum to 1: for the training-free pipeline, 0.20 (decomposition) + 0.20 (CLIP) + five × 0.10 = 0.90, and the GRPO pipeline adds two more 0.10 weights, totaling 1.10. Please clarify whether the weights are normalized or whether the sum is intentionally not 1.
  2. [Section 3.2, Appendix D] The user study section should state whether annotators were compensated and report any IRB or ethics-approval details in the main text, rather than only in the checklist. It should also report inter-annotator agreement or variance for the Likert ratings.
  3. [Throughout the paper] There are inconsistent renderings of model names, such as 'GPT-4 O' versus 'GPT-4o' and 'IMAGEN-3' versus 'Imagen-3'. Please standardize these names.
  4. [Section 2.4] The GRPO loss formula is written in a simplified form and cites a debiased variant from reference [21]; please provide the exact loss definition used in the code, including the advantage normalization and clipping, or cite the precise TRL implementation.
  5. [Figure 4] Figure 4 reports best-image vote counts without indicating the number of metaphors per length category or any uncertainty. A short caption note stating the sample sizes would help readers interpret the 'performance by length' panels.

Circularity Check

2 steps flagged · score 7.0 of 10

The reported MA/decomposition wins are generated by the same Qwen2.5-VL judge that supplied the reward/feedback, so the headline 'strongest alignment' claim is not independently verified; the paper's own human study reverses the ranking.

  1. fitted input called prediction [Section 2.2 (Multi-Faceted Evaluation Framework), Section 2.3 (Training-Free In-Context Refinement), Section 3.3 (Discussion)]
    "VLM-based Analysis: A VLM (Qwen-VL) is prompted to perform a critical analysis of the generated image, leveraging VLM-as-a-judge [20]. It identifies the perceived source (S′), target (T′), and meaning (M′) from the visual content. It then provides scores for the presence and clarity of the source (vlms_presence), the effectiveness of the symbolic representation of the target (vlmt_presence), and the alignment of the perceived meaning with the original intent (vlmm_align). ..."

    The training-free loop selects the image that maximizes the composite reward R, and vlmm_align (the meaning-alignment score) is one of the reward terms. The same Qwen2.5-VL judge then produces the MA values reported in Table 2 and the Section 3.3 claim of 'strongest alignment.' This is selecting on the target metric: the reported MA advantage over zero-shot GPT-4o/Imagen is the value of the very objective used for refinement, not an independent measurement. The GRPO pipeline is the same: the policy is trained to maximize R, which includes the VLM decomposition and alignment scores, and those same scores are later reported as evaluation results.

  2. fitted input called prediction [Section 2.4 (GRPO-based Refinement) and Table 2]
    "Reward Computation: Each generated candidate is evaluated using the shared multi-faceted evaluation framework. ... Policy Update: The GRPO algorithm uses the computed rewards to update the model’s LoRA weights."

    In the GRPO pipeline, the decomposition reward and VLM-based MA are part of the reward used to update GEMMA-3-4B's LoRA weights. Table 2 then reports decomposition and MA scores from the same evaluator as evidence of performance. These are the values of the training objective, so the 'strong decomposition' and 'strong MA' rows do not constitute out-of-sample validation. The paper's own Limitations paragraph acknowledges that automated model-based scoring 'drive[s] both selection and GRPO rewards' and that these signals 'do not fully capture human aesthetic preference.'

full rationale

The paper's central contribution is a self-evaluating framework, and the self-evaluation loop is genuine: a VLM supplies feedback, the LLM refines the prompt, and the image with the highest composite reward is selected. That loop is not itself circular; the circularity enters when the same VLM scores are then presented as the evidence for 'strongest alignment' in Section 3.3. Because MA and decomposition are defined as outputs of Qwen2.5-VL and are components of the reward R, the reported gains on those metrics are partly constructed by the optimization procedure. The CLIP score is an external signal and prevents a score of 10, but the paper's own user study (GPT-4o: 127 best-image votes vs. training-free: 79) reverses the ranking on the central construct, and the paper attributes the gap to 'aesthetics bias' and subset sampling without independent support. No load-bearing self-citation or uniqueness-theorem import is present; the problem is the evaluator/optimizer overlap, which the Limitations paragraph explicitly concedes. Overall, the automatic MA/decomposition claim reduces to a self-referential judge, so the circularity score is 7 rather than 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 3 invented entities

The central results depend on hand-chosen reward weights, fixed iteration counts, sampling hyperparameters, and unvalidated VLM judgments. The paper assumes clean S-T-M decompositions, that embedding similarity captures metaphor alignment, and that a VLM judge can stand in for human perception. The MA and decomposition scores are new instruments with no external validation, and they act as both the optimization signal and the evaluation ruler.

free parameters (4)
  • reward function weights = w_decomp=0.20, w_clip=0.20, w_others=0.10; w_format=0.10, w_length=0.10 for GRPO
    Hand-chosen weights determine which image is selected in the training-free pipeline and the GRPO training signal; no sensitivity analysis is provided for them.
  • refinement iterations N = 10
    The training-free loop runs a fixed 10 iterations; the choice affects final reward and compute cost, with no reported convergence analysis.
  • GRPO hyperparameters = LoRA rank 32, alpha 64, learning rate 5e-5, batch size 4, 4 generations
    RL hyperparameters are chosen without a reported sweep; they influence the GRPO pipeline results, and the beta in the loss formula is not specified numerically.
  • image-generation sampling parameters = guidance scale 1.5-4.5, 8-20 inference steps, resolution 768-1024
    The paper reports sensitivity to these settings; the best reported numbers use a particular configuration, and this tuning affects the comparisons.
assumptions (3)
  • domain assumption Every test metaphor can be decomposed into a single source, target, and intended meaning by a language model.
    The S-T-M representation is the backbone of the framework; novel, ambiguous, or culture-specific metaphors may not admit such a clean decomposition.
  • domain assumption CLIPScore and BERTScore are valid measures of semantic alignment between image and prompt and between textual components.
    These embeddings are known to have biases and limited compositional understanding; they are used both as rewards and as evaluation signals.
  • domain assumption The VLM judge's scalar scores for source, target, and meaning are stable, interpretable, and aligned with human judgment.
    No calibration or agreement analysis is reported, and the user study shows human raters prefer a system with lower VLM scores, contradicting this assumption.
invented entities (3)
  • Meaning Alignment (MA) metric
    purpose: Quantifies how well the intended meaning of a metaphor is perceived by a VLM in a generated image; used as a reward and as the headline evaluation metric.
    No evidence ties MA to human perception; it is computed by the same VLM family used for optimization, creating a circularity risk.
  • Metaphor Decomposition Score
    purpose: VLM-assessed quality of the S-T-M decomposition; used as a reward component and as an evaluation metric.
    It is a self-assessment by the same VLM family used to generate feedback; no external validation is provided.
  • S-T-M structured decomposition
    purpose: Intermediate representation that splits a metaphor into source, target, and meaning to guide prompt generation and refinement.
    The representation is grounded in Conceptual Metaphor Theory from the external literature, but as operationalized here it lacks a falsifiable handle outside the paper's own pipeline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation." pith.science (2026). https://pith.science/paper/P37EIKSR

@misc{pith2026250818569,
  author       = {Pith},
  title        = {Pith review of: The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P37EIKSR}},
  note         = {Machine review of arXiv:2508.18569}
}
read the original abstract

Visual metaphor generation is a challenging task that aims to generate an image given an input text metaphor. Inherently, it needs language understanding to bind a source concept with a target concept, in a way that preserves meaning while ensuring visual coherence. We propose a self-evaluating visual metaphor generation framework that focuses on metaphor alignment. Our self-evaluation approach combines existing metrics with our newly proposed metaphor decomposition score and a meaning alignment (MA) metric. Within this setup, we explore two novel approaches: a training-free pipeline that explicitly decomposes prompts into source-target-meaning (S-T-M) mapping for image synthesis, and a complementary training-based pipeline that improves alignment using our proposed self-evaluation reward schema, without any large-scale retraining. On the held-out test set, the training-free approach surpasses strong closed baselines (GPT-4o, Imagen) on decomposition, CLIP, and MA scores, with the training-based approach close behind. We evaluate our framework output using a user-facing study, and observed that participants preferred GPT-4o overall, while our training-free pipeline led open-source methods and edged Imagen on abstract metaphors. Our analyses show S-T-M prompting helps longer or more abstract metaphors, with closed models excelling on short, concrete cases; we also observe sensitivity to sampler settings. Overall, structured prompting and lightweight RL perform metaphor alignment well under modest compute, and remaining gaps to human preference appear driven by aesthetics and sampling.

Figures

Figures reproduced from arXiv: 2508.18569 by the authors.

Figure 1
Figure 1. Overview of our framework. Given a metaphorical text input, the pipeline decomposes it into source, target, intended meaning, and visual prompt using a large language model (LLM). A text-to-image model generates an initial image, which is analyzed by a vision-language model (VLM) to assess metaphor alignment. Feedback from the VLM guides iterative refinement in the training-free setting or reinforcement learning fin… view at source ↗
Figure 2
Figure 2. Our proposed Visual Metaphor generation architecture for both training-free and GRPO pipelines. For in-context learning (training-free), we remove the ‘TRL GRPO Trainer’ and provide all the feedback (CLIP, VLM analysis, and BERT scores) through a prompt for refined prompt generation from the LLM. For GRPO, however, the LLM undergoes LoRA fine-tuning with GRPO and the feedback is provided at every training step. pres… view at source ↗
Figure 3
Figure 3. Structure of the user study for evaluating metaphorical image generation. Participants were shown images generated by different methods for the same metaphorical prompt and asked to rate them on metaphor alignment, image quality, and creativity. Ratings were collected and analyzed to compare the performance of zero-shot baselines, training-free pipeline, and GRPO fine-tuning approach. Dataset images are added only f… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Best Model Overall and Model Perfor￾mance w.r.t Metaphor Length Results, summarized in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Model’s Performance w.r.t Metaphor Source and Target [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Performance by Length and Model Win Rates [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: User study instructions provided to the participants. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Simile Understanding in Text-to-Image Models: An Evaluation Framework

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A YOLO-based evaluation framework shows that text-to-image models consistently render the literal vehicle of a simile, a failure that CLIPScore and PickScore miss.

Reference graph

Works this paper leans on

64 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    A. R. Akula, B. Driscoll, P. Narayana, S. Changpinyo, Z. Jia, S. Damle, G. Pruthi, S. Basu, L. Guibas, W. T. Freeman, Y . Li, and V . Jampani. MetaCLUE: Towards Comprehensive Visual Metaphors Research . In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23201–23211, Los Alamitos, CA, USA, June 2023. IEEE Computer Society....

    arXiv 2023

  2. [2]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  3. [3]

    Balaji et al

    Y . Balaji et al. ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers,

    work page

  4. [4]

    Baldridge, J

    J. Baldridge, J. Bauer, M. Bhutani, N. Brichtova, A. Bunner, L. Castrejon, K. Chan, Y . Chen, S. Dieleman, Y . Du, et al. Imagen 3.arXiv preprint arXiv:2408.07009, 2024

    arXiv 2024

  5. [5]

    Black, M

    K. Black, M. Janner, Y . Du, I. Kostrikov, and S. Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301, 2023

    arXiv 2023

  6. [6]

    Chakrabarty, A

    T. Chakrabarty, A. Saakyan, O. Winn, A. Panagopoulou, Y . Yang, M. Apidianaki, and S. Mure- san. I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors, July 2023. URL http://arxiv.org/abs/2305.14724. arXiv:2305.14724 [cs]

    arXiv 2023

  7. [7]

    X. Chen, Z. Wu, X. Liu, Z. Pan, W. Liu, Z. Xie, X. Yu, and C. Ruan. Janus-pro: Uni- fied multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811, 2025

    arXiv 2025

  8. [8]

    Esser, S

    P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024

    2024

Show all 64 references
  1. [9]

    Y . Fan, O. Watkins, Y . Du, H. Liu, M. Ryu, C. Boutilier, P. Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:79858–79885, 2023

    2023

  2. [10]

    Forceville

    C. Forceville. Pictorial Metaphor in Advertising. Routledge, London, UK, 1996

    1996

  3. [11]

    R. W. Gibbs. The Cambridge Handbook of Metaphor and Thought . Cambridge University Press, Cambridge, UK, 2008

    2008

  4. [12]

    Hessel, A

    J. Hessel, A. Holtzman, M. Forbes, R. L. Bras, and Y . Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718, 2021

    2021 arXiv

  5. [13]

    Y . Hu, B. Liu, J. Kasai, Y . Wang, M. Ostendorf, R. Krishna, and N. A. Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering, 2023. URL https://arxiv.org/abs/2303.11897

    2023 arXiv

  6. [14]

    M. Johnson. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. The University of Chicago Press, Chicago, US, 1990

    1990

  7. [15]

    Kirstain, A

    Y . Kirstain, A. Polyak, U. Singer, S. Matiana, J. Penna, and O. Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation, 2023. URL https://arxiv.org/ abs/2305.01569

    2023 arXiv

  8. [16]

    O. Kramer. Conceptual Metaphor Theory as a Prompting Paradigm for Large Language Models, Feb. 2025. URL http://arxiv.org/abs/2502.01901. arXiv:2502.01901 [cs]

    2025 arXiv

  9. [17]

    Kövecses

    Z. Kövecses. Metaphor: A Practical Introduction. Oxford University Press, Oxford, UK, 2nd edition, 2010

    2010

  10. [18]

    Lakoff and M

    G. Lakoff and M. Johnson. Metaphors We Live By. University of Chicago Press, 1980. 9

    1980

  11. [19]

    Lakoff and M

    G. Lakoff and M. Johnson. Metaphors We Live By. University of Chicago Press, Chicago, US, 2nd edition, 2008

    2008

  12. [20]

    S. Lee, S. Kim, S. Park, G. Kim, and M. Seo. Prometheus-vision: Vision-language model as a judge for fine-grained evaluation. In Findings of the association for computational linguistics ACL 2024, pages 11286–11315, 2024

    2024

  13. [21]

    Z. Liu, C. Chen, W. Li, P. Qi, T. Pang, C. Du, W. S. Lee, and M. Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  14. [22]

    Introducing 4o image generation, 2025

    OpenAI. Introducing 4o image generation, 2025. URL https://openai.com/index/ introducing-4o-image-generation/

    2025

  15. [23]

    B. J. Phillips and E. F. McQuarrie. Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing Theory, 4(1-2):113–136, 2004. doi: 10.1177/1470593104044089

    2004 doi

  16. [24]

    Podell et al

    D. Podell et al. Sdxl: Improving latent diffusion models for high-resolution image synthesis,

  17. [25]

    Rajakumar Kalarani, P

    A. Rajakumar Kalarani, P. Bhattacharyya, and S. Shekhar. Unveiling the invisible: Captioning videos with metaphors. In Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 6306–6320, Miami, Florida, ...

    2024 doi

  18. [26]

    Ramesh, P

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    2022 arXiv

  19. [27]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022. doi: 10.1109/CVPR52688.2022. 01042

    2022

  20. [28]

    Saakyan, S

    A. Saakyan, S. Kulkarni, T. Chakrabarty, and S. Muresan. Understanding Figurative Meaning through Explainable Visual Entailment, Feb. 2025. URL http://arxiv.org/abs/2405. 01474. arXiv:2405.01474 [cs]

    2025 arXiv

  21. [29]

    Saharia et al

    C. Saharia et al. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  22. [30]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  23. [31]

    G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025

    2025 arXiv

  24. [32]

    J. Xu, X. Liu, Y . Wu, Y . Tong, Q. Li, M. Ding, J. Tang, and Y . Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation.Advances in Neural Information Processing Systems, 36:15903–15935, 2023

    2023

  25. [33]

    Z. Xue, J. Wu, Y . Gao, F. Kong, L. Zhu, M. Chen, Z. Liu, W. Liu, Q. Guo, W. Huang, et al. Dancegrpo: Unleashing grpo on visual generation. arXiv preprint arXiv:2505.07818, 2025

    2025 arXiv

  26. [34]

    Zhang, J

    L. Zhang, J. Liu, L. Jin, H. Wang, K. Wei, and G. Xu. GOME: Grounding-based metaphor binding with conceptual elaboration for figurative language illustration. In Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, editors,Proceedings of the 2024 Conference on Empirical Methods in Natu...

    2024 doi

  27. [35]

    ** Pe rc ei ve d Source ( S ’) **: What is the primary visual element in the image that r e p r e s e n t s ’{ s } ’?

  28. [36]

    ** Pe rc ei ve d Target ( T ’) **: What visual elements , if any , sy mb ol iz e or evoke the abstract concept of ’{ t } ’? If the image ONLY shows ’{ s } ’ without any clear visual link to ’{ t } ’ , state that the target is not r e p r e s e n t e d

  29. [37]

    ** Pe rc ei ve d Meaning ( M ’) **: What is the overall meaning the image conveys ? 12

  30. [38]

    ** S Presence Score **: How clearly is ’{ s } ’ depicted ? (0.0 for not present , 1.0 for clearly present )

  31. [39]

    A high score requires clear symbolic elements ( e

    ** T Presence Score **: How e f f e c t i v e l y does the image use the visuals of S to s ym bo li ze or evoke T ? A score of 0.1 should be given if the image only shows S without any m e t a p h o r i c a l c o n n e c t i o n to T . A high score requires clear symbolic elem...

  32. [40]

    "" C.2 VLM Prompt Without STM V L M _ P R O M P T _ W O _ S T M =

    ** Meaning Ali gn me nt Score **: Based on your analysis , how well does the Pe rc ei ved Meaning ( M ’) align with the original intended Meaning ( M ) ? Provide a score from 0.0 ( no al ig nme nt ) to 1.0 ( perfect a li gn me nt ) . Respond with a JSON object with keys ’ s_pr...

  33. [41]

    ** Visual D e s c r i p t i o n **: Briefly describe the main elements and style of the image

  34. [42]

    ** M e t a p h o r i c a l A li gn men t **: How well does the image capture the essence and meaning of the metaphor ?

  35. [43]

    "" You are an expert in l i n g u i s t i c s and s em ant ic s . Evaluate the fo ll ow in g d e c o m p o s i t i o n of a metaphor . Original Metaphor :

    ** Al ig nm en t Score **: Provide a single score from 0.0 ( no c o n n e c t i o n ) to 1.0 ( pe rf ect ly r e p r e s e n t s the metaphor ) for how well the image v i s u a l i z e s the metaphor . Respond with a JSON object with keys ’ v i s u a l _ d e s c r i p t i o n ’...

  36. [44]

    Identify the Source ( S ) , Target ( T ) , and Meaning ( M ) of the metaphor 14

  37. [45]

    "" # An example for one - shot learning e x a m p l e _ u s e r = ’ Analyze the fo ll owi ng metaphor :

    Generate a detailed visual prompt for an image g e n e r a t i o n model I n s t r u c t i o n s : - S should be the concrete concept used to explain T - T should be the abstract concept being ex pl ain ed - M should be the intended c o n n e c t i o n or i n t e r p r e t a t...

  38. [46]

    Better re pr es ent the original Source ( S ) , Target ( T ) , and Meaning ( M ) in the image

  39. [47]

    Address any w e a k n e s s e s in di ca ted by the scores ( e . g . , if S ’ is di ff er en t from S , or if M ’ m isa li gn s with M )

  40. [48]

    Provide * only * the new , revised image g e n e r a t i o n prompt as a single string

    If the d e c o m p o s i t i o n quality is low , focus on the most reliable aspects of the S , T , M br ea kd ow n . Provide * only * the new , revised image g e n e r a t i o n prompt as a single string . Do not include any other e x p l a n a t o r y text or labels , and th...

  41. [49]

    The conclusion supports the paper’s contributions and scope

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: Section 3 discusses related experiments and results claimed in the abstract and introduction in full detail. The concl...

  42. [50]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Section 3.3 and the Limitations paragraph discuss them in detail. Guidelines: • The answer NA means that the paper has no limitation while the answer...

  43. [51]

    Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] 17 Justification: The paper does not include theoretical results. Guidelines: • The answer NA means that the...

  44. [52]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and da...

  45. [53]

    Guidelines: • The answer NA means that paper does not include experiments requiring code

    Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material? 18 Answer: [No] Justification: The code will be publishe...

  46. [54]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: Appendix A and C include deta...

  47. [55]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [No] Justification: The statistical significance tests were not carried...

  48. [56]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments? Answer: [Yes] Justification: The paper includes the ...

  49. [57]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: The paper conforms with the Code of Ethics. Guidelines: • The answer NA means t...

  50. [58]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [NA] Justification: The paper mainly focuses on the creative abilities of generative AI models and is not related to the socie...

  51. [59]

    Guidelines: • The answer NA means that the paper poses no such risks

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper does ...

  52. [60]

    Guidelines: • The answer NA means that the paper does not use existing assets

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: Existing assets ...

  53. [61]

    Guidelines: • The answer NA means that the paper does not release new assets

    New assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: No new assets are introduced. Guidelines: • The answer NA means that the paper does not release new assets. • Researc...

  54. [62]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? A...

  55. [63]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

  56. [64]

    Answer: [Yes] Justification: The paper is about the creative abilities of generative AI models

    Declaration of LLM usage Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the ...

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.