Pith. sign in

REVIEW 4 major objections 6 minor 40 references

PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PoemTale Diffusion claims that splitting a poem by emotional shifts, refining each segment with an LLM loop, and sharing attention tokens across images sharply reduces the information lost when poems become pictures.

desk verdict Useful training-free pipeline for poem-to-image with a fine new dataset, but the quantitative evaluation is partly circular and needs an independent metric and significance testing before the information-loss claims are credible. read the letter →

arxiv 2507.13708 v2 pith:EUQ4QIVE submitted 2025-07-18 cs.CV

Sofia Jamil , Bollampalli Areen Reddy , Raghvendra Kumar , Sriparna Saha , Koustava Goswami , K.J. Joseph This is my paper
classification cs.CV
keywords poem-to-imagegenerationdiffusionmodelspromptrefinementconsistentself-attentionEPEsegmentationP4IdatasetLong-CLIPinformationretention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a training-free pipeline called PoemTale Diffusion for turning poems into sequences of images. Its thesis is that the failure of text-to-image models on poetry is an information-loss problem: a single image cannot hold a poem's layered meaning, and unrefined multi-image attempts fragment it. To fix this, the pipeline segments poems by shifts in entities and emotions, rewrites each segment into detailed image instructions through a self-terminating LLM refinement loop, and generates the segment images with cross-image consistent self-attention so characters stay recognizable. The authors claim this approach consistently outperforms both single-image and naive per-stanza baselines on quantitative scores and expert human ratings, and they release a 1111-poem dataset to support further work.

What carries the argument

The load-bearing object is the three-module pipeline: (1) the Entity Plus Emotion (EPE) segmentation algorithm, which cuts a poem at lines where named-entity tags or emotion labels change, detected with NLTK and a fine-tuned DistilRoBERTa; (2) the Multi-Stage Prompt Refinement (MSPR) loop, in which GPT-4o-mini rewrites each segment into image instructions, iterating until the Long-CLIP cosine similarity between poem and generated description saturates across three consecutive rounds; (3) Consistent Self-Attention, which replaces the standard self-attention in a diffusion U-Net so that key and value tokens are sampled from other segment images, tying identity across frames with no retraining. The refinement loop is what converts abstract verse into concrete instructions; the segmentation decides the granularity; the self-attention swap enforces visual continuity.

What would settle it

Take a set of poems held out from the P4I dataset, run the MSPR loop to convergence, and have poetry experts rate the final image instructions and images for semantic fidelity; if expert ratings do not rise with Long-CLIP saturation, or if random refinements score just as well on Long-CLIP, then the convergence criterion is not evidence of better poem understanding. A direct correlation test between Long-CLIP scores and expert semantic-alignment ratings would settle this.

Watch

Extended reading notes

Core claim

The central claim is that existing diffusion models can generate images that retain a poem's meaning if the poem is first restructured and the images are generated as a coherent set rather than one picture. The paper's discovery is that three interventions—entity-plus-emotion segmentation, a multi-stage prompt refinement loop that iterates until a Long-CLIP alignment score saturates, and consistent self-attention borrowed from StoryDiffusion—combine to produce the maximum information retention among the approaches compared. Playground V3 with the full pipeline yields the best results on the poem-alignment metric (Long-CLIP 0.5897, emotion CLIP 0.4249, BLIP 0.4333), and human experts rate the pipeline 3.9–4.1 on a 5-point scale versus 1.7–2.2 for the two baselines.

Load-bearing premise

The argument rests on Long-CLIP scores faithfully measuring how well an image captures a poem's meaning; if that score ignores poetic nuance, the refinement loop stops at the wrong point and the reported gains do not reflect true information retention.

Editorial extensions

If this is right

  • Applied to any diffusion model with self-attention, the pipeline improves poetic information retention without retraining; the paper demonstrates this on Janus, SDXL, and Playground V3.
  • Segment-by-segment generation preserves emotional shifts in a poem better than a single image, since each image is tied to one detected emotional and entity context.
  • Human expert ratings (semantic alignment 3.9, emotional resonance 4.1 on a 5-point scale) support that the pipeline's images are more aligned and emotionally resonant than baseline approaches.
  • Quantitative scores increase across prompt–image alignment (1.68↑), emotion depiction (0.21↑), and character consistency (0.57↑) compared with the baselines, per Section 7.1.
  • The release of the 1111-poem P4I dataset gives the community a benchmark for poem-to-image generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The pipeline treats a poem as a storyboard rather than a single illustration; a natural extension is to apply the same segmentation-and-refine loop to other long-form creative texts such as song lyrics, play scenes, or narrative prose.
  • Because the refinement loop's stopping rule is an automatic alignment score, it could be swapped for any improved long-text vision-language model; gains would likely track the semantic coverage of that model rather than poetry itself.
  • The claim that Long-CLIP is a valid proxy for poetic meaning is testable but untested; a validation study against expert ratings on poetic inputs would determine whether the reported information-retention gains are metric artifacts.
  • The consistent self-attention mechanism is applied across segment images within one poem; the approach could be extended to cross-poem character consistency, for example illustrating an entire poetry collection with a recurring protagonist.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes PoemTale Diffusion, a training-free pipeline for generating a sequence of images from a poem. The method has three components: an Entity Plus Emotion (EPE) segmentation module that splits poems at shifts in entities and emotions; a Multi-Stage Prompt Refinement (MSPR) loop that uses GPT-4o-mini to iteratively rewrite each segment into an image description until a Long-CLIP score saturates; and Consistent Self-Attention, borrowed from StoryDiffusion, to keep characters and style consistent across the generated images. The authors also introduce the P4I dataset of 1,111 annotated poems and report quantitative metrics (BLIP, Long-CLIP, CLIP) plus a human expert evaluation. The central claim is that the approach 'minimises information loss' and 'generates images with the maximum information retention' compared with using the whole poem as a single prompt or using human-annotated stanza segments without refinement.

Significance. If the claims were supported, this would be a useful and novel contribution: it tackles an underexplored task, is training-free and model-agnostic, introduces a sizable poetry dataset with annotation guidelines, and makes code and data publicly available. The human evaluation, despite its limitations, points in the direction of the paper's main claim, and the qualitative figures do show noticeable differences in coherence. However, the quantitative evidence as presented is not trustworthy, because the primary metric used to evaluate the method is the same metric used as the optimization target inside the prompt-refinement loop, and the other main automatic metric is partly tautological. The strengths of the paper are the task formulation, the dataset, and the engineering of a modular pipeline; the weakness is the absence of a valid, non-circular evaluation of information retention.

major comments (4)
  1. [§5.2, §6.2, Table 2] The central quantitative claim rests on a circular evaluation. Section 5.2 states that the MSPR loop iterates until the Long-CLIP score shows saturation, i.e., Long-CLIP is the objective that selects the final image descriptions. Section 6.2 then reports Long-CLIP as the primary measure of poem-image alignment, and Table 2 uses it to claim large gains (e.g., Playground V3: 0.5897 vs. 0.2193 for Approach 1). Because the generated images are conditioned on descriptions that were explicitly chosen to maximize Long-CLIP similarity to the poem, the reported gains may reflect optimization to this specific scorer rather than greater preservation of the poem's meaning. The paper provides no validation that Long-CLIP correlates with human judgment for poetic language, which is particularly important for poetry with layered or abstract meanings. I ask the authors to either validate Long-CLIP against human ratings on poetic inputs, or use an evaluation protocol in which the metric used for convergence is not the same as the metric used for the final comparison, for example by holding out a set of poems and descriptions that were not used in the refinement loop.
  2. [§6.2, Table 2] The BLIP score is reported as evidence in Table 2, but, as described in Section 6.2, it compares captions generated from an image with the 'original image instruction prompt.' For the PoemTale condition, that prompt is exactly the MSPR-refined description used to generate the image. The BLIP score therefore measures whether the image is self-consistent with its own generative prompt, not whether the image retains information from the poem. For the baseline approaches, the corresponding prompt is either the raw poem or a poem segment, so the comparison is not apples-to-apples. This metric should be replaced or supplemented with an image-caption or image-text similarity measure that does not use the generation prompt as the reference, or the authors should justify why high prompt-image consistency under a refined prompt implies high poem-image information retention.
  3. [Table 3, §4.3, §7.2] The human evaluation, which is the only non-circular evidence for the central claim, is reported too thinly to support it. Table 3 gives only average scores with no standard deviations, no per-expert breakdown, no inter-rater agreement statistic, and no significance tests. The reported sample sizes are inconsistent: Section 4.3 says 170 poems were selected (15% of the P4I dataset), while Section 7.2 and Table 3 refer to 150 poems. Section 4.3 describes a team of four experts (one poet, two professors, one doctoral researcher), while Section 7.2 says 'two renowned poet, one university professors, and a doctoral researcher' and then states that ratings were averaged over 'three experts.' Moreover, the experts were blinded only to the model and approach names, not to the prompts: the MSPR-refined prompts are longer and more descriptive than raw poem segments, so the raters could be responding to the informativeness of the prompt rather than to the information retention of the image-generation pipeline. I request a complete reporting of the human study: exact sample size, rater identities, agreement metrics, per-criterion distributions, and a control condition that matches prompt length and detail.
  4. [Table 2, §4.2, §5.1, §5.2, §5.3] Table 2 compares the full PoemTale pipeline against two baselines, but the design does not isolate the contribution of the three proposed modules. The 'Only With Poem Segments' baseline (Approach 2) uses the human-annotated segments described in Section 4.2, whereas PoemTale uses automatic EPE segmentation; the two conditions also differ in the prompt-refinement loop and in the use of Consistent Self-Attention. Consequently, the gains in Table 2 could be driven almost entirely by MSPR or by the attention mechanism, with EPE contributing little or nothing. The paper claims to evaluate 'each component' of the pipeline, but no ablation study is reported. I ask for ablations that hold all components fixed while toggling EPE vs. human segments, MSPR on/off, and Consistent Self-Attention on/off, so that the effect of each module can be assessed.
minor comments (6)
  1. [§2] The text cites 'DreamBooth [8]', but reference [8] is actually Gal et al., 'An image is worth one word: Personalizing text-to-image generation using textual inversion.' Please correct the citation or the reference entry.
  2. [§4.3, §7.2, Table 3] The number of poems in the human evaluation is inconsistent: Section 4.3 states 170 poems, while Section 7.2 and Table 3 state 150 poems. Please unify these numbers and clarify whether the same sample was used in both analyses.
  3. [§7.1] The sentence 'We observe an increasing trend in CLIP scores across the following aspects: emotion depiction (0.21↑), prompt alignment with images (1.68↑), character consistency across images (0.57↑), and resemblance to the reference character in the poem (0.82↑)' does not map transparently to the numbers in Table 2, and no statistical significance is reported. Please specify how these deltas are computed and add confidence intervals or significance tests.
  4. [§5.2] The MSPR loop has a free parameter, the convergence threshold for Long-CLIP saturation, and the paper reports convergence after '4–5 iterations' on a small subset but does not describe how the threshold was chosen or how sensitive the final results are to it. A sensitivity analysis would improve reproducibility.
  5. [§5.1] The EPE algorithm is described only at a high level; details such as how emotion predictions from DistilRoBERTa are thresholded to detect a 'shift,' how NER tag changes are combined with emotion changes, and how multi-line stanzas are handled are missing. These details are needed for replication.
  6. [Abstract, §1] There are several grammatical and typographical issues, including 'a multi stage prompt refinement loop into Language Models' in the abstract and inconsistent hyphenation of 'multi-stage'/'multistage.' A careful proofreading pass is recommended.

Circularity Check

2 steps flagged · score 6.0 of 10

The MSPR loop's Long-CLIP saturation criterion is the same Long-CLIP metric reported as the main quantitative outcome, so the headline 'information retention' gains are partly a construction artifact; BLIP is likewise evaluated against the very prompt that generated the image.

  1. fitted input called prediction [Section 5.2 (MSPR loop) and Section 6.2 / Table 2 (evaluation)]
    "The description is evaluated using the Long CLIP [36] score. ... The loop terminates when the Long CLIP score shows saturation, indicating that there is no significant improvement across three consecutive iterations, showing convergence."

    In Section 5.2, Long-CLIP is the signal used to accept and terminate the refined prompt: the loop iterates until the Long-CLIP score saturates. In Section 6.2 and Table 2, the same Long-CLIP model is then used to measure 'cosine similarity between the poem and the generated image' and is reported as the primary evidence of information retention. Because the generated image is conditioned on the Long-CLIP-selected prompt, the reported Long-CLIP gains (e.g., Playground V3: 0.5897 vs 0.2193) are substantially a consequence of optimizing the prompt against the evaluation scorer rather than independent evidence that the poem's meaning was preserved. This is optimization-target-as-evaluator circularity.

  2. other [Section 6.2 (BLIP Score) and Table 2]
    "To evaluate the alignment between the generated image and its corresponding prompt, we utilized the BLIP [15] Score, where captions were generated for images, and their similarity to the original image instruction prompt was computed."

    The 'original image instruction prompt' is precisely the prompt from which the generated image was produced. Comparing a caption of the image back to that prompt measures prompt-to-image self-consistency, not poem-to-image information retention. Any image that follows its prompt will score well regardless of whether the prompt captures the poem's layered meaning, so the BLIP column in Table 2 is, by construction, a measure of how well the image can be re-described by its own input. It therefore cannot support the paper's central claim that information from the poem, rather than from the refined prompt, is retained.

full rationale

The central quantitative claim that PoemTale Diffusion 'minimises information loss' rests on Table 2, whose Long-CLIP column is the same metric used as the convergence criterion inside the MSPR loop. The loop selects a refined prompt by iterating until Long-CLIP saturates, and the reported evaluation then measures Long-CLIP between the poem and images generated from that selected prompt. This is a genuine fitted-input-called-prediction pattern: the evaluation score is partly forced by the selection procedure. The BLIP metric is similarly self-referential because it compares image captions to the exact prompt that generated the image. The human evaluation (Table 3) provides some independent evidence, but it covers only 150 poems, reports no inter-rater agreement or variance, and does not control for the confound that refined prompts are longer and more descriptive. No load-bearing self-citation chain was found: the citation to the authors' prior COLING paper [12] supplies only the Stage 1 prompt-design starting point and is not the argument that forces the result. The EPE segmentation and Consistent Self-Attention components are not themselves circular, but the headline 'maximum information retention' claim is substantially supported by metrics that the method itself optimizes or defines. Score 6 reflects this partial, construction-level circularity rather than a fully tautological derivation.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim depends on several unvalidated assumptions about the reliability of CLIP-family metrics for poetic language, the accuracy of an off-the-shelf emotion classifier for poetry, and the sufficiency of NER tags for detecting meaningful poem segments. These are modeling assumptions, not fitted parameters, but any failure would weaken the pipeline's claimed benefits.

free parameters (1)
  • MSPR convergence threshold = No significant improvement over 3 consecutive iterations; optimal iterations observed as 4-5
    The loop termination criterion in Section 5.2 is a hand-chosen heuristic. The 'significant improvement' threshold is not quantified, and the 4-5 iteration optimum was determined from a small subset with human feedback, not by a formal procedure.
assumptions (4)
  • domain assumption Long-CLIP text-image similarity reliably measures poem-to-image semantic alignment
    Used both for prompt selection in MSPR (Section 5.2) and for evaluation (Section 6.2). No validation is provided that Long-CLIP correlates with human judgments for poetic language.
  • domain assumption The fine-tuned DistilRoBERTa emotion classifier correctly identifies emotions in poetic text
    EPE segmentation (Section 5.1) relies on this classifier to detect emotional shifts. Off-the-shelf emotion models are typically trained on non-poetic text, so accuracy on figurative poetry is unverified.
  • domain assumption NLTK NER tag changes indicate entity shifts relevant for poem segmentation
    EPE uses NLTK NER tags (Section 5.1) as boundaries. NER may miss poetic entities or produce spurious changes, which would affect segment quality.
  • domain assumption CLIP-family scores correlate with perceived information retention in generated images
    The quantitative evaluation uses CLIP, BLIP, and Long-CLIP scores (Section 6.2). The paper itself acknowledges these metrics may not fully capture image quality (Section 7.2), yet uses them as primary evidence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement." pith.science (2026). https://pith.science/paper/EUQ4QIVE

@misc{pith2026250713708,
  author       = {Pith},
  title        = {Pith review of: PoemTale Diffusion: Minimising Information Loss in Poem to Image Generation with Multi-Stage Prompt Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUQ4QIVE}},
  note         = {Machine review of arXiv:2507.13708}
}
read the original abstract

Recent advancements in text-to-image diffusion models have achieved remarkable success in generating realistic and diverse visual content. A critical factor in this process is the model's ability to accurately interpret textual prompts. However, these models often struggle with creative expressions, particularly those involving complex, abstract, or highly descriptive language. In this work, we introduce a novel training-free approach tailored to improve image generation for a unique form of creative language: poetic verse, which frequently features layered, abstract, and dual meanings. Our proposed PoemTale Diffusion approach aims to minimise the information that is lost during poetic text-to-image conversion by integrating a multi stage prompt refinement loop into Language Models to enhance the interpretability of poetic texts. To support this, we adapt existing state-of-the-art diffusion models by modifying their self-attention mechanisms with a consistent self-attention technique to generate multiple consistent images, which are then collectively used to convey the poem's meaning. Moreover, to encourage research in the field of poetry, we introduce the P4I (PoemForImage) dataset, consisting of 1111 poems sourced from multiple online and offline resources. We engaged a panel of poetry experts for qualitative assessments. The results from both human and quantitative evaluations validate the efficacy of our method and contribute a novel perspective to poem-to-image generation with enhanced information capture in the generated images.

Figures

Figures reproduced from arXiv: 2507.13708 by the authors.

Figure 1
Figure 1. Workflow of our proposed PoemTale Diffusion approach 5.1 Segmentation Module To effectively break down complex poetic verses into smaller, inde￾pendent units, we introduce an Entity Plus Emotion (EPE) approach to detect segment boundaries in a poem. Our approach incrementally processes each line of the poem, assigning named entity recognition (NER) tags and emotions to each line. When there is a change in NER tags o… view at source ↗
Figure 2
Figure 2. Working of Multi Stage Prompt Refinement Approach 5.3 Poem to Image Generation using Consistent Self-Attention After we have detailed image instructions for all segments of the poem, our final step involves generating images for multiple segments (if present) of a poem using a diffusion model. To maintain coherence across multiple images, the main character or central theme of the poem should remain visually consist… view at source ↗
Figure 3
Figure 3. Qualitative Comparison of Different Image Generation Models Integrated with PoemTale Diffusion approach. We recommend that the reader zoom in for better readability. 7 Results Analysis and Discussion 7.1 Quantitative Evaluation The results detailed in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Results of various approaches in poem to image generation corresponding to various segments of the poem (shown in the grey boxes). We recommend that the reader zoom in for better readability [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Results of various approaches in poem to image generation corresponding to various segments of the poem (shown in the grey boxes). We recommend that the reader zoom in for better readability [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Results of various approaches in poem to image generation corresponding to various segments of the poem (shown in the grey boxes). We recommend that the reader zoom in for better readability. eliminate potential bias, the experts were not informed of the names of the m…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 32 canonical work pages

  1. [1]

    Agarwal and K

    R. Agarwal and K. Kann. Acrostic poem generation. In B. Web- ber, T. Cohn, Y. He, and Y. Liu, editors,Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1230–1240, Online, Nov. 2020. Association for Com- putational Linguistics. doi: 10.18653/v1/2020.emnlp-main.94. URL https://aclanthology.org/2020.emnlp-main.94

    work page doi:10.18653/v1/2020.emnlp-main.94 2020

  2. [2]

    G.AstlinPereiraandM.Hussain.Areviewoftransformer-basedmodels forcomputervisiontasks:Capturingglobalcontextandspatialrelation- ships.arXiv e-prints, pages arXiv–2408, 2024

    work page 2024

  3. [3]

    Betker, G

    J. Betker, G. Goh, L. Jing, T. Brooks, J. Wang, L. Li, L. Ouyang, J.Zhuang,J.Lee,Y.Guo,etal. Improvingimagegenerationwithbetter captions.Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023

    work page 2023

  4. [4]

    Chakrabarty, V

    T. Chakrabarty, V. Padmakumar, and H. He. Help me write a poem - instruction tuning as a vehicle for collaborative poetry writing. In Y. Goldberg, Z. Kozareva, and Y. Zhang, editors,Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process- ing, pages 6848–6863, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computat...

    doi:10.18653/v1/2022 2022

  5. [5]

    Cheng, C.-C

    W.-F. Cheng, C.-C. Wu, R. Song, J. Fu, X. Xie, and J.-Y. Jie. Image inspired poetry generation in xiaoice.” eprint arxiv (august 9, 2018)

    work page 2018

  6. [6]

    Cherti, R

    M. Cherti, R. Beaumont, R. Wightman, M. Wortsman, G. Ilharco, C.Gordon,C.Schuhmann,L.Schmidt,andJ.Jitsev. Reproduciblescal- inglawsforcontrastivelanguage-imagelearning. InProceedingsofthe IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2818–2829, 2023

    work page 2023

  7. [7]

    M.Ding,Z.Yang,W.Hong,W.Zheng,C.Zhou,D.Yin,J.Lin,X.Zou, Z. Shao, H. Yang, et al. Cogview: Mastering text-to-image generation via transformers.Advances in neural information processing systems, 34:19822–19835, 2021

    work page 2021

  8. [8]

    Cohen-or

    R.Gal,Y.Alaluf,Y.Atzmon,O.Patashnik,A.H.Bermano,G.Chechik, and D. Cohen-or. An image is worth one word: Personalizing text- to-image generation using textual inversion. InThe Eleventh Inter- national Conference on Learning Representations, 2023. URL https: //openreview.net/forum?id=NAQvF08TcyG

    work page 2023

Show all 40 references
  1. [9]

    Ghazvininejad, X

    M. Ghazvininejad, X. Shi, Y. Choi, and K. Knight. Generating topical poetry. InProceedings of the 2016 conference on empirical methods in natural language processing, pages 1183–1191, 2016

    2016

  2. [10]

    Hartmann

    J. Hartmann. Emotion english distilroberta-base. https://huggingface. co/j-hartmann/emotion-english-distilroberta-base/, 2022

    2022

  3. [11]

    T. M. Huynh and Q. L. Bao. Vietnamese poem generation & the prospect of cross-language poem-to-poem translation.arXiv preprint arXiv:2401.01078, 2024

    2024 arXiv

  4. [12]

    Jamil, B

    S. Jamil, B. A. Reddy, R. Kumar, S. Saha, J. K. J, and K. Goswami. Poetry in pixels: Prompt tuning for poem image generation via diffu- sion models. In O. Rambow, L. Wanner, M. Apidianaki, H. Al-Khalifa, B.D.Eugenio,andS.Schockaert,editors,Proceedingsofthe31stInter- national C...

    2025

  5. [13]

    Kumari, B

    N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J.-Y. Zhu. Multi- conceptcustomizationoftext-to-imagediffusion. InProceedingsofthe IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1931–1941, 2023

    1931

  6. [14]

    Paint4poem: A dataset for artistic visualization of classical chinese po- ems.arXiv preprint arXiv:2109.11682, 2021

    D.Li,S.Wang,J.Zou,C.Tian,E.Nieuwburg,F.Sun,andE.Kanoulas. Paint4poem: A dataset for artistic visualization of classical chinese po- ems.arXiv preprint arXiv:2109.11682, 2021

    2021 arXiv

  7. [15]

    J. Li, D. Li, C. Xiong, and S. Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,

  8. [16]

    L. Lian, B. Li, A. Yala, and T. Darrell. LLM-grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models.Transactions on Machine Learning Re- search, 2024. ISSN 2835-8856. URL https://openreview.net/forum? id=hFALpTb4fR. Feat...

    2024

  9. [17]

    Beyondnarrativedescrip- tion: Generating poetry from images by multi-adversarial training

    B.Liu,J.Fu,M.P.Kato,andM.Yoshikawa. Beyondnarrativedescrip- tion: Generating poetry from images by multi-adversarial training. In Proceedings of the 26th ACM international conference on Multimedia, pages 783–791, 2018

    2018

  10. [18]

    Souza, S

    B.Liu,E.Akhgari,A.Visheratin,A.Kamko,L.Xu,S.Shrirao,C.Lam- bert, J. Souza, S. Doshi, and D. Li. Playground v3: Improving text-to- image alignment with deep-fusion large language models, 2024. URL https://arxiv.org/abs/2409.10695

    2024 arXiv

  11. [19]

    C. Liu, H. Wu, Y. Zhong, X. Zhang, Y. Wang, and W. Xie. Intelligent grimm-open-ended visual storytelling via latent diffusion models. In ProceedingsoftheIEEE/CVFConferenceonComputerVisionandPat- tern Recognition, pages 6190–6200, 2024

    2024

  12. [20]

    Z. Liu, Z. Fu, J. Cao, G. De Melo, Y.-C. Tam, C. Niu, and J. Zhou. Rhetoricallycontrolledencoder-decoderformodernchinesepoetrygen- eration. InProceedings of the 57th annual meeting of the Association for Computational Linguistics, pages 1992–2001, 2019

    1992

  13. [21]

    K.-L. Lo, R. Ariss, and P. Kurz. Gpoet-2: A gpt-2 based poem genera- tor.ArXiv,abs/2205.08847,2022. URLhttps://api.semanticscholar.org/ CorpusID:248863108

    2022 arXiv

  14. [22]

    Podell, Z

    D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. InThe Twelfth International Con- ference on Learning Representations, 2024. URL https://openreview. net...

    2024

  15. [23]

    Sutskever

    A.Ramesh,M.Pavlov,G.Goh,S.Gray,C.Voss,A.Radford,M.Chen, and I. Sutskever. Zero-shot text-to-image generation. InInternational conference on machine learning, pages 8821–8831. Pmlr, 2021

    2021

  16. [24]

    Ramesh, P

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with clip latents. arxiv 2022.arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  17. [25]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High- resolutionimagesynthesiswithlatentdiffusionmodels. InProceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pages 10684–10695, 2022

    2022

  18. [26]

    N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500–22510, 2023

    2023

  19. [27]

    Saharia, W

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K.Ghasemipour,R.GontijoLopes,B.KaragolAyan,T.Salimans,etal. Photorealistic text-to-image diffusion models with deep language un- derstanding.Advances in neural information processing systems, 35: 36479–36494, 2022

    2022

  20. [28]

    Sauer, D

    A. Sauer, D. Lorenz, A. Blattmann, and R. Rombach. Adversarial dif- fusion distillation. InEuropean Conference on Computer Vision, 2023. URL https://api.semanticscholar.org/CorpusID:265466173

    2023

  21. [29]

    Deep unsupervised learning using nonequilibrium thermodynamics

    J.Sohl-Dickstein,E.Weiss,N.Maheswaranathan,andS.Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. InIn- ternational conference on machine learning, pages 2256–2265. PMLR, 2015

    2015

  22. [30]

    Understandinginterobserveragreement: the kappa statistic.Fam med, 37(5):360–363, 2005

    A.J.Viera,J.M.Garrett,etal. Understandinginterobserveragreement: the kappa statistic.Fam med, 37(5):360–363, 2005

    2005

  23. [31]

    C. Wu, X. Chen, Z. Wu, Y. Ma, X. Liu, Z. Pan, W. Liu, Z. Xie, X. Yu, C. Ruan, and P. Luo. Janus: Decoupling visual encoding for unified multimodalunderstandingandgeneration,2024. URLhttps://arxiv.org/ abs/2410.13848

    2024 arXiv

  24. [32]

    R. Yan, H. Jiang, M. Lapata, S.-D. Lin, X. Lv, and X. Li. I, poet: auto- matic chinese poetry composition through a generative summarization framework under constrained optimization. InIJCAI 2013, Proceed- ingsofthe23rdInternationalJointConferenceonArtificialIntelligence, page...

    2013

  25. [33]

    H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang. Ip-adapter: Text com- patible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721, 2023

    2023 arXiv

  26. [34]

    H.-X. Yu, H. Duan, J. Hur, K. Sargent, M. Rubinstein, W. T. Freeman, F. Cole, D. Sun, N. Snavely, J. Wu, et al. Wonderjourney: Going from anywhere to everywhere. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6658–6667, 2024

    2024

  27. [35]

    S. Yuan, A. Dai, Z. Yan, R. Liu, M. Chen, B. Chen, Z. Qiu, and X. He. Learningtogeneratepoeticchineselandscapepaintingwithcalligraphy,

  28. [36]

    Long-clip:Unlock- ing the long-text capability of clip.arXiv preprint arXiv:2403.15378, 2024

    B.Zhang,P.Zhang,X.Dong,Y.Zang,andJ.Wang. Long-clip:Unlock- ing the long-text capability of clip.arXiv preprint arXiv:2403.15378, 2024

    2024 arXiv

  29. [37]

    Chinesepoetrygenerationwithrecurrentneu- ralnetworks

    X.ZhangandM.Lapata. Chinesepoetrygenerationwithrecurrentneu- ralnetworks. InProceedingsofthe2014conferenceonempiricalmeth- ods in natural language processing (EMNLP), pages 670–680, 2014

    2014

  30. [38]

    Y. Zhou, D. Zhou, M.-M. Cheng, J. Feng, and Q. Hou. Storydiffusion: Consistent self-attention for long-range image and video generation. In TheThirty-eighthAnnualConferenceonNeuralInformationProcessing Systems, 2024. URL https://openreview.net/forum?id=VFqzxhINFU

    2024

  31. [2022]

    URL https://arxiv.org/abs/2201.12086

    arXiv

  32. [2023]

    URL https://arxiv.org/abs/2305.04719

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.