Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

CharGen: High Accurate Character-Level Visual Text Generation Model with MultiModal Encoder

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CharGen pairs character-by-character glyph encoding with a destylization perceptual loss to reduce stroke and spelling errors in diffusion-generated visual text.

desk verdict CharGen has a plausible character-level encoder and a helpful ablation, but the headline gains over AnyText are confounded by swapping in Long-CLIP, so the central attribution is not yet established. read the letter →

arxiv 2412.17225 v1 pith:BQVAF53Q submitted 2024-12-23 cs.CV

Lichen Ma , Tiezhu Yue , Pei Fu , Yujie Zhong , Kai Zhou , Xiaoming Wei , Jie Hu This is my paper · ORCID
classification cs.CV
keywords visualtextgenerationcharacter-levelmultimodalencoderglyphimagesperceptuallossOCRdestylizationmodelingChineserenderingdiffusionmodelsControlNet
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CharGen, a visual text generation and editing model built on ControlNet, and argues that two changes make diffusion-based text rendering markedly more accurate. First, instead of encoding a whole line of text as one visual token, it renders each character alone into a glyph image, encodes those glyphs character by character, and fuses the visual features with per-character text embeddings before injecting them into the diffusion U-Net. Second, it replaces OCR-style perceptual losses with a multi-scale loss computed from the features of a pretrained destylization encoder, which is designed to strip away font style and expose stroke-level glyph structure. On the AnyText-benchmark, the method reports English sentence accuracy of 80.96% (up 8.8 points over AnyText-v1.1) and Chinese accuracy of 74.99% (up 5.5 points), with gains on MARIO-Eval as well. The paper's central claim is that the character-level multimodal encoding and the destylization loss, not the extra training data or model capacity, are what drive these accuracy gains.

What carries the argument

The two load-bearing components are the character-level multimodal encoder and the CharGen loss. The encoder renders each input character individually (80x80 pixels, Arial Unicode font), encodes the glyph with PP-OCRv3 to obtain visual embeddings, and encodes the same character with Long-CLIP to obtain text embeddings; two decoupled linear mappers fuse the two modalities into a per-character multimodal embedding that is concatenated with the prompt embedding and passed to a transformer text encoder before cross-attending to the U-Net. The CharGen loss (Eq. 2) extracts multi-scale features from the original and predicted images with the frozen ODM encoder—a pretrained text-image alignment model whose destylization objective restores clean glyphs while ignoring style, color, and texture—and penalizes the averaged L2 distance over spatial positions at each scale. The first component supplies the missing stroke-level conditions; the second supplies supervision that is sensitive to glyph shape errors but invariant to the font used.

What would settle it

Render the same correct Chinese character in two different fonts and render two characters that differ by one missing stroke (e.g., 词 versus 词 with the left component removed), then measure the ODM feature distance between each pair; if the style pair distance is not clearly smaller than the stroke-deletion distance, the loss cannot be providing the stroke-level, style-invariant supervision the paper claims.

Watch

Extended reading notes

Core claim

CharGen's central discovery is that visual text generation fails mainly because conventional pipelines compress glyph information at the wrong granularity and supervise it in the wrong feature space. The authors show that a character-level multimodal encoder—encoding each rendered character glyph independently with a visual encoder and combining it with the character's text embedding through learned mappers—provides the diffusion backbone with the fine-grained cross-modal conditions needed to render complex words and multi-stroke characters, and that a multi-scale perceptual loss based on an OCR destylization encoder (ODM) supervises glyph shape while remaining insensitive to font style. The paper reports that this combination outperforms prior methods on both Chinese and English public benchmarks, resolving error categories such as added/dropped strokes, repeated glyphs, merged glyphs, and misshapen characters that earlier methods leave unsolved.

Load-bearing premise

The accuracy gains attributed to the CharGen loss depend on the assumption that the ODM encoder's destylized features are sensitive to fine-grained glyph shape errors (added or dropped strokes, misshapen characters) while remaining insensitive to font style; if those features miss stroke-level errors or encode style, the reported 6.7% ablation gain from the loss would not be a true glyph-accuracy gain.

Editorial extensions

If this is right

  • Existing ControlNet-based visual text generation systems can adopt the character-level multimodal encoder and CharGen loss as plug-in components, since CharGen's design does not change the base diffusion model.
  • The method should transfer to other scripts with complex glyph structures, such as Arabic, Devanagari, and Korean, because the encoder operates at the character level rather than the language-model token level.
  • Because the CharGen loss supervises in a destylized feature space, generated text style is intended to follow the scene description rather than the training-render font, improving consistency between text and background style.
  • The benchmark results imply that character-level granularity plus style-invariant perceptual supervision is the key to eliminating added strokes, dropped strokes, and merged glyphs, not larger training data alone, since CharGen trains on 1.39 million English images yet beats models trained on 10 million.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that a pure-glyph variant of CharGen, dropping the per-character text branch, may be sufficient for glyph-heavy prompts, since the ablation shows visual embedding contributes far more than text embedding (Exp. 4 vs Exp. 5).
  • The destylization loss could find use outside generation, as a training signal for text-shape faithfulness in OCR-guided inpainting, video text rendering, or font synthesis systems.
  • A testable extension the paper does not run: feed the ODM encoder two images of the same word in different fonts and confirm that the CharGen loss stays near zero, and feed images with one stroke altered and confirm the loss increases; that would directly validate the style-invariance assumption.
  • A practical deployment consideration is that the per-character visual encoder and the ODM loss add inference-time complexity; the paper does not measure the added computational cost per generation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CharGen, a ControlNet-based visual text generation and editing model built on SD1.5. Its two main contributions are a character-level multimodal encoder that fuses per-character glyph visual features with per-character text embeddings, and a multi-scale perceptual loss (CharGen loss) computed from an ODM encoder. The model is trained on AnyWord-3M and evaluated on AnyText-benchmark and MARIO-Eval, reporting large gains over AnyText-v1.1 and Glyph-ByT5, including 8.8% English and 5.5% Chinese Sentence Accuracy improvements on AnyText-benchmark. Ablations on a 200K subset attribute the gains to the character-level embedding, the multimodal fusion, and the CharGen loss.

Significance. If the causal attribution is correct, CharGen is a practically useful contribution to visual text generation: it combines a sensible character-level visual-text fusion with a perceptual loss targeted at glyph shapes, and the main comparisons use a controlled inference protocol (identical DDIM sampler, CFG, seed, and prompts) plus external OCR metrics. The paper also attempts a fine-grained error taxonomy (Table 1) that is useful for the community. However, the reported gains are not yet convincingly attributed to the proposed modules because the text encoder was changed relative to the main baseline, the ablations lack variance estimates, and several recent state-of-the-art baselines are missing. These issues prevent the paper from currently supporting its central claim and SOTA statements.

major comments (4)
  1. [Section 3.2, Table 4] The central attribution claim is not established because of a text-encoder confound. All CharGen experiments, including every row of Table 4, use Long-CLIP as the text encoder, while the primary baseline AnyText-v1.1 uses standard CLIP with a 77-token limit. Long-CLIP extends the token limit to 248 and is designed to improve text understanding, so it can independently improve OCR-based metrics, especially for long or multi-text prompts. Since Table 4 has no row that replaces Long-CLIP with standard CLIP, or equivalently re-runs AnyText with Long-CLIP, the abstract's claim that the proposed character-level encoder and CharGen loss cause the 8.8% English and 5.5% Chinese gains is not supported. Please add a control that isolates the text-encoder choice.
  2. [Section 4.4, Table 4] The ablation experiments are performed once on a 200K subset with no reported training seeds or variance estimates. Several of the component differences used to support the design, such as 0.535 vs. 0.572 (+3.7%) for multimodal fusion and 0.620 vs. 0.639 (+1.9%) for the full model, are small enough that single-run noise could affect the conclusions. Please report results over multiple seeds, or at least provide confidence intervals, so the component contributions can be assessed reliably.
  3. [Section 3.3, Eq. (2)] The CharGen perceptual loss assumes that the pretrained ODM encoder's destylized features are sensitive to fine-grained glyph errors (added/dropped strokes, misshapen characters) while being invariant to font style. No supporting analysis of ODM features is presented, such as a perturbation study or a visualization showing that ODM feature distances increase for stroke-level errors but not for style changes. Without such evidence, the 6.7% ablation gain from this loss (Exp. 5 vs. 7 in Table 4) could in principle reflect style alignment rather than glyph accuracy, and the use of an encoder co-authored by two CharGen authors makes this point particularly important to address.
  4. [Tables 2 and 3] The claim of state-of-the-art performance is weakened by the omission of several recent baselines. In particular, Glyph-ByT5-v2, a 2024 method directly aimed at accurate multilingual visual text rendering, is cited in the paper but not included in either comparison table, and Glyph-ByT5 is not evaluated on AnyText-benchmark. To support the 'state-of-the-art' and 'recent methods' statements, please include these baselines or explicitly justify their exclusion.
minor comments (5)
  1. [Abstract] The phrase 'High Accurate' should be 'High-Accuracy' for grammatical correctness.
  2. [Section 3.3, Eq. (2)] The summation notation 'kX' and 'h,wX' appears to be a formatting artifact; please rewrite the equation with explicit sums over layers and spatial positions.
  3. [Figure 2] The text 'Multi-model Encoder' is inconsistent with the term 'multimodal' used elsewhere in the paper; please unify the terminology.
  4. [References] References [34] and [35] both cite the same ControlNet paper; please merge or distinguish them appropriately.
  5. [Section 4.4] In the bullet listing the CharGen perceptual loss comparisons, the phrase 'Exp.5 and Exp.7 highlight' should be 'Exp. 5 and Exp. 7 highlight' for consistency with the table notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CharGen's claims rest on external OCR benchmarks and controlled ablations; the ODM citation with overlapping authors is independent published evidence, and the Long-CLIP comparison issue is a confound rather than a circular reduction.

full rationale

The derivation chain is self-contained. The character-level multimodal encoder (Sec. 3.2) and the CharGen perceptual loss (Sec. 3.3, Eqs. 1-2) are evaluated on public benchmarks (AnyText-benchmark, MARIO-Eval) with an external OCR recognizer (DuguangOCR, Sec. 4.1), not by the proposed loss itself. The ablations in Table 4 isolate each proposed component, and the loss weight lambda is selected via the ablation study rather than fitted to the final test metric. The ODM encoder [7] shares two authors with CharGen, but it is a published CVPR 2024 model with its own external benchmarks, so the citation is independent support rather than a self-referential premise. The adoption of Long-CLIP (Sec. 3.2) is a potential confound because no ablation row replaces it with standard CLIP, so the improvement over AnyText may be partly due to Long-CLIP; however, this is an attribution and correctness risk, not circularity, since the reported numbers are measured empirically and no equation reduces its output to its input by construction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four domain assumptions, each inherited from pretrained models or datasets used without in-paper verification: Long-CLIP's compatibility, PP-OCRv3's glyph discriminability, ODM's destylization property, and AnyWord-3M's representativeness. Additionally, the loss weight lambda is fitted to a validation subset. No new physical entities or free parameters beyond lambda are introduced.

free parameters (1)
  • lambda (CharGen loss weight) = 0.01
    Chosen via ablation on a 200K subset (Exp. 9-11 in Table 4); the reported accuracy gains depend on this tuned value.
assumptions (4)
  • domain assumption Long-CLIP maintains CLIP's latent space and zero-shot generalization, enabling seamless integration into SD1.5 cross-attention.
    Section 3.2 relies on this to combine fused character embeddings with the prompt embeddings.
  • domain assumption PP-OCRv3 provides character-level visual features that are discriminative enough to distinguish visually similar glyphs.
    Section 3.2 uses PP-OCRv3 as the per-character visual encoder; its discriminative power is assumed, not demonstrated here.
  • domain assumption ODM encoder features are destylized: sensitive to glyph shape and invariant to font style, suitable for perceptual loss.
    Section 3.3 states this; no empirical evidence in this paper verifies the style-invariance or stroke-sensitivity.
  • domain assumption AnyWord-3M is representative enough to train a model that transfers to AnyText-benchmark and MARIO-Eval test sets.
    Section 4.2.1; the model is trained on AnyWord-3M and evaluated on these benchmarks without leakage checks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CharGen: High Accurate Character-Level Visual Text Generation Model with MultiModal Encoder." pith.science (2026). https://pith.science/paper/BQVAF53Q

@misc{pith2026241217225,
  author       = {Pith},
  title        = {Pith review of: CharGen: High Accurate Character-Level Visual Text Generation Model with MultiModal Encoder},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BQVAF53Q}},
  note         = {Machine review of arXiv:2412.17225}
}
read the original abstract

Recently, significant advancements have been made in diffusion-based visual text generation models. Although the effectiveness of these methods in visual text rendering is rapidly improving, they still encounter challenges such as inaccurate characters and strokes when rendering complex visual text. In this paper, we propose CharGen, a highly accurate character-level visual text generation and editing model. Specifically, CharGen employs a character-level multimodal encoder that not only extracts character-level text embeddings but also encodes glyph images character by character. This enables it to capture fine-grained cross-modality features more effectively. Additionally, we introduce a new perceptual loss in CharGen to enhance character shape supervision and address the issue of inaccurate strokes in generated text. It is worth mentioning that CharGen can be integrated into existing diffusion models to generate visual text with high accuracy. CharGen significantly improves text rendering accuracy, outperforming recent methods in public benchmarks such as AnyText-benchmark and MARIO-Eval, with improvements of more than 8% and 6%, respectively. Notably, CharGen achieved a 5.5% increase in accuracy on Chinese test sets.

Figures

Figures reproduced from arXiv: 2412.17225 by the authors.

Figure 1
Figure 1. (A.1) A conventional text encoder without visual glyph information. (A.2) A character-level text embedding that indirectly [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The framework of CharGen. tion: control condition, text encoder, and perceptual super￾vision. Control Condition. To guide text generation or editing, auxiliary glyph images of the text to be generated or edited are commonly used. For instance, TextDiffuser [5] initially generates segmentation masks of characters and then uti￾lizes these character segmentation masks along with posi￾tion masks to facilitate image gene… view at source ↗
Figure 3
Figure 3. Visual text image generated by CharGen. the CLIP latent space, enabling seamless integration into downstream frameworks without requiring additional adap￾tation. 3.3. Perceptual Loss Previous methods often faced challenges in generating ac￾curate strokes for complex words, resulting in outputs that were either unreadable or of low quality to users. We be￾lieve this issue arises from the use of inaccurate glyph loss … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: A qualitative comparison of CharGen with AnyText on English and Chinese text generation, using test captions from the AnyText [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: A qualitative comparison of CharGen with AnyText on visual text editing. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. InnoText: A Unified Model for Visual Text Generation and Editing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A unified DiT model with font-size-aware modulation and region-weighted loss outperforms existing visual text generation and editing systems on bilingual benchmarks.

Reference graph

Works this paper leans on

39 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Improving Image Gen- eration with Better Captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, Wesam Manassra, Prafulla Dhariwal, Casey Chu, Yunxin Jiao, and Aditya Ramesh. Improving Image Gen- eration with Better Captions. https://cdn.openai. com/papers/dall-e-3.pdf , 2023. Accessed:2024- 07-30. 1

    work page 2023

  2. [2]

    Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative transformers, 2023. 1

    work page 2023

  3. [3]

    Diffute: Universal text editing diffusion model

    Haoxing Chen, Zhuoer Xu, Zhangxuan Gu, Yaohui Li, Changhua Meng, Huijia Zhu, Weiqiang Wang, et al. Diffute: Universal text editing diffusion model. Advances in Neural Information Processing Systems, 36, 2024. 1, 2, 3

    work page 2024

  4. [4]

    TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering, 2023

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering, 2023. arXiv:2311.16465 [cs]. 6

    arXiv 2023

  5. [5]

    Textdiffuser: Diffusion models as text painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. Advances in Neural Information Processing Sys- tems, 36, 2024. 1, 2, 3, 6

    work page 2024

  6. [6]

    Diffusion Models Beat GANs on Image Synthesis, 2021

    Prafulla Dhariwal and Alex Nichol. Diffusion Models Beat GANs on Image Synthesis, 2021. arXiv:2105.05233 [cs, stat]. 1

    arXiv 2021

  7. [7]

    Odm: A text-image further alignment pre-training approach for scene text detection and spotting

    Chen Duan, Pei Fu, Shan Guo, Qianyi Jiang, and Xiaom- ing Wei. Odm: A text-image further alignment pre-training approach for scene text detection and spotting. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15587–15597, 2024. 2, 4

    work page 2024

  8. [8]

    Duguangocr

    Duguangocr. Duguangocr. https://modelscope.cn/ docs/convnextTiny_ocr-recognition, 2023. 6

    work page 2023

Show all 39 references
  1. [9]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Mach...

    2024

  2. [10]

    Wukong: 100 million large-scale chinese cross-modal pre-training dataset and a foundation frame- work, 2022

    Jiaxi Gu, Xiaojun Meng, Guansong Lu, Lu Hou, Minzhe Niu, Hang Xu, Xiaodan Liang, Wei Zhang, Xin Jiang, and Chunjing Xu. Wukong: 100 million large-scale chinese cross-modal pre-training dataset and a foundation frame- work, 2022. 6

    2022

  3. [11]

    Denoising Dif- fusion Probabilistic Models, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Dif- fusion Probabilistic Models, 2020. 1

    2020

  4. [12]

    Auto-encoding varia- tional bayes, 2022

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes, 2022. 4

    2022

  5. [13]

    Pp-ocrv3: More attempts for the improvement of ultra lightweight ocr system,

    Chenxia Li, Weiwei Liu, Ruoyu Guo, Xiaoting Yin, Kaitao Jiang, Yongkun Du, Yuning Du, Lingfeng Zhu, Baohua Lai, Xiaoguang Hu, Dianhai Yu, and Yanjun Ma. Pp-ocrv3: More attempts for the improvement of ultra lightweight ocr system,

  6. [14]

    Character-aware models improve visual text rendering, 2023

    Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mo- hammad Norouzi, and Noah Constant. Character-aware models improve visual text rendering, 2023. 1

    2023

  7. [15]

    Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, R. J. Mi- cal, Mohammad Norouzi, and Noah Constant. Character- Aware Models Improve Visual Text Rendering, 2023. arXiv:2212.10562 [cs]. 3

    2023 arXiv

  8. [16]

    Glyph-ByT5: A Cus- tomized Text Encoder for Accurate Visual Text Rendering,

    Zeyu Liu, Weicong Liang, Zhanhao Liang, Chong Luo, Ji Li, Gao Huang, and Yuhui Yuan. Glyph-ByT5: A Cus- tomized Text Encoder for Accurate Visual Text Rendering,

  9. [17]

    Glyph-ByT5-v2: A Strong Aesthetic Base- line for Accurate Multilingual Visual Text Rendering, 2024

    Zeyu Liu, Weicong Liang, Yiming Zhao, Bohan Chen, Ji Li, and Yuhui Yuan. Glyph-ByT5-v2: A Strong Aesthetic Base- line for Accurate Multilingual Visual Text Rendering, 2024. arXiv:2406.10208 [cs]. 3

    2024 arXiv

  10. [18]

    GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to- Image Generation, 2023

    Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, and Xiaodong Lin. GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to- Image Generation, 2023. arXiv:2303.17870 [cs]. 1, 2, 3

    2023 arXiv

  11. [19]

    Glyphdraw2: Automatic generation of complex glyph posters with diffusion models and large language models,

    Jian Ma, Yonglin Deng, Chen Chen, Haonan Lu, and Zhenyu Yang. Glyphdraw2: Automatic generation of complex glyph posters with diffusion models and large language models,

  12. [20]

    Midjourney

    Midjourney. Midjourney. https://www.midjourney. com, 2023. 1

    2023

  13. [21]

    Improved denoising dif- fusion probabilistic models, 2021

    Alex Nichol and Prafulla Dhariwal. Improved denoising dif- fusion probabilistic models, 2021. 1

    2021

  14. [22]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  15. [23]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 3

    2020

  16. [24]

    Zero-shot text-to-image generation, 2021

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation, 2021. 1

    2021

  17. [25]

    Hierarchical text-conditional image gener- ation with clip latents, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents, 2022. 1

    2022

  18. [26]

    Ocr-vqgan: Taming text- within-image generation

    Juan A Rodriguez, David Vazquez, Issam Laradji, Marco Pedersoli, and Pau Rodriguez. Ocr-vqgan: Taming text- within-image generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 3689–3698, 2023. 3

    2023

  19. [27]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 4

    2022

  20. [28]

    Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic Text-to- Image Diffusion M...

    2022 arXiv

  21. [29]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021. 6

    2021 arXiv

  22. [30]

    Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions, 2021. 1

    2021

  23. [31]

    AnyText: Multilingual Visual Text Gen- eration And Editing, 2024

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. AnyText: Multilingual Visual Text Gen- eration And Editing, 2024. arXiv:2311.03054 [cs]. 1, 2, 3, 4, 6

    2024 arXiv

  24. [32]

    Glyphcontrol: Glyph conditional control for visual text generation

    Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, and Kai Chen. Glyphcontrol: Glyph conditional control for visual text generation. Advances in Neural Information Processing Systems, 36, 2024. 1, 2, 3, 6

    2024

  25. [33]

    Long-clip: Unlocking the long-text capability of clip, 2024

    Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. Long-clip: Unlocking the long-text capability of clip, 2024. 4

    2024

  26. [34]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 4, 6

    2023

  27. [35]

    Adding Conditional Control to Text-to-Image Diffusion Models,

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding Conditional Control to Text-to-Image Diffusion Models,

  28. [36]

    Brush your text: Synthesize any scene text on im- ages via diffusion model

    Lingjun Zhang, Xinyuan Chen, Yaohui Wang, Yue Lu, and Yu Qiao. Brush your text: Synthesize any scene text on im- ages via diffusion model. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 7215–7223, 2024. 1, 2

    2024

  29. [37]

    UDiffText: A Uni- fied Framework for High-quality Text Synthesis in Arbi- trary Images via Character-aware Diffusion Models, 2023

    Yiming Zhao and Zhouhui Lian. UDiffText: A Uni- fied Framework for High-quality Text Synthesis in Arbi- trary Images via Character-aware Diffusion Models, 2023. arXiv:2312.04884 [cs]. 1, 2, 3

    2023 arXiv

  30. [2023]

    arXiv:2302.05543 [cs]. 2, 3

    arXiv

  31. [2024]

    arXiv:2403.09622 [cs]. 3, 6

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.