Pith. sign in

REVIEW 3 major objections 6 minor 43 references

Beyond Flat Text: Dual Self-inherited Guidance for Visual Text Generation

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free framework makes diffusion models render slanted and curved text far more accurately, by borrowing the model's own skill with flat text.

desk verdict A clever training-free plug-in that clearly lifts OCR accuracy on rotated masks, but the paper never checks whether text actually follows the slanted/curved layout, so the headline claim is only half-supported. read the letter →

arxiv 2501.05892 v2 pith:SR3IS246 submitted 2025-01-10 cs.CV

Minxing Luo , Zixun Xia , Liaojun Chen , Zhenhang Li , Weichao Zeng , Jianye Wang , Wentao Cheng , Yaxing Wang
show 2 more authors
Yu Zhou Jian Yang
This is my paper · ORCID
classification cs.CV
keywords visualtextgenerationdiffusionmodelslatentspaceguidancelayouttraining-freemethodOCRaccuracyadaptiveinstancenormalizationtext-imagecoherence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusion models that render text into images tend to distort letters when the requested text region is slanted or curved, because training data for such layouts is scarce. The paper claims that a training-free dual-branch framework, STGen, fixes this by reusing the model's own strength with flat text: it generates an accurate flat-text latent with the same prompt, rotates that latent into the target region as a semantic guide, and separately injects glyph-shape structure from the rendered text image. An AdaIN-based blending step keeps the injected latents in the same statistical range as the noisy latent, preserving background coherence. The authors report substantial OCR-accuracy gains on a new rotated-mask benchmark, e.g., English sentence accuracy rising from 38.04 to 45.43 and Chinese from 44.78 to 49.96 when their framework is added to a strong baseline, and they show the same scheme improves several other base models.

What carries the argument

Dual-branch latent guidance. The Semantic Rectification Branch runs the base visual text generation model on a flattened version of the requested mask, rotates the resulting $z_0$ latent into the user's position, and substitutes it into the text region (Eq. 1) after AdaIN normalization. The Structure Injection Branch encodes a rendered glyph image into latent $z_g$ and injects it with the same AdaIN operation. The two priors are merged by a weighted sum with hyperparameters $\rho$ and $\lambda$ and a timestep-decaying factor $\kappa_t$, so that guidance is strong early in denoising and fades later (Eqs. 6–7). The divide-and-conquer step splits curved or multi-part masks into near-straight segments using the mask's bounding-box direction so each segment can be flattened and rotated back.

What would settle it

Take a 75° slanted mask, run STGen with the correct rotation, then rerun it with the flat-text latent rotated by 10°–15° off the true angle while keeping everything else fixed. If OCR accuracy on the text region stays at the correctly-aligned level rather than falling toward the unguided baseline, the semantic-rectification branch is not carrying the claimed improvement.

Watch

Extended reading notes

Core claim

STGen's central claim is that visual-text generation for hard layouts can be decomposed into two inherited guides: the model's own output for flat text supplies semantics, and glyph images supply structure, so no training is needed. The paper argues that existing latent-space drift on slanted masks can be corrected by replacing, inside the text region, the noisy latent with a rotated, AdaIN-normalized flat-text latent from the same prompt, and by adding a glyph latent as structural grounding. The result, if correct, means the hard problem of arbitrary-layout text is reduced to a geometric alignment and blending problem on top of a capable flat-text generator.

Load-bearing premise

The load-bearing assumption is that a flat-text image latent, when rotated, stays semantically aligned with a slanted or curved text region of the same prompt; if the trained latent space is not roughly rotation-equivariant, the injected prior would place wrong or distorted text tokens and the accuracy gains would vanish.

Editorial extensions

If this is right

  • Existing visual text generation models can be upgraded for slanted, curved, or multi-part text layouts at inference time, with no retraining and no new training data.
  • OCR-based sentence accuracy on rotated masks improves, with the largest relative gains on hard rotations (60°–90°).
  • The framework improves several different base models, not just a single architecture, suggesting the semantic and structural priors transfer across backbones.
  • Background-text harmony improves because the semantic prior also carries background information, reducing occlusion by unrelated scene elements.
  • The proposed rotated-mask benchmark gives a reusable protocol for future work to measure text accuracy across easy, medium, and hard slant levels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If STGen works as reported, it implies the latent space of these diffusion models is approximately rotation-equivariant for text regions, and the same geometric-transfer trick might extend to perspective-distorted or non-rigid text on surfaces such as fabric, cans, and banners.
  • The AdaIN-based prior-blending recipe is a general mechanism: any external condition that can be encoded into a latent, such as a sketch, logo, or color palette, could be injected into the denoising process with the same range-preserving normalization.
  • Because the flat-text latent is generated from the same prompt, one could precompute it once and reuse it across multiple layout variations, making the approach more practical for interactive applications.
  • A direct upper-bound test would be to replace the generated flat-text latent with a ground-truth flat rendering; if OCR accuracy still falls short of perfect, the remaining bottleneck lies in the base model's text rendering rather than in geometric transfer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes STGen, a training-free dual-branch inference-time guidance method for pre-trained visual text generation models. A Semantic Rectification Branch generates a flat-text latent from the same prompt and a simplified mask, rotates it to match the user-provided position mask, and blends it into the main latent (with AdaIN) to provide a semantic prior. A Structure Injection Branch encodes the glyph image and injects its latent as a structural prior. A divide-and-conquer strategy splits curved or multi-part masks into straight segments. Experiments on a rotated-mask benchmark derived from the AnyText benchmark report improved OCR sentence accuracy and normalized edit distance over AnyText, Diff-Text, and GlyphControl in both English and Chinese across easy, medium, and hard rotation buckets; ablations support the contribution of both branches and the AdaIN combination. The paper also reports user-study preferences and FID results on the vanilla AnyText benchmark.

Significance. If the central claim holds, STGen is a practically valuable plug-and-play method: it improves several existing visual text generation models without retraining, which is demonstrated by consistent OCR gains across two languages and three difficulty levels, and the method is model-agnostic in design. The paper also contributes a synthetic benchmark for slanted/curved text generation, which addresses a real gap in evaluation. The ablations are clean and attribute the gain to the two branches plus AdaIN. However, the evidence is weakened by the absence of any metric that verifies the generated text follows the requested slanted or curved layout, and by the lack of statistical significance reporting. These are load-bearing issues for the paper's headline claim.

major comments (3)
  1. [4.2, Table 2] The benchmark and the reported metrics in Sec. 4.2 and Table 2 evaluate OCR sentence accuracy, normalized edit distance, and CLIP score, but none of these verifies that the generated text follows the user-provided rotated or curved position mask lp. Since the paper's headline claim is accurate generation 'in challenging scenarios (e.g., slanted or curved text layouts)', OCR readability alone is insufficient: a method that flattens text towards horizontal would improve OCR accuracy (OCR engines are trained on horizontal text) while failing the requested layout, and this is exactly the kind of shortcut the hard-level bucket (60-90 degrees, AnyText English 2.02 -> 6.60, Chinese 2.02 -> 8.55) could hide. Please add a quantitative layout-adherence evaluation, for example the angle difference between detected text bounding boxes and the mask orientation, text-region IoU with lp, or a user study that explicitly asks whether the text is placed at the requested slant/curve.
  2. [3.1, Eq. (1)] Eq. (1) rotates the flat-text latent zf0 and blends it into the slanted region, implicitly assuming the latent space of the VTGM is approximately rotation-equivariant so that the rotated semantic prior aligns with the slanted text region. This assumption is not tested anywhere in the paper. If the rotated latent injects misplaced or distorted tokens, the guidance could improve OCR readability without honoring the mask, or could harm generation for rotations outside the tested range. An explicit experiment that varies the rotation angle and measures both OCR accuracy and mask alignment (e.g., detected text angle vs. requested angle) would test this assumption directly and would also strengthen the evidence for the central claim.
  3. [4.3, Table 2] The hard-level absolute accuracy remains very low even with STGen (AnyText+Ours: 6.60 Sen.Acc English, 8.55 Chinese). The reported improvements are computed from a single run on a synthetic benchmark, and no variance, confidence intervals, or per-seed numbers are provided. Given the low absolute values and the small improvements at the hard level, please report results over multiple seeds or provide a statistical significance test (e.g., paired bootstrap over prompts) to establish that the gains are robust rather than noise. This is important because the hard-level bucket is where the paper's 'challenging scenarios' claim is strongest.
minor comments (6)
  1. [Table 3 caption] The caption says 'the numbers on the right are results from the English set while the numbers on the right are from the Chinese set'; the second 'right' should presumably be 'left' (or the wording should be corrected).
  2. [4.3] The sentence 'with an improvement of approximately 10%. over the baseline' contains a stray period after '10%'.
  3. [4.1] The text states that 'As shown in Tab. 1, our method demonstrates robustness for hyper-parameter sensitivity', but Table 1 only varies lambda and rho, not the temporal schedule kappa_t = 10t - T. Please either add a sensitivity study for kappa_t or revise the claim.
  4. [4.2] The evaluation does not report which OCR engine is used, its version or parameters, the exact number of prompts per difficulty level, or the random seed used for mask rotation; please add these details for reproducibility.
  5. [3.1] STGen requires an additional full denoising pass (T steps) in the Semantic Rectification Branch, but the paper reports no runtime or computational overhead compared with the baselines; for a training-free inference-time method, this cost should be quantified.
  6. [4.2] The paper states that 'we preprocess the masks using the Divide and Conquer strategy' for all methods before generation; please clarify whether this preprocessing is neutral to all methods or whether it inherently favors STGen, since the divide-and-conquer strategy is also a component of the proposed method.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: STGen's dual-branch guidance is an explicit algorithmic composition that is evaluated against external OCR ground truth, not a derivation that reduces to its own inputs.

full rationale

The paper's central operation is the Semantic Rectification Branch, which generates a flat-text latent zf with the same frozen VTGM and blends it into the target latent via Eq. (1) and Eq. (2) (tilde z_t = AdaIN(zf, zt) ⊙ lp + zt ⊙ (1-lp)). This is an explicitly stated algorithmic intervention, not a hidden equivalence: the target is slanted or curved text, while the prior is a flat rendering of the same prompt, so the method's output is not defined as its input. The Structure Injection Branch similarly injects a VAE-encoded glyph latent (Eq. 5) as an external structural condition. No fitted parameter is renamed as a prediction: λ and ρ are tuned in Table 1 and then fixed in Table 2, which is a test-set-tuning concern for absolute numbers but does not make the reported OCR accuracies equal to the fitted values by construction. The paper's citations (AnyText, AdaIN, Masui et al.) are to external prior work, not to the authors' own unverified results, and no uniqueness theorem or ansatz is imported from the authors' other papers. The benchmark is derived from AnyText-benchmark and scored with OCR Sen.Acc, NED, and CLIP against ground truth, so the main claims are externally measurable rather than self-referential. One non-circular concern is that the evaluation lacks a layout-adherence metric for the rotated masks, so OCR gains could partly reflect readability rather than geometric fidelity, but that is a metric-validity issue, not circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central method relies on two domain assumptions about the pretrained VTGM: flat-text latents are accurate enough to serve as priors, and latent-space rotation approximates image rotation. Hyperparameters lambda and rho are fitted on the proposed benchmark via sensitivity analysis. No new entities are postulated.

free parameters (3)
  • lambda (prior blend weight) = 0.5
    Chosen via sensitivity analysis in Table 1 as a balanced setting for both English and Chinese; used in Eq. 7 to weigh merged prior vs. original latent.
  • rho (semantic/structure balance) = 0.5
    Same sensitivity analysis; used in Eq. 6 to balance SRB and SIB priors.
  • kappa_t temporal schedule = 10t - T (as written, likely typo)
    Hand-designed decay factor in Eq. 7; exact schedule ambiguous as printed.
assumptions (3)
  • domain assumption Flat-text output of AnyText (or other VTGM) is sufficiently accurate to serve as a semantic prior
    Sec. 3.1: 'we inherit the high-quality flat text generation capabilities of the existing visual text model'; not proven for all prompts and languages.
  • domain assumption The model's latent space is approximately rotation-equivariant
    Sec. 3.1: 'We rotate zf0 to match the user-given position'; if rotation in latent space is not valid, the semantic prior would be misaligned.
  • domain assumption Complex masks can be decomposed into straight sections via Bezier curves without losing text semantics
    Sec. 3.1 Divide and Conquer strategy; no evidence that this segmentation preserves glyph shape for arbitrary curved text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Flat Text: Dual Self-inherited Guidance for Visual Text Generation." pith.science (2026). https://pith.science/paper/SR3IS246

@misc{pith2026250105892,
  author       = {Pith},
  title        = {Pith review of: Beyond Flat Text: Dual Self-inherited Guidance for Visual Text Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SR3IS246}},
  note         = {Machine review of arXiv:2501.05892}
}
read the original abstract

In real-world images, slanted or curved texts, especially those on cans, banners, or badges, appear as frequently, if not more so, than flat texts due to artistic design or layout constraints. While high-quality visual text generation has become available with the advanced generative capabilities of diffusion models, these models often produce distorted text and inharmonious text background when given slanted or curved text layouts due to training data limitation. In this paper, we introduce a new training-free framework, STGen, which accurately generates visual texts in challenging scenarios (\eg, slanted or curved text layouts) while harmonizing them with the text background. Our framework decomposes the visual text generation process into two branches: (i) \textbf{Semantic Rectification Branch}, which leverages the ability in generating flat but accurate visual texts of the model to guide the generation of challenging scenarios. The generated latent of flat text is abundant in accurate semantic information related both to the text itself and its background. By incorporating this, we rectify the semantic information of the texts and harmonize the integration of the text with its background in complex layouts. (ii) \textbf{Structure Injection Branch}, which reinforces the visual text structure during inference. We incorporate the latent information of the glyph image, rich in glyph structure, as a new condition to further strengthen the text structure. To enhance image harmony, we also apply an effective combination method to merge the priors, providing a solid foundation for generation. Extensive experiments across a variety of visual text layouts demonstrate that our framework achieves superior accuracy and outstanding quality.

Figures

Figures reproduced from arXiv: 2501.05892 by the authors.

Figure 1
Figure 1. STGen for visual text generation in challenging layout. Using a pre-trained visual text generation model (e.g., AnyText [36]), our method, STGen, guides the model to adjust the text region in latent space during image synthesis, producing images that more faithfully represent the input prompt with precise visual text. Abstract In real-world images, slanted or curved texts, especially those on cans, banners, or badge… view at source ↗
Figure 2
Figure 2. Failure cases of AnyText [36]. The top row illustrates two failure cases: textual distortion (left) and background occlu￾sion (right). The bottom row displays results using our method. gle to handle users’ diverse inputs, such as commonly seen slanted or curved texts in real-world images. Given these user inputs, the model often leads to text distortion and background occlusion, as shown in [PITH_FULL_IMAGE:figures… view at source ↗
Figure 4
Figure 4. Pipeline of our method. Given user input on the leftmost side, which contains a prompt and a mask lp specifying positions for generating visual texts, we first split the lp using Divide and Conquer Strategy and obtain glyph image lg and flat position mask ˜lp. Then ˜lp and lg are input to the Semantic Rectification Branch and Structure Injection Branch respectively. In the Semantic Rectification Branch, based on ˜lp… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison of our method and state-of-the-art models in both English and Chinese text generation. accuracy of visual text generation without compromising the baseline performance. Additionally, our method slightly improves text-image similarity due to our d…
Figure 6
Figure 6. Figure 6: More results on Background coherence and complex text layouts. The left two columns show how our approach seamlessly blends text with the background, while the remaining images highlight its ability to produce multi-sentence and circular text. w/ SRB, w/o SIB, w/o AdaI…
Figure 7
Figure 7. Figure 7: Ablation Study Visualization. The first column shows accuracy improvement with SRB. The second reveals that SIB en￾hances accuracy but compromises coherence. The third confirms that combining both branches enhances both. Finally, the fourth shows that adding AdaIN furt…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 22 canonical work pages

  1. [1]

    ediff-i: Text-to-image diffusion models with ensemble of expert denoisers

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, Tero Karras, and Ming-Yu Liu. ediff-i: Text-to-image diffusion models with ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. 1

    arXiv 2022

  2. [2]

    Text2live: Text-driven layered image and video editing

    Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kas- ten, and Tali Dekel. Text2live: Text-driven layered image and video editing. In European conference on computer vi- sion, pages 707–723. Springer, 2022. 7

    2022

  3. [3]

    Deep residual network for steganalysis of digital images

    Mehdi Boroumand, Mo Chen, and Jessica Fridrich. Deep residual network for steganalysis of digital images. IEEE Transactions on Information Forensics and Security , 14(5): 1181–1193, 2019. 1

    work page 2019

  4. [4]

    Freeman, Michael Ru- binstein, Yuanzhen Li, and Dilip Krishnan

    Huiwen Chang, Han Zhang, Jarred Barber, Aaron Maschinot, Jos ´e Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Patrick Murphy, William T. Freeman, Michael Ru- binstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to- image generation via masked generative transformers. In In- ternational Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawa...

    work page 2023

  5. [5]

    Textdiffuser-2: Unleashing the power of language models for text rendering

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser-2: Unleashing the power of language models for text rendering. In Computer Vi- sion - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part V , pages 386–402. Springer, 2024. 1, 3, 5, 6, 7

    work page 2024

  6. [6]

    Textdiffuser: Diffusion models as text painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. Advances in Neural Information Processing Sys- tems, 36, 2024. 1, 3, 5, 6, 7

    work page 2024

  7. [7]

    Icdar2019 robust reading challenge on arbitrary-shaped text-rrc-art

    Chee Kheng Chng, Yuliang Liu, Yipeng Sun, Chun Chet Ng, Canjie Luo, Zihan Ni, ChuanMing Fang, Shuaitao Zhang, Junyu Han, Errui Ding, et al. Icdar2019 robust reading challenge on arbitrary-shaped text-rrc-art. In 2019 Interna- tional Conference on Document Analysis and Recognition (ICDAR), pages 1571–1576. IEEE, 2019

    work page 2019

  8. [8]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 3

    2021

Show all 43 references
  1. [9]

    Synthetic data for text localisation in natural images

    Ankush Gupta, Andrea Vedaldi, and Andrew Zisserman. Synthetic data for text localisation in natural images. In Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 2315–2324, 2016. 3

    2016

  2. [10]

    Not every image is worth a thousand words: Quantifying originality in stable diffusion

    Adi Haviv, Shahar Sarfaty, Uri Hacohen, Niva Elkin-Koren, Roi Livni, and Amit H Bermano. Not every image is worth a thousand words: Quantifying originality in stable diffusion. arXiv preprint arXiv:2408.08184, 2024. 2

    2024 arXiv

  3. [11]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Processing, pages 7514–7528, 2021. 6

    2021

  4. [12]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In Proceedings of the 34th Inter- national Conference on Neural Information Processing Sys- tems, pages 6840–6851, 2020. 3

    2020

  5. [13]

    Arbitrary style transfer in real-time with adaptive instance normalization

    Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceed- ings of the IEEE international conference on computer vi- sion, pages 1501–1510, 2017. 4

    2017

  6. [14]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding vari- ational bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14- 16, 2014, Conference Track Proceedings, 2014. 5

    2014

  7. [15]

    Textstylebrush: transfer of text aes- thetics from a single example

    Praveen Krishnan, Rama Kovvuri, Guan Pang, Boris Vas- silev, and Tal Hassner. Textstylebrush: transfer of text aes- thetics from a single example. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):9122–9134, 2023. 1

    2023

  8. [16]

    Imagenet classification with deep convolutional neural net- works

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works. Advances in neural information processing systems , 25, 2012

    2012

  9. [17]

    Character-aware models improve visual text rendering

    Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, RJ Mical, Mo- hammad Norouzi, and Noah Constant. Character-aware models improve visual text rendering. arXiv preprint arXiv:2212.10562, 2022. 3

    2022 arXiv

  10. [18]

    Character-aware models improve visual text rendering

    Rosanne Liu, Dan Garrette, Chitwan Saharia, William Chan, Adam Roberts, Sharan Narang, Irina Blok, Rj Mical, Mo- hammad Norouzi, and Noah Constant. Character-aware models improve visual text rendering. In Proceedings of the 61st Annual Meeting of the Association for Computatio...

  11. [19]

    Glyph-byt5: A customized text encoder for accurate visual text rendering.arXiv preprint arXiv:2403.09622, 2024

    Zeyu Liu, Weicong Liang, Zhanhao Liang, Chong Luo, Ji Li, Gao Huang, and Yuhui Yuan. Glyph-byt5: A customized text encoder for accurate visual text rendering.arXiv preprint arXiv:2403.09622, 2024. 1, 3

    2024 arXiv

  12. [20]

    Glyph-byt5-v2: A strong aesthetic base- line for accurate multilingual visual text rendering

    Zeyu Liu, Weicong Liang, Yiming Zhao, Bohan Chen, Ji Li, and Yuhui Yuan. Glyph-byt5-v2: A strong aesthetic base- line for accurate multilingual visual text rendering. arXiv preprint arXiv:2406.10208, 2024. 3

    2024 arXiv

  13. [21]

    Glyphdraw: Seamlessly ren- dering text with intricate spatial structures in text-to-image generation

    Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, and Xiaodong Lin. Glyphdraw: Seamlessly ren- dering text with intricate spatial structures in text-to-image generation. arXiv preprint arXiv:2303.17870, 2023. 1, 3, 5

    2023 arXiv

  14. [22]

    Harnessing the latent diffusion model for training-free image style transfer

    Kento Masui, Mayu Otani, Masahiro Nomura, and Hideki Nakayama. Harnessing the latent diffusion model for training-free image style transfer. arXiv preprint arXiv:2410.01366, 2024. 4

    2024 arXiv

  15. [23]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR,

  16. [24]

    Any- trans: Translate anytext in the image with large scale models

    Zhipeng Qian, Pei Zhang, Baosong Yang, Kai Fan, Yiwei Ma, Derek F Wong, Xiaoshuai Sun, and Rongrong Ji. Any- trans: Translate anytext in the image with large scale models. arXiv preprint arXiv:2406.11432, 2024. 3 9

    2024 arXiv

  17. [25]

    Exploring stroke-level mod- ifications for scene text editing

    Yadong Qu, Qingfeng Tan, Hongtao Xie, Jianjun Xu, Yuxin Wang, and Yongdong Zhang. Exploring stroke-level mod- ifications for scene text editing. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 2119– 2127, 2023. 1

    2023

  18. [26]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  19. [27]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. 3

    2020

  20. [28]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International confer- ence on machine learning, pages 8821–8831. Pmlr, 2021. 3

    2021

  21. [29]

    Hierarchical text-conditional image gen- eration with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents. arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  22. [30]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 5

    2022

  23. [31]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  24. [32]

    Norm-guided latent space exploration for text-to-image generation

    Dvir Samuel, Rami Ben-Ari, Nir Darshan, Haggai Maron, and Gal Chechik. Norm-guided latent space exploration for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  25. [33]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. arXiv:2010.02502, 2020

    2010 arXiv

  26. [34]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In International Conference on Learning Represen- tations, 2021. 3

    2021

  27. [35]

    Strive: Scene text re- placement in videos

    Jeyasri Subramanian, Varnith Chordia, Eugene Bart, Shaobo Fang, Kelly Guan, Raja Bala, et al. Strive: Scene text re- placement in videos. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 14549– 14558, 2021. 1

    2021

  28. [36]

    Anytext: Multilingual visual text genera- tion and editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text genera- tion and editing. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. 1, 2, 3, 5...

    2024

  29. [37]

    Swaptext: Image based texts transfer in scenes

    Qiangpeng Yang, Jun Huang, and Wei Lin. Swaptext: Image based texts transfer in scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14700–14709, 2020

    2020

  30. [38]

    Glyphcontrol: Glyph conditional control for visual text generation

    Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, and Kai Chen. Glyphcontrol: Glyph conditional control for visual text generation. Advances in Neural Information Processing Systems, 36, 2024. 1, 3, 5, 6, 7

    2024

  31. [39]

    How control information influences multilingual text image generation and editing? arXiv preprint arXiv:2407.11502 ,

    Boqiang Zhang, Zuan Gao, Yadong Qu, and Hongtao Xie. How control information influences multilingual text image generation and editing? arXiv preprint arXiv:2407.11502 ,

    arXiv

  32. [40]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 3

    2023

  33. [41]

    Brush your text: Synthesize any scene text on im- ages via diffusion model

    Lingjun Zhang, Xinyuan Chen, Yaohui Wang, Yue Lu, and Yu Qiao. Brush your text: Synthesize any scene text on im- ages via diffusion model. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 7215–7223, 2024. 1, 5, 6, 7

    2024

  34. [42]

    Harmonizing visual text comprehension and gen- eration

    Zhen Zhao, Jingqun Tang, Binghong Wu, Chunhui Lin, Shu Wei, Hao Liu, Xin Tan, Zhizhong Zhang, Can Huang, and Yuan Xie. Harmonizing visual text comprehension and gen- eration. In Advances in Neural Information Processing Sys- tems 38: Annual Conference on Neural Information Pro...

    2024

  35. [43]

    Visual text generation in the wild

    Yuanzhi Zhu, Jiawei Liu, Feiyu Gao, Wenyu Liu, Xing- gang Wang, Peng Wang, Fei Huang, Cong Yao, and Zhibo Yang. Visual text generation in the wild. arXiv preprint arXiv:2407.14138, 2024. 1, 3, 5, 6, 7 10

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.