Pith. sign in

REVIEW 3 major objections 5 minor 78 references

SceneVTG++: Controllable Multilingual Visual Text Generation in the Wild

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a two-stage pipeline—a multimodal language model proposing text positions and content, followed by a pixel-level diffusion model rendering the text locally—can generate natural-scene multilingual text that is…

desk verdict Solid extension with useful datasets, but the SOTA claim is undercut by a possible train/test overlap on MLT2017 and mixed numbers against the authors' own prior version. read the letter →

arxiv 2501.02962 v2 pith:BJIDAAJF submitted 2025-01-06 cs.CV

Jiawei Liu , Yuanzhi Zhu , Feiyu Gao , Zhibo Yang , Peng Wang , Junyang Lin , Xinggang Wang , Wenyu Liu This is my paper · ORCID
classification cs.CV
keywords visualtextgenerationscenemultilingualrenderingconditionaldiffusionmodelsmultimodallargelanguagelayoutsyntheticOCRtrainingdataattributecontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SceneVTG++ is an attempt to solve natural-scene visual text generation by splitting the task into two stages: first, a multimodal large language model looks at a background image and proposes where text should go and what it should say; second, a dedicated local diffusion model renders the text at that spot. The paper argues that this division lets one system simultaneously satisfy four criteria that earlier methods meet only in part: fidelity (no stroke errors or artificial seams), reasonability (text on sensible carriers with content related to the scene), utility (generated images work as training data for OCR detection and recognition), and controllability (font and color can be specified). To train and test this, the paper contributes a 197K-pair multilingual text-erase dataset with color and font labels, a 100K-pair synthetic extension, and a benchmark built from existing scene-text sets. If the central claim is right, synthetic scene text can replace some manually collected real data for training OCR systems in multiple languages.

What carries the argument

The load-bearing machinery is the two-stage architecture itself. TLCG handles the semantic problem: a visual-instruction-tuned multimodal language model, fine-tuned with a two-step chain-of-thought-style prompt, proposes text locations and contents that fit the scene, and the second step determines layout-box geometry from the proposed text length. CLTD handles the visual problem: a conditional diffusion model without latent compression, operating on a cropped background patch with a text-line mask and optional stroke mask as image-level conditions and an OCR recognizer's image/text embeddings plus an RGB color embedding as embedding-level conditions. The pixel-level design and the extra foreground loss computed in recognizer feature space are what allow small characters to survive and keep the rendered text aligned with the intended annotations.

What would settle it

Run the full pipeline on a held-out set of scene types, scripts, and carrier materials absent from SceneVTG-Erase++—for example handwritten menus, Arabic script, or curved bottle labels—and count whether generated text stays on the carrier, remains fully readable, and is judged semantically relevant by human raters; if those rates collapse while metrics on the paper's own benchmark stay high, the generality of the reasonability claim would be shown to depend on dataset overlap rather than on the method itself.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that decomposing scene text generation into a language-model planning stage and a local pixel-level rendering stage solves the trade-off that pushes rendering-based and diffusion-based methods to opposite corners. The Text Layout and Content Generator (TLCG) is a multimodal large language model fine-tuned with a two-step prompt: first it outputs candidate anchor points and semantically relevant multilingual text strings, then it outputs quadrilateral layout boxes that enclose those points without crossing carrier boundaries. The Controllable Local Text Diffusion (CLTD) is a conditional diffusion model that operates at pixel level rather than in a compressed latent space; conditioned on a background crop, a text-line mask, an optional stroke mask, and image/text/color embeddings, it denoises only the local text region and stitches it back seamlessly. Trained on the new SceneVTG-Erase++ and SceneVTG-Syn datasets, the combined pipeline is reported to achieve the best results among compared methods on fidelity, reasonability, utility, and controllability, including multilingual text that previous diffusion methods render poorly.

Load-bearing premise

Everything rests on the assumption that a multimodal language model fine-tuned on erased scene-text images can propose sensible text locations and contents for arbitrary natural backgrounds; if that proposal fails, the diffusion stage produces well-rendered text in the wrong place or with the wrong meaning.

Editorial extensions

If this is right

  • If the reported results hold, a single pipeline can produce synthetic multilingual scene text whose text and annotations align well enough to train OCR detectors and recognizers without manual relabeling.
  • Because font is controlled through a stroke mask and color through RGB input, one background image can be turned into many text variants, enabling targeted augmentation over rare colors, fonts, and writing systems.
  • The reported benchmark numbers imply that generated data can approach real photographed data for training English OCR and can substantially improve multilingual recognition compared with prior synthetic pipelines.
  • The same TLCG+CLTD division could be applied to full-image generation tasks where the user wants text at a specific location while keeping the rest of the photograph untouched.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An unstated but direct extension is to use TLCG's two-step prompting for scene text editing: replace an existing sign's wording while preserving its carrier and surroundings, since the module already outputs location, content, and layout independently of rendering.
  • The architecture suggests that script coverage is limited mainly by the training data and the OCR recognizer's embedding vocabulary rather than by the diffusion renderer, meaning scripts absent from SceneVTG-Erase++ could fail even if the local diffusion stage can draw arbitrary strokes; the paper does not test this.
  • One could turn CLTD into a controllable data engine for rare OCR conditions by sampling colors, fonts, and languages from a specified distribution and then measuring recognizer gains on real benchmarks, making the utility claim testable beyond the paper's fixed test sets.
  • The paper's evaluation of reasonability is confined to benchmark images drawn from the same data sources as the training set, so an honest test of the claimed generalization would require holding out entire scene types, languages, and carrier materials.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SceneVTG++, a two-stage framework for generating multilingual visual text in natural scene images. The first stage, TLCG, uses a fine-tuned multimodal large language model to propose reasonable text locations and semantically relevant text content from a background image. The second stage, CLTD, is a pixel-level conditional diffusion model that renders controllable text with specified color and font at the proposed locations. The authors introduce two datasets, SceneVTG-Erase++ (197K real image pairs) and SceneVTG-Syn (100K synthetic pairs), and evaluate their method against rendering-based and diffusion-based baselines on fidelity, reasonability, utility, and controllability using their SceneVTG-benchmark++ test set. The paper claims to simultaneously satisfy all four criteria and to achieve state-of-the-art text generation performance.

Significance. If the reported results hold, the paper would provide a practically useful pipeline for synthesizing scene text data for OCR training, and it would be one of the first systems to combine context-aware layout/content reasoning with controllable multilingual text rendering in natural scenes. The authors contribute substantial datasets and a well-structured two-stage design, and they evaluate utility by training OCR models on the generated data, which is a meaningful downstream test. However, the central claims of state-of-the-art performance and simultaneous satisfaction of all four criteria are not cleanly supported by the paper's own tables, and a potential train/test overlap in the benchmark requires explicit resolution before the results can be trusted.

major comments (3)
  1. [Section 4; Section 5.2 and 5.4; Tables 1 and 3] The paper does not rule out train/test overlap. Section 4 states that SceneVTG-Erase++ is built from 'multiple publicly available multilingual scene text datasets [58,...,70]' (reference [69] is MLT2017), and the same section defines SceneVTG-benchmark++ as including 'the validation set of the multilingual natural scene dataset MLT2017 [69]'. Nowhere do the authors state that MLT2017 validation images, or their CTRNet-erased and DiffBIR-restored versions, were excluded from the training pairs. If any such image appears in training, the fidelity and utility numbers in Tables 1 and 3 are partially measured on training data. The authors must explicitly confirm the split exclusion or re-evaluate on provably disjoint data. Even with exclusion, the benchmark comes from the same dataset family used to construct the training corpus, so the reasonability and fidelity results do not demonstrate generalization to unseen scene distributions as the 'in the wild' claim implies.
  2. [Abstract; Section 5.2.1; Tables 1 and 3] The claim of 'state-of-the-art text generation performance' is contradicted by the paper's own English end-to-end results: SceneVTG++ yields FID 29.99 versus SceneVTG's 26.28 and LA 74.93 versus 75.62 (Table 1). In the utility table, SceneVTG++ is also below SceneVTG on detection F for IC13 (74.68 vs 75.36), IC15 (65.13 vs 66.31), and MLT17 (53.51 vs 53.90). The manuscript should either qualify the SOTA claim (for example, to the multilingual setting or to an aggregated ranking over all four criteria), or provide a statistical justification for why the performance is considered superior overall. As written, the headline claim overstates what Tables 1 and 3 show.
  3. [Section 5.5; Figure 12] Controllability, one of the four criteria the paper claims to satisfy, is evaluated only qualitatively. Figure 12 shows example images under color and font conditions, but there is no quantitative metric such as color prediction accuracy, font classification accuracy, or OCR success rate under specified attributes, and no user study. Without such evidence, the claim of 'precise attribute control' across languages and text lengths is not substantiated. A quantitative controllability evaluation is needed to support the paper's central claim of simultaneously satisfying all four criteria.
minor comments (5)
  1. [Section 5.6; Table 4] The text states that full fine-tuning outperforms LoRA on all metrics except readability, but Table 4 also shows full fine-tuning has a lower CLIPScore (22.73 vs 24.75). The sentence should be corrected to acknowledge both exceptions.
  2. [Section 3.2.3; Eq. (4); Section 5.1] Equation (4) introduces hyperparameters λ_f and λ_b, but Section 5.1 does not report their values. Please provide these values for reproducibility.
  3. [Table 2] The CLIPScore entry for PosterLlama is listed as '-', but the metric is otherwise defined for all compared methods; please either report the value or explain why it is omitted.
  4. [Section 2.1.2; Table 3] The method 'VISD' is mentioned in Section 2.1.2 without a citation or reference, although it appears in Table 3. A citation and a brief description should be added.
  5. [Throughout] The names 'SceneVTG-Erase ++' and 'SceneVTG-benchmark ++' contain inconsistent spacing around the '++'; the notation should be unified.

Circularity Check

0 steps flagged · score 2.0 of 10

No demonstrated circularity: the derivation is anchored to external metrics and benchmarks, with a potential MLT2017 train/test overlap that is a benchmark-validity risk rather than a reduction-by-construction.

full rationale

The claimed derivation chain is largely self-contained. CLTD is a pixel-level conditional diffusion model trained from scratch on SceneVTG-Erase++ and SceneVTG-Syn; its fidelity is judged by FID, PaddleOCR-based F/LA, and local-image comparisons against external real benchmarks, not by its training loss. TLCG is an MMLM fine-tuned on ground-truth layout/content pairs, and its reasonability is measured by IoU against real ground-truth boxes, PD-Edge, readability, and CLIPScore; these metrics are external to the model's objective. Utility is measured by training OCR detectors/recognizers on generated images and testing on standard benchmarks (IC13, IC15, MLT2017, etc.), which is an external downstream check. The only self-citations are to the conference version SceneVTG [19] as a baseline/extension point, which is not load-bearing. The one substantive concern is that Section 4 lists MLT2017 among the datasets used to build SceneVTG-Erase++ ('collects multiple publicly available multilingual scene text datasets [58,...,70]') and separately defines SceneVTG-benchmark++ as including 'the validation set of the multilingual natural scene dataset MLT2017 [69]', without stating that the MLT2017 validation split was excluded from training. If validation images entered the erased training pairs, Tables 1-3 would report on training data; but the paper does not actually state this overlap, and standard dataset use often implies the training split. This is an omitted split specification and a benchmark-validity/correctness risk, not a demonstrated circular reduction under the hard-rule standard.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim rests on training data and heuristic modeling choices rather than on derived constants. The free parameters are the two unreported loss weights, and the axioms are domain assumptions about scene text placement, color extraction, stroke masks, MMLM generalization, and architecture choice. No invented entities are introduced.

free parameters (2)
  • Lambda_f (foreground loss weight) = not reported
    Equation (4) defines the training loss as L_cdm + lambda_f * L_fore + lambda_b * L_back. The value of lambda_f is never given, yet it controls the balance between the diffusion objective and text fidelity.
  • Lambda_b (background loss weight) = not reported
    Same equation (4); no value is reported, and it controls how strongly the model preserves the background pixels outside the text mask.
assumptions (5)
  • domain assumption Text in natural scenes should appear on carriers such as signs, boards, and walls, and not on sky or ground.
    This norm is used in Section 1 and Figure 1 to define 'reasonability' and in the TLCG prompt (Figure 4) to constrain layout proposals. It is a cultural and scene prior, not a mathematical truth, and may fail for other scene distributions.
  • domain assumption The dominant color removed by text erasure is the text color.
    Section 3.3 reconstructs text color by comparing palettes before and after erasure and taking the color with the largest decrease. This assumes text is the main chromatic difference, which fails for textured or color-varying backgrounds.
  • domain assumption Adaptive threshold binarization of text regions yields a usable stroke mask representing the font.
    Section 3.3 and Figure 6 use binarized text strokes as the font condition. This assumes clean stroke segmentation from thresholding alone; complex backgrounds or low contrast can produce noisy masks.
  • domain assumption A pixel-level diffusion model (without latent compression) preserves small text features better than latent diffusion.
    Section 3.2.3 states this rationale for choosing a pixel-level model. It is a plausible engineering claim but not demonstrated by an ablation comparing latent vs pixel-level backbones.
  • domain assumption Fine-tuned LLaVA can infer reasonable text content and positions from image context across languages.
    Section 3.1 relies on the world knowledge of a fine-tuned MMLM to generate semantically relevant text content and layouts; this capability is only evaluated on the authors' constructed benchmark, not on out-of-distribution scenes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SceneVTG++: Controllable Multilingual Visual Text Generation in the Wild." pith.science (2026). https://pith.science/paper/BJIDAAJF

@misc{pith2026250102962,
  author       = {Pith},
  title        = {Pith review of: SceneVTG++: Controllable Multilingual Visual Text Generation in the Wild},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BJIDAAJF}},
  note         = {Machine review of arXiv:2501.02962}
}
read the original abstract

Generating visual text in natural scene images is a challenging task with many unsolved problems. Different from generating text on artificially designed images (such as posters, covers, cartoons, etc.), the text in natural scene images needs to meet the following four key criteria: (1) Fidelity: the generated text should appear as realistic as a photograph and be completely accurate, with no errors in any of the strokes. (2) Reasonability: the text should be generated on reasonable carrier areas (such as boards, signs, walls, etc.), and the generated text content should also be relevant to the scene. (3) Utility: the generated text can facilitate to the training of natural scene OCR (Optical Character Recognition) tasks. (4) Controllability: The attribute of the text (such as font and color) should be controllable as needed. In this paper, we propose a two stage method, SceneVTG++, which simultaneously satisfies the four aspects mentioned above. SceneVTG++ consists of a Text Layout and Content Generator (TLCG) and a Controllable Local Text Diffusion (CLTD). The former utilizes the world knowledge of multi modal large language models to find reasonable text areas and recommend text content according to the nature scene background images, while the latter generates controllable multilingual text based on the diffusion model. Through extensive experiments, we respectively verified the effectiveness of TLCG and CLTD, and demonstrated the state-of-the-art text generation performance of SceneVTG++. In addition, the generated images have superior utility in OCR tasks like text detection and text recognition. Codes and datasets will be available.

Figures

Figures reproduced from arXiv: 2501.02962 by the authors.

Figure 1
Figure 1. In terms of fidelity, reasonability, utility, and controllability, our proposed SceneVTG [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Comparison of pipelines for different visual text generation methods counter difficulties in generating small characters. (2) The reasonability of generating text layout and content depends on the given prompts, but a rational prompt requires careful design manually. (3) Existing methods cannot precisely generate text at the given layout location and may generate unexpected text. (4) The generated text are of￾ten un… view at source ↗
Figure 3
Figure 3. The overall pipeline of SceneVTG++. With background images and predefined text prompts as input, TLCG generates reasonable text layouts and recommends appropriate text contents. CLTD then generates text on the background image based on TLCG outputs along with some other text attributes. However, these methods are not specifically designed for text and encounter significant difficulties when generating text. In this … view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: An example of TLCG workflow that generate reasonable text layout and content in two steps. The [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: The detailed architecture of the Controllable Local Text Di [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: The process for constructing color and font conditions, as well as data synthesis. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results of end-to-end English text generation. (a) SynthText; (b) Textdi [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Visualizations of end-to-end multilingual text generation results: (a) AnyText; (b) SceneVTG [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Visualizations of local text generation results. The first row visualizes local English text genera [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Visualizations of text layout and content generation. (a) and (b) show the comparison between [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Visualizations of generated text with annotations. SceneVTG [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]
Figure 12
Figure 12. Figure 12: Visualizations of generated text with attribute conditions. SceneVTG [PITH_FULL_IMAGE:figures/full_fig_p024_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 62 canonical work pages

  1. [69]

    Nayef, F

    N. Nayef, F. Yin, I. Bizid, H. Choi, Y . Feng, D. Karatzas, Z. Luo, U. Pal, C. Rigaud, J. Chazalon, W. Khlif, M. M. Luqman, J. Burie, C. Liu, J. Ogier, ICDAR2017 robust reading challenge on multi-lingual scene text detection and script identification - RRC-MLT, in: 14th IAPR International Conference on Document Analysis and Recognition, ICDAR, 2017, pp. 1454–1459

    work page 2017

  2. [1]

    Ramesh, M

    A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. V oss, A. Radford, M. Chen, I. Sutskever, Zero-shot text-to-image generation, in: Proceedings of the 38th International Conference on Machine Learning, ICML, 2021, pp. 8821–8831

    work page 2021

  3. [2]

    Ramesh, P

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, M. Chen, Hierarchical text- conditional image generation with clip latents, arXiv preprint arXiv:2204.06125 (2022)

    arXiv 2022

  4. [3]

    Betker, G

    J. Betker, G. Goh, L. Jing, TimBrooks, J. Wang, L. Li, LongOuyang, Jun- tangZhuang, JoyceLee, YufeiGuo, WesamManassra, PrafullaDhariwal, Casey- Chu, YunxinJiao, A. Ramesh, Improving image generation with better captions, 2023

    work page 2023

  5. [4]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, High-resolution im- age synthesis with latent di ffusion models, in: IEEE /CVF Conference on Com- puter Vision and Pattern Recognition, CVPR, 2022, pp. 10674–10685

    work page 2022

  6. [5]

    Podell, Z

    D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. M ¨uller, J. Penna, R. Rombach, SDXL: improving latent di ffusion models for high-resolution im- age synthesis, in: The Twelfth International Conference on Learning Represen- tations,ICLR, 2024

    work page 2024

  7. [6]

    Daras, A

    G. Daras, A. G. Dimakis, Discovering the hidden vocabulary of DALLE-2, arXiv preprint arXiv:2206.00169 (2022)

    arXiv 2022

  8. [7]

    R. Liu, D. Garrette, C. Saharia, W. Chan, A. Roberts, S. Narang, I. Blok, R. Mical, M. Norouzi, N. Constant, Character-aware models improve visual text rendering, in: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), ACL, 2023, pp. 16270–16297

    work page 2023

Show all 78 references
  1. [8]

    S. Fang, H. Xie, Y . Wang, Z. Mao, Y . Zhang, Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition,CVPR, 2021, pp. 7098–7107. 26

    2021

  2. [9]

    Y . Liu, H. Chen, C. Shen, T. He, L. Jin, L. Wang, Abcnet: Real-time scene text spotting with adaptive bezier-curve network, in: IEEE /CVF Conference on Computer Vision and Pattern Recognition,CVPR, 2020, pp. 9806–9815

    2020

  3. [10]

    B. Shi, X. Bai, C. Yao, An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition, IEEE Trans. Pattern Anal. Mach. Intell. (2017) 2298–2304

    2017

  4. [11]

    X. Zhou, C. Yao, H. Wen, Y . Wang, S. Zhou, W. He, J. Liang, EAST: an efficient and accurate scene text detector, in: IEEE /CVF Conference on Computer Vision and Pattern Recognition,CVPR, 2017, pp. 2642–2651

    2017

  5. [12]

    Gupta, A

    A. Gupta, A. Vedaldi, A. Zisserman, Synthetic data for text localisation in natural images, in: IEEE /CVF Conference on Computer Vision and Pattern Recogni- tion,CVPR, 2016, pp. 2315–2324

    2016

  6. [14]

    A. Q. Nichol, P. Dhariwal, Improved denoising diffusion probabilistic models, in: Proceedings of the 38th International Conference on Machine Learning,ICML, 2021, pp. 8162–8171

    2021

  7. [15]

    J. Song, C. Meng, S. Ermon, Denoising di ffusion implicit models, in: The Twelfth International Conference on Learning Representations,ICLR, 2021

    2021

  8. [16]

    J. Chen, Y . Huang, T. Lv, L. Cui, Q. Chen, F. Wei, Textdiffuser: Diffusion models as text painters, in: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, (NeurIPS), 2023

    2023

  9. [17]

    J. Chen, Y . Huang, T. Lv, L. Cui, Q. Chen, F. Wei, Textdiffuser-2: Unleashing the power of language models for text rendering, arXiv preprint arXiv:2311.16465 (2023). 27

    2023 arXiv

  10. [18]

    Y . Tuo, W. Xiang, J. He, Y . Geng, X. Xie, Anytext: Multilingual visual text generation and editing, in: The Twelfth International Conference on Learning Representations,ICLR, 2024

    2024

  11. [19]

    Y . Zhu, J. Liu, F. Gao, W. Liu, X. Wang, P. Wang, F. Huang, C. Yao, Z. Yang, Visual text generation in the wild, arXiv preprint arXiv:2407.14138 (2024)

    2024 arXiv

  12. [20]

    J. Li, J. Yang, A. Hertzmann, J. Zhang, T. Xu, Layoutgan: Synthesizing graphic layouts with vector-wireframe adversarial networks, IEEE Trans. Pattern Anal. Mach. Intell. (2021) 2388–2399

    2021

  13. [21]

    A. A. Jyothi, T. Durand, J. He, L. Sigal, G. Mori, Layoutvae: Stochastic scene layout generation from a label set, in: 2019 IEEE /CVF International Conference on Computer Vision, ICCV, 2019, pp. 9894–9903

    2019

  14. [22]

    D. P. Kingma, M. Welling, Auto-encoding variational bayes, in: 2nd International Conference on Learning Representations, ICLR 2014,, 2014

    2014

  15. [23]

    Gupta, J

    K. Gupta, J. Lazarow, A. Achille, L. Davis, V . Mahadevan, A. Shrivastava, Lay- outtransformer: Layout generation and completion with self-attention, in: 2021 IEEE/CVF International Conference on Computer Vision, ICCV, 2021, pp. 984– 994

    2021

  16. [24]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017, pp. 5998–6008

    2017

  17. [25]

    Inoue, K

    N. Inoue, K. Kikuchi, E. Simo-Serra, M. Otani, K. Yamaguchi, Layoutdm: Dis- crete diffusion model for controllable layout generation, in: IEEE /CVF Con- ference on Computer Vision and Pattern Recognition, CVPR 2023, 2023, pp. 10167–10176

    2023

  18. [26]

    S. Long, C. Yao, Unrealtext: Synthesizing realistic scene text images from the unreal world, arXiv preprint arXiv:2003.10608 (2020). 28

    2020 arXiv

  19. [27]

    Z. Tang, T. Miyazaki, S. Omachi, A scene-text synthesis engine achieved through learning from decomposed real-world data, IEEE Trans. Image Process. (2023) 5837–5851

    2023

  20. [28]

    M. Zhou, C. Xu, Y . Ma, T. Ge, Y . Jiang, W. Xu, Composition-aware graphic layout GAN for ‘-textual presentation designs, in: Proceedings of the Thirty- First International Joint Conference on Artificial Intelligence, IJCAI, 2022, pp. 4995–5001

    2022

  21. [29]

    H. Hsu, X. He, Y . Peng, H. Kong, Q. Zhang, Posterlayout: A new benchmark and approach for content-aware visual-textual presentation layout, in: IEEE /CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2023, pp. 6018–6026

    2023

  22. [30]

    F. Li, A. Liu, W. Feng, H. Zhu, Y . Li, Z. Zhang, J. Lv, X. Zhu, J. Shen, Z. Lin, J. Shao, Relation-aware diffusion model for controllable poster layout generation, in: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM, 2023, pp...

    2023

  23. [31]

    J. Lin, J. Guo, S. Sun, Z. Yang, J. Lou, D. Zhang, Layoutprompter: Awaken the design ability of large language models, in: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS, 2023

    2023

  24. [32]

    J. Seol, S. Kim, J. Yoo, Posterllama: Bridging design ability of langauge model to contents-aware layout generation, arXiv preprint arXiv:2404.00995 (2024)

    2024 arXiv

  25. [33]

    Mirza, Conditional generative adversarial nets, arXiv preprint arXiv:1411.1784 (2014)

    M. Mirza, Conditional generative adversarial nets, arXiv preprint arXiv:1411.1784 (2014)

    2014 arXiv

  26. [34]

    K. Sohn, H. Lee, X. Yan, Learning structured output representation using deep conditional generative models, in: Advances in Neural Information Process- ing Systems 28: Annual Conference on Neural Information Processing Systems, 2015, pp. 3483–3491. 29

    2015

  27. [35]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: Proceedings of the 38th International Conference on Machine Learni...

    2021

  28. [36]

    Zhang, A

    L. Zhang, A. Rao, M. Agrawala, Adding conditional control to text-to-image diffusion models, in: IEEE /CVF International Conference on Computer Vision, ICCV, 2023, pp. 3813–3824

    2023

  29. [37]

    Huang, D

    L. Huang, D. Chen, Y . Liu, Y . Shen, D. Zhao, J. Zhou, Composer: Creative and controllable image synthesis with composable conditions, in: International Conference on Machine Learning, ICML, 2023, pp. 13753–13773

    2023

  30. [38]

    Y . Yang, D. Gui, Y . Yuan, W. Liang, H. Ding, H. Hu, K. Chen, Glyphcontrol: Glyph conditional control for visual text generation, in: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems, NeurIPS, 2023

    2023

  31. [39]

    Z. Liu, W. Liang, Z. Liang, C. Luo, J. Li, G. Huang, Y . Yuan, Glyph-byt5: A customized text encoder for accurate visual text rendering, arXiv preprint arXiv:2403.09622 (2024)

    2024 arXiv

  32. [40]

    Zhang, Z

    B. Zhang, Z. Gao, Y . Qu, H. Xie, How control information influences multilingual text image generation and editing?, arXiv preprint arXiv:2407.11502 (2024)

    2024 arXiv

  33. [41]

    L. Kang, P. Riba, Y . Wang, M. Rusi ˜nol, A. Forn ´es, M. Villegas, Ganwriting: Content-conditioned generation of styled handwritten word images, in: Computer Vision - ECCV, 2020, pp. 273–289

    2020

  34. [42]

    C. Luo, Y . Zhu, L. Jin, Z. Li, D. Peng, SLOGAN: handwriting style synthesis for arbitrary-length and out-of-vocabulary text, IEEE Trans. Neural Networks Learn. Syst. (2023) 8503–8515

    2023

  35. [43]

    Nikolaidou, G

    K. Nikolaidou, G. Retsinas, V . Christlein, M. Seuret, G. Sfikas, E. B. Smith, H. Mokayed, M. Liwicki, Wordstylist: Styled verbatim handwritten text gen- 30 eration with latent di ffusion models, in: Document Analysis and Recognition - ICDAR, 2023, pp. 384–401

    2023

  36. [44]

    Z. Yang, D. Peng, Y . Kong, Y . Zhang, C. Yao, L. Jin, Fontdi ffuser: One-shot font generation via denoising diffusion with multi-scale content aggregation and style contrastive learning, in: Thirty-Eighth AAAI Conference on Artificial In- telligence, AAAI, 2024, pp. 6603–6611

    2024

  37. [45]

    Y . Zhu, Z. Li, T. Wang, M. He, C. Yao, Conditional text image generation with diffusion models, in: IEEE /CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2023, pp. 14235–14244

    2023

  38. [46]

    Avrahami, D

    O. Avrahami, D. Lischinski, O. Fried, Blended di ffusion for text-driven editing of natural images, in: IEEE /CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2022, pp. 18187–18197

    2022

  39. [47]

    Brooks, A

    T. Brooks, A. Holynski, A. A. Efros, Instructpix2pix: Learning to follow image editing instructions, in: IEEE /CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2023, pp. 18392–18402

    2023

  40. [48]

    Couairon, J

    G. Couairon, J. Verbeek, H. Schwenk, M. Cord, Diffedit: Diffusion-based seman- tic image editing with mask guidance, in: The Eleventh International Conference on Learning Representations, ICLR, 2023

    2023

  41. [49]

    F. Wei, X. Zhang, A. Zhang, B. Zhang, X. Chu, Lenna: Language enhanced reasoning detection assistant, arXiv preprint arXiv:2312.02433 (2023)

    2023 arXiv

  42. [50]

    X. Lai, Z. Tian, Y . Chen, Y . Li, Y . Yuan, S. Liu, J. Jia, Lisa: Reasoning segmen- tation via large language model, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2024, pp. 9579–9589

    2024

  43. [51]

    H. Liu, C. Li, Q. Wu, Y . J. Lee, Visual instruction tuning, Advances in neural information processing systems, NeurIPS 36 (2024)

    2024

  44. [52]

    H. Liu, C. Li, Y . Li, Y . J. Lee, Improved baselines with visual instruction tuning, in: Proceedings of the IEEE /CVF Conference on Computer Vision and Pattern Recognition, CVPR, 2024, pp. 26296–26306. 31

    2024

  45. [53]

    J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, J. Zhou, Qwen- vl: A frontier large vision-language model with versatile abilities, arXiv preprint arXiv:2308.12966 (2023)

    2023 arXiv

  46. [54]

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, S. C. H. Hoi, Instructblip: Towards general-purpose vision-language models with instruc- tion tuning, in: Advances in neural information processing systems, NeurIPS, 2023

    2023

  47. [55]

    Zhang, L

    S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu, G. Wang, Instruction tuning for large language models: A survey, arXiv preprint arXiv:2308.10792 (2023)

    2023

  48. [56]

    C. Li, W. Liu, R. Guo, X. Yin, K. Jiang, Y . Du, Y . Du, L. Zhu, B. Lai, X. Hu, D. Yu, Y . Ma, Pp-ocrv3: More attempts for the improvement of ultra lightweight OCR system, arXiv preprint arXiv:2206.03001 (2022)

    2022 arXiv

  49. [57]

    C. Liu, Y . Liu, L. Jin, S. Zhang, C. Luo, Y . Wang, Erasenet: End-to-end text removal in the wild, IEEE Trans. Image Process. (2020) 8760–8775

    2020

  50. [58]

    Y . Liu, L. Jin, S. Zhang, S. Zhang, Detecting curve text in the wild: New dataset and new solution, arXiv preprint arXiv:1712.02170 (2017)

    2017 arXiv

  51. [59]

    Karatzas, F

    D. Karatzas, F. Shafait, S. Uchida, M. Iwamura, L. G. i Bigorda, S. R. Mestre, J. Mas, D. F. Mota, J. Almaz ´an, L. de las Heras, ICDAR 2013 robust read- ing competition, in: 12th International Conference on Document Analysis and Recognition, ICDAR, 2013, pp. 1484–1493

    2013

  52. [60]

    Karatzas, L

    D. Karatzas, L. Gomez-Bigorda, A. Nicolaou, S. K. Ghosh, A. D. Bagdanov, M. Iwamura, J. Matas, L. Neumann, V . R. Chandrasekhar, S. Lu, F. Shafait, S. Uchida, E. Valveny, ICDAR 2015 competition on robust reading, in: 13th International Conference on Document Analysis and Recog...

    2015

  53. [61]

    Singh, G

    A. Singh, G. Pang, M. Toh, J. Huang, W. Galuba, T. Hassner, Textocr: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text, in: IEEE Con- 32 ference on Computer Vision and Pattern Recognition, CVPR, 2021, pp. 8802– 8812

    2021

  54. [62]

    C. Chng, C. S. Chan, C. Liu, Total-text: toward orientation robustness in scene text detection, Int. J. Document Anal. Recognit. (2020) 31–52

    2020

  55. [63]

    S. Long, S. Qin, D. Panteleev, A. Bissacco, Y . Fujii, M. Raptis, ICDAR 2023 competition on hierarchical text detection and recognition, in: Document Analy- sis and Recognition - ICDAR, 2023, pp. 483–497

    2023

  56. [64]

    Zhang, L

    Y . Zhang, L. Gueguen, I. Zharkov, P. Zhang, K. Seifert, B. Kadlec, Uber-text: A large-scale dataset for optical character recognition from street-level imagery, in: SUNw: Scene Understanding Workshop-CVPR, 2017, p. 5

    2017

  57. [65]

    C. K. Chng, E. Ding, J. Liu, D. Karatzas, C. S. Chan, L. Jin, Y . Liu, Y . Sun, C. C. Ng, C. Luo, Z. Ni, C. Fang, S. Zhang, J. Han, ICDAR2019 robust reading challenge on arbitrary-shaped text - rrc-art, in: 2019 International Conference on Document Analysis and Recognition, IC...

    2019

  58. [66]

    B. Shi, C. Yao, M. Liao, M. Yang, P. Xu, L. Cui, S. J. Belongie, S. Lu, X. Bai, ICDAR2017 competition on reading chinese text in the wild (RCTW-17), in: 14th IAPR International Conference on Document Analysis and Recognition, ICDAR, 2017, pp. 1429–1434

    2017

  59. [67]

    Zhang, M

    R. Zhang, M. Yang, X. Bai, B. Shi, D. Karatzas, S. Lu, C. V . Jawahar, Y . Zhou, Q. Jiang, Q. Song, N. Li, K. Zhou, L. Wang, D. Wang, M. Liao, ICDAR 2019 robust reading challenge on reading chinese text on signboard, in: 2019 Inter- national Conference on Document Analysis and...

    2019

  60. [68]

    Y . Sun, D. Karatzas, C. S. Chan, L. Jin, Z. Ni, C. K. Chng, Y . Liu, C. Luo, C. C. Ng, J. Han, E. Ding, J. Liu, ICDAR 2019 competition on large-scale street view text with partial labeling - RRC-LSVT, in: 2019 International Conference on Document Analysis and Recognition, ICD...

    2019

  61. [70]

    Nayef, C

    N. Nayef, C. Liu, J. Ogier, Y . Patel, M. Busta, P. N. Chowdhury, D. Karatzas, W. Khlif, J. Matas, U. Pal, J. Burie, ICDAR2019 robust reading challenge on multi-lingual scene text detection and recognition - RRC-MLT-2019, in: 2019 International Conference on Document Analysis ...

    2019

  62. [71]

    C. Liu, L. Jin, Y . Liu, C. Luo, B. Chen, F. Guo, K. Ding, Don’t forget me: Accu- rate background recovery for text removal via modeling local-global context, in: Computer Vision - ECCV, 2022, pp. 409–426

    2022

  63. [72]

    X. Lin, J. He, Z. Chen, Z. Lyu, B. Fei, B. Dai, W. Ouyang, Y . Qiao, C. Dong, Diffbir: Towards blind image restoration with generative di ffusion prior, arXiv preprint arXiv:2308.15070 (2023)

    2023 arXiv

  64. [73]

    A. Veit, T. Matera, L. Neumann, J. Matas, S. J. Belongie, Coco-text: Dataset and benchmark for text detection and recognition in natural images, arXiv preprint arXiv:1601.07140 (2016)

    2016 arXiv

  65. [74]

    Heusel, H

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, S. Hochreiter, Gans trained by a two time-scale update rule converge to a local nash equilibrium, in: Ad- vances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Ne...

    2017

  66. [75]

    J. Li, D. Li, S. Savarese, S. Hoi, Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models, in: International conference on machine learning, PMLR, 2023, pp. 19730–19742

    2023

  67. [76]

    Z. Su, W. Liu, Z. Yu, D. Hu, Q. Liao, Q. Tian, M. Pietik ¨ainen, L. Liu, Pixel dif- 34 ference networks for efficient edge detection, in: IEEE /CVF International Con- ference on Computer Vision, ICCV, 2021, pp. 5097–5107

    2021

  68. [77]

    Mishra, K

    A. Mishra, K. Alahari, C. V . Jawahar, Scene text recognition using higher order language priors, in: British Machine Vision Conference, BMVC, 2012, pp. 1–11

    2012

  69. [78]

    K. Wang, B. Babenko, S. J. Belongie, End-to-end scene text recognition, in: IEEE International Conference on Computer Vision, ICCV, 2011, pp. 1457–1464

    2011

  70. [79]

    Risnumawan, P

    A. Risnumawan, P. Shivakumara, C. S. Chan, C. L. Tan, A robust arbitrary text detection system for natural scene images, Expert Syst. Appl. (2014) 8027–8048. 35

    2014

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.