{"id":"7d475178-9de0-4401-91c5-c33bcfd92d57","arxiv_id":"2412.17225","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A character-level multimodal encoder plus a glyph-aware perceptual loss lets a diffusion model render text more accurately, beating prior methods by 6 to 9 points on English and Chinese benchmarks.","lead":"CharGen is a new AI model that generates images containing accurate text, such as signs, posters, or book covers, by reading the text character by character instead of as a whole block. It improves text spelling accuracy on public benchmarks by roughly 6 to 9 percentage points over prior methods, which matters for making AI image tools produce readable and correctly spelled text.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains over AnyText may stem from Long-CLIP rather than the proposed character-level encoder and CharGen loss; no ablation controls for this confound.","rationale":"The reader's weakest assumption targets the ODM loss's sensitivity to stroke-level errors, which is a genuine risk: if ODM features are not sensitive to glyph shape, the CharGen loss may not provide the described supervision, and the 6.7% ablation gain in Table 4 could stem from other effects. However, the single most load-bearing issue for the central claim is the Long-CLIP confound. The headline numbers compare an integrated system (CharGen with Long-CLIP) against AnyText, which uses vanilla CLIP. Long-CLIP is a known, pretrained improvement that extends the token limit from 77 to 248 and can independently boost text-rendering accuracy, especially on benchmarks with longer captions. The paper presents Long-CLIP as a design choice, not as a contribution, and no experiment varies it. Table 4's ablations hold Long-CLIP constant, so they establish that the character-level encoder and CharGen loss help within a Long-CLIP system, but they do not establish that those components are the reason CharGen outperforms AnyText. A controlled swap of the text encoder is inexpensive and would settle the attribution. The CONDITIONAL verdict remains appropriate because the concern is testable and the ablation evidence still suggests the proposed components have some positive effect; the experiment is needed before accepting the causal claim. I partially agree with the reader: the ODM assumption is also worth testing, but the Long-CLIP confound is more fundamental to the paper's headline improvements.","tokens_in":10987,"tokens_out":8931,"duration_ms":84156,"concrete_test":"Retrain CharGen with the original OpenAI CLIP text encoder (77-token limit) in place of Long-CLIP, keeping all other settings (training data, epochs, ControlNet, CharGen loss, inference protocol) fixed, and evaluate on the AnyText-benchmark. If English Sen.ACC drops from 0.8096 toward AnyText-v1.1's 0.7222 (or Chinese from 0.7499 toward 0.6943), the headline improvements are largely attributable to Long-CLIP rather than the proposed modules. Conversely, if accuracy remains above roughly 0.78 English / 0.72 Chinese, the confound is minor. A complementary check is to substitute Long-CLIP into AnyText-v1.1's pipeline without other changes; if AnyText+Long-CLIP matches or exceeds CharGen, the character-level encoder and CharGen loss contribute little independently.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the proposed character-level multimodal encoder and CharGen loss cause the reported accuracy gains. This attribution is not established because all experiments, including ablations in Table 4, use Long-CLIP as the text encoder (Section 3.2), while the main baseline AnyText-v1.1 uses the standard CLIP encoder with a 77-token limit. Long-CLIP raises the token limit to 248 and is designed to improve text understanding, which can independently increase OCR-based accuracy metrics, especially for long or multi-text prompts. The 8.8% English and 5.5% Chinese improvements over AnyText in Table 2 may therefore be substantially or entirely due to Long-CLIP, not to the character-level visual fusion or the CharGen loss. Table 4 shows the proposed components help relative to box-level baselines, but every row uses Long-CLIP, so the contribution of the text encoder is never isolated. Since the abstract attributes the gains to the proposed encoder and loss without this control, the central claim is currently unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CharGen, a ControlNet-based visual text generation and editing model built on SD1.5. Its two main contributions are a character-level multimodal encoder that fuses per-character glyph visual features with per-character text embeddings, and a multi-scale perceptual loss (CharGen loss) computed from an ODM encoder. The model is trained on AnyWord-3M and evaluated on AnyText-benchmark and MARIO-Eval, reporting large gains over AnyText-v1.1 and Glyph-ByT5, including 8.8% English and 5.5% Chinese Sentence Accuracy improvements on AnyText-benchmark. Ablations on a 200K subset attribute the gains to the character-level embedding, the multimodal fusion, and the CharGen loss.","tokens_in":11156,"tokens_out":3637,"duration_ms":34535,"significance":"If the causal attribution is correct, CharGen is a practically useful contribution to visual text generation: it combines a sensible character-level visual-text fusion with a perceptual loss targeted at glyph shapes, and the main comparisons use a controlled inference protocol (identical DDIM sampler, CFG, seed, and prompts) plus external OCR metrics. The paper also attempts a fine-grained error taxonomy (Table 1) that is useful for the community. However, the reported gains are not yet convincingly attributed to the proposed modules because the text encoder was changed relative to the main baseline, the ablations lack variance estimates, and several recent state-of-the-art baselines are missing. These issues prevent the paper from currently supporting its central claim and SOTA statements.","major_comments":[{"comment":"The central attribution claim is not established because of a text-encoder confound. All CharGen experiments, including every row of Table 4, use Long-CLIP as the text encoder, while the primary baseline AnyText-v1.1 uses standard CLIP with a 77-token limit. Long-CLIP extends the token limit to 248 and is designed to improve text understanding, so it can independently improve OCR-based metrics, especially for long or multi-text prompts. Since Table 4 has no row that replaces Long-CLIP with standard CLIP, or equivalently re-runs AnyText with Long-CLIP, the abstract's claim that the proposed character-level encoder and CharGen loss cause the 8.8% English and 5.5% Chinese gains is not supported. Please add a control that isolates the text-encoder choice.","section":"Section 3.2, Table 4"},{"comment":"The ablation experiments are performed once on a 200K subset with no reported training seeds or variance estimates. Several of the component differences used to support the design, such as 0.535 vs. 0.572 (+3.7%) for multimodal fusion and 0.620 vs. 0.639 (+1.9%) for the full model, are small enough that single-run noise could affect the conclusions. Please report results over multiple seeds, or at least provide confidence intervals, so the component contributions can be assessed reliably.","section":"Section 4.4, Table 4"},{"comment":"The CharGen perceptual loss assumes that the pretrained ODM encoder's destylized features are sensitive to fine-grained glyph errors (added/dropped strokes, misshapen characters) while being invariant to font style. No supporting analysis of ODM features is presented, such as a perturbation study or a visualization showing that ODM feature distances increase for stroke-level errors but not for style changes. Without such evidence, the 6.7% ablation gain from this loss (Exp. 5 vs. 7 in Table 4) could in principle reflect style alignment rather than glyph accuracy, and the use of an encoder co-authored by two CharGen authors makes this point particularly important to address.","section":"Section 3.3, Eq. (2)"},{"comment":"The claim of state-of-the-art performance is weakened by the omission of several recent baselines. In particular, Glyph-ByT5-v2, a 2024 method directly aimed at accurate multilingual visual text rendering, is cited in the paper but not included in either comparison table, and Glyph-ByT5 is not evaluated on AnyText-benchmark. To support the 'state-of-the-art' and 'recent methods' statements, please include these baselines or explicitly justify their exclusion.","section":"Tables 2 and 3"}],"minor_comments":[{"comment":"The phrase 'High Accurate' should be 'High-Accuracy' for grammatical correctness.","section":"Abstract"},{"comment":"The summation notation 'kX' and 'h,wX' appears to be a formatting artifact; please rewrite the equation with explicit sums over layers and spatial positions.","section":"Section 3.3, Eq. (2)"},{"comment":"The text 'Multi-model Encoder' is inconsistent with the term 'multimodal' used elsewhere in the paper; please unify the terminology.","section":"Figure 2"},{"comment":"References [34] and [35] both cite the same ControlNet paper; please merge or distinguish them appropriately.","section":"References"},{"comment":"In the bullet listing the CharGen perceptual loss comparisons, the phrase 'Exp.5 and Exp.7 highlight' should be 'Exp. 5 and Exp. 7 highlight' for consistency with the table notation.","section":"Section 4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically sound in its controlled evaluation protocol, but the Long-CLIP confound is a genuine threat to the central attribution claim and should be resolved with an additional control experiment. The ODM perceptual loss also deserves a careful independence analysis. The missing recent baselines are likely to be raised by external reviewers as well; including them would substantially strengthen the SOTA claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: the paper's real contribution is the character-level multimodal encoder plus an ODM-based perceptual loss, and the ablation shows they help. But the paper compares against AnyText-v1.1 with a different text encoder (standard CLIP) while CharGen uses Long-CLIP, so the 8.8% English and 5.5% Chinese gains are not cleanly attributable to the proposed components. That is the main thing to know.\n\nWhat is actually new: the character-level encoder, fusing per-character glyph visual features (PP-OCRv3) with per-character text embeddings, is a genuine departure from box-level or line-level glyph encoders in AnyText and GlyphControl. The ablation in Table 4 is structured and shows a clear progression: character-level over box-level, multimodal over visual-only, ODM loss over OCR loss. The inference protocol is controlled (same sampler, CFG, seed, prompts). That is real evidence that the proposed pieces do something.\n\nSoft spots: the Long-CLIP confound is serious. Every CharGen row in Tables 2 and 4 uses Long-CLIP, while AnyText-v1.1 uses standard CLIP with a 77-token limit. Long-CLIP is designed to improve text understanding and can independently raise OCR accuracy, especially for long or multi-text prompts. The paper never isolates the text encoder. The abstract attributes the gains to the proposed encoder and loss without that control. This is a load-bearing omission. It does not kill the ablation logic, since all rows share Long-CLIP, but the headline SOTA claim is currently unsupported. Adding AnyText+Long-CLIP or a CharGen-with-CLIP row would fix it. Also: ODM is co-authored by two CharGen authors, which is a self-citation, but the evaluation uses external OCR, so it is not circular in a damaging way. Lambda is selected on the Wukong test set, which is mild benchmark tuning. Minor: no error bars, no code, and a few missing recent baselines (Glyph-ByT5-v2, TextDiffuser-2 on the AnyText-benchmark).\n\nWho this is for: people working on visual text generation who want a concrete recipe. It deserves a serious referee, conditionally; the confound is addressable and the core mechanism is plausible. I would send it to review and ask for the Long-CLIP control, code, and variance estimates.","headline":"CharGen has a plausible character-level encoder and a helpful ablation, but the headline gains over AnyText are confounded by swapping in Long-CLIP, so the central attribution is not yet established.","tokens_in":11739,"tokens_out":2130,"would_cite":false,"duration_ms":16848,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CharGen pairs character-by-character glyph encoding with a destylization perceptual loss to reduce stroke and spelling errors in diffusion-generated visual text.","keywords":["visual text generation","character-level multimodal encoder","glyph images","perceptual loss","OCR destylization modeling","Chinese text rendering","diffusion models","ControlNet"],"falsifier":"Render the same correct Chinese character in two different fonts and render two characters that differ by one missing stroke (e.g., 词 versus 词 with the left component removed), then measure the ODM feature distance between each pair; if the style pair distance is not clearly smaller than the stroke-deletion distance, the loss cannot be providing the stroke-level, style-invariant supervision the paper claims.","tokens_in":10779,"feed_emoji":"🔤","tokens_out":7163,"duration_ms":56643,"temperature":0.7,"pith_summary":"The paper proposes CharGen, a visual text generation and editing model built on ControlNet, and argues that two changes make diffusion-based text rendering markedly more accurate. First, instead of encoding a whole line of text as one visual token, it renders each character alone into a glyph image, encodes those glyphs character by character, and fuses the visual features with per-character text embeddings before injecting them into the diffusion U-Net. Second, it replaces OCR-style perceptual losses with a multi-scale loss computed from the features of a pretrained destylization encoder, which is designed to strip away font style and expose stroke-level glyph structure. On the AnyText-benchmark, the method reports English sentence accuracy of 80.96% (up 8.8 points over AnyText-v1.1) and Chinese accuracy of 74.99% (up 5.5 points), with gains on MARIO-Eval as well. The paper's central claim is that the character-level multimodal encoding and the destylization loss, not the extra training data or model capacity, are what drive these accuracy gains.","feed_headline":"Glyph-by-glyph encoding lifts diffusion text accuracy by 8.8 points","feed_subtitle":"Character-level glyph encoding plus a style-invariant loss cuts spelling and stroke errors in text-to-image output.","key_machinery":"The two load-bearing components are the character-level multimodal encoder and the CharGen loss. The encoder renders each input character individually (80x80 pixels, Arial Unicode font), encodes the glyph with PP-OCRv3 to obtain visual embeddings, and encodes the same character with Long-CLIP to obtain text embeddings; two decoupled linear mappers fuse the two modalities into a per-character multimodal embedding that is concatenated with the prompt embedding and passed to a transformer text encoder before cross-attending to the U-Net. The CharGen loss (Eq. 2) extracts multi-scale features from the original and predicted images with the frozen ODM encoder—a pretrained text-image alignment model whose destylization objective restores clean glyphs while ignoring style, color, and texture—and penalizes the averaged L2 distance over spatial positions at each scale. The first component supplies the missing stroke-level conditions; the second supplies supervision that is sensitive to glyph shape errors but invariant to the font used.","core_discovery":"CharGen's central discovery is that visual text generation fails mainly because conventional pipelines compress glyph information at the wrong granularity and supervise it in the wrong feature space. The authors show that a character-level multimodal encoder—encoding each rendered character glyph independently with a visual encoder and combining it with the character's text embedding through learned mappers—provides the diffusion backbone with the fine-grained cross-modal conditions needed to render complex words and multi-stroke characters, and that a multi-scale perceptual loss based on an OCR destylization encoder (ODM) supervises glyph shape while remaining insensitive to font style. The paper reports that this combination outperforms prior methods on both Chinese and English public benchmarks, resolving error categories such as added/dropped strokes, repeated glyphs, merged glyphs, and misshapen characters that earlier methods leave unsolved.","pith_inferences":["We infer that a pure-glyph variant of CharGen, dropping the per-character text branch, may be sufficient for glyph-heavy prompts, since the ablation shows visual embedding contributes far more than text embedding (Exp. 4 vs Exp. 5).","The destylization loss could find use outside generation, as a training signal for text-shape faithfulness in OCR-guided inpainting, video text rendering, or font synthesis systems.","A testable extension the paper does not run: feed the ODM encoder two images of the same word in different fonts and confirm that the CharGen loss stays near zero, and feed images with one stroke altered and confirm the loss increases; that would directly validate the style-invariance assumption.","A practical deployment consideration is that the per-character visual encoder and the ODM loss add inference-time complexity; the paper does not measure the added computational cost per generation."],"forward_implications":["Existing ControlNet-based visual text generation systems can adopt the character-level multimodal encoder and CharGen loss as plug-in components, since CharGen's design does not change the base diffusion model.","The method should transfer to other scripts with complex glyph structures, such as Arabic, Devanagari, and Korean, because the encoder operates at the character level rather than the language-model token level.","Because the CharGen loss supervises in a destylized feature space, generated text style is intended to follow the scene description rather than the training-render font, improving consistency between text and background style.","The benchmark results imply that character-level granularity plus style-invariant perceptual supervision is the key to eliminating added strokes, dropped strokes, and merged glyphs, not larger training data alone, since CharGen trains on 1.39 million English images yet beats models trained on 10 million."],"supporting_citations":[{"why":"Supplies the pretrained ODM encoder whose destylized features define the CharGen perceptual loss.","marker":"[7]"},{"why":"Provides the AnyWord-3M training set, the AnyText-benchmark, and the AnyText-v1.1 baseline that CharGen compares against and outperforms.","marker":"[31]"},{"why":"The ControlNet architecture that CharGen builds upon for injecting image-based and text-based conditions into the diffusion model.","marker":"[34]"},{"why":"Long-CLIP raises the text token limit from 77 to 248, enabling longer prompts while staying in the CLIP latent space.","marker":"[33]"},{"why":"PP-OCRv3 acts as the character-wise glyph visual encoder that extracts visual embeddings from each rendered glyph image.","marker":"[13]"},{"why":"Glyph-ByT5 is the strongest previous method on MARIO-Eval and serves as the main comparison baseline for image-level accuracy.","marker":"[16]"},{"why":"TextDiffuser is a key baseline on both benchmarks and represents the character-segmentation-mask approach that CharGen goes beyond.","marker":"[5]"},{"why":"GlyphControl contributes the glyph-image rendering strategy with ControlNet that CharGen adapts for its per-character glyph inputs.","marker":"[32]"}],"fun_headline_variants":["CharGen: encoding each glyph separately sharpens text-to-image output","Perceptual loss on glyph shapes reduces stroke errors in visual text","Fine-grained character encoding lifts diffusion text accuracy 8%","Glyph-level multimodal encoding fixes misshapen characters in text generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy gains attributed to the CharGen loss depend on the assumption that the ODM encoder's destylized features are sensitive to fine-grained glyph shape errors (added or dropped strokes, misshapen characters) while remaining insensitive to font style; if those features miss stroke-level errors or encode style, the reported 6.7% ablation gain from the loss would not be a true glyph-accuracy gain.","fun_headline_variants_meta":{"raw":{"variants":["CharGen: encoding each glyph separately sharpens text-to-image output","Perceptual loss on glyph shapes reduces stroke errors in visual text","Fine-grained character encoding lifts diffusion text accuracy 8%","Glyph-level multimodal encoding fixes misshapen characters in text generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000557,"raw_usage":{"total_tokens":2627,"prompt_tokens":902,"completion_tokens":1725,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1653}},"tokens_in":518,"tokens_out":1725,"duration_ms":13368,"temperature":1.0,"reasoning_tokens":1653,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:41:29.849886+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render the same correct Chinese character in two different fonts and render two characters that differ by one missing stroke (e.g., 词 versus 词 with the left component removed), then measure the ODM feature distance between each pair; if the style pair distance is not clearly smaller than the stroke-deletion distance, the loss cannot be providing the stroke-level, style-invariant supervision the paper claims.","supporting_citations":[{"cited_title":"Odm: A text-image further alignment pre-training approach for scene text detection and spotting","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained ODM encoder whose destylized features define the CharGen perceptual loss."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"The ControlNet architecture that CharGen builds upon for injecting image-based and text-based conditions into the diffusion model."},{"cited_title":"Long-clip: Unlocking the long-text capability of clip, 2024","cited_arxiv_id":null,"evidence_quote":"Long-CLIP raises the text token limit from 77 to 248, enabling longer prompts while staying in the CLIP latent space."},{"cited_title":"Pp-ocrv3: More attempts for the improvement of ultra lightweight ocr system,","cited_arxiv_id":null,"evidence_quote":"PP-OCRv3 acts as the character-wise glyph visual encoder that extracts visual embeddings from each rendered glyph image."},{"cited_title":"Glyph-ByT5: A Cus- tomized Text Encoder for Accurate Visual Text Rendering,","cited_arxiv_id":null,"evidence_quote":"Glyph-ByT5 is the strongest previous method on MARIO-Eval and serves as the main comparison baseline for image-level accuracy."},{"cited_title":"Textdiffuser: Diffusion models as text painters","cited_arxiv_id":null,"evidence_quote":"TextDiffuser is a key baseline on both benchmarks and represents the character-segmentation-mask approach that CharGen goes beyond."},{"cited_title":"Glyphcontrol: Glyph conditional control for visual text generation","cited_arxiv_id":null,"evidence_quote":"GlyphControl contributes the glyph-image rendering strategy with ControlNet that CharGen adapts for its per-character glyph inputs."}],"review_version":1}