Pith. sign in

REVIEW 5 major objections 6 minor 19 references

Syn3DTxt: Embedding 3D Cues for Scene Text Generation

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Encoding surface normals as RGB masks in synthetic data gives scene text editing models the geometric context they need, lifting SSIM by 15%, FID by 17.7%, and accuracy by 72.1% over 2D baselines.

desk verdict Novel dataset and plausible 3D-cue idea, but the central comparison is confounded and the abstract numbers are inconsistent; worth reviewing with major-revision expectations. read the letter →

arxiv 2505.18479 v1 pith:6522XUG2 submitted 2025-05-24 cs.CV

Li-Syun Hsiung , Jun-Kai Tu , Kuan-Wu Chu , Yu-Hsuan Chiu , Yan-Tsung Peng , Sheng-Luen Chung , Gee-Sern Jison Hsu This is my paper · ORCID
classification cs.CV
keywords scenetexteditingsyntheticdatasetsurfacenormalmasks3Dgeometriccuesperspectiveconsistencyrenderingcurved
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that scene text editors fail on real-world images because their training data flattens text into 2D masks, hiding the perspective, rotation, and curvature that text has in physical scenes. It proposes a new data standard: synthetic images whose text masks are RGB-coded surface normal maps rather than binary foreground maps. Models trained on this 3D-augmented data, using the MOSTEL architecture with a widened output head, are claimed to achieve 15% better SSIM, 17.7% better FID, and 72.1% better word accuracy than 2D-trained baselines. If this holds, the field gains a cheap recipe: keep the model and pipeline, and change the mask to carry geometric cues.

What carries the argument

The central object is the surface-normal RGB mask: a text-region mask in which each pixel's color encodes the direction the surface faces at that point, via a spherical-to-RGB mapping. The masks are produced by a generator that controls rotation order (roll before pitch and yaw, with pitch/yaw order depending on viewing distance) and applies per-character bending so adjacent characters carry different normals. This gives the training signal a per-pixel geometric meaning that a binary mask lacks, and it is the channel through which the paper claims the model learns perspective and curvature.

What would settle it

Run the paper's own control experiment with matched capacity: give the 2D control model the same three-channel decoder but feed it binary masks repeated across channels, or random RGB masks, through the identical fine-tuning pipeline on Syn3DTxt. If that control matches the 3D-mask model's SSIM, FID, and accuracy, then the surface-normal geometry is not the active ingredient; if it does not, the geometric cue is confirmed.

Watch

Extended reading notes

Core claim

The paper claims that surface-normal maps, encoded as RGB colors, are the missing supervision for scene text editing. It builds a synthetic data engine that renders text in 3D with controlled single-, dual-, and triple-axis rotations and arc curvature, then represents each pixel's surface orientation through the spherical-coordinate mapping $(R,G,B)=(\sin\theta\cos\phi,\sin\theta\sin\phi,\cos\theta)$ in the mask. Using MOSTEL as the backbone, with its decoder extended from one to three output channels, the paper reports that fine-tuning on the Syn3DTxt datasets improves perspective-consistent text editing over 2D-mask baselines by 15% in SSIM, 17.7% in FID, and 72.1% in accuracy, including on the real-world ScenePair set.

Load-bearing premise

The controlled comparison assumes the two fine-tuned models differ only in the geometric information inside their training masks, but the 3D model's output head was enlarged from one to three channels and the paper does not say whether the 2D control also had the larger head.

Editorial extensions

If this is right

  • Training on Syn3DTxt with surface-normal RGB masks improves perspective-consistent scene text editing over 2D-mask training, with the paper reporting 15% higher SSIM, 17.7% better FID, and 72.1% higher accuracy on its benchmarks.
  • The improvement carries to a real-world out-of-domain evaluation set, ScenePair, where the 3D-fine-tuned model lifts word accuracy from 37.69% to 67.58% compared with the reproduced MOSTEL baseline.
  • Fine-tuning on the 3D-augmented data is a practical route to geometric awareness: it reuses the pretrained 2D model and requires no new architecture beyond a wider output head.
  • The paper's data generator can produce large, diverse rotated and curved text samples with normal masks, and the toolkit plus datasets are released for other methods to train on.
  • Despite the gains, editing text with highly arbitrary shapes and complex curvature remains an acknowledged limitation of the approach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test the paper does not run is a matched-capacity control with three-channel binary masks; it would isolate whether the gain comes from the geometric cue or from the larger output head.
  • The same surface-normal RGB masks could serve as supervision for other geometry-grounded tasks, such as normal estimation or relighting, since they encode per-pixel orientation in a standard color map; the paper only evaluates text editing.
  • Feeding the normal masks to a diffusion-based editor rather than the GAN-style MOSTEL would test whether the geometric cue transfers across architectures; the paper tests only MOSTEL variants.
  • The rotation-angle distribution in the dataset is manually chosen, so an automatic sampler fit to real signboard viewpoint statistics might generalize even further; this is a testable extension, not a claim of the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper introduces Syn3DTxt, a synthetic scene-text dataset in which each rendered text image is accompanied by an RGB mask encoding surface normals, together with a curved/rotated generation pipeline. The authors modify MOSTEL's decoder to output three channels and use a three-phase training schedule (baseline, 3D fine-tuning, curvature fine-tuning). They report quantitative gains over SRNet, TextCtrl, and MOSTEL on Syn3DTxt-eval, Tamper-Syn2k, and ScenePair, and claim 15% SSIM, 17.7% FID, and 72.1% accuracy improvements.

Significance. The idea of supervising scene-text editing with explicit 3D surface-normal cues is timely, and the external ScenePair results (ACC 67.58 vs 41.41 for the 2D-finetuned control) suggest the approach has real potential. The dataset and toolkit release would be a useful community asset if the generation code is made public. However, the headline comparisons are currently underdetermined by an architecture confound and possible test-set leakage, and several abstract-level numbers are not supported by the tables, so the significance cannot be fully assessed from the present manuscript.

major comments (5)
  1. [§4.2, Tables 3–4] The central 2D-vs-3D comparison is confounded. The decoder is changed from one output channel to three "to accommodate richer geometric representations", but the text never states whether the "MOSTEL + 2D Finetuned" control also uses the three-channel decoder or keeps a one-channel head. If the control has a smaller output head, the gains (e.g., ScenePair ACC 67.58 vs 41.41) may come from added capacity rather than from surface-normal cues. Please specify the exact decoder configuration of both models and, ideally, include a matched-capacity control (e.g., a three-channel decoder fed a broadcast binary mask).
  2. [§4.1, §4.2] The Syn3DTxt-wrap evaluation appears to leak into training. Section 4.1 describes Syn3DTxt-wrap-2k only as a 2,000-image test set, while Section 4.2, Phase 3, says the model is "fine-tuned using the Syn3DTxt-wrap dataset". If the 2k evaluation images are the same ones used for fine-tuning, the Syn3DTxt-wrap row of Table 3 is invalid. Please state the size and composition of the training split of Syn3DTxt-wrap and confirm that the test subset is disjoint.
  3. [Abstract, §1, Tables 3–4] The quantitative claims in the abstract and introduction do not match the tables. "Perspective-Aware SSIM" is never defined in §4.3; the relative SSIM gain on Syn3DTxt-eval-2k is 13.0% (0.7215 to 0.8151), not 15%, and the FID gain there is 12.2%, not 17.7%. On Tamper-Syn2k the 3D-finetuned model's FID (35.277) is worse than the reproduced MOSTEL baseline (29.484), contradicting the claim of consistent improvement across all metrics. The reported "72.1% in Accuracy" does not appear in Table 4, where the best ACC values are 67.58 and 71.25. Please correct the abstract and specify how each reported percentage is computed (relative vs absolute, and over which dataset).
  4. [§3.1, Eqs. (1)–(3)] The rotation definitions are internally inconsistent. The text says Tx, Ty, Tz correspond to rotations about X, Y, Z and to roll, pitch, yaw, but Eq. (1) is a Z-axis rotation, Eq. (2) is a Y-axis rotation, and Eq. (3) is an X-axis rotation; this also disagrees with the standard assignment roll=X, pitch=Y, yaw=Z. Since these rotations define the geometric content of the dataset, the exact intended transformation order and axis conventions must be stated unambiguously and matched in the released generator.
  5. [Tables 3–4, §4.2] The proposed incremental training strategy is not consistently better than training from scratch. On ScenePair, "MOSTEL 3D from scratch" outperforms "MOSTEL + 3D Finetuned" in ACC (71.25 vs 67.58), SSIM (0.4185 vs 0.4074), and MSE; on Syn3DTxt-eval-2k and Syn3DTxt-eval-advanced, the from-scratch model has better FID. The claims about the value of the three-phase strategy should be moderated or supported with an ablation that isolates each phase.
minor comments (6)
  1. [§3.2 vs §4.1] The dataset size is given as "over 200k paired training samples and 6k testing samples" in Section 3.2 but "150,000 images" in Section 4.1; please reconcile these numbers.
  2. [§4.3] The heading reads "Evaluation Metries", and the metric list does not include Perspective-Aware SSIM despite its use in the abstract.
  3. [§4.4] The statement "averaging improvements of approximately 10 percentage points" is not derived; please report the per-dataset gains and specify how the average is computed.
  4. [Fig. 6 caption] The phrase "likely along the θ-axis" should be replaced with the exact axis used in the experiment.
  5. [Table 3] TextCtrl has no entries for Tamper-Syn2k; please explain why this baseline is unavailable there.
  6. [Tables 1–2] The rotation angle categories are given as ranges (e.g., 45° to 60°) but the sampling distribution within each category is unspecified, which is needed for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 3D-vs-2D comparison is a controlled empirical test on external benchmarks, and no result reduces to its own inputs by construction.

full rationale

The paper's central claim is that RGB surface-normal masks provide useful 3D geometric supervision for scene text editing. This is tested by comparing MOSTEL + 2D Finetuned against MOSTEL + 3D Finetuned, where the only intended difference is the mask modality used during fine-tuning. The evaluation includes external benchmarks (Tamper-Syn2k from MOSTEL and ScenePair from TextCtrl) on which the 3D-finetuned model also improves, so the result does not rest solely on self-generated test data. The Syn3DTxt-eval sets are produced by the same pipeline as the training data, which limits the external-generalization strength of those numbers, but this is an in-domain evaluation design, not a circular reduction: at inference the models do not receive the RGB normal masks, and the comparison still discriminates between the 2D and 3D training cues. No parameter is fitted to the target metrics; the rotation-angle distributions are hand-chosen rather than optimized to maximize reported scores. The abstract's reported percentages (15% / 17.7% / 72.1%) do not exactly match the tables, and 'Perspective-Aware SSIM' is not defined in Sec. 4.3, but these are reporting/overstatement issues, not circularity. The potential decoder-channel confound in Sec. 4.2 (1-channel vs 3-channel output head) is an experimental-validity concern that could undercut causal attribution, but it is not a case where the prediction is equivalent to the input by definition or by self-citation. There is no load-bearing self-citation chain or imported uniqueness theorem. The derivation is self-contained as an empirical comparison, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen dataset generation parameters (rotation angle distributions, curvature levels) and on domain assumptions about the usefulness of RGB normal masks. No new physical entities are introduced. The paper does not ablate the training phases, so the necessity of its three-phase strategy is assumed.

free parameters (3)
  • Rotation distribution percentages = 20, 20, 20, 20, 5, 5, 10 (percent)
    Hand-chosen in Tab. 1 to mimic assumed real-world frequency of rotation axis combinations; no empirical or theoretical justification.
  • Rotation angle categories = 30, 45-60, 65-70 (degrees)
    Hand-chosen in Tab. 2 based on 'visual inspection after coordinate calculations' to define small, medium, and large rotations.
  • Arc distortion levels = 0, 60, 120 (degrees)
    Three curvature levels chosen without justification; these shape the distribution of curved text in the dataset.
assumptions (4)
  • domain assumption The RGB mapping in Eq. (5) (R = sin theta cos phi, G = sin theta sin phi, B = cos theta) is an invertible and informative encoding of surface orientation.
    The paper assumes this encoding conveys enough geometric information to the decoder; no analysis of information loss or perceptual equivalence is given.
  • domain assumption The roll/pitch/yaw ordering convention (roll before pitch/yaw, pitch before yaw for close-ups) matches real-world viewpoint changes.
    Sec. 3.1 justifies this by an intuitive visual argument and Eq. (4), but it is a modeling choice that shapes the dataset distribution.
  • ad hoc to paper Three-phase incremental training (baseline, 3D finetune, curvature) is required; each phase contributes to the gain.
    No ablation isolates the contribution of each phase; the training strategy is presented as the method without testing alternatives.
  • standard math Standard rotation matrices and spherical coordinates are used correctly despite inconsistent labeling of theta and phi in Eqs. (1)-(3).
    The notation swaps names (Ty uses cos phi, Tz uses cos theta) though the prose says Ty modifies pitch theta and Tz yaw phi; if the actual code follows the matrices, the labels are just confusing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Syn3DTxt: Embedding 3D Cues for Scene Text Generation." pith.science (2026). https://pith.science/paper/6522XUG2

@misc{pith2026250518479,
  author       = {Pith},
  title        = {Pith review of: Syn3DTxt: Embedding 3D Cues for Scene Text Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6522XUG2}},
  note         = {Machine review of arXiv:2505.18479}
}
read the original abstract

This study aims to investigate the challenge of insufficient three-dimensional context in synthetic datasets for scene text rendering. Although recent advances in diffusion models and related techniques have improved certain aspects of scene text generation, most existing approaches continue to rely on 2D data, sourcing authentic training examples from movie posters and book covers, which limits their ability to capture the complex interactions among spatial layout and visual effects in real-world scenes. In particular, traditional 2D datasets do not provide the necessary geometric cues for accurately embedding text into diverse backgrounds. To address this limitation, we propose a novel standard for constructing synthetic datasets that incorporates surface normals to enrich three-dimensional scene characteristic. By adding surface normals to conventional 2D data, our approach aims to enhance the representation of spatial relationships and provide a more robust foundation for future scene text rendering methods. Extensive experiments demonstrate that datasets built under this new standard offer improved geometric context, facilitating further advancements in text rendering under complex 3D-spatial conditions.

Figures

Figures reproduced from arXiv: 2505.18479 by the authors.

Figure 1
Figure 1. Example of previous Dataset (a) MARIO-10M, constructed by [ [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visualization of RGB-encoded normal vectors within a spherical coordinate system. Each point on the sphere represents [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Example of generated text data with three [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparison between 2D and 3D models [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison between the original MOSTEL and our enhanced MOSTEL 3D on ScenePair across three repre [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Four visual examples of different models (a) Horizontal 3D Rotation Comparison, Visualization of model outputs under [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 16 canonical work pages

  1. [1]

    Textdiffuser: Diffusion models as text painters

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. Advances in Neural Information Processing Systems, 36:9353–9387, 2023. 1, 2, 3

    work page 2023

  2. [2]

    Textdiffuser-2: Unleashing the power of language models for text rendering

    Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser-2: Unleashing the power of language models for text rendering. In Eu- ropean Conference on Computer Vision, pages 386–402. Springer, 2024. 1

    work page 2024

  3. [3]

    Total-text: Towards orientation robustness in scene text detection

    Chee Kheng Ch’ng, Chee Seng Chan, and Chenglin Liu. Total-text: Towards orientation robustness in scene text detection. International Journal on Document Analysis and Recognition (IJDAR), 23:31–52, 2020. 1

    work page 2020

  4. [4]

    Synthetic data for text localisation in natural images

    Ankush Gupta, Andrea Vedaldi, and Andrew Zisserman. Synthetic data for text localisation in natural images. In IEEE Conference on Computer Vision and Pattern Recognition, 2016. 1, 3, 4

    work page 2016

  5. [5]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. 2, 7

    work page 2017

  6. [6]

    Reading text in the wild with convo- lutional neural networks

    Max Jaderberg, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Reading text in the wild with convo- lutional neural networks. International Journal of Com- puter Vision, 116(1):1–20, 2016. 3, 4

    work page 2016

  7. [7]

    Icdar 2013 robust reading competition

    Dimosthenis Karatzas, Faisal Shafait, Seiichi Uchida, Masakazu Iwamura, Lluis Gomez i Bigorda, Sergi Rob- les Mestre, Joan Mas, David Fernandez Mota, Jon Al- mazan Almazan, and Lluis Pere De Las Heras. Icdar 2013 robust reading competition. In 2013 12th interna- tional conference on document analysis and recognition, pages 1484–1493. IEEE, 2013. 6

    work page 2013

  8. [8]

    First creating backgrounds then rendering texts: A new paradigm for visual text blending

    Zhenhang Li, Yan Shu, Weichao Zeng, Dongbao Yang, and Yu Zhou. First creating backgrounds then rendering texts: A new paradigm for visual text blending. arXiv preprint arXiv:2410.10168, 2024. 1

    arXiv 2024

Show all 19 references
  1. [9]

    Synthtext3d: synthesizing scene text images from 3d virtual worlds

    Minghui Liao, Boyu Song, Shangbang Long, Minghang He, Cong Yao, and Xiang Bai. Synthtext3d: synthesizing scene text images from 3d virtual worlds. Science China Information Sciences, 63:1–14, 2020. 1, 3, 4

    2020

  2. [10]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th Eu- ropean conference, zurich, Switzerland, September 6-12, 2014, proceed...

    2014

  3. [11]

    Icdar 2023 competition on hierarchical text detec- tion and recognition

    Shangbang Long, Siyang Qin, Dmitry Panteleev, Alessandro Bissacco, Yasuhisa Fujii, and Michalis Rap- tis. Icdar 2023 competition on hierarchical text detec- tion and recognition. In International Conference on Document Analysis and Recognition , pages 483–497. Springer, 2023. 6

    2023

  4. [12]

    Glyph- draw: Seamlessly rendering text with intricate spatial structures in text-to-image generation

    Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, and Xiaodong Lin. Glyph- draw: Seamlessly rendering text with intricate spatial structures in text-to-image generation. arXiv preprint arXiv:2303.17870, 2023. 1

    2023 arXiv

  5. [13]

    Icdar2017 robust reading challenge on multi-lingual scene text de- tection and script identification-rrc-mlt

    Nibal Nayef, Fei Yin, Imen Bizid, Hyunsoo Choi, Yuan Feng, Dimosthenis Karatzas, Zhenbo Luo, Umapada Pal, Christophe Rigaud, Joseph Chazalon, et al. Icdar2017 robust reading challenge on multi-lingual scene text de- tection and script identification-rrc-mlt. In 2017 14th IAPR ...

    2017

  6. [14]

    Exploring stroke- level modifications for scene text editing

    Yadong Qu, Qingfeng Tan, Hongtao Xie, Jianjun Xu, Yuxin Wang, and Yongdong Zhang. Exploring stroke- level modifications for scene text editing. InProceedings of the AAAI Conference on Artificial Intelligence , pages 2119–2127, 2023. 1, 3, 6, 7

    2023

  7. [15]

    A robust arbitrary text detection system for natural scene images

    Anhar Risnumawan, Palaiahankote Shivakumara, Chee Seng Chan, and Chew Lim Tan. A robust arbitrary text detection system for natural scene images. Expert Systems with Applications, 41(18):8027–8048, 2014. 2

    2014

  8. [16]

    Anytext: Multilingual visual text generation and editing

    Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuansong Xie. Anytext: Multilingual visual text generation and editing. arXiv preprint arXiv:2311.03054, 2023. 1

    2023 arXiv

  9. [17]

    Editing text in the wild

    Liang Wu, Chengquan Zhang, Jiaming Liu, Junyu Han, Jingtuo Liu, Errui Ding, and Xiang Bai. Editing text in the wild. In Proceedings of the 27th ACM international conference on multimedia, pages 1500–1508, 2019. 1, 2, 3, 4, 7

    2019

  10. [18]

    Synthtiger: Synthetic text image generator towards better text recognition models, 2021

    Moonbin Yim, Yoonsik Kim, Han-Cheol Cho, and Sun- grae Park. Synthtiger: Synthetic text image generator towards better text recognition models, 2021. 3, 4

    2021

  11. [19]

    Textctrl: Diffusion-based scene text edit- ing with prior guidance control

    Weichao Zeng, Yan Shu, Zhenhang Li, Dongbao Yang, and Yu Zhou. Textctrl: Diffusion-based scene text edit- ing with prior guidance control. Advances in Neu- ral Information Processing Systems, 37:138569–138594,

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.