{"id":"122baea1-4e2b-497e-abc4-0d1565f343bd","arxiv_id":"1908.08207","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An end-to-end neural network detects and recognizes arbitrary-shape scene text using instance segmentation, character segmentation, and spatial attention, setting state-of-the-art results on several benchmarks.","lead":"Mask TextSpotter is a neural network that spots text in photos by detecting and reading it at the same time, using image segmentation and attention instead of separate detection and recognition steps. It handles curved and irregularly shaped text, not just straight text, and reports state-of-the-art results on standard benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'first completely end-to-end trainable' claim rests on an unverified classification of FOTS [46] as training recognition on ground-truth boxes; if FOTS actually back-propagates through predicted RoIs, the central novelty claim is false.","rationale":"I read Mask TextSpotter as making two central claims: (1) it is the first completely end-to-end trainable text spotter, and (2) it handles arbitrary shapes via a two-dimensional representation. The reader's weakest assumption concerns axis-aligned horizontal rectangles around curved text. That concern is real but not the most load-bearing: the mask branch explicitly segments the true polygon inside the horizontal proposal, and the strong Total-Text results suggest the assumption degrades gracefully, though a failure-mode analysis is absent. The 'first' claim is more brittle because it depends on a historical classification of prior work, specifically the assertion that FOTS trains recognition on ground-truth text regions. This is externally checkable and is not supported in the manuscript. If the characterization is wrong, the novelty claim is objectively false even though the architecture remains valuable. I therefore recommend keeping the reader's CONDITIONAL verdict while adding a prior-art verification condition. The concrete test is to trace FOTS's training graph. I disagree with the reader's choice of weakest assumption only in the sense that a different assumption is more likely to break the stated claim.","tokens_in":25010,"tokens_out":8421,"duration_ms":88808,"concrete_test":"Verify the training data flow in FOTS (Liu et al., CVPR 2018) from the paper or released code: during training, does the recognition branch receive region features extracted from the detection branch's predicted boxes via RoIRotate, with gradients flowing jointly, or from ground-truth text regions? Also check whether any other pre-2019 spotting method trains detection and recognition with a single joint loss without curriculum or alternating steps. If FOTS uses predicted proposals, Mask TextSpotter is not the first; if it uses GT crops, this particular objection is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.1 and the Introduction assert that prior spotters are 'not completely end-to-end trainable' because they 'adopt the curriculum learning paradigm [5], [26], [39], or apply the alternating training scheme [7], or train the recognition part with the ground truth text regions instead of the predicted proposals [46].' The last clause is load-bearing: it is the reason FOTS [46] is excluded and Mask TextSpotter can call itself 'the first framework that is completely end-to-end trainable for text spotting.' The paper gives no citation-level evidence for this characterization. FOTS is explicitly a unified detector plus recognizer with a differentiable RoIRotate module; whether its recognition branch consumes detection-branch proposals during training is a factual question, not an internal design choice. If FOTS is trained end-to-end on predicted text regions, the 'first' claim is false, and the Introduction's dichotomy between Mask TextSpotter and all previous spotters collapses. The rest of the paper, including mask segmentation, SAM, and the benchmarks, would remain a substantive extension, but the headline novelty claim would need to be narrowed or withdrawn.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Mask TextSpotter, an extension of the authors' ECCV 2018 work, in which text detection, character segmentation, and text-sequence recognition are integrated into a single Mask R-CNN-style network with a shared FPN backbone. Detection is performed by text-instance segmentation; recognition is performed jointly by a character-segmentation branch and a new Spatial Attention Module (SAM) that decodes sequences in two-dimensional space. The paper claims that this is the first completely end-to-end trainable text spotting framework, that it handles arbitrary-shaped (including curved) text, and that it achieves state-of-the-art results on ICDAR2013, ICDAR2015, COCO-Text, Total-Text, and MLT for detection/spotting, plus seven standard recognition benchmarks. The experiments include ablations on the attention module, position embedding, weighted edit distance, and the need for character-level annotations.","tokens_in":25199,"tokens_out":5933,"duration_ms":54095,"significance":"If the claims hold, the contribution is substantial: a single network with shared features obtains competitive or state-of-the-art detection and recognition on horizontal, oriented, and curved text, while reducing the need for character-level annotations. The paper's strengths include the breadth of evaluation (five spotting/detection datasets and seven recognition benchmarks), the explicit comparisons against the authors' conference version and ASTER under matched training data, and the ablations isolating the effects of SAM, position embedding, weighted edit distance, and real-world character annotations. The standalone recognition experiments (Table 9) are especially convincing in showing consistent gains on irregular text. However, the headline novelty claim depends on a factual characterization of prior work that is not substantiated in the manuscript, and the specification of one proposed component is incomplete.","major_comments":[{"comment":"The claim that Mask TextSpotter is 'the first framework that is completely end-to-end trainable for text spotting' rests on the sentence in §2.1 that prior methods 'train the recognition part with the ground truth text regions instead of the predicted proposals [46]' and the corresponding statement in the Introduction. This is a factual claim about how FOTS [46] trains its recognition branch, and no citation-level evidence is provided. FOTS is described in its own abstract as a unified network with a differentiable RoIRotate module, and the distinction depends entirely on whether the recognition branch consumes predicted text regions during training. If FOTS is trained on predicted proposals, the 'first' claim is false and the dichotomy in §2.1 collapses. Please verify this architectural detail against the FOTS paper or public code and either provide direct evidence for the characterization or revise the novelty claim.","section":"§2.1 and Introduction"},{"comment":"The 'arbitrary shapes' claim is implemented through axis-aligned horizontal rectangles: ground-truth polygons are converted to minimal-area horizontal rectangles in §3.5, and at inference the mask branch receives the Fast R-CNN boxes as proposals in §3.7.1. For long or strongly curved words, such a rectangle can include substantial background or cut off parts of the glyphs, but no analysis is given of how performance degrades with curvature or word length, and the failure cases in §4.11 do not address this dimension. Please add a quantitative analysis of this failure mode, for example performance stratified by curvature or by box-occupancy ratio, or qualify the claim that the method 'easily handles text instances of irregular shapes.'","section":"§3.5 and §3.7.1"},{"comment":"The weighted edit distance is a stated contribution and is credited in §4.8 with consistent gains, but the algorithm is not fully specified. The text says that the deletion, insertion, and replacement costs Cd, Ci, and Cr depend on the character probabilities pc_index, yet Eq. (20) only defines the recurrence and no formula is given for how the costs are computed from those probabilities. As written, the method cannot be reproduced. Please provide the exact cost definitions.","section":"§3.7.3, Eq. (20)"}],"minor_comments":[{"comment":"Several conclusions rest on differences of 0.1–0.3 percentage points, such as Ours versus the Conference version on ICDAR2013 word spotting with strong and weak lexicons, and the paper itself attributes these to the small test set. Since all numbers come from a single training run, please report variance across multiple runs or soften the comparative wording.","section":"§4.3, Table 2"},{"comment":"The onehot notation is under-specified: please state whether indices i and j are 0-based or 1-based, since the embedding shape (Wp+Hp, Hp, Wp) depends on this choice.","section":"§3.3.1, Eqs. (1)–(2)"},{"comment":"The notation (rx, ry) for the proposal vertices is ambiguous because r is used both as a proposal and as a matched rectangle; please rename to avoid confusion.","section":"§3.5, Eqs. (13)–(14)"},{"comment":"The citation [5] (Bengio et al., curriculum learning) is a general learning method, not a text-spotting system; the sentence 'They adopt the curriculum learning paradigm [5], [26], [39]' would be clearer if [5] were cited as the source of the paradigm rather than as an example of a text-spotting method.","section":"§2.1"},{"comment":"The text says 'extra 1162 images (SCUT)' but cites [81] (DeepText); please provide the correct reference for the SCUT dataset.","section":"§4.2.1"},{"comment":"The two failure cases are illustrative, but the section would benefit from a short discussion of the most frequent failure types and their frequency, especially in view of the paper's 'arbitrary shapes' claim.","section":"§4.11"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the FOTS characterization. I could not verify from the manuscript alone whether FOTS trains its recognition branch on ground-truth boxes; if it does not, the 'first' claim is false and the paper would need a major repositioning. The rest of the technical contribution is sound, and the experimental evidence is strong. I would not accept the paper until the authors supply the missing verification of the FOTS training scheme and the exact specification of the weighted edit distance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a competent extension of the authors' ECCV 2018 Mask TextSpotter, not a brand-new architecture. What is actually new is the Spatial Attention Module: a 2D attention decoder that reads text directly from two-dimensional features, removes the need for character-level annotations in training, and complements the character segmentation branch. That is a real improvement, and the paper is honest that it is adapting existing spatial attention ideas rather than inventing attention from scratch.\n\nStrengths: experiments are extensive and mostly supportive. Five datasets covering horizontal, oriented, curved, and multi-language text; ablations for SAM, position embedding, weighted edit distance, and annotation requirements. Standalone recognition is genuinely strong: beats ASTER on all 12 settings in Table 9, with large margins on SVTP and CUTE. Comparisons against their own conference version and ASTER are explicit and fair. The claim that arbitrary-shape spotting works survives on the tested benchmarks.\n\nSoft spots are real. The 'first completely end-to-end trainable' claim is load-bearing and under-supported. In Sec. 2.1 the paper dismisses FOTS by saying it trains recognition with ground truth regions rather than predicted proposals, but there is no citation-level or implementation-level evidence for that. If FOTS actually backpropagates through predicted RoIs, the 'first' claim is false and the intro's dichotomy collapses. The rest of the paper would remain substantive, but the headline would need to be narrowed. The stress-test concern lands.\n\nSecond, the architecture assumes a single axis-aligned horizontal rectangle can enclose every text instance (Sec. 3.5, 3.7.1). That is an obvious potential failure for very long or highly curved words; the benchmarks say it works often enough, but the paper does not analyze when it fails. Third, no error bars or multiple-run variance; several 'state-of-the-art' margins are fractions of a percent. Code is not released, which matters for reproducibility. Minor: the drop on ICDAR2013 vs the conference version is hand-waved as 'natural disturbance.'\n\nBottom line: for text spotting researchers this is a useful system paper and deserves a serious referee. It should not be desk-rejected. But before publication the authors need to verify the FOTS characterization, qualify the 'first' claim, and ideally release code or add a reproducibility appendix. I would accept it into a journal with major/minor revisions, not as is.","headline":"Solid empirical extension of the ECCV18 Mask TextSpotter, but the 'first fully end-to-end trainable' headline is only as strong as an unverified claim about FOTS.","tokens_in":25767,"tokens_out":3031,"would_cite":true,"duration_ms":32018,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a fully end-to-end trainable network can spot scene text of arbitrary shapes—horizontal, rotated, and curved—by performing both detection and recognition in two-dimensional space via shared semantic segmentation.","keywords":["scene text spotting","end-to-end training","arbitrary-shape text recognition","instance segmentation","spatial attention","character segmentation","curved text","scene text recognition"],"falsifier":"A direct test is to generate synthetic text wrapped around circles and spirals with varying radius and word length, then plot end-to-end accuracy against curvature and length; if accuracy collapses as curvature grows beyond the benchmark range, or the predicted masks clip characters at the ends of the minimal horizontal rectangle, the arbitrary-shapes claim is bounded by the proposal assumption.","tokens_in":24783,"feed_emoji":"🔤","tokens_out":6211,"duration_ms":58503,"temperature":0.7,"pith_summary":"Mask TextSpotter claims that scene text spotting, detecting and reading text in natural images, can be done by one fully end-to-end trainable network that treats both tasks as two-dimensional problems. Previous spotters chained a proposal detector to a sequence recognizer trained in stages; this paper replaces recognition with semantic segmentation of characters plus a spatial-attention decoder, so detection and recognition share features and are optimized jointly. If right, it removes the two limitations the paper identifies in prior work: training that is not fully end-to-end, and an inability to read curved or otherwise irregular text. The paper reports state-of-the-art or competitive results on horizontal, oriented, curved, and multi-language benchmarks, and its standalone recognizer outperforms prior recognizers on irregular-text benchmarks.","feed_headline":"One network spots curved text by reading 2D maps, end to end","feed_subtitle":"Detection and recognition share one feature space, so curved and rotated words are read in a single training pass.","key_machinery":"Central machinery is a single mask branch shared by three tasks: it predicts a binary text-instance mask, so detections are arbitrary polygons rather than boxes; per-character class maps, supervised locally where character annotations exist; and, through a Spatial Attention Module, a character sequence decoded directly from the two-dimensional feature map with position embeddings and a recurrent decoder. During inference the text-instance mask is contoured into a polygon, character maps are read by a pixel-voting algorithm that assigns each connected region its highest-scoring class, and the two recognition outputs are combined by confidence. A weighted edit distance then matches any lexicon. The design's work is to let detection and recognition share one feature space and one differentiable loss.","core_discovery":"The paper's central discovery is that text detection and recognition can be unified in an instance-segmentation architecture in which a single mask branch simultaneously outputs a text instance mask, per-character segmentation maps, and a character sequence produced by a spatial attention module. Because detection is instance segmentation, the predicted region can take any shape; because recognition decodes a two-dimensional feature map rather than a flattened one-dimensional sequence, curved and perspective text can be read without rectification. The paper further claims that this is the first completely end-to-end trainable text spotter, and that joint training improves detection accuracy, evidenced by comparisons with a detection-only variant. On ICDAR2015 with a generic lexicon, it reports gains of 8.6 points in word spotting and 10.5 points in end-to-end recognition over the previous best.","pith_inferences":["Editorial inference: The horizontal-rectangle proposal step is the likely ceiling of the arbitrary-shapes claim, because very long or tightly curved words could produce proposals that either include large background regions or clip the text, and the paper does not analyze failure rates along that dimension.","Editorial inference: The two-dimensional attention design suggests a natural stress test: text laid out vertically, diagonally, or wrapped around a circle, where left-to-right pixel voting would struggle but global attention might still decode; a synthetic dataset could probe this directly.","Editorial inference: The recognition module could be detached and paired with a stronger proposal generator, since it decodes from feature maps; one could verify whether end-to-end spotting gains exactly match the detector's improvement."],"forward_implications":["A single model can be trained once and deployed on datasets combining horizontal, oriented, and curved text, because detection is shape-agnostic and recognition reads two-dimensional maps.","Joint optimization means recognition errors can help suppress false-positive detections; the paper's ablation shows detection f-measure rises when the recognition branch is present.","Because the Spatial Attention Module needs only word-level labels, training scales to datasets without character-level annotation; ablations show only small losses when real character annotations are withheld.","On large or missing lexicons, the method is comparatively strong: it reports its largest gains over prior work precisely in the no-lexicon and generic-lexicon settings."],"supporting_citations":[{"why":"Supplies the mask-branch design and RoIAlign pooling that the paper repurposes for text instance and character segmentation.","marker":"[22]"},{"why":"Supplies the region proposal network and Fast R-CNN box classification and regression machinery that generate and refine text proposals.","marker":"[61]"},{"why":"Supplies the feature pyramid backbone that gives the network multi-scale features for text of varying sizes.","marker":"[44]"},{"why":"Supplies the large synthetic scene-text corpus used to pretrain the detector and recognizer.","marker":"[21]"},{"why":"Supplies the Total-Text benchmark with curved-text polygon annotations used to demonstrate arbitrary-shape spotting.","marker":"[10]"},{"why":"Supplies the prior attention-based recognizer baseline that the standalone recognition model is compared against and outperforms on irregular text.","marker":"[66]"},{"why":"Supplies the ICDAR2015 benchmark where the method reports its largest end-to-end gains with a generic lexicon.","marker":"[36]"}],"fun_headline_variants":["Mask TextSpotter reads curved text in one pass, no rectification","End-to-end network spots and reads arbitrary-shape text via masks","Arbitrary-shape text spotted and read by one neural network","End-to-end neural spotter reads curved and rotated text"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that for every text instance, however curved or elongated, a single axis-aligned horizontal rectangle from the region proposal network can cover the whole word tightly enough for the mask and recognition branches to read it.","fun_headline_variants_meta":{"raw":{"variants":["Mask TextSpotter reads curved text in one pass, no rectification","End-to-end network spots and reads arbitrary-shape text via masks","Arbitrary-shape text spotted and read by one neural network","End-to-end neural spotter reads curved and rotated text"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000827,"raw_usage":{"total_tokens":3611,"prompt_tokens":937,"completion_tokens":2674,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":2600}},"tokens_in":553,"tokens_out":2674,"duration_ms":17742,"temperature":1.0,"reasoning_tokens":2600,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:46:00.774191+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to generate synthetic text wrapped around circles and spirals with varying radius and word length, then plot end-to-end accuracy against curvature and length; if accuracy collapses as curvature grows beyond the benchmark range, or the predicted masks clip characters at the ends of the minimal horizontal rectangle, the arbitrary-shapes claim is bounded by the proposal assumption.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the mask-branch design and RoIAlign pooling that the paper repurposes for text instance and character segmentation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the region proposal network and Fast R-CNN box classification and regression machinery that generate and refine text proposals."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the feature pyramid backbone that gives the network multi-scale features for text of varying sizes."},{"cited_title":"Gupta, A","cited_arxiv_id":null,"evidence_quote":"Supplies the large synthetic scene-text corpus used to pretrain the detector and recognizer."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Total-Text benchmark with curved-text polygon annotations used to demonstrate arbitrary-shape spotting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the prior attention-based recognizer baseline that the standalone recognition model is compared against and outperforms on irregular text."},{"cited_title":"Karatzas, L","cited_arxiv_id":null,"evidence_quote":"Supplies the ICDAR2015 benchmark where the method reports its largest end-to-end gains with a generic lexicon."}],"review_version":1}