{"id":"23d80a53-da1c-48b5-85ca-c1be02df5844","arxiv_id":"2506.04999","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A full-image, mask-guided CLIP model with two-stage multi-granularity alignment training sets a new state of the art for Chinese scene text retrieval and introduces a diverse-layout benchmark.","lead":"This paper introduces a new benchmark for Chinese scene text retrieval that tests vertical, cross-line, and partial text layouts, plus a model that reads the full image instead of only cropped text regions. The model outperforms prior methods by 18.8 mAP points on the existing CSVTR benchmark and ranks best on the new layout benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DL-CSVTR may overlap with ReCTS training images; without a deduplication analysis, the benchmark results could reflect memorization rather than layout generalization.","rationale":"The strongest claim is that CSTR-CLIP outperforms crop-based baselines on DL-CSVTR, and that DL-CSVTR is a valid probe of diverse-layout generalization. The most load-bearing condition is therefore that DL-CSVTR tests unseen images. The paper trains on ReCTS, a public dataset of Chinese signboard images, and constructs DL-CSVTR around common signboard query terms collected from image search engines. No overlap analysis is reported. If near-duplicates exist between DL-CSVTR and ReCTS, the model could retrieve by memorizing training images, making the Table 2 gains (especially cross-line +11.58 and partial +18.66 from RAGP/global features in the ablations) an artifact of test-set contamination rather than evidence for the proposed mechanisms. This is a falsifiable, concrete gap that can be settled by a near-duplicate search once the benchmark is public. The reader's RAGP concern is legitimate for out-of-distribution transfer, but it is secondary: the benchmark numbers themselves do not depend on the spatial-proximity assumption being true globally, only on the augmented pairs being helpful for the layouts tested. The leakage concern, if confirmed, would invalidate the central benchmark contribution outright. Since the artifacts are not yet public and no deduplication statement is made, the appropriate verdict remains conditional.","tokens_in":16989,"tokens_out":6728,"duration_ms":77343,"concrete_test":"After the dataset and code release, compute a near-duplicate index between all 2,070 DL-CSVTR images and the ReCTS training split (plus SynthText-CH) using perceptual hashing and CLIP feature cosine similarity; manually inspect the top matches. Then recompute Table 2 mAP after excluding any near-duplicate test images (or, alternatively, report the overlap fraction). If the cross-line and partial gains shrink materially once overlapping images are removed, the benchmark results are inflated by training-set leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 trains on ReCTS (real scene Chinese signboard text) for both stages, while Section 3 constructs DL-CSVTR from query terms that are common signboard phrases (e.g., '瑞幸咖啡', '如家酒店', '封顶大吉') collected via image search engines. The paper never states that DL-CSVTR images are disjoint from ReCTS, nor reports any overlap or near-duplicate check. Since ReCTS is a public dataset of Chinese street-sign photos, and DL-CSVTR is a street-view signboard test set, the same or near-duplicate images could easily appear in both. If so, the high mAP values in Table 2 (84.44/65.56/61.85) would partly reflect the model having seen the test images and their text annotations during training, invalidating the claim that CSTR-CLIP generalizes to diverse layouts. This is more load-bearing than the RAGP spatial-proximity assumption: even if RAGP is perfectly sound, a contaminated test set would make the headline benchmark results uninterpretable. The absence of any deduplication statement or overlap statistic is a concrete, checkable gap.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses Chinese scene text retrieval, arguing that existing benchmarks and methods, inherited from English, fail on diverse Chinese text layouts. It introduces DL-CSVTR, a new test-only benchmark with vertical, cross-line, and partial layout subsets (2,070 images, 89 queries). It also proposes CSTR-CLIP, a two-stage CLIP-based model: stage 1 trains region-focused OCR via segmentation-map-guided single-granularity alignment; stage 2 injects frozen full-image global features and applies a Random Alignment Granularity Processing (RAGP) augmentation that randomly masks, expands, and splices text and segmentation maps to simulate multi-granularity layouts. Experiments report 88.57 mAP on CSVTR (an 18.82-point gain over the prior best) and strong results on DL-CSVTR (84.44/65.56/61.85 mAP on vertical/cross-line/partial), with ablations supporting the contributions of global features and RAGP.","tokens_in":17250,"tokens_out":3803,"duration_ms":44123,"significance":"If the empirical results hold, the paper makes two useful contributions: a benchmark that targets a real gap (Chinese layout diversity in street-view text), and a method that demonstrates the value of moving beyond cropped text regions to full-image features combined with region guidance. The ablations in Table 3 are informative and the authors state that code and data will be released. The main novelty—multi-granularity alignment with global feature injection—is plausible and could influence future scene-text retrieval work. However, the central claims depend on two empirical guarantees that are not yet established: the DL-CSVTR test set must be disjoint from the ReCTS training data, and the RAGP hyperparameters must not be silently selected on the test set. These issues are directly addressable but are load-bearing for the benchmark's validity and the method's reported gains.","major_comments":[{"comment":"The paper does not report any overlap or near-duplicate analysis between the new DL-CSVTR test images and the ReCTS training images. ReCTS is a dataset of real Chinese street-sign photos, and DL-CSVTR is constructed by searching for common signboard phrases (e.g., '封顶大吉', '瑞幸咖啡'); it is entirely plausible that the same or near-duplicate images appear in both. Because ReCTS is used for training in both stages, any overlap would mean the DL-CSVTR results in Table 2 (84.44/65.56/61.85 mAP) partly reflect memorization of test images rather than layout generalization. Please provide a concrete deduplication analysis (e.g., perceptual hashing, image retrieval-based near-duplicate check, or manual inspection) and report the overlap statistics; if overlap exists, remove those images and re-run the evaluation.","section":"§3 and §5 (Training Datasets)"},{"comment":"The RAGP hyperparameters β=0.2, α=0.3, and θ=0.2 are stated without any description of a validation protocol or sensitivity analysis. Since RAGP produces large improvements on DL-CSVTR-CL (45.98→60.08) and DL-CSVTR-P (33.25→52.51) in Table 3, the reader cannot rule out that these values were chosen by observing the test-set numbers. Please specify how the hyperparameters were selected (e.g., a held-out validation split from ReCTS or a separate development set) and provide a sensitivity study over these parameters, or the RAGP contribution remains under-validated.","section":"§5 (Implementation Details) and §5.3 (Table 3)"},{"comment":"The RAGP augmentation explicitly relies on the assumption that 'semantically related text elements are spatially close in the image.' This assumption is load-bearing because RAGP merges each text with the nearest-neighbor text by bounding-box centroid. If nearby text boxes are semantically unrelated, the augmented pairs train the model to align a text string with a mismatched mask, and the reported cross-line/partial gains would likely not transfer. Please provide an empirical test of the assumption—for example, measure the semantic relatedness of the merged pairs in the training set, or compare the current nearest-neighbor rule against a random-neighbor or same-image-random rule to show that the spatial proximity heuristic is actually beneficial.","section":"§4 (RAGP)"},{"comment":"The paper cites Zeng et al. (2024a), a CLIP-based scene text retrieval method that is directly relevant to the proposed approach, but this method is not included as a baseline in Table 1 or Table 2. Given the paper's claim of state-of-the-art performance on CSVTR, omitting a recent, closely related method weakens that claim. Please add this baseline (or justify its exclusion) and report its performance on CSVTR and, if possible, on DL-CSVTR.","section":"§2, §5.1, Table 1"}],"minor_comments":[{"comment":"The paper states '18.82% accuracy' when Table 1 reports mAP; please use 'mAP' or 'percentage points' for precision.","section":"Abstract and §5.1"},{"comment":"Figure 9's caption says 'Distribution of text layouts...' but the figure appears to display the query-term lists for the three DL-CSVTR subsets; this caption seems mismatched with the figure content.","section":"Appendix A.3, Figure 9"},{"comment":"The sentence 'The details of the specific query word settings can be found in the supplementary material' is followed in the appendix by 'as shown in Figure 2'; the query lists actually appear in Figure 9, so the cross-reference appears incorrect.","section":"Appendix A.2"},{"comment":"The phrase 'might include unrelated extra not unrelated characters' appears to be a grammatical error; it should likely read 'extra unrelated characters.'","section":"§3 (Partial layout)"},{"comment":"All results in Table 3 are reported as single runs with no indication of variance; please state whether results are averaged over multiple seeds or runs, or note that only single runs were performed.","section":"Table 3"},{"comment":"In the Random Expand step, the pseudocode says 'Find Tip whose centroid is closest to Tik' but Tik is a text string, not a bounding box; please clarify that the centroid is taken from Tik's bounding box, to avoid ambiguity.","section":"Algorithm 3"}],"recommendation":"major_revision","confidential_remarks":"The benchmark contamination concern is the most serious issue; if a deduplication analysis reveals substantial overlap with ReCTS, the paper's central benchmark claim would be undermined and a reject might be warranted. The omission of Zeng et al. (2024a) from the experimental comparison is surprising given that the paper cites it and claims state-of-the-art; this should be addressed. The paper otherwise contains a well-structured method and ablations that largely support the internal logic of the approach."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read for you on arXiv:2506.04999. The paper does two real things: it builds DL-CSVTR, a Chinese scene text retrieval benchmark with vertical, cross-line, and partial layouts, and it proposes CSTR-CLIP, a two-stage CLIP-based model using full-image features, text-region segmentation maps, and a random multi-granularity alignment (RAGP) training procedure. The numbers are strong: 88.57 mAP on CSVTR, about 19 points over the previous best, and clear gains on the new DL-CSVTR subsets. The ablations are coherent: global features help vertical layouts, RAGP helps cross-line and partial, and combining them gives the best results. The writing is clear, the pseudo-code helps, and the per-query AP plots in the appendix add useful detail.\n\nThe soft spots, in order of seriousness. First, the paper trains on ReCTS, a public dataset of Chinese street-sign photos, and builds DL-CSVTR from image-search results for common signboard phrases like 瑞幸咖啡 and 封顶大吉. The text never reports any overlap or near-duplicate check between DL-CSVTR and ReCTS or any other training source. ReCTS images are exactly what a search engine would return for those queries, so the possibility that test images appear in training is real. If so, the headline mAP values partly reflect memorization, not layout generalization. This is a concrete, checkable gap, and the authors need to provide a deduplication analysis or otherwise show the sets are disjoint.\n\nSecond, every result is a single run with no error bars, and the RAGP hyperparameters (β=0.2, α=0.3, θ=0.2) appear without a described validation protocol. Minor compared to the contamination issue, but it weakens reproducibility. Third, RAGP assumes semantically related text elements are spatially close. That is a reasonable heuristic for signboards, but the large ablation gains on cross-line and partial layouts could be inflated if that assumption fails elsewhere. I would want to see it tested on an independent set.\n\nWho is this for: people working on scene text retrieval, especially Chinese. The benchmark is a useful contribution if the data is clean and released. The method is worth engaging with. I would not trust the absolute numbers until the overlap question is answered, but this deserves a serious referee who pushes on data provenance. My recommendation: send it to review, but require the overlap analysis and dataset release as conditions for acceptance.","headline":"Useful benchmark-and-baseline paper with a serious, unaddressed data-contamination question that should be resolved before the numbers are trusted.","tokens_in":17748,"tokens_out":2196,"would_cite":false,"duration_ms":25757,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Crop-based Chinese text retrieval fails on vertical, cross-line, and partial layouts; CSTR-CLIP encodes the full image with text-region guidance and multi-granularity alignment, reaching 88.57 mAP on CSVTR, 18.82 points over the prior best.","keywords":["Chinese scene text retrieval","DL-CSVTR benchmark","diverse text layouts","CLIP","multi-granularity alignment","text-region segmentation map","random alignment granularity processing","cross-modal retrieval"],"falsifier":"Build a test set of street-view images in which the text box nearest to each query is deliberately unrelated to it, and compare DL-CSVTR cross-line and partial mAP with RAGP enabled versus disabled; if accuracy drops toward the stage-1 level whenever neighbors are unrelated, the spatial-proximity assumption is the fragile load-bearing premise.","tokens_in":16798,"feed_emoji":"🔍","tokens_out":13213,"duration_ms":122861,"temperature":0.7,"pith_summary":"The paper claims that Chinese scene text retrieval fails in real scenes because it inherited an English-oriented recipe: crop the detected text region, then match it to the query at a single granularity. Chinese text is laid out vertically, spans rows and columns, and embeds query terms inside longer strings, so a crop either cuts the query short or pulls in unrelated characters while discarding the surrounding context. To make the case, the authors contribute DL-CSVTR, a 2,070-image benchmark whose queries appear in vertical, cross-line, and partial layouts, and CSTR-CLIP, a CLIP-based model that encodes the full image steered by a text-region segmentation map and trains first on whole-text alignment, then on randomly varied alignment granularities. They report 88.57 mean average precision (mAP) on the previous CSVTR benchmark, 18.82 points above the best crop-based method, and 84.44, 65.56, and 61.85 mAP on the three new layout subsets, all above every crop-based baseline.","feed_headline":"18.8-point gain: full-image CLIP reads Chinese text layouts","feed_subtitle":"CSTR-CLIP reaches 88.57 mAP on the existing benchmark and tops the new layout subsets.","key_machinery":"The load-bearing components are the Text Position Convolution, a single-channel convolution that converts the text segmentation map into a guided embedding added to the CLIP RGB embedding before the image encoder; the Random Alignment Granularity Processing (RAGP) module, which during stage-two training randomly masks characters from the text, merges the nearest neighboring segmentation map, and extends the text toward the nearest box; and per-layer 1x1-convolution fusion layers that inject the frozen original CLIP encoder's global features into the fine-tuned encoder. The segmentation map is a single-channel grayscale image with text pixels highlighted, and the same map used for guidance in training doubles as the user interface for region-specified retrieval at inference.","core_discovery":"The central claim is that scene text retrieval should move from matching cropped text regions to matching a whole image whose attention is steered by text positions. CSTR-CLIP sums the CLIP RGB embedding with a Text Position Convolution that encodes the text segmentation map, runs the result through the CLIP image encoder, and in a second stage adds the frozen original CLIP encoder's global features at every layer plus a random multi-granularity alignment. The authors attribute the CSVTR improvement (69.75 to 88.57 mAP) to retaining visual features outside the text region, and the DL-CSVTR gains on the hardest layouts to RAGP, which deliberately breaks the exact segmentation-map-to-text match during training so that the model perceives text elements inside and near the guided region. On vertical layouts the full-image information itself is credited with most of the gain.","pith_inferences":["The two-stage curriculum of exact alignment followed by deliberately broken alignment could transfer to other vision-language tasks where a region only partially encodes a target concept, such as referring expression grounding.","A controlled stress test of RAGP's spatial-proximity assumption, using images where neighboring text boxes are semantically unrelated, would reveal how much of the cross-line and partial gains depend on that assumption; if gains collapse, a language-aware neighbor selector would be the natural fix.","The same full-image-plus-guidance design should simplify English scene text retrieval as well, since retrieval accuracy would no longer be gated by whether the detected crop exactly contains the query word."],"forward_implications":["Keeping the whole image instead of cropping detected boxes lifts CSVTR mAP from 69.75 (the best crop-based method) to 88.57, quantifying how much context crop-based retrieval discards.","Adding global features and RAGP in stage two raises cross-line mAP from 45.98 to 65.56 and partial mAP from 33.25 to 61.85 on DL-CSVTR, so multi-granularity alignment is the ingredient that handles the hardest layouts.","Because the segmentation map is user-controllable, the same model enables region-specified retrieval: highlighting part of the map narrows the search to a chosen area of the image.","If the detector finds no text, a fully highlighted segmentation map still lets the model retrieve from visual context alone, something crop-based pipelines cannot do."],"supporting_citations":[{"why":"Establishes the CSVTR benchmark and the TDSL cross-modal baseline; the existing benchmark the paper must outperform and the crop-based paradigm it argues against.","marker":"(Wang et al., 2021)"},{"why":"The prior state-of-the-art visual embedding method (69.75 mAP on CSVTR) that CSTR-CLIP beats by 18.82 points.","marker":"(Luo et al., 2024)"},{"why":"CLIP, the contrastive image-text backbone whose frozen encoder supplies the global features and whose architecture seeds the model.","marker":"(Radford et al., 2021)"},{"why":"Chinese CLIP, the pre-trained weights that initialize CSTR-CLIP's encoders.","marker":"(Yang et al., 2023)"},{"why":"ReCTS, the real-world Chinese signboard corpus used for training in both stages.","marker":"(Zhang et al., 2019)"},{"why":"SynthText, whose synthetic data pipeline motivates the SynthText-CH 300K images used in stage one.","marker":"(Gupta et al., 2016)"},{"why":"VSTR, the visual-matching baseline compared on CSVTR and representative of the crop-based paradigm.","marker":"(Wen et al., 2023)"}],"fun_headline_variants":["New benchmark: CLIP model nails Chinese text in any layout","Full-image CLIP beats cropped-text retrieval by 18.8 points","Diverse layout benchmark pushes Chinese text retrieval forward","CSTR-CLIP: whole-image encoding wins Chinese text search","Forget cropping: CLIP reads Chinese text in complex layouts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The RAGP augmentation in Section 4 assumes that semantically related text elements are spatially close in the image, so merging a nearby text box into the training pair teaches a useful lesson; when neighboring text is semantically unrelated, the augmented pairs train the model to align a mask with a mismatched string, and the reported cross-line and partial gains would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["New benchmark: CLIP model nails Chinese text in any layout","Full-image CLIP beats cropped-text retrieval by 18.8 points","Diverse layout benchmark pushes Chinese text retrieval forward","CSTR-CLIP: whole-image encoding wins Chinese text search","Forget cropping: CLIP reads Chinese text in complex layouts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000181,"raw_usage":{"total_tokens":1322,"prompt_tokens":972,"completion_tokens":350,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":263}},"tokens_in":588,"tokens_out":350,"duration_ms":4730,"temperature":1.0,"reasoning_tokens":263,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:28:38.584424+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a test set of street-view images in which the text box nearest to each query is deliberately unrelated to it, and compare DL-CSVTR cross-line and partial mAP with RAGP enabled versus disabled; if accuracy drops toward the stage-1 level whenever neighbors are unrelated, the spatial-proximity assumption is the fragile load-bearing premise.","supporting_citations":[{"cited_title":"Scene text retrieval via joint text detection and similarity learning","cited_arxiv_id":null,"evidence_quote":"Establishes the CSVTR benchmark and the TDSL cross-modal baseline; the existing benchmark the paper must outperform and the crop-based paradigm it argues against."},{"cited_title":"Visual and semantic guided scene text retrieval","cited_arxiv_id":null,"evidence_quote":"The prior state-of-the-art visual embedding method (69.75 mAP on CSVTR) that CSTR-CLIP beats by 18.82 points."},{"cited_title":"W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al","cited_arxiv_id":null,"evidence_quote":"CLIP, the contrastive image-text backbone whose frozen encoder supplies the global features and whose architecture seeds the model."},{"cited_title":"Icdar 2019 robust reading challenge on reading chinese text on signboard","cited_arxiv_id":null,"evidence_quote":"ReCTS, the real-world Chinese signboard corpus used for training in both stages."},{"cited_title":"Synthetic data for text localisation in natural images","cited_arxiv_id":null,"evidence_quote":"SynthText, whose synthetic data pipeline motivates the SynthText-CH 300K images used in stage one."},{"cited_title":"Visual matching is enough for scene text retrieval","cited_arxiv_id":null,"evidence_quote":"VSTR, the visual-matching baseline compared on CSVTR and representative of the crop-based paradigm."}],"review_version":1}