{"id":"7d859572-1e30-4991-abf5-b0f5434ffffc","arxiv_id":"2505.10921","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A new dataset of 5,726 Chinese cultural heritage image-text pairs plus a training-free local alignment method LACLIP that improves Chinese-CLIP retrieval only marginally on it.","lead":"This paper introduces a new Chinese cultural heritage dataset, CulTi, with 5,726 image-text pairs from books on ancient silk and Dunhuang murals, plus an inference-time method LACLIP for cross-modal retrieval. The dataset fills a real gap, but the method's reported gain over a strong CLIP baseline is small and lacks statistical support.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 0.6-point MR gain in Table 3 (47.9 vs 47.3) is never shown to exceed run-to-run noise: inference is stochastic random-crop, no error bars or significance tests are reported, and no ablation isolates the Eq. (5) weighting.","rationale":"The most load-bearing claim is the abstract's 'LACLIP significantly outperforms existing models.' The entire quantitative support is the 0.6-point MR gap in Table 3, and the paper's own Table 3 shows a decline in image-to-text R@1, so the improvement is not even directionally consistent across both subtasks. Fine-tuning a ViT-H/14 plus RoBERTa-wwm-Large on 5,726 pairs for only 3 epochs is exactly the regime where seed sensitivity and optimizer noise are expected to be non-negligible. The inference protocol adds another stochastic layer: random crops with unspecified n, scale, and seed. With no error bars or repeated runs, a 0.6-point difference cannot be distinguished from sampling noise. This is not a matter of disagreeing with consensus; it is a missing piece of evidence for the central claim. The Eq. (5) weighting scheme is a second soft spot: weighting by similarity to the full image is not the same as weighting by relevance to the text, and the paper provides no ablation to show the weighting beats uniform averaging. The dataset itself is a real contribution: it is documented, has a clear construction pipeline, is released, and fills a gap. Therefore the paper should not be rejected; it should be conditional on releasing exact inference settings, reporting variance, adding the uniform-weight ablation, and softening the significance language. This matches the reader's CONDITIONAL verdict, though my primary concern is statistical reliability of the gain rather than the specific Eq. (5) premise.","tokens_in":12642,"tokens_out":6196,"duration_ms":62382,"concrete_test":"Run one controlled experiment with 10 random seeds comparing three conditions on CulTi: (a) CN-CLIP ViT-H, (b) LACLIP as described in Sec. 4.2 with exact crop parameters reported, and (c) LACLIP with Eq. (5) replaced by uniform weights 1/n. Report mean +/- std MR and paired 95% confidence intervals for (b)-(a) and (b)-(c). If the (b)-(a) interval includes 0, 'significantly outperforms' is unsupported; if (b)-(c) is also insignificant, the Eq. (5) mechanism is not the source of any gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 3 reports LACLIP at MR 47.9 versus 47.3 for fine-tuned CN-CLIP ViT-H, a 0.6-point gain, and the abstract/conclusion call this 'significantly outperforms existing models.' The image-to-text direction actually declines (R@1 drops from 24.5 to 23.6). No variance estimate, significance test, or seed is provided. The method is stochastic by design: Section 4.2 says the image is 'randomly cropped to multiple local patches,' but the paper never states the number of patches n, the crop scale/aspect-ratio distribution, or the random seed, so the reported numbers are not reproducible and could depend on a favorable draw. The mechanism is also questionable: Eq. (5) assigns weight by similarity of the patch to the full image, not by similarity to the text. For descriptions that target a small local motif, a crop containing the motif can receive low weight while a dominant-background crop receives high weight. The paper offers no ablation replacing Eq. (5) with uniform weights or with text-selected patches, so there is no evidence that the local-alignment premise, rather than random-crop ensembling or fine-tuning noise, produces the gain. The CulTi dataset contribution is credible and useful; the unsupported part is the method claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents CulTi, a Chinese cultural heritage multimodal dataset of 5,726 image-text pairs built from two book series on ancient silk patterns and Dunhuang murals. The construction pipeline is described in four steps: source selection, document digitization, OCR-based information extraction, and data summarization with ChatGPT-4o augmentation and manual proofreading; category and volume distributions are reported. The paper also proposes LACLIP, which fine-tunes Chinese-CLIP (ViT-H/14, RoBERTa-wwm-Large) on CulTi and, at inference, computes a weighted average of similarities between random image crops and the text, with crop weights derived from cosine similarity to the full image embedding (Eqs. 5-6). Experiments compare zero-shot and fine-tuned models on R@K and mean recall; LACLIP is reported at MR 47.9 versus 47.3 for CN-CLIP ViT-H.","tokens_in":12895,"tokens_out":5746,"duration_ms":55599,"significance":"If the dataset is as described, CulTi addresses a real gap: a domain-specific Chinese image-text retrieval benchmark for cultural heritage, with a documented construction process and a public code/data link. This is a potentially useful community resource. The method contribution is currently much weaker: the reported improvement over the strongest baseline is 0.6 mean-recall points, the image-to-text direction declines, no statistical support is provided, and the key design assumption is untested. The dataset part is credible and worth publishing; the method claim needs substantial additional evidence before it can be accepted. I therefore see this as a major-revision rather than a rejection, because the dataset contribution is fixable and the method issues are addressable with ablations and more rigorous evaluation.","major_comments":[{"comment":"The central claim that LACLIP 'significantly outperforms existing models' rests on a 0.6-point mean-recall gain (47.9 vs 47.3) over CN-CLIP ViT-H, while the image-to-text direction declines at every recall level (R@1: 23.6 vs 24.5; R@5: 49.9 vs 50.4; R@10: 62.9 vs 64.0). No variance estimates, significance tests, or repeated runs with different seeds are reported, so the reported gain cannot be distinguished from run-to-run noise. The abstract, §5.2, and §6 should either provide statistical support or use more measured language.","section":"§5.2, Table 3"},{"comment":"The method's key premise, that patches similar to the full image contain the subjects the text describes, is not tested. Eq. (5) weights each patch by exp(α·S(u_i^k, u_i^I)), i.e., similarity to the global image embedding, not to the text. For a repetitive pattern where the text describes a single motif, a crop containing that motif can receive low weight while a dominant-background crop receives high weight. The paper provides no ablation replacing Eq. (5) with uniform weights, no cropping, or text-selected patches, and no sensitivity analysis for α; without this, the reported gain could come from random-crop ensembling or fine-tuning noise rather than the proposed local-alignment mechanism.","section":"§4.2, Eqs. (5)-(6)"},{"comment":"The inference procedure is not reproducible as reported. The paper states that the image is 'randomly cropped to multiple local patches' but never specifies the number of patches n, the crop size, the scale/aspect-ratio range, or the random seed, even though n and the crop distribution directly determine Eq. (6). The fine-tuning description is also incomplete: no optimizer, learning-rate schedule, image resolution, or per-split data sizes are given. These details are needed before the LACLIP numbers in Table 3 can be independently reproduced or compared.","section":"§4.2 and §5.1"},{"comment":"The evaluation reports only aggregate MR and R@K over the full test set. Since CulTi contains four visually distinct categories (pattern, original textile, cropped pattern, mural) from two book series, and the local-alignment assumption is argued from the pattern case in Fig. 2, a per-category or per-source breakdown is necessary to show where the method helps and where it hurts. Without it, the claim that LACLIP 'handles fine-grained semantic associations' is not localized.","section":"§5.2"}],"minor_comments":[{"comment":"There are several typographical errors, including 'T able 1' in §2.1 and 'Mciji Mountain' in Fig. 5 and the text; these should be corrected to 'Table 1' and 'Maiji Mountain'.","section":"Throughout"},{"comment":"The qualitative examples contain small, low-resolution Chinese text that is difficult to verify from the PDF; higher-resolution crops or zoomed insets would help readers assess the failure cases.","section":"Figures 4, 7, 8"},{"comment":"The conclusion contains grammatical errors (e.g., 'CulTi fulfill the gap in multimodal cultural area provides valuable resources') that should be corrected.","section":"§6"},{"comment":"The paper should specify the license and copyright status of the images extracted from the two published book series, and whether the dataset is intended for non-commercial research use only.","section":"§3 and GitHub link"},{"comment":"The Related Work mentions local-alignment methods LexVLA [11] and SEA [22] but does not compare LACLIP to them; a comparison or a clear explanation of why they are inapplicable would help position the method.","section":"§2.2"}],"recommendation":"major_revision","confidential_remarks":"The dataset component is the strongest part of this submission and, in my view, is publishable as a dataset contribution. If the requested experiments do not support the LACLIP method, the authors should consider reframing the paper as a dataset-and-baselines paper. I also note that the 'first dataset' claim is hard to verify from the manuscript alone; the authors should document their search of existing resources or soften the claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The CulTi dataset is the real contribution here, and it's a good one for a niche that had nothing. The LACLIP method, as presented, is not supported by the evidence in the paper.\n\nWhat's new: a 5,726-pair Chinese cultural heritage image-text dataset from two serious book series (silk patterns, Dunhuang murals), with a documented pipeline: scanning, OCR, ChatGPT-4o augmentation for 28 pairs, manual proofreading, and per-volume splits. That's a legitimate resource for digital humanities and cross-modal retrieval in a domain that previously had only unimodal or natural-scene Chinese datasets. The paper is honest about the construction steps and shows examples and failure cases. I'd trust the dataset more than most new benchmarks because the source is physical publications and the extraction process is described in enough detail to reproduce or audit.\n\nWhat's soft: the method. LACLIP is fine-tuned Chinese-CLIP plus a weighted random-crop ensemble at inference. The gain over the strongest baseline is 0.6 mean recall points (47.9 vs 47.3), while image-to-text recall actually drops (R@1 from 24.5 to 23.6). There are no error bars, no seeds, no statement of how many patches are sampled or at what scales, and alpha is hand-set at 1.02 with no sensitivity analysis. The weighting in Eq. (5) is by similarity to the full image, not to the text, so there's no direct evidence that the local-alignment premise is what helps; a uniform random-crop ensemble could plausibly produce the same or better result. The paper says 'significantly outperforms,' but with stochastic inference and no variance report, a 0.6-point gap on a 1,145-item test set is within plausible noise.\n\nThis is a case where the dataset deserves a serious referee and the method claim needs to be scaled back or properly ablated. I'd send it to review, but the authors should release exact inference settings, add ablations separating the weighting from random cropping, report variance across seeds, and either soften the significance claim or back it with a test.","headline":"A genuine new dataset for a neglected niche, saddled with a method claim that the reported numbers do not support.","tokens_in":13442,"tokens_out":1791,"would_cite":true,"duration_ms":18076,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new 5,726-pair Chinese cultural heritage dataset, plus a training-free local alignment method, lifts cross-modal retrieval to a mean recall of 47.9 on CulTi.","keywords":["Chinese cultural heritage","cross-modal retrieval","image-text retrieval","local alignment","Chinese-CLIP","Dunhuang murals","ancient silk patterns","multimodal dataset"],"falsifier":"Run LACLIP on CulTi with the patch weights in Eq. (5) replaced by uniform weights, keeping the same random crops and all other settings; if mean recall does not drop below 47.9, the similarity-based weighting is not doing the work, and the result reduces to fine-tuning plus random-crop averaging.","tokens_in":12431,"feed_emoji":"🖼️","tokens_out":5968,"duration_ms":53299,"temperature":0.7,"pith_summary":"The paper aims to close a gap: there is no specialized multimodal dataset for Chinese cultural heritage, so cross-modal retrieval models are tested only on generic scenes. It introduces CulTi, 5,726 simplified-Chinese image-text pairs drawn from two professional book series on ancient silk patterns and Dunhuang murals. It then proposes LACLIP, an inference-only local alignment strategy that re-weights image patches by how similar each patch is to the full image before comparing them with the text. On CulTi, LACLIP reports a mean recall of 47.9, slightly above the strongest fine-tuned baseline at 47.3, with the gain concentrated in text-to-image retrieval.","feed_headline":"Local alignment lifts heritage retrieval to 47.9 mean recall","feed_subtitle":"A new 5,726-pair Chinese cultural heritage dataset and a training-free method close a domain gap.","key_machinery":"The load-bearing mechanism is the local alignment module defined by Eqs. (5)-(6): patch weights\n$$w_k^i = \\frac{\\exp(\\$\\alpha$ \\, S(u_k^i, u_I^i))}{\\sum_{t=1}^n \\exp(\\$\\alpha$ \\, S(u_t^i, u_I^i))}$$\nand final similarity\n$$S_{\\mathrm{final}} = \\sum_{k=1}^n w_k^i \\, S(u_k^i, u_T^i).$$\nHere $u_k^i$ is the embedding of a random crop, $u_I^i$ the full-image embedding, and $u_T^i$ the text embedding. The weight makes a crop count more if its embedding resembles the whole image, on the assumption that such crops contain the main subject the text describes. The dataset construction pipeline (OCR extraction, layout-specific cropping, and LLM-based text augmentation) supplies the training pairs needed to fine-tune the base model so that the local weights operate on domain-aligned embeddings.","core_discovery":"CulTi is the paper's first contribution: 5,726 image-text pairs in Simplified Chinese, assembled from scanned pages of two authoritative publication series, with four data categories (pattern, original textile, cropped pattern, and mural) and a per-category 7:1:2 train/validation/test split. The second contribution is LACLIP, which starts from a Chinese-CLIP model with ViT-H/14 and RoBERTa-wwm-Large backbones fine-tuned on CulTi and adds a local alignment module at inference. For each image, several random crops are encoded, and each crop embedding is given a weight proportional to its cosine similarity with the full-image embedding; the final image-text similarity is the weighted sum of the crop-text similarities. The paper reports that this raises mean recall from 47.3 for the fine-tuned CN-CLIP ViT-H baseline to 47.9, mainly through text-to-image retrieval, while image-to-text recall slightly decreases.","pith_inferences":["If the similarity-weighting premise holds, the same random-crop weighting could be applied to other repetitive decorative arts, such as ceramics, brocades, or architectural ornament, where a text describes one motif among many repeats.","The method's reliance on random crops suggests a natural extension: replacing random crops with detected motif regions or saliency proposals should either sharpen the gain or reveal that random cropping alone is sufficient.","The 0.6-point margin over the best baseline is small relative to the 24.7-point gain from fine-tuning itself, so the practical contribution may be the dataset and fine-tuning procedure rather than the local alignment module.","A controlled test varying the number of crops and the scaling factor $\\alpha$ would show whether the weighting is robust or whether performance depends on these free parameters."],"forward_implications":["CulTi gives the cross-modal retrieval community a domain-specific benchmark in Simplified Chinese where generic models perform poorly in zero-shot mode, with the best baseline mean recall at 22.6 before fine-tuning.","LACLIP's inference-stage weighting can be attached to any fine-tuned Chinese-CLIP without retraining, so the reported improvement is portable across backbone choices.","The reported gain is concentrated in text-to-image retrieval; image-to-text recall is slightly lower than the baseline, so the method is a targeted fix rather than a symmetric win.","The per-category 7:1:2 split supports standard supervised evaluation, and the four data categories allow retrieval difficulty to be studied separately for patterns, original textiles, cropped patterns, and murals."],"supporting_citations":[{"why":"Supplies the Chinese-CLIP model whose pre-trained weights are fine-tuned and whose encoders are reused by LACLIP.","marker":"[21]"},{"why":"Provides the CLIP baselines and the contrastive pretraining paradigm that the retrieval evaluation builds on.","marker":"[18]"},{"why":"Supplies the large language model used to augment brief textual descriptions during dataset construction.","marker":"[7]"},{"why":"Supplies the OCR techniques used to extract text from the scanned cultural-heritage documents.","marker":"[14]"},{"why":"Provides a general-domain Chinese image-caption dataset that CulTi is contrasted with.","marker":"[9]"},{"why":"Provides another general-domain Chinese multimodal dataset used as a comparison baseline.","marker":"[10]"},{"why":"Represents the unimodal Dunhuang dataset that CulTi extends by adding aligned textual descriptions.","marker":"[23]"},{"why":"Supplies the ViT-H/14 visual backbone used in the fine-tuned Chinese-CLIP model.","marker":"[5]"},{"why":"Supplies the RoBERTa-wwm-Large text encoder used for the Chinese text modality.","marker":"[13]"},{"why":"Supplies a Chinese cross-modal benchmark model used as both a zero-shot and fine-tuned baseline.","marker":"[20]"}],"fun_headline_variants":["CulTi dataset and LACLIP boost heritage retrieval","Training-free local alignment improves heritage retrieval","New CulTi dataset challenges cross-modal heritage retrieval","LACLIP adds local alignment to hit 47.9 on CulTi"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central premise is that a random crop whose embedding resembles the full image is the region the text actually describes; if that mapping fails, the weighting adds noise rather than alignment.","fun_headline_variants_meta":{"raw":{"variants":["CulTi dataset and LACLIP boost heritage retrieval","Training-free local alignment improves heritage retrieval","New CulTi dataset challenges cross-modal heritage retrieval","LACLIP adds local alignment to hit 47.9 on CulTi"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000679,"raw_usage":{"total_tokens":3104,"prompt_tokens":984,"completion_tokens":2120,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":2053}},"tokens_in":600,"tokens_out":2120,"duration_ms":17561,"temperature":1.0,"reasoning_tokens":2053,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:00:42.387010+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run LACLIP on CulTi with the patch weights in Eq. (5) replaced by uniform weights, keeping the same random crops and all other settings; if mean recall does not drop below 47.9, the similarity-based weighting is not doing the work, and the result reduces to fine-tuning plus random-crop averaging.","supporting_citations":[{"cited_title":"ACM Computing Surveys (CSUR)54(6), 1–37 (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the OCR techniques used to extract text from the scanned cultural-heritage documents."},{"cited_title":"In: Pro- ceedings of the 25th ACM international conference on Multimedia","cited_arxiv_id":null,"evidence_quote":"Provides a general-domain Chinese image-caption dataset that CulTi is contrasted with."},{"cited_title":"IEEE Transactions on Multimedia 21(9), 2347–2360 (2019)","cited_arxiv_id":null,"evidence_quote":"Provides another general-domain Chinese multimodal dataset used as a comparison baseline."},{"cited_title":"In: Proceedings of the 31st ACM International Conference on Multimedia","cited_arxiv_id":null,"evidence_quote":"Supplies a Chinese cross-modal benchmark model used as both a zero-shot and fine-tuned baseline."}],"review_version":1}