{"id":"d28297f4-f84a-410d-978e-dc376dada6f1","arxiv_id":"2501.01699","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RSHNL combines contrastive learning, class-center aggregation, and self-paced weighting to make cross-modal hashing robust to noisy labels.","lead":"This paper proposes a cross-modal hashing method that learns image-text hash codes while filtering out training samples with wrong labels. It sorts training pairs by difficulty, ignores the hardest ones, and reports better retrieval accuracy than 11 existing methods on four datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The NSH noise-detection rule assumes loss magnitude separates noisy from clean pairs; the paper provides no quantitative evidence that hard clean pairs are not discarded or that mislabeled pairs that fit current centers are not retained.","rationale":"The reader's conditional verdict is appropriate, and I find the same weakest link. The method's novelty and empirical gains are real, and the authors provide code and extensive tables. However, the specific claim that NSH 'distinguishes noisy labels through the difficulty level' is not supported beyond a qualitative figure. This matters because the self-paced weight in Eq. (12) is only optimal for a fixed model; the model itself is trained with labels that may be wrong, so the loss ℓ_i can be high for correct but hard pairs. The absence of any quantitative noise-detection metric means we cannot tell whether the reported robustness comes from removing noise or from some other property (e.g., the contrastive loss or warm-up). The oracle test and precision/recall measurements would settle this directly. Until then the paper should not be accepted without the conditional request. I also note the suspicious identical values in Table 4 for RSHNL-1, which should be corrected, but the primary concern is the logical gap in the NSH selection rule.","tokens_in":15658,"tokens_out":4626,"duration_ms":46984,"concrete_test":"Use the released code on INRIA-Websearch (and XMediaNet) with 128-bit codes and 0.6 and 0.8 symmetric noise. Inject a fixed noise mask, keep it fixed across seeds, and at every 10 epochs after warm-up compute the confusion matrix of the rule w_i=0 against the true noisy set: report noisy-pair recall, clean-pair false-positive rate, and AUC. Then run an oracle ablation that replaces w_i with 1 for true clean pairs and 0 for true noisy pairs while keeping all other hyperparameters. If the oracle gains more than about 3 MAP points at 0.8 noise, or the rule's noisy-pair recall falls below 80% while false-positive rate exceeds 20%, the NSH mechanism does not reliably identify noisy labels and the paper's explanation of its robustness must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central robustness claim rests on the NSH selection rule in Eqs. (7)-(12): a pair is treated as noisy iff its center-aggregation loss ℓ_i exceeds the pace parameter γ. This is load-bearing because every reported gain at noise rates 0.2-0.8 is attributed to removing noisy pairs and learning clean pairs easy-to-hard. The paper does not show that ℓ_i separates the two populations. Hard clean samples (large intra-class variation, ambiguous modality content) can have ℓ_i > γ and be assigned w_i=0, discarding valuable training data; conversely, a mislabeled sample whose features happen to align with its corrupted label can have small ℓ_i and be retained. Eqs. (10)-(12) only derive the optimal w_i for a fixed model; they do not prove any monotone relationship between ℓ_i and the event 'y_i is wrong'. The only evidence is the qualitative density plot in Fig. 3, which shows relative shifts but reports no precision, recall, or AUC of the w_i=0 decision against the known injected noise mask. The ablation row RSHNL-1 in Table 4 with identical MAP values across all noise rates further undermines confidence in the experimental support, though the primary logical gap remains the unvalidated selection mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RSHNL, a supervised cross-modal hashing method designed to tolerate symmetric label noise. It combines three losses: a contrastive hashing loss (CHL) for cross-modal consistency, a center-aggregation loss (CAL) for intra-class compactness, and a self-paced hashing loss (NSH) that assigns an importance weight to each training pair based on its center-aggregation loss. For a fixed model, the optimal SPL weight is derived in closed form as max(0, 1 - ell_i/gamma), and pairs with ell_i > gamma are given zero weight and declared noisy. The method is evaluated on four datasets against 11 baselines under noise rates 0.2-0.8 and hash lengths 16-128, with code released on GitHub.","tokens_in":16030,"tokens_out":8518,"duration_ms":85144,"significance":"The paper's strengths are its breadth of evaluation and reproducibility: four datasets, four noise rates, four hash lengths, and a public code link. The closed-form SPL weight update is mathematically correct, and the reported MAP margins over prior methods are large, especially at high noise rates. If the noise-selection mechanism truly removes corrupted pairs, the method would be a practically useful contribution to noisy-label cross-modal hashing. However, the central claim that a high center-aggregation loss identifies label noise is not validated beyond a qualitative density plot, and several experimental reporting issues weaken the quantitative evidence. These issues are fixable but require additional experiments and corrections.","major_comments":[{"comment":"The key assumption of the method is that a high center-aggregation loss ell_i indicates a noisy label. The text after Eq. (12) states that when ell_i > gamma, 'we consider this sample pair could be mislabeled and assign the weight as zero.' The derivation of w* only gives the optimal weight for a fixed model and fixed gamma; it does not establish that the event {ell_i > gamma} coincides with, or even strongly overlaps, the corrupted-label set. Hard clean pairs with large intra-class variation can also have large ell_i and be discarded, while corrupted pairs whose features happen to align with the wrong label can have small ell_i and be retained. Figure 3 is only a qualitative density plot and does not quantify the overlap. Because the injected noise mask is known in these synthetic experiments, please report the precision, recall, and AUC of the w_i=0 decision with respect to that mask, and show that the discarded set is dominated by corrupted rather than hard clean pairs.","section":"Noise-tolerance Self-paced Hashing, Eqs. (7)-(12)"},{"comment":"The ablation row RSHNL-1 (removing the warm-up phase) reports exactly 84.3 for all four noise rates on XMedia and exactly 38.6 for all four noise rates on INRIA-Websearch. This is inconsistent with the expected behavior of a variant that removes the warm-up phase, and it undermines the claim that the warm-up component is crucial. Please provide corrected numbers for this row and clarify what RSHNL-1 actually runs.","section":"Ablation Study, Table 4"},{"comment":"All MAP results are reported as single values with no standard deviations, number of seeds, or significance tests. The strong claim of state-of-the-art performance across all noise rates and bit lengths would be substantially more convincing with at least three independent runs and mean±std for the main comparisons; this would also help assess whether the large reported margins are stable.","section":"Tables 1-3 and Experiments Settings"},{"comment":"The learning pace parameter gamma is load-bearing for the entire NSH mechanism, yet its selection is deferred: the text says 'we perform the sensitivity analysis in the appendix,' but the appendix is not part of the submitted manuscript. The paper should include the sensitivity analysis or a clear schedule for gamma in the main text or a provided appendix, along with the values of Nw, Nm, alpha, tau, and r used in each experiment.","section":"Theoretical Justification, Eq. (14) and following paragraph"}],"minor_comments":[{"comment":"The phrase 'the doji represents hash centers' appears to contain a typo; 'doji' should likely be 'circle' or 'center'.","section":"Figure 1 caption"},{"comment":"The x-axis begins at 0, but the fraction of instances with weight exactly 0 is not shown; a histogram with a zero-weight bar or a separate percentage would make the separation claim visible.","section":"Figure 3"},{"comment":"The upper bound for gamma is loose because v_i^m is a convex combination of positive softmax outputs, so the maximum value of ell_i cannot actually be attained; this should be acknowledged.","section":"Eq. (14)"},{"comment":"The novelty claim that this is 'the first work' introducing SPL to distinguish noisy labels in CMH is hedged, but the related-work section would benefit from a more explicit comparison with existing robust hashing methods such as CMMQ and DHRL that also down-weight unreliable samples.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable candidate after revision, but the two main concerns are the unvalidated noise-selection rule and the anomalous RSHNL-1 ablation row. The submission also needs the referenced appendix, since the sensitivity analysis for gamma is essential to reproduce the method."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: RSHNL is a genuine new combination — self-paced learning applied to separate noisy labels in cross-modal hashing, with contrastive hashing and center aggregation as supporting loss terms. The experiments are extensive: four datasets, four noise rates, four bit lengths, 11 baselines, and the gains over prior noisy-label CMH methods are large and consistent. Code is public. On the strength of the empirical work alone, this deserves referee time.\n\nThe new piece is the NSH module: a linear SPL regularizer yields w_i = max(0, 1 - ℓ_i/γ), so pairs with center-aggregation loss above γ get zero weight. That derivation is correct but standard. The claim that this separates noisy from clean labels is what carries the paper, and the evidence is mostly indirect. Fig. 3 shows density shifts in the weights for true vs. false labels, and the robustness analysis with RSHNL-D (all pairs allowed) supports the idea that dropping high-loss pairs helps. But the paper never reports how accurate the w_i=0 decision is against the known injected noise mask — no precision, recall, or AUC. The concern that hard clean pairs get discarded is real, though it's an addressable one. Adding a diagnostic that quantifies the separation would strengthen the paper considerably.\n\nThe other soft spots are smaller. The main tables report single MAP values with no error bars or significance tests; for a benchmark comparison that's common but the margins are sometimes a few points, so it would help. The ablation row RSHNL-1 (no warm-up) has identical values across all noise rates on both datasets (84.3 84.3 84.3 84.3; 38.6 38.6 38.6 38.6), which looks like the variant collapsed or a reporting error — needs checking. Wikipedia results are deferred to an appendix that wasn't in the copy I saw, so I can't verify them. Implementation details are also there.\n\nOverall the central empirical claim is likely valid: RSHNL does better than the baselines at this task, and the components all seem to contribute. The mechanism is plausible but not proven. For a CV/MM conference paper, that's a normal state of affairs. I don't see a load-bearing flaw. The right call is to engage, get the revision with better statistics and the noise-selection diagnostic, and then it can be accepted.","headline":"Genuine new combination with large benchmark gains; the noise-detection mechanism is plausible but not directly validated, and the experiments need error bars and one suspicious ablation row checked.","tokens_in":16462,"tokens_out":2280,"would_cite":true,"duration_ms":22016,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A self-paced regularizer that assigns zero weight to high-loss sample pairs lets cross-modal hashing keep high retrieval accuracy even when 80 percent of labels are randomly corrupted.","keywords":["cross-modal hashing","noisy labels","self-paced learning","contrastive hashing","center aggregation","cross-modal retrieval","Hamming space"],"falsifier":"Measure the centroids of clean and noisy sample-pair losses at the epoch when the threshold is applied: if the distributions overlap substantially—for example, if hard-but-correctly-labeled pairs have $\\ell_i$ above $\\gamma$ as often as mislabeled pairs—then the zero-weight rule discards valuable clean data. A direct experiment would corrupt labels in a controlled way, track which pairs receive weight zero across training, and compare that set to the known ground-truth noise; low precision or low recall would show the loss does not separate noise from hard clean samples.","tokens_in":15426,"feed_emoji":"🔎","tokens_out":6057,"duration_ms":52977,"temperature":0.7,"pith_summary":"This paper proposes RSHNL, a cross-modal hashing method built from three losses: contrastive hashing learning to align modalities, center aggregation learning to pull same-class hash codes to shared centers, and a noise-tolerance self-paced hashing loss that weights each sample pair by its estimated learning difficulty. The central claim is that using a self-paced regularizer to zero out high-loss pairs—treated as mislabeled—lets the hash model learn clean pairs from easy to hard, making retrieval robust to symmetric label noise up to 80 percent. Experiments on four benchmark datasets compare RSHNL with 11 methods, and it reports the highest MAP scores in nearly all settings. The contribution is framed as the first use of self-paced learning to distinguish and eliminate noisy labels specifically in cross-modal hashing.","feed_headline":"Self-paced hashing beats label noise up to 80%","feed_subtitle":"Zero-weight filtering of high-loss pairs keeps retrieval accuracy high on four benchmarks.","key_machinery":"The load-bearing object is the self-paced regularizer $R(w_i, \\gamma) = \\gamma\\bigl(\\tfrac{1}{2}w_i^2 - w_i\\bigr)$ inside the noise-tolerance self-paced hashing loss $L_S$. Minimizing $w_i\\ell_i + R(w_i,\\gamma)$ with $w_i \\in [0,1]$ yields the closed-form weight $w^*_i = \\max\\bigl(0, 1 - \\ell_i/\\gamma\\bigr)$, which is the mechanism that both detects noisy labels (weight 0 when $\\ell_i > \\gamma$) and implements easy-to-hard curriculum learning. The paper also bounds $\\gamma$ between $0$ and $M(r^2 - r + 1)/r$ using the minimum and maximum of $\\ell_i$ across the $M$ modalities, where $r$ is the weight factor in the generalized contrastive loss.","core_discovery":"RSHNL's core discovery is that the learning-difficulty weights of a self-paced regularizer can double as a noise detector for cross-modal hashing. Under the proposed noise-tolerance self-paced hashing loss, the optimal weight for a sample pair is $w^*_i = \\max(0, 1 - \\ell_i/\\gamma)$, where $\\ell_i$ is that pair's center-aggregation loss and $\\gamma$ is the pace parameter: pairs with $\\ell_i > \\gamma$ receive weight zero and are treated as mislabeled, while remaining pairs are weighted by their easiness and enter training progressively. The paper argues that because clean pairs begin with smaller center-aggregation loss, this schedule both filters noise and orders learning from easy to hard, and the reported experiments show RSHNL retaining high MAP scores where competing supervised methods degrade sharply as the noise rate rises.","pith_inferences":["The same weight-threshold scheme could be applied to other pair-based losses, such as ranking or metric-learning objectives, wherever a per-pair loss is computable during training.","A natural stress test beyond symmetric noise is asymmetric or instance-dependent label noise, which the paper does not evaluate; the threshold rule may need a noise-rate-dependent or adaptively scheduled $\\gamma$ in that setting.","Because the detector is purely loss-based and relies on early-training robustness, its success may hinge on the warm-up epoch $N_w$; tuning that schedule could be decisive on datasets with intrinsically harder examples.","The novelty statement that this is the first SPL-based noise discrimination in CMH is a historical claim; the practical contribution—a single hyperparameter that controls both filtering and curriculum—is what later work is likely to build on."],"forward_implications":["Under symmetric label noise from 0.2 to 0.8, RSHNL reports higher MAP than 11 compared CMH methods across XMedia, INRIA-Websearch, XMediaNet, and Wikipedia, including methods specifically designed for noisy labels.","Removing the self-paced mechanism (the variant that forces all weights to 1) leads to severe overfitting and a sharp MAP drop, indicating that the zero-weight filtering is the component that confers noise robustness.","The closed-form weight update means the noise-tolerance mechanism adds no extra learned parameters; the pace parameter $\\gamma$ is bounded and set by sensitivity analysis.","Because the weights evolve during training, the model incorporates more pairs over time, so the method behaves as a curriculum that ends with nearly all clean pairs included, as shown by the density plots of instance weights.","The performance degradation of most supervised baselines as bit length grows under high noise is reported, while RSHNL stays comparatively stable, suggesting the noise filter prevents long hash codes from absorbing corrupted semantic information."],"supporting_citations":[{"why":"Introduces self-paced learning, the paradigm from which the noise-tolerance regularizer is derived.","marker":"(Kumar, Packer, and Koller 2010)"},{"why":"Supplies the linear interpolation self-paced regularizer that produces the closed-form weight update in Eq. (12).","marker":"(Jiang et al. 2014b)"},{"why":"Documents the early-training robustness of deep networks, which justifies the warm-up phase before the noise-tolerance loss is applied.","marker":"(Song et al. 2022b)"},{"why":"CMMQ, a cross-modal hashing method that selects confident samples via the memorization effect; it is the primary noise-robust baseline RSHNL outperforms.","marker":"(Yang et al. 2022)"},{"why":"DHRL, a cross-modal hashing method for noisy labels using ranking and swapping for uncertainty estimation; it serves as a noisy-label SOTA comparison.","marker":"(Shu et al. 2024)"},{"why":"DSCMH, a dual self-paced cross-modal hashing method that handles feature noise but not label noise, marking the gap RSHNL fills.","marker":"(Sun et al. 2024a)"},{"why":"DHaPH, a self-paced hashing baseline that upweights hard pairs, the opposite strategy to RSHNL's zero-weight noise filtering.","marker":"(Huo et al. 2024)"}],"fun_headline_variants":["Self-paced hashing filters noisy labels via difficulty weights","Noise-tolerant hashing: easy-first learning cleans labels","Cross-modal hashing that learns from clean pairs only","RSHNL: Self-paced weights spot mislabeled pairs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"A sample pair's center-aggregation loss reliably indicates whether its label is wrong: noisy pairs are assumed to always have higher loss than clean pairs at the stage where the threshold is applied.","fun_headline_variants_meta":{"raw":{"variants":["Self-paced hashing filters noisy labels via difficulty weights","Noise-tolerant hashing: easy-first learning cleans labels","Cross-modal hashing that learns from clean pairs only","RSHNL: Self-paced weights spot mislabeled pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000701,"raw_usage":{"total_tokens":3193,"prompt_tokens":1001,"completion_tokens":2192,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":2127}},"tokens_in":617,"tokens_out":2192,"duration_ms":14847,"temperature":1.0,"reasoning_tokens":2127,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:21:11.432739+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the centroids of clean and noisy sample-pair losses at the epoch when the threshold is applied: if the distributions overlap substantially—for example, if hard-but-correctly-labeled pairs have $\\ell_i$ above $\\gamma$ as often as mislabeled pairs—then the zero-weight rule discards valuable clean data. A direct experiment would corrupt labels in a controlled way, track which pairs receive weight zero across training, and compare that set to the known ground-truth noise; low precision or low recall would show the loss does not separate noise from hard clean samples.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces self-paced learning, the paradigm from which the noise-tolerance regularizer is derived."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CMMQ, a cross-modal hashing method that selects confident samples via the memorization effect; it is the primary noise-robust baseline RSHNL outperforms."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DHRL, a cross-modal hashing method for noisy labels using ranking and swapping for uncertainty estimation; it serves as a noisy-label SOTA comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DHaPH, a self-paced hashing baseline that upweights hard pairs, the opposite strategy to RSHNL's zero-weight noise filtering."}],"review_version":1}