REVIEW 4 major objections 4 minor 41 references
Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels
T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A self-paced regularizer that assigns zero weight to high-loss sample pairs lets cross-modal hashing keep high retrieval accuracy even when 80 percent of labels are randomly corrupted.
desk verdict Genuine new combination with large benchmark gains; the noise-detection mechanism is plausible but not directly validated, and the experiments need error bars and one suspicious ablation row checked. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the self-paced regularizer $R(w_i, \gamma) = \gamma\bigl(\tfrac{1}{2}w_i^2 - w_i\bigr)$ inside the noise-tolerance self-paced hashing loss $L_S$. Minimizing $w_i\ell_i + R(w_i,\gamma)$ with $w_i \in [0,1]$ yields the closed-form weight $w^*_i = \max\bigl(0, 1 - \ell_i/\gamma\bigr)$, which is the mechanism that both detects noisy labels (weight 0 when $\ell_i > \gamma$) and implements easy-to-hard curriculum learning. The paper also bounds $\gamma$ between $0$ and $M(r^2 - r + 1)/r$ using the minimum and maximum of $\ell_i$ across the $M$ modalities, where $r$ is the weight factor in the generalized contrastive loss.
What would settle it
Measure the centroids of clean and noisy sample-pair losses at the epoch when the threshold is applied: if the distributions overlap substantially—for example, if hard-but-correctly-labeled pairs have $\ell_i$ above $\gamma$ as often as mislabeled pairs—then the zero-weight rule discards valuable clean data. A direct experiment would corrupt labels in a controlled way, track which pairs receive weight zero across training, and compare that set to the known ground-truth noise; low precision or low recall would show the loss does not separate noise from hard clean samples.
Extended reading notes
Core claim
RSHNL's core discovery is that the learning-difficulty weights of a self-paced regularizer can double as a noise detector for cross-modal hashing. Under the proposed noise-tolerance self-paced hashing loss, the optimal weight for a sample pair is $w^*_i = \max(0, 1 - \ell_i/\gamma)$, where $\ell_i$ is that pair's center-aggregation loss and $\gamma$ is the pace parameter: pairs with $\ell_i > \gamma$ receive weight zero and are treated as mislabeled, while remaining pairs are weighted by their easiness and enter training progressively. The paper argues that because clean pairs begin with smaller center-aggregation loss, this schedule both filters noise and orders learning from easy to hard, and the reported experiments show RSHNL retaining high MAP scores where competing supervised methods degrade sharply as the noise rate rises.
Load-bearing premise
A sample pair's center-aggregation loss reliably indicates whether its label is wrong: noisy pairs are assumed to always have higher loss than clean pairs at the stage where the threshold is applied.
Editorial extensions
If this is right
- Under symmetric label noise from 0.2 to 0.8, RSHNL reports higher MAP than 11 compared CMH methods across XMedia, INRIA-Websearch, XMediaNet, and Wikipedia, including methods specifically designed for noisy labels.
- Removing the self-paced mechanism (the variant that forces all weights to 1) leads to severe overfitting and a sharp MAP drop, indicating that the zero-weight filtering is the component that confers noise robustness.
- The closed-form weight update means the noise-tolerance mechanism adds no extra learned parameters; the pace parameter $\gamma$ is bounded and set by sensitivity analysis.
- Because the weights evolve during training, the model incorporates more pairs over time, so the method behaves as a curriculum that ends with nearly all clean pairs included, as shown by the density plots of instance weights.
- The performance degradation of most supervised baselines as bit length grows under high noise is reported, while RSHNL stays comparatively stable, suggesting the noise filter prevents long hash codes from absorbing corrupted semantic information.
Reading between the lines
- The same weight-threshold scheme could be applied to other pair-based losses, such as ranking or metric-learning objectives, wherever a per-pair loss is computable during training.
- A natural stress test beyond symmetric noise is asymmetric or instance-dependent label noise, which the paper does not evaluate; the threshold rule may need a noise-rate-dependent or adaptively scheduled $\gamma$ in that setting.
- Because the detector is purely loss-based and relies on early-training robustness, its success may hinge on the warm-up epoch $N_w$; tuning that schedule could be decisive on datasets with intrinsically harder examples.
- The novelty statement that this is the first SPL-based noise discrimination in CMH is a historical claim; the practical contribution—a single hyperparameter that controls both filtering and curriculum—is what later work is likely to build on.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RSHNL, a supervised cross-modal hashing method designed to tolerate symmetric label noise. It combines three losses: a contrastive hashing loss (CHL) for cross-modal consistency, a center-aggregation loss (CAL) for intra-class compactness, and a self-paced hashing loss (NSH) that assigns an importance weight to each training pair based on its center-aggregation loss. For a fixed model, the optimal SPL weight is derived in closed form as max(0, 1 - ell_i/gamma), and pairs with ell_i > gamma are given zero weight and declared noisy. The method is evaluated on four datasets against 11 baselines under noise rates 0.2-0.8 and hash lengths 16-128, with code released on GitHub.
Significance. The paper's strengths are its breadth of evaluation and reproducibility: four datasets, four noise rates, four hash lengths, and a public code link. The closed-form SPL weight update is mathematically correct, and the reported MAP margins over prior methods are large, especially at high noise rates. If the noise-selection mechanism truly removes corrupted pairs, the method would be a practically useful contribution to noisy-label cross-modal hashing. However, the central claim that a high center-aggregation loss identifies label noise is not validated beyond a qualitative density plot, and several experimental reporting issues weaken the quantitative evidence. These issues are fixable but require additional experiments and corrections.
major comments (4)
- [Noise-tolerance Self-paced Hashing, Eqs. (7)-(12)] The key assumption of the method is that a high center-aggregation loss ell_i indicates a noisy label. The text after Eq. (12) states that when ell_i > gamma, 'we consider this sample pair could be mislabeled and assign the weight as zero.' The derivation of w* only gives the optimal weight for a fixed model and fixed gamma; it does not establish that the event {ell_i > gamma} coincides with, or even strongly overlaps, the corrupted-label set. Hard clean pairs with large intra-class variation can also have large ell_i and be discarded, while corrupted pairs whose features happen to align with the wrong label can have small ell_i and be retained. Figure 3 is only a qualitative density plot and does not quantify the overlap. Because the injected noise mask is known in these synthetic experiments, please report the precision, recall, and AUC of the w_i=0 decision with respect to that mask, and show that the discarded set is dominated by corrupted rather than hard clean pairs.
- [Ablation Study, Table 4] The ablation row RSHNL-1 (removing the warm-up phase) reports exactly 84.3 for all four noise rates on XMedia and exactly 38.6 for all four noise rates on INRIA-Websearch. This is inconsistent with the expected behavior of a variant that removes the warm-up phase, and it undermines the claim that the warm-up component is crucial. Please provide corrected numbers for this row and clarify what RSHNL-1 actually runs.
- [Tables 1-3 and Experiments Settings] All MAP results are reported as single values with no standard deviations, number of seeds, or significance tests. The strong claim of state-of-the-art performance across all noise rates and bit lengths would be substantially more convincing with at least three independent runs and mean±std for the main comparisons; this would also help assess whether the large reported margins are stable.
- [Theoretical Justification, Eq. (14) and following paragraph] The learning pace parameter gamma is load-bearing for the entire NSH mechanism, yet its selection is deferred: the text says 'we perform the sensitivity analysis in the appendix,' but the appendix is not part of the submitted manuscript. The paper should include the sensitivity analysis or a clear schedule for gamma in the main text or a provided appendix, along with the values of Nw, Nm, alpha, tau, and r used in each experiment.
minor comments (4)
- [Figure 1 caption] The phrase 'the doji represents hash centers' appears to contain a typo; 'doji' should likely be 'circle' or 'center'.
- [Figure 3] The x-axis begins at 0, but the fraction of instances with weight exactly 0 is not shown; a histogram with a zero-weight bar or a separate percentage would make the separation claim visible.
- [Eq. (14)] The upper bound for gamma is loose because v_i^m is a convex combination of positive softmax outputs, so the maximum value of ell_i cannot actually be attained; this should be acknowledged.
- [Related Work] The novelty claim that this is 'the first work' introducing SPL to distinguish noisy labels in CMH is hedged, but the related-work section would benefit from a more explicit comparison with existing robust hashing methods such as CMMQ and DHRL that also down-weight unreliable samples.
Circularity Check
No significant circularity: the NSH weight update is the closed-form solution of the stated self-paced regularizer, and retrieval gains are measured by held-out test MAP; the noise-separation assumption is an empirical heuristic, not a self-referential derivation.
full rationale
The paper's derivation chain is self-contained. The NSH loss in Eq. (7) defines a per-pair loss ℓ_i and a self-paced regularizer R(w_i, γ) = γ(½w_i² − w_i). Optimizing Eq. (10) over w_i with fixed network parameters yields the closed-form solution w*_i = max(0, 1 − ℓ_i/γ) in Eq. (12). This is a standard SPL update and is exactly the solution of the stated optimization problem; it does not presuppose which pairs are clean. The subsequent rule 'When ℓ_i > γ, we consider this sample pair could be mislabeled and assign the weight as zero' is an assumption that loss magnitude separates noisy from clean pairs. That assumption is empirically plausible but not proven by the equations; however, it is not circular, because the paper does not define ground-truth label noise as ℓ_i > γ. Instead, experiments inject symmetric noise and evaluate MAP on test data, and Fig. 3 compares the learned weights against known true/false label masks. The evaluation protocol reports MAP on the test set when it peaks on the validation set, so the method is not fitted to the test result. Self-citations to DSCMH (Sun et al. 2024a) and related SPL hashing work appear in the related-work discussion and as baselines; they are contextual and are not load-bearing for the claimed derivation or the experimental comparison. The anomalously identical ablation rows in Table 4 (RSHNL-1 at all noise rates) are an experimental reporting concern, not a circularity of the derivation. Overall, no load-bearing step reduces to its own inputs by construction.
Assumptions & free parameters
free parameters (5)
- alpha (weight of contrastive loss LC) =
unknown
- tau (temperature in CHL and CAL probabilities) =
unknown
- r (weight factor in losses) =
unknown
- gamma (self-paced learning pace) =
unknown
- Nw (warm-up epoch count) =
unknown
assumptions (3)
- domain assumption DNNs are robust in the early training stage, so warm-up training with Lp does not overfit noisy labels.
- ad hoc to paper A sample pair whose loss ell_i is greater than the pace parameter gamma is mislabeled.
- domain assumption Learning from easy to hard samples improves generalization and noise robustness.
Cite this review
Pith. "Pith review of Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels." pith.science (2026). https://pith.science/paper/X7N2SF5M
@misc{pith2026250101699,
author = {Pith},
title = {Pith review of: Robust Self-Paced Hashing for Cross-Modal Retrieval with Noisy Labels},
year = {2026},
howpublished = {\url{https://pith.science/paper/X7N2SF5M}},
note = {Machine review of arXiv:2501.01699}
}
read the original abstract
Cross-modal hashing (CMH) has appeared as a popular technique for cross-modal retrieval due to its low storage cost and high computational efficiency in large-scale data. Most existing methods implicitly assume that multi-modal data is correctly labeled, which is expensive and even unattainable due to the inevitable imperfect annotations (i.e., noisy labels) in real-world scenarios. Inspired by human cognitive learning, a few methods introduce self-paced learning (SPL) to gradually train the model from easy to hard samples, which is often used to mitigate the effects of feature noise or outliers. It is a less-touched problem that how to utilize SPL to alleviate the misleading of noisy labels on the hash model. To tackle this problem, we propose a new cognitive cross-modal retrieval method called Robust Self-paced Hashing with Noisy Labels (RSHNL), which can mimic the human cognitive process to identify the noise while embracing robustness against noisy labels. Specifically, we first propose a contrastive hashing learning (CHL) scheme to improve multi-modal consistency, thereby reducing the inherent semantic gap. Afterward, we propose center aggregation learning (CAL) to mitigate the intra-class variations. Finally, we propose Noise-tolerance Self-paced Hashing (NSH) that dynamically estimates the learning difficulty for each instance and distinguishes noisy labels through the difficulty level. For all estimated clean pairs, we further adopt a self-paced regularizer to gradually learn hash codes from easy to hard. Extensive experiments demonstrate that the proposed RSHNL performs remarkably well over the state-of-the-art CMH methods.
Figures
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Cao, Y.; Gao, Y.; Chen, N.; Lin, J.; and Chen, S. 2023. Generative Adversarial Network Based Asymmetric Deep Cross-Modal Unsupervised Hashing. In International Conference on Algorithms and Architectures for Parallel Processing, 30--48. Springer
work page 2023
-
[4]
Chen, N.; Cao, Y.; and Liu, C. 2021. Deep Cross-Modal Supervised Hashing Based on Joint Semantic Matrix. In International Conference on Network and System Security, 258--274. Springer
work page 2021
-
[5]
Hu, P.; Zhu, H.; Lin, J.; Peng, D.; Zhao, Y.-P.; and Peng, X. 2022. Unsupervised contrastive cross-modal hashing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3): 3877--3889
work page 2022
-
[6]
Huo, Y.; Qin, Q.; Zhang, W.; Huang, L.; and Nie, J. 2024. Deep Hierarchy-aware Proxy Hashing with Self-paced Learning for Cross-modal Retrieval. IEEE Transactions on Knowledge and Data Engineering, 1--14
work page 2024
-
[7]
Jiang, L.; Meng, D.; Yu, S.-I.; Lan, Z.; Shan, S.; and Hauptmann, A. 2014 a . Self-paced learning with diversity. Advances in Neural Information Processing Systems, 27
work page 2014
-
[8]
Jiang, L.; Meng, D.; Yu, S.-I.; Lan, Z.; Shan, S.; and Hauptmann, A. 2014 b . Self-paced learning with diversity. Advances in Neural Information Processing Systems, 27
work page 2014
Show all 41 references
-
[9]
Jiang, Q.-Y.; and Li, W.-J. 2017. Deep cross-modal hashing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3232--3240
2017
-
[10]
Krapac, J.; Allan, M.; Verbeek, J.; and Juried, F. 2010. Improving web image search results using query-relative classifiers. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1094--1101. IEEE
2010
-
[11]
Kumar, M.; Packer, B.; and Koller, D. 2010. Self-paced learning for latent variable models. Advances in Neural Information Processing Systems, 23
2010
-
[12]
Kuznetsova, A.; Rom, H.; Alldrin, N.; Uijlings, J.; Krasin, I.; Pont-Tuset, J.; Kamali, S.; Popov, S.; Malloci, M.; Kolesnikov, A.; et al. 2020. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. Internationa...
2020
-
[13]
Li, F.; Wang, B.; Zhu, L.; Li, J.; Zhang, Z.; and Chang, X. 2024 a . Cross-Domain Transfer Hashing for Efficient Cross-modal Retrieval. IEEE Transactions on Circuits and Systems for Video Technology
2024
-
[14]
Li, Y.; Qin, Y.; Sun, Y.; Peng, D.; Peng, X.; and Hu, P. 2024 b . RoMo: Robust Unsupervised Multimodal Learning With Noisy Pseudo Labels. IEEE Transactions on Image Processing
2024
-
[15]
Liang, J.; Li, Z.; Cao, D.; He, R.; and Wang, J. 2016. Self-paced cross-modal subspace matching. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, 569--578
2016
-
[16]
Liu, K.; Gong, Y.; Cao, Y.; Ren, Z.; Peng, D.; and Sun, Y. 2024. Dual semantic fusion hashing for multi-label cross-modal retrieval. In International Joint Conferences on Artificial Intelligence Organization, IJCAI, 4569--4577
2024
-
[17]
Peng, Y.; Huang, X.; and Zhao, Y. 2018. An Overview of Cross-media Retrieval: Concepts, Methodologies, Benchmarks and Challenges. IEEE Transactions on Circuits and Systems for Video Technology, 2372–2385
2018
-
[18]
Peng, Y.; Zhai, X.; Zhao, Y.; and Huang, X. 2015. Semi-supervised cross-media feature learning with unified patch graph regularization. IEEE Transactions on Circuits and Systems for Video Technology, 26(3): 583--596
2015
-
[19]
T.; Peng, X.; and Hu, P
Qin, Y.; Sun, Y.; Peng, D.; Zhou, J. T.; Peng, X.; and Hu, P. 2023. Cross-modal Active Complementary Learning with Self-refining Correspondence. Advances in Neural Information Processing Systems, 36
2023
-
[20]
R.; Levy, R.; and Vasconcelos, N
Rasiwasia, N.; Costa Pereira, J.; Coviello, E.; Doyle, G.; Lanckriet, G. R.; Levy, R.; and Vasconcelos, N. 2010. A new approach to cross-modal multimedia retrieval. In Proceedings of the 18th ACM International Conference on Multimedia, 251--260
2010
-
[21]
Shu, Z.; Bai, Y.; Yong, K.; and Yu, Z. 2024. Deep Cross-Modal Hashing With Ranking Learning for Noisy Labels. IEEE Transactions on Big Data
2024
-
[22]
Song, H.; Kim, M.; Park, D.; Shin, Y.; and Lee, J.-G. 2022 a . Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems, 34(11): 8135--8153
2022
-
[23]
Song, H.; Kim, M.; Park, D.; Shin, Y.; and Lee, J.-G. 2022 b . Learning From Noisy Labels With Deep Neural Networks: A Survey. IEEE Transactions on Neural Networks and Learning Systems, 1–19
2022
-
[24]
Sun, Y.; Dai, J.; Ren, Z.; Chen, Y.; Peng, D.; and Hu, P. 2024 a . Dual Self-Paced Cross-Modal Hashing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 15184--15192
2024
-
[25]
Sun, Y.; Liu, K.; Li, Y.; Ren, Z.; Dai, J.; and Peng, D. 2024 b . Distribution Consistency Guided Hashing for Cross-Modal Retrieval. In Proceedings of the 32nd ACM International Conference on Multimedia, 5623--5632
2024
-
[26]
Sun, Y.; Qin, Y.; Peng, D.; Ren, Z.; Yang, C.; and Hu, P. 2024 c . Dual Self-Paced Hashing for Image Retrieval. IEEE Transactions on Multimedia
2024
-
[27]
Sun, Y.; Ren, Z.; Hu, P.; Peng, D.; and Wang, X. 2023. Hierarchical consensus hashing for cross-modal retrieval. IEEE Transactions on Multimedia, 26: 824--836
2023
-
[28]
Tan, W.; Zhu, L.; Li, J.; Zhang, H.; and Han, J. 2022. Teacher-student learning: Efficient hierarchical message aggregation hashing for cross-modal retrieval. IEEE Transactions on Multimedia, 25: 4520--4532
2022
-
[29]
Wang, L.; Qin, Y.; Sun, Y.; Peng, D.; Peng, X.; and Hu, P. 2024. Robust Contrastive Cross-modal Hashing with Noisy Labels. In Proceedings of the 32nd ACM International Conference on Multimedia, 5752--5760
2024
-
[30]
Wang, R.; Yu, G.; Zhang, H.; Guo, M.; Cui, L.; and Zhang, X. 2021. Noise-robust deep cross-modal hashing. Information Sciences, 581: 136--154
2021
-
[31]
Wei, J.; Xu, X.; Wang, Z.; and Wang, G. 2021. Meta self-paced learning for cross-modal matching. In Proceedings of the 29th ACM International Conference on Multimedia, 3835--3843
2021
-
[32]
Xu, T.; Liu, X.; Huang, Z.; Guo, D.; Hong, R.; and Wang, M. 2022. Early-learning regularized contrastive learning for cross-modal retrieval with noisy labels. In Proceedings of the 30th ACM International Conference on Multimedia, 629--637
2022
-
[33]
Yang, D.; Wu, D.; Zhang, W.; Zhang, H.; Li, B.; and Wang, W. 2020. Deep semantic-alignment hashing for unsupervised cross-modal retrieval. In Proceedings of the 2020 International Conference on Multimedia Retrieval, 44--52
2020
-
[34]
Yang, E.; Yao, D.; Liu, T.; and Deng, C. 2022. Mutual quantization for cross-modal search with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7551--7560
2022
-
[35]
Yu, J.; Zhou, H.; Zhan, Y.; and Tao, D. 2021. Deep graph-neighbor coherence preserving network for unsupervised cross-modal hashing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 4626--4634
2021
-
[36]
Zhang, C.; Li, H.; Gao, Y.; and Chen, C. 2023 a . Weakly-Supervised Enhanced Semantic-Aware Hashing for Cross-Modal Retrieval. IEEE Transactions on Knowledge and Data Engineering, 35(6): 6475--6488
2023
-
[37]
Zhang, X.; Liu, X.; Nie, X.; Kang, X.; and Yin, Y. 2023 b . Semi-supervised semi-paired cross-modal hashing. IEEE Transactions on Circuits and Systems for Video Technology
2023
-
[38]
Zhang, Z.; Luo, H.; Zhu, L.; Lu, G.; and Shen, H. T. 2023 c . Modality-Invariant Asymmetric Networks for Cross-Modal Hashing. IEEE Transactions on Knowledge and Data Engineering, 35(5): 5091--5104
2023
-
[39]
H.; and Hoon, G
Zhou, K.; Hassan, F. H.; and Hoon, G. K. 2023. The State of the Art for Cross-Modal Retrieval: A Survey. IEEE Access
2023
-
[40]
Zhu, L.; Wu, X.; Li, J.; Zhang, Z.; Guan, W.; and Shen, H. T. 2022. Work together: Correlation-identity reconstruction hashing for unsupervised cross-modal retrieval. IEEE Transactions on Knowledge and Data Engineering, 35(9): 8838--8851
2022
-
[41]
Zhu, L.; Zheng, C.; Guan, W.; Li, J.; Yang, Y.; and Shen, H. T. 2023. Multi-modal hashing for efficient multimedia retrieval: A survey. IEEE Transactions on Knowledge and Data Engineering, 36(1): 239--260
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.