{"id":"fb5cf3bd-e862-4cf4-8d34-274cfa61f575","arxiv_id":"2607.20874","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A dual-branch multi-label contrastive learning framework with self-corrected noisy labels improves webly supervised multi-label recognition on two new benchmark datasets.","lead":"The paper builds two large web-scraped benchmarks for multi-label recognition (Web-COCO, Web-Pascal) and proposes a dual-branch contrastive learning method that corrects noisy web labels during training. The method reports improved average precision over several baselines on the COCO and Pascal VOC test sets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported superiority may be confounded by unequal weight decay: DBMLCL uses wd=0 while baselines use paper-specific decays, so mAP gains in Table 1 are not isolated from a regularization effect.","rationale":"The reader's CONDITIONAL verdict is appropriate, but the most direct threat to the central claim is not exactly the label-correction loop; it is the fairness of the headline comparison. The paper explicitly says baselines use their original weight-decay settings while DBMLCL uses wd=0, a deviation from the 'unified' protocol. Regularization strength is a known sensitivity in noisy-label training, and the authors state that wd=0 is chosen to help fit corrected labels. Without an ablation that isolates this single hyperparameter, the reported mAP gains in Table 1 could stem from this choice rather than from the proposed architecture, contrastive losses, or label correction. The reader's concern about the label-correction loop is related but secondary: §4.3 uses model predictions to overwrite labels with no held-out clean set to halt self-reinforcement, and Eq. (12) as written updates thresholds only on corrected-positive samples, which would make false-positive removal ineffective; however, even fixing that would not address the weight-decay confound. Credit is due for constructing a useful benchmark, releasing code/data links, and including ablations showing that the dual-branch design and contrastive losses help over its own baseline, but the lack of weight-decay control leaves the superiority claim conditional. Hence the reader's verdict remains unchanged.","tokens_in":16398,"tokens_out":10412,"duration_ms":101509,"concrete_test":"Run a 2×2 hyperparameter study on Web-COCO 100%: {DBMLCL, CSRA} × {wd=0, wd=1e-4}, with all other settings from §5.1.1 held fixed. If CSRA with wd=0 reaches or exceeds DBMLCL with wd=0, or if DBMLCL with wd=1e-4 no longer beats CSRA, the reported margin is a regularization artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"§5.1.1 promises 'a unified training strategy for all algorithms,' but the experiments do not actually unify regularization: baselines keep their original weight-decay settings, while §4.4 says DBMLCL 'set the weight decay parameter to 0.' Since wd=0 is chosen deliberately to make the model fit corrected labels, and noisy-web multi-label training is sensitive to regularization strength, the Table 1 mAP gaps (68.6/70.1/71.4 vs. CSRA 64.9/67.5/70.3) cannot be attributed to the proposed dual-branch contrastive learning or label correction unless this hyperparameter is controlled. The Table 4 ablation compares DBMLCL to a dual-branch SSGRL with the same strategy, but it does not vary weight decay, and no Table 1 baseline is re-run with wd=0. This is the most load-bearing threat to the central 'superior performance' claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a benchmark for webly supervised multi-label recognition (WS-MLR), comprising two new web-retrieved training sets, Web-COCO and Web-Pascal, aligned with the MS-COCO and Pascal VOC category vocabularies. It re-implements several multi-label recognition baselines under a supposedly common evaluation protocol and proposes a Dual-Branch Multi-Label Contrastive Learning (DBMLCL) framework that combines instance-level and prototype-level contrastive losses with a self-training label-correction module. The label-correction module uses the model's own prediction probabilities and feature-prototype similarities to identify and rewrite noisy keyword-derived labels. Experiments on Web-COCO and Web-Pascal report improved mAP, OF1, and CF1 over the re-implemented baselines at 20%, 40%, and 100% training-data proportions.","tokens_in":16731,"tokens_out":7159,"duration_ms":72106,"significance":"If the benchmark is made available and the results are reproducible, the WS-MLR benchmark itself is a potentially valuable community resource: the datasets are large, category-aligned with widely used test sets, and evaluation on the original manually annotated COCO/VOC validation/test splits is the correct protocol. The paper also quantifies keyword-label noise (precision 46.1%, recall 64.6%), which is useful for future work. However, the central empirical claim is currently weakened by a protocol confound and by a logical gap in the label-correction mechanism. The proposed label correction, as written, cannot remove false-positive labels, and the claimed improvements over baselines are not isolated from a difference in weight decay. These issues are fixable within the manuscript's scope, but the experiments and derivations must be corrected before the claim of 'superior performance' can be considered established.","major_comments":[{"comment":"The adaptive threshold update in Eq. (12) is restricted to samples with ŷ_i,k = 1. Since θ2 and α2 are initialized to 0 and never receive an update from a sample with ŷ_i,k = 0, θ2 remains 0 throughout training. Because p_i,k is the average of two sigmoid outputs, p_i,k ∈ (0,1), so the condition p_i,k < θ2 in Eq. (8) can never be true. Consequently the algorithm cannot detect or correct any false-positive keyword label, contradicting the claims in §5.4.4 and Figure 9. The update rule must be split into positive and negative cases, or otherwise updated from rectified negatives; otherwise the false-positive-correction component is vacuous. This must be fixed and the experiments rerun because it directly affects the method's central mechanism.","section":"§4.3, Eq. (12), Algorithm 1"},{"comment":"The paper claims a unified training strategy for all algorithms, but §4.4 sets weight decay to 0 for DBMLCL while §5.1.1 keeps 'the original settings in each paper' for all baselines. As a result, the Table 1 mAP/OF1/CF1 gaps (e.g., 71.4 vs. 70.3 for CSRA at 100% data) are not attributable to the proposed contrastive learning and label correction alone; they may reflect the regularization difference, which is known to matter strongly when training on noisy labels. Please rerun all Table 1 baselines with weight decay 0, and DBMLCL with the baselines' decays, or otherwise hold this hyperparameter fixed. Report the sensitivity of the main result to this choice.","section":"§5.1.1 and §4.4"},{"comment":"All reported results are single-run with no error bars, standard deviations, or significance tests. Webly supervised training is stochastic and the noise level is high; the Table 1 improvements over the second-best method shrink from 3.6 mAP at 20% data to 1.1 mAP at 100% data. Please report at least 3 seeds with means and standard deviations for the main comparisons, or explicitly justify why the differences are robust to training stochasticity.","section":"§5.2, Tables 1 and 2"},{"comment":"The label-correction thresholds are derived from the model's own probability outputs and feature-prototype similarities (Eqs. 8–12) after only a 5-epoch warm-up, with no external clean set used to monitor or halt the self-training loop. If the early model is confidently wrong on a category, corrections will reinforce its errors. The paper does analyze corrected labels on 20,000 manually annotated samples (§5.4.4), which is helpful, but this analysis is post-hoc and not used to select or stop the training. Please add a diagnostic (e.g., correction precision/recall over training epochs on that annotated subset) or compare with fixed-threshold correction and with a variant that uses the clean subset for threshold selection or early stopping, to show the loop does not drift.","section":"§4.3 label-correction loop"}],"minor_comments":[{"comment":"Line 12 contains a typo: 'if then' should be 'if ... then'.","section":"Algorithm 1"},{"comment":"The notation in Eq. (12) is ambiguous. As written, the update condition set {i | ŷ_i,k = 1} applies to both α and θ, but the text says all thresholds are adaptive. Please clarify the update rule for θ2 and α2.","section":"Eq. (12)"},{"comment":"The abstract and Section 3 give a GitHub URL for datasets/code, while the Data Availability section gives a Baidu Cloud link with a password. Please verify both links and state the license and access terms.","section":"Data availability"},{"comment":"Please define whether 'retrieved labels' indicates only added positive labels or also removed false positives. Given the threshold issue in Eq. (12), the figure currently cannot support the claim that false-positive labels are corrected.","section":"Figure 8"}],"recommendation":"major_revision","confidential_remarks":"The benchmark is likely the most valuable contribution and could justify publication after correction. The threshold-update bug and the weight-decay confound are serious; if the experiments are rerun with fixed thresholds and controlled regularization and the gains persist, I would support acceptance. Please ensure the authors have the resources to rerun the baselines with the controlled settings, as this is required to validate the central claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper has two parts: a new WS-MLR benchmark (Web-COCO, Web-Pascal; ~300k web images, 80/20 categories, clean COCO/VOC test splits) and a dual-branch contrastive-label-correction method (DBMLCL). The benchmark is the more solid contribution. The datasets are collected and filtered sensibly, the noise analysis (46.1% precision, 64.6% recall on Web-COCO labels) is honest, and the re-implementations of six baselines under a common protocol are a real service to this subfield. The method is reasonable: two branches, instance-level and prototype contrastive losses, and a self-training loop that overwrites keyword labels based on prediction confidence and feature-prototype similarity.\n\nWhat I'd flag: the headline 'superior performance' claim is undercut by a hyperparameter confound. Section 5.1.1 promises a unified training strategy, but weight decay 'adopts the original settings' for baselines, while Section 4.4 sets DBMLCL's weight decay to 0 to 'encourage the model to fit the corrected labels.' That is a deliberate, method-specific change, and noisy-label training is known to be sensitive to regularization. Table 1 gains (e.g., 68.6 vs 64.9 mAP at 20% data) could be partly due to reduced regularization rather than the contrastive losses or label correction. The ablation in Table 4 compares DBMLCL against dual-branch SSGRL variants, but apparently never varies weight decay, so it doesn't isolate the effect. This is fixable — rerun baselines with wd=0 and run DBMLCL with wd>0 — but as reported, the central comparison is not conclusive.\n\nAlso, the label-correction loop is self-referential. The thresholds and corrected labels come from the model's own predictions and prototypes, with no external clean set to monitor drift. The paper does validate corrected labels on 20k annotated samples (Figure 8), which is useful, but that's after training, not during. A small clean validation set or early-stopping based on held-out labels would strengthen the design.\n\nMinor issues: all results are single-run, no error bars or significance tests; the threshold update in Eq. 12 only uses samples corrected to positive, which looks asymmetric and needs justification.\n\nBottom line: the benchmark and reimplementations are valuable, and the method is plausible. The paper deserves a proper review, but the authors need to control weight decay and report variance before I'd trust the 1–3.6 mAP gains. I'd send it to reviewers and bring the benchmark to a reading group, but I wouldn't cite the superiority claim as settled.","headline":"Useful benchmark and plausible method, but the headline superiority claim isn't clean because DBMLCL trains with weight decay 0 while baselines use their own, and the label-correction loop lacks an external check.","tokens_in":17111,"tokens_out":5145,"would_cite":true,"duration_ms":45616,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Webly supervised multi-label recognition is feasible if noisy keyword labels are corrected at the category level, and the paper's dual-branch contrastive method shows a 6.8-point mAP gain on an 80-category benchmark.","keywords":["webly supervised learning","multi-label recognition","label noise","contrastive learning","prototype learning","label correction","evaluation benchmark","image classification"],"falsifier":"Take a random sample of web images that were never used for training, have humans label them, and compare the original keyword labels and the model's corrected labels against those human labels. If the corrected labels are not more precise, or not more balanced in precision and recall, than the original keyword labels on this sample, the correction mechanism is not doing what the paper claims. An even more direct test is to train the same network with the correction module disabled and see whether the reported mAP gap disappears.","tokens_in":16373,"feed_emoji":"🖼️","tokens_out":7016,"duration_ms":62768,"temperature":0.7,"pith_summary":"This paper is trying to establish that multi-label image recognition models can be trained on noisy web-search keyword labels instead of clean manual annotations, and that the noise does not have to be fatal. To do this it assembles two large web-retrieved datasets over the 80 and 20 categories of two standard multi-label benchmarks, and measures that the keyword-derived labels are only about 46% precise and 65% recall. The central claim is that a dual-branch network that learns category-specific instance features and category prototypes, then uses the similarity between them to overwrite suspicious labels, outperforms representative existing methods on both datasets: on the 80-category set it raises mAP from 64.6 to 71.4 at 100% data. If this holds, webly supervised multi-label recognition becomes a workable alternative for scaling recognition systems without manual label cost.","feed_headline":"Noisy web labels corrected with category prototypes, mAP up 6.8 points","feed_subtitle":"Only 46% of web keyword labels are precise; the method rewrites them via feature-to-prototype similarity to reach 71.4 mAP.","key_machinery":"DBMLCL (Dual-Branch Multi-Label Contrastive Learning) is the load-bearing machinery: two branches with unshared parameters, each trained on a disjoint half of the data. Two contrastive losses do the work: an instance contrastive loss makes the two branches' category-level feature vectors agree for positive categories and disagree with a queue of negatives, and a prototype contrastive loss makes each feature vector cling to its own category prototype and repel others. The resulting feature–prototype similarity, averaged over the two branches, is what the label-correction rule uses to retrieve missing labels and delete false positives, with adaptive per-category thresholds updated by momentum.","core_discovery":"The paper's central claim is that label noise in webly supervised multi-label recognition should be modeled and corrected at the category level rather than at the level of whole images. Each training image is passed through two branches of the same network; each branch produces a feature vector per category, and the network maintains a running prototype vector per category. After a five-epoch warm-up, the model uses its predicted probabilities to flag label entries that look wrong, and then rewrites those entries according to the cosine similarity between the image's category feature and the category prototype. The authors report that this correction loop yields the best mAP, overall F1, and","pith_inferences":["Because correction is conservative, the method likely under-recovers rare, small, or heavily occluded categories; a testable tweak is to set category-specific recall targets and adjust the similarity thresholds until the precision/recall trade-off matches class frequency.","The correction loop has no external clean labels, so threshold drift is a real risk; one cheap safeguard would be to reserve a small human-labeled sample purely to monitor whether flipped labels become more accurate as training proceeds.","The two-branch design doubles training cost, but since the 50%/50% split loses little, the same contrastive agreement could be distilled into a single branch at inference, cutting parameters and FLOPs in half.","The same category-specific prototype and noise-correction recipe could transfer to other tag-supervised tasks, such as video multi-label recognition or noisy web retrieval, where labels come from queries rather than human annotators."],"forward_implications":["Multi-label models can be trained from hundreds of thousands of keyword-tagged web images, with the main remaining cost being compute rather than annotation.","Category-level feature–prototype similarity is a practical signal for both recovering missing labels and rejecting false positives in noisy supervision.","The benchmark's fixed protocol lets future WS-MLR algorithms be compared on the same training data, same evaluation sets, and same metrics.","The reported ablation shows the gain comes from contrastive learning plus correction, not from merely doubling the network branches.","A vision-language baseline can achieve high ranking mAP but lower F1 on this data, implying that ranking quality is not enough for usable multi-label predictions."],"fun_headline_variants":["Webly multi-label recognition gets a benchmark and dual-branch contrastive method","Category prototypes correct noisy web labels, mAP up 6.8 points","Web-COCO and Web-Pascal: New benchmark for webly supervised multi-label recognition","Dual-branch contrastive learning fixes label noise in webly multi-label recognition"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that, in the label-correction step, after five warm-up epochs the model's predictions and feature–prototype similarities are already reliable enough to overwrite the keyword labels, and that this self-correction makes labels more accurate rather than locking in the model's early mistakes.","fun_headline_variants_meta":{"raw":{"variants":["Webly multi-label recognition gets a benchmark and dual-branch contrastive method","Category prototypes correct noisy web labels, mAP up 6.8 points","Web-COCO and Web-Pascal: New benchmark for webly supervised multi-label recognition","Dual-branch contrastive learning fixes label noise in webly multi-label recognition"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000537,"raw_usage":{"total_tokens":2399,"prompt_tokens":708,"completion_tokens":1691,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":452,"completion_tokens_details":{"reasoning_tokens":1605}},"tokens_in":452,"tokens_out":1691,"duration_ms":13547,"temperature":1.0,"reasoning_tokens":1605,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T09:05:01.607841+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of web images that were never used for training, have humans label them, and compare the original keyword labels and the model's corrected labels against those human labels. If the corrected labels are not more precise, or not more balanced in precision and recall, than the original keyword labels on this sample, the correction mechanism is not doing what the paper claims. An even more direct test is to train the same network with the correction module disabled and see whether the reported mAP gap disappears.","supporting_citations":[],"review_version":1}