{"id":"baf66b63-2a9b-4a2c-b0ab-22b378dec4cb","arxiv_id":"2506.19496","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"COLUR restores a noise-damaged model by having a clean teacher flag confident disagreements, unlearning them with label-smoothed gradient ascent, and relearning from mixed soft labels, beating nine baselines on four datasets.","lead":"A machine-learning framework repairs a model whose accuracy has been damaged by training on mislabeled data: it first unlearns the wrong labels, then relearns from cleaned soft labels. On four image datasets with up to 90 percent corrupted labels, it reports the highest restored accuracy among all compared methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparison protocol for MRR is unspecified and likely unfair; several LNL baselines underperform Degrade at 10% noise, so \"consistently outperforms SOTA\" is not yet supported.","rationale":"The reader's declared weakest assumption is that the teacher model f(θ0) is trustworthy for identifying noisy labels. That is a real mechanism-level condition, but in these experiments the teacher is the original same-domain model with high accuracy (e.g., 84.85% on CIFAR-10), so it is plausibly reliable for the tested settings. The more immediate threat to the central claim is the baseline comparison itself. The paper claims consistent superiority over SOTA methods, but never explains how SOTA LNL methods are adapted to model restoration. The implausibly low low-noise baseline numbers (CoTe. 45.12 at 10% CIFAR-10 noise, far below Degrade) indicate that the comparison protocol is not a standard LNL evaluation. If the baselines are handicapped by the adaptation, the headline result is not evidence of a better restoration method. This does not require rejecting the mechanism; the framework may still work, but the empirical claim needs a verified, fair baseline protocol. The reader's rationale does mention the MRR adaptation protocol as a condition, so there is partial agreement, but their formal weakest_assumption field points elsewhere. A CONDITIONAL verdict remains appropriate: the paper should be accepted only if the authors release exact baseline adaptation scripts and demonstrate that Co-teaching and similar methods recover their expected performance on low-noise splits; otherwise the comparison should be re-run. I do not see grounds for REJECT because the COLUR mechanism is coherent and the high-noise margins are large, and I do not see grounds for ACCEPT until the baseline protocol is verified.","tokens_in":25054,"tokens_out":6425,"duration_ms":64729,"concrete_test":"Run the authors' released baseline wrapper for Co-teaching on CIFAR-10 at 10% noise under the stated MRR protocol, and check whether the wrapper initializes from f(θ0) or f(θu), which subset of Du it trains on, and the epoch and learning-rate schedule; compare the final accuracy to Degrade (84.82). If Co-teaching cannot exceed Degrade, or if the wrapper trains only on Dn_u instead of the full Du, the Table 6 comparison is an artifact and the \"consistently outperforms SOTA\" claim is unsubstantiated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central empirical claim (abstract and Section 4.2) is that COLUR consistently outperforms SOTA LNL/MU methods after MRR. The comparisons in Tables 1 and 6 depend on how each baseline is adapted to the MRR setting, but Section 4.1 does not specify this protocol: whether baselines are initialized from f(θ0), from f(θu), or trained from scratch on Du, and with what epochs and data subsets. Several reported numbers are implausible under any standard LNL protocol. At CIFAR-10 with 10% noise, Du is 90% clean; Co-teaching on such data should reach roughly 85-90% accuracy, yet Table 6 reports CoTe. at 45.12 and CoTe.+ at 73.29, well below the Degrade reference of 84.82. Similarly, on CIFAR-100 at 10% noise, CoTe. reports 37.75 versus Degrade 63.70. These patterns suggest an artifact, such as baselines being trained only on the noisy subset Dn_u, or using incompatible initialization, loss, or stopping criteria. If the baselines are not run under a faithful MRR protocol, the uniformly favorable comparison does not establish the claimed superiority. This is more load-bearing than the teacher-reliability assumption because the original model is strong in these same-domain experiments; the baseline protocol directly determines whether the headline empirical claim is credible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces COLUR, a framework for restoring a model whose performance has degraded after incremental training on a dataset with noisy labels. The pipeline has three stages. First, learning: the original model f(θ0) is trained or fine-tuned on D_u to produce a degraded model f(θu). Second, unlearning: teacher/student disagreement sets are extracted, where the teacher is a copy of f(θ0), and high-confidence disagreements are unlearned via label-smoothed gradient ascent. Third, relearning: high-confidence agreements are relearned with label smoothing, and low-confidence samples are mixed with high-confidence samples via Mixup using soft labels. The unlearning and relearning steps are iterated. Experiments on CIFAR-10, CIFAR-100, Flower-102 and Oxford-IIIT Pet under symmetric and asymmetric noise at noise ratios from 10% to 90% compare COLUR with nine LNL and four MU baselines. The headline result is that COLUR achieves the highest accuracy in every reported setting, e.g., restoring CIFAR-10 from 30.04% (degraded) to 80.34% at 90% noise and exceeding the original model's accuracy at 50% noise.","tokens_in":25272,"tokens_out":8867,"duration_ms":86621,"significance":"The LUR-inspired framing is a genuinely useful reframing: instead of training an LNL model from scratch, the paper aims to repair a deployed model by unlearning the influence of noisy labels and relearning with refined soft labels. If the reported results are reproducible under a fair protocol, this would be a practical contribution, because full retraining is often too expensive and MU methods alone cannot correct labels. The paper also provides an anonymized code link and hyper-parameter tables, which aids reproducibility. The main weakness is that the empirical evidence, as currently presented, does not yet support the 'consistently outperforms SOTA' claim: the baseline-adaptation protocol is unspecified, several baseline numbers are difficult to reconcile with known LNL behavior, and no variance information is provided. The core idea is defensible and the defects are addressable, so the appropriate response is a major revision rather than rejection.","major_comments":[{"comment":"The protocol by which LNL and MU baselines are adapted to the MRR setting is not specified, and several reported baseline numbers are implausible under any standard LNL protocol. The text only says that hyperparameters were 'carefully tuned' and gives no initialization rule (from f(θ0), from f(θu), or from scratch), no training data subset (D_u, D_c^u, or D_n^u), and no epoch or stopping criterion. At CIFAR-10 with 10% noise, Co-teaching is reported at 45.12%, a result far below the Degrade reference of 84.82%; at CIFAR-100 with 10% noise, Co-teaching is 37.75% versus Degrade at 63.70%. On a 90%-clean dataset Co-teaching should be much closer to standard noisy-label training performance, so these numbers suggest an artifact in the adaptation rather than genuine method behavior. Because every claim of 'consistently outperforms' is built on these comparisons, the paper must state the exact adaptation protocol for every baseline and reconcile or correct these anomalous entries.","section":"Section 4.1, Tables 1 and 6"},{"comment":"The object being unlearned is ambiguous. In Eq. (6), Y^u is defined as the student's predictive label, and Eq. (9) builds S^u_τ from that predicted label. Eq. (11) then applies label smoothing to y^u and Eq. (12) runs gradient ascent on that quantity. The surrounding prose nevertheless describes S^u_τ as 'high-confidence noisy label data,' which suggests that the observed noisy labels from D_u should be the unlearning targets. If the implementation unlearns the student's own predicted labels, the method is not unlearning the influence of the noisy labels but of the model's current beliefs on disagreement samples; if it unlearns the observed labels, Eqs. (6)-(10) use a misleading notation. This distinction changes the mechanism and must be clarified. The same section also contains a concrete mismatch: Algorithm 1 defines A^u_τ and A^t_τ using c(x; θ_t, θ_u), whereas Eq. (14) defines the agreement confidence as c(x; θ^ul_u, θ_t) using the unlearned student.","section":"Section 3.3, Eqs. (6)-(12), Algorithm 1"},{"comment":"No experiment is run with multiple seeds, and no error bars, standard deviations, or significance tests are reported. This matters because several claimed wins are small: at 10% noise on Oxford-IIIT Pet, COLUR (92.59%) leads PENCIL (92.01%) by 0.58 points, and on CIFAR-100, COLUR (66.78%) leads PENCIL (65.02%) by 1.76 points. With a single run and no protocol details it is impossible to tell whether these margins are real or within run-to-run variation. At minimum, the paper should report means and standard deviations over at least three seeds for all methods and noise levels.","section":"Section 4.1, Tables 3 and 6"},{"comment":"The number of unlearning/relearning iterations N is never reported, and the unlearning update in Eq. (12) has no associated epoch count or stopping criterion. Table 5 lists τ_c, α_mix, α_ls, learning rates, and batch size, but not N, despite Algorithm 1 stating that unlearning and relearning are 'alternatively executed for N iterations.' Since the ablation shows that the unlearning module contributes several accuracy points, N is a load-bearing hyperparameter; the manuscript must report it per dataset and noise level or state the convergence criterion used.","section":"Section 9, Algorithm 1, Table 5"}],"minor_comments":[{"comment":"The lines labeled 'Relearn student model' in fact update θ_t, the teacher; correct the labels.","section":"Algorithm 1, lines 18-19 and Eqs. (19), (21)"},{"comment":"The Mixup operation mixes SA_mix_<τ with A_mix_τ, but no pairing rule is given for samples from the two sets; specify how the sets are matched or subsampled.","section":"Eq. (17)"},{"comment":"The caption says CIFAR-100 under 'Symmetric Label Noise Ratios,' but Section 7.1 and Table 2 say CIFAR-100 uses asymmetric noise; fix the caption.","section":"Figure 7 caption"},{"comment":"The definition of A^u_τ uses 'for (x, y_u, p_u) ∈ S^u' where it should be '∈ A^u'; fix the typo.","section":"Algorithm 1"},{"comment":"The text says AdamW and SGD were tried, but does not state which optimizer was finally used for each model; report the final configuration.","section":"Section 4.1, Training Details"},{"comment":"Eq. (15) uses β_m while Eq. (16) uses β without a subscript; unify the Mixup notation.","section":"Eqs. (15)-(16)"},{"comment":"The figure would be easier to interpret with axis labels, a color scale, and panel annotations; as printed, the reader cannot tell what the matrices display besides class indices.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about the baseline protocol is well founded: the reported Co-teaching numbers at low noise are hard to explain unless the baselines were run under a different or broken adaptation, and the paper does not specify that adaptation. The revision should be checked carefully on this point. The method may be sound, but the current headline claim is not supported until the protocol is fixed and the comparisons are re-run and reported with variance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The problem is real and worth working on: restoring a model that has been degraded by incremental training on noisy labels, rather than retraining from scratch. The proposed LUR pipeline (confidence-thresholded disagreement extraction, LS-based gradient ascent unlearning, then mixup-based relearning on soft labels) is a genuine composition that I don't recall seeing in the LNL or machine-unlearning literature. The ablation gives each module credit, and the source code is available for review. The high-noise margins — e.g., 80.34% vs. 76.39% on CIFAR-10 at 90% noise — are large enough that they would probably survive seed variance, so the mechanism is worth verifying.\n\nThe main soft spot is the baseline protocol. Section 4.1 says the comparison methods were \"carefully tuned\" and points to the online extended version, but that version never specifies how each LNL or MU method is adapted to the MRR setting: initialized from f(θ0), from f(θu), or trained from scratch on Du? With what epochs, data subsets, and stopping criteria? Until that is specified, the uniformly favorable comparison is not interpretable. Some numbers look like artifacts: on CIFAR-10 with 10% noise, Co-teaching reports 45.12%, far below the 84.82% degraded model, which makes no sense under any standard Co-teaching protocol on 90%-clean data. That is not a minor issue; it directly affects the headline claim.\n\nSecond, the paper never includes the do-nothing baseline: keep the pre-degradation model f(θ0) and simply refuse to update on Du. At 90% noise, the original CIFAR-10 model (84.85%) beats COLUR (80.34%), so the paper should say when updating at all is the right move. Third, Algorithm 1 extracts high-confidence agreements using c(x; θt, θu), while Eq. (14) uses c(x; θu_ul, θt); the difference matters if the unlearned student is the one doing the relearning. Minor, but needs reconciliation. Also, no error bars, seeds, or significance tests anywhere, so the 1–2 point margins at low noise are not established.\n\nA serious referee should engage with this, because the problem is important and the mechanism is plausible. But the paper needs a full, explicit MRR adaptation protocol, error bars and seeds, the do-nothing baseline, and a fix to the Algorithm/Eq. discrepancy. I'd accept it for peer review with those demands.","headline":"Real problem, plausible mechanism, and large high-noise margins — but the baseline comparison protocol is under-specified and some baseline numbers look like artifacts, so the headline claim needs verification.","tokens_in":25918,"tokens_out":2716,"would_cite":true,"duration_ms":27542,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A model degraded by noisy labels can be restored to better-than-original accuracy by unlearning high-confidence teacher-student disagreements and relearning from confidence-refined soft labels, with no need to know which labels are wrong.","keywords":["model restoration and refinement","learning with noisy labels","machine unlearning","teacher-student co-training","label smoothing","mixup","model confidence","label noise"],"falsifier":"Take a model trained on clean data, then present it with an incremental set of clean but hard examples (for instance naturally ambiguous images or a mild distribution shift) where the teacher is confidently wrong on many labels; if COLUR's unlearning stage removes those high-confidence disagreements and final accuracy falls below a control that skips unlearning, the sample-selection premise is refuted.","tokens_in":24756,"feed_emoji":"🧠","tokens_out":5644,"duration_ms":50876,"temperature":0.7,"pith_summary":"The paper claims that a trained model whose accuracy has collapsed after incremental training on noisy labels can be restored, and even improved beyond its original level, by a pipeline that unlearns the influence of suspicious labels and then relearns from soft, confidence-refined labels. It introduces COLUR, a teacher-student framework that never needs to be told which labels are wrong. On CIFAR-10 with 90% label noise, COLUR lifts accuracy from 30.04% to 80.34%; at 50% noise it reaches 87.30%, above the original model's 84.85%. Across four datasets and noise levels from 10% to 90%, COLUR reports the best restored accuracy among both learning-with-noisy-labels and machine-unlearning baselines. If correct, this makes retraining from scratch unnecessary for noise-damaged models.","feed_headline":"Noisy-label models restored: 90% noise to 80% accuracy","feed_subtitle":"A teacher-student loop unlearns confidently wrong labels and relearns from refined soft labels, beating retraining baselines.","key_machinery":"The load-bearing object is the joint confidence score $c(x;\\theta_t,\\theta_u)=\\sqrt{c(x;\\theta_t)\\cdot c(x;\\theta_u)}$, the geometric mean of teacher and student max-class probabilities on a sample where the two models disagree. Samples scoring above a threshold $\\tau$ form the high-confidence noisy set that is unlearned by gradient ascent on a label-smoothed target. The same teacher-student agreement geometry, with the student replaced by the unlearned model, produces the high-confidence agreement set that is relearned with label smoothing, while low-confidence data are mixup-augmented with high-confidence data to create soft-label training pairs. This confidence geometry is what turns unlearning from a data-removal operation into a self-supervised label-refinement loop.","core_discovery":"The paper's central claim is that model performance lost to noisy-label training can be actively undone rather than merely tolerated: a copy of the original model acts as teacher, the degraded model as student, and samples where both predict different labels with high confidence are treated as mislabeled and unlearned through label-smoothing gradient ascent. The unlearned model is then relearned on high-confidence agreed labels, smoothed to avoid overconfidence, and on mixup-augmented data that blends low-confidence and high-confidence soft predictions. The two steps alternate. In the paper's experiments this restores CIFAR-10 from 30.04% to 80.34% accuracy at 90% noise and from 65.06% to 87.30% at 50% noise, surpassing the original undegraded model's 84.85%; similar gains are reported on CIFAR-100, Flower-102, and Oxford-IIIT Pet. The claim is that this repair path works without the user identifying any noisy label in advance.","pith_inferences":["The teacher-student disagreement criterion could be adapted to detect other forms of training-data corruption, such as poisoned examples, since it only requires confident disagreement rather than knowledge of the corruption type.","A testable extension is to apply COLUR under distribution shift between the original data and the incremental noisy data; the fixed teacher's authority would then decay, so replacing it with an iteratively updated teacher might extend the framework to non-stationary domains.","The ablation shows the unlearning module alone gives modest gains while the full pipeline gives the large improvement, suggesting most of the benefit comes from relearning on refined soft labels; a natural experiment is to test whether simple downweighting of disagreement samples replaces the unlearning step.","Reporting accuracy on standard test splits leaves open how the restored model behaves under harder or shifted test sets; an independent check on corrupted or out-of-distribution variants would clarify whether the model generalizes or re-fits the clean structure of the incremental data."],"forward_implications":["COLUR is model-agnostic: it only needs a copy of the original model and the degraded model, so it can repair different backbone architectures without architectural changes.","The unlearning stage removes the need for a pre-specified forget set: noisy samples are discovered by teacher-student disagreement rather than given by the user.","At high noise levels, LNL baselines collapse on several datasets while COLUR retains a large fraction of accuracy, suggesting the unlearn-then-relearn loop is most valuable exactly where standard robust training fails.","Because COLUR surpasses the original model at moderate noise, the relearning stage is not just restoration but refinement: the model can end up better than it was before degradation.","MU baselines that are handed the true noisy-label set still underperform COLUR, implying that targeted unlearning alone is insufficient and the relearning step is essential."],"supporting_citations":[{"why":"Supplies the co-training architecture with two networks whose agreement selects training samples, the basis for COLUR's teacher-student split.","marker":"[Han et al., 2018]"},{"why":"Co-teaching+ uses disagreement-based sample selection, directly motivating the disagreement extraction in Eq. (7).","marker":"[Yu et al., 2019]"},{"why":"Defines label smoothing, the operation used both in the unlearning target and in relearning agreed labels.","marker":"[Szegedy et al., 2016]"},{"why":"Provides the label-smoothing-improved gradient ascent mechanism adopted in Eq. (12) for unlearning noisy samples.","marker":"[Di et al., 2024]"},{"why":"Supplies mixup, the augmentation used to combine low-confidence and high-confidence soft labels in relearning.","marker":"[Zhang et al., 2018]"},{"why":"Supports mixup as an effective regularizer against label noise, justifying its use in the relearning stage.","marker":"[Carratino et al., 2022]"},{"why":"Provides the neuroscience grounding that forgetting enables efficient relearning, the inspiration for the unlearn-then-relearn loop.","marker":"[Ryan and Frankland, 2022]"},{"why":"Supplies the online learning, unlearning, and relearning paradigm that COLUR operationalizes for model restoration.","marker":"[Ramkumar et al., 2023]"}],"fun_headline_variants":["Unlearning noisy labels restores model to 80% accuracy","Teacher-student unlearning repairs noisy-label damage","Forget wrong labels: relearn to restore model performance","Confident mielabels unlearned, model accuracy revived","Unlearn to relearn: fix noisy-label trained models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole repair rests on the assumption that when the original teacher model and the degraded student both predict different labels with high confidence, the sample is mislabeled; if the original model is confidently wrong about a clean-but-hard example, the unlearning step removes correct knowledge and relearning can bake in the damage.","fun_headline_variants_meta":{"raw":{"variants":["Unlearning noisy labels restores model to 80% accuracy","Teacher-student unlearning repairs noisy-label damage","Forget wrong labels: relearn to restore model performance","Confident mielabels unlearned, model accuracy revived","Unlearn to relearn: fix noisy-label trained models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1327,"prompt_tokens":921,"completion_tokens":406,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":325}},"tokens_in":537,"tokens_out":406,"duration_ms":4909,"temperature":1.0,"reasoning_tokens":325,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:33:13.523927+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a model trained on clean data, then present it with an incremental set of clean but hard examples (for instance naturally ambiguous images or a mild distribution shift) where the teacher is confidently wrong on many labels; if COLUR's unlearning stage removes those high-confidence disagreements and final accuracy falls below a control that skips unlearning, the sample-selection premise is refuted.","supporting_citations":[{"cited_title":"On mixup regularization","cited_arxiv_id":null,"evidence_quote":"Supports mixup as an effective regularizer against label noise, justifying its use in the relearning stage."},{"cited_title":"Forgetting as a form of adaptive engram cell plasticity","cited_arxiv_id":null,"evidence_quote":"Provides the neuroscience grounding that forgetting enables efficient relearning, the inspiration for the unlearn-then-relearn loop."},{"cited_title":"Learn, unlearn and relearn: An online learning paradigm for deep neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the online learning, unlearning, and relearning paradigm that COLUR operationalizes for model restoration."}],"review_version":1}