{"id":"091efa04-9773-4827-a5df-eafb169cffa5","arxiv_id":"2412.00150","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"CUFIT uses a frozen-feature linear probe to select clean samples, then curriculum-trains two adapters, improving noisy-label medical image classification.","lead":"This paper proposes CUFIT, a way to fine-tune visual foundation models for medical image classification when training labels are unreliable. It reports consistent accuracy gains over prior sample-selection methods on four medical benchmarks and a real-world noisy retina dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"At 60% symmetric noise the paper's stated justification for agreement selection is false; the high-noise results depend on an unverified modal-prediction condition, since LPM- and IAM-selected sets are never analyzed above 40% noise.","rationale":"The most load-bearing part of the central claim is not the architecture but the selection signal. The paper's headline numbers at 40% noise are large, and the method has independent support in the released code and consistent gains across datasets, VFMs, adapters, and a real-world benchmark. However, the same mechanism is claimed to work at 60% symmetric noise, where the paper's explicit justification is wrong. The reader's weakest assumption captures this: the selection criterion needs the true class to be the modal prediction of the LPM, not a majority of clean labels in the observed class. I checked the balanced symmetric-noise arithmetic: with k classes and r=0.6, an observed class is majority noisy, so Section 4.2's sentence cannot be the reason the method works. It is still possible the method works: if the frozen DINOv2 features make the LPM predict the true class with high probability, agreement selects mostly clean samples because wrong annotations almost never equal the true-class prediction. The paper strongly suggests this in Figure 1 and Figure 3, but Figure 3 stops at 40% noise. At 60%, no precision/recall analysis is provided. This is the single missing link: the high-noise row of Table 1 is the strongest evidence for the paper's claim and the row least grounded in the stated mechanism. Other concerns, such as the absence of error bars and the Figure 2 caption that says IAM selects samples for LPM, are real but secondary; a seed-variance run and a caption fix would not test the mechanism. Therefore I keep the reader's CONDITIONAL verdict: the claim is plausible and well-supported at moderate noise, but the 60% result needs the selected-set precision check before full acceptance.","tokens_in":13889,"tokens_out":7261,"duration_ms":66286,"concrete_test":"Using the released code, reproduce the 60% symmetric-noise runs on HAM10000 and BloodMnist with 5 seeds. During training, log the LPM-selected set and IAM-selected set (ground-truth labels are available in simulated noise) and compute per-epoch label precision and recall, plus the final LAM test accuracy with mean and standard deviation. If either selected set ever has precision <= 0.5, or if the CUFIT-versus-CoDis margin at 60% collapses to within seed noise, the high-noise claim fails; if both selected sets are consistently majority clean and the margin persists across seeds, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that CUFIT beats prior sample-selection methods at 60% symmetric noise (Table 1) rests on the agreement criterion in Eqs. (6)-(7): a sample is selected as clean when the current module's argmax equals its annotation. Section 4.2 justifies this with 'clean labels are in the majority within a noisy class.' For balanced k-class symmetric noise with r=0.6, each observed class contains (1-r)=40% clean samples, so the stated majority premise is false. The selection mechanism can still work if the LPM's argmax is a better predictor of the true label than the noisy annotation is (i.e., the true class is the modal prediction), and for r=0.6 with k>=6 this is plausible because wrong labels are spread among k-1 classes. But the paper provides no evidence for this modal condition at 60%: Figure 3 reports label precision/recall only for 40% noise, Table 1 reports only final test accuracy, and no selected-set precision/recall at 60% is given. Since the LPM is trained on all samples including a noisy majority, it could in principle predict the noisy label distribution strongly enough that agreement selects predominantly noisy samples; without measuring this, the best-result row at 60% is not supported by the stated mechanism. The concern is empirical, not a formal contradiction: adding a precision measurement would settle it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CUFIT, a curriculum fine-tuning paradigm for vision foundation models under label noise. A linear probing module is trained on all samples; an intermediate adapter module is trained only on samples whose linear-probe argmax agrees with the annotation; a final adapter module is trained on samples selected by the intermediate adapter. At inference, only the last adapter is used. Experiments on four medical datasets with symmetric noise rates of 10-60%, on a real-world noisy retinopathy benchmark, and on natural image benchmarks compare against full fine-tuning, linear probing, Rein, Co-teaching, JoCor, and CoDis. Table 1 reports consistent gains, e.g., 5.0%, 2.1%, 4.6%, and 5.8% at 40% noise on HAM10000, APTOS-2019, BloodMnist, and OrgancMnist, and Section 6 analyzes label precision/recall, adapter/backbone variations, and CNN baselines.","tokens_in":14159,"tokens_out":8842,"duration_ms":73427,"significance":"If the empirical claims are supported, the contribution is practically valuable: it shows that frozen VFM features can drive clean-sample selection without a noise-rate hyperparameter, and the curriculum ordering (linear probe then adapters) is simple and general. Strengths include public code, evaluation on external test sets, no fitted constants in the selection criterion, and process-level diagnostics in Figure 3 that show higher label precision and recall than CoDis at 40% noise. The main gaps are the missing empirical support for the 60% noise claims and the absence of any variance reporting; both are addressable by additional experiments. I find no circularity in the evaluation, since the reported accuracies are measured on held-out test sets.","major_comments":[{"comment":"The justification for the agreement criterion is that clean labels are in the majority within a noisy class. Under the symmetric noise model used in Sec. 5.1, at r=0.6 each observed class contains, in expectation, 40% clean and 60% noisy samples, so the stated premise is false. The selection rule could still work if the classifier's argmax is a better predictor of the true label than the observed annotation, i.e., if the true class is the modal prediction because wrong labels are spread over k-1 classes, but this condition is neither stated in Sec. 4.2 nor verified empirically. Figure 3 reports label precision/recall only at 40% noise, and Table 1 reports only final test accuracy; the 60% rows, including the large BloodMnist margin (98.2 vs 88.7), are therefore not backed by the mechanism described. Please add selected-set precision/recall at 60% noise (or an equivalent diagnostic) and either prove or empirically test the modal-prediction condition.","section":"Sec. 4.2, Eqs. (6)-(7); Table 1 at noise rate 0.6"},{"comment":"No standard deviations, confidence intervals, or number of seeds are reported; the values are stated only as test accuracy averaged over the last ten epochs. Without across-seed variability, the abstract's specific margins (e.g., 5.0%, 2.1%, 4.6%, and 5.8% at 40% noise) cannot be distinguished from run-to-run variation. Because the paper's central claim is comparative, report mean and standard deviation over at least three seeds for the main tables, or otherwise justify the stability of the margins.","section":"Tables 1-4"},{"comment":"Co-teaching, JoCor, and CoDis are two-network methods, but the manuscript does not specify how these baselines are instantiated with DINOv2 and the Rein adapter. It is unclear whether each of the two networks has its own frozen VFM backbone and adapter, whether they share the frozen backbone, and how the noise-rate hyperparameter required by these methods is set for the simulated benchmarks (and for the real-world benchmark via the estimation method of [56]). Without these details, the fairness of the comparison cannot be assessed. Please document the exact two-network setup and hyperparameter choices, and ideally release the baseline code.","section":"Sec. 5.1, Baselines"}],"minor_comments":[{"comment":"The sentence claiming CUFIT achieves '0.85% relateively higher accuracy' than CoDis at 10% noise and '3.7%' at 60% noise on HAM10000 does not match Table 1 (82.6 vs 81.9 is 0.7 percentage points; 70.1 vs 66.1 is 4.0 percentage points). Correct the numbers or state explicitly that they are relative improvements.","section":"Sec. 5.2"},{"comment":"The header row is malformed, with entries like 'DINOv2ResNet' that make it hard to distinguish ResNet and DINOv2 columns. Also, the HAM10000 full-training DINOv2 value at 0.2 noise is 66.5 here but 62.6 in Table 1; the discrepancy should be resolved.","section":"Table 3"},{"comment":"The paragraph says 'we utilize four backbones' and lists BioMedCLIP and LoRA, while the preceding sentence and the Figure 4 caption describe three VFMs and three adapters. Please align the text with the figure.","section":"Sec. 6.2"},{"comment":"The text says that the modules are trained simultaneously on the current batch, while Figure 2 and the notation LPM→IAM→LAM suggest strictly sequential training; please clarify the temporal relationship.","section":"Sec. 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope, and the central idea is useful and clearly presented. The main concern is that the strongest empirical claims, especially the 60% noise results, rely on a justification that is false at that noise level, and the missing variance reporting makes the headline margins hard to evaluate. These issues are fixable with additional experiments and reporting. I would not reject on novelty grounds, but the baseline implementation details should be checked carefully during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this is a genuinely useful method paper, not a breakthrough. The CUFIT pipeline—linear probe trained on all samples, agreement-based selection feeding an intermediate adapter, then a last adapter—is a new combination, and the gains over Co-teaching/JoCor/CoDis at 40% noise are consistent across four datasets. Code is released. I'd take it seriously.\n\nWhat it does well: it targets a real practical bottleneck (frozen VFM features under noisy medical labels), and the curriculum idea is simple and plausible. The linear probe is robust because it doesn't update features; the adapter then specializes on cleaner samples. The real-world EyePACS evaluation is a plus.\n\nSoft spots, in order of importance:\n\n1. The 60% symmetric-noise row is where the stated mechanism breaks. Section 4.2 justifies agreement selection by 'clean labels are in the majority within a noisy class.' At r=0.6 that's false; a class contains 40% clean samples. The method can still work if the LPM's prediction is a better estimate of the true label than the noisy annotation is, and for 7-11 class problems that's plausible. But the paper never checks selected-set precision/recall at 60%; Figure 3 only shows 40%. So the best-result rows are undersupported by the paper's own explanation.\n\n2. No error bars or seed variance anywhere. Table 1 shows single numbers. For differences of 1-3% against strong baselines, that matters.\n\n3. Baseline adaptation is under-specified. Co-teaching/JoCor/CoDis are two-network methods; it's not clear how they were instantiated on top of a frozen DINOv2 plus Rein adapter. If their hyperparameters weren't tuned, the comparison could be unfair.\n\nMinor: no comparison to loss-modification methods (they state this), and the 'full-training' numbers look oddly low, which suggests the setup favors adapters.\n\nIs the central claim likely right? I think yes—the method is coherent and the margins at 40% are sizeable. The main fix is empirical: report seeds, add precision/recall at 60%, and either correct the mechanism statement or verify the modal-prediction condition. That's a revise-and-resubmit path, not a rejection.\n\nThis paper is for readers working on label noise, parameter-efficient fine-tuning, or medical image classification with foundation models. It deserves a serious referee—someone who will check the baseline implementations and ask for the missing measurements. Send it.","headline":"CUFIT is a sensible and likely-reproducible method with real gains, but the 60%-noise mechanism claim is overstated and the missing error bars and baseline details need referee attention.","tokens_in":14670,"tokens_out":2853,"would_cite":true,"duration_ms":25744,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CUFIT, a curriculum fine-tuning paradigm, uses a frozen linear probe to select clean samples for two adapters and reports consistent gains over prior noisy-label methods on four medical benchmarks.","keywords":["curriculum learning","label noise","vision foundation models","medical image classification","clean sample selection","linear probing","parameter-efficient fine-tuning","adapters"],"falsifier":"Compute the precision of the samples selected by the linear probing module on a synthetic 60% symmetric-noise version of any of the four medical benchmarks; if that selected set is not majority clean, meaning precision falls to 50% or below, the curriculum's input is corrupted and CUFIT's reported advantage would be expected to disappear.","tokens_in":13712,"feed_emoji":"🩺","tokens_out":8425,"duration_ms":70674,"temperature":0.7,"pith_summary":"Clean-sample selection methods for learning under label noise usually assume training starts from scratch, which wastes the rich features of pretrained vision foundation models. CUFIT instead uses a linear probe on a frozen foundation model as the first selector: because it never updates the feature extractor, it is less prone to memorizing wrong labels. The samples it flags as clean train an intermediate adapter, whose selections train a final adapter, and only this final adapter is used for prediction. The paper reports that this curriculum consistently beats prior selection methods, e.g., by 5.0%, 2.1%, 4.6%, and 5.8% at 40% symmetric noise on HAM10000, APTOS-2019, BloodMnist, and OrgancMnist. If correct, it makes pretrained models usable on noisy medical datasets without noise-rate hyperparameters or architectural loss changes.","feed_headline":"Three-module curriculum beats noisy-label baselines on four medical sets","feed_subtitle":"A frozen-feature linear probe filters clean samples for two adapters, lifting accuracy at 40% noise by 2.1 to 5.8 points.","key_machinery":"The load-bearing object is the three-module curriculum LPM→IAM→LAM with the agreement criterion as the selection rule: a training sample is treated as clean when the selecting module's argmax prediction equals the annotated label. The linear probing module never updates the frozen vision foundation model, so its selections stay stable under label noise; the intermediate adapter module adapts the features and selects a larger clean set; the last adapter module is trained on that set and used for final predictions. The chain needs no noise-rate hyperparameter and applies to any adapter inserted into a frozen transformer.","core_discovery":"The central discovery is that the resistance of linear probing to label noise can be converted into a training curriculum for adapters. Given a frozen pretrained vision foundation model, CUFIT trains a linear classifier on all samples; this linear probing module (LPM) selects as clean any sample whose predicted class equals its annotation (the agreement criterion). Those samples train an intermediate adapter module (IAM), which in turn selects clean samples for the last adapter module (LAM). Because the LPM improves precision and the IAM improves recall relative to it, the curriculum expands the clean set available to the final classifier. In the paper's experiments on four simulated noisy medical benchmarks and one real-world noisy retina benchmark, this scheme outperforms prior sample-selection baselines and stays effective at 60% and even 80% noise.","pith_inferences":["The agreement criterion's real requirement is that the true label be the linear probe's modal prediction within each annotated class; under symmetric noise above 50% this is stricter than the paper's majority-clean phrasing and is directly measurable, so the method's failure boundary could be mapped empirically.","Because the selection modules and the adapter can be swapped independently, CUFIT-like curricula could be applied to other parameter-efficient fine-tuning methods or to any frozen backbone in domains far from medical imaging; nothing in the design is medical-specific.","One could decouple the curriculum from the hard argmax match by using the linear probe's confidence as a threshold, tuning the precision-recall balance of the selected clean set and potentially pushing the method past the 60% noise regime reported."],"forward_implications":["At 40% symmetric noise, CUFIT improves over the next-best baselines by 5.0, 2.1, 4.6, and 5.8 accuracy points on HAM10000, APTOS-2019, BloodMnist, and OrgancMnist.","Gains over the strongest sample-selection baseline widen as noise rises, from about 0.7 point at 10% noise to 4.0 points at 60% noise on HAM10000.","On the real-world Kaggle-EyePACS noisy retina training set, CUFIT achieves the highest combined accuracy on the APTOS-2019 and FGADR test sets, 64.4% versus 63.9% for the next method.","The same curriculum lifts accuracy under 80% symmetric noise on CIFAR-10 and CIFAR-100 and on the real-world ANIMAL10N benchmark, so the mechanism is not restricted to medical images.","Because CUFIT works across several vision foundation models and several adapter types, the curriculum itself, not a particular adapter, is the source of the reported gain."],"supporting_citations":[{"why":"DINOv2 is the default pretrained vision foundation model whose frozen features make the linear probing module stable under label noise.","marker":"[19]"},{"why":"Rein is the default adapter used for the intermediate and last adapter modules in the CUFIT experiments.","marker":"[21]"},{"why":"Co-teaching is the sample-selection baseline whose symmetric-noise generation protocol CUFIT follows and outperforms.","marker":"[11]"},{"why":"JoCor is the co-regularization baseline compared across every noise rate in the main results table.","marker":"[13]"},{"why":"CoDis is the strongest sample-selection baseline, and CUFIT reports larger gains against it as noise rises.","marker":"[14]"},{"why":"LoRA is one of the adapter variants used to show that CUFIT is not tied to a single adapter design.","marker":"[24]"},{"why":"VPT is another adapter variant used in the cross-adapter validation.","marker":"[25]"},{"why":"AdaptFormer is another adapter variant used in the cross-adapter validation.","marker":"[26]"}],"fun_headline_variants":["Frozen probe filters noise, then two adapters learn","Probe-then-adapt curriculum beats noisy-label baselines","Linear probe selects clean samples for adapter fine-tuning","CUFIT: robust curriculum from frozen features for noisy labels","Noise-proof curriculum: probe first, then adapt for medical images"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a sample whose label agrees with the current module's prediction is genuinely clean; this requires the true label to be the module's most common prediction within each annotated class, which is not guaranteed at the 60% symmetric-noise rate tested.","fun_headline_variants_meta":{"raw":{"variants":["Frozen probe filters noise, then two adapters learn","Probe-then-adapt curriculum beats noisy-label baselines","Linear probe selects clean samples for adapter fine-tuning","CUFIT: robust curriculum from frozen features for noisy labels","Noise-proof curriculum: probe first, then adapt for medical images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000788,"raw_usage":{"total_tokens":3496,"prompt_tokens":990,"completion_tokens":2506,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":606,"completion_tokens_details":{"reasoning_tokens":2423}},"tokens_in":606,"tokens_out":2506,"duration_ms":16757,"temperature":1.0,"reasoning_tokens":2423,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:08:31.072216+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the precision of the samples selected by the linear probing module on a synthetic 60% symmetric-noise version of any of the four medical benchmarks; if that selected set is not majority clean, meaning precision falls to 50% or below, the curriculum's input is corrupted and CUFIT's reported advantage would be expected to disappear.","supporting_citations":[{"cited_title":"Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation","cited_arxiv_id":"2312.04265","evidence_quote":"Rein is the default adapter used for the intermediate and last adapter modules in the CUFIT experiments."},{"cited_title":"Combating noisy labels with sample selection by mining high-discrepancy examples","cited_arxiv_id":null,"evidence_quote":"CoDis is the strongest sample-selection baseline, and CUFIT reports larger gains against it as noise rises."},{"cited_title":"Visual prompt tuning","cited_arxiv_id":null,"evidence_quote":"VPT is another adapter variant used in the cross-adapter validation."},{"cited_title":"Adapt- former: Adapting vision transformers for scalable visual recognition","cited_arxiv_id":null,"evidence_quote":"AdaptFormer is another adapter variant used in the cross-adapter validation."}],"review_version":1}