{"id":"fc6c6c0b-ba80-43bb-b24d-5a5334026be2","arxiv_id":"2412.05029","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A class-wise embedding method with class associative and prototype discriminative losses reports state-of-the-art accuracy on six instance-dependent partial label learning benchmarks.","lead":"This paper introduces a training method for partial label learning, where each training image comes with a set of candidate labels instead of one true label. The method builds a separate embedding for every class, then uses two new loss functions to pull candidate labels together and align confident predictions with class prototypes, improving accuracy on six image benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Prototype discriminative loss relies on argmax predictions that may be wrong; the paper does not measure prototype contamination, and Eq. (6) is inconsistent with the prose about which class is used.","rationale":"The reader's weakest assumption is that prototype quality depends on the reliability of argmax predictions inside the candidate set, and I agree this is the most load-bearing point. The PDL loss is the component explicitly designed to reduce label ambiguity, and its entire effect flows through prototypes built from predicted labels. If those prototypes are contaminated by early errors, the loss can reinforce wrong classes rather than correct them, and the paper provides no check on this contamination. The two-stage training is a reasonable mitigation, but T_w = 250 is chosen without reporting classification accuracy at that epoch or the precision of the argmax-in-candidate selection. The ablation in Table 4 shows only the combined effect of adding PDL, not the quality of the prototypes themselves. The Eq. (6) notation issue further weakens the description: the text specifies argmax within the candidate set, but the equation uses a global argmax with a membership condition, and these differ when the global argmax is outside S_i. This is a concrete, testable inconsistency that could change the behavior of prototype updates and bias coverage. I do not believe this invalidates the paper's empirical result; the method may well work as intended, and the reported gains could be real. But the prototype-quality question is exactly where the central argument is least secure, and it deserves a direct measurement before accepting the causal story that the two losses produce the improvement. Since the reader's verdict was already CONDITIONAL, I recommend leaving that verdict unchanged, with the condition being the requested prototype-quality experiment. This is an honest non-prosecutorial read: the concern is specific, the paper is transparent about its components, and the code is available to settle it quickly.","tokens_in":14527,"tokens_out":9448,"duration_ms":104153,"concrete_test":"Using the released code, instrument CEL on CUB200 and DOGS120: (1) log the precision of argmax-in-S_i at epoch 250 and the fraction of prototype updates from wrong predictions; (2) run a variant where prototypes are updated with ground-truth labels (upper bound) and with randomly chosen candidate labels (lower bound), keeping all other settings fixed; (3) run a variant where Eq. (6) is implemented as argmax over S_i and compare with the global-argmax version. If predicted-argmax prototypes match oracle prototypes within about 0.5% final accuracy, the concern is minor; if the oracle gap exceeds the PDL ablation gain of 0.71% average, then the self-training loop is a primary determinant of performance and should be reported and mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central disambiguation mechanism is the prototype discriminative loss (Eq. 9), which pulls the class-wise embedding of the predicted high-confidence class toward that class's prototype and away from all other prototypes. The prototypes in Eq. (6) are updated from predicted labels: Q_c is updated only when c = argmax(P_i) and c ∈ S_i. If the argmax is a noisy candidate, the embedding of the wrong class is added to the prototype, and PDL amplifies that error by making the wrong class's embedding more canonical. The paper's only safeguard is a two-stage schedule with T_w = 250, but no evidence is provided that model predictions at epoch 250 are accurate enough. The ablation in Table 4 measures the total contribution of PDL (0.71% average) but never compares prototypes built from predicted labels against ground-truth or random prototypes, so the contamination effect is unquantified. Moreover, the prose before Eq. (6) says “we only select the class with the highest model output probability in the candidate label set,” which would be c = argmax_{j∈S_i} P_i^j, while the displayed equation uses a global argmax with a membership test. If the implementation follows the equation, samples whose global argmax falls outside S_i do not update any prototype, biasing prototypes and reducing coverage; if it follows the prose, the equation is miswritten. In either case, the prototype loop is underspecified, making it the weakest link in the claim that the two proposed losses, rather than the architecture or training schedule, drive the reported gains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CEL, a method for instance-dependent partial label learning (IDPLL). For each sample, CEL produces class-wise embeddings, one per class, via an ML-Decoder head. It introduces two losses: a class associative loss (CAL) that pulls together embeddings of candidate labels and pushes apart embeddings of candidate and non-candidate labels, and a prototype discriminative loss (PDL) that aligns the embedding of the model's high-confidence candidate class with a class prototype and repels it from other prototypes. Prototypes are updated online from model predictions. Training is split into two stages, with PDL added after a warm-up of T_w=250 epochs. Experiments on six benchmarks against twelve methods report first place on all six data sets, with significance tests, ablations, parameter sensitivity plots, and a public code release.","tokens_in":14814,"tokens_out":5249,"duration_ms":47789,"significance":"If the method's gains are robust, CEL is a meaningful step for IDPLL: it is among the first to exploit per-class embeddings in this setting, it explicitly addresses the two-sided nature of instance-dependent noise, and the empirical evaluation is broad, with six benchmarks, twelve baselines, and public code. The reported improvements on fine-grained data sets (e.g., CUB200 from 66.60% to 68.60%, DOGS120 from 75.97% to 78.18%) are substantial. The main risks are technical: the prototype update rule appears inconsistent with the prose, the CAL objective as written contains a diagonal-term bias, and the self-training loop behind PDL is not diagnosed. The absence of a validation-based hyperparameter selection protocol also tempers the strength of the comparative claims.","major_comments":[{"comment":"The sums over j,k ∈ S_i include the diagonal terms j=k, each of which contributes ⟨E_j,E_j⟩=1 to the numerator and 1 to the denominator. Since the class-wise embeddings are L2-normalized, this introduces a positive floor of 1/|S_i| into s_cal and prevents the first term of L_cal in Eq. (5) from being minimized to zero even under perfect candidate-set alignment. The pairwise similarity should be computed over ordered pairs with j≠k (or at least the diagonal should be excluded). Please correct Eq. (3) and confirm that the implementation matches the corrected definition.","section":"Section 3.3, Eq. (3)"},{"comment":"The prose before Eq. (6) states that the authors select the class with the highest model output probability in the candidate label set, but Eq. (6) uses c = argmax(P_i) together with the condition c ∈ S_i. Under the displayed equation, samples whose global argmax falls outside S_i do not update any prototype, which both reduces prototype coverage and biases prototypes toward samples whose global argmax happens to be a candidate. Under the prose, the equation is miswritten. Please specify the exact selection rule used in the released code, correct the notation in Eq. (6) (and in Eqs. (7)-(8)), and clarify how the normalization in Eq. (6) is accumulated across updates.","section":"Section 3.4, Eq. (6)"},{"comment":"Because prototypes are updated from the model's own argmax predictions, PDL creates a self-training loop: wrong early predictions contaminate prototypes, and Eq. (9) then reinforces those errors. The paper does not report any diagnostic of the quality of the high-confidence selections used in Eq. (6), nor does it compare prototypes built from predicted labels against ground-truth or random prototypes. Table 4 reports only the aggregate contribution of PDL (about 0.71% average), which cannot distinguish a genuine disambiguation gain from error amplification. Please add a plot or table showing selection precision/recall over training and at least one control experiment (e.g., ground-truth prototypes, fixed random prototypes) to quantify contamination.","section":"Section 3.4 and Table 4"},{"comment":"The hyperparameters α, β, γ1, γ2 are selected by grid search on the same data sets whose test accuracies are reported in Table 2, and T_w=250 is fixed without a validation-based justification. Since the baselines use hyperparameters from their original papers, this asymmetric tuning can inflate the reported gains. Figure 5 shows sensitivity on two data sets, but the headline numbers are still best grid values on the test data. Please describe a validation split or other selection protocol, report the chosen hyperparameter values per data set, or provide full grid results for all benchmarks.","section":"Section 4.1.3 and Table 2"},{"comment":"All experiments use a single synthetic IDPLL noise generation process, the one proposed by VALEN. Given that the paper motivates IDPLL by real-world annotation scenarios, the robustness of CAL and PDL to other instance-dependent noise models is unknown. Please either add experiments with at least one additional generation mechanism or discuss this limitation explicitly and temper the general claim of effectiveness.","section":"Section 4.1.1"}],"minor_comments":[{"comment":"The PICO row reports '67.80 ±-0.06%'; the standard deviation should be positive, so this is presumably a typo for 0.06%.","section":"Table 2"},{"comment":"For CIFAR-100H, the reported average number of candidate labels is 3.41 while the rate is 0.6; for a 100-class problem, rate=0.6 would typically imply about 60 candidates, not 3.41. Please clarify what 'rate' means for CIFAR-100H and make the table internally consistent.","section":"Table 1"},{"comment":"The x-axis labels in panels (a)-(d) are visually misaligned (e.g., '1' and '2' appear at the bottom of panels (c) and (d) in a way that makes it hard to read the grid values). Please redraw the figure with clearer axis ticks.","section":"Figure 5"},{"comment":"The claim of being 'the first to create class-wise embeddings for each sample' is too broad, because class-wise embeddings already appear in multi-label classification (e.g., Query2Label and ML-Decoder, both cited). The novelty claim should be scoped to partial label learning or IDPLL.","section":"Abstract and Section 1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an important and timely problem, and the empirical results are promising. However, the prototype construction is both under-specified and potentially unsafe without diagnostics, and the CAL objective contains a mathematical slip that, if present in the code, would weaken the intended behavior. The evaluation also needs a validation-based hyperparameter protocol. These issues are fixable within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on partial label learning. The core idea—giving each sample a separate embedding per class, then using those embeddings to enforce candidate-set similarity and prototype-based disambiguation—is new for IDPLL, and the results are consistently positive: CEL ranks first on all six benchmarks, with gains of 1–3 points over the previous best on fine-grained sets. The ablations show both losses contribute (CAL ~1.2%, PDL ~0.7% on average), and the significance tests are a nice addition. The comparison setup is reasonably even-handed: same backbones, same training schedule, baselines given their reported settings, and code is public. I believe the central claim.\n\nTwo real weaknesses. First, the prototype update in Eq. (6) is underspecified and internally inconsistent. The prose says you pick the highest-probability class within the candidate set, but the equation uses a global argmax with a membership test. If the implementation follows the equation, samples whose global argmax falls outside S_i never update any prototype, which biases coverage; if it follows the prose, the equation is miswritten. Either way, the reader can't tell what the actual loop does. Second, the self-training nature of the prototypes isn't probed. Prototypes are built from model predictions, and the paper never compares against ground-truth or random prototypes, so contamination from early wrong argmaxes is unquantified. Hyperparameters are also tuned on the same benchmarks, and all synthetic noise comes from one generation process (VALEN), so the generality of the gains is an open question.\n\nThese are fixable with additional experiments rather than fatal flaws. The empirical core is believable; the gains are consistent and the ablations support the contribution of each loss. The novelty claim ('first to create class-wise embeddings') is a bit overbroad since class-wise encoders exist in the cited multi-label literature, but the specific combination for IDPLL is new.\n\nWho is this for? Anyone working on weakly supervised learning, specifically PLL/IDPLL. It deserves a serious referee: the method is clearly described (modulo the prototype issue), the experiments are thorough, and the code is public. Recommendation: send to peer review, and ask the authors to fix Eq. (6)/prose inconsistency and add a prototype-quality analysis before acceptance.","headline":"Solid empirical paper on instance-dependent PLL; gains look real, but the prototype self-training loop is underspecified and needs fixing before publication.","tokens_in":15411,"tokens_out":2529,"would_cite":true,"duration_ms":21360,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CEL claims that in instance-dependent partial label learning, per-class embeddings with two new losses turn the noisy labels' mixed blessing into a measurable accuracy gain, reaching the top of all six reported benchmarks.","keywords":["partial label learning","instance-dependent partial label learning","class-wise embedding","label disambiguation","class prototypes","weakly supervised learning","fine-grained image classification"],"falsifier":"Train CEL on one of the reported data sets (for example CUB200) with the prototype update in Eq. (6) replaced by a random candidate class, and compare final accuracy with the reported 68.60%. If the random-update control matches or beats the argmax-update version, the self-training assumption behind PDL is not doing the work the paper claims. A second check is a per-epoch ablation that adds PDL at different epochs; if adding it before epoch 250 does not hurt, the stated rationale for the two-stage schedule is falsified.","tokens_in":14303,"feed_emoji":"🎯","tokens_out":7424,"duration_ms":69303,"temperature":0.7,"pith_summary":"This paper argues that instance-dependent partial label learning is a mixed blessing: the candidate labels are noisy, yet because they are generated from the image itself they carry genuine information about the sample. The proposed method, CEL, exploits both sides by giving each sample one embedding per class, then training with two additional losses. A class associative loss pulls candidate-set embeddings together and pushes non-candidate embeddings apart, while a prototype discriminative loss aligns the model's most confident candidate class with global class prototypes. Across six benchmarks, including four fine-grained image sets, CEL reports the best accuracy among twelve compared PLL and IDPLL methods.","feed_headline":"Per-class embeddings beat 12 methods on partial-label learning","feed_subtitle":"Two new losses on per-class embeddings top all six benchmarks, with the largest gains on fine-grained image sets.","key_machinery":"The key machinery is the class-wise embedding: instead of one feature vector per image, the model outputs a matrix $E_i \\in \\mathbb{R}^{q \\times l}$, where row $j$ is the embedding of sample $x_i$ seen from class $j$, produced by an ML-Decoder encoder on top of a ResNet backbone. The class associative loss (CAL) uses cosine similarity to raise the average similarity of class-wise embeddings within the candidate set and lower it for pairs crossing the candidate boundary. The prototype discriminative loss (PDL) maintains a normalized prototype $Q_c$ for each class, updated only with embeddings of the highest-confidence candidate class, and then pulls that class's embedding toward $Q_c$ while pushing it away from all other prototypes. The two-stage schedule adds PDL only after $T_w$ epochs, on the ground that early prototypes are too unreliable to guide disambiguation.","core_discovery":"The central claim is that the instance-dependence of noisy labels, usually treated only as a source of ambiguity, can be used as structured supervision. CEL equips each sample with class-wise embeddings, one vector per class, and imposes two geometric constraints: embeddings inside the candidate set should be mutually similar, embeddings between candidate and non-candidate classes should be dissimilar, and the embedding of the model's highest-confidence candidate class should sit close to that class's prototype while staying far from other prototypes. The paper reports that this combination gives first place on all six tested data sets, with the largest margins on fine-grained data such as CUB200 (68.60% versus 66.60% for the previous best method) and DOGS120 (78.18% versus 75.97%).","pith_inferences":["The prototype update in Eq. (6) is a self-training loop: if the argmax within the candidate set is wrong early on, wrong embeddings enter the prototypes and PDL amplifies the error. The paper waits 250 epochs before adding PDL; a natural testable extension is a confidence-weighted or thresholded prototype update that softens this dependence.","The class associative loss encodes the prior that all candidate labels are semantically alike, which is true under instance-dependent noise but false under instance-independent partial labels. CEL may therefore transfer poorly to classic PLL benchmarks, and a simple experiment would be to run CEL on instance-independent candidate sets and compare with methods designed for that case.","Because CAL and PDL operate directly on the embedding geometry, the recipe is agnostic to the classifier head; a plausible extension is to test whether the same two losses give comparable gains with transformer backbones or higher-resolution inputs than the ResNet/ML-Decoder setup reported here.","The paper attributes the early training speed to CAL and the late-stage accuracy to PDL, but the ablation only measures final accuracy. A per-epoch ablation would directly test that attribution and, if it failed, would point to a different division of labor between the two losses."],"forward_implications":["If CEL's claim holds, instance-dependent noisy labels should be treated as structured supervision rather than noise, and methods that ignore the relationship between noisy labels and image features are leaving accuracy on the table.","The ablation argues for a two-phase recipe: first learn representations that encode candidate-set structure with CAL, then disambiguate with global prototypes via PDL, with the paper attributing average gains of 1.22% and 0.71% to the two losses respectively.","The largest reported gaps on fine-grained data suggest that class-wise embeddings help most when candidate labels share a superclass and are visually similar, which is exactly the setting where ordinary PLL methods degrade.","The embedding-length experiments indicate that a length of 512 is a safe default across both small (32x32) and large (224x224) inputs, since oversizing dilutes features on small images and undersizing compresses features on large ones.","The win/tie/loss counts, 66 wins out of 72 paired comparisons at the 0.05 significance level, make the accuracy ranking a statistical claim and not just a point estimate."],"supporting_citations":[{"why":"Supplies the ML-Decoder class-wise encoder that produces the per-class embedding matrix central to the method.","marker":"[25]"},{"why":"Provides the progressive label disambiguation loss (Eqs. 1-2) that CEL adopts as its classification term and also serves as a baseline.","marker":"[22]"},{"why":"Introduces the IDPLL problem and the instance-dependent noisy-label generation process used to build all six benchmark candidate sets.","marker":"[38]"},{"why":"The strongest prior IDPLL method and the source of the weak/strong augmentation protocol; its reported accuracies are the main numbers CEL claims to beat.","marker":"[34]"},{"why":"An IDPLL baseline using normalized-entropy sample selection, representing the alternative to prototype-based disambiguation that CEL compares against.","marker":"[8]"},{"why":"An IDPLL baseline that progressively purifies the candidate set, the closest competing approach to CEL's separation of helpful and harmful aspects of noisy labels.","marker":"[37]"}],"fun_headline_variants":["Class-wise embeddings turn noisy labels into a training boost","Partial-label learning: per-class vectors beat 12 baselines","Class prototypes and embeddings top all six PLL benchmarks","Mixed blessing strategy wins partial-label learning by large margins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the class the model is most confident about inside the candidate set is usually the true label, so accumulating that class's embedding into the class prototype builds a trustworthy global picture to guide later disambiguation.","fun_headline_variants_meta":{"raw":{"variants":["Class-wise embeddings turn noisy labels into a training boost","Partial-label learning: per-class vectors beat 12 baselines","Class prototypes and embeddings top all six PLL benchmarks","Mixed blessing strategy wins partial-label learning by large margins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000176,"raw_usage":{"total_tokens":1298,"prompt_tokens":960,"completion_tokens":338,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":273}},"tokens_in":576,"tokens_out":338,"duration_ms":4788,"temperature":1.0,"reasoning_tokens":273,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:57:51.604082+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CEL on one of the reported data sets (for example CUB200) with the prototype update in Eq. (6) replaced by a random candidate class, and compare final accuracy with the reported 68.60%. If the random-update control matches or beats the argmax-update version, the self-training assumption behind PDL is not doing the work the paper claims. A second check is a per-epoch ablation that adds PDL at different epochs; if adding it before epoch 250 does not hurt, the stated rationale for the two-stage schedule is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ML-Decoder class-wise encoder that produces the per-class embedding matrix central to the method."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the progressive label disambiguation loss (Eqs. 1-2) that CEL adopts as its classification term and also serves as a baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the IDPLL problem and the instance-dependent noisy-label generation process used to build all six benchmark candidate sets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The strongest prior IDPLL method and the source of the weak/strong augmentation protocol; its reported accuracies are the main numbers CEL claims to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"An IDPLL baseline using normalized-entropy sample selection, representing the alternative to prototype-based disambiguation that CEL compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"An IDPLL baseline that progressively purifies the candidate set, the closest competing approach to CEL's separation of helpful and harmful aspects of noisy labels."}],"review_version":1}