{"id":"71be8ac6-d4b2-4a81-b86a-443191e06dd0","arxiv_id":"2608.11287","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Class-wise reliability-weighted aggregation of experts trained on progressively clipped subsets improves few-shot accuracy on long-tailed benchmarks.","lead":"This paper introduces CLEAR, an ensemble method for long-tailed image classification that trains multiple experts on progressively more balanced subsets and combines them with per-class trust weights. It reports strong gains on rare classes, such as 41.25% few-shot accuracy on CIFAR-100-LT, with competitive overall accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No ablation isolates the class-wise trust weighting, so the paper's central design claim is not yet established.","rationale":"The reader's weakest_assumption was the in-bag bias in trust estimates for tail classes (Algorithm 1, lines 7-8), which is indeed a real weakness and is acknowledged in Sec. 6. My stress-test agrees that this is a problem, but identifies a more fundamental attribution issue: even if the trust estimates were unbiased, the paper does not show that class-wise weighting itself contributes anything beyond structured sampling and the modular BSM/BCL/LA components. The sharpness ablation in Fig. 4 is insufficient because tau=0 is not a global-weight control. This matters because the abstract and contributions claim class-wise expert reliability as the novel design principle; without a direct aggregation-level ablation, the empirical results support structured sampling and modular losses at least as strongly as they support class-wise trust. The in-bag bias amplifies the risk because it means the trust weights for the exact classes where the method claims advantage are computed from training predictions. My proposed test, a global-weight control on the same experts, would decisively separate the contribution of the proposed aggregation from the other components. This concern does not overturn the empirical results, but it means the paper's central claim is not yet supported as stated, which aligns with the reader's CONDITIONAL verdict.","tokens_in":15976,"tokens_out":10208,"duration_ms":97083,"concrete_test":"Re-run the CIFAR-100-LT experiments with the same 15 EDC experts, BSM/BCL/LA settings, and hyperparameters, but replace Eq. (9) with a global trust weighting w_m = softmax(tau * (1/C) sum_c q_{m,c}) applied to all classes, and also run a simple average of expert softmax probabilities. If either control matches or exceeds the reported 41.25% few-shot accuracy, the class-wise aggregation mechanism is not the source of the improvement and the central claim should be weakened.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's core contribution is the class-wise trust-weighted gPoE aggregation in Eqs. (9)-(10), but no experiment compares it to a global-weight or simple-average ensemble built from the same experts. Figure 4 only varies the sharpness parameter tau inside the class-wise scheme; tau=0 is a uniform-per-expert geometric mean, not a global-trust control. Table 2 shows that CLEAR(BSM) already beats SADE on few-shot accuracy (37.45 vs. 33.9), and the headline 41.25 is reached only after adding LA and BCL, which are external modules. Without an ablation that holds the expert set fixed and swaps only the aggregation rule, the reported few-shot gains cannot be attributed to class-wise reliability estimation. The in-bag trust bias acknowledged in Sec. 6 (Algorithm 1, lines 7-8) strengthens this concern: for the tail classes that drive the claimed few-shot advantage, q_{m,c} is estimated from training-side predictions for every expert, so even if the weights do influence results, they may reflect memorization rather than genuine test-time reliability.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes CLEAR, a multi-expert ensemble for long-tailed classification. Using exponential decay clipping, the method trains M experts on progressively more balanced subsets of the original training set. For each expert and class it estimates a trust score q_{m,c} as the posterior mean of a Beta-binomial model of class-wise precision, based on held-out predictions when the class has more than the clipping threshold and on in-bag predictions otherwise. Trust scores are normalized across experts with a softmax temperature τ, and test predictions are aggregated in log space as S_c(x)=Σ_m w_{m,c} log p_m(c|x). The method is evaluated on CIFAR-100-LT, ImageNet-LT, and Places-LT with several backbones, reporting competitive overall accuracy and especially strong few-shot accuracy on CIFAR-100-LT (41.25% few-shot) and Places-LT (40.82%).","tokens_in":16151,"tokens_out":6762,"duration_ms":62922,"significance":"The central idea—that expert reliability in long-tailed recognition should be modeled per class rather than globally—is timely and plausible, and the framework is modular and broadly evaluated. The paper is honest about its limitations, including the in-bag trust bias and computational overhead. If the reported gains are attributable to the class-wise trust weighting, this would be a useful design principle for ensemble-based long-tailed classification. However, the current experimental design does not isolate that mechanism, and the theoretical derivation of the aggregation rule is not fully sound. The contribution is therefore conditional on additional controlled experiments.","major_comments":[{"comment":"No experiment isolates the class-wise trust weighting from the rest of the pipeline. Figure 4 varies τ within the proposed class-wise scheme; τ=0 corresponds to uniform per-expert geometric-mean aggregation, not to a global-trust or simple-average ensemble built from the same experts. Consequently, the few-shot gains in Table 2 (e.g., 37.45 for CLEAR(BSM) vs. 33.9 for SADE) could in principle be produced by the structured expert set or by the external BCL/LA components rather than by the class-wise reliability estimates. I request an ablation that fixes the expert set and compares (i) uniform aggregation, (ii) global expert weights (e.g., softmax of overall held-out precision), and (iii) the proposed class-wise weights, both with and without BCL/LA.","section":"5.3 (Figure 4), 4.2 (Table 2)"},{"comment":"The trust scores for the classes that drive the claimed few-shot advantage are estimated from in-bag predictions. As Algorithm 1 (lines 6-8) states, for classes with n_c ≤ T_m no held-out samples remain, so q_{m,c} is computed from training-side predictions. Section 6 explicitly acknowledges that this 'may lead to optimistic trust estimates, particularly for rare classes.' Because the headline few-shot accuracy is measured on exactly those rare classes, this bias is load-bearing: an expert that memorized tail training examples can receive inflated trust and dominate the aggregation at test time. To support the central claim, the authors should quantify this bias, for example by comparing in-bag and held-out trust estimates on classes with n_c > T_m, or by adding a cross-validation-based trust estimation variant.","section":"3.3 and Algorithm 1, Section 6"},{"comment":"The gPoE derivation is not internally consistent. Eq. (1) defines θ_{m,c} as a probability of correctness, and Eq. (7) uses it as an exponent on p_m(c|x). In Eq. (9) this exponent is replaced by w_{m,c}, a softmax-normalized weight that sums to one across experts, which is a different quantity. The text says 'replacing θ_{m,c} with the normalized trust w_{m,c}' but offers no justification for why a normalized score should enter the exponent of a product-of-experts posterior. As written, Eq. (10) is a heuristic class-wise log-linear ensemble; that is acceptable as a method, but the Bayesian/gPoE motivation should be either corrected or explicitly downgraded to a heuristic.","section":"3.2, Eqs. (7)-(9)"}],"minor_comments":[{"comment":"'Macro Accuracy' is used throughout Section 5 and Figures 2-6 but is not defined in the evaluation metrics; define it explicitly, presumably as mean per-class accuracy.","section":"4.1"},{"comment":"'Uniform Interval', 'Quantile Selection', and 'K-means Clustering' are used without a definition in Section 3.3 or the experimental setup; add a sentence describing each schedule.","section":"Table 6"},{"comment":"Tables 2 and 3 report different numbers for CLEAR(BSM) on CIFAR-100-LT with ResNet-32 (53.09 vs. 52.70 overall, 37.45 vs. 36.53 few-shot) because Table 2 uses δ=0.95 and Table 3 uses δ=0.9; state this explicitly to prevent reader confusion.","section":"Tables 2 and 3"},{"comment":"The 'without smoothing' limit should be written as α0,β0→0, since α0=β0=0 makes the posterior mean undefined for N_{m,c}=0.","section":"3.1, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"This is a plausible contribution, but the missing aggregation-level ablation is a serious gap for a paper whose central claim is about aggregation. The gPoE derivation should also be revised. I would encourage the editor to invite a revision rather than reject, because the issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is worth taking seriously: instead of a global weight per expert, estimate a Beta-smoothed class-wise precision score for each expert and use it as a per-class exponent in a gPoE aggregation. That is not in the RIDE/SADE/MDCS line, and the paper is upfront that the conditional-independence motivation is just a heuristic. The structured sampling schedule (EDC) is also a clean, distinct way to generate diverse experts, and the modular integration with BSM/BCL/LA is a strength, not a weakness.\n\nThe reported few-shot gains are real and sizable: 41.25% on CIFAR-100-LT and 40.82% on Places-LT beat the published SADE numbers by a healthy margin, and the paper is unusually honest about the in-bag trust bias for tail classes. That is genuine empirical value.\n\nBut the stress-test note is on target. There is no ablation that holds the expert set fixed and swaps only the aggregation rule. The τ sweep in Figure 4 varies sharpness inside the class-wise scheme; τ=0 gives a uniform-per-expert geometric mean, not a global-trust-weight control. So you cannot attribute the gains to class-wise reliability estimation rather than the structured sampling or the BSM/BCL/LA additions. That is a load-bearing gap for a paper whose entire selling point is the class-wise trust weighting.\n\nThe in-bag bias is more than a minor caveat. For the tail classes that drive the few-shot claim, every expert is trained on all available samples in every stage (because n_c stays below T_m), so trust for those classes is always estimated from training-side predictions. The Beta prior helps but does not remove the bias. A validation split or a per-class held-out procedure for tail classes would strengthen the central claim substantially.\n\nThe gPoE derivation is shaky as written: Eq. (7)-(9) replace the reliability exponent θ with normalized weights that sum to 1 over experts, which changes the meaning of the exponent. The paper calls this an approximation, but it should either be defended more carefully or presented purely as a design choice. Also, hyperparameters (τ, δ, stage count) are tuned per dataset, and there is no code release.\n\nThese are all fixable with an ablation plus a hold-out trust estimator. The paper deserves a serious referee; I would send it to review, ask for those additions, and otherwise let it through. It is a credible empirical contribution that just needs to prove its mechanism.","headline":"A genuinely new class-wise reliability weighting for long-tailed ensembles, with strong few-shot gains, but the experiments never isolate the proposed aggregation rule from the other moving parts.","tokens_in":16731,"tokens_out":3089,"would_cite":false,"duration_ms":29879,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Modeling expert reliability per class improves few-shot accuracy while keeping overall accuracy competitive on long-tailed benchmarks.","keywords":["long-tailed recognition","class-wise trust","expert aggregation","structured sampling","product of experts","few-shot accuracy","ensemble learning","reliability estimation"],"falsifier":"Train CLEAR on CIFAR-100-LT but estimate every class-wise trust score from held-out predictions (e.g., by splitting each class's samples before training, or using out-of-bag predictions), then re-measure few-shot accuracy. If the few-shot gains over single experts and over SADE shrink to near zero, the in-bag optimism is the source of the reported tail improvement.","tokens_in":15679,"feed_emoji":"🎯","tokens_out":6935,"duration_ms":52706,"temperature":0.7,"pith_summary":"Long-tailed classifiers are reliably accurate on frequent classes but unreliable on rare ones, and existing multi-expert methods typically assign each expert a global weight or route samples rather than asking which expert to trust for each class. This paper proposes CLEAR, a modular ensemble that trains several experts on progressively clipped versions of the training set and then gives each expert a separate trust score per class, estimated as a smoothed class-wise precision. Predictions are combined with a class-wise product-of-experts rule, so a class can lean on the expert that handles it best. On CIFAR-100-LT, ImageNet-LT, and Places-LT, CLEAR keeps overall accuracy competitive while improving few-shot accuracy, with the largest tail gains on CIFAR-100-LT (41.25% few-shot) and Places-LT (40.82%). The paper's point is that class-wise expert reliability is a useful design principle for long-tailed ensembles.","feed_headline":"Class-wise trust lifts few-shot accuracy on long-tailed benchmarks","feed_subtitle":"Per-class expert weighting boosts tail accuracy while keeping overall scores competitive.","key_machinery":"The load-bearing machinery is the class-wise trust score q_{m,c}, defined as the posterior mean of a Beta-binomial model over expert m's precision when predicting class c: q_{m,c} = (alpha_0 + n_{m,c}) / (alpha_0 + beta_0 + N_{m,c}), where N_{m,c} is the number of times expert m predicts class c and n_{m,c} is the number of those that are correct. These scores are normalized across experts by a softmax with sharpness tau, yielding weights w_{m,c}, and the final logit for class c is S_c(x) = sum_m w_{m,c} log p_m(c|x). This class-wise generalized product-of-experts rule lets different experts dominate different classes, and tau interpolates between geometric-mean aggregation and hard selection of the most trusted expert per class.","core_discovery":"CLEAR's central claim is that expert reliability in long-tailed classification is inherently class-dependent, so ensemble weighting should be class-wise rather than global. To instantiate this, the paper introduces structured sampling: a schedule of clipping thresholds (exponential decay by default) generates sub-training sets that move from the original long-tailed distribution toward balanced ones, with every class preserved in every subset, and one expert is trained per subset. For each expert and class, a trust score is the posterior mean of a Beta-binomial model of class-wise precision, and these scores are sharpened with a softmax temperature and used as exponents in a generalized product-of-experts aggregation. The authors report that this scheme yields competitive overall accuracy and particularly strong few-shot performance on three benchmarks, supporting the claim that class-wise reliability-aware aggregation is a useful principle for long-tailed ensemble learning.","pith_inferences":["Beyond the paper, the same class-wise trust mechanism could be applied to any collection of pretrained models, not only the clipped-distribution experts, as a post-hoc ensemble weighting that requires no retraining.","The in-bag trust estimation for rare classes is the most likely source of optimistic few-shot gains; replacing it with out-of-bag or cross-validated estimates would test whether the reported tail improvements are real or partly an artifact.","The Beta-binomial trust model could be extended to estimate uncertainty in the trust scores themselves, allowing the aggregation to down-weight classes where trust is uncertain as well as classes where trust is low.","One could test the same design principle in other imbalanced domains, such as medical imaging or fraud detection, where rare classes are consequential and multiple models are already available."],"forward_implications":["Adding class-wise trust weighting to a standard long-tailed objective, such as Balanced Softmax, improves few-shot accuracy without retraining or changing the loss.","The sharpness parameter tau lets practitioners slide between uniform averaging and per-class hard expert selection, so the aggregation can be tuned to how noisy the trust estimates are.","Strong few-shot gains appear on both object-centric (CIFAR-100-LT) and scene-centric (Places-LT) benchmarks, suggesting the principle transfers across dataset types.","Most of the accuracy gain comes from the first 5–8 experts, so the ensemble can be kept compact for deployment."],"supporting_citations":[{"why":"Defines the ImageNet-LT and Places-LT benchmarks used to evaluate CLEAR.","marker":"[10]"},{"why":"Introduces the CIFAR-100-LT long-tailed version used in the experiments.","marker":"[1]"},{"why":"Balanced Softmax, the default training objective for every CLEAR expert.","marker":"[15]"},{"why":"Balanced Contrastive Learning, the optional representation-level module added to experts.","marker":"[28]"},{"why":"Logit Adjustment, the optional inference-time calibration applied to expert logits.","marker":"[13]"},{"why":"SADE, a key multi-expert baseline that CLEAR compares against on few-shot accuracy.","marker":"[23]"},{"why":"RIDE, a representative multi-expert method used as a baseline for ensemble performance.","marker":"[20]"}],"fun_headline_variants":["Class-wise expert trust lifts few-shot accuracy","CLEAR: per-class expert weighting for long-tailed learning","Tail classes benefit from class-wise expert aggregation","Reliability-aware experts improve long-tail few-shot performance","Class-wise aggregation sharpens long-tailed classification"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"For classes whose samples are all used to train an expert, the trust score is computed from the expert's in-bag training predictions, which can be optimistically biased for rare classes and may inflate the reported few-shot advantage.","fun_headline_variants_meta":{"raw":{"variants":["Class-wise expert trust lifts few-shot accuracy","CLEAR: per-class expert weighting for long-tailed learning","Tail classes benefit from class-wise expert aggregation","Reliability-aware experts improve long-tail few-shot performance","Class-wise aggregation sharpens long-tailed classification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000176,"raw_usage":{"total_tokens":1259,"prompt_tokens":881,"completion_tokens":378,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":306}},"tokens_in":497,"tokens_out":378,"duration_ms":4015,"temperature":1.0,"reasoning_tokens":306,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:12:45.029504+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CLEAR on CIFAR-100-LT but estimate every class-wise trust score from held-out predictions (e.g., by splitting each class's samples before training, or using out-of-bag predictions), then re-measure few-shot accuracy. If the few-shot gains over single experts and over SADE shrink to near zero, the in-bag optimism is the source of the reported tail improvement.","supporting_citations":[{"cited_title":"Large-scale long-tailed recognition in an open world","cited_arxiv_id":null,"evidence_quote":"Defines the ImageNet-LT and Places-LT benchmarks used to evaluate CLEAR."},{"cited_title":"Learning imbalanced datasets with label-distribution-aware margin loss","cited_arxiv_id":null,"evidence_quote":"Introduces the CIFAR-100-LT long-tailed version used in the experiments."},{"cited_title":"Balanced meta-softmax for long-tailed visual recognition","cited_arxiv_id":null,"evidence_quote":"Balanced Softmax, the default training objective for every CLEAR expert."},{"cited_title":"Balanced contrastive learning for long-tailed visual recognition","cited_arxiv_id":null,"evidence_quote":"Balanced Contrastive Learning, the optional representation-level module added to experts."},{"cited_title":"Long-tail learning via logit adjustment","cited_arxiv_id":null,"evidence_quote":"Logit Adjustment, the optional inference-time calibration applied to expert logits."},{"cited_title":"Self-supervised aggregation of diverse experts for test-agnostic long-tailed recognition","cited_arxiv_id":null,"evidence_quote":"SADE, a key multi-expert baseline that CLEAR compares against on few-shot accuracy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"RIDE, a representative multi-expert method used as a baseline for ensemble performance."}],"review_version":1}