{"id":"900341ba-cd12-4d56-8803-9f8feac82a45","arxiv_id":"2508.19498","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A knowledge transfer framework that aggregates features and logits from over 100 heterogeneous pre-trained teacher models via sign voting and pseudo-class voting, improving unsupervised object recognition accuracy.","lead":"UNIFORM trains a single image recognition model by combining knowledge from many pre-trained models that have different architectures and were trained on different data. It uses voting to filter disagreements among teachers, and reports accuracy gains over existing knowledge distillation baselines on 11 datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported numbers may be transductive: 'All the samples in the datasets selected for combination will be used for training' implies the official test splits enter the student's training set, making comparisons with inductive teacher baselines invalid.","rationale":"The reader's weakest assumption identifies the same load-bearing issue: the evaluation protocol does not state that only official training splits are used, and the sentence 'All the samples in the datasets selected for combination will be used for training' at least permits, and arguably prescribes, transductive training on test images. This concern is more important than other potential weaknesses because it determines what every accuracy number in the paper means. If the protocol is transductive, the comparison to supervised teacher test accuracies is unfair and the generalization/scalability claims are not supported by the experiments. I agree with the reader that the method is a reasonable engineering contribution with a coherent voting mechanism, and the ablations in Tables 4–7 give some internal support that the voting components help under the paper's setup. However, internal consistency does not resolve the split ambiguity. I also considered the scalability claim, which is only demonstrated up to 60 descriptive teachers (20 predictive + 60 descriptive = 80 total teachers in the 5-dataset setting), whereas the abstract claims benefit from over one hundred teachers; this is a real overreach, but it is secondary to the data-split issue because even the 80-teacher scaling result is suspect if the evaluation includes test images. The lack of code and exact teacher identifiers compounds the problem, but the split ambiguity is the single decisive check. Since the ambiguity is unresolved and the reader already issued a CONDITIONAL verdict, I recommend no change to the verdict: the paper should be accepted only if the authors confirm the train/test split and rerun or justify the results. If the transductive interpretation is confirmed, the verdict should move to REJECT because the central empirical claim would rest on invalid comparisons.","tokens_in":20528,"tokens_out":5623,"duration_ms":57137,"concrete_test":"Inspect or release the exact data-loading code used to precompute teacher features and to train the student, and identify which files from each of the 11 datasets enter the training set. Then rerun the 11-dataset experiment (or at minimum the 5-dataset Table 2 configuration) with the student trained exclusively on the official training splits and evaluated on the official test splits. If the reported accuracies (e.g., Avg. (D) 82.87 in Table 3) drop materially, or if the ranking against the inductive teacher baselines changes, the transductive interpretation is confirmed and the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that UNIFORM benefits from over one hundred teachers and outperforms knowledge-transfer baselines on standard object recognition benchmarks. Section 4, Evaluation Protocol, states: 'All the samples in the datasets selected for combination will be used for training.' If taken literally, every image in each dataset—including the official test images—is used to train the student, with only teacher features and voted pseudo-labels as supervision. The per-dataset accuracies reported in Tables 1–3 would then be measured on the same images seen during training, i.e., transductive accuracies. This breaks the comparison with the 'Predictive Teacher' rows, which are standard supervised test accuracies: the student is being compared on a different, easier task of fitting the evaluation set rather than generalizing to unseen data. The claim that UNIFORM 'sometimes even outperform[s] the teachers' is exactly what such a protocol would inflate. The scaling study in Figure 5 is also affected: if all methods train on test images, the curves measure adaptation to the test distribution rather than generalization. No sentence in the paper clarifies that 'all samples' means only the official training splits; the footnote in Table 1 says the KD methods are 'trained on the joint label spaces and training data,' but the protocol sentence is broader and unqualified. Because the headline accuracy numbers are the primary evidence for every table, this ambiguity is load-bearing. If the authors intended official train splits only, the protocol needs to say so explicitly; if not, the results are not comparable with the supervised baselines or with standard benchmark numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents UNIFORM, a knowledge-distillation framework for training a single student model from a large, heterogeneous set of pre-trained teachers without using manual labels. Teachers are split into predictive teachers that output class predictions on the target label space and descriptive teachers that only provide feature representations. The method maps teacher features into a shared latent space, uses a sign-based voting rule to filter conflicting dimensions, and aggregates the surviving entries; on the logit side, it derives a pseudo-label by hard voting and then applies a decoupled KD loss that weights the pseudo-class and the remaining classes separately. Experiments are conducted on 11 benchmark datasets, combining up to 104 public teachers, and report consistent improvements over adapted baselines (KD, CFL, OFA, CFL+).","tokens_in":20845,"tokens_out":8770,"duration_ms":78545,"significance":"If the reported numbers are taken at face value, the paper makes a useful empirical contribution: it demonstrates that a single student can distill knowledge from a model zoo containing over 100 models with different architectures and training data, using two relatively simple voting mechanisms, and that this approach scales better with the number of teachers than a strong CFL+ baseline. The method makes only mild assumptions about teacher architectures and label spaces, and the ablation study isolating the two voting modules supports their individual contributions. The paper also carefully documents the teacher pool in the appendix, which is useful for reproducibility. The main qualification is that the evaluation protocol is ambiguous about whether test images enter the training set; the headline claims of 'outperforming teachers' and 'unsupervised object recognition' depend on resolving this ambiguity.","major_comments":[{"comment":"The sentence 'All the samples in the datasets selected for combination will be used for training' is unqualified. If the official test splits of the 11 benchmark datasets are included in the unlabeled training set, then the reported accuracies are transductive fits to the evaluation set. In that case, comparisons with the 'Predictive Teacher' rows (which are inductive supervised accuracies) are invalid, and the statement that UNIFORM 'sometimes even outperform[s] the teachers' (Section 1) is an artifact of the protocol. This issue affects every table in Section 4 and the scaling study in Figure 5, because all methods would be measured on images they have already seen during training. The manuscript needs to state explicitly which images are used for training and which for evaluation, and if the test splits were used, the experiments must be redone with a proper inductive split or explicitly presented as transductive with appropriate transductive baselines.","section":"Section 4, Evaluation Protocol"},{"comment":"The feature encoders are described as consisting of four 2D convolutional layers with ReLU activation, which forces all mapped teacher features to be non-negative. Consequently, the sign function in Eq. (3) can only take values in {0, +1}, so the 'sign conflicts' between positive and negative elements that motivate the voting mechanism cannot occur in the mapped space as described. Either the encoder output can be negative (e.g., a linear layer before the sign function), or the mechanism actually implements a sparsity filter that discards zero activations; the current text and Figure 2 do not match the implementation. This needs to be clarified or corrected, since it affects the interpretation of the central feature-voting contribution.","section":"Section 3.1.1, Eqs. (3)-(4) and Section 4, Implementation details"},{"comment":"The reported accuracies are single-run numbers with no standard deviations or significance tests. Some per-dataset differences between UNIFORM and CFL+ are small or negative (e.g., in Table 3, Flowers102: 96.83 vs 96.96; Cifar100: 88.54 vs 89.00), and the 'superior performance' claim rests on averaged gains. The authors should provide variance across at least three seeds and, where possible, a paired significance test over the 11 datasets, especially if the protocol is inductive.","section":"Section 4.1, Tables 1-3"},{"comment":"The scaling experiment varies only the number of descriptive teachers (up to 60) while keeping the 20 predictive teachers fixed. The conclusion that 'existing methods saturate at a much smaller scale' is drawn from a single baseline (CFL+) on one dataset configuration. The claim about benefiting from 'over one hundred teachers' in the abstract is not directly tested because no experiment compares 40 vs. 100 teachers in the same setup; the 104-teacher result appears only in Table 3. The scaling claim would be stronger if the experiment included the full 104-teacher pool and at least one more baseline.","section":"Section 4.3 and Figure 5"}],"minor_comments":[{"comment":"The term H(\\hat{p}^t) is written as a loss term but is constant with respect to the student parameters; consider removing it or explicitly stating that it does not affect gradients.","section":"Eq. (6)"},{"comment":"If for some dimension all teachers have sign equal to zero or the denominator sum is zero, the aggregation in Eq. (4) would divide by zero; the paper should state how such dimensions are handled.","section":"Eq. (4)"},{"comment":"The text contains a duplicated fragment: 'adopts the same encoder and decoder structure (fe(·) and fd(·)) as UNIFORM. and fd(·)) as UNIFORM.' This should be corrected.","section":"Appendix 6.2, CFL+ description"},{"comment":"In the last row, 'SwinTransfromer-Base' is misspelled and should be 'SwinTransformer-Base'.","section":"Table 9"},{"comment":"No code is released; providing the teacher-model list and code would substantially aid reproducibility, especially given the large number of external models used.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The evaluation protocol ambiguity is the main gate to acceptance. If the authors confirm that only official training splits were used for training and report accuracies on the standard test splits, the paper is likely salvageable with the other revisions. If the test splits were used in training, the headline claims are misleading and the experiments need to be rerun. I also recommend asking the authors to clarify the activation of the feature encoder, since the ReLU output contradicts the sign-conflict narrative. The paper is otherwise within the scope of the journal and the empirical study is extensive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on 2508.19498. The core idea is clean: take a hundred-plus off-the-shelf models, split them into predictive teachers (share target classes) and descriptive teachers (features only), map all features into a common space, vote on sign conflicts, and distill into one student. The logit side does majority-vote pseudo-labeling and then applies decoupled KD separately to pseudo and non-pseudo classes. That combination is new in this setting, and the paper shows it beats three strong baselines and scales better with teacher count than the CFL+ extension. The ablations give credit to both voting modules. The math in Eqs. (2)-(6) is straightforward and internally consistent. So as an engineering contribution, this is legitimate.\n\nBut the evaluation protocol has a sentence that is either a serious flaw or a serious ambiguity: \"All the samples in the datasets selected for combination will be used for training.\" The paper reports per-dataset test accuracy, and the baseline rows are supervised teacher test accuracies. If the student trains on test images—even without labels—those accuracy numbers are transductive and not comparable to the supervised teacher numbers. I checked the appendix for a clarifying statement and there is none. The footnote on Table 1 says KD methods are \"trained on the joint label spaces and training data,\" which is narrower, but the protocol sentence is unqualified and appears in the main text. This has to be fixed before the paper means what it claims.\n\nOther soft spots are smaller. No error bars, and many differences (e.g., 0.5% on Flowers102) could be noise. The paper does not ship code, and Appendix Table 8 lists teacher models as \"link\" placeholders rather than actual identifiers, so the exact teacher pool is not reproducible as written. The scaling study in Figure 5 stops at 60 descriptive teachers; the \"over one hundred teachers\" headline is the main configuration, not the scaling curve. That is a minor mismatch, not a fatal one.\n\nWould I send this to referees? Yes. The method is sensible, the results are suggestive, and the protocol issue is fixable with a clarifying sentence and a discussion of transductive versus inductive evaluation. A serious referee can help the authors tighten the claims. But I would not cite the reported numbers in my own work until the train/test split question is resolved.","headline":"UNIFORM is a sensible multi-teacher distillation method, but the evaluation protocol is ambiguous about whether test images enter training, which must be resolved before the numbers can be trusted.","tokens_in":21385,"tokens_out":2298,"would_cite":false,"duration_ms":22321,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UNIFORM claims one student model, trained without labels, can learn recognition from the collective consensus of over one hundred off-the-shelf pretrained models by voting away teacher disagreement.","keywords":["knowledge distillation","multi-teacher learning","unsupervised object recognition","feature voting","logit voting","model zoo","pseudo-labeling","heterogeneous architectures"],"falsifier":"Re-run the 11-dataset experiment once with the official test images included only for evaluation and never used to compute teacher features or student training targets; if average accuracy drops materially, the headline numbers depend on transductive access to the test set.","tokens_in":20349,"feed_emoji":"🧠","tokens_out":7157,"duration_ms":65389,"temperature":0.7,"pith_summary":"UNIFORM claims that a single student network can learn object recognition without manual labels by distilling the collective opinion of a large, heterogeneous crowd of off-the-shelf pretrained models. It divides those teachers into predictive teachers, which can score some of the target classes, and descriptive teachers, which only supply visual features. Instead of averaging teacher outputs directly, it keeps only opinions that survive two voting procedures: an element-wise sign vote over mapped features and a pseudo-class vote over logits. Across 11 benchmark datasets with 104 teachers, the paper reports the highest average accuracy among the knowledge-transfer baselines it compares against, and the margin grows as teachers are added while baselines saturate. If true, this means freely available model collections can act as a substitute for labeled data in building a compact recognizer.","feed_headline":"One student network learns from 104 teachers","feed_subtitle":"Feature and logit voting turn disagreeing pretrained models into label-free supervision for recognition.","key_machinery":"Two voting mechanisms carry the method. Feature voting is coordinate-wise: after each teacher's features are mapped into a shared latent space by a learned encoder, the sign of each dimension is tallied across teachers, and only features whose signs match the majority are averaged into the target the student must imitate; reconstruction decoders keep these mapped features from collapsing. Logit voting is class-wise: each predictive teacher contributes one hard vote, the class with the most votes becomes the pseudo-label, and the student's distillation loss is decomposed into pseudo and non-pseudo logit terms so the voted class is emphasized over the distracting tails of teacher distributions.","core_discovery":"On its own terms, UNIFORM establishes that teacher disagreement, not teacher diversity, is the obstacle to large-scale knowledge distillation. Naive averaging of teacher features can cancel almost all information when sign patterns conflict, and naive averaging of teacher logits mixes contradictory predictions; UNIFORM's feature voting filters each coordinate to the majority sign before aggregation, and its logit voting first selects a pseudo-class by hard votes, then transfers teacher logits with the pseudo class weighted more heavily. With these mechanisms, 104 public teachers of varied architectures and training data give the student enough supervision to beat strong baselines on all three combinations studied (2, 5, and 11 datasets) and to keep improving as descriptive teachers are added beyond the point where the CFL+ baseline stops improving. The paper reports that the Swin Transformer student reaches 91.89% average dataset accuracy on the 5-dataset setting and that UNIFORM sometimes exceeds the supervised predictive teachers.","pith_inferences":["The evaluation protocol says all samples in the combined datasets are used during training but does not state whether the official test splits enter the unlabeled pool; if they do, the reported accuracies are transductive and would not be comparable to the supervised test numbers. Re-running with held-out images removed from the teacher-feature pool would settle this.","Coordinate-wise sign voting is essentially a majority operation, so it may discard information that only a minority of teachers carry; if the target domain is far from most teachers' training data, the student may inherit the majority's blind spots even when a few specialized teachers saw the relevant distribution.","The same teacher-consensus interface could be applied to dense prediction tasks such as segmentation or detection, but the paper's experiments are image classification only; whether spatial feature voting transfers to pixel-level labels is untested."],"forward_implications":["A student trained without labels from 104 teachers can match or exceed the supervised teachers on several of the benchmark datasets, so public model zoos can plausibly serve as label-free supervision at scale.","UNIFORM keeps scaling with teacher count, at least up to 60 descriptive teachers, whereas the CFL+ baseline saturates near 30; the voting mechanisms are what absorb the extra teachers without being overwhelmed by disagreement.","The framework makes no assumption about teacher architecture or label space, so predictive teachers with only partial class coverage can be combined with purely descriptive teachers under one loss.","Only the student model survives deployment; the teachers and their encoders and decoders are discarded after training, converting a large model collection into one compact classifier.","Stronger student architectures, such as Swin Transformer and ConvNeXt-base, translate teacher consensus into higher accuracy, so the student architecture remains a free design choice."],"supporting_citations":[{"why":"defines the logit-distillation baseline whose plain averaging of teacher logits UNIFORM replaces with pseudo-class voting.","marker":"[22]"},{"why":"provides the common-feature-learning baseline CFL, the backbone of the CFL+ extension, and the main predictive-only comparison.","marker":"[37]"},{"why":"is the heterogeneous-architecture distillation baseline (OFA) that UNIFORM must beat when teachers and student differ in design.","marker":"[19]"},{"why":"supplies the smooth-L1-plus-cosine feature loss used in the ablation that isolates the contribution of UNIFORM's feature loss.","marker":"[46]"},{"why":"motivates decomposing the logit loss into pseudo and non-pseudo classes during distillation.","marker":"[69]"},{"why":"establishes feature-based knowledge transfer, which UNIFORM extends by unifying features into a shared space and voting.","marker":"[47]"},{"why":"is cited as a possible distance metric for the feature-transfer loss in Eq. (4).","marker":"[16]"}],"fun_headline_variants":["UNIFORM learns from 104 disagreeing models via voting","Voting turns teacher disagreement into student knowledge","104 teachers, one student: majority vote wins","UNIFORM: scale to over 100 teachers by voting","Diverse pretrained models, unified student via voting"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported accuracies measure true recognition on held-out data, but the protocol says all samples in each combined dataset are used for training without clarifying whether the official test images appear there as unlabeled inputs.","fun_headline_variants_meta":{"raw":{"variants":["UNIFORM learns from 104 disagreeing models via voting","Voting turns teacher disagreement into student knowledge","104 teachers, one student: majority vote wins","UNIFORM: scale to over 100 teachers by voting","Diverse pretrained models, unified student via voting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000421,"raw_usage":{"total_tokens":2164,"prompt_tokens":945,"completion_tokens":1219,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":1142}},"tokens_in":561,"tokens_out":1219,"duration_ms":9603,"temperature":1.0,"reasoning_tokens":1142,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:51:31.567996+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 11-dataset experiment once with the official test images included only for evaluation and never used to compute teacher features or student training targets; if average accuracy drops materially, the headline numbers depend on transductive access to the test set.","supporting_citations":[{"cited_title":"Knowledge Amalgamation from Het- erogeneous Networks by Common Feature Learning","cited_arxiv_id":null,"evidence_quote":"provides the common-feature-learning baseline CFL, the backbone of the CFL+ extension, and the main predictive-only comparison."},{"cited_title":"One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distil- lation","cited_arxiv_id":null,"evidence_quote":"is the heterogeneous-architecture distillation baseline (OFA) that UNIFORM must beat when teachers and student differ in design."},{"cited_title":"Learning Transferable Visual Models From Natural Language Supervision","cited_arxiv_id":null,"evidence_quote":"supplies the smooth-L1-plus-cosine feature loss used in the ablation that isolates the contribution of UNIFORM's feature loss."},{"cited_title":"Decoupled Knowledge Distillation","cited_arxiv_id":null,"evidence_quote":"motivates decomposing the logit loss into pseudo and non-pseudo classes during distillation."},{"cited_title":"Borgwardt, Malte J","cited_arxiv_id":null,"evidence_quote":"is cited as a possible distance metric for the feature-transfer loss in Eq. (4)."}],"review_version":2}