{"id":"222252f4-4619-44d9-9b26-dcb67d4a9907","arxiv_id":"2607.15531","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A tiny 82k-parameter student jointly distills two experiments' jet taggers and uses a per-sample reliability gate to beat the teachers at several flavor-tagging working points.","lead":"JetCoRD distills two large LHC jet-tagging networks (ATLAS GN2 and JetClass ParT) into one 82k-parameter model that uses a per-jet reliability score to correct teacher mistakes, reporting small gains at several working points. A generalist reader might care because it is a cheap way to combine and refine expensive detectors' taggers, if the gains hold up.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The student's gains may stem from simply training on the teacher's embedding; the paper never tests a plain-CE baseline, so the reliability mechanism's contribution is unproven.","rationale":"The reader's weakest assumption focused on the information bottleneck: the teacher's embedding may discard information needed to repair teacher errors. That is a valid concern, but the paper at least provides indirect evidence (the student does improve) that enough information survives. My concern is more direct and cuts to the attribution: even if enough information survives, nothing shows the reliability-aware mechanism is what extracts it. The paper's own assertion that a simple classifier on teacher embeddings cannot surpass the teacher is unsupported and likely false for a non-linear classifier against a linear teacher head. If a plain-CE student also surpasses the teacher, the central contribution — the reliability-coupled design — is not needed to achieve the headline numbers, and the paper's novelty claim collapses. The paper should report this baseline. Without it, the verdict should remain conditional: the claimed mechanism may be real, but the evidence in the paper does not yet establish it. If the plain-CE baseline fails to surpass the teacher, the current verdict could be upgraded. This is a concrete, cheap, and decisive experiment.","tokens_in":14311,"tokens_out":14084,"duration_ms":156154,"concrete_test":"Train the exact same JetCoRD architecture and training protocol (same teacher embeddings/logits as inputs, same backbone, same per-domain heads) but with the loss reduced to plain cross-entropy on the student's own logits (no distillation loss, no prototype repair, no gate anchor, gate fixed to 0 so the final output is the standalone student logits). Evaluate on all 19 working points in Table 1. If this plain-CE student matches or exceeds the teacher at the headline WPs (e.g., b-vs-c@0.77, c-vs-u@0.3, Tbqq@0.5, Hbb@0.5), then the reliability mechanism is not necessary for the central claim and the novelty is undermined. If the plain-CE student fails to exceed the teacher at those WPs, the concern is resolved and the reliability mechanism is implicated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that reliability-weighted distillation (A-CoRD) and the gate-anchored mixture (RAI) allow an 82k student to exceed two teachers at selected working points. However, the student operates only on the teacher's 128-dim embedding and logits (Sec. 3.1). The paper asserts that 'a simple classifier on teacher embeddings can at best match the teacher, never surpass it' — but this is unproven. The teacher's classification head is a simple (presumably linear) function on the embedding, while the student is a non-linear MLP; a non-linear classifier on the same embedding could plausibly beat the teacher's head. If that is what happens, the reported improvements (+4.3% b-vs-c@0.77, +1.5% c-vs-u@0.3, +1.6% Tbqq@0.5, +1.4% Hbb@0.5) are not evidence for the reliability mechanism; they are what any supervised model on those features would achieve. The ablations (Tables 3–4) compare only KD and internally ablated variants; no plain supervised baseline on the same inputs is reported. Thus the attribution of the gains specifically to A-CoRD/RAI is load-bearing and currently unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes JetCoRD, an 82k-parameter student that jointly distills two pre-trained jet taggers — ATLAS GN2 (3 classes) and CMS ParT (10 classes). The student consumes the teacher embeddings and logits as inputs and is trained with a reliability-weighted distillation loss (A-CoRD) plus a reliability-gated inference-time convex mixture of teacher and student logits (RAI). The authors report that the student matches both teachers in overall accuracy and exceeds them at selected working points (e.g., +4.3% on b-vs-c at ε=0.77, +1.5% on c-vs-u at ε=0.30, +1.6% on T→bqq at ε=0.5, +1.4% on H→bb at ε=0.5). The contribution is framed as the first cross-experiment distillation in HEP and a new reliability-coupled training/inference framework.","tokens_in":14701,"tokens_out":5095,"duration_ms":51810,"significance":"If the central claims hold, the work is significant: it would demonstrate that a very small unified student can exploit teacher reliability signals to improve upon heavyweight teachers at physics-relevant working points, and it introduces a training/inference coupling (A-CoRD/RAI) that appears novel in HEP distillation. The paper has concrete strengths: it releases code and trained checkpoints, adopts a 10-seed protocol, and includes architecture and algorithm ablations that isolate the contributions of RAI and CCB. However, the statistical evidence for 'exceeds teacher' is currently weakened by the absence of a plain supervised baseline on the same teacher features, by selection of the headline working points from 19 without multiple-comparison correction, and by an implausible zero-standard-deviation entry. These issues are fixable and do not invalidate the overall idea, but they currently prevent firm acceptance.","major_comments":[{"comment":"The paper asserts 'a simple classifier on teacher embeddings can at best match the teacher, never surpass it' (Sec. 3.1), but this assertion is unproven and load-bearing. The teacher's classification head is a simple function on e_T, while the student is a nonlinear MLP (two ResMLP blocks) operating on the same 128-dim embedding plus logits and domain token. A nonlinear supervised classifier on these features could plausibly already beat the teacher's head, independent of any reliability mechanism. Table 4 ablations compare only KD, CoRD, A-CoRD, and PCG — all of which contain the RAI mixture — but no plain CE-trained student on (e_T, t) with the same backbone is reported. The attribution of the reported gains to A-CoRD/RAI therefore lacks a necessary control.","section":"§3.1, §6.2"},{"comment":"The four headline working points (+4.3%, +1.5%, +1.6%, +1.4%) are selected from the 19 rows in Table 1, and no multiple-comparison correction is applied. Since these working points are positively correlated, the expected maximum improvement under a null hypothesis is non-negligible. In addition, the Tbqq@0.5 row reports 32786.9±0.0 over n=10 seeds, which is inconsistent with any seed variation and likely reflects a reporting artifact; this row is one of the four headline results. The teacher rows are shown without uncertainty (the teacher is deterministic, but student seed variation should be propagated to the Δ% columns). The 'exceeds teacher' claim needs a multi-seed, multiple-comparison-aware analysis.","section":"Table 1, §5.2"},{"comment":"The final logits are a convex mixture s_final = g·t + (1−g)·s_student. Consequently, if the gate g is close to 1, the student output is nearly identical to the teacher's, so the reported 'matches teacher in overall accuracy' is at least partly by construction rather than evidence of learned compression. The non-trivial part is the exceedance at selected WPs, which requires g<1 on relevant samples. The paper should report the distribution of g over the full test set (e.g., the fraction of jets with g>0.9, per domain) and demonstrate that the WP gains are not driven by a minuscule subset of jets where the gate happens to defer to the student. Currently Table 5 gives only conditional means, which is insufficient to rule out this explanation.","section":"Eq. (3.8), §3.3.2"}],"minor_comments":[{"comment":"The conclusion says the student exceeds the teachers at 'three physics-actionable working points', but the abstract and Table 1 list four (including T→bqq). Please reconcile the count.","section":"§8.1 vs Abstract"},{"comment":"Eq. (3.4) uses acc_c^T, which must be a fraction (0–1), while Table 6 reports teacher accuracy as a percentage (e.g., 76.8%). The notation should explicitly state the normalization to avoid confusion.","section":"Eq. (3.4), Table 6"},{"comment":"The abbreviations 'Hqql' and 'Tbl' in Table 1 are not introduced; use the full names H→q¯qℓ and t→bℓν, as elsewhere in the text.","section":"Table 1"},{"comment":"Table 5 reports quantities (mean reliability, mean gate, correlations) without error bars. Since these are computed on the test set from one checkpoint (or averaged over 10 seeds?), please clarify whether these are single-run or seed-averaged, and include standard deviations if averaged.","section":"§4.6, Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of JHEP and the idea is interesting. The central claim (exceeding the teacher at selected WPs) is plausible but currently rests on an uncontrolled comparison with no plain supervised baseline on the teacher features. The statistical handling of multiple working points also needs tightening. These are fixable within the manuscript's scope. I did not run the code."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — this is a serious and well-written ML-for-HEP paper. The genuinely new piece is the single per-sample reliability signal r_i that is reused three ways: reweighting the KD loss, gating an EMA-prototype repair, and anchoring an inference-time gate that mixes teacher and student logits. The cross-experiment setup (ATLAS GN2 + CMS ParT, disjoint label spaces) is also new in HEP distillation. The paper does the work: ten seeds, ablations that separate architecture (RAI/CCB) from algorithm (KD/CoRD/A-CoRD/PCG), a hyperparameter scan, and code released. The emergent sparse repair budget — essentially only u in ATLAS and H→4q in JetClass get nonzero beta — is a nice, interpretable result.\n\nThe soft spots are real but not fatal. First, the missing baseline: the student is a nonlinear MLP on teacher embeddings and logits; the claim that a simple classifier on those features cannot surpass the teacher is asserted, not tested. A plain cross-entropy baseline on the same inputs would settle whether the reliability mechanism, not just the nonlinear head, is responsible for the WP gains. The ablations do compare KD (teacher-logit targets) to the reliability variants, which partly addresses this, but a CE-only run is still needed. Second, the statistical presentation: teacher rows have no uncertainty, headline WPs are chosen from 19 without multiple-comparison correction, and one row (Tbqq@0.5) reports ±0.0 std, which looks like a rounding artifact. Third, the paper's own tail regressions on H→qql@0.99 and t→blv@0.995 are honestly disclosed and explained, but they mean the claim should be phrased as 'gains at selected WPs' rather than a blanket improvement. The 'matches teacher by construction' concern raised by the mixture logits is real in principle, but the gate-anchor loss and the fact that KD collapses to a teacher copy while the reliability variants do not show the mechanism is doing work.\n\nOverall: the method is novel, the experiments are reproducible in principle, and the limitations are acknowledged. The missing CE baseline and statistical hygiene are fixable. I would send this to peer review and ask for those additions; the core idea deserves referee time.\n\nRead it if you work on distillation or deployment-side compression in HEP; otherwise it's a solid methods paper, not a physics result.","headline":"A genuinely new reliability-coupled distillation scheme for jet taggers, with honest ablations and code; the 'exceeds teacher' claim is plausible but needs a plain-supervised baseline and uncertainty on teacher rows.","tokens_in":15149,"tokens_out":2925,"would_cite":true,"duration_ms":32446,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An 82k-parameter student jet tagger can match and beat two multi-million-parameter teachers by explicitly modeling when to trust them.","keywords":["jet tagging","knowledge distillation","reliability estimation","model compression","cross-experiment transfer","teacher-student learning","high-energy physics","particle transformer"],"falsifier":"A concrete test would be to construct a test set where teacher errors are dominated by a known information-loss mechanism—for example, by zeroing out the embedding components that carry secondary-vertex topology—and then check whether the student's improvement over the teacher at b-vs-c and c-vs-u disappears. If the gate's correlation with reliability (currently 0.49–0.59) drops to near zero on such a set, the central claim that reliability-based repair is the cause of the gains would be falsified.","tokens_in":14231,"feed_emoji":"⚛️","tokens_out":3296,"duration_ms":35395,"temperature":0.7,"pith_summary":"This paper tries to prove that knowledge distillation of jet taggers works even when the teacher is imperfect, provided the student learns a per-sample reliability signal that controls both training and inference. The authors distill two large, detector-specific teachers (ATLAS GN2 and CMS ParT, together 7M parameters) into a single 82k-parameter student, which matches both teachers on overall accuracy and exceeds them at four physics-actionable working points. The core idea is that the same reliability estimate that down-weights the distillation loss on teacher mistakes also anchors a learned gate that decides, per jet, whether to trust the teacher's logits or the student's own prediction. If true, this suggests that state-of-the-art tagging capability can be compressed to about 1% of the parameter budget and shared across experiments, while treating teacher errors as information rather than ground truth.","feed_headline":"Tiny jet-tagging student beats teachers at four working points","feed_subtitle":"An 82k-parameter model with a single reliability signal outdoes 7M-parameter teachers while sharing across experiments.","key_machinery":"The central object is the per-sample reliability signal r_i = 1{y_hat_T = y_i} · σ(κ · (top1(t_i) − top2(t_i))), which is nonzero only when the teacher's prediction agrees with the ground-truth label and scales with the teacher's logit margin. This scalar is used in three coupled mechanisms: (1) it reweights the temperature-scaled KL distillation loss; (2) it gates an EMA-prototype repair loss that pulls the student embedding toward the correct class prototype when the teacher is wrong; and (3) it anchors the learned gate g_i = σ(MLP(z_i)) through a BCE loss, so that the final prediction is a convex mixture of teacher and student logits. The repair strength per class is controlled by learnab","core_discovery":"The central claim is that a single per-sample reliability signal r_i can simultaneously carry three jobs: reweight the KL distillation loss so teacher mistakes contribute almost nothing, drive a prototype-based repair loss that pulls the student's representation toward the true class when the teacher is wrong, and anchor an inference-time gate g_i that mixes teacher and student logits via s_final = g_i * t_i + (1 - g_i) * s_student. The gate is trained to reproduce r_i through a BCE anchor loss, so the same notion of \"when to trust the teacher\" applies at train and test time. The authors show that this coupling allows an 82k-parameter student to exceed the teachers at b-vs-c (ε=0.77, +4.3%),","pith_inferences":["The student's ability to repair teacher errors is bounded by the information content of the 128-dimensional teacher embedding; if a teacher mistake is caused by a feature that is discarded before the embedding (e.g., secondary-vertex topology), the gate and repair loss cannot recover the true label.","The measured correlation between the inference gate and the training-time reliability is only 0.49–0.59, suggesting that the gate has room to better identify unreliable samples; a stronger anchor or a richer gate input could yield further gains.","The small but reproducible regression at extreme working points (ε ≥ 0.99) might be addressed by the saturation mask the paper suggests; this is a concrete, testable extension.","The design points toward a theoretical question: under bounded teacher calibration error, is a reliability-anchored gate asymptotically Bayes-optimal? Proving this would elevate the approach from an empirical recipe to a principled framework."],"forward_implications":["A tiny 82k-parameter model can deliver teacher-level or better performance, making it feasible to deploy state-of-the-art jet tagging in low-latency or resource-limited settings.","A single student can serve two different detector environments, potentially removing the need for each experiment to train and maintain its own tagger.","Reliability-weighted distillation stops the student from inheriting the teacher's systematic mistakes, which is a known failure mode of vanilla knowledge distillation.","The learned repair budget is interpretable and sparse: the student automatically concentrates its corrective effort on classes where the teacher is statistically weak, which could guide further model development.","The gate mechanism provides a principled way to combine any imperfect teacher with a student, extending beyond jet tagging to other high-energy physics classification tasks."],"fun_headline_variants":["82k-param student outguns 7M-param teachers via reliability gate","Reliability signal lets 82k jet tagger beat 7M teachers","Cross-experiment distillation: tiny student tops teachers at key points","JetCoRD: single reliability signal yields 4 working-point wins","Reliability-aware distillation: 82k student beats 7M teachers"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire correction mechanism assumes that the teacher's 128-dimensional embedding and logits retain enough information to identify and fix the teacher's own mistakes; if a teacher error is caused by information that was discarded before the embedding, neither the repair loss nor the gate can recover the true label.","fun_headline_variants_meta":{"raw":{"variants":["82k-param student outguns 7M-param teachers via reliability gate","Reliability signal lets 82k jet tagger beat 7M teachers","Cross-experiment distillation: tiny student tops teachers at key points","JetCoRD: single reliability signal yields 4 working-point wins","Reliability-aware distillation: 82k student beats 7M teachers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000656,"raw_usage":{"total_tokens":2903,"prompt_tokens":871,"completion_tokens":2032,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":1936}},"tokens_in":615,"tokens_out":2032,"duration_ms":13516,"temperature":1.0,"reasoning_tokens":1936,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T23:02:23.966674+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test would be to construct a test set where teacher errors are dominated by a known information-loss mechanism—for example, by zeroing out the embedding components that carry secondary-vertex topology—and then check whether the student's improvement over the teacher at b-vs-c and c-vs-u disappears. If the gate's correlation with reliability (currently 0.49–0.59) drops to near zero on such a set, the central claim that reliability-based repair is the cause of the gains would be falsified.","supporting_citations":[],"review_version":1}