{"id":"427d1413-c557-44df-9857-b82e8963aaf5","arxiv_id":"1908.08959","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A systematic comparison shows that data-augmentation jet taggers (planing and PCA scaling) achieve background-preserving performance similar to adversarial networks and uBoost, with much lower training cost.","lead":"This paper compares jet tagging methods that keep the background's mass distribution unchanged while still picking out signal jets. The authors find that simple preprocessing tricks, called planing and PCA scaling, perform about as well as much more complex training schemes, at far lower computational cost.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'nearly identical' planed-vs-adversarial comparison in Fig. 15 rests on a single adversarial training run with no error bars; run-to-run spread could exceed the claimed gap.","rationale":"Reader's weakest assumption is representation invariance. I regard that as a legitimate limitation but it concerns generalization beyond the paper's declared scope (Sec. 5 explicitly lists other representations as future work). The missing statistical uncertainty, by contrast, threatens the validity of the comparison even inside the stated scope. The paper itself flags the absence of uncertainties only implicitly (Table 2 footnote for adversarial; note about running out of background for 4-prong), but does not propagate them to the central distance curves. The proposed retraining test is cheap relative to the original adversarial training (one λ value, not nine) and would settle whether the planed-vs-adversarial gap is real. This supports the reader's CONDITIONAL verdict rather than overturning it; I would keep the verdict and add the repeated-adversarial check as an explicit condition.","tokens_in":26926,"tokens_out":7911,"duration_ms":85429,"concrete_test":"Retrain the λ=50 adversarial network for the 3- and 4-prong signals with 10 different random seeds, using the same architecture, data split, and alternating training schedule described in Sec. 3.3.2. For each seed, recompute the Bhattacharyya distance versus background-rejection curve shown in Fig. 15, then form the pointwise mean and 1σ band across seeds. Also compute the corresponding mean planed-network curve from 10 (or more) planed trainings. If the vertical separation between the two mean curves is smaller than the combined 1σ band at the relevant rejection values, the \"nearly identical\" statement is supported; if the band is as large as or larger than the separation, the comparison is unresolved and the abstract's \"similar performance\" claim must be qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative support for the abstract claim is Fig. 15 (Sec. 4.3), where for the 3- and 4-prong signals the network trained on planed data produces \"nearly identical\" Bhattacharyya-distance curves to the adversarial network. This comparison has no error bars, and the adversarial side is a single training run. Table 2's footnote states that the adversarial statistics are \"over sampled once over each of the nine different values of λ due to the long training time\"; unlike the base, PCA, and Planed networks, there are no 10-seed repeats for the λ=50 classifier shown in Fig. 15. Thus one cannot distinguish a true equivalence from run-to-run noise. The problem is compounded at high background rejection or low signal efficiency, where the Bhattacharyya distance is estimated from few surviving QCD events (the paper itself notes that for 4-prong jets the background is exhausted near ε_S ≈ 0.1). If repeated adversarial runs have a spread comparable to the planed-adversarial separation, the paper's headline \"similar performance\" claim is not currently established; it would be a single-seed anecdote for the expensive method. This is the load-bearing gap because the entire practical message is that the cheap method matches the expensive one.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a systematic comparison of methods for decorrelating jet taggers from the jet mass, applied to 2-, 3-, and 4-prong signal jets against a QCD background. The methods considered are single-variable taggers (τ21, τ32, τ43, and the analytic τDDT21), standard multivariate classifiers (BDT and NN), and four decorrelation approaches: PCA-based rescaling, Planing, uBoost, and adversarial neural networks. All multivariate classifiers use the same 11-variable N-subjettiness input basis of Eq. (4), and performance is evaluated with ROC curves, background-rejection curves, the Bhattacharyya distance between the background mass distribution before and after cuts, and measured training times. The main quantitative claim is that the data-augmentation methods Planing and PCA deliver performance comparable to the training-augmentation methods uBoost and adversarial networks, while requiring a fraction of the training time.","tokens_in":27187,"tokens_out":6574,"duration_ms":60492,"significance":"If the stated equivalence holds, the paper gives practically useful guidance for LHC analyses: cheap and simple data-augmentation methods can replace expensive adversarial training in several regimes, and the comparison extends beyond the 2-prong case studied by ATLAS. The study is well organized, uses multiple quantitative metrics, provides a parameter scan for the adversarial weight, and states that reproducing code is available on GitHub. The main weakness is that the headline 'similar performance' is not yet backed by uncertainty quantification: the adversarial network is represented by a single training run at the chosen λ, and none of the Bhattacharyya-distance curves carry error bars. The paper is therefore more convincing as a benchmark study than as a proof of equivalence between methods.","major_comments":[{"comment":"The central claim that Planing performs nearly identically to the adversarial network for 3- and 4-prong jets is not supported with uncertainty quantification. Table 2 states that the adversarial training statistics are 'over sampled once over each of the nine different values of λ due to the long training time'; unlike the base, PCA, and Planed networks, there are no 10-seed repeats for the λ=50 classifier shown in Fig. 15. The plotted separation between the planed and adversarial curves could therefore be smaller than run-to-run seed variation. Please add repeated adversarial training runs at the chosen λ (with at least the same number of seeds as the other methods) and show error bands on the Bhattacharyya-distance curves, or explicitly rephrase the 'nearly identical' claim as a single-run observation.","section":"4.3, Fig. 15; Table 2 footnote"},{"comment":"The 4-prong comparisons are affected by low statistics in the high-rejection tail. The text notes that for the 4-prong signal the background is exhausted near ε_S ≈ 0.1, so the Bhattacharyya distances at high background rejection are estimated from very few surviving QCD events. Without statistical error bars on dB, the apparent agreement between Planing and the adversarial network in the right-hand panels of Fig. 15 cannot be distinguished from noise. This is especially relevant because the abstract's conclusion is based on the 3- and 4-prong panels.","section":"4.1, Fig. 15"},{"comment":"The abstract states without qualification that Planing and PCA 'deliver similar performance' to adversarial and uBoost. This is a statement about the specific 11-variable N-subjettiness representation of Eq. (4); Sec. 5 explicitly leaves other representations to future work. Because the relative difficulty of decorrelation could depend on the input representation, the conclusion should either be phrased as representation-dependent or the abstract and conclusion should carry the qualifier 'for the N-subjettiness inputs studied here.'","section":"5; Abstract"}],"minor_comments":[{"comment":"The heading 'Adverserial Neural Networks' contains a typo ('Adverserial' should be 'Adversarial').","section":"3.3 heading"},{"comment":"The abstract's 'similar performance' glosses over the 2-prong ordering; at high background rejection the adversarial network clearly has the smallest Bhattacharyya distance, and the Planing curve has lower signal efficiency at fixed rejection. Suggest a more nuanced summary.","section":"4.3, Fig. 15"},{"comment":"The phrase 'Bhattacharya distance' should be 'Bhattacharyya distance'.","section":"Appendix B"},{"comment":"The quantity used is a normalized variant of the Bhattacharyya distance; please cite a source or add a phrase such as 'normalized Bhattacharyya distance' to avoid confusion with the standard −ln BC definition.","section":"Eq. (9)"},{"comment":"The background-rejection axes are plotted with decreasing numerical values; consider reversing the axis or adding an arrow to clarify the 'better' direction.","section":"Figs. 12 and 14, bottom rows"},{"comment":"The statement 'Code to reproduce our results can be found on GitHub' would benefit from a URL or repository identifier.","section":"6, Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The comparison involves two methods (Planing, Ref. [9], and PCA, Ref. [7]) developed in papers co-authored by members of this group. I see no sign of unfair implementation in the manuscript, but the lack of a disclosure and the asymmetry in statistical effort (10-seed repeats for Planing/PCA, single-run adversarial) creates a perception risk. The editor may wish to ask the authors to state the relationship and to provide the adversarial seed repeats requested in the major comments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this paper benchmarks four mass-decorrelation methods—planing, PCA rescaling, uBoost, and adversarial training—on 2-, 3-, and 4-prong jets, and argues that the cheap data-augmentation methods (planing and PCA) match the expensive training-augmentation methods in preserving the QCD background shape. That is a useful practical message for the jet-substructure community, and the study is the first to do this systematically beyond the 2-body case that ATLAS covered.\n\nThe paper does several things well. The method descriptions are clear, the choice of metrics (Bhattacharyya distance, with Jensen-Shannon as a cross-check) is sensible, and the training-time comparison in Table 2 is genuinely informative. The extension to 3- and 4-prong topologies matters because the analytic DDT approach does not generalize, and the paper shows where the alternatives fill that gap. They also ship code, which makes the benchmark reproducible. Credit where due: this is a solid, well-organized study.\n\nThe soft spot is exactly where the reader and the stress-test point. The central claim that planing and adversarial training give “nearly identical” background distortion for 3- and 4-prong jets is built on Fig. 15, and the adversarial curve there appears to come from a single training run (the Table 2 footnote says adversarial statistics are one sample per λ value). The planed networks have 10-seed statistics; the adversarial one does not. Without error bars on that curve, the claimed equivalence could be run-to-run noise. The problem is worse at high background rejection, where the 4-prong background is sparse. This is a legitimate referee issue, not a manufactured one. I would also have liked the abstract to say “comparable performance” rather than “similar performance” until the uncertainty is quantified.\n\nA second, milder limitation is that the whole comparison uses one input representation (the 11-variable N-subjettiness basis). The authors acknowledge this and leave images/sequences/graphs to future work. That is fine for a first systematic study, but it means the conclusion might not transfer to other representations. I do not see this as a fatal flaw—the paper is explicit about the scope.\n\nBottom line: the paper is worth engaging with seriously. The qualitative conclusion—planing and PCA are competitive with uBoost and adversarial training at a fraction of the cost—is probably correct, but the quantitative “nearly identical” claim needs error bars before I would trust it. Send it to peer review; with a moderate revision it would be a useful reference.","headline":"A competent and genuinely useful benchmark of mass-decorrelation taggers, but the headline equivalence between cheap and expensive methods rests on a single adversarial run without error bars, so the central comparison needs a revision before the claim is solid.","tokens_in":27749,"tokens_out":2324,"would_cite":true,"duration_ms":24481,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Planing and PCA match adversarial taggers at ~100x less training time","keywords":["jet substructure","mass decorrelation","jet tagging","data planing","PCA rescaling","adversarial networks","uBoost","Bhattacharyya distance"],"falsifier":"Repeat the comparison with jet-image inputs instead of the 11 N-subjettiness variables and recompute Bhattacharyya distance at fixed background rejection; if planing and PCA no longer track adversarial and uBoost, the representation-invariance premise fails.","tokens_in":26730,"feed_emoji":"⚛️","tokens_out":7245,"duration_ms":66669,"temperature":0.7,"pith_summary":"Jet taggers that separate new-physics signals from ordinary QCD background tend to distort the background's invariant-mass shape, making bump-hunt searches harder. This paper compares four decorrelation strategies on 2-, 3-, and 4-prong jets and claims that the two data-augmentation approaches—Planing and PCA-based rescaling—match the two training-augmentation approaches—adversarial networks and uBoost—in preserving the background distribution, while being far simpler and roughly 100 times faster to train. The authors quantify sculpting with the Bhattacharyya distance between the original background mass distribution and the distribution after cuts, and find that a network trained on planed data comes close to the adversarial network for 3- and 4-prong jets, while PCA-based scaling parallels uBoost. If true, this gives experimental analyses a cheap way to gain most of the decorrelation benefit without the tuning overhead of adversarial training.","feed_headline":"Planing and PCA match adversarial taggers at ~100x less training time","feed_subtitle":"For higher-prong jets, simple data reshaping preserves background shape almost as well as adversarial nets.","key_machinery":"The machinery has three parts. The input representation is the 5-body N-subjettiness basis, 11 variables $\\tau^{(\\beta)}_N$ from Eq. (4), chosen so no classifier sees the jet scale. The decorrelation procedures are (i) planing, with event weights $[w(X_i)]^{-1}=C\\,d\\sigma(X_i)/dm|_{m=m_i}$ that flatten the jet-mass distribution of each class, and (ii) PCA rescaling, $\\vec\\tau^{\\rm PCA}_i = R_i^{-1} S_i R_i \\vec\\tau^{\\rm std}_i$, which removes per-mass-bin linear correlations of the inputs. Against these are set uBoost, a boosted-decision-tree variant that up-weights events whose local efficiency deviates from the target, and the adversarial network, whose classifier minimizes $L_{\\rm tagger}=L_{\\rm classifier}-\\lambda L_{\\rm adversary}$ with the adversary predicting the mass bin. The comparison metric is the Bhattacharyya distance, Eq. (9), between the QCD mass distribution before and after applying a signal-efficiency cut.","core_discovery":"At the paper's core is a quantitative comparison, on a single 11-variable N-subjettiness basis, of two families of mass-decorrelation methods for tagging 2-, 3-, and 4-prong jets against a QCD background. The claim is that the data-augmentation methods—planing, which reweights events so both signal and background have a uniform jet-mass distribution, and PCA rescaling, which per-mass-bin whitens and rotates the input features—reach approximately the same level of background-shape preservation as the training-augmentation methods—adversarial networks and uBoost—when measured by the Bhattacharyya distance between the background mass distribution before and after cuts. For 3- and 4-prong jets, a neural network trained on planed data tracks the adversarial network's sculpting curve nearly identically while training about 100 times faster; PCA-based scaling gives curves similar to uBoost while costing far less time. Because no analytic decorrelated tagger exists for 3- or 4-prong jets, these results position data augmentation as the practical route to mass-agnostic multivariate tagging.","pith_inferences":["If the method ordering is representation-independent, the same study on jet images or graph-based representations should reproduce the qualitative result; this is a direct experimental test of the paper's implicit assumption.","The near-agreement between planing and adversarial training suggests the adversary may be learning to flatten the same background density that planing flattens explicitly; comparing the implied event weights could reveal the two methods are solving the same optimization.","Extending planing to two protected variables, jet mass and transverse momentum, would likely yield a tagger robust to both dominant kinematic systematics; the paper lists multidimensional planing as future work."],"forward_implications":["Analyses can adopt planing or PCA rescaling as drop-in preprocessing without changing the classifier architecture or tuning a new loss-function hyperparameter.","For 3- and 4-prong signals, planed neural networks preserve the background shape almost as well as adversarial networks while cutting training time by a factor of about 100.","PCA rescaling paired with a boosted decision tree gives uBoost-like sculpting at less than 1/20 of the training time, enabling fast iteration during analysis development.","Since the taggers rely on substructure and not the absolute jet scale, a single planed or PCA-trained classifier may serve across a range of signal masses, avoiding mass-by-mass retraining."],"supporting_citations":[{"why":"supplies the planing method: reweighting each event to flatten a chosen feature before training","marker":"[9]"},{"why":"introduces the PCA-based rescaling that the paper uses as the other data-augmentation method","marker":"[7]"},{"why":"defines uBoost, the boosted-decision-tree algorithm with uniform selection efficiencies","marker":"[30]"},{"why":"introduces adversarial neural networks for decorrelated jet substructure tagging","marker":"[27]"},{"why":"provides the analytic DDT single-variable decorrelation that serves as the 2-prong baseline","marker":"[28]"},{"why":"gives the prior experimental comparison of mass-decorrelated taggers that this study extends to 3- and 4-prong jets","marker":"[33]"},{"why":"establishes the 5-body phase-space N-subjettiness basis used as the common input representation","marker":"[44]"},{"why":"introduces the uniform phase-space weighting scheme that planing builds on","marker":"[4]"}],"fun_headline_variants":["Simple reshaping beats adversarial jets at 100x speed","Planed jets: adversarial performance, 100x faster","Data tricks rival adversarial nets for jet tagging","Mass-agnostic jet taggers: planing wins on speed","PCA scaling matches adversarial taggers for less compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All classifiers were trained on the same 11-variable N-subjettiness basis, so the relative ranking of methods could differ if a different input representation changed how hard each decorrelation task is.","fun_headline_variants_meta":{"raw":{"variants":["Simple reshaping beats adversarial jets at 100x speed","Planed jets: adversarial performance, 100x faster","Data tricks rival adversarial nets for jet tagging","Mass-agnostic jet taggers: planing wins on speed","PCA scaling matches adversarial taggers for less compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1272,"prompt_tokens":882,"completion_tokens":390,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":313}},"tokens_in":498,"tokens_out":390,"duration_ms":3661,"temperature":1.0,"reasoning_tokens":313,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:24:45.337448+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the comparison with jet-image inputs instead of the 11 N-subjettiness variables and recompute Bhattacharyya distance at fixed background rejection; if planing and PCA no longer track adversarial and uBoost, the representation-invariance premise fails.","supporting_citations":[{"cited_title":"Performance of mass-decorrelated jet substructure observables for hadronic two-body decay tagging in ATLAS,","cited_arxiv_id":null,"evidence_quote":"gives the prior experimental comparison of mass-decorrelated taggers that this study extends to 3- and 4-prong jets"}],"review_version":1}