{"id":"b4fd2778-9d80-4e78-8139-e38ee176b8ff","arxiv_id":"2412.19675","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new scanner package combines a similarity-learning neural network with VEGAS adaptive sampling to collect valid points in BSM parameter scans faster than earlier ML-based methods.","lead":"This paper presents DLScanner, an open-source Python package that uses deep learning and adaptive VEGAS sampling to speed up scans of particle physics model parameter spaces. It could help physicists find allowed regions of new-physics theories faster than standard random or MCMC scanning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fig. 5 reports one run per method; the 53-vs-57 SL+VEGAS lead is only 4 iterations and is not tested against seed-to-seed noise, so the headline convergence ranking is not yet established.","rationale":"I agree with the reader's weakest-assumption assessment: the single-run comparison in Fig. 5 is the weakest link in the paper's central claim. I considered other possible objections—the unexplained 0.4 rescaling in Fig. 5, the absence of a direct MCMC/MultiNest baseline, and the abstract's high-dimensional claim resting on a 5D example—but the single-run issue is the most load-bearing because it directly undermines the numerical ranking that motivates the package. If repeated seeds show overlapping distributions, no additional baselines would rescue the specific 53-iteration claim; conversely, if the ordering survives repetition, the core comparison is substantially strengthened. The proposed test is cheap and decisive. I do not see an internal inconsistency in the method description; the concern is about evidentiary robustness rather than a flaw in the algorithm. Because the reader already assigned CONDITIONAL, my stress test does not move the verdict; it reinforces the conditionality.","tokens_in":24030,"tokens_out":3908,"duration_ms":40450,"concrete_test":"Run each of the six configurations from Fig. 5 (SL+VEGAS, MLP+VEGAS, regressor+VEGAS, and their random-sampling counterparts) with 10 independent seeds, keeping all hyperparameters fixed; record iterations to 10,000 valid points and report mean ± std. Apply a bootstrap or Mann-Whitney test for SL+VEGAS vs MLP+VEGAS and for each VEGAS method vs its random counterpart. If the 53-vs-57 gap is within noise (p>0.05), drop the claim that SL+VEGAS is fastest; if the distributions separate, the ranking stands. Add, if feasible, a plain VEGAS-only arm and an MCMC/MultiNest run to support the 'compared to other scanning methods' wording.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 6.2 and Fig. 5 are the only quantitative support for the central claim that DLScanner, and especially the SL+VEGAS configuration, converges fastest. With K=300, the 53-iteration SL result and the 57-iteration MLP result differ by only 1,200 evaluated points out of roughly 16,000; DL training, VEGAS adaptation, and point selection are all stochastic, and no seed repetition, error bar, or significance test is reported. The paper itself states in Sec. 6.2 that the SL-vs-MLP difference is minimal in this 5D example. The larger apparent gains (VEGAS vs random, classifier vs regressor) also come from single runs with no uncertainty. In addition, the conclusion claims superiority over MCMC/MultiNest without a direct baseline in this paper, citing Ref. [22] instead. The load-bearing condition for the headline claim—that the observed ordering is reproducible and not noise—is therefore unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents DLScanner, an open-source Python package for scanning parameter spaces in BSM phenomenology with deep-learning assistance. The scanning loop trains a network on accumulated labeled points, uses the network to preselect candidate points from a large random or VEGAS-adapted sample, evaluates the selected points with an external tool (SPheno, micrOMEGAs, or a user function), and retrains the network. The package implements an MLP regressor, an MLP classifier, and a two-stage similarity-learning (SL) classifier, each optionally combined with a VEGAS map. The main demonstration is an MSSM scan over five parameters targeting m_h in [124,126] GeV, with convergence measured by the number of iterations needed to collect 10,000 valid points. The paper reports that the SL classifier with VEGAS converges in 53 iterations versus 57 for MLP+VEGAS, that VEGAS speeds up the classifiers by factors of 4 to 5.5 relative to random sampling, and that the MLP regressor is markedly slower.","tokens_in":24229,"tokens_out":3956,"duration_ms":388348,"significance":"If the performance claims hold, DLScanner would be a useful addition to the BSM scanning toolbox: it is released on PyPI and GitHub, provides a generic sampler interface, and includes reproducible SPheno and micrOMEGAs modules. The combination of similarity learning with an adaptive VEGAS map is a reasonable design idea, and making the code publicly available is a service to the community. However, the current manuscript does not yet establish the central quantitative claims: the comparison rests on one run per method in a 5-dimensional example, the SL-vs-MLP advantage is only four iterations, the high-dimensional claim is extrapolated rather than demonstrated, and the comparison with MCMC/MultiNest is cited from prior work rather than measured here.","major_comments":[{"comment":"The central convergence comparison is based on a single run per method. The headline SL+VEGAS advantage over MLP+VEGAS is 53 versus 57 iterations, i.e., about 1,200 evaluated points out of roughly 17,000 at K=300, and no repeated seeds, error bars, or significance tests are reported. Since network initialization, training, VEGAS adaptation, and point selection are all stochastic, this difference is well within plausible seed-to-seed fluctuations. The larger VEGAS-versus-random and classifier-versus-regressor gaps are also single-run observations. Please report repeated-seed statistics or otherwise quantify the variability before claiming substantial gains.","section":"Sec. 6.2, Fig. 5"},{"comment":"The abstract claims that the package addresses 'slow convergence in high-dimensional scans,' but the only physics demonstration is five-dimensional, and the text itself states in Sec. 6.2 that the SL advantage is minimal in this 5D example and that a more distinct performance is 'anticipated' in higher dimensions. Since the high-dimensional claim is load-bearing for the paper's motivation, either add a higher-dimensional benchmark (the generic sampler in Sec. 5.1 is well suited for a synthetic target) or soften the abstract and conclusion accordingly.","section":"Abstract and Sec. 6.2"},{"comment":"The statement that all DL-based methods 'exhibit superior performance and faster convergence compared to adaptive sampling techniques such as MCMC or MultiNest' is not tested in this paper; it is deferred to Ref. [22]. Given that the abstract promises gains 'compared to other scanning methods,' please include a direct baseline on the same MSSM target with the same stopping rule, or restrict the conclusions to the methods actually benchmarked in Fig. 5.","section":"Conclusion, final paragraph"}],"minor_comments":[{"comment":"The text says the SL network error is computed using the 'contrastive loss function, Eq. (12),' but Eq. (12) is the cosine similarity distance; the contrastive loss is defined in Eq. (13). Please correct the cross-reference.","section":"Sec. 6.1"},{"comment":"The installation examples contain spaces inside the package name, e.g., 'pip install DL Sc an ne r' and the corresponding git clone URL. If these appear literally in the published version, copy-pasting them will fail; please use monospaced 'DLScanner' consistently.","section":"Sec. 3"},{"comment":"The legend entry 'MLP regressor with Random sampling ×0.4' is unclear; if the curve is scaled to fit the axis, please state the scaling explicitly or use an inset or broken axis instead.","section":"Fig. 5"},{"comment":"The input-file description states that target labels must match SPheno output exactly, but it would be helpful to warn explicitly about whitespace, case sensitivity, and the need for a unique match when the same label appears in multiple output blocks.","section":"Sec. 5.2"}],"recommendation":"major_revision","confidential_remarks":"The package is real, open source, and reproducible, which argues against rejection. However, the headline quantitative claims are not yet supported by the evidence shown: the single-run 53-versus-57 iteration comparison is the kind of difference that needs seed-to-seed statistics, and the high-dimensional and MCMC/MultiNest superiority claims are currently unmeasured. I would ask for either additional experiments (multi-seed runs, a higher-dimensional benchmark, a direct baseline) or a careful scaling back of the claims in the abstract and conclusion."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nDLScanner is a genuinely useful piece of software: it packages an iterative deep-learning scanner for BSM parameter spaces, with a clean API, a generic sampler, SPheno and micrOMEGAs modules, and a trial script to reproduce the MSSM example. The new bit is the combination of a supervised similarity-learning network with a VEGAS map that is retrained each iteration; that specific pairing is not in the authors' prior ML-scan paper or elsewhere in the cited literature. For a software paper, the level of documentation is good, and the authors are honest about one key limitation: they state in Sec. 6.2 that the SL-vs-MLP difference is minimal in their 5D example.\n\nThe soft spots are about evidence for the headline claims, not about the code. All iteration counts in Fig. 5 come from one run per method. No seeds, no error bars, no repeated experiments. So the 53-iteration SL result versus 57 for MLP is essentially noise-level, and the paper's own text concedes as much. The larger VEGAS-versus-random gains (roughly 4-5x) are more plausible, but they are also single-run numbers. The abstract promises \"high-dimensional scans\" but the example is 5-dimensional, with the higher-dimensional case only anticipated. The comparison to MCMC/MultiNest is cited from Ref. [22], not measured here, so \"compared to other scanning methods\" in the abstract oversells this paper's direct evidence. Fig. 5 also scales the slowest baseline by 0.4 without explaining the axis break, which is sloppy. Minor: they refer to Eq. (12) when they mean the contrastive loss in Eq. (13).\n\nNone of this is fatal. The package is real, the code is available, and the qualitative claim that adaptive VEGAS sampling speeds up DL-based scanning is consistent with the numbers shown. What needs work is the statistical discipline: repeated seeds, spread of iteration counts, a direct MCMC/MultiNest benchmark in the same figure, and matching the abstract to what is actually measured.\n\nWho should read it: phenomenologists who do parameter scans and want off-the-shelf ML-assisted scanning. It deserves a serious referee; with a revision that adds error bars and tones down the claims, it would be a solid contribution to CPC or similar.","headline":"A genuinely useful open-source ML scanner package whose headline performance claims are under-supported by single-run benchmarks.","tokens_in":24766,"tokens_out":2505,"would_cite":true,"duration_ms":23040,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pairing a similarity-learning network with a VEGAS map makes deep-learning parameter scans converge faster than MLP classifiers or regressors.","keywords":["parameter space scan","deep learning","similarity learning","VEGAS sampling","MSSM","beyond the Standard Model","active learning","classifier"],"falsifier":"Run each sampling method with, say, 20 random seeds and an identical SPheno setup, then plot the distribution of iterations needed to collect 10,000 valid points; if the 53-iteration SL plus VEGAS result falls inside the spread of the MLP plus VEGAS runs, the claimed advantage is not established. A second check is to repeat the comparison in 8 to 10 dimensions and see whether SL plus VEGAS still leads.","tokens_in":23793,"feed_emoji":"🔭","tokens_out":5933,"duration_ms":51325,"temperature":0.7,"pith_summary":"DLScanner is an open-source Python tool that accelerates scans of physics parameter spaces by replacing random proposal points with a smarter loop: a deep network learns where the valid region is, and a VEGAS map suggests new points concentrated there. The paper's central claim is that combining a similarity-learning (SL) classifier with VEGAS converges fastest, collecting 10,000 valid points in the worked MSSM example in 53 iterations, versus 57 for an MLP classifier with VEGAS and 214 or 316 for classifiers with random sampling. The regressor variants lag far behind, needing 417–1580 iterations. The practical stakes are that expensive spectrum and dark-matter calculators get called far fewer times when the proposal points are already near the target region.","feed_headline":"Similarity learning plus VEGAS finds valid physics points 4x faster","feed_subtitle":"New scanner collects 10,000 MSSM points meeting Higgs-mass constraints in 53 iterations, against 214 with random sampling.","key_machinery":"The central object is the similarity-learning classifier used inside the predict-train loop. It is a two-encoder network with shared weights, trained with a contrastive loss so that valid points cluster together and invalid points are pushed apart in a normalized representation space; after training, one encoder is frozen and a fully connected layer maps the embeddings to a binary class probability. The other load-bearing piece is the VEGAS map, a piecewise-constant change of variables that is retrained each iteration on the accumulated in-target points and then used to generate the large pool $L$ from which the network selects the batch $K$ to send to the expensive calculator.","core_discovery":"The paper demonstrates, in a 5-dimensional MSSM scan with SPheno calculating the SM-like Higgs mass, that the number of iterations needed to collect 10,000 valid points depends strongly on the network and the proposal mechanism. The SL classifier with VEGAS reaches the target in 53 iterations, the MLP classifier with VEGAS in 57; switching to random sampling raises those to 214 and 316, and the MLP regressor needs 417 with VEGAS or 1580 with random sampling. The authors attribute the SL advantage to its two-step mapping: inputs are first embedded in a fixed-dimensional representation space where in-target and out-target points are separated by a contrastive loss, and only then classified, so the network's effective capacity is tied to the embedding dimension rather than the raw parameter dimension. They further attribute the VEGAS improvement to a map trained on accumulated in-target points, which generates proposal points at higher density near the valid region.","pith_inferences":["The 53-versus-57 iteration gap between SL and MLP is small; a more discriminative test would be higher-dimensional or multi-modal target regions, where the paper itself anticipates a clearer separation.","The VEGAS map is trained on accumulated valid points only, so in a target region with several disconnected islands the map may concentrate on one island; comparing coverage across islands per iteration would reveal whether the method explores or exploits.","For a community that already uses MCMC and nested sampling, SL plus VEGAS could serve as a proposal generator rather than a standalone scanner, combining the DL's boundary awareness with the convergence guarantees of existing samplers.","Wall-clock time, not iteration count, is the real cost when SPheno evaluations dominate; parallelizing the true-label calculations in the generic sampler would make the iteration-count advantage translate directly to runtime."],"forward_implications":["Adding VEGAS speeds up the MLP classifier from 316 to 57 iterations and the SL classifier from 214 to 53, so the proposal distribution matters as much as the network architecture.","Forcing the classifier to train on balanced valid and invalid points, and to keep uncertain points, means the final sample traces the decision boundary of the target region rather than only its center.","Because the SL network's representation space has fixed dimension, the paper expects the SL advantage over MLP to grow in higher-dimensional scans, where the MLP must directly approximate a mapping from a high-dimensional input.","The same iterative loop, with the same hyperparameters, applies to micrOMEGAs and to user-defined black-box functions, so the package can scan relic-density constraints or any custom observable without changing the sampling logic."],"supporting_citations":[{"why":"Supplies the VEGAS adaptive map used to generate proposal points concentrated near the target region.","marker":"[33]"},{"why":"The earlier machine-learning-assisted sampling loop that this package builds on and compares with.","marker":"[22]"},{"why":"Introduces active-learning DL scanning, the approach the paper's SL method is designed to improve.","marker":"[20]"},{"why":"The spectrum calculator used to produce true Higgs-mass labels in the MSSM scan.","marker":"[39]"},{"why":"The original SPheno program whose executable the scanning module wraps.","marker":"[40]"},{"why":"Represents the adaptive-sampling baseline that the paper claims DL methods converge faster than.","marker":"[5]"},{"why":"The dark-matter calculator interfaced by the package's second preconfigured scanning module.","marker":"[41]"},{"why":"Supplies the similarity and self-supervised learning idea at the heart of the SL classifier.","marker":"[26]"}],"fun_headline_variants":["SL network plus VEGAS: 4x faster scans for valid points","DLScanner: deep learning and adaptive sampling speed up scans","From 214 to 53 iterations: DLScanner accelerates parameter scans","AI-powered scanner finds physics points 4x quicker","DLScanner: similarity learning and VEGAS reduce scan iterations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported iteration counts come from a single run of each method, so the claimed ordering of convergence speeds assumes those runs are representative, and the promise of high-dimensional gains assumes the 5-dimensional MSSM example extends to higher dimensions—neither is demonstrated with error bars or repeated trials.","fun_headline_variants_meta":{"raw":{"variants":["SL network plus VEGAS: 4x faster scans for valid points","DLScanner: deep learning and adaptive sampling speed up scans","From 214 to 53 iterations: DLScanner accelerates parameter scans","AI-powered scanner finds physics points 4x quicker","DLScanner: similarity learning and VEGAS reduce scan iterations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000297,"raw_usage":{"total_tokens":1691,"prompt_tokens":886,"completion_tokens":805,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":718}},"tokens_in":502,"tokens_out":805,"duration_ms":29193,"temperature":1.0,"reasoning_tokens":718,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:59:13.331921+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run each sampling method with, say, 20 random seeds and an identical SPheno setup, then plot the distribution of iterations needed to collect 10,000 valid points; if the 53-iteration SL plus VEGAS result falls inside the spread of the MLP plus VEGAS runs, the claimed advantage is not established. A second check is to repeat the comparison in 8 to 10 dimensions and see whether SL plus VEGAS still leads.","supporting_citations":[],"review_version":1}