REVIEW 3 major objections 4 minor 1 cited by
DLScanner: A parameter space scanner package assisted by deep learning methods
T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Pairing a similarity-learning network with a VEGAS map makes deep-learning parameter scans converge faster than MLP classifiers or regressors.
desk verdict A genuinely useful open-source ML scanner package whose headline performance claims are under-supported by single-run benchmarks. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the similarity-learning classifier used inside the predict-train loop. It is a two-encoder network with shared weights, trained with a contrastive loss so that valid points cluster together and invalid points are pushed apart in a normalized representation space; after training, one encoder is frozen and a fully connected layer maps the embeddings to a binary class probability. The other load-bearing piece is the VEGAS map, a piecewise-constant change of variables that is retrained each iteration on the accumulated in-target points and then used to generate the large pool $L$ from which the network selects the batch $K$ to send to the expensive calculator.
What would settle it
Run each sampling method with, say, 20 random seeds and an identical SPheno setup, then plot the distribution of iterations needed to collect 10,000 valid points; if the 53-iteration SL plus VEGAS result falls inside the spread of the MLP plus VEGAS runs, the claimed advantage is not established. A second check is to repeat the comparison in 8 to 10 dimensions and see whether SL plus VEGAS still leads.
Extended reading notes
Core claim
The paper demonstrates, in a 5-dimensional MSSM scan with SPheno calculating the SM-like Higgs mass, that the number of iterations needed to collect 10,000 valid points depends strongly on the network and the proposal mechanism. The SL classifier with VEGAS reaches the target in 53 iterations, the MLP classifier with VEGAS in 57; switching to random sampling raises those to 214 and 316, and the MLP regressor needs 417 with VEGAS or 1580 with random sampling. The authors attribute the SL advantage to its two-step mapping: inputs are first embedded in a fixed-dimensional representation space where in-target and out-target points are separated by a contrastive loss, and only then classified, so the network's effective capacity is tied to the embedding dimension rather than the raw parameter dimension. They further attribute the VEGAS improvement to a map trained on accumulated in-target points, which generates proposal points at higher density near the valid region.
Load-bearing premise
The reported iteration counts come from a single run of each method, so the claimed ordering of convergence speeds assumes those runs are representative, and the promise of high-dimensional gains assumes the 5-dimensional MSSM example extends to higher dimensions—neither is demonstrated with error bars or repeated trials.
Editorial extensions
If this is right
- Adding VEGAS speeds up the MLP classifier from 316 to 57 iterations and the SL classifier from 214 to 53, so the proposal distribution matters as much as the network architecture.
- Forcing the classifier to train on balanced valid and invalid points, and to keep uncertain points, means the final sample traces the decision boundary of the target region rather than only its center.
- Because the SL network's representation space has fixed dimension, the paper expects the SL advantage over MLP to grow in higher-dimensional scans, where the MLP must directly approximate a mapping from a high-dimensional input.
- The same iterative loop, with the same hyperparameters, applies to micrOMEGAs and to user-defined black-box functions, so the package can scan relic-density constraints or any custom observable without changing the sampling logic.
Reading between the lines
- The 53-versus-57 iteration gap between SL and MLP is small; a more discriminative test would be higher-dimensional or multi-modal target regions, where the paper itself anticipates a clearer separation.
- The VEGAS map is trained on accumulated valid points only, so in a target region with several disconnected islands the map may concentrate on one island; comparing coverage across islands per iteration would reveal whether the method explores or exploits.
- For a community that already uses MCMC and nested sampling, SL plus VEGAS could serve as a proposal generator rather than a standalone scanner, combining the DL's boundary awareness with the convergence guarantees of existing samplers.
- Wall-clock time, not iteration count, is the real cost when SPheno evaluations dominate; parallelizing the true-label calculations in the generic sampler would make the iteration-count advantage translate directly to runtime.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents DLScanner, an open-source Python package for scanning parameter spaces in BSM phenomenology with deep-learning assistance. The scanning loop trains a network on accumulated labeled points, uses the network to preselect candidate points from a large random or VEGAS-adapted sample, evaluates the selected points with an external tool (SPheno, micrOMEGAs, or a user function), and retrains the network. The package implements an MLP regressor, an MLP classifier, and a two-stage similarity-learning (SL) classifier, each optionally combined with a VEGAS map. The main demonstration is an MSSM scan over five parameters targeting m_h in [124,126] GeV, with convergence measured by the number of iterations needed to collect 10,000 valid points. The paper reports that the SL classifier with VEGAS converges in 53 iterations versus 57 for MLP+VEGAS, that VEGAS speeds up the classifiers by factors of 4 to 5.5 relative to random sampling, and that the MLP regressor is markedly slower.
Significance. If the performance claims hold, DLScanner would be a useful addition to the BSM scanning toolbox: it is released on PyPI and GitHub, provides a generic sampler interface, and includes reproducible SPheno and micrOMEGAs modules. The combination of similarity learning with an adaptive VEGAS map is a reasonable design idea, and making the code publicly available is a service to the community. However, the current manuscript does not yet establish the central quantitative claims: the comparison rests on one run per method in a 5-dimensional example, the SL-vs-MLP advantage is only four iterations, the high-dimensional claim is extrapolated rather than demonstrated, and the comparison with MCMC/MultiNest is cited from prior work rather than measured here.
major comments (3)
- [Sec. 6.2, Fig. 5] The central convergence comparison is based on a single run per method. The headline SL+VEGAS advantage over MLP+VEGAS is 53 versus 57 iterations, i.e., about 1,200 evaluated points out of roughly 17,000 at K=300, and no repeated seeds, error bars, or significance tests are reported. Since network initialization, training, VEGAS adaptation, and point selection are all stochastic, this difference is well within plausible seed-to-seed fluctuations. The larger VEGAS-versus-random and classifier-versus-regressor gaps are also single-run observations. Please report repeated-seed statistics or otherwise quantify the variability before claiming substantial gains.
- [Abstract and Sec. 6.2] The abstract claims that the package addresses 'slow convergence in high-dimensional scans,' but the only physics demonstration is five-dimensional, and the text itself states in Sec. 6.2 that the SL advantage is minimal in this 5D example and that a more distinct performance is 'anticipated' in higher dimensions. Since the high-dimensional claim is load-bearing for the paper's motivation, either add a higher-dimensional benchmark (the generic sampler in Sec. 5.1 is well suited for a synthetic target) or soften the abstract and conclusion accordingly.
- [Conclusion, final paragraph] The statement that all DL-based methods 'exhibit superior performance and faster convergence compared to adaptive sampling techniques such as MCMC or MultiNest' is not tested in this paper; it is deferred to Ref. [22]. Given that the abstract promises gains 'compared to other scanning methods,' please include a direct baseline on the same MSSM target with the same stopping rule, or restrict the conclusions to the methods actually benchmarked in Fig. 5.
minor comments (4)
- [Sec. 6.1] The text says the SL network error is computed using the 'contrastive loss function, Eq. (12),' but Eq. (12) is the cosine similarity distance; the contrastive loss is defined in Eq. (13). Please correct the cross-reference.
- [Sec. 3] The installation examples contain spaces inside the package name, e.g., 'pip install DL Sc an ne r' and the corresponding git clone URL. If these appear literally in the published version, copy-pasting them will fail; please use monospaced 'DLScanner' consistently.
- [Fig. 5] The legend entry 'MLP regressor with Random sampling ×0.4' is unclear; if the curve is scaled to fit the axis, please state the scaling explicitly or use an inset or broken axis instead.
- [Sec. 5.2] The input-file description states that target labels must match SPheno output exactly, but it would be helpful to warn explicitly about whitespace, case sensitivity, and the need for a unique match when the same label appears in multiple output blocks.
Circularity Check
Core convergence benchmarks are measured, not circular; a peripheral MCMC/MultiNest superiority claim rests on the authors' own Ref. [22].
-
self citation load bearing
[Sec. 6.2, final paragraph (p. 27); echoed in Sec. 7 Conclusion]
"Finally, we note that all DL-based methods, whether classifiers or regressors, exhibit superior performance and faster convergence compared to adaptive sampling techniques such as MCMC or MultiNest, as highlighted in Ref. [22]."
This sentence is the only support in the paper for the claim that DLScanner-type methods outperform MCMC/MultiNest, and it is a citation to Ref. [22] (Hammad, Park, Ramos and Saha, CPC 293 (2023) 108902), a prior paper sharing two of the present authors. No MCMC or MultiNest comparison is run in this study. The superiority claim therefore reduces to the authority of the authors' own earlier work rather than to a measurement made here. This is self-citation, though it is peripheral: the paper's main quantitative claim (SL+VEGAS converges in 53 iterations vs 57 for MLP+VEGAS) is measured internally in Fig. 5 and is not derived from Ref. [22].
full rationale
The paper does not present a mathematical derivation whose output is equivalent to its input. DLScanner's central results are empirical benchmarks: each method iteratively proposes points, SPheno evaluates the true Higgs mass, and convergence is measured by the number of iterations needed to collect 10,000 points satisfying 124 <= m_hSM <= 126 GeV. The SL network is trained with labels generated by exactly this target condition, and VEGAS is trained on accumulated valid points; this is the intended supervised/adaptive loop, not a circular prediction, because the reported figure of merit (valid points collected) is not a fitted parameter used to produce those same counts. The SL vs MLP vs regressor ordering in Fig. 5 is an internal, measured comparison, and the paper explicitly acknowledges that the SL advantage is minimal in the 5-dimensional example. The absence of repeated seeds or error bars is a statistical robustness limitation, but it is not circularity. The only self-referential element is the concluding claim of superiority over MCMC/MultiNest, which is imported from the authors' own Ref. [22] rather than demonstrated in this paper. Because the core benchmark is self-contained and the self-citation affects only a peripheral comparative statement, the circularity score is low.
Assumptions & free parameters
free parameters (6)
- classifier confidence threshold =
0.75
- regressor likelihood threshold =
0.9
- contrastive loss margin m =
1
- VEGAS map parameters =
ninc=100, nitn=5, alpha=1.0
- K batch size =
300
- L prediction set size =
50000
assumptions (4)
- domain assumption SPheno's spectrum calculation gives the true value of the SM-like Higgs mass used for in-target labeling.
- domain assumption The VEGAS map trained on accumulated valid points does not systematically miss otherwise reachable target regions.
- domain assumption The DL networks trained on accumulated points generalize to unseen regions well enough to propose useful new points.
- standard math Standard VEGAS algorithm properties from Ref. [33] hold as implemented in the vegas Python package.
Cite this review
Pith. "Pith review of DLScanner: A parameter space scanner package assisted by deep learning methods." pith.science (2026). https://pith.science/paper/ISTTMX63
@misc{pith2026241219675,
author = {Pith},
title = {Pith review of: DLScanner: A parameter space scanner package assisted by deep learning methods},
year = {2026},
howpublished = {\url{https://pith.science/paper/ISTTMX63}},
note = {Machine review of arXiv:2412.19675}
}
read the original abstract
In this paper, we introduce a scanner package enhanced by deep learning (DL) techniques. The proposed package addresses two significant challenges associated with previously developed DL-based methods: slow convergence in high-dimensional scans and the limited generalization of the DL network when mapping random points to the target space. To tackle the first issue, we utilize a similarity learning network that maps sampled points into a representation space. In this space, in-target points are grouped together while out-target points are effectively pushed apart. This approach enhances the scan convergence by refining the representation of sampled points. The second challenge is mitigated by integrating a dynamic sampling strategy. Specifically, we employ a VEGAS mapping to adaptively suggest new points for the DL network while also improving the mapping when more points are collected. Our proposed framework demonstrates substantial gains in both performance and efficiency compared to other scanning methods.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Normalizing Flow-Assisted Nested Sampling on Type-II Seesaw Model
A RealNVP normalizing flow trained inside nested sampling accelerates Bayesian scans of the Type-II seesaw parameter space and yields posterior constraints on scalar masses and couplings.
Reference graph
Works this paper leans on
- [22]
-
[1]
Speagle,A Conceptual Introduction to Markov Chain Monte Carlo Methods, 1909.12313
J.S. Speagle,A Conceptual Introduction to Markov Chain Monte Carlo Methods, 1909.12313
arXiv 1909
-
[2]
An effective introduction to the Markov Chain Monte Carlo method
W. Wang,An effective introduction to the markov chain monte carlo method, arXiv preprint arXiv:2204.10145 (2022)
work page Pith review arXiv 2022
-
[3]
C. Robert and G. Casella,A short history of markov chain monte carlo: Subjective recollections from incomplete data, Statistical Science26 (2011)
work page 2011
-
[4]
Introduction to Markov Chain Monte Carlo Simulations and their Statistical Analysis
B.A. Berg,Introduction to Markov chain Monte Carlo simulations and their statistical analysis, cond-mat/0410490
-
[5]
F. Feroz, M.P. Hobson and M. Bridges,MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics, Mon. Not. Roy. Astron. Soc.398 (2009) 1601 [0809.3437]
arXiv 2009
-
[6]
F. Feroz, M.P. Hobson, E. Cameron and A.N. Pettitt,Importance Nested Sampling and the MultiNest Algorithm, Open J. Astrophys.2 (2019) 10 [1306.2144]
arXiv 2019
-
[7]
P. Bechtle, K. Desch and P. Wienemann,Fittino, a program for determining MSSM parameters from collider observables using an iterative method, Comput. Phys. Commun. 174 (2006) 47 [hep-ph/0412012]
arXiv 2006
Show all 42 references
-
[8]
GAMBIT collaboration, Comparison of statistical sampling methods with ScannerBit, the GAMBIT scanning module, Eur. Phys. J. C77 (2017) 761 [1705.07959]
2017 arXiv
-
[9]
Goodsell and A
M.D. Goodsell and A. Joury,BSMArt: Simple and fast parameter space scans, Comput. Phys. Commun.297 (2024) 109057 [2301.01154]
2024 arXiv
-
[10]
Shang and Y
L. Shang and Y. Zhang,EasyScan_HEP: A tool for connecting programs to scan the parameter space of physics models, Comput. Phys. Commun.296 (2024) 109027 [2304.03636]
2024 arXiv
-
[11]
Lewis and S
A. Lewis and S. Bridle,Cosmological parameters from CMB and other data: A Monte Carlo approach, Phys. Rev. D66 (2002) 103511 [astro-ph/0205436]. 32
2002 arXiv
-
[12]
Ruiz de Austri, R
R. Ruiz de Austri, R. Trotta and L. Roszkowski,A Markov chain Monte Carlo analysis of the CMSSM, JHEP 05 (2006) 002 [hep-ph/0602028]
2006 arXiv
-
[13]
Allanach and C.G
B.C. Allanach and C.G. Lester,Sampling using a ‘bank’ of clues, Comput. Phys. Commun. 179 (2008) 256 [0705.0486]
2008 arXiv
-
[14]
Strege, G
C. Strege, G. Bertone, G.J. Besjes, S. Caron, R. Ruiz de Austri, A. Strubig et al., Profile likelihood maps of a 15-dimensional MSSM, JHEP 09 (2014) 081 [1405.0622]
2014 arXiv
-
[15]
Han, K.-i
C. Han, K.-i. Hikasa, L. Wu, J.M. Yang and Y. Zhang,Status of CMSSM in light of current LHC Run-2 and LUX data, Phys. Lett. B769 (2017) 470 [1612.02296]
2017 arXiv
-
[16]
Bagnaschi et al.,Likelihood Analysis of the pMSSM11 in Light of LHC 13-TeV Data, Eur
E. Bagnaschi et al.,Likelihood Analysis of the pMSSM11 in Light of LHC 13-TeV Data, Eur. Phys. J. C78 (2018) 256 [1710.11091]
2018 arXiv
-
[17]
Brinckmann and J
T. Brinckmann and J. Lesgourgues,MontePython 3: boosted MCMC sampler and other features, Phys. Dark Univ.24 (2019) 100260 [1804.07261]
2019 arXiv
-
[18]
M.A. Diaz, G. Cerro, S. Dasmahapatra and S. Moretti,Bayesian Active Search on Parameter Space: a 95 GeV Spin-0 Resonance in the (B−L)SSM, 2404.18653
-
[19]
M.A. Diaz, S. Dasmahapatra and S. Moretti,hep-aid: A Python Library for Sample Efficient Parameter Scans in Beyond the Standard Model Phenomenology, 2412.17675
-
[20]
Goodsell and A
M.D. Goodsell and A. Joury,Active learning BSM parameter spaces, Eur. Phys. J. C 83 (2023) 268 [2204.13950]
2023 arXiv
-
[21]
J. Ren, L. Wu, J.M. Yang and J. Zhao,Exploring supersymmetry with machine learning, Nucl. Phys. B943 (2019) 114613 [1708.06615]
2019 arXiv
-
[23]
Graff, F
P. Graff, F. Feroz, M.P. Hobson and A. Lasenby,BAMBI: blind accelerated multimodal Bayesian inference, Monthly Notices of the Royal Astronomical Society (2012) no [1110.2997]
2012 arXiv
-
[24]
Staub,xBIT: an easy to use scanning tool with machine learning abilities, 1906.03277
F. Staub,xBIT: an easy to use scanning tool with machine learning abilities, 1906.03277
1906 arXiv
-
[25]
Caron, T
S. Caron, T. Heskes, S. Otten and B. Stienen,Constraining the Parameters of High-Dimensional Models with Active Learning, Eur. Phys. J. C79 (2019) 944 [1905.08628]
2019 arXiv
-
[26]
Dillon, G
B.M. Dillon, G. Kasieczka, H. Olischlager, T. Plehn, P. Sorrenson and L. Vogel, Symmetries, safety, and self-supervision, SciPost Phys.12 (2022) 188 [2108.04253]
2022 arXiv
-
[27]
Esmail, A
W. Esmail, A. Hammad and S. Moretti,Sharpening the A→ Z(∗)h signature of the Type-II 2HDM at the LHC through advanced Machine Learning, JHEP 11 (2023) 020 [2305.13781]
2023 arXiv
-
[28]
Dillon, L
B.M. Dillon, L. Favaro, F. Feiden, T. Modak and T. Plehn,Anomalies, representations, and self-supervision, SciPost Phys. Core7 (2024) 056 [2301.04660]. 33
2024 arXiv
-
[29]
Hammad, M.M
A. Hammad, M.M. Nojiri and M. Yamazaki,Quantum similarity learning for anomaly detection, 2411.09927
-
[30]
Hallin, G
A. Hallin, G. Kasieczka, S. Kraml, A. Lessa, L. Moureaux, T. von Schwartz et al., Universal New Physics Latent Space, 2407.20315
-
[31]
Lepage,A New Algorithm for Adaptive Multidimensional Integration, J
G.P. Lepage,A New Algorithm for Adaptive Multidimensional Integration, J. Comput. Phys.27 (1978) 192
1978
-
[32]
Lepage,VEGAS: An Adaptive Multidimensional Integration Program, CLNS-80/447 (1980)
G.P. Lepage,VEGAS: An Adaptive Multidimensional Integration Program, CLNS-80/447 (1980)
1980
-
[33]
Lepage,Adaptive multidimensional integration: VEGAS enhanced, J
G.P. Lepage,Adaptive multidimensional integration: VEGAS enhanced, J. Comput. Phys. 439 (2021) 110386 [2009.05112]
2021 arXiv
-
[34]
gplepage/vegas: vegas version 6.1.3
G.P. Lepage, “gplepage/vegas: vegas version 6.1.3.” URL: https://doi.org/10.5281/zenodo.12687656, July, 2024
2024 doi
-
[35]
Install tensorflow with pip
TensorFlow, “Install tensorflow with pip.” URL: https://www.tensorflow.org/install/pip
-
[36]
Tested build configurations
TensorFlow, “Tested build configurations.” URL: https://www.tensorflow.org/install/source#gpu
-
[37]
DLScanner
A. Hammad and R. Ramos, “DLScanner.” URL: https://github.com/raalraan/DLScanner
-
[38]
Kingma and J
D.P. Kingma and J. Ba,Adam: A method for stochastic optimization, 2017
2017
-
[39]
Porod and F
W. Porod and F. Staub,SPheno 3.1: Extensions including flavour, CP-phases and models beyond the MSSM, Comput. Phys. Commun.183 (2012) 2458 [1104.1573]
2012 arXiv
-
[40]
Porod,SPheno, a program for calculating supersymmetric spectra, SUSY particle decays and SUSY particle production at e+ e- colliders, Comput
W. Porod,SPheno, a program for calculating supersymmetric spectra, SUSY particle decays and SUSY particle production at e+ e- colliders, Comput. Phys. Commun. 153 (2003) 275 [hep-ph/0301101]
2003 arXiv
-
[41]
Alguero, G
G. Alguero, G. Belanger, F. Boudjema, S. Chakraborti, A. Goudelis, S. Kraml et al., micrOMEGAs 6.0: N-component dark matter, Comput. Phys. Commun.299 (2024) 109133 [2312.14894]
2024 arXiv
-
[42]
Djouadi,The Anatomy of electro-weak symmetry breaking
A. Djouadi,The Anatomy of electro-weak symmetry breaking. II. The Higgs bosons in the minimal supersymmetric model, Phys. Rept.459 (2008) 1 [hep-ph/0503173]. 34
2008 arXiv
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.