REVIEW 3 major objections 5 minor 1 cited by
Enforcing Fundamental Relations via Adversarial Attacks on Input Parameter Correlations
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper claims that adversarial examples made by shuffling feature values along their own distributions can serve as data augmentation, improving classification performance on several benchmarks by forcing models to attend to…
desk verdict RDSA is a simple but genuinely effective correlation-breaking attack; the augmentation story needs a control before the mechanism claim holds. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Random Distribution Shuffle Attack (RDSA), which resamples selected input features from their empirical one-dimensional histograms, so that marginals are preserved while inter-feature correlations are broken by the shuffling. The load-bearing mechanism is the contrast between these two effects, measured by the Jensen-Shannon distance (small) and the mean absolute difference of correlation matrices (large); after retraining on such adversaries, a classifier that relied on marginal statistics can no longer do so and must encode correlations to classify correctly.
What would settle it
On the VBF and Rain in Australia panels of Figure 8, every tested augmentation method, RDSA included, has mean AUROC below the data-starved baseline, so reproducing that systematic degradation is the concrete check against any general claim that correlation-breaking augmentation improves classification.
Extended reading notes
Core claim
RDSA is a non-gradient attack: for each correctly predicted input it selects nVars features at random, redraws their values from empirical histograms built over the whole dataset, and repeats the shuffle until the model returns a different class or a step limit is reached. The resulting adversarial examples achieve high fooling ratios on all six benchmarks while keeping the Jensen-Shannon distance between original and adversarial feature distributions small, often around $10^{-2}$, and while increasing the mean absolute difference between the clean and adversarial correlation matrices. Used as augmentation, RDSA retrains the data-starved model on the reduced clean set plus correctly labeled adversaries, and the paper reports improved AUROC on TopoDNN, MIMIC-IV, MNIST784, and HAR, with all augmentation methods including RDSA degrading AUROC on the VBF and Rain in Australia tasks.
Load-bearing premise
The load-bearing premise is that training a network on correctly labeled examples whose inter-feature correlations have been destroyed, while keeping each feature's own distribution intact, makes it rely on the true correlations and generalize better on clean data.
Editorial extensions
If this is right
- RDSA produces high-fooling-ratio adversaries while keeping one-dimensional feature distributions almost unchanged, a property gradient-based attacks like LowProFool do not share.
- Adversarial training with RDSA improves AUROC over a data-starved baseline on TopoDNN, MIMIC-IV, MNIST784, and HAR, and is competitive with or better than CTGAN, TVAE, and LPF on several of these tasks.
- RDSA applies to tabular data across physics, weather, medicine, and activity recognition, so the approach is not domain-specific.
- Shuffling all continuous features drives inter-feature correlations toward zero while preserving marginals, providing a controlled way to test how much a classifier depends on correlations.
Reading between the lines
- RDSA could be used as a diagnostic probe: the fooling ratio as a function of how many features are shuffled measures how strongly a model depends on pairwise correlations, useful for comparing models trained on real versus simulated data.
- Because RDSA stays on the empirical marginal manifold, combining it with gradient-based attacks would probe a complementary class of vulnerabilities, giving a fuller robustness profile than either attack alone.
- A testable extension is to shuffle only selected feature pairs and track the AUROC drop; the pairs whose shuffling hurts most are the correlation structure the model actually relies on, turning the attack into an interpretability tool.
- The negative VBF and Rain in Australia results suggest RDSA augmentation should not be applied blindly; a confidence-weighted or per-class version might recover gains where the current recipe fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces Random Distribution Shuffle Attack (RDSA), an adversarial attack that resamples input features from their one-dimensional histograms to break inter-feature correlations while preserving marginal distributions. The authors evaluate RDSA on six classification tasks (VBF, TopoDNN, Rain in Australia, MIMIC-IV, MNIST784, HAR), reporting high fooling ratios with small Jensen-Shannon distances and substantial correlation-matrix changes. They further apply RDSA-generated adversaries as a data-augmentation strategy under data-starved training regimes, comparing against CTGAN, TVAE, and LowProFool (LPF). The paper claims that adversarial training with RDSA improves classifier performance and robustness, with the proposed mechanism being that retraining on correlation-broken adversaries forces networks to focus on correlations. The reported experiments show AUROC gains over the data-starved baseline on TopoDNN, MIMIC-IV, MNIST784, and HAR, while VBF and Rain in Australia show mostly negative effects.
Significance. The attack itself is a simple and interesting contribution: it directly targets the correlation structure of tabular data while preserving marginals, and the paper demonstrates this behavior quantitatively across diverse domains. The evaluation is broad, including six datasets and comparisons with three established augmentation baselines, with uncertainty estimated over repeated runs. The pseudo-code and the detailed appendices facilitate replication, provided the issues noted below are fixed. If the central mechanism claim were established, the work would be a useful addition to adversarial robustness and tabular data augmentation. However, the load-bearing hypothesis that augmentation gains arise from 'focusing on correlations' is not directly tested, and the negative results on two datasets limit the generality of the conclusions as currently stated.
major comments (3)
- [Sections 1 and 6; Figure 8] The central claim that retraining on RDSA adversaries forces the model to 'focus on correlations' and thereby improves clean-data generalization is not supported by the experimental design. Section 1 states this hypothesis, and Section 6 invokes it to interpret the MIMIC-IV results, but the experiments do not isolate the correlation-focusing mechanism. RDSA combines two operations: marginal-preserving independent resampling of features and selection of samples that fool the model. A control that breaks correlations in the same way but without conditioning on model misclassification (e.g., random shuffles of the same variables, or shuffles drawn without the adversarial query) is missing. Without such a control, the observed AUROC improvements on four datasets are equally consistent with generic label-preserving noise acting as a regularizer. Moreover, the hypothesis predicts benefits whenever correlations are informative, yet for VBF and Rain in Australia all augmentation methods including RDSA decrease AUROC below the data-starved baseline (Figure 8c,d). The paper should either provide a direct test of the correlation-focusing mechanism or substantially weaken the causal language in the abstract and conclusions.
- [Appendix D, pseudo-code; Section 3] The provided pseudo-code for RDSA does not match the textual description and is not runnable as written. The function call `random.sample(range(0, len(input), nVars))` is missing the required second argument `k`; moreover, using `nVars` as the step of `range` selects indices 0, nVars, 2*nVars, ..., rather than a random subset of `nVars` variables as described in Section 3 (the intended call is presumably `random.sample(range(len(input)), nVars)`). In addition, the pseudo-code's inner loop resamples every selected variable at every one of the `max_steps` attempts, while the text in Section 3 suggests that after each full shuffling attempt the model is queried; the code as written would re-shuffle all variables repeatedly before a single prediction, which is inconsistent with the description and with the reported 'up to 100 shuffle attempts'. The handling of failed examples (when the function returns `None`) is also unspecified, which affects the fooling-ratio calculation. The method definition should be corrected and clarified.
- [Abstract and Section 8 (Conclusions) versus Figure 8] The conclusions overstate the generality of the augmentation results. The abstract claims that 'correct application of the proposed novel attack can result in a significant improvement in classification performance', and Section 8 says retraining 'showed the capability to significantly improve their performance and robustness, often outperforming standard data augmentation methods'. This is contradicted by the Rain in Australia and VBF results, where Figure 8 shows that all RDSA configurations (and indeed all augmentation methods tested) perform below the base model within uncertainties. The paper should revise these statements to acknowledge that the benefit is dataset-dependent, or provide an analysis of why the mechanism fails on these datasets. As written, the conclusion is not supported by the full set of reported results.
minor comments (5)
- [Abstract] The sentence 'Given that correlations between input features are also crucial in many other disciplines.' is an incomplete sentence fragment and should be integrated into the surrounding text.
- [Section 5, equations for metrics] The correlation-difference metric is defined as ⟨cf⟩? Actually, the average correlation difference is defined with the equation after '⟨cc⟩ :=' in Section 5, but the text and Figure 5 caption refer to it as ⟨s⟩. This notation inconsistency should be fixed.
- [Section 6 and 7 (data augmentation pipeline)] Section 6 states the process is repeated 100 times, but the pipeline description in the same section says 'using a fixed random seed of 42 to ensure consistent results across 50 iterations.' The number of iterations should be unified.
- [Appendix A] The data and code availability statement is a placeholder ('Currently not in here, as it is not anonymous. We will put the links back in here once the anonymous phase is over.'). For reproducibility, the final version should include the actual links or a statement of availability.
- [Section 4] There is a typo: 'continouous' should be 'continuous'. Similar minor typos appear elsewhere (e.g., 'actuall' in Section 1).
Circularity Check
No significant circularity: RDSA's advertised properties follow from its definition, but the central augmentation claim is evaluated against external baselines and data-starved controls.
full rationale
The paper's central claim is that adversarial training with RDSA-generated examples improves classification performance relative to a data-starved baseline and competes with CTGAN, TVAE, and LPF. This claim is tested empirically on six datasets and is not obtained by fitting any parameter to the reported AUROC values. RDSA is defined as a shuffling procedure that resamples feature values from one-dimensional histograms; consequently, low Jensen-Shannon distances and increased correlation-matrix differences are properties of the construction rather than derived predictions, and the paper presents them as expected method characteristics rather than as independent confirmations. The attack's high fooling ratio similarly follows from the rejection-sampling loop that retains only misclassified candidates, but this is the algorithm's stated goal, not a hidden circular inference. The 'focus on correlations' hypothesis is used as an interpretive mechanism, not as a mathematical premise that forces the outcome; indeed, the augmentation degrades performance on two of the six datasets, showing that the results are not tautological. No load-bearing self-citations appear, and the comparisons to CTGAN, TVAE, and LPF provide external anchors. The absence of a control that breaks correlations without conditioning on model misclassification is a legitimate experimental-design concern about causal attribution, but it does not make the derivation circular. Overall, the derivation chain is self-contained and the empirical claims are externally benchmarked.
Assumptions & free parameters
free parameters (4)
- nBins =
1000
- max_steps (Ns) =
100
- nVars per dataset =
VBF 1-8, TopoDNN 10-87, Rain 1-9, MIMIC 1-33, MNIST 1-560, HAR 1-548
- Reduced training set sizes =
VBF 1000, TopoDNN 60878, Rain 1396, MIMIC 9198, MNIST 11200, HAR 1647
assumptions (3)
- domain assumption Correlations between observables are governed by fundamental principles and are more stable than individual feature distributions.
- domain assumption Independently resampling each feature from its marginal histogram reduces correlations toward zero.
- ad hoc to paper Retraining on correlation-broken adversaries with original labels encourages a model to focus on correlations and generalize better.
Cite this review
Pith. "Pith review of Enforcing Fundamental Relations via Adversarial Attacks on Input Parameter Correlations." pith.science (2026). https://pith.science/paper/FDFGXHIT
@misc{pith2026250105588,
author = {Pith},
title = {Pith review of: Enforcing Fundamental Relations via Adversarial Attacks on Input Parameter Correlations},
year = {2026},
howpublished = {\url{https://pith.science/paper/FDFGXHIT}},
note = {Machine review of arXiv:2501.05588}
}
read the original abstract
Correlations between input parameters play a crucial role in many scientific classification tasks, since these are often related to fundamental laws of nature. For example, in high energy physics, one of the common deep learning use-cases is the classification of signal and background processes in particle collisions. In many such cases, the fundamental principles of the correlations between observables are often better understood than the actual distributions of the observables themselves. In this work, we present a new adversarial attack algorithm called Random Distribution Shuffle Attack (RDSA), emphasizing the correlations between observables in the network rather than individual feature characteristics. Correct application of the proposed novel attack can result in a significant improvement in classification performance - particularly in the context of data augmentation - when using the generated adversaries within adversarial training. Given that correlations between input features are also crucial in many other disciplines. We demonstrate the RDSA effectiveness on six classification tasks, including two particle collision challenges (using CERN Open Data), hand-written digit recognition (MNIST784), human activity recognition (HAR), weather forecasting (Rain in Australia), and ICU patient mortality (MIMIC-IV), demonstrating a general use case beyond fundamental physics for this new type of adversarial attack algorithms.
Figures
Figures from the paper (12 more)
Forward citations
Cited by 1 Pith paper
-
MiniFool -- Physics-Constraint-Aware Minimizer-Based Adversarial Attacks in Deep Neural Networks
MiniFool generates uncertainty-aware adversarial perturbations and uses the attack strength needed to flip a classification as a measure of how robust the network's decision is.
Reference graph
Works this paper leans on
-
[1]
A. Adelmann, W. Hopkins, E. Kourlitis, M. Kagan, G. Kasieczka, C. Krause, D. Shih, V. Mikuni, B. Nachman, K. Pedro, D. Winklehner, New directions for surrogate models and differentiable programming for high energy physics detector simulation (2022).arXiv:2203.08806
arXiv 2022
-
[2]
V. S. Ngairangbam, A. Bhardwaj, P. Konar, A. K. Nayak, Invisible Higgs search through vector boson fusion: a deep learning approach, The European Physical Journal C 80 (11) (Nov. 2020).doi:10.1140/epjc/ s10052-020-08629-w. URL http://dx.doi.org/10.1140/epjc/s10052-020-08629-w
doi:10.1140/epjc/ 2020
-
[3]
M. Migliorini, J. Pazzini, A. Triossi, M. Zanetti, A. Zucchetta, Muon trigger with fast neural networks on fpga, a demonstrator, Journal of Physics: Conference Series 2374 (1) (2022) 012099.doi:10.1088/1742-6596/ 2374/1/012099. URL http://dx.doi.org/10.1088/1742-6596/2374/1/012099 11
-
[4]
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, R. Fergus, Intriguing properties of neural networks (2014).arXiv:1312.6199
arXiv 2014
-
[6]
S.-M. Moosavi-Dezfooli, A. Fawzi, P. Frossard, Deepfool: a simple and accurate method to fool deep neural networks (2016). arXiv:1511.04599
arXiv 2016
- [7]
-
[8]
I. J. Goodfellow, J. Shlens, C. Szegedy, Explaining and harnessing adversarial examples (2015). arXiv: 1412.6572. URL https://arxiv.org/abs/1412.6572
arXiv 2015
-
[9]
S. Chatrchyan, et al., The CMS Experiment at the CERN LHC, JINST 3 (2008) S08004.doi:10.1088/ 1748-0221/3/08/S08004
work page 2008
Show all 30 references
-
[10]
LHC Machine, JINST 3 (2008) S08001.doi:10.1088/1748-0221/3/08/S08001
2008 doi
-
[11]
Z. Kong, J. Guo, A. Li, C. Liu, Physgan: Generating physical-world-resilient adversarial examples for autonomous driving (2021). arXiv:1907.04449
2021 arXiv
-
[12]
Ballet, X
V. Ballet, X. Renard, J. Aigrain, T. Laugel, P. Frossard, M. Detyniecki, Imperceptible adversarial attacks on tabular data (2019). arXiv:1911.03274. URL https://arxiv.org/abs/1911.03274
2019 arXiv
-
[13]
L. Xu, M. Skoularidou, A. Cuesta-Infante, K. Veeramachaneni, Modeling tabular data using conditional gan (2019). arXiv:1907.00503
2019 arXiv
-
[14]
URL https://opendata.cern.ch/
CERN, CERN Open Data Portal (2024). URL https://opendata.cern.ch/
2024
-
[15]
CMS collaboration, Simulated dataset VBFToHToWWToLAndTauNuQQ_M-125_8TeV-powheg-pythia6 in AODSIM format for 2012 collision data (2017).doi:10.7483/OPENDATA.CMS.YA1K.II60
2017 doi
-
[16]
doi:10.7483/OPENDATA.CMS.YN3L.COMS
CMS collaboration, Simulated dataset LplusNuVBF_Mqq-120_8TeV-madgraph in AODSIM format for 2012 collision data (2017). doi:10.7483/OPENDATA.CMS.YN3L.COMS
2017 doi
-
[17]
CMS collaboration, Simulated dataset WpWmJJToLNuQQ_TuneZ2star_8TeV-vbfnlo-pythia6 in AODSIM format for 2012 collision data (2017).doi:10.7483/OPENDATA.CMS.52HO.31V8
2017 doi
-
[18]
CMS collaboration, Simulated dataset WpWmJJToQQLNuBar_TuneZ2star_8TeV-vbfnlo-pythia6 in AODSIM format for 2012 collision data (2017).doi:10.7483/OPENDATA.CMS.4HOX.06QU
2017 doi
-
[19]
CMS collaboration, Simulated dataset ZJetToMuMu_Pt-80to120_TuneEE3C_8TeV_herwigpp in AODSIM format for 2012 collision data (2017).doi:10.7483/OPENDATA.CMS.TYJU.X3NA
2017 doi
-
[20]
CMS collaboration, Simulated dataset TTJets_FullLeptMGDecays_TuneP11TeV_8TeV-madgraph-tauola in AODSIM format for 2012 collision data (2017).doi:10.7483/OPENDATA.CMS.7RZ3.0BXP
2017 doi
-
[21]
CMS collaboration, Simulated dataset WWJetsTo2L2Nu_TuneZ2star_8TeV-madgraph-tauola in AODSIM format for 2012 collision data (2017).doi:10.7483/OPENDATA.CMS.V2C6.O1P4
2017 doi
-
[22]
Kasieczka, T
G. Kasieczka, T. Plehn, A. Butter, K. Cranmer, D. Debnath, B. M. Dillon, M. Fairbairn, D. A. Faroughy, W. Fedorko, C. Gay, L. Gouskos, J. F. Kamenik, P. Komiske, S. Leiss, A. Lister, S. Macaluso, E. Metodiev, L. Moore, B. Nachman, K. Nordström, J. Pearkes, H. Qu, Y. Rath, M. R...
2019 doi
-
[23]
URL https://www.kaggle.com/datasets/jsphyg/weather-dataset-rattle-package
Kaggle, Rain in Australia (2020) [cited 17.03.2024]. URL https://www.kaggle.com/datasets/jsphyg/weather-dataset-rattle-package
2020
-
[24]
Deng, The mnist database of handwritten digit images for machine learning research, IEEE Signal Processing Magazine 29 (6) (2012) 141–142
L. Deng, The mnist database of handwritten digit images for machine learning research, IEEE Signal Processing Magazine 29 (6) (2012) 141–142
2012
-
[25]
Garcia-Gonzalez, D
D. Garcia-Gonzalez, D. Rivero, E. Fernandez-Blanco, M. R. Luaces, A public domain dataset for real-life human activity recognition using smartphone sensors, Sensors 20 (8) (2020).doi:10.3390/s20082200. URL https://www.mdpi.com/1424-8220/20/8/2200
2020 doi
-
[26]
Johnson, L
A. Johnson, L. Bulgarelli, T. Pollard, S. Horng, L. A. Celi, R. Mark, MIMIC-IV 2.2 (Jan. 2023).doi:https: //doi.org/10.13026/6mm1-ek67. URL https://physionet.org/content/mimiciv/2.2/
2023 doi
-
[27]
A. E. W. Johnson, L. Bulgarelli, L. Shen, A. Gayles, A. Shammout, S. Horng, T. J. Pollard, S. Hao, B. Moody, B. Gow, L.-w. H. Lehman, L. A. Celi, R. G. Mark, MIMIC-IV, a freely accessible electronic health record dataset, Scientific Data 10 (1) (2023) 1.doi:10.1038/s41597-022-...
2023 doi
-
[28]
A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, H. E. Stanley, PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research 12 resource for complex physiologic signals, Circulation 101 (23...
-
[29]
Gupta, B
M. Gupta, B. Gallamoza, N. Cutrona, P. Dhakal, R. Poulain, R. Beheshti, An Extensive Data Processing Pipeline for MIMIC-IV, in: Proceedings of the 2nd Machine Learning for Health symposium, Vol. 193 of Proceedings of Machine Learning Research, PMLR, 2022, pp. 311–325. URL http...
2022
-
[30]
Menéndez, J
M. Menéndez, J. Pardo, L. Pardo, M. Pardo, The jensen-shannon divergence, Journal of the Franklin Institute 334 (2) (1997) 307–318.doi:https://doi.org/10.1016/S0016-0032(96)00063-4. URL https://www.sciencedirect.com/science/article/pii/S0016003296000634
1997 doi
-
[31]
__main__
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, İ. Polat, Y. Feng,...
2020
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.