REVIEW 3 major objections 6 minor 31 references
Neural Architecture Codesign for Fast Physics Applications
T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A two-stage neural architecture codesign pipeline automatically discovers hardware-efficient models that match or beat hand-crafted baselines for Bragg peak localization and jet tagging.
desk verdict Useful engineering integration with open code and concrete FPGA results, but the headline accuracy claims are overstated and the baseline comparison is asymmetric. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the two-stage codesign loop. Stage one is a global search: a multi-objective genetic algorithm (NSGA-II) samples a modular, block-based search space of convolutional, attention, and fully-connected layers, scoring each candidate on task performance and bit operations (BOPs) so that hardware cost is part of the fitness. Stage two is a local search: the top candidates get hyperparameter optimization (tree-structured Parzen estimators), then quantization-aware training at 4/8/16/32-bit precision combined with iterative magnitude-based unstructured pruning up to 99% sparsity. The surviving models are synthesized to FPGA high-level synthesis code via hls4ml, with reuse factor and strategy tuned for latency or resource use. BOPs serve as the cheap proxy that lets the search rank hardware efficiency before expensive synthesis.
What would settle it
Run the original BraggNN and deep-sets baselines through the same local-search procedure (quantization-aware training at 8 bits, iterative magnitude pruning to 80% sparsity, and hls4ml synthesis at the same reuse factor) and compare accuracy, BOPs, latency, and resource use against the discovered models. For BraggNN, since its attention block cannot currently be synthesized, the test would be to remove the attention block from the original model, retrain it identically, and compare; if the resulting baseline matches or beats the NAC models on all metrics, the central claim is falsified.
Extended reading notes
Core claim
The central claim is that neural architecture codesign — optimizing architecture and hardware constraints together rather than sequentially — yields models that are Pareto-improved over hand-crafted baselines. Concretely, the paper reports that its discovered Bragg models match BraggNN's peak-fitting accuracy (mean distance 0.201 px vs 0.202 px) with 5.9x fewer bit operations, and that its medium jet classifier beats the deep-sets baseline's accuracy (65.06% vs 64.0%) with 7.2x fewer bit operations. On FPGA synthesis, the optimized models achieve latencies of 4.92 microseconds (Bragg small model) and 70 nanoseconds (jet tiny model) while using a small fraction of available DSPs, LUTs, and flip-flops. The paper also claims that the attention block in BraggNN is unnecessary for the task, since the discovered attention-free architectures perform equally well.
Load-bearing premise
The comparison treats the original BraggNN and deep-sets models as fixed baselines and does not apply the same quantization, pruning, hyperparameter tuning, and FPGA synthesis treatment to them; if those baselines were compressed and tuned equally, the reported advantages of the discovered models could shrink or disappear.
Editorial extensions
If this is right
- If the pipeline works as described, domain scientists can obtain deployable, FPGA-ready models for new physics tasks without hand-designing layers or manually tuning quantization and pruning.
- The reported results imply that large hand-crafted models often contain removable structure: BraggNN's attention block was found unnecessary, and both baselines could be pruned past 80% sparsity with less than 10% performance loss.
- The Pareto fronts produced by global and local search give end users a menu of accuracy-versus-latency tradeoffs, such as a jet tiny model with 30.25x fewer BOPs at only 2.8% lower accuracy.
- Because the search space is modular, the same pipeline should transfer to other scientific tasks, with hardware constraints folded in from the start.
Reading between the lines
- The paper does not compress and retune the original baselines, so a fairer head-to-head would likely shrink the reported margins; the absence of BraggNN from the hardware table makes this gap explicit.
- Because the authors acknowledge BOPs do not translate directly to latency, adding a latency or resource surrogate inside the global search could change which architectures are selected and is a natural next step.
- A testable extension is to apply the same two-stage loop to a third physics task with a strong hand-crafted baseline, to see whether the Pareto improvements generalize.
- The discovery that BraggNN's attention block is unnecessary suggests a general heuristic: for small local image patches, searching over a 'None' block can automatically eliminate attention, providing a cheap ablation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents Neural Architecture Codesign (NAC), a two-stage pipeline that combines global multi-objective NAS (NSGA-II) over a modular block-based search space with a local search stage that performs hyperparameter optimization, quantization-aware training, and iterative magnitude pruning, followed by FPGA synthesis via hls4ml. The method is evaluated on two physics tasks: Bragg peak localization in X-ray diffraction microscopy and jet classification in high-energy physics. The authors report that the discovered models match or exceed hand-crafted baselines in accuracy or distance while reducing bit operations by up to 5.9x (Bragg large) and 7.2x (jet medium), with synthesized FPGA latencies as low as 4.92 us for the Bragg task and 70 ns for the jet task.
Significance. The paper is a useful engineering contribution: it prototypes an end-to-end codesign pipeline that integrates NAS, compression, and hardware synthesis, and it releases the code publicly (nac-opt). The FPGA synthesis numbers are concrete and were produced independently by hls4ml/Vivado, and the BOPs metric is explicitly defined in Appendix A. The paper also honestly acknowledges in Section 6 that no discovered model exceeds baselines on all criteria and that BOPs do not translate directly to latency. However, the central comparative claim that NAC architectures outperform hand-crafted baselines is not yet fully supported because the baselines are not given the same compression and tuning treatment, and because no statistical controls (repeated seeds, test-set separation) are reported. If these comparison gaps are closed, the practical significance would be substantial; as presented, the magnitude of the architectural contribution is not isolated from the effect of extra compression effort.
major comments (3)
- [§4.2 and §5.2, Tables 1 and 3] The BOPs improvements are confounded by asymmetric baseline treatment. The NAC models are reported after QAT at multiple precisions and iterative pruning to high sparsity, while the original BraggNN (34,540 MegaBOPs) and deep sets (12.10 MegaBOPs) appear to be full-precision, unpruned baselines. Since the local-search stage includes compression, the reported 5.9x and 7.2x BOPs reductions cannot be attributed to architecture codesign alone. The central claim requires running the original baselines through the same local-search/compression pipeline (or at least the same QAT and pruning schedule) and comparing accuracy-versus-BOPs curves, rather than single points.
- [§4.2, Table 2] The BraggNN baseline is absent from the FPGA hardware table because its attention block cannot be synthesized, so no direct latency/resource comparison is available for the task with the largest claimed BOPs reduction. Without a synthesized baseline, the Bragg-task hardware claims are only absolute, not relative. For the jet task, Table 4 lists a Baseline row, but the text does not state whether that baseline received the same 8-bit quantization, reuse-factor tuning, and the custom hls4ml branch; this must be clarified before the latency comparison is meaningful.
- [§4.2 and §5.2, Figures 4 and 6] No error bars, repeated-seed variation, or train/test split description is provided. The reported differences are small (mean distance 0.202 vs 0.201 pixels; accuracy 64.0% vs 65.06%), and the models are selected from large Pareto searches (1000–2500 trials), so the reported numbers may partly reflect selection bias. The authors should specify the data split, report mean and standard deviation over several seeds for both the selected models and the baselines, and describe how the Pareto-front selection used a validation set distinct from the final test set.
minor comments (6)
- [§4.2] “resuse factor” should be “reuse factor”.
- [§5.2] “sparisty” should be “sparsity”.
- [§6] The phrase “0.87% relative improvement in LUT utilization” is ambiguous; it should specify the comparison being made and whether “relative” refers to percentage points or percent change.
- [Table 4] The text says “All models are quantized to 8 bits,” but the table does not show bit widths per row; state explicitly whether the Baseline row is also 8-bit quantized and whether it used the same custom hls4ml branch.
- [Figures 4 and 6] The legend for model size (Large/Medium/Small/Tiny) is difficult to distinguish in grayscale; consider using distinct markers, and label the pruning-iteration direction more clearly.
- [Appendix C] The search-space tables list channel dimensions up to 64, while §4.1 says the space was restricted “to ensure compatibility” with the target FPGA; describe the explicit compatibility filtering used during the Bragg search.
Circularity Check
No circularity found; the central claims rest on external task metrics, independent FPGA synthesis, and open-source tooling rather than on the paper's own outputs.
full rationale
I found no load-bearing step that reduces to its own inputs by construction. The NAC pipeline optimizes BOPs directly during search, so the reported BOPs reductions are expected optimization outcomes rather than predictions derived from the fitted quantities; the paper does not present them as independent discoveries. Accuracy is measured on the physics task itself, and FPGA latency, initiation interval, and resource utilization come from Vivado synthesis of hls4ml output, which is an external, independent evaluation path. The self-citations to hls4ml, the custom hls4ml branch, and the released NAC code are tooling/repository references with public implementations, not unverified theoretical claims invoked to close an argument. The most serious limitation is baseline asymmetry: the original BraggNN and deep sets baselines are not given the same local search, quantization-aware training, and pruning treatment, so the magnitude of the reported BOPs and latency gains may partly reflect extra compression effort rather than architecture alone. That is an experimental-control concern, not circularity, because the quoted baseline and NAC numbers are still independently measured. The paper also acknowledges its own limitation that no model exceeds all criteria and that BOPs do not translate directly to latency, further indicating the reported hardware numbers are not manufactured from the search objective. Overall, the derivation chain is self-contained with respect to the external benchmarks, and no circular step was identified.
Assumptions & free parameters
free parameters (4)
- Reuse factor =
4 (Bragg), 2 (jet)
- Pruning schedule =
20 iterations, 20% removal per iteration
- Quantization bit widths =
4, 8, 16, 32 bits; reported models use 8 bits
- Search compute budgets =
1000 trials/96 h (Bragg), 1000 trials/61 h (jet) global; 29 h/26 h local
assumptions (6)
- domain assumption BOPs formulas from Ref. [31] accurately rank computational cost of layers
- domain assumption hls4ml and AMD Vivado synthesis reports faithfully represent FPGA resource use and latency
- domain assumption The gold diffraction dataset and the jet dataset are representative and correctly labeled
- standard math NSGA-II and TPE provide reliable optimization over the search spaces
- domain assumption Permutation-invariant deep sets is an appropriate inductive bias for jet tagging
- domain assumption Unstructured pruning and QAT preserve accuracy at high sparsity on these tasks
Cite this review
Pith. "Pith review of Neural Architecture Codesign for Fast Physics Applications." pith.science (2026). https://pith.science/paper/K67FGP5U
@misc{pith2026250105515,
author = {Pith},
title = {Pith review of: Neural Architecture Codesign for Fast Physics Applications},
year = {2026},
howpublished = {\url{https://pith.science/paper/K67FGP5U}},
note = {Machine review of arXiv:2501.05515}
}
read the original abstract
We develop a pipeline to streamline neural architecture codesign for physics applications to reduce the need for ML expertise when designing models for novel tasks. Our method employs neural architecture search and network compression in a two-stage approach to discover hardware efficient models. This approach consists of a global search stage that explores a wide range of architectures while considering hardware constraints, followed by a local search stage that fine-tunes and compresses the most promising candidates. We exceed performance on various tasks and show further speedup through model compression techniques such as quantization-aware-training and neural network pruning. We synthesize the optimal models to high level synthesis code for FPGA deployment with the hls4ml library. Additionally, our hierarchical search space provides greater flexibility in optimization, which can easily extend to other tasks and domains. We demonstrate this with two case studies: Bragg peak finding in materials science and jet classification in high energy physics, achieving models with improved accuracy, smaller latencies, or reduced resource utilization relative to the baseline models.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Neural architecture search: A survey
T. Elsken, J. H. Metzen, and F. Hutter, “Neural architecture search: A survey”, J. Mach. Learn. Res. 20 (2019), no. 55, 1, arXiv:1808.05377
arXiv 2019
-
[2]
A survey on evolutionary neural architecture search
Y. Liu et al., “A survey on evolutionary neural architecture search”, IEEE Trans. Neural Netw. Learn. Syst. 34 (2021), no. 2, 550, doi:10.1109/tnnls.2021.3100554, arXiv:2008.10937
arXiv 2021
-
[3]
A survey on computationally efficient neural architecture search
S. Liu, H. Zhang, and Y. Jin, “A survey on computationally efficient neural architecture search”, J. Autom. Intell. 1 (2022), no. 1, 100002, doi:10.1016/j.jai.2022.100002, arXiv:2206.01520
arXiv 2022
-
[4]
A survey on multi-objective neural architecture search
S. M. Shariatzadeh, M. Fathy, R. Berangi, and M. Shahverdy, “A survey on multi-objective neural architecture search”, 2023. arXiv:2307.09099
arXiv 2023
-
[5]
Once for all: Train one network and specialize it for efficient deployment
H. Cai et al., “Once for all: Train one network and specialize it for efficient deployment”, in International Conference on Learning Representations . 2020. arXiv:1908.09791
arXiv 2020
-
[6]
A. Pappalardo, “Xilinx/brevitas: v0.11.0”, 2023. doi:10.5281/zenodo.3333552, https://github.com/Xilinx/brevitas
-
[7]
C. N. Coelho Jr. et al., “Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors”, Nat. Mach. Intell. 3 (2021) 675, doi:10.1038/s42256-021-00356-5 , arXiv:2006.10159
arXiv 2021
-
[8]
Optuna: A next-generation hyperparameter optimization framework
T. Akiba et al., “Optuna: A next-generation hyperparameter optimization framework”, in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , p. 2623. 2019
work page 2019
Show all 31 references
-
[9]
Fast inference of deep neural networks in FPGAs for particle physics
J. Duarte et al., “Fast inference of deep neural networks in FPGAs for particle physics”, JINST 13 (2018), no. 07, P07027, doi:10.1088/1748-0221/13/07/P07027, arXiv:1804.06913
2018 arXiv
-
[10]
BraggNN: fast X-ray Bragg peak analysis using deep learning
Z. Liu et al., “BraggNN: fast X-ray Bragg peak analysis using deep learning”, IUCrJ 9 (2022), no. 1, arXiv:2008.08198
2022 arXiv
-
[11]
Deep sets
M. Zaheer et al., “Deep sets”, in Advances in Neural Information Processing Systems, I. Guyon et al., eds., volume 30. Curran Associates, Inc., 2017. arXiv:1703.06114
2017 arXiv
-
[12]
Energy Flow Networks: Deep Sets for Particle Jets
P. T. Komiske, E. M. Metodiev, and J. Thaler, “Energy Flow Networks: Deep Sets for Particle Jets”, JHEP 01 (2019) 121, doi:10.1007/JHEP01(2019)121, arXiv:1810.05165
2019 arXiv
-
[13]
Ultrafast jet classification at the hl-lhc
P. Odagiu et al., “Ultrafast jet classification at the hl-lhc”, Mach. Learn. Sci. Tech. 5 (2024), no. 3, 035017, doi:10.1088/2632-2153/ad5f10, arXiv:2402.01876
2024 arXiv
-
[14]
Machine learning in high energy physics: a review of heavy-flavor jet tagging at the LHC
S. Mondal and L. Mastrolorenzo, “Machine learning in high energy physics: a review of heavy-flavor jet tagging at the LHC”, Eur. Phys. J. ST 233 (2024), no. 15, 2657, doi:10.1140/epjs/s11734-024-01234-y , arXiv:2404.01071. REFERENCES 20
2024 arXiv
-
[15]
fastmachinelearning/nac-opt: v0.2.0
D. Demler et al., “fastmachinelearning/nac-opt: v0.2.0”, 2025. doi:10.5281/zenodo.14618369, https://github.com/fastmachinelearning/nac-opt
2025 doi
-
[16]
A comprehensive survey of neural architecture search: Challenges and solutions
P. Ren et al., “A comprehensive survey of neural architecture search: Challenges and solutions”, ACM Computing Surveys (CSUR) 54 (2021), no. 4, 1
2021
-
[17]
Zero-cost proxies for lightweight NAS
M. S. Abdelfattah, A. Mehrotra, L. Dudziak, and N. D. Lane, “Zero-cost proxies for lightweight NAS”, in International Conference on Learning Representations
-
[18]
Algorithms for hyper-parameter optimization
J. Bergstra, R. Bardenet, Y. Bengio, and B. K´ egl, “Algorithms for hyper-parameter optimization”, in Advances in Neural Information Processing Systems, J. Shawe-Taylor et al., eds., volume 24. Curran Associates, Inc., 2011
2011
-
[19]
A fast and elitist multiobjective genetic algorithm: Nsga-ii
K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: Nsga-ii”, IEEE Trans. Evol. Comput. 6 (2002), no. 2, 182, doi:10.1109/4235.996017
2002
-
[20]
Pruning and quantization for deep neural network acceleration: A survey
T. Liang et al., “Pruning and quantization for deep neural network acceleration: A survey”, Neurocomputing 461 (2021) 370, arXiv:2101.09671
2021 arXiv
-
[21]
What is the state of neural network pruning?
D. Blalock, J. J. Gonzalez Ortiz, J. Frankle, and J. Guttag, “What is the state of neural network pruning?”, Proceedings of machine learning and systems 2 (2020) 129, arXiv:2003.03033
2020 arXiv
-
[22]
A hardware-friendly high-precision CNN pruning method and its FPGA implementation
X. Sui et al., “A hardware-friendly high-precision CNN pruning method and its FPGA implementation”, Sensors 23 (2023), no. 2, 824
2023
-
[23]
Learning best combination for efficient N:M sparsity
Y. Zhang et al., “Learning best combination for efficient N:M sparsity”, Advances in Neural Information Processing Systems 35 (2022) 941, arXiv:2206.06662
2022 arXiv
-
[24]
Applications and Techniques for Fast Machine Learning in Science
A. M. Deiana et al., “Applications and Techniques for Fast Machine Learning in Science”, Front. Big Data 5 (2022) 787421, doi:10.3389/fdata.2022.787421, arXiv:2110.13041
2022
-
[25]
fastmachinelearning/hls4ml: v0.8.0
FastML Team, “fastmachinelearning/hls4ml: v0.8.0”, 2023. doi:10.5281/zenodo.10140081, https://github.com/fastmachinelearning/hls4ml
2023 doi
-
[26]
Far-field high-energy diffraction microscopy: a non-destructive tool for characterizing the microstructure and micromechanical state of polycrystalline materials
J.-S. Park et al., “Far-field high-energy diffraction microscopy: a non-destructive tool for characterizing the microstructure and micromechanical state of polycrystalline materials”, Microscopy Today 25 (2017), no. 5, 36
2017
-
[27]
OpenHLS: High-Level Synthesis for Low-Latency Deep Neural Networks for Experimental Science
M. Levental et al., “OpenHLS: High-Level Synthesis for Low-Latency Deep Neural Networks for Experimental Science”, 2023. arXiv:2302.06751
2023 arXiv
-
[28]
fastmachinelearning/l1-jet-id: v0.2.0
Z. W. Que, J. Duarte, P. Odagiu, and A. Sznajder, “fastmachinelearning/l1-jet-id: v0.2.0”, 2024. doi:10.5281/zenodo.14531416, https://github.com/fastmachinelearning/l1-jet-id . REFERENCES 21
2024 doi
-
[29]
Sherlock: A multi-objective design space exploration framework
Q. Gautier, A. Althoff, C. Crutchfield, and R. Kastner, “Sherlock: A multi-objective design space exploration framework”, ACM Trans. Des. Autom. Electron. Syst. 27 (2022) doi:10.1145/3511472
2022 doi
-
[30]
rule4ml: An Open-Source Tool for Resource Utilization and Latency Estimation for ML Models on FPGA
M. M. Rahimifar, H. E. Rahali, and A. C. Therrien, “rule4ml: An Open-Source Tool for Resource Utilization and Latency Estimation for ML Models on FPGA”,
-
[31]
End-to-end codesign of hessian-aware quantized neural networks for fpgas
J. Campos et al., “End-to-end codesign of hessian-aware quantized neural networks for fpgas”, ACM Trans. Reconfigurable Technol. Syst. 17 (2024), no. 3, 1, doi:10.1145/3662000, arXiv:2304.06745
2024 arXiv
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.