REVIEW 5 major objections 6 minor 1 cited by
Quark: Implementing Convolutional Neural Networks Entirely on Programmable Data Plane
T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A pruned, 7-bit CNN can run entirely inside a Tofino switch pipeline
desk verdict First real Tofino deployment of full in-pipeline CNN inference — the feasibility result is real, but the line-rate claim needs a trigger-rate bound before it can be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The object that carries the argument is the CAP-Unit—a pipeline module that fuses a convolutional or fully connected layer with ReLU and maxpooling and processes two feature positions per pass. Three mechanisms make it fit: channel pruning plus quantization-aware training shrink the model; a multiplication table stored in SRAM replaces hardware multipliers by precomputing all products; and recirculation, bounded by Theorem 1 as at most $\lceil (T + L_{\mathrm{conv}} + L_{\mathrm{fc}}) \cdot C^2 \rceil$ passes, lets one unit service a whole network. The quantization arithmetic is driven by the scale $S$, zero-point $Z$, and the precomputed multiplier $M = S_w S_x / S_a$, which turns the quantized convolution into integer multiply-accumulate plus a bit shift.
What would settle it
Run Quark on a Tofino switch while replaying a 40 Gbps trace in which every arriving flow immediately triggers a first-packet inference with no cached prediction, and measure forwarding throughput and packet drops. If throughput falls substantially below 39.7 Gbps or packets drop, the unconditional line-rate claim is false; otherwise the claim is supported.
Extended reading notes
Core claim
The central claim is that the usual reasons CNNs cannot run on Tofino-class switches—no floating point, no multiplication, no loops, small per-stage SRAM—can all be worked around without leaving the P4 pipeline. Quark keeps model accuracy by pruning channels at rate 0.8 and training with fake-quantize nodes before 7-bit quantization, then converts every remaining multiply into a lookup in a SRAM match-action table and every layer into one CAP-Unit pass. The full three-convolutional, two-fully-connected network is completed by recirculating the packet 102 times through the single deployed unit. Measured on the hardware switch, the on-pipeline model matches the controller-trained model's F1 score to within about one percentage point while consuming only about a quarter of the available SRAM and reaching 39.696 Gbps throughput.
Load-bearing premise
The load-bearing assumption is that recirculating the inference packet 102 times does not become a throughput bottleneck; if flows that trigger inference arrive at a sufficiently high rate, the measured 39.696 Gbps may not hold.
Editorial extensions
If this is right
- Any CNN whose layers can be expressed in the CAP-Unit template inherits the same per-pass resource footprint; deeper or wider networks cost additional recirculations, not additional pipeline stages.
- The reported resource usage means the same Tofino switch can host Quark alongside other forwarding functions: 75.73% of SRAM, 75% of table IDs, and 86.4% of PHV bits remain free.
- On BMv2, the all-units-per-pipeline variant beats the one-unit variant by 15.6% in throughput, so switches with more stages or more per-stage resources would reduce recirculation overhead.
- Keeping at least 7 quantization bits is necessary: 6-bit quantization causes accuracy to degrade sharply on the reported anomaly-detection task, placing a lower bound on the compression this method can tolerate.
Reading between the lines
- Because each inference consumes 102 pipeline passes, the system's sustainable load is better described by flows-per-second than bits-per-second; a future test should report the maximum trigger rate that preserves 39.7 Gbps.
- The same modular recirculation strategy could extend to other PISA targets, but the SRAM multiplication tables scale with the product of input values and weight values, so wider bit widths or larger input alphabets would need a sparse or decomposed multiplication scheme.
- Applying Quark to live traffic rather than replayed datasets would test whether the eight-packet feature set and 42.66µs latency are sufficient for online detection in high-speed spines.
- A direct extension would be to report the accuracy-latency trade-off curve across pruning rates and bit widths on the hardware pipeline, since the paper evaluates model performance and hardware performance separately.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Quark, a framework for deploying pruned and quantized CNN inference entirely within the P4 data plane, with implementations on both the Intel Tofino hardware switch and the BMv2 software switch. The control plane performs channel pruning and quantization-aware training, while the data plane implements a modular CAP-Unit that performs convolution, activation, and pooling; when a CNN does not fit in the pipeline, packets are recirculated across the unit. The evaluation reports 97.3% accuracy for anomaly detection, an average inference latency of 42.66 us, throughput of 39.696 Gbps on a 40 Gbps link, and low SRAM/PHV usage, and the authors provide a public code repository.
Significance. If the central claims hold, this is a substantial engineering result: it would be one of the first demonstrations of a full CNN running on a commodity P4 hardware switch without external compute elements, and the modular CAP-Unit plus the recirculation analysis could be reused by other in-network ML efforts. The paper has notable strengths: a real Tofino implementation, measurement against public datasets, comparison with N3IC and INQ-MLT, per-class F1/recall/precision results, a resource breakdown, and public code. However, the headline line-rate claim is not yet established because the interaction between inference triggering, the 102 recirculations, and sustained forwarding throughput is left unquantified; there are also inconsistencies in the reported SRAM utilization and in the derivation of the quantized convolution formula that need to be resolved before the central claims can be accepted.
major comments (5)
- [Section VI-E, Figure 10] The line-rate claim is conditional on the traffic mix and is not supported by the reported measurement. Each inference consumes 102 recirculations through the single CAP-Unit (Section V-D1), and inference is triggered only for certain flows or flow states (Section V-B); therefore sustainable throughput depends on how many flows per second actually trigger inference. The replay experiment reports 39.696 Gbps versus a 39.712 Gbps baseline, but it does not state how many inference triggers occurred during the replay, what fraction of packets belonged to flows that triggered inference, or what the maximum sustainable trigger rate is at 40 Gbps. The authors should report the trigger counts in the experiment, derive or measure the worst-case trigger load that the 102-pass design can sustain, and either add a worst-case evaluation or qualify the 'line rate' claim in the abstract.
- [Section IV-C, Equations (9)-(10)] The step from Equation (9) to Equation (10) replaces the bias scale S_b with S_w*S_x and sets Z_b=0. This is exact only if the bias is intentionally quantized with the product scale and zero-point 0; otherwise it introduces an unquantified approximation error. Because QAT is used, the network may learn to compensate, but the paper should state this constraint explicitly and provide an error analysis or an ablation that quantifies the accuracy impact of this substitution. As written, the derivation claims a generality that the formulas do not have.
- [Section V-D1 and Section VI-E] The relation between Theorem 1 and the reported 102 recirculations is not shown. Applying Theorem 1 to the model described in Section VI-A (three 16-channel convolutional layers and two fully connected layers of 16 and 15 units) with the stated formula and C=16 gives a required recirculation count far larger than 102 for any plausible input-feature count. If the pruning rate of 0.8 changes the channel counts, the post-pruning dimensions should be reported and the calculation of 102 should be shown explicitly. This is load-bearing because the 42.66 us latency claim is tied directly to the recirculation count.
- [Abstract and Table VI] The SRAM utilization figures are inconsistent: the abstract states that Quark uses 22.7% of SRAM resources, while Table VI reports an SRAM usage of 24.27% and states that 75.73% of SRAM remains available. The authors should reconcile these numbers and ensure that all instances report the same definition of SRAM usage, since the abstract's resource-efficiency claim is a headline result.
- [Section VI-B and VI-C] The pruning rate (0.8) and quantization bit width (7 bits) are selected based on experiments performed on the same anomaly-detection dataset that is later used for the final reported accuracy. As described, this is a form of test-set selection that can make the final 97.3% accuracy optimistic. The authors should describe a validation-based selection procedure or evaluate the selected configuration on a held-out test set that was not used for hyperparameter choice.
minor comments (6)
- [Section IV-B, Equation (3)] The zero-point formula uses 'Rmax' without defining it; it should be written consistently as rmax with explicit rounding, e.g., Z = Round(q_max - rmax/S).
- [Section IV-C and Section V-C] Equation (11) says M can be approximated as a fixed-point number with a bit shift, but Section V-C step (iv) says the product of M and previous results is precomputed and stored in a MAT. The relationship between these two implementation paths should be clarified.
- [Table VI] The column header 'ComputationaleMatch xBar tMatch xBar' appears corrupted; the column names should be printed consistently.
- [Section V-C, Listing 1] The comment 'Skip (vi) when fully connected layer, do result_1 += result_2' is unclear; it is not obvious how the accumulation across fully connected layer outputs is performed in the P4 code.
- [Section VI-E, Figure 11b] The figure's axis label 'Flows count per Second' and the legend entries would benefit from clearer units and an explicit statement of which day corresponds to each line style.
- [Table V] The text states that Quark and INQ-MLT show 'no significant difference' in flow classification F1, but no variance or statistical test is reported; a single overall F1 of 0.712 vs. 0.718 does not by itself establish equivalence.
Circularity Check
No circularity: Quark's accuracy, latency, and resource claims are measured against public datasets and external baselines; self-citations are background only.
full rationale
The central derivation chain is self-contained. The quantization formulation in Section IV follows the standard affine quantization mapping (Equations 1-5) and derives the quantized convolution expression in Equations 8-11; these are mathematical transformations of stated assumptions, not redefinitions of the target result. The pruning and quantization choices are evaluated against held-out data (ISCX Botnet and CICIDS-2017), and the reported 97.3% accuracy is measured on the Tofino implementation and compared with the controller result, so no fitted parameter is renamed as a prediction. The CAP-Unit modularization and the recirculation bound in Theorem 1 are proved from model dimensions rather than imported from prior work. The comparison schemes N3IC and INQ-MLT are external systems, and the hardware resource and latency figures are direct measurements. The authors do cite their own prior work in references [9], [21], and [25], but those citations are used only as related-work context and background on programmable data plane constraints; none of them supplies a load-bearing premise, a uniqueness theorem, or an ansatz that forces the paper's conclusions. The skeptical concern about the line-rate claim under 102 recirculations is a potential measurement or workload-scoping limitation (the trigger rate is not explicitly bounded), not a circular reduction: the throughput number is measured against a basic-switch baseline rather than derived from the claim itself. No self-definitional, fitted-input-as-prediction, or self-citation-load-bearing step is present.
Assumptions & free parameters
free parameters (2)
- pruning_rate =
0.8
- quantization_bit_width =
7 bits
assumptions (4)
- domain assumption Quantization mapping r=S(q-Z) and the integer-arithmetic convolution approximation in Eq. (10), with bias scale Sb replaced by SwSx and Zb set to 0, preserve accuracy after QAT.
- domain assumption All products needed for a layer fit in one match-action table in a single Tofino stage, so multiplication can be emulated by table lookup without exceeding SRAM or stage limits.
- domain assumption Recirculating inference packets, 102 passes for the deployed CNN, leaves enough pipeline capacity for the forwarding workload to run near line rate.
- domain assumption Channel pruning based on an importance criterion removes low-contribution channels with negligible accuracy loss, and the accuracy loss measured on the two selected datasets generalizes.
Cite this review
Pith. "Pith review of Quark: Implementing Convolutional Neural Networks Entirely on Programmable Data Plane." pith.science (2026). https://pith.science/paper/WUCUOJ74
@misc{pith2026250115100,
author = {Pith},
title = {Pith review of: Quark: Implementing Convolutional Neural Networks Entirely on Programmable Data Plane},
year = {2026},
howpublished = {\url{https://pith.science/paper/WUCUOJ74}},
note = {Machine review of arXiv:2501.15100}
}
abstract
The rapid development of programmable network devices and the widespread use of machine learning (ML) in networking have facilitated efficient research into intelligent data plane (IDP). Offloading ML to programmable data plane (PDP) enables quick analysis and responses to network traffic dynamics, and efficient management of network links. However, PDP hardware pipeline has significant resource limitations. For instance, Intel Tofino ASIC has only 10Mb SRAM in each stage, and lacks support for multiplication, division and floating-point operations. These constraints significantly hinder the development of IDP. This paper presents \quark, a framework that fully offloads convolutional neural network (CNN) inference onto PDP. \quark employs model pruning to simplify the CNN model, and uses quantization to support floating-point operations. Additionally, \quark divides the CNN into smaller units to improve resource utilization on the PDP. We have implemented a testbed prototype of \quark on both P4 hardware switch (Intel Tofino ASIC) and software switch (i.e., BMv2). Extensive evaluation results demonstrate that \quark achieves 97.3\% accuracy in anomaly detection task while using only 22.7\% of the SRAM resources on the Intel Tofino ASIC switch, completing inference tasks at line rate with an average latency of 42.66$\mu s$.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
INSIGHT: A Survey of In-Network Systems for Intelligent, High-Efficiency AI and Topology Optimization
A survey of in-network AI computing that catalogs architectures, model-compression methods, aggregation frameworks, and applications, but offers no new experimental results.
Reference graph
Works this paper leans on
-
[9]
pHeavy: Predicting heavy flows in the programmable data plane,
X. Zhang, L. Cui, F. P. Tso, and W. Jia, “pHeavy: Predicting heavy flows in the programmable data plane,”IEEE Transactions on Network and Service Management, vol. 18, no. 4, pp. 4353–4364, 2021
work page 2021
-
[21]
IN3: A framework for in-network computation of neural networks in the programmable data plane,
X. Zhang, L. Cui, F. P. Tso, W. Li, and W. Jia, “IN3: A framework for in-network computation of neural networks in the programmable data plane,”IEEE Communications Magazine, vol. 62, no. 4, pp. 96–102, 2024
work page 2024
-
[25]
X. Zhang, L. Cui, F. P. Tso, Z. Li, and W. Jia, “Dapper: Deploying service function chains in the programmable data plane via deep rein- forcement learning,”IEEE Transactions on Services Computing, vol. 16, no. 4, pp. 2532–2544, 2023
work page 2023
-
[1]
The programmable data plane: Abstractions, architectures, algorithms, and applications,
O. Michel, R. Bifulco, G. R ´etv´ari, and S. Schmid, “The programmable data plane: Abstractions, architectures, algorithms, and applications,” ACM Computing Surveys (CSUR), vol. 54, no. 4, pp. 1–36, 2021
work page 2021
-
[2]
Programmable data plane intelligence: Advances, opportunities, and challenges,
W.-X. Liu, C. Liang, Y . Cui, J. Cai, and J.-M. Luo, “Programmable data plane intelligence: Advances, opportunities, and challenges,”IEEE Network, vol. 37, no. 5, pp. 122–128, 2023
work page 2023
-
[3]
Re-architecting traffic analysis with neural network interface cards,
G. Siracusano, S. Galea, D. Sanvito, M. Malekzadeh, G. Antichi, P. Costa, H. Haddadi, and R. Bifulco, “Re-architecting traffic analysis with neural network interface cards,” in19th USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2022, pp. 513– 533
work page 2022
-
[4]
Taurus: A data plane architecture for per-packet ml,
T. Swamy, A. Rucker, M. Shahbaz, I. Gaur, and K. Olukotun, “Taurus: A data plane architecture for per-packet ml,” inProceedings of the 27th ACM International Conference on Architectural Support for Program- ming Languages and Operating Systems, 2022, pp. 1099–1114
work page 2022
-
[5]
Line-speed and scalable intrusion detection at the network edge via federated learning,
Q. Qin, K. Poularakis, K. K. Leung, and L. Tassiulas, “Line-speed and scalable intrusion detection at the network edge via federated learning,” inIFIP Networking Conference (Networking), 2020, pp. 352–360
work page 2020
Show all 35 references
-
[6]
Flowlens: Enabling efficient flow classification for ml- based network security applications,
D. Barradas, N. Santos, L. Rodrigues, S. Signorello, F. M. V . Ramos, and A. Madeira, “Flowlens: Enabling efficient flow classification for ml- based network security applications,” inNetwork and Distributed System Security Symposium (NDSS), 2021
2021
-
[7]
In-network intelligence control: Toward a self-driving networking architecture,
T. Mai, S. Garg, H. Yao, J. Nie, G. Kaddoum, and Z. Xiong, “In-network intelligence control: Toward a self-driving networking architecture,” IEEE Network, vol. 35, no. 2, pp. 53–59, 2021
2021
-
[8]
Offloading machine learning to programmable data planes: A systematic survey,
R. Parizotto, B. L. Coelho, D. C. Nunes, I. Haque, and A. Schaeffer- Filho, “Offloading machine learning to programmable data planes: A systematic survey,”ACM Computing Surveys (CSUR), vol. 56, no. 1, pp. 1–34, 2023
2023
-
[10]
Programmable switches for in-networking classification,
B. M. Xavier, R. S. Guimar ˜aes, G. Comarela, and M. Martinello, “Programmable switches for in-networking classification,” inIEEE Conference on Computer Communications (IEEE INFOCOM), 2021, pp. 1–10
2021
-
[11]
In-network neural networks,
G. Siracusano and R. Bifulco, “In-network neural networks,”arXiv preprint arXiv: 1801.05731, 2018
2018 arXiv
-
[12]
Do switches dream of machine learning? Toward in-network classification,
Z. Xiong and N. Zilberman, “Do switches dream of machine learning? Toward in-network classification,” inProceedings of the 18th ACM Workshop on Hot Topics in Networks, 2019, p. 25–33
2019
-
[13]
IIsy: Practical in-network classification,
C. Zheng, Z. Xiong, T. T. Bui, S. Kaupmees, R. Bensoussane, A. Bern- abeu, S. Vargaftik, Y . Ben-Itzhak, and N. Zilberman, “IIsy: Practical in-network classification,”arXiv preprint arXiv:2205.08243, 2022
2022 arXiv
-
[14]
A survey of convolutional neural networks: Analysis, applications, and prospects,
Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional neural networks: Analysis, applications, and prospects,”IEEE Transac- tions on Neural Networks and Learning Systems, vol. 33, no. 12, pp. 6999–7019, 2022
2022
-
[15]
A machine learning-based toolbox for p4 programmable data-planes,
K. Zhang, N. Samaan, and A. Karmouch, “A machine learning-based toolbox for p4 programmable data-planes,”IEEE Transactions on Net- work and Service Management, 2024
2024
-
[16]
NetNN: Neural intrusion detection system in programmable networks,
K. Razavi, S. D. Fard, G. Karlos, V . Nigade, M. M ¨uhlh¨auser, and L. Wang, “NetNN: Neural intrusion detection system in programmable networks,” in29th IEEE Symposium on Computers and Communications (ISCC), 2024
2024
-
[17]
Can the network be the ai accelerator?
D. Sanvito, G. Siracusano, and R. Bifulco, “Can the network be the ai accelerator?” inProceedings of the 2018 Morning Workshop on In- Network Computing, 2018, pp. 20–25
2018
-
[18]
Intel Tofino switch ASIC,
“Intel Tofino switch ASIC,” 2024, https://www.intel.com/content/www/ us/en/products/details/network-io/intelligent-fabric-processors/tofino. html, accessed on Jul. 30, 2024
2024
-
[19]
NetFC: Enabling accurate floating-point arithmetic on programmable switches,
P. Cui, H. Pan, Z. Li, J. Wu, S. Zhang, X. Yang, H. Guan, and G. Xie, “NetFC: Enabling accurate floating-point arithmetic on programmable switches,” inIEEE 29th International Conference on Network Protocols (ICNP), 2021, pp. 1–11
2021
-
[20]
Brain-on- Switch: Towards advanced intelligent network data plane via NN-Driven traffic analysis at line-speed,
J. Yan, H. Xu, Z. Liu, Q. Li, K. Xu, M. Xu, and J. Wu, “Brain-on- Switch: Towards advanced intelligent network data plane via NN-Driven traffic analysis at line-speed,” in21st USENIX Symposium on Networked Systems Design and Implementation (NSDI), 2024, pp. 419–440
2024
-
[22]
In-network neural networks: Challenges and opportunities for innovation,
M. C. Luizelli, R. Canofre, A. F. Lorenzon, F. D. Rossi, W. Cordeiro, and O. M. Caicedo, “In-network neural networks: Challenges and opportunities for innovation,”IEEE Network, vol. 35, no. 6, pp. 68– 74, 2021
2021
-
[23]
Deep learning approach for intelligent intrusion detection system,
R. Vinayakumar, M. Alazab, K. P. Soman, P. Poornachandran, A. Al- Nemrat, and S. Venkatraman, “Deep learning approach for intelligent intrusion detection system,”IEEE Access, vol. 7, pp. 41 525–41 550, 2019
2019
-
[24]
End-to-end encrypted traffic classification with one-dimensional convolution neural networks,
W. Wang, M. Zhu, J. Wang, X. Zeng, and Z. Yang, “End-to-end encrypted traffic classification with one-dimensional convolution neural networks,” inIEEE international conference on intelligence and security informatics (ISI), 2017, pp. 43–48
2017
-
[26]
Survey on FPGA architecture and recent applications,
S. Gandhare and B. Karthikeyan, “Survey on FPGA architecture and recent applications,” inInternational Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN), 2019, pp. 1–4
2019
-
[27]
Network processor - A simplified approach for transport layer offloading on nic,
G. Gadre, S. Badhe, and K. Kulkarni, “Network processor - A simplified approach for transport layer offloading on nic,” in2016 International Conference on Advances in Computing, Communications and Informat- ics (ICACCI), 2016, pp. 2542–2548
2016
-
[28]
Binary neural networks: A survey,
H. Qin, R. Gong, X. Liu, X. Bai, J. Song, and N. Sebe, “Binary neural networks: A survey,”Pattern Recognition, vol. 105, p. 107281, 2020
2020
-
[29]
p4lang/behavioral-model,
“p4lang/behavioral-model,” https://github.com/p4lang/ behavioral-model, accessed on Jul. 30, 2024
2024
-
[30]
A white paper on neural network quantization,
M. Nagel, M. Fournarakis, R. A. Amjad, Y . Bondarenko, M. van Baalen, and T. Blankevoort, “A white paper on neural network quantization,” arXiv preprint arXiv:2106.08295, 2021
2021 arXiv
-
[31]
Pruning and quantization for deep neural network acceleration: A survey,
T. Liang, J. Glossner, L. Wang, S. Shi, and X. Zhang, “Pruning and quantization for deep neural network acceleration: A survey,”Neuro- computing, vol. 461, pp. 370–403, 2021
2021
-
[32]
Understanding straight-through estimator in training activation quantized neural nets,
P. Yin, J. Lyu, S. Zhang, S. J. Osher, Y . Qi, and J. Xin, “Understanding straight-through estimator in training activation quantized neural nets,” inInternational Conference on Learning Representations (ICLR), 2019
2019
-
[33]
mininet/mininet,
“mininet/mininet,” https://github.com/mininet/mininet, accessed on Jul. 30, 2024
2024
-
[34]
Towards effective feature selection in machine learning-based botnet detection approaches,
E. Biglar Beigi, H. Hadian Jazi, N. Stakhanova, and A. A. Ghorbani, “Towards effective feature selection in machine learning-based botnet detection approaches,” inIEEE Conference on Communications and Network Security, 2014, pp. 247–255
2014
-
[35]
Toward generating a new intrusion detection dataset and intrusion traffic characterization,
I. Sharafaldin, A. H. Lashkari, A. A. Ghorbaniet al., “Toward generating a new intrusion detection dataset and intrusion traffic characterization,” inInternational Conference on Information Systems Security and Pri- vacy (ICISSp), vol. 1, 2018, pp. 108–116
2018
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.